If you don’t know what video compression is or what’s a codec, please read this thoroughly (it’s quite short): http://en.wikipedia.org/wiki/Video_compression
Video files that are highly compressed are bad news for any realtime software, not just Jitter. Especially bad are compression formats that compress temporal (as opposed to spatial) information, like most of the advanced codecs such as the popular h.264, or any of the mpeg formats. Therefore what we want for Jitter is a spatial codec such as Photo or Motion Jpeg. In this PDF, excerpted from the Cycling74 Jitter documentation, you’ll find a detailed explanation. The summary for the “favorite setting” is:
• 320×240 frame size // You can use 640 by 480 as well if you have a newer computer
• 15 frames per second // 25-30 fps also work, again depends on the strength of your processor
• Video tracks: Photo-JPEG codec, using a medium to high spatial quality setting
• Audio tracks: no compression