Codec Comparison 2

Introduction

Note:  This comparison is from January 2001, when Windows Media 8 was still in beta.  Since then, most of the bugs have been fixed, but I haven't noticed any changes in the quality.  Therefore, the comparisons found below should still be valid.

Microsoft has released version 8 of their video and audio codecs, which are known as Windows Media Video 8 and Windows Media Audio 8.  To test the quality and usefulness of these new codecs, I encoded a number of videos in Windows Media 8 (WM 8), RealMedia (RM 8), DivX Fast-Motion (FM), and DivX Low-Motion (LM) formats.   Below you will find the results of my experiments, including many frame grabs for comparison.  I would like to say that although I attempted to encode a wide variety of types of video, I cannot guarantee that my results will be achieved for every type of video.  (Note: DivX was included as a baseline only.  The images below were selected to demonstrate the weaknesses of WM 8 and RM 8, not DivX.  Although both WM 8 and RM 8 produced better videos than DivX on average, this may or may not be represented in the pictures below.)

Before I begin, let me point out that Windows Media 8 is currently in a early beta state.  There is no built-in graphical interface, although you can download several user-created ones from http://www.doom9.org/.  In addition, the Windows Media 8 Encoding Utility (WM8EU) currently has a number of bugs and limitations.  First of all, the input audio must be in a 16-bit uncompressed PCM format.  Second, the files produced by WM8EU are sometimes unplayable by all players except Windows Media Player 7 (WMP 7), and even WMP 7 has trouble.  For example, sometimes the encoded files are not indexed properly, which causes the video to stutter and prevents you from manually seeking past a certain point in the video.  Finally, despite what the documentation says, WM8EU does not natively support files over 2 GB.  However, you may be able to get around this limitation with two useful utilities (explained below).  Hopefully Microsoft will have corrected these bugs by the time they decide to release the final version of the Windows Media Encoder 8.  Right now, I wouldn't advise anyone except computer gurus to use WM 8 as their video format because there are still many bugs to be ironed out. 

Now we're about ready to move on to the comparisons.  Unless otherwise noted, all the WM 8 files were encoded using two-pass VBR encoding with the following command:
wm8eutil -input video.avi -output video.wmv -v_mode 3 -v_bitrate bitrate -v_keydist 10 -v_buffer 0
The DivX files (both Fast-Motion and Low-Motion) were encoded with 10 seconds/keyframe and a crispness of 100.  The RM 8 files were encoded as described in the RealMedia 8 Encoding section of my guide.  I encoded all the files so that their final file sizes did not differ by more than a few KB.

320 x 240 Video

Cartoons:
Windows Media 8 has made some impressive gains over the previous MPEG-4 based codecs for animated video.  The amount of blockiness and artifacts have been noticeably reduced.  However, RM 8 still appears to be slightly superior than WM 8 for this type of video.  In the videos that I tested, RM 8 produced a sharper image with less artifacts than WM 8 at any bitrate.  I tested bitrates ranging from 300 kbps to approximately 1000 kbps.  (The codecs wouldn't go much higher than 1000 kbps since the video was only 320 x 240.)  Here are some comparison frame grabs:

300 kbps video test frame #1:  Source* - RM 8 - WM 8 - DivX FM - DivX LM (8% larger than other files)
300 kbps video test frame #2:  Source* - RM 8 - WM 8 - DivX FM - DivX LM (8% larger than other files)
900 kbps video test frame #1:  Source* - RM 8 - WM 8 - DivX LM - (DivX FM couldn't achieve this bitrate)

* = I lost the source video, so the source frames are actually from maximum bitrate DivX LM and RM 8 files.

Real Life Video:
This is the part where things began to become tricky.  Depending on the type of video, RM 8 and WM 8 each have certain advantages.  In video with low and high action sequences, RM 8 appears to assign more bits to the low motion parts and less bits to the high action parts when compared to WM 8.  Because of this, WM 8 produces a slightly blurrier image with more artifacts when there is little motion, especially during dark scenes.  Here are some frame grabs of this:

350 kbps video frame:  Source - RM 8 - WM 8 - DivX FM - DivX LM
600 kbps video frame:  Source - RM 8 - WM 8 - DivX FM - DivX LM

In high action sequences, WM 8 seems to have the advantage.  WM 8 retains higher detail on fast-moving objects, while RM 8 has a more blurred image.  Here are some grabs demonstrating this:

600 kbps video test frame #1:  Source - RM 8 - WM 8 - DivX FM - DivX LM
600 kbps video test frame #2:  Source - RM 8 - WM 8 - DivX FM - DivX LM

However, if you encode at lower bitrates, WM 8 loses its sharpness advantage during the high-motion scenes of a video with high and low action sequences.  Here are some images of this:

350 kbps video frame:  Source - RM 8 - WM 8 - DivX FM - DivX LM

Understandably, the weaknesses of the two codecs gradually fade as the bitrate increases, but they remain visible at even 1000 kbps.  WM 8 still produces slightly more artifacts for low motion scenes, and RM 8 still isn't quite as sharp.

CPU Utilization:
In its current form, WM 8 needs far more horsepower than RM 8 at 320 x 240.  WM 8 requires 2-3 times as much processing power as RM 8.  On my system (an Athlon 650), playing a RM 8 file produced an average CPU utilization of 25%.  Playing an the same video in WM 8 utilized 60% of my CPU on average.  Hopefully, Microsoft will be able to decrease the CPU requirements in the release version of WM 8.

Conclusions:
For cartoons at 320x240, I'd recommend RM 8 because it appears to have an advantage over WM 8 at any bitrate.
For 320 x 240 real life video with a mixture of high and low action sequences, I'd also have to suggest RM 8.  Although RM 8 doesn't produce as sharp at image during fast motion scenes, the decreased detail isn't that visible because the human eye is poor at distinguishing detail when an object is moving fast.  The artifacts of WM 8, on the other hand, are clearly visible because they occur during low motion sequences.  The higher CPU utilization of WM 8 is also an issue.
For primarily low-motion videos, I'd once again choose RM 8.  RM 8 seems to produce less artifacts than WM 8 for this type of video.
For videos where fast motion sequences are in the majority and CPU utilization is not an issue, WM 8 would probably be the better choice.  On a related note, if you are attempting to encode a video that just contains one long action sequence, using the 1-pass CBR of WM 8 (-v_mode 0) will produce the best quality at a given file size.  This is odd because you'd expect two-pass VBR to always produce identical or higher quality than a CBR file at the same file size.  However, this is not the case.  RM also does better when encoding an action clip in CBR mode.  This looks to me like it might be a possible inefficiency in the 2-pass VBR engines of both formats.  Here's a comparison of 1-pass CBR and 2-pass VBR on a high-action clip in WM 8.  Please note that both screenshots were taken from videos of the same file size:

1100 kbps video frame:  Source - RM 8 (2-pass VBR) - WM 8 (2-pass VBR) - RM 8 (1-pass CBR) - WM 8 (1-pass CBR)

640 x 480 Video

This is where RM 8 begins to have problems.  When you encode a video in RM 8 using the "Normal Motion Video" setting, the image quality lacks in sharpness.  It almost looks like RM 8 is encoding the video at a lower resolution and then resampling it to full resolution on playback.  You can get around this by using the "Sharpest Image Video" option, but then other problems surface.  First of all, unless you specify a really high bitrate, the encoder drops numerous frames of video.  The dropped frames occur in parts of the video where the bitrate is not sufficient to produce a sharp image.  For example, during high-motion scenes it might drop 1 out of every 5 frames in order to keep a sharp image in the 4 remaining frames.  Second, RealPlayer drops numerous frames when decoding a Sharpest Image video.  On my Athlon 650, it seems to drop small bunches of frames every few seconds.  It even does this during low-motion scenes, despite the fact that my CPU utilization averages around 65%.

Relatively speaking, WM 8 does a much better job with high resolution video.  The image is consistently sharper than that of a RM 8 video.  And although you'll need a speedy processor, WM 8 doesn't have the decoding problems that RM 8 has at 640 x 480.  Playing a 640 x480 video in WM 8 uses about 70% of my processor during low-motion scenes and 100% of my processor during really high-motion scenes.  Because of this, it does drop a few frames, but not nearly as many as RM 8.

Because of the issues with RM 8 I just mentioned, I don't feel it's necessary to write up a more lengthy comparison for 640 x 480 video.  Therefore, I'm only going to include a couple frame grabs:

2200 kbps high-motion video frame:  Source - RM 8 (Normal Motion Video) - RM 8 (Sharpest Image Video) - WM 8 - (DivX files excluded because they dropped more than 10% of the frames)
750 kbps low-motion video frame:  Source - RM 8 (Normal Motion Video) - RM 8 (Sharpest Image Video) - WM 8 - DivX FM - DivX LM

Conclusion:
RM 8 does a poor job with high resolution video, so I have to recommend WM 8.

Audio

Since everyone hears a little differently, this is the most subjective part of my comparison.  Unlike the video comparisons, I cannot take a single snapshot and declare which audio format sounds better at that instant.  For these reasons, you might want to test the two formats for yourself and determine which sounds better to your ears.  However, I'll still offer my opinion because I feel that at lower bitrates there are certain artifacts that most people should be able to notice.

Mono:
For RealAudio 8 (RA 8), I feel that acceptable quality for a video with voice and background music can be obtained at 32 kbps (the 32 Kbps Music codec).  There is a 14.5 KHz cutoff frequency, so the audio certainly isn't CD-quality.  However, it sounds reasonably close to the source audio, and there aren't that many noticeable artifacts.  If you want to improve the quality and increase the frequency response to 20 KHz, you can increase the bitrate to 44 kbps or 64 kbps.

For Windows Media Audio 8 (WMA 8), I wouldn't use a bitrate lower than 64 kbps.  This is because WMA 8 produces artifacts that sound "metallic" at lower bitrates.  Microsoft has decreased these artifacts from Windows Media Audio 7, but I can still hear them at bitrates up to 96 kbps.  You may or may not be annoyed be these artifacts, but I personally can tolerate them below 64 kbps.

Here are some sample audio clips:

RA 8 (32 Kbps Music) - WMA 8 (32 kbps, 44.1 KHz) - Source WAV (compressed with Monkey's Audio 3.80)

Stereo:
For RA 8, I think the 44 Kbps Stereo Music High codec provides adequate quality for a stereo soundtrack.  RA 8 does a good job of representing the source audio while preventing most of the undesirable artifacts.  Admittedly, some people feel that RA 8 does a poor job with stereo separation, but I haven't noticed any serious problem.  Any problems with the stereo separation are probably more likely to show up when attempting to encode CD-quality music rather than when encoding a video soundtrack.

For WMA 8, I feel that 64 kbps is the lowest bitrate you should use for stereo soundtrack.  As I mentioned above, WMA 8 has an annoying metallic sound below 64 kbps.  However, for bitrates higher than 64 kbps, it's possible that WMA 8 will sound better than RA 8.  This is because RA 8 might have a poorer stereo separation whereas WMA 8 tries to remain closer to the source audio.

Here are some sample audio clips:

RA 8 (44 Kbps Stereo Music High - RA 8) - WMA 8 (48 kbps, 44.1 KHz) - Source WAV (compressed with Monkey's Audio 3.80)

Conclusions:
For the low bitrate soundtracks that are normally used for video, I'd say that RA 8 is the better choice.  It seems to produce less unwanted artifacts at lower bitrates for mono or stereo audio.
If you are trying to encode audio that approaches CD-quality, you might be better off using WMA 8 at 96 kbps or 128 kbps.  This is because people who have better ears than mine say that WMA 8 more accurately represents the source audio at higher bitrates.