For the Google Lunar Xprize I’m looking into video compression in order to estimate the bandwidth that is needed to stream video from the moon to earth.

Plan

In order to make an estimate, uncompressed footage from the moon would be best to use as a base. In stead of that footage we have panoramic shot from the moon available in tiff format with a resolution of 17531 x 2333 pixels. Cutting this panoramic shot in seperate pictures and animating those pictures in a video pan shot should give some information on what compression is acceptable and the required bandwith it needs.

All processing will be done on the following system:

• AMD Athlon Processor 4850e @ 2500MHz
• 8GB RAM
• Debian GNU/Linux snapshot sid install from 2011/01/15

If possible the whole process will be executed using open source software. The first test will be done in 720p HD format (1280x720).

Converting to pictures

Using ImageMagick it is quite easy to convert an image to different formats and different resolutions.

The first step consists of scaling the original image to a vertical resultion that is the same as the target video steam, 720 pixels. I stored the image in the lossless png format using:

convert -scale x720 MoonPano4.tif pano.png

This results in a pano.png image with a resultion of 5410 x 720 pixels.

Using this as the base image, the idea is to cut seperate 1280 x 720 pixel images. The first image will start at x coordinate 0 and the following will increment 1 pixel further on the x axis. This should result in 5410 - 1280 = 4130 frames that can be used to create movies. To automate this process the following script can be used to convert the pano.png to seperate frames.

#!/bin/sh
width=5410
base=pano.png

count=0
for i in $( seq 0 1 $(( $width - 1280 )) )
do
   filename=$( printf "%04d.bmp" $count )
   echo "Converting $filename..."
   convert $base -crop 1280x720+$i+0 $filename
   count=$(( $count + 1 ))
done

When the script is finished 4130 files are created in bmp (losless) format.

Compiling software

To convert the frames into movies the ffmpeg software is used. Below is the procedure used to compile ffmpeg and the x264 library.

Compiling x264

Using git the repository for x264 can be checked out, compiled and installed.

git clone git://git.videolan.org/x264.git
cd x264
./configure
make
make install

Compiling ffmpeg

Using git the repository for ffmpeg can be checked out, compiled and installed.

git clone git://git.ffmpeg.org/ffmpeg.git
cd ffmpeg
./configure --enable-gpl --enable-version3 --enable-nonfree --enable-postproc \
    --enable-libfaac --enable-libmp3lame --enable-libopencore-amrnb \
    --enable-libopencore-amrwb --enable-libtheora --enable-libvorbis \
    --enable-libvpx --enable-libx264 --enable-libxvid --enable-x11grab
make
make install

H264

To get an initial idea of compression and picture quality, the h264 codec is choosen.

Configuration

This codec has a number of parameters that can be configured. For this test 2 are defined:

• Video preset
• Rate control method

Video Preset

The h264 codec has a number of video presets to choose from:

• ultrafast
• superfast
• veryfast
• faster
• fast
• medium (default)
• slow
• slower
• veryslow
• placebo

Rate control method

Assuming passing twice over the input isn’t an option in the real application, there are two rate control methods which are interesting to try:

• constant rate factor
• one pass average bitrate

The constant rate factor is aimed at picture quality. Bit rate and output size can vary in this situation. The constant rate factor method is choosen by passing the -crf option followed by a float value (lower value is higher picture quality and higher bitrate).

The one pass average bitrate is aimed at keeping a constant bitrate. Picture quality can vary depeding on the type of movement etc. in the input. The one pass averate bitrate method is choosen by passing the -b option followed by the desired bitrate.

Commandline settings

The following commandline options used are:

• -t 30: Limit output video length to 30 seconds
• -f image2: Input format is in image2 format, seperate input images
• -y: overwrite any existing output
• -i frames/%04d.bmp: refer to the pictures in the frames directory
• -vcodec libx264: use h264 for encoding the output
• -vpre slow: select the slow quality profile
• -threads 0: don’t limit the number of threads

Output processing

The ffmpeg command outputs information on the screen at intervals. It displays lines with the following information:

frame=  628 fps=  8 q=31.0 size=     686kB time=22.84 bitrate= 246.1kbits/s

One of the interesting variables in the output is the bitrate. Depending on the encoding used the bitrate can vary through the processing of the frames. It can be quite useful to have a bitrate vs. time graph based on the output of ffmpeg.

After capturing the output of ffmpeg some simple scripting can process the bitrate vs. time graph.

#!/bin/bash
PLOTDATA=/tmp/plot.$$

if test ! -f "$1"
then
   echo "FFMpeg output file $1 can not be found."
   exit 1
fi

cat $1 | awk 'BEGIN {RS="\r";FS=" "} /kbits/ { t=substr($8,6) ; i=index($0,"bitrate="); \
br=substr($0, i+8, index($0,"kbit")-i-8); if (t != "10000000000.00") printf("%s %s\n", t, br);}' \
 > $PLOTDATA

gnuplot -e "set title \"FFMpeg bitrate for render $1\"; set xlabel \"Time (seconds)\"; set ylabel \
\"Bitrate (kbit/sec)\"; set output \"$1.png\"; set terminal png small; set grid; \
plot \"$PLOTDATA\" using 1:2 with lines"

rm -f $PLOTDATA

Encoding at 25 frames per second

For the initial test the frame rate is kept at the basic 25 frames per second. All commands used limit the target video duration to 30 seconds to limit rendering time.

Constant rate factor

The constant rate factor was tested with 3 values:

• 26
• 35
• 40

CRF 26

ffmpeg -t 30 -f image2 -y -i frames/%04d.bmp -vcodec libx264 -vpre slow -crf 26 -threads 0 h264-crf26.mp4  > h264-crf26 2>&1

Average bitrate: 201.41kbit/second

Bitrate variation over the whole video:

Result video:

h264-crf26.mp4

CRF 35

ffmpeg -t 30 -f image2 -y -i frames/%04d.bmp -vcodec libx264 -vpre slow -crf 35 -threads 0 h264-crf35.mp4 > h264-crf35 2>&1

Average bitrate: 119.53kbit/second

Bitrate variation over the whole video:

Result video:

h264-crf35.mp4

CRF 40

ffmpeg -t 30 -f image2 -y -i frames/%04d.bmp -vcodec libx264 -vpre slow -crf 40 -threads 0 h264-crf40.mp4 > h264-crf40 2>&1

Average bitrate: 91.87kbit/second

Bitrate variation over the whole video:

Result video:

h264-crf40.mp4

one pass average bitrate

The one pass average bitrate was tested with 2 values:

• 150kbit/second
• 120kbit/second

150kbit/second

ffmpeg -t 30 -f image2 -y -i frames/%04d.bmp -vcodec libx264 -vpre slow -b 150k -threads 0 h264-br150.mp4  > h264-br150 2>&1

Average bitrate: 134.08kbit/second

Bitrate variation over the whole video:

Result video:

h264-br150.mp4

120kbit/second

ffmpeg -t 30 -f image2 -y -i frames/%04d.bmp -vcodec libx264 -vpre slow -b 120k -threads 0 h264-br120.mp4  > h264-br120 2>&1

Average bitrate: 108.52kbit/second

Bitrate variation over the whole video:

Result video:

h264-br120.mp4

Conclusions

In all cases it is clear the bitrate variation over the whole video peaks at the start. This seems logical, as the initial picture needs to be established. As soon as this information is available, only differences have to be encoded which takes a lot less bandwidth.

When using the average bitrate method, the initial amount of bandwidth required is much less then when using the constant rate factor. However, the lower bandwidth requirement results in a much worse initial picture. Due to the bandwidth constraint it takes longer for a high quality initial picture to form. The quality of the picture improves while the actual movement is taking place. With the constant rate factor the initial frame is formed to high quality after which movement starts.

The spikes in bitrate that can be seen in bitrate graphs indicate the use of i-frames every 10 seconds. The i-frame uses more bandwidth because it doesn’t require references to other frames to allow for decoding (like with p or b frames). For more information see a relevant article on <a href=“http://en.wikipedia.org/wiki/B-frame#Bi-directional_predicted_frames_.28or_slices.2C.29_a.k.a._B_pictures" data-wikipedia<=”" data-a="">.

For this specific pan shot is seems like the big difference is in the startup bandwidth required. The difference in bitrate during the remainder of the video is very low.

Encoding at 10 frames per second

To see what the effect of the framerate is on the resulting videostream, a test was done with the framerate lowered to 10. To keep an acceptable motion effect, framerates lower then 10 shouldn’t be selected. The resulting video will become very choppy. The same test were re-run as before with the 25 frames per second encoding.

Constant rate factor

The constant rate factor was tested with 3 values:

• 26
• 35
• 40

CRF 26

ffmpeg -t 30 -f image2 -y -i frames/%04d.bmp -vcodec libx264 -vpre slow -crf 26 -threads 0 -r 10 h264-crf26-10.mp4  > h264-crf26-10 2>&1

Average bitrate: 161.07kbit/second

Bitrate variation over the whole video:

Result video:

h264-crf26-10.mp4

CRF 35

ffmpeg -t 30 -f image2 -y -i frames/%04d.bmp -vcodec libx264 -vpre slow -crf 35 -threads 0 -r 10 h264-crf35-10.mp4 > h264-crf35-10 2>&1

Average bitrate: 91.53kbit/second

Bitrate variation over the whole video:

Result video:

h264-crf35-10.mp4

CRF 40

ffmpeg -t 30 -f image2 -y -i frames/%04d.bmp -vcodec libx264 -vpre slow -crf 40 -threads 0 -r 10 h264-crf40-10.mp4 > h264-crf40-10 2>&1

Average bitrate: 64.80kbit/second

Bitrate variation over the whole video:

Result video:

h264-crf40-10.mp4

one pass average bitrate

The one pass average bitrate was tested with 2 values:

• 150kbit/second
• 120kbit/second

150kbit/second

ffmpeg -t 30 -f image2 -y -i frames/%04d.bmp -vcodec libx264 -vpre slow -b 150k -threads 0 -r 10 h264-br150-10.mp4  > h264-br150-10 2>&1

Average bitrate: 165.41kbit/second

Bitrate variation over the whole video:

Result video:

h264-br150-10.mp4

120kbit/second

ffmpeg -t 30 -f image2 -y -i frames/%04d.bmp -vcodec libx264 -vpre slow -b 120k -threads 0 -r 10 h264-br120-10.mp4  > h264-br120-10 2>&1

Average bitrate: 137.10kbit/second

Bitrate variation over the whole video:

Result video:

h264-br120-10.mp4

Conclusions

As might be expected, a lower framerate yields a lower bitrate. This seems to be true for the constant rate factor encoding, but not for the average bitrate encoding. While the constant rate factor bitrate graphs display the same initial peak bandwidth usage, the constant bitrate graphs display a big increase in initial bitrate.

Both types of encoding also don’t display the characteristic i-frame spikes in the bitrate graph.

It seems that lowering the frames per second has some interesting side effects that need to be reviewed more closely if this technique is chosen to lower the bandwidth.

Results summary