Previous Post: Live Coding with Video Recording

This POST started 10 days ago with a Lived Coding recording.
See this previous post: Live Coding - Java Coding Exercise: a tool for synchronizing 2 dirs (backup) using SHA comparisons

Live Coding Video Recording

For preparing 30 minutes of live-coding, it actually took me more than 10 hours of work:

1 hour to peek tools in linux to do video screen capture, and key-press/mouse-click display
1 hour to code (first recording)
1 hour to convert the video file to a compressed but standard file format
3 hours to write this post on github.io
4 hours to repeat faster code recording

And because I am curious about everything in computer sciences, I have spent the last few days searching for tools, documentations, research papers on the subject of “screencast video compression” !

At first, the longest was to find the tool to show the key-press/mouse-clicks and to do a video capture of all.
I tend to avoid mouse-clicking wherever possible, because it is just slower… and more harmfull for your “carpal tunnel” (you might get surgical operation on your hand sometimes otherwise…). see wikipedia: carpal tunnel

For displaying key-press and mouse-click, I have chosen to install and launched “key-mon” tool

When watching at the video, you can see that I hardly ever use the mouse, but I use few eclipse shortcuts here and there.

I should also (but did not) have configured “MouseFeed”, which is and Eclipse plugin that shows you when you mouse-click instead of using shortcuts. This plugin is very cool to learn important shortcuts, and force you improving your Eclipse efficiency.
For example, for fast switching from java source code editer to corresponding Test code, I use “CTRL+J” which is defined in an eclipse plugin called “MoreUnit”.

Finally, here we are, I recorder me coding for 30 minutes, converted the file to .MKV file format (=4.6 Mo) , and here it is:

live-coding.mkv

For training myself, I might redo and redo again this exercise:

try to reduce time from 1 hour to 45 minutes ?
try to reduse code size from 200 lines to 150 ? (using shorter code style, java 8 lambda, ..)
try not to use the mouse at all (from ~ 20 clicks to 0?)
try to do do less typo errors (edit then delete 1 char)
try to do less do-write and undo-write of codes (type directly the final code, when choosing local variables names, method names, if-tehn-else / for / try-catch structures)
less hesitations
always write Tests code first before Code (TDD=Test Driven Development, Agile practise),
etc.

An interesting measurment would be to see the number of “key stroke”, and the number of “Delete” or “Up arrow” pressed.
Even better, I would like to see the speed of “key strokes”, as a function of time, to see when coding is slowing.

More on screen capture video compression file formats

To do screencast video capture, I have searched a tool for my Debian Linux workstation… After googling it, there was a dozen of candidates.

Strangely (or not!!), most of them dit not work at all (crashed when launched !!!), where unusable, or very un-intuitive to use. I am a busy guy, I don’t want to read 600 pages of manual for that kind of task. I repeated 4 times:

get a soft candidate from google
apt-get install ..
launch it
click on the big green button if found it, and if it seems to work
trash it, and get another one

I finally choose a very simple and efficient one from github (using java source-code for it!) It is called “screen-recorder”, and there is also a tool “screen-player”, and “screen-converter”

https://github.com/bspkrs/java-screen-recorder.git

For recording my video, I launched “screen-recorder.sh” which internally was

java -cp screen-recorder.jar com.wet.wired.jsr.recorder.JRecorder

It worked like a charm. Presenting me a minimalistic and efficient app with only a “start” button. Internally, it writes to a temporary file… And once started, you have a button “Stop”, and “Save” (which rename the temp file to the destination you ask).

The huge advantage of it is that it writes the video file on the fly in a proprietary(!) file format “.cap”, but it is very very efficiently, using an internal GZIP stream of bitmaps… simple but efficient! Then you can replay it, either normal speed, or fast speed… and convert it to “.mov” standard format.

For 28mn of video capture of 1900x1080, the .cap file is 31Mo ONLY … while the converted .mov file is 518 Go, and the .MKV file is 30% smaller: 22 Mo
(For another recording lasting 1 hour, I got bigger files: 36M of .CAP, 13M of .MKV, and 1.5 Go of .MOV !)

$ ls -lh
  22M  live-coding.h264
  22M  live-coding.mkv
  22M  live-coding.m4v
  23M  live-coding.webm
  31M  live-coding.cap
  36M  live-coding.mp4
  65M  live-coding.wmv
  65M  live-coding.asf
  69M  live-coding.avi
  86M  live-coding.flv
  86M  live-coding.swf
 518M  live-coding.mov
  25G  live-coding.yuv

This is the main reason why I put on this website the .cap file … if you are interrested in seing the video! Someday, I might find a converter to a much better compressed and standard file format.

To play the video in the recorded proprietary .CAP format … I have launched

java -cp java-screen-player.jar com.wet.wired.jsr.player.JPlayer live-coding.cap

Then to convert it to .MOV, I launched

java -cp screen-converter.jar:lib/jmf.jar  com.wet.wired.jsr.converter.RecordingConverter live-coding.cap

Unfortunatly, .MOV file is huge, something like 20x bigger than .CAP file.

So, I converted it to a more efficient and standard format: .MKV, using “handbrake” UI conversion tool.

MOV file compression algorithm is based on lossy JPEG (Fourier cosinus transformationon using float numbers and roundings), so it is clearly not adpated to compress RGB integers image generated from computer screen (using only horizontal/vertical rectangles, Fonts display and few other vector drawing primitives)

Basically, a video of someone typing text in an editor is a video with a very LOW BIT RATE information between each frame! We could describe that images are combinations of these low level primitives:

Only the few pixels around the cursor are changing from one frame image to the next one.
around the cursor change, image block are often shifted to the left or to the right
most changing blocks in the image are often copy&paste image blocks from already played frames (example: click => change color, then revert to previous color)
many image fragments comes from graphical drawing primitives: line, rectangle, text fonts, icons

Screen capture and teleconferencing to share a screen is SO MUCH common, it is used in Citrix applications, YouTube tutorials, Team work programming, Chats … There should be extremely efficient algorithms to use the knowledge of image and video structures, and compress them much more efficiently than a regular camera based film (where .MOV is efficient).

I did few searches with Google (and DuckDuckGo), and I found that lossless .MKV, .WMV, .VP9 might be interresting candidates.

My first try was to convert the .mov to .mkv, which I did using “handbrake” application.

The result I got confirmed my thoughts… .MKT file is 3x to 10x smaller than .cap file !

It is a pity than the project java-screen-converter does not convert directly to .MKV file format, but produces hugly intermediate .MOV file (which is huge, and loss quality with JPEG computations) … then recompressing it with .MKV is not optimal… it should be even much better with a direct .CAP to .MKV conversion.

Someone interrested to contribute to github: java-screen-recorder.git for doing this ?

Yet Another Github project : screencast-compression-tool.git

After playing/refactoring a little with https://github.com/bspkrs/java-screen-recorder.git, I felt the need to do more image analysis algorithm on it. I also wanted to have better video file format conversion or tool supports, so I have tested humble.io project.

I have created a new repository, not cloning the previous one (I have reused only 2 files from java-screen-recorder, but restarting other all from the beginning).

https://github.com/Arnaud-Nauwynck/screencast-compression-tool

more things to come …