Part of the big fat boatload of new operations in 1.0.17 are chromakey (matting) operations. This is literally the “green screen” effect blending the foreground and background together effectively.
There are many algorithms in this domain and make for fun reading. MediaLab implements color range and alpha blend matting.
Types of Matting
In color range, the cv::inRange creates a mask on the key color (range), e.g. “green”. The mask and its compliment guide the foreground and background copy/paste of pixels. There are numerous examples of this process laying around in C++ and Python. You can do these individual steps now in MediaLab. We also packaged it into one operation for convenience and performance.
Alpha blending translates the “key” color into a blend (composite) between foreground and background. A process courtesy of Petro Vlahos calculates the color of the combined pixels. The idea is to make the foreground “transparent” where the key color is, so the background color is opaque. Since this involves lots of Linear Algebra, it’s packaged for your convenience.
These new operations alone are useful, but some other new operations help too.
New Operations
First issue: how do we get a background? The Frame Source is our only image stream, the chromakeyed foreground. The new LoadImage operation handles it. You can now load multiple out-of-band images for other purposes, like compositing.
Second issue: this stuff only works if the two images are the same dimensions (and pixel format). It would be most convenient to get a “piece” of the background matching the size of the foreground in the right location. The new RoiExtract and RoiCombine operations handle it.
Example Pipeline
You can extract (copy) a background rectangle, chromakey it, and combine (paste) it back. A good example is putting a “logo” in the corner of an image. Let’s run through that pipeline quickly.
- Load Image – static background (1360×906) ->
background - Load Image – logo with green key (64×64) ->
logo - Crop – background size must match incoming frames (1280×720)
background->background_crop - Chromakey Range – take incoming frame (with blue key) and combine (
input,background_crop) ->combined - Roi Extract – copy patch (64×64) matching logo
combined->roi - Chromakey Alpha – take the patch and combine (
roi,logo) ->chromakeylogo - Roi Combine – take modified patch and paste back (
chromakeylogo,combined) ->output
