Actionscript 3 Bitmap Segmentor (Source Code Included)
22.01.08 @ 6:00 amThe demo below shows the basic functionality of the BitmapSegmentor class. See the notes below for playback limitations.
The BitmapSegmentor can segment any type of DisplayObject, however, I find segmenting video to be an especially effective method of showing both the features of this project and its performance. Unfortunately, due to changes in Flash Player 9.0.115.0, this demo will only work for the latest version of Flash Player (9.0.115.0) and does not seem to work on 9.0.115.0 for Mac OS X 10.5 (Leopard). If anyone on the Flash Player dev team is reading this, I am getting the well-known security violation when performing Bitmap.draw() for Leopard. Once everything gets straightened out with performing Bitmap.draw() on video, this collection of code will be all the more useful.
Download the Source
The source code for this project has gone through a fairly solid pass of cleanup and optimization. I have also put in your basic code documentation. I will continue to release the source within the demo provided so people can see how the classes are implemented and executed. As always, the only thing I would appreciate in return is adding this project to your del.icio.us bookmarks.
View/Download the source of the full presentation
This code is licensed under the GNU Public License. Make this code better (please), and (for everyone's sake) share it.
What it Does and How it Works
The core class, BitmapSegmentor, takes a specified DisplayObject and determines the individual segment size based on the number of rows and columns set in the constructor. A "grid" is then created by initializing the necessary amount of BitmapData and BitmapSegment objects. The BitmapSegment object is a custom class to contain properties of each segment's bitmapData and cooresponding x,y position. When the segment() method is called, Bitmap.draw() is performed on the specified target. The arrays which contain the pooled BitmapData and BitmapSegment objects are iterated through to copy the pixels of the target's bitmapData for each segment. The resulting copied pixel bitmap data is then linked to associated BitmapSegment object along with the bitmapData's x and y coordinates relative to the target's bitmapData.
The extension class, SampleableBitmapSegmentor uses the ColorSampler class I wrote a while back to not only break up a DisplayObject into a grid of Bitmaps, but also allow you to sample the Bitmap's pixel data to get properties such as the red channel, green channel, blue channel, hue, saturation and brightness. The SampleableBitmapSegmentor object contains the sample() method which performs the same operations as the segment() method except it also samples the color data of each segment. All segments are stored in an array of SampleableBitmapSegment objects. The SampleableBitmapSegment object contains ColorSampler instance to perform an extra sampling executions.
The latter class offers up the ability to easily sample and visualize a DisplayObject of any kind. The 'Interpret Sampled Color Values' in the demo uses the sample() method. As expected, the performance lowers when using this method.
actionscript, actionscript 3, as3, bitmap, bitmap segmentor, bitmapdata, color, color sampler, demo, display object, flash, flex, grid, sample, sampling, segments, segmet video
January 22nd, 2008 at 10:28 pm
$1.00 in Comment Love for May
If you just want to process video (or any IBitmapDrawable) to arrive at a down-sampled, interpolated grid, a cheap way to do it is to just ‘mosaic’ the source by way of scaling it down into a bitmap, then look at the related value of the color in each mosaic tile region (bitmap pixel). This allows you to rely on sampling and manipulating a lot less info and taking advantage of whatever flash is doing behind the scenes to scale. The cost here is you don’t end up with an object containing a clone of the original region’s whole data to easily display and transform (although that is sort of a different problem vs. getting a ‘value’ for the region’s average color/hue/etc. — and you could always reverse a grid back to its source by way of some simple factory methods performed on the downsampled data vs. the original source).
For example:
(1) Play a video (or set up any bitmap data provider)
(2) ‘Mosaic’ it… down-sample it into a bitmap data… this can either be done by (a) using a single bitmap data as a single buffer to capture the video into, scaling to fit the buffer during capture or (b) using 2 bitmap datas as a double buffer where the initial capture is unscaled, then is copied to a second bitmap (scaling prior to or during this second step).
(2a) Duplicate the video into a bitmap data whose size maps 1 pixel to each ‘region’ being captured. For example a 320 x 240 video source with ‘10 rows’ and ‘20 columns’ would be captured into a bitmap that is 32 x 12 pixels. Use a matrix transform to scale with the draw() method to scale the source to the output resolution during capture. You can apply smoothing as desired (depending on what sort of interpolation you want).
(2b) Duplicate the video into a bitmap data whose size matches the source (using draw()). Then either:
(2b1) nest that bitmap (buffer#1) into a parent object, then draw() the parent object into a lower resolution bitmap (buffer#2). This should get around having to bother with a matrix transform of any sort during the draw() step since you are drawing from a parent’s display space (so the scaled-down child data is captured in its scaled-down form vs. its unscaled form).
(2b2) copyPixels() that bitmap (buffer#1) into a lower resolution bitmap (buffer#2), using a matrix transformation to reduce the resolution during the copyPixels() operation. You can also decide whether you want to selectively mess with channels in this step.
(3) Once you essentially have a low-resolution bitmap of the original video (where flash has done the interpolation for you) you can then quickly traverse the low-resolution data to get a given sample regions average value (and in turn can look at that sample’s color as RGB, HSV or whatever else).
In this scenario instead of having an object for every slice you’d basically have a buffer (a bunch of pixel data where each pixel represents a sample region’s average value), and some utility methods for using the buffer’s data. You could still then allocate an object acting as a view for each sample - whereby the buffer is basically the model data provided to that view. You could just set an update() method on the view that gets passed the new buffer’s value when it is updated (so for each sample period you dispatch or notify the views that the buffer data has changed, passing them or referencing them their region’s pixel ‘value’).
Maybe I only took a fast look at your code - it looked quite good, and well-written. Where when you ’sample’ you are actually performing an average on each region by going through each pixel of the region (and keeping each pixel’s RGB in a sample set). This step you don’t need, since you can arrive at an average pretty efficiently just relying on flash to down-sample the region for you (and you can define from a few common interpolation methods — linear = no smoothing, bilinear = smoothing). Once you have the average you can easily compute HSV/brightness/etc. using external utility methods vs. methods on the object/data itself (the rgb2hsv, hex2rgb, etc. color space methods can certainly be static, and/or external to the samples themselves).
Another advantage to this hack is that you can use common Bitmap filters and transformations that flash applies (fast) to manipulate the downsampled buffer data (increase contrast, invert values, blur it, edge contrast it, etc.).
Hope you find this useful!…
N
January 22nd, 2008 at 11:37 pm
Wow Neil, thanks so much for writing such a thought-out and intelligent comment - you definitely win the best comment of the month award.
Your solution for sampling color is so much more graceful and simpler than mine - I can’t wait to implement it and observe the performance increases. I really do appreciate you taking the time to share this method as it definitely made this article much, much better.
April 4th, 2008 at 3:50 am
$0.00 in Comment Love for May
can we use colour matrix to transform images that are loaded in a UIloader through xml ,the images are animated using xml values(copy motion as xml).
i want to modify my xml in such a way that i could apply grey scale to it.
May 11th, 2008 at 9:54 pm
$0.50 in Comment Love for May
Great example, but perhaps a silly question, could this be used in Flash CS3? I’m thinking of an application with a picture in the background.
Magnus’s last blog post..EMUG Meetings in Abu Dhabi?