Bitscapes
'Bitscapes' provided a proof-of-concept client-server platform for streaming and rendering artwork into
a browser.
An image (or sequence of images) are pre-processed offline and the following data extracted:
- SURF features (SVG)
- Prewitt edges (SVG)
- Hough lines (SVG)
- Haar Cascade faces (SVG) - intersected with SURF and Prewitt to improve false-positives
- Palette (SVG) - essentially a matrix of colour
- Diff (SVG) - computed through standard python
- Blobs (SVG) - Python > ImageMagick > Potrace > SVG
All the SVG files are stored in files and streamed to the client (via PHP) when required.
The Javascript client comes in 3 parts: server, stage and renderer.
The server is a wrapper around a jQuery AJAX function, essentially just receiving the frames of
data from the server and posting them to the render worker.
The render worker implements the worker interface, receives the frame data and generates an
image for passing back to the stage.
The stage choreographs the rendering, asset disposing and any interactivity. Here is an example of how
a specific pre-processed MPG is streamed:
$(document).ready(function() {
stage.init(server);
stage.setRenderer(renderFrame_Voronoi);
//stage.setRenderer(renderFrame_Blur);
//stage.setRenderer(renderFrame_Curve);
//stage.setRenderer(renderFrame_Raw);
server.streamData('M2U00264.MPG', ["palette", "surf", "prewitt", /*"lines", "blobs", "diff", */"mini"]);
stage.play();
});
4 renderers were explored: RAW (no filter), Blur (BoxStack blur), Curve (quadraticCurveTo) and Voronoi.
SVG parsing was done in the client where needed.
As a first attempt at exploring generative techniques and data composition this was successful. The use of
SVG meant that some layers could be handled direct in a DIV layer with no canvas involvement (i.e. the actual
source image, blobs and pallete matrix could all bypass canvas).
The pre-processing of the images meant that any language could be used on the server to generate frame layer
data, extensive use of Python and OpenCV was used.
The worker implementation in the browser is stable and reliable, at the time there were less Javascript libraries
available for browser-based rendering but now this is less of a problem with tools like Processing.js
Additional tooling (a proper API server for accessing processed image sets and a tagging system) lead to a fairly
comprehensive proof-of-concept piece of work.