As you might already know my forthcoming iOS game, Toxin involves shooting rapidly dividing toxic cells which eventually fill the playfield or burst to spawn flying enemies.

Now, not long ago I read an article about generating heat maps to display web statistics, and I thought it would be cool to use this technique to colourise the ever changing clumps of cells in Toxin. After finding out how they work, I knocked up a prototype in Processing before adding the effect to the game.

Generating heatmaps is pretty simple. Take a 2d grid of values, perform a Gaussian blur on it, then use each resulting value as an index into a gradient of colours.

For Toxin, the first step was to create a simple array of numbers representing the current state of the cell grid. Toxin cells can be different sizes. To simplify things, I store each big cell as several small cells.

Each cell is given an initial value of 255. When we draw our cells, we use a range from black to white, where values of 0 are black, 255 is white and values in between are shades of gray.

The cells are represented as a grid of numbers. Each cell is given the initial value of 255

The next step is to apply a simple box blur to this grid of numbers. Normally heat maps use Gaussian blur, but since Toxin’s grid is such a low resolution we can get away with using a simpler algorithm which is much faster to calculate. To learn more about blurring techniques, I recommend Alex Evan’s article on Gamasutra.

The cell grid after being blurred. Notice how the values have changed

Now look what happens when we draw our original cell grid using the values from the blur operation. Cells in the middle of large clumps have higher values than solitary cells.

Drawing the original grid with the blurred values

The final step is to colourise these grayscale values so they look cool. We do this by mapping these values onto a gradient I made in Photoshop. Values of 0 map onto the left of this gradient, 255 to the right, and the others to the colours in between.

When we draw the grid with the colours from the gradient we get our heatmap!

However, Photoshop’s scripting system is far from perfect. You cannot access pixel data from within scripts, nor can you draw lines or circles or anything like that. These limitations have become really frustrating to me.

For example, on the day I started work on Toxin, I needed to draw an ellipse with precise dimensions. The whole game takes place inside an ellipse, you see. I couldn’t do it in Photoshop by clicking and dragging, it was too inaccurate, so I wrote a C program using the Allegro game library to create the precise shapes I needed. These were then loaded into Photoshop and used as guides while I created the game artwork.

Later I discovered Processing, which is much quicker to develop in than C. I soon developed a workflow where I would go back and forth between Photoshop and Processing, generating procedural content then processing it or running it through scripts until I got the desired result.

Processing, however has some frustrating quirks of its own. As a simplified front end to Java, you often get Java errors that are incomprehensible without digging into the underlying platform. Also, it cannot effectively save semi-transparent PNGs which means that the anti-aliased sprites I was creating all had a solid background which had to be removed. I ended up redesigning my workflow and using the excellent Ghost and AntiGhost plugins from Flaming Pear to overcome these problems.

From graphics app to graphics platform

All these issues got me thinking about what I really want from a high end graphics program. What I want is something like a programmable, graphical Lisp Machine; a powerful graphics programming platform with the UI of an art application, not an art application with a bolted-on scripting system.

I want the ability to programmatically control every aspect of Photoshop from both external scripts and from a real-time REPL inside Photoshop that works like the command line used in some CAD programs.

I want to be able to type line(10,10,100,100) and have a line appear on my presently selected image. I want to be able to open the REPL and assign my current selection to a variable which I can later re-select with a simple command. I want to be able to access the pixel data in my images, and write my own filters in Javascript within the scripting system itself. I think anyone in high-end graphics or scientific imaging would love something like this.

A true graphics platform like I have described would likely have to be written from the ground up with programmability built in. Nevertheless I did spend some time finding out how far I could go with Photoshop’s native capabilities.

I had the idea of writing a Photoshop plugin that contained a number of drawing functions which could be called from scripts via a Javascript library that integrates with the Photoshop document model. So you could write PSDraw.line(10,10,100,100); in ExtendScript Toolkit, and my Javascript library would construct a call to the plugin which would draw the line.

Unfortunately, it’s not possible to get information from a plugin back to the Photoshop script thats running it, so it wouldn’t be possible to do a getPixel() or anything like that. At best, this plugin/script combo would give us a somewhat clunky way of drawing programmatically but little else that would justify the effort involved. (have you ever seen the Photoshop plugin API?)

As a secondary option I thought about actually embedding V8 inside a plugin which would contain an editor and debugger. Besides the ugliness of having 2 distinct Javascript engines running in the same app, the plugin architecture put paid to this idea. Photoshop plugins are one-shot operations. Load the plugin, set the parameters, perform the operation, then shutdown. My idea would only be effective if the plugin was a persistent part of the Photoshop environment.

So the only options then, would be for Adobe to integrate scripting more completely, or for someone else to start making what would be a true sucessor to Photoshop.

Processing provides a simplified front end to Java, ideal for quick experiments

The green cells in Toxin were all generated using a Processing program. Each cell is a circle made up of a number of connected spline segments. I animate the cell by displacing the spline control points using Perlin noise. The cells are then resized and colourized in Photoshop.

Left, a cell being rendered in Processing. Right, the finished result

As you can see, the finished product does not look exactly like the original Processing output. I had to do a lot of tweaking, re-rendering and reprocessing until I got them looking right. It was very time consuming. There were many moments where I wondered if I was going to get useable results at all.

Many of Toxin’s graphical effects have their origins in experiments I did with Photoshop blend modes. I discovered that certain combinations of layers and modes resulted in nice abstract animations when the layers were dragged around. I wrote photoshop scripts to perform these movements and spit out animation frames. Unfortunately these scripts revealed many bugs and inaccuracies in Photoshop and its scripting system. You only have to look through the source of my AtlasMaker Photoshop script to see how many strange workarounds are required to accomplish straightforward tasks.

To solve these problems I implemented Photoshop’s blending modes in Processing, and use this to create my animations. Below are a few frames from Toxin’s shot animation that demonstrate the technique. The way the colours shift throughout the animation is down to the movement of multiple layers blended into a simple white shot sprite. You can get a better idea of how this looks in the game by looking at the screenshots I posted a couple of weeks ago.

Many of Toxin’s sprites use animated image processing effects

I also used Processing as a test bed to work out algorithms before implementing them in the game. For example, Toxin has a weapon which freezes the edges of groups of cells. It didn’t take long to prototype this in Processing.

Testing a powerup that uses a flood fill algorithm to find edge cells

Stay tuned for more posts on Toxin, and a new version of my texture atlas Photoshop script which is in final testing…

Also, Toxin is the first game I have worked on since the Smartphone stuff I did around 2002-2003. I was desperate to get back into game development and I mistakenly thought I could begin again where I left off. It was like running into a brick wall. The first artwork I did was so bad I deleted everything, took the bus into Stratford on Avon and spent the rest of the day sitting by the side of the river in despair!

It took me some time to reach my old level again and then improve on it. As much as I have enjoyed developing Toxin and the creative discoveries I have made, there has been a great deal of struggle and fruitless experimentation involved.

The interior designer and UK TV legend Laurence Llewelyn-Bowen was an unlikely influence. On his TV home makeover shows he’d get his clients to collect images they liked and pin them to a board for inspiration. I had a Toxin folder on my laptop which I filled with images from ffffound and other design inspiration sites. When I got stuck I’d fire up IrfanView and spend half an hour blanky staring at images until some ideas popped up. I highly recommend this practice. A few of the images from my Toxin folder are in the image below.

That watch is a Dievas Aqualuna Blue Professional. No I can’t afford one yet. Buy Toxin!

The first thing I did was the main ship, which is the only thing I got right first time; it remained unchanged throughout development whereas everything else got remade at least three times! I think it works really well and has a nicely balanced, iconic look. I designed it in a vector drawing package which I later abandoned in favour of using rendered, animated splines created in a 3d program.

The background was next. It was essential to get the background style nailed as it would define the look of the whole game. I tried a few different styles as you can see in the rough drafts below.

One of my biggest goals here was to create something that looked good within the phone. Too often, mobile games just present windows on to the game world without any consideration for the device that is running the game. To me this is like designing a watch face without taking the case into account.

Next time I’ll talk about how Toxin uses procedural graphics and describe how I used Photoshop scripting and Processing, a Java based generative arts platform to create much of the game’s artwork.