Last week I received my Oculus Rift DK2 headset which I ordered back in August. I had my eye on the Oculus DK1, the first version of their headset, but I was too busy getting Antigen out of the door to let myself be distracted by other projects. Now Antigen is out of the way, I can let myself experiment a little while I figure out what my next big project will be.

It comes in a padded, reusable cardboard box, which is fine for storage but if I was going to take it anywhere I would probably get a Pelican camera case or something similar, and cut some custom foam inserts for it, like this Oculus user has done.

The headset is surprisingly light, and connects to your computer with a sturdy cable that runs over the top of your head along one of the headset’s straps. I imagine this was done for weight distribution and to help keep the cable out of the way when you move around.

The cable connects to your computer via 1 USB and 1 HDMI port. There is a motion tracking IR camera that clips to your monitor like a webcam. This uses 1 USB port. There’s also an optional power supply if you want to use the USB port built into the headset. One device that uses this is the Leap Motion, which I’d like to get my hands on sooner or later…

The Rift also comes with two sets of detachable lenses; a longer lens that is already installed in the headset, and a shorter lens intended for nearsighted users.

Setup was easy. I downloaded the drivers from the Oculus website, installed them and as instructed, updated the headset firmware using a simple and straightforward preferences utility. This tool also lets you create user profiles for the Rift, containing your height and other details.

The Rift appears to your computer as a second monitor. You can view anything you drag on there, but it will look like crap unless it is rendered stereoscopically and positioned correctly. There is a DirectX style “Direct Mode” that lets applications talk directly to the Rift, but it doesn’t work on the Mac yet. (The Mac is a bit of a second-class citizen for the Rift at the moment. Unfortunately, most of the coolest demos are Windows only too.)

At this point I could hardly contain my urge to jack in to the matrix and become the console cowboy I’d always dreamed of being, so I hit “Show Demo” and braced myself.

I found myself sitting at a desk in cyberspace, somewhere on an infinite Tron-esque plain. The first thing that struck me was the stereoscopy, the 3d-ness of the lamp and the tower of playing cards on my desk. I really felt I could reach out and touch them.

The resolution is good at 960×1080 for each eye, but I don’t think it’s high enough for consumer use yet. I suspect that in order to approximate the usability and experience of a desktop monitor, VR displays will require a much higher relative pixel density. There is also a kind of “screen door effect” where you can see the pixel grid. Most of the time this isn’t too bad, but it is noticeable when you try and view details that are further away, such as the faces of game characters.

Text is readable as long as it is large and in the middle of the display. It’s a bit like looking at text on a C64 with a bad TV set. Towards the edges of the display it gets blurry and there is some chromatic aberration. You need to look at text directly in order to read it properly, which will have ramifications for UI design, particularly games where information is usually positioned at the edge of the screen.

All the same, these aren’t really criticisms, they are just statements about where the developers kit is right now. The DK2 is a prototype that a deranged experimenter committed person works with and adapts himself to, rather than a consumer product. I’ve not had a chance to try the new Crescent Bay prototype, but reports suggest that it eliminates the screen door effect and that we can expect the consumer version to be a real advance on what is currently available.

There’s a nice demo included with the SDK where you can walk around an Italian country house and its surrounding grounds. The house is on a cliff over the sea. You really feel like you can fall off when you approach the cliff. One thing I found interesting is how your body is tricked into reacting to things when you accidentally bump into them. It feels suddenly like there is something real there. This demo also shows how high quality shading and lighting can mitigate some of the shortcomings of the headset such as the resolution and screen-door effect. Another demo I tried, 4thFlrStudio by Brendan Coyle demonstrates this clearly, with its remarkable lighting and detail.

After a bit of messing about I got Steam and Half-Life 2 working on the Rift. Within minutes it gave me motion sickness! It clearly showed how VR requires both a rock-solid framerate and well-calibrated motion settings in order to be effective. Anything less than 75fps will likely send the user running to the toilet!

The presence of other characters in the game is startling. This was the first time I had encountered another being in VR, albeit a simulated one. At the beginning of HL2 you go through a police checkpoint and one of them thumps you in the chest. I actually felt it! The enemies were extremely intimidating. I dread to think what it is like on the later levels when you encounter the Hunters and Striders! I really felt I was inside the game, it’s kind of like a fuzzy lucid dream. I will have to experiment with the settings to see how far I can minimise the motion sickness.

The next thing for me to do is get the SDK going and see what I can create with it. I might also need a more powerful computer. Some of the demos I tried only ran at 16fps. My 2012 Macbook Pro is a great for general use and for my iOS development, but its NVidia 650m has about 1/7th the power of a top desktop GPU.

As you might already know my forthcoming iOS game, Toxin involves shooting rapidly dividing toxic cells which eventually fill the playfield or burst to spawn flying enemies.

Now, not long ago I read an article about generating heat maps to display web statistics, and I thought it would be cool to use this technique to colourise the ever changing clumps of cells in Toxin. After finding out how they work, I knocked up a prototype in Processing before adding the effect to the game.

Generating heatmaps is pretty simple. Take a 2d grid of values, perform a Gaussian blur on it, then use each resulting value as an index into a gradient of colours.

For Toxin, the first step was to create a simple array of numbers representing the current state of the cell grid. Toxin cells can be different sizes. To simplify things, I store each big cell as several small cells.

Each cell is given an initial value of 255. When we draw our cells, we use a range from black to white, where values of 0 are black, 255 is white and values in between are shades of gray.

The cells are represented as a grid of numbers. Each cell is given the initial value of 255

The next step is to apply a simple box blur to this grid of numbers. Normally heat maps use Gaussian blur, but since Toxin’s grid is such a low resolution we can get away with using a simpler algorithm which is much faster to calculate. To learn more about blurring techniques, I recommend Alex Evan’s article on Gamasutra.

The cell grid after being blurred. Notice how the values have changed

Now look what happens when we draw our original cell grid using the values from the blur operation. Cells in the middle of large clumps have higher values than solitary cells.

Drawing the original grid with the blurred values

The final step is to colourise these grayscale values so they look cool. We do this by mapping these values onto a gradient I made in Photoshop. Values of 0 map onto the left of this gradient, 255 to the right, and the others to the colours in between.

When we draw the grid with the colours from the gradient we get our heatmap!

Creating good looking graphics for an abstract game like Toxin is harder than it first appears. Not only are your images stripped bare of most signifying information, requiring a real eye for the principles of graphic design, but creating the artwork itself – giving each line the right balance of sharpness and glow, demands careful image processing and a great deal of experimentation.

When I started making Toxin, the graphics were a real struggle. It took me an enormous amount of work to develop the look of the game. To give you an idea, I have a folder on my laptop with all of Toxin’s artwork in, and it weighs in at 967meg! All for a little mobile game!

Eventually all my experimentation began to pay off, and I developed some powerful techniques for creating animated multicoloured abstract sprites. In this post I want to describe one of my favourite methods.

I start by creating our sprite out of spline shapes in 3D Studio Max. Unlike many 3D programs, Max can render splines at various thicknesses, animate them, and can even apply texture maps and materials to them.

I then use gradient maps to mask out areas of the shapes that I want to be different colours. These gradient maps are then animated, so the different coloured areas can actually move around the shape.

Then, I do two sets of renders. First I render the shape with the masks on, so there’ll be a hole wherever the coloured area will be. Then I do another render with the mask inverted, so that only the coloured area is visible.

This diagram shows each step in the creation of one frame of animation for a Toxin sprite. The total animation is 64 frames long. This intricate process is largely automated using Photoshop scripts

Then its time to start up Photoshop. After deciding what colours I am going to use, I run each set of sprites through a Photoshop script that colourises them and adds a glow. To create the glow, I duplicate the image into another layer, Gaussian blur it, then additive-blend it on top. Sometimes I use several glow layers with different Gaussian blur parameters. I also find that using image->adjustments->Hue/Saturation to colourise gives me the best results.

The final step is to compose the two sets of colourised renders into a single sprite. I do this with another Photoshop script. The end result is a lot more interesting than the basic Geometry Wars style sprites everyone else is using, I think.

However, Photoshop’s scripting system is far from perfect. You cannot access pixel data from within scripts, nor can you draw lines or circles or anything like that. These limitations have become really frustrating to me.

For example, on the day I started work on Toxin, I needed to draw an ellipse with precise dimensions. The whole game takes place inside an ellipse, you see. I couldn’t do it in Photoshop by clicking and dragging, it was too inaccurate, so I wrote a C program using the Allegro game library to create the precise shapes I needed. These were then loaded into Photoshop and used as guides while I created the game artwork.

Later I discovered Processing, which is much quicker to develop in than C. I soon developed a workflow where I would go back and forth between Photoshop and Processing, generating procedural content then processing it or running it through scripts until I got the desired result.

Processing, however has some frustrating quirks of its own. As a simplified front end to Java, you often get Java errors that are incomprehensible without digging into the underlying platform. Also, it cannot effectively save semi-transparent PNGs which means that the anti-aliased sprites I was creating all had a solid background which had to be removed. I ended up redesigning my workflow and using the excellent Ghost and AntiGhost plugins from Flaming Pear to overcome these problems.

From graphics app to graphics platform

All these issues got me thinking about what I really want from a high end graphics program. What I want is something like a programmable, graphical Lisp Machine; a powerful graphics programming platform with the UI of an art application, not an art application with a bolted-on scripting system.

I want the ability to programmatically control every aspect of Photoshop from both external scripts and from a real-time REPL inside Photoshop that works like the command line used in some CAD programs.

I want to be able to type line(10,10,100,100) and have a line appear on my presently selected image. I want to be able to open the REPL and assign my current selection to a variable which I can later re-select with a simple command. I want to be able to access the pixel data in my images, and write my own filters in Javascript within the scripting system itself. I think anyone in high-end graphics or scientific imaging would love something like this.

A true graphics platform like I have described would likely have to be written from the ground up with programmability built in. Nevertheless I did spend some time finding out how far I could go with Photoshop’s native capabilities.

I had the idea of writing a Photoshop plugin that contained a number of drawing functions which could be called from scripts via a Javascript library that integrates with the Photoshop document model. So you could write PSDraw.line(10,10,100,100); in ExtendScript Toolkit, and my Javascript library would construct a call to the plugin which would draw the line.

Unfortunately, it’s not possible to get information from a plugin back to the Photoshop script thats running it, so it wouldn’t be possible to do a getPixel() or anything like that. At best, this plugin/script combo would give us a somewhat clunky way of drawing programmatically but little else that would justify the effort involved. (have you ever seen the Photoshop plugin API?)

As a secondary option I thought about actually embedding V8 inside a plugin which would contain an editor and debugger. Besides the ugliness of having 2 distinct Javascript engines running in the same app, the plugin architecture put paid to this idea. Photoshop plugins are one-shot operations. Load the plugin, set the parameters, perform the operation, then shutdown. My idea would only be effective if the plugin was a persistent part of the Photoshop environment.

So the only options then, would be for Adobe to integrate scripting more completely, or for someone else to start making what would be a true sucessor to Photoshop.

Processing provides a simplified front end to Java, ideal for quick experiments

The green cells in Toxin were all generated using a Processing program. Each cell is a circle made up of a number of connected spline segments. I animate the cell by displacing the spline control points using Perlin noise. The cells are then resized and colourized in Photoshop.

Left, a cell being rendered in Processing. Right, the finished result

As you can see, the finished product does not look exactly like the original Processing output. I had to do a lot of tweaking, re-rendering and reprocessing until I got them looking right. It was very time consuming. There were many moments where I wondered if I was going to get useable results at all.

Many of Toxin’s graphical effects have their origins in experiments I did with Photoshop blend modes. I discovered that certain combinations of layers and modes resulted in nice abstract animations when the layers were dragged around. I wrote photoshop scripts to perform these movements and spit out animation frames. Unfortunately these scripts revealed many bugs and inaccuracies in Photoshop and its scripting system. You only have to look through the source of my AtlasMaker Photoshop script to see how many strange workarounds are required to accomplish straightforward tasks.

To solve these problems I implemented Photoshop’s blending modes in Processing, and use this to create my animations. Below are a few frames from Toxin’s shot animation that demonstrate the technique. The way the colours shift throughout the animation is down to the movement of multiple layers blended into a simple white shot sprite. You can get a better idea of how this looks in the game by looking at the screenshots I posted a couple of weeks ago.

Many of Toxin’s sprites use animated image processing effects

I also used Processing as a test bed to work out algorithms before implementing them in the game. For example, Toxin has a weapon which freezes the edges of groups of cells. It didn’t take long to prototype this in Processing.

Testing a powerup that uses a flood fill algorithm to find edge cells

Stay tuned for more posts on Toxin, and a new version of my texture atlas Photoshop script which is in final testing…

The main way of going about it is to use a custom editing program. Developers either write their own specifically for each game, or they’ll use one provided by a third-party game engine like Unity, Source or UDK. Sometimes, they might take a popular modding tool like one of the Quake editors and use it for their own ends.

These editors allow you to create levels visually; you can draw maps, position items and enemies, draw movement paths, edit dialog text and set up events to occur at different points in the level. All this information is saved in a structured file that the game is programmed to understand.

The complexity of these tools and the files they create depends on the game. Some of them are as large and complex as the 3d modelling tools used by animators and architects; others are simple programs coded up in an afternoon that allow a programmer to avoid some laborious data entry.

I’ve always enjoyed writing game development tools. My first map editor was written in 1994 in Blitz Basic on the Amiga for a strategy game I designed that went nowhere. I probably spent more time polishing the editor than working on the game itself. Not long after I wrote an isometric editor that let you sculpt terrain like in Populous, again, for a game that failed to materialise.

My last game, Starblaster didn’t use an editor; the levels were stored as a script, but when I started Toxin, my forthcoming iPhone game, I knew it was time to break out the old skills.

The Toxin Level Editor

The Toxin Editor uses the same Opengl graphics engine as the iOS game

The Toxin Level Editor is my first ever Mac program, and it’s pretty clunky. It was my first attempt at writing a program in Objective C, using the famous Interface Builder which originally came out of NeXTStep. I always had a thing about NeXTStep, so it was exciting to finally write a program with its direct descendant.

Once I got the basics up and running, I needed to decide how I was going to save the levels. While many developers create their own file formats, I decided to use an existing one. That way I could use an off-the-shelf library to save time and effort.

Having worked in web application development for the past five years or so, I had some experience with XML and JSON. These are text based formats commonly used by web services to exchange information. I chose JSON as my level format because of it’s simplicity and readability. I use the SBJSON library in the editor and in the game for loading and saving.

Levels in detail

Toxin levels contain four main things. A list of cells, a list of linear events, a list of timed events and a set of general preferences that are applied to the whole level.

The cells are toxic blobs which spawn and multiply and gradually fill the playfield. The editor allows me to draw cells into the level and directly set their parameters.

Events in Toxin are instructions to the game to perform an action such as add an enemy to the game, set off an explosion or play a sound effect.

Linear events are fired off one after the other from the start of the level. They are the main way I add mobile enemies to the game. I can split the events into waves using a WAIT event which instructs the game not to add events until the previously created enemies are all dead. Enemies have standard parameters for position and angle, and a set of custom parameters defined as key, value pairs.

Timed events let me perform actions and spawn enemies at a specific time, or at a random time within a range. I did toy with the idea of having rule based events that I could fire off when certain game conditions were met. In my next game I plan to embed a scripting language to handle this kind of thing.

Adding an entity to the level. The linear events list is used to construct consecutive waves of enemies. The entities can have many custom parameters

The levels are saved as a bunch of text files which are then zipped and put into the game. I think this was a mistake. While it’s a good technique for larger games, with Toxin I could have got away with storing them in a single file.

If I were working on a desktop game I would try and build the editor into the game itself, as well as give the game a console that lets me manipulate level data and game parameters during play. This isn’t really feasible on a mobile app, though you might be able to do it on a tablet. I do get tired of having to zip all my level files, then replace the zip inside the project and recompile every time I want to try something.

I have a lot more Toxin posts planned, so stay tuned. Let me know if you have any questions or want more details.

Over the next few posts I want to write a bit about how my first iPhone game, Toxin, is being developed. Although I’ve been programming for a long time, this is my first ever project in the Apple ecosystem; I hadn’t even used a Mac before starting this game.

Toxin is written in C++ inside a thin layer of Objective C. I did this for a number of reasons. Firstly I am much more familiar with C++. Secondly, as the most common game development language I knew I would have better luck finding additional libraries and answers to any development problems if I worked in it.

It also meant I could reuse some of my own code. Toxin actually contains code used in my first game, Starblaster (2003). Some of it goes back to DirectX stuff I did in the late 90’s. I always liked having a sense of continuity in my work, a feeling that all my games are connected.

The Objective C layer handles iOS specific stuff like getting the app started up, passing touch information to the game logic, and setting up threads for Toxin’s concurrent level loading. The sound library I use, CocosDenshion is also in Objective C. I wouldn’t rule out doing a whole project in Objective C. I’ve enjoyed learning it and I’ve used it to create several utilities already, including the Toxin Level Editor, which I will probably write about soon. I would, however, want to spend some time comparing the performance of the Objective C Container libraries with the STL beforehand.

Engine? What Engine?

Now, some of my friends believe that the main reason Toxin has taken me so long is because I didn’t use one of the popular 2d engines out there like Cocos2d and instead wrote all my own graphics code. They don’t believe me when I tell them that large engines are unnecessary for small games (though I would never stop anyone from using them) and that I could rewrite my entire graphics system in about 2 weeks.

My “engine” if you could call it that, is a basic set of classes for things like sprite drawing, line drawing, loading textures, and drawing basic primitive shapes. All sprite and line drawing is batched using vertex buffers for maximum speed. The classes are loosely related; I just add something when I need it, not worrying about architecture or anything. The whole thing is about 7000 lines of code, and that includes some significant utility classes I wrote, such as my keyframe interpolation system. I use this all over the place to animate transitions, colour blends and movements of all kinds. I wrote interpolators for many different data types and I’m sure I could reduce the lines of code by making them into template classes.

I used this graphics system to build Toxin’s particle engine which I based on John van der Burg’s Gamasutra article, Building an advanced Particle System. The particles use some simple physics to bounce them off the inside of the elliptical playfield.

At one point, the game had elliptical gravity: When a particle hit the ellipse, gravity would activate for that particle and it would “fall” into the ellipse, over 360 degrees. It was a cool effect but it didn’t look like anything you would expect to happen, so I replaced it with simple bouncing which is much more satisfying to watch.

Some of the enemies in Toxin use the Chipmunk physics engine. One of the things I’m proud of in this game is how you can easily mix and match different types of objects. Some use Chipmunk, others use simple bounding box and oriented-bounding box collision.

Game structure is accounted for by a basic game state manager with simple transitions and there’s a resource managment system that lets me load and unload graphics whenever I need to.

After reading Mike McShaffry’s Game Coding Complete I wrote a messaging system which allows different parts of my game to easily communicate with each other with a minimal amount of dependency. I use this messaging system for spawning enemies, creating explosions, and playing sound effects.

Next time I’ll write something about how Toxin levels work, and I’ll describe my first native Mac program, the Toxin Level Editor. Please let me know if you have any questions or if you want more information about any of the things in this post.

Toxin is an abstract puzzle/shooter for the iPhone and iPod Touch, running on iOS3.0 and above. The basic gameplay is simple. You control a ship that rotates around the edge of an elliptical playfield, shooting inwards. If anything touches the edge you lose energy. Lose too much energy and you die.

Inside the ellipse are 99 levels of swarming biological foes, multiplying toxic cells, physics puzzles and colourful particle effects.

Toxin was inspired by a number of different games including Geometry Wars, Tempest, Space Invaders and Ebonstar, but the biggest influence was Microbes, released by Vision Software in 1991. I really enjoyed this back in the day, and like many innovative Amiga titles it seems to have disappeared from the consensual history of video games. You can play it with UAE, but I dont think the emulation is perfect.

I also tried to add some simple puzzle concepts to give the game more depth but without affecting the pace too much. The design process involved me sitting down with a coffee every night after work, staring at my sketchpad and thinking, “now what can I do inside an ellipse?”

Once this game is done, I am thinking about writing a Toxin “Remix” called Antigen. It will have the same ship-in-an-ellipse structure but the core game will be more puzzle based and slower paced as well as having different artwork. I think the idea of a game remix could work well at low pricepoints, a little like the way remix CDs used to accompany singles.

Soon I’ll write a little on how the artwork was done, and how it went through a long, painful process of development. I am also looking into producing a gameplay video, so stay tuned…

This StackOverflow discussion was useful, along with this guide to the png format from the W3C, but unfortunately, Apple threw a spanner in the works.

If you’ve been programming the iPhone for a while you’ll know that by default, Xcode premultiplies the alpha channel of PNG files before adding them to the application bundle. It also adds its own undocumented data chunk to the beginning of the file, breaking PNG guidelines which state that the IHDR chunk should always be at the beginning.

This Objective C function will return the size of an image for both standard and Apple PNGs. It scans through the file looking for the IHDR chunk, making no assumptions about its position. It should be easy enough to convert to other members of the C family.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57

#include <stdio.h>
 
//----------------------------------------------------------------------------
// GetPNGSize() - gets PNG Dimensions without loading any data
//----------------------------------------------------------------------------
BOOL GetPNGSize(const char *fileName,CGSize *pngSize)
{
    // PNG Signature
    unsigned char png_signature[8] = {137, 80, 78, 71, 13, 10, 26, 10};
    // IHDR chunk identifier
    unsigned char ihdr_signature[4]={'I','H','D','R'};
 
    FILE *fp;
    if( (fp = fopen(fileName,"rb")) == NULL )
    {
        return NO;
    }
 
    // load png signature and check it
    unsigned char sigBuf[8];
    fread(sigBuf, 1, 8, fp);
 
    if(memcmp(&sigBuf[0], &png_signature[0], 8))
    {
        fclose(fp);
        return NO;
    }
 
    // examine chunks until we find IHDR
    BOOL done = NO;
    while(!done && !feof(fp))
    {
        // get chunk length and type
        unsigned char chunkInfo[8];
        fread(&chunkInfo, 1, 8, fp);
 
        long length = (chunkInfo[0] << 24) | (chunkInfo[1] << 16) | (chunkInfo[2] << 8) | chunkInfo[3];
 
        // is this the IHDR?
        if(!memcmp(&chunkInfo[4], &ihdr_signature[0], 4))
        {
            unsigned char sizeInfo[8];
            fread(sizeInfo,1,8,fp);
 
            pngSize->width = (sizeInfo[0] << 24) | (sizeInfo[1] << 16) | (sizeInfo[2] << 8) | sizeInfo[3]; 
            pngSize->height= (sizeInfo[4] << 24) | (sizeInfo[5] << 16) | (sizeInfo[6] << 8) | sizeInfo[7];
            done = YES;
        }
        else 
        {
            // skip over the data and checksum and go to next chunk
            fseek(fp, length+4 , SEEK_CUR);
        }
    }
    fclose(fp);
    return done;
}