Particle System Language

The vast majority of paricle system implementations are authored with certain assumptions in place. Particles will have a lifetime and be expired after it, all points will be rendered as camera facing quads, all attributes will be animated across lifetime, etc. I started to view these things as simply channels of data and small “Programs” that modify them.

TL;DR;

I made a particle system engine which utilizes a domain specific language (DSL) for defining the data layout and the modifications to be run against that data on each simulation step. There are videos over there on those youtubes the kids are always talking about.

Basics

Initially a lot of time was spent prototyping the DSL. Either the google protocol buffer textual format (which is a lot like JSON) or Lua to represent the data. This was a convenience so that existing tech in the engine could be used and obviate the need to implement and maintain a language parser.

Structure

The data conditioning and utilization is as follows:

Lua Format -> Particle Compiler -> Byte Code -> Engine -> Simulate data systems like a boss.

There are two separate program types; System Programs and Particle Programs. Additionally there are init functions for both types.

Domain Specific Language (Lua files)

The actual language itself is a bit stiff for my tastes due to the fact that it needs to conform to Lua syntax given the current structure of things. The architecture of the pipeline would allow for an alternate layer that generates the intermediate byte code data (if not the actual binary byte code itself) by implementing and/or engaging a language parser on either the Lua or engine side. It could even be an external program. Likewise there are a wealth of language parser generation tools out there in the world and standardized ways of defining a language’s syntax. That will be a nice improvement and make it far more fluid and natural to author effects. There would also be opportunities to be had there to enforce certain paradigms through the syntax.

A particular trade-off here in the DSL is that the granularity of available commands is somewhat coarse. Edge-cases or potentially killer features need to be handled by explicitly adding new commands (or adjust old-ones) to support the new functionality. While that can be limiting or bothersome at times, it also gives the author a more concise set of tools. Reducing options reduces variables and that in turn can reduce the complexity of a problem space. Honestly I think that that simple equation pays dividends almost without fail. Minecraft (and derivative works such as Terreria), as well as most popular iOS games, offer very convenient and solid arguments to this effect; people armed with simple but powerful tools can and will get super creative… let’s hope that this works out similarly for me… I’d really like that…

Particle Compiler

LuaJIT introduced a system/feature called FFI. This allows you to use C structures and functions natively. There’s a lot more information over at LuaJIT’s site. The particle compiler tool is written in Lua and engaged by the input pipeline’s compile script (which will be discussed at some other time). It uses FFI to work with the same structures that the Engine Runtime uses internally and serializes those out to disk.

There is some foul, foul chicanery at play here to make the lua format feel and operate more like it’s own DSL. The particle system file is evaluated as lua in a limited environment where only functions that generate bytcode and a few others are available; mostly in an effort to keep the main environment clean, but also to allow for some trickery like defining a table called “_output” in the environment and letting the generation functions write to that.

Byte Code format

The byte code itself is basically an operation ID, a bitmask specifying which parameters are channel identifiers and which are statics, and the parameters themselves. This format could further be structured to allow for things like strict vectorized operations (e.g. taking exactly 4 float parameters per operation).

Engine Runtime

Inside of it all, well, there’s a big switch statement… not exactly the most interesting code ever written, but it gets the job done. The switch happens against the operation ID in byte code for this operation.

The operations generally take in the same data and work against all of the particles at once. The idea here is to maximize cache coherency and to limit the cost of processing of the byte code. Does this implementation achieve that? No, not as much as it should.

One particular issue is that the parameters to the operation can be channel identifiers or static values. In order to facilitate that there needs to be some conditional checks prior to the processing of each particle. Specifically because operations in ‘particle programs’ are expected to operate against a data channel for each particle.

As an example, the following particle schema/program would add ‘velocity’ to ‘position’ on each simulation step.

:::lua
particle_schema = {
    { name = 'position'; size = 3; };
    { name = 'velocity'; size = 3; };
}

particle_program = {
    AddVec3( 'position', 'velocity' );
}

This makes it necessary to read the ‘velocity’ channel for each particle. However if that were instead:

:::lua
    AddVec3( 'position', 0, 1, 0 );

We wouldn’t technically need to check to see if that parameter was a channel specifier or static values and try to process it… we could just use the same values (i.e. 0, 1, 0) for all of the particles. The most obvious optimization here would be to have multiple implementations of each operation for every permutation of parameters. Currently everything has been changing enough that it hasn’t made sense to really endeavor that, and to do it in a maintainable way some careful CPP macros or similar would probably be the way to go. Otherwise it would become far too easy to end up with bugs that only existed in certain permutations of parameter types. Even writing those words just now sort of bums me out.

Simulation

Initially the emission of particles was done purely based on time in a compulsory way. This was then moved into specific operations such as Emit, EmityByTime, and EmityByDistance that are executed in the system program. That shift really opened the door for a lot of interesting possibilities like a static number of particles; no lifetime, no creation, no destruction.

Results

The particle system grab bag:

Ribbon/Strand rendering:

Fire:

I’ll Be Back

Still to be done are interfaces for setting data in a system externally. That would enable one to use and abuse the particle engine in interesting ways; e.g. as a decal system, for bunches of bullets drawn in a single draw call, etc., etc. Something I’ve been wanting to play with as well is an operation that will do raycasts against a physics environment so paricles can be made to organically interact with the world.

…which sounds like a really good use of my time, doesn’t it? It’s not as though there are games to be made…

-r