Journal Entry 63 – Profiling Reflections

This will be a quick one, because I’m knee deep in refactoring some of the reflection code. I use occlusion culling on reflective surfaces to ensure that I only render the reflection when necessary. Unfortunately, occlusion querying requires the GPU to finish rendering the surface that is being tested against. This is basically equivalent to calling glFlush, which can stall the pipeline. On my previous GPU this didn’t appear to be a huge deal, but on my new GPU this is a big problem! Check out this screenshot from GPU PerfStudio:

I've never seen my CPU so busy and my GPU so under utilized.  What's going on?
I’ve never seen my CPU so busy and my GPU so under utilized. What’s going on?

What is my CPU so busy with? Lets dig into the OpenGL API calls using the same profiler:

Wow, my CPU is sitting there polling for an occlusion query.  This can't be good.
Wow, my CPU is sitting there polling for an occlusion query. This can’t be good.

Ok, I think I’ve found the problem. Lets track down the offending C# code.

while (queryState[0] == 0)
{
   Gl.GetQueryObjectiv(queryObjects[y], GetQueryObjectParam.QueryResultAvailable, queryState);
}

Yaaaaa, that’s a problem. We can use the query results from the previous frame, incurring a single frame latency before the reflection will be rendered, but will anyone notice? It just takes a few lines to re-order the commands and draw the water based on the previous frame occlusion query. Here are the results!

By waiting until the next frame to perform the query, we can guarantee the query is ready and remove the bottle neck.  This is all at the expense of one frame delay before updating the reflection.
By waiting until the next frame to perform the query, we can guarantee the query is ready and remove the bottle neck. This is all at the expense of one frame delay before updating the reflection.

Looks like we’re back in business! Profiling is a very nice way to see what is impacting the performance of the application. Until next time,

Giawa

Journal Entry 62 – Lua Integration

I ended up playing with my CNC machine longer than expected, so tonight will be a pretty quick blog post! I want to take a bit of time to chat about the Lua scripting language and how Summer Dawn (and LLF) interface to Lua. I use Lua as a scripting language to help enable the modding community, and to make it easy for anyone else to create content for the game. Summer Dawn uses NLua, which are bindings and a wrapper for using Lua within the .NET framework. It makes interfacing with Lua pretty straight forward. After acquiring your copy of NLua (which is under the permissive MIT license) you have to create a Lua environment.

var lua = new NLua.Lua();

Now we can execute Lua from either C# code or from a file. Here’s an example of a fibonacci calculation written in Lua, with some simple memoization to cache intermediate values. There are probably more efficient ways to do this, but it gets the job done.

lua.DoString(@"
cache = {}
 
function fibonacci (val)
  if (val < 1) then
    return 0
  elseif val < 3 then 
    return 1
  elseif (cache[val] ~= nil) then
    return cache[val]
  else
    cache[val] = fibonacci(val - 2) + fibonacci(val - 1)
    return cache[val]
  end
end");

We can then get a reference to this function and call it from C# code!

var scriptFunc = lua["fibonacci"] as NLua.LuaFunction;
 
for (int i = 1; i < 35; i++)
{
    double? result = (double?)scriptFunc.Call(i).First();
    Console.WriteLine("fibonacci({0}) = {1}", i, result);
}

As an aside, I use ‘First()’ here because Lua functions can return multiple items, so Call returns an array of objects. I only want the first object returned, so I am ussing the Linq extension method. I could have simply used Call(i)[0] for the same effect.

Okay, so we can now execute Lua code The next step is figuring out how to call C# code directly from Lua. Well, that’s pretty easy too. Here’s a simple class I wrote which has the ability to write to the Console:

public class ConsoleWriter
{
    public ConsoleWriter()
    {
    }
 
    public void Print(string text)
    {
        Console.WriteLine(text);
    }
}

We need to assign an instance of this class to a Lua variable. In this case, I will assign a new instance of ConsoleWriter to a Lua variable named instance. Next I will call the Print method on the instance variable, which will call the corresponding Print method in C# code.

lua["instance"] = new ConsoleWriter();
lua.DoString("instance:Print(\"Hello from Lua\")");
Here's a screenshot of the fibonacci sequence and printing to the console from Lua!
Here’s a screenshot of the fibonacci sequence and printing to the console from Lua!

The result is “Hello from Lua” printed to the Console. This is very powerful, as Lua code can execute C# code directly, and vice versa. This forms the basis for questing, NPC text and so on. I hope to have an example of a quest written in Lua soon. I am now working on transitioning from the current hard-coded quest system to the flexible and scriptable system that Lua enabled.

I hope to have more time tomorrow for a bigger blog post. I know I promised a water wheel, and that is in the works, I just couldn’t get it finished tonight. I also want to take a minute to talk about today’s reddit post and my new twitter followers. I want to thank you all for the positive feedback I’ve gotten. Here are a few of the common questions I am getting.

When is your game coming out?

I can’t promise a release date, but I’m hoping to release an alpha version in Q1 of 2015. No promises though! I want to have a solid feature set before releasing.

Can I stream your game once I get it?

Absolutely, you are free to stream the game and/or record and store your game sessions in any format or medium that you would like.

How much will the game cost?

The alpha is absolutely free. I wouldn’t feel right charging a fee for a game that isn’t complete.

What’s the best way to get new information about the game?

Follow me on Twitter! That’s where you’ll find out the latest information about LLF and my work.

Thanks again, and happy coding!

Giawa

Journal Entry 61 – Rotating Voxel Chunks

Tonight will be a pretty quick write up, but I’d like to share some of what I’ve been working on these past few days. Over the weekend I was thinking about procedural generation, and decided to start working on a procedural farm. One of the first items I tackled was a fence around the property, which was difficult to do. I would have to create fence assets that ran in both north to south and east to west directions, as well as any corner pieces, etc that I would require. It would be nice if I could somehow create an asset once, and then include information in the chunk ID to rotate the chunk by an arbitrary amount.

The final result was simply adding a new voxel type. As I’ve mentioned before, each of my voxels is a 16 bit ushort. This 16 bit value has many different uses. It can represent a color with 5 bits for each of the R, G and B components. It can also be a transparent color, or contain an ID which references another voxel chunk, it can be a plant, or water, or anything else that I would like to dream up (as long as it fits in the 16 bit space). I’ve created a new voxel type that includes a 10 bit ID and a 2 bit rotation, allowing for the item to be rotated 0, 90, 180 or 270 degrees.

Here we use the same model, but rotate it 0, 90, 180 and 270 degrees to get a more detailed scene.
Here we use the same model, but rotate it 0, 90, 180 and 270 degrees to get a more detailed scene.

This was actually very simple to do, and only required a small modification to the code that deals with processing what I’ve called ‘detailed voxel chunks’. Detailed voxel chunks are voxels that exist in a world chunk that reference another voxel chunk. A simple example would be a plant, which is itself a voxel chunk that is placed within a parent voxel chunk. These chunks can be embedded indefinitely, and I needed to make a small modification to this bit of a code to allow for rotation.

// if this is a chunk that can be rotated then we need to do a bit more work
if ((t & 0xF000) == 0x9000)
{
    // swap the x and z sizes since we've rotated this chunk either 90 or 270 degrees
    if ((t & 0x0C00) == 0x0C00 || (t & 0x0C00) == 0x0400)
    {
        position = BoundingBox.Min + new Vector3(x - center.z * scale + 0.5, y, z - center.x * scale + 0.5) + new Vector3(chunk.minz, chunk.miny, chunk.minx) * scale;
        box = new AxisAlignedBoundingBox(position, position + new Vector3(chunk.maxz - chunk.minz, chunk.maxy - chunk.miny, chunk.maxx - chunk.minx) * scale);
    }
    else box = new AxisAlignedBoundingBox(position, position + new Vector3(chunk.maxx - chunk.minx, chunk.maxy - chunk.miny, chunk.maxz - chunk.minz) * scale);
 
    // calculate the rotation angle of the detailed voxel chunk
    float angle = (float)((t & 0x0C00) >> 10) * (float)Math.PI * 0.5f;
 
    // first we must translate the chunk to the origin, then rotate it, then translate it to the desired position
    modelMatrix = Matrix4.CreateTranslation(new Vector3(-center.x, 0, -center.z)) * 
                  Matrix4.CreateRotationY(angle) * 
                  Utilities.FastMatrix4(new Vector3(x + 0.5, y, z + 0.5), new Vector3(scale, scale, scale));
}

The code will first check to see if the voxel is a rotated voxel chunk. If so, it will then calculate the correct bounding box for the chunk. The bounding box is used for picking, and depends on the rotation (we need to swap the x and z when the chunk is rotated either 90 or 270 degrees). Then we compute the rotation, which is determined via 2 bits in the 16 bit voxel identifier. Finally, we create a model matrix for this voxel chunk. The model matrix is a bit of a pain, because rotations must happen around the origin. To accommodate this we first translate the voxel chunk into the origin, then we perform the rotation, and then we translate it back out to the designated position.

Finally, we can build the fence that we set out to build! This uses the same fence voxel chunk for the entire fence, but it is rotated in code. This makes it easier on the asset creators (you only need one model and can re-use several times) and on the procedural generation. I’m hoping to be able to make all sort of cool new effects with the help of this voxel type.

This fence can be easily built procedurally with the ability to rotate voxel chunks.
This fence can be easily built procedurally with the ability to rotate voxel chunks.

Tomorrow I’m going to start tackling the creation of a full water mill! I’d like to be able to demo the crafting system and be able to turn wheat into flour. Stay tuned,

Giawa

Journal Entry 60 – One Year Later

It has been exactly one year since my last journal entry! Yes, I’m alive, and the news of awful things happening to me have been greatly exaggerated. Let’s take a look to see what I’ve been up to, and where things are going from here.

First, I’ve continued to be active on Twitter, posting several screenshots throughout the past several months. If you aren’t already following me there, I totally recommend it if you want updates on Live Love Farm! That brings me to my second bit:

I’m now working on a game that uses the Voxel engine, and it is called Live Love Farm (LLF). You can sign up for alpha access here: http://llf.giawa.com and the alpha is totally free! This is a Voxel game that is inspired by games such as Harvest Moon, Zelda and World of Warcraft. I hope to have the alpha available shortly, although there’s still lots of work to do!

Anyways, you probably don’t want to hear my ramblings. You’re here for some code! So, let’s take a look at some of the stuff I am playing with. A few days ago I posted this screen shot:

With the new classes and particle effects this water wheel comes to life!
With the new classes and particle effects this water wheel comes to life!

The first step in creating this rotating water wheel is to actually create a model for the wheel. There’s a few ways to do this, and the editor comes in handy. However, instead of trying to draw perfect circles in voxels I enlisted the help of my Utilities classes. I have a simple ‘FillCircle’ and ‘DrawCircle’ method which can also be used to place voxels only at particular points along the circle (via an override). Here’s the method prototype for that:

public static void FillCircle(ushort[] data, Point3D point, int radius, ushort c, double step = 0.05)

I can use this FillCircle method to create the spokes in the water wheel, by setting the ‘step’ value to only try to fill the circle along 8 points. Since a circle goes from 0 to Math.PI * 2 we can set the step value to Math.PI / 4 to get 8 spokes. I also add the hub that connects the water wheel to the wheel house, providing power to whatever machinery lies inside.

// draw the spokes of the water wheel
Geometry.FillCircle(voxelData, new Point3D(16, 0, 16), 14, 2319, Math.PI / 4);
Geometry.FillCircle(voxelData, new Point3D(16, 7, 16), 14, 2319, Math.PI / 4);
 
// draw the center bar which will connect to the wheel house
Geometry.DrawLine(voxelData, new Point3D(16, 0, 16), new Point3D(16, 7, 16), 12684);

Now we can draw the wheel itself, which is pretty simple. I just used two circles to double up the width.

// draw the wheel itself
Geometry.DrawCircle(voxelData, new Point3D(16, 0, 16), 13, 2319, Math.PI / 80);
Geometry.DrawCircle(voxelData, new Point3D(16, 0, 16), 14, 2319, Math.PI / 80);
Geometry.DrawCircle(voxelData, new Point3D(16, 7, 16), 13, 2319, Math.PI / 80);
Geometry.DrawCircle(voxelData, new Point3D(16, 7, 16), 14, 2319, Math.PI / 80);

Finally, we can draw the buckets, which are added via a few more DrawCircle calls. I have created a loop here which varies the y value so that the buckets have some height to them.

// draw that water wheel buckets
for (int i = 1; i < 7; i++)
{
    Geometry.DrawCircle(voxelData, new Point3D(16, i, 16), 13, 12684, Math.PI / 10);
    Geometry.DrawCircle(voxelData, new Point3D(16, i, 16), 14, 12684, Math.PI / 10);
    Geometry.DrawCircle(voxelData, new Point3D(16, i, 16), 15, 12684, Math.PI / 10);
}
Using math to make some object in the world!
Using math to make some object in the world!

Now we need to orient the water wheel and let the engine know that it needs to be rotated a little bit every frame. This is very simple using the MovingVoxelChunk class. MovingVoxelChunk implements the IDrawable interface, which allows us to place these objects in the World easily. An update callback is run every frame which can affect the position (and other aspects) of the voxel chunk.

// load the water wheel and build it
VoxelChunk waterwheelBucket = new VoxelChunk(TerrainShader.Program, "vox/windmill/water_wheel.zvc");
waterwheelBucket.BuildWithOcclusion();
 
// this is the initial Z rotation of the wheel
float rotation = 0f;
 
// now we create a MovingVoxelChunk object with a callback
MovingVoxelChunk waterwheel = new MovingVoxelChunk(new Vector3(9, 8, 16.5), waterwheelBucket, (chunk) =>
    {
        // the callback updates the rotation value and generates a new model matrix
        rotation += Time.DeltaTime * 0.5f;
 
        // the model matrix consists of first moving the water wheel to the origin (-16, 0, -16)
        // now create a Z rotation (the is what rotates the water wheel)
        // and translate/scale the water wheel to the correct position
        return Matrix4.CreateTranslation(new Vector3(-16, 0, -16)) * Matrix4.CreateRotationZ((float)-Math.PI / 2) * Utilities.FastMatrix4(chunk.Position, new Vector3(0.5, 0.5, 0.5), Vector3.Right, rotation);
    });
 
// add the MovingVoxelChunk to the world and we're done!
World.AddObject(waterwheel);

Of course, there’s nothing to stop us from adding particles and performing other operations in the callback function, which is how I generate the particles for the original screenshot. This technique is what is used for wind mills, fires, and pretty much any voxel chunk that is moving around each frame.

To wrap up, let’s take a look at some screenshots of what has been accomplished over the last year! 4 legged animals:

Some of the bugs I had while creating these four legged creatures were hilarious.
Some of the bugs I had while creating these four legged creatures were hilarious.

A* Pathfinding:

A* Pathfinding

Shadows:

Shadows

Pixel perfect picking of world objects:

Pixel Perfect Picking

Terrain generation using Voronoi diagrams:

Voronoi Procedural Generation

Cuddly critters that roam around the world:

Raccoons and Cats!

Basic farming (picking up items, planting, growing day to day, throw on ground, store in shipping container/backpack, etc):

Bok Choy Gone To Seed

3D Sound using OpenAL:

The white lines are drawn to each sound source that is active.  Sound sources fade out and in depending on their location with respect to the character.
The white lines are drawn to each sound source that is active. Sound sources fade out and in depending on their location with respect to the character.

Atmospherics such as snow and rain (with corresponding changes to cloud cover):

It's Snowing!

Reflections:

Reflective Water

Light scattering:

Lots of Post Processing

Transparencies, ordering, lighting, etc:

Transparencies

There’s also a lot of work that went on behind the scenes that isn’t possible to show in a screenshot. The UI frame work has been drastically updated with new controls, performance improvements, etc. The meshing algorithms are much more efficient and robust, threading is better, and there are lots of gameplay items. For example, there is full support for inventories, crafting, recipes, NPC text, quests, achievements and so on.

The big step that needs to occur before alpha release is better world procedural generation, and the integration of all the different abilities of the engine. I’ll also need to work on generating some in-game content, such as different stages of plants, tools, and so on. I also need to begin work on my character animations, although I have some good tools for that!

Until next time,

Giawa

Journal Entry 59 – Better late than never?

Happy December everyone! I’ve had a pretty crazy week, which is why my blog postings haven’t been happening. I do have some cool progress to share. The first news (that you may have already read on Twiter) is that I have some help with creating the art assets in the game! Hopefully there will be some cool stuff to show off soon :D I have been back and forth with her quite a bit to fix issues with the client she is using. She is using a Mac, which meant that I had to go back and support the Mac operating system. Since I am using C# and OpenGL, it isn’t a far stretch to get Mac support via Mono. However, there are some issues to overcome.

The first issue is regarding shaders. Mac OS X only supports up to #version 120 of GLSL. Since all of my code is written with newer versions of GLSL, I need some way to fallback to the older version for the Mac. Well, it turns out I am only really using the newer versions of GLSL for speeding up certain things and making life easier (for example – Uniform blocks). I ended up writing a C# method that can convert pretty much 100% of my GLSL code into Mac OS X compliant code with a few simple string operations. I then need to do a bit more work on the Mac for setting my camera and other functions that take advantage of the newer version of GLSL.

One of the first scenes I got up and running on the Mac OS X using Mono+OpenGL.  All of the normal editor functionality is available to use on either Mac or PC (or Linux, too!).
One of the first scenes I got up and running on the Mac OS X using Mono+OpenGL. All of the normal editor functionality is available to use on either Mac or PC (or Linux, too!).
Here I've added full multiplayer support and server/client connectivity to the Mac client.  It didn't take too long to get up and running around with friends in-game.
Here I’ve added full multiplayer support and server/client connectivity to the Mac client. It didn’t take too long to get up and running around with friends in-game.

Next, I wanted to tackle a few visual aspects. It is tough to render dense forest, since I cannot draw easily across Voxel Chunk boundaries. You might already see from the first screenshot that this issue has been addressed with the latest builds. Here’s a crazy jungle!

Adjacent chunks get certain information about their neighbour, which allows trees and other objects to be seamlessly connected across chunk boundaries.
Adjacent chunks get certain information about their neighbour, which allows trees and other objects to be seamlessly connected across chunk boundaries.

It gets even crazier once you are under the tree canopy, with all of the shadows and dense trunks/leafs/clutter.

Things look pretty clear above the canopy, but below is a different story.  Real jungle biomes will need all sorts of cool and bushy ground clutter too.
Things look pretty clear above the canopy, but below is a different story. Real jungle biomes will need all sorts of cool and bushy ground clutter too.

Finally, I did a bit of work on the stability of the client, optimization and some lighting! I’m working on making sure that lit objects actually have some specularity. Here’s a preview of what’s in the works:

I've turned off the normal lighting (aside from basic ambient lighting) and instead am relying upon the 'gem' in this stone to cast light.  Specularity has been added to make the light sort of 'glow'.
I’ve turned off the normal lighting (aside from basic ambient lighting) and instead am relying upon the ‘gem’ in this stone to cast light. Specularity has been added to make the light sort of ‘glow’.

Anyways, that’s it for now! I just wanted to let everyone know I’m still working on stuff.

Giawa

PS: I got a new computer, which is why all of the new screenshots are Windows 8.1. Takes some getting used to, but I’m loving the speed of the processor.