For various reasons, one of them being to test LxEngine with “real” data, I’ve been experimenting with loading and displaying the game data from Bethesda Softwork’s 2002 game, The Elder Scrolls III: Morrowind (buy it here on Steam). There’s a fair amount of information out there about the Morrowind file formats – as it is a highly moddable game.  I’ve been using NifTools to parse the actual models and been using custom code for the ESM/BSA parsing (neither are very complicated formats).

The primary purpose of the project has been to test out LxEngine with dated, but production-quality data and data formats.  The experiment thus far has been serving it’s purpose.  It has raised questions like, “Hey, what should the engine do when the current cell has 17 lights and the current shader only supports 8 at a time?”  The LxEngine project has hardly been lacking in TODOs, but in any case, this is helping identify the necessities versus the niceties.

A secondary purpose of the project is to learn a bit more about how Morrowind works, so that potentially as a side-effect of working on my own goals produce some useful contribution to the OpenMW project.  (This project certainly isn’t meant to compete with OpenMW – the goals here are to demo some basic rendering, physics, sound, etc. from Morrowind to test out LxEngine’s capabilities.  The goal of the OpenMW project is to produce a fully playable game with full fidelity to the original.)

As for the current progress, here are some screenshots:

Morrowind data game rendered (with obvious limitations!) with LxEngine

An actual, in-game screenshot from Bethesda Softworks' Morrowind of that same scene

Update: Texture Mapping

Adding texture mapping involved a couple core changes:

• Adding UVs and texture samplers to the LxEngine GLSL shader builder. This is less complex than some of the existing features of the shader builder, but hadn’t yet been added.  The support is somewhat minimal and will require revisiting for multi-texturing.
• Adding DDS texture format support to the GL Rasterizer.  DDS stands for DirectDraw Surface, i.e. a Microsoft DirectX format, that furthermore has some strange patent issues, which seems to bode poorly but video cards usually handle this format natively. There’s a EXT_texture_compression_s3tc OpenGL extension that allows DDS format data to be passed more or less directly to the card.  There’s a simple nVidia tutorial showing how to do this.
• Passing DDS streams from within a BSA understood by the TES3 loader to the LxEngine Rasterizer which knows nothing about Morrowind format data.  This was the fun one – which actually still requires a bit of work – abstracting the LxEngine rasterizer from the texture data source in a flexible, general way that both (a) allows the Rasterizer to know nothing about BSA files while the BSA loader knows nothing about OpenGL and (b) still streams the data directly from a disk-based std::istream to OpenGL without any superfluous copies.    The Rasterizer now allows textures to be created with a “type” and “acquire callback”.  In this case, the type is a stream and the callback is over in the TES3 loader: the only shared concept necessary is the std::istream.

Ever wonder what the Morrowind UV mapping looked like?

And after a couple bug fixes (like, ehem, remembering to open the binary DDS stream in std::ios::binary mode)…

Morrowind cell rendered with LxEngine

Next, I need to add multisampling support to the renderer: these screenshots would look so much better with it enabled!

Update: Multisampling

Multisampling…

16x Multisampling

The image below is not in fact of a checker procedural material, but rather is a first image from some work towards adding voxel rendering to LxEngine.  As version 0, it’s more or less simply rendering an array of cubes.

A single cell of 16x16x4 voxels

Here’s a rough the change list:

• Added support for solid colored materials (i.e. no lighting, just color)
• Added support for specifying the light set and camera to use a particular pass (previously this had to be specified per item being rendered)
• Improved the OpenGL error checking in the rasterizer a bit

Update: 2011.06.25

Procedurally generated voxel "world"

The sample has been updated to support multiple voxel cells with a procedurally generated (noise-based) height function.  The rendering has been rewritten to create a single mesh for each cell which – as anticipated – is orders of magnitude faster than the original naive voxel-by-voxel rendering algorithm.

Building the mesh…

The algorithm for building the mesh is quite simple:

• For each voxel in the cell (i.e. each of the 16x16x4 blocks)…
• Check if the block above is empty or there is no block (i.e. the block is on the top boundary); if so add a quad for the top of the block to the vertex buffer
• Repeat the prior step for the other five sides of the block

This has two major advantages:

The entire cell is treated as a single vertex buffer.  This means one draw call for all 16x16x4 blocks, rather than 1024 draw calls.  Graphics hardware operates far more efficiently one large batches; thus, the single draw call for the whole cell takes roughly as much time as each of the 1024 calls took when drawing each block individually.  (Combining the buffer, of course, means the same shader and parameters need to be used for the entire buffer.  The solution to handle textured blocks – not yet implemented – will be to use a texture atlas containing all the textures so that texture parameters do not have to be reset.)

The second advantage is all the interior blocks of a cell get discard implicitly.  In the naive algorithm, every block has drawn, regardless of whether it was completely obscured or not.  Due to the grid nature of the cell, checking neighbors is sufficient to see if a particular face is never visible.  Since “most” blocks in the anticipated data sets are fully interior, this potentially trims out a sizeable chunk of obscured vertices and faces.  In a “solid” cell of 16x16x4 block, there are potentially 24,576 vertices and 6144 faces (assuming each face requires a unique set of vertex parameters for each of its vertices); with the interior cells stripped out, this becomes 768 faces with 3,072 vertices.  That’s 12.5% the number of vertices and faces.  If vertices can be shared between faces, the number of vertices drops to 768 as well, or 3.1% of the original count.