Links Worth Reading

• Faster Photorealism in Wonderland: Physically based shading and lighting at Sony Pictures Imagework – thanks Ray Tracey
• Git Submodules: Adding, Using, Removing, Updating – thanks Chris Jean

Update: Notes on git submodule updates

Chris Jean’s article linked to above is great (thank you, Chris!), but the “Updating Submodules” section is not really aimed at how to update the  top-level repository to pick up changes made and committed locally to that submodule.  Step #1 of his article will run “git submodule init” and “git submodule update”, which would throw away any local changes.  Chris’ steps still work, but the steps basically discard the locally made changes, re-pull them, and then commit them. Let’s just commit them directly:

• After committing the submodule changes:
• Change the directory to the top-level repository root
• “git add” the submodule directory
• Commit the change

Spelling it out via a more specific example, here’s how to directly incorporate locally made changes to the submodule into the top-level repository – using a change to the GLGeom lib within LxEngine as an example.  Steps 7 & 8 are the key.  It’s quite simple once you know what you need to do:

1. Assume there’s with a working, buildable LxEngine git clone that you’re working in…
2. Make the desired code changes to glgeom with lxengine/lx0/dev/libs/glgeom
3. cd lx0/dev/libs/glgeom
4. git add .
5. git commit -m ‘text description of the changes’
6. git push origin master
7. cd lxengine
8. git add lx0/dev/libs/glgeom
9. git commit -m ‘updating lxengine’s version of glgeom’
10. git push origin master

Walking through the steps: (1) start with the assumption a buildable local repository of LxEngine is available – i.e. git submodule init and git submodule update have been invoked, everything builds, everything is initially setup correctly before any changes are made, etc., (2) make the code changes to the glgeom submodule that are desired, (3) change to the glgeom submodule directory – this more or less acts as if git is working within glgeom and now no longer really “knows” about the outer LxEngine repository, (4, 5, & 6) standard steps for adding and committing a change and then pushing it to github – nothing special here, (7) to update the submodule the directory needs to be changed back to the root of the top-level repository, (8) “git add” the submodule directory to convey the update to top-level project, and (9 & 10) standard steps to commit a change and push to github.

Done.

A prototype quality BlendReader class has been integrated into LxEngine.   The interface is small and simple. Part of the core vision for LxEngine is excellent usability for the development team during development. This means removing steps from the development process that technically can be automated.  Direct support for the Blender file format is an exciting addition in this regard.  No more export step: simply save the updated file and run the application.

I call the code ‘prototype quality’ at this stage because I know it won’t work correctly on .blend files from 32-bit systems or from big endian systems.  Both these issues will be trivial to solve, but have not yet been addressed.   There’s also likely a bit of room for optimization, but for the most part, I doubt that matters for anything but massive scenes (in which case, it’ll likely be better not to be loading a .blend file directly, but rather some leaner format).

If nothing else, the .blend file format is fairly interesting in itself.

Usage

The BlendReader interface is trivial to use:

1. Create the BlendReader object
2. Call reader.open(std::string filename)
   1. This call opens the file, reads the blend file’s “DNA” structure and build indices so objects can be read out of the file easily
3. Call reader.getBlocksByType(std::string type)
   1. This returns a list of the info about the blocks in the file of the given type.  For example, “Mesh” or “Scene”.
4. Call reader.readObject(address)
   1. This takes the address of a block and reads it in as a typed object.  It’s not a C++ type, but rather a wrapper on the block that lets the user grab the fields by name without any error-prone pointer arithmetic.
5. Call obj.field<type>(name, index)
   1. Reads a particular named field out of the object and casts it to the given type.  Of course the caller needs to get the type right – but this is inevitable as at some point the opaque chunk of binary data needs to be cast into native C++ types

Example Code

Here’s the chunk of prototype code that loads the blend files into an LxEngine document (using the LxEngine Mesh structure):

    float normalizeShort (short s)
    {
        return float (s) / float(std::numeric_limits<short>::max());
    }
 
 
    Mesh*
    load_blend (std::string filename)
    {
        BlendReader reader;
        if ( reader.open(filename) )
        {
            Mesh* pMesh = new Mesh;
 
            auto meshBlocks = reader.getBlocksByType("Mesh");
 
            if (meshBlocks.size() != 1)
            {
                lx_warn("More than one mesh found in .blend file.  Processing only the "
                        "first one that is found.");
            }
 
            auto spObj = reader.readObject(meshBlocks[0]->address);
            auto numVerts = spObj->field<int>("totvert");
            auto numFaces = spObj->field<int>("totface");
 
            pMesh->mVertices.reserve(numVerts);
            pMesh->mFaces.reserve(numFaces);
 
            pMesh->mFlags.mVertexNormals = true;
 
            auto spVerts = reader.readObject(spObj->field<unsigned __int64>("mvert"));
            for (int i = 0; i < numVerts; ++i)
            {
                Mesh::Vertex v;
                v.position = spVerts->field<point3>("co", 0);
 
                // Normals are encoded as shorts
                v.normal.x = normalizeShort( spVerts->field<short>("no", 0) );
                v.normal.y = normalizeShort( spVerts->field<short>("no", 1) );
                v.normal.z = normalizeShort( spVerts->field<short>("no", 2) );
 
                pMesh->mVertices.push_back(v);
                spVerts->next();
            }
            spVerts.reset();
 
            auto spFaces = reader.readObject(spObj->field<unsigned __int64>("mface"));
            for (int i = 0; i < numFaces; ++i)
            {
                Mesh::Quad q;
                q.index[0] = spFaces->field<int>("v1");
                q.index[1] = spFaces->field<int>("v2");
                q.index[2] = spFaces->field<int>("v3");
                q.index[3] = spFaces->field<int>("v4");
 
                pMesh->mFaces.push_back(q);
                spFaces->next();
            }
            spFaces.reset();
 
            return pMesh;
        }
        else
        {
            lx_error("Could not open file '%s'", filename.c_str());
            return nullptr;
        }
    }

Full Source Code

The latest code (assuming future submissions haven’t moved it) is available here on github.

And some related links on the .blend file format:

• http://www.atmind.nl/blender/blender-sdna.html – table on SDNA fields
• http://www.blender.org/development/architecture/blender-file-format/