Tech/LxEngine/Internal Documentation/Core Architecture

From Athile

Core Architecture

API Structure

As much data as possible is described by XML documents (which in turn can refer to data sources). Conversions and caching should happen transparently behind the scenes as much as possible. This design principle is based on the assumption of large amounts of available disk space (not true on mobile platforms and some gaming platforms) and willingness to pay a one-time cost in priming caches on an initial run (not always true).

The game logic should occur in script as much as possible. Only core data types and core algorithms should be coded in C++. Plug-ins can inject new methods into the script. There is not necessarly a 1:1 C++ to script API, as the C++ API is intened to be the core functionality, whereas the script API is intended to be the broad, 'usable' API. The objective is to promote rapid, easy change without recompilation to change gaming parameters.

The program does not have a entry-point in the traditional sense; instead, the DOM is used to register and associate scripts with particular events. This can be done via attributes on particular elements or via the DOM API directly.

Document Object Model (DOM)

The LxEngine DOM is intended to be as HTML-like to allow an easy, rapid transfer of skills as well as to leverage the design work that has gone into the (obviously) successful HTML format. That said, however, the focus should be on a HTML-like API that is far more lightweight and trim than HTML. A concise core needs to be developed, not a complete mirror of all HTML can do.

Note: at this time there is no notion of a Node - only an Element.

Element

• An Element is either in a Document or not in a Document
• If an Element is in a Document, then all its children are in the Document; and vice-versa
• An Element has a single value - not a set of Text Nodes
• An Element has either a value or children; having both is considered an error

Element values by default are parsed as lxvars. The exception is if the value is enclosed as a comment, in which case it becomes a string (encapsulated in an lxvar). The value of an element can be defined by a src attribute rather than an inline definition in the case of external files.

<Root>
    <Header>
      <Script language="javascript" src="media/scripts/base/ois_constants.js" />
      <Script language="javascript" src="data/sm_lx_cube_rain/lxquery.js" />
      <Script language="javascript">
      <!--
        window.onKeyDown = function(e)
        {
          alert(e);
        }
      -->
      </Script>
    </Header>
    <Library>
      <Mesh id="mesh_ship" src="media/models/ship_00.lxson" />
    </Library>
    <Scene>
      <Camera>
        {
          position : [ 9.0, 8.0, 10.0 ],
          lookAt   : [ 0.0, 0.0, 0.0 ],
          near     : 0.1,
          far      : 100,
        }
      </Camera>
      <Ref id="ship" ref="mesh_ship" max_extent="1.0" color="Red"/>
    </Scene>
</Root>

Attributes should be used for lighter weight, optional properties.

The value, by contrast is expected to change infrequently - or if it does change, there is a expectation that the operation might be slower.

Examples:

• A Mesh scale factor is a good attribute (easy default of 0,0,0, quick to change in the scene graph node)
• The Mesh vertex data should be part of the value (will require resending the mesh to HW, recomputing any cached bounds, etc.)

LxCSS

Attributes are specified on several levels:

• Pseudo-attributes - e.g. "wind: gale northwest" becomes "wind_direction: [-1 1 0]; wind_speed: 17;"
• User values - what the client document gave as a value; e.g. "color: Red"
• Normalized values - values converted to a more limited standard type rather than the flexibility of a user value; e.g. "color: [1, 0, 0]"
• Computed values - values computed as a direct value, rather than indirect; e.g. "color: [1, 0, 0]" rather than "color: inherit"

Example: "wind : gale northwest;" could be a LxCSS specified value. This is a pseudo-attribute with a pseudo-value. The "wind" pseudo-value gets translated to the real attributes "wind_velocity: 17.0" (gale = 17/ms) and "wind_direction : [ 0.707, 0.707, 0.0 ]" (a normalized vector in the +X/+Y direction).

Attribute Parsers

Views

Views are created on Documents. The Document is the implicit owner of a View. A View not associated with a Document has undefined behavior.

In Javascript, the views are accessible as:

• document.views["name"]
• document.views[0]
• window is implicitly document.view[0] (for HTML compatibility/similarity)

Components

A Component in LxEngine terminology is a concept for plugging in additional functionality to the Engine, Document, or Element. The DOM object will forward all major events to the respective Component. This pluggable, loose coupling allows keeps the code modular and maintainable; the primary motivation for the Component-centric design is code maintainability, not runtime efficiency or flexibility.

Another way of understanding Components: a particular subsystem, such as the Bullet Physics SDK or OGRE 3D Rendering Engine, is designed to be implemented as a set of Component classes. This way the core engine may have some abstract notion of 'physics' or 'rendering' so that it can forward relevant events to and between the Components, but the core engine itself knows nothing about Bullet or OGRE in particular.

That which contains a Component is known as a Host.

The pattern loosely looks like this:

• Bootstrap: at initialization, a subsystem creates a bootstrap ^g Engine::Component that, following the lazy-evaluation paradigm of LxEngine, listens for the first use of the subsystem or for idle cycles in which to begin its full initilization
• Engine Component: on the first use, the Bootstrap swaps in a full Engine Component implementation. This should represent all application-wide data needed for the subsystem. This Component will not be destructed until application shutdown.
• Document Component: on per-document. Should listen for Elements being added and attach the appropriate Element Component implementations to those specific Elements.
• Element Components:

Component Levels

1. Major subsystem
2. Scripting subsystem

Each Engine::Component is given a specific level. The subsystems with the lowest level are guaranteed to be initialized at the given level. The purpose of this is so that, for example, the physics subsystem can register its custom Element functions and properties before the scripting component exposes the functions and properties to the scripting environment.

Tending to think the virtual method mechanism is preferable. It gives the callback mechanism a logical home in the code, whereas with slots, the generality actually offers too much freedom and it is unclear where to register and de-register a callback mechanism.

Seems useful, but don't want to crowd the core API if this doesn't fit nicely.

lxvar

The lxvar is variant data type that can store simple and complex data types. It is designed for convenient and efficient marshaling of JSON and Javascript data to native C++ storage and processing.

Any implementation can be by manipulated via the lxvar interface can pose as an lxvar. Simply provide a wrapper and a new implementation of lxvalue. This allows the object to be passed around as an lxvar with less data translation and duplication.

LxSON

A superset of JSON notation, used for parsing lxvar variables. The grammar is the same as JSON with several additions:

• Named maps notation

If the pattern:

<alphanumeric> { }

is found, then data will be parsed as equivalent to a 2-element array of the form: [ <alphanumeric>, { } ]. This allows for easier object creation / specification where the alphanumeric represents the object type and the map holds the creation parameters.

• Long-string notation

If the pattern:

<<" ">> or <<' '>>

is found, then a multi-line string is read. Special rules apply to the whitespace in a long-string.

• The opening delimiter is expected to be followed immediately by a newline, or by a string of whitespace then a newline
• The closing delimiter is expected to be on its own line, preceded only by whitespace
• All intermediate lines are appended together with the newlines replaced with space characters
• The start column of the text chunk is indicated by the left-most intermediate line
• All whitespace prior to the start column is removed from every intermediate line.
• A double-newline translates to a literal newline which will be in the final string
• The above rule applies to the end of the string as well: to end the string with a newline, a double-newline must be used

    {
        mykey : <<"
                In this string, the opening two lines will not contain any newline
                characters.  The parser should replace the newlines with space characters.
 
                The double-newline in the string will be parsed as single newline in the
                final string.  Note: this literal does not end with a newline since the
                end of the string would require a double-newline.
                ">>,
        mykey2: <<"
                In this example, the value does end in the newline since there is a second 
                newline.
 
                ">>
    }

Currently, the long-string notation does not support this. Standard JSON notation with embedded newline characters would need to be used instead.

Currently, this is not supported in the long-string notation. Standard JSON notation should be used instead.

Graphics

Cameras

• A Document knows of zero or more Camera Elements in its Scene Element.
• A View knows of one implicit Camera that may or may not be linked to a Document Camera Element

Materials

The materials system does not yet exist, so this information is highly subject to change. It will be based on the early LxEngine prototype work seen here on the blog.

The basic idea is a JSON-like (actually lxson, a superset of JSON) material description. It is a procedural description composed of pluggable fragments. Each fragment has a set of named inputs and a single output (i.e. a shader graph).

Inputs:

• name
• type
• default value
• semantic name

† semantic name: if a parent fragment has set this a value to the semantic name or the object itself has set this field, then the value corresponding to that semantic name is used instead of the default value if an explicit value is not given.

Outputs:

• type

Fragement:

• name
• source for each supported shader language

The compiler looks at a material description...

• Determines which inputs are connected to other fragments
• Determines which inputs are unconnected or given explicit values

The runtime at activation of the material...

• Activates the shader based on a cached hash of the shader graph
• Sets all unconnected inputs to default or semantic values
• Sets all explicitly set inputs to their values (as stored in the cached rep of the shader)

The graph of connected fragments defines a unique shader. The unconnected inputs become the shader uniform variables. A

Psuedo-code

checker 
{
    output      : "vec3",
    input       : 
    {
        uv          : [ "vec2", [0,0]       "uvMapper"          ],
        primary     : [ "vec3", [1,1,1],    "primaryColor"      ],
        secondary   : [ "vec3", [0,0,0],    "secondaryColor"    ],
    },
    source : <<"
        vec2 t = abs( fract(uv) );
        ivec2 s = ivec2(trunc(2 * t));
 
        if ((s.x + s.y) % 2 == 0)
            return primary;
        else
            return secondary;
    ">>,
}

Physics

LxEngine is based entirely on the metric system.

• http://www.microbiologybytes.com/maths/1010-7.html

density (kg/m³), surface area (m²)

Time

All physics take place in application time, not user time. The user can pause, accelerate, or decelerate the pace of application time.

Wind

How wind is applied in LxEngine...

Wind is specified via a velocity vector (m/s) at any given point in space at any given point in time. A useful mapping of descriptive speeds to metric speeds is given by the Beaufort scale.

To think about the effect of wind on an object, it is helpful to ask two separate questions:

• How much air mass is hitting the object over a given time slice?
• What is the impulse (or momentum) of that air hitting the object?

Solving this:

• V = air velocity (m/s)
• D = air density (kg/m^3)
• A = surface area perpendicular to the air velocity (m^2)

Mass passing through the surface for a time dt:

• M = D * V * A * dt

Therefore, the impulse is the mass times the velocity of that mass:

• I = M * V

Or written out in full:

• I = (D * V * A * t) * V

Notes

• D = the average density of surface air is 1.29 kilograms per cubic meter (source)
• A = surface area of object; requires an approximation of the surface area perpendicular to the wind direction
• This does not account for air resistance: the air mass simply disappears after contact

CSS attributes for wind scale should allow the Beaufort scale descriptive values.

• http://www.physicalgeography.net/fundamentals/7n.html

i.e. a function that takes a point in space and returns a velocity for that point in space. The returned value could be used for as constant across the surface of any single object - given the likely rate of change of the velocity field comparative to the size of an object. However, this still might allow objects higher off the ground to be affected more noticeable, which might give a desirable effect.

Since velocity is a vector, this could be used to create cyclones that vary in speed and strength based on vertical and horizontal distance from the epicenter.

Example Code

    void
    PhysicsDoc::_applyWind (const float timeStep)
    {
        const btVector3 airVelocity (1.4, 0, 0);      // 1.4 m/s ~= 5 km / hr
        const btScalar  airDensity = 1.29;            // kg/ m^3
 
        btCollisionObjectArray objects = mspDynamicsWorld->getCollisionObjectArray();
        mspDynamicsWorld->clearForces();
        for (int i = 0; i < objects.size(); i++) 
        {
            btRigidBody* pRigidBody = btRigidBody::upcast(objects[i]);
            if (pRigidBody) 
            {
                // Compute an approximate surface area in each direct by simply taking some
                // percentage of the bounding radius.
                //
                btScalar radius;
                btCollisionShape* pShape = pRigidBody->getCollisionShape();
                pShape->getBoundingSphere(btVector3(0,0,0), radius);
                btScalar approxArea = radius * radius * .66f;
                btVector3 surfaceArea (approxArea, approxArea, approxArea);
 
                // Velocity * density * surface area = amount of mass per second hitting the area
                // " * time * velocity = momentum of that mass over that period of time
                //
                btVector3 impulse = (airVelocity * airDensity * surfaceArea * timeStep) * airVelocity;
                pRigidBody->applyCentralImpulse(impulse);
            }
        }
    }

Input Devices

As much as possible the system should be based upon "actions" rather than input events. There should be scriptable translation layer that abstracts the actual device events into application actions. I.e. the application should, in most cases, never know whether the W key was pressed, if the mouse was moved forward, or if a script created an simulated device event: the application should only know it received a "move forward" event.

The architecture does not currently do this.

Input should be chained into several layers:

• Platform abstraction: map platform APIs -> generic interface for devices
• Device abstraction: map device input -> action table -> application actions

The action table is a configuration mapping that takes a generic device event (key down, mouse up, etc.) and maps it to a application action. The application itself never should know what device generated the input. For example, if a script is simulating device input via directly actions, this should be transparent to the app.