As an academic exercise, I put together some code to create a “interleaved threads”: i.e. create two threads each with a manual yield() function that “swaps” the current thread to the other thread.

The trivial example I used was to print all the numbers from 0 to 99 with each thread counting one digit then transferring to the other thread: i.e. one thread prints all the evens and the other prints all the odds. The threading code then synchronizes things such that the numbers come out in order. It uses a common “yield” pattern to notify when the active thread should be swapped.

Before we get any further, two major caveats:

• I’m not sure what a good real world application for this would be. This completely defeats the purpose of using threads to run things in parallel, though it might be useful for the code architecture since each thread still has it’s own stack and instruction pointer and might be used for something akin to what corountines are sometimes used for?
• I’m pretty sure the implementation wastes a bit of time in the scheduler since it doesn’t really “swap” the current thread, but rather thread A simply refuses to run until the scheduler decides to let thread B run

As I said, an academic exercise. However, I thought it still might be worth sharing. I’d be very interested in hearing feedback if someone is aware of a case where this stratagem might be useful in practice.

Counting Evens and Odds

Here’s what the code looks like to produce an output of:

0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99,

#include <boost/thread.hpp>
#include <boost/thread/mutex.hpp>
 
void count_odds (YieldFunc yield)
{
    for (int i = 1; i < 100; i += 2)
    {
        std::cout << i << ", ";
        std::cout.flush();
        yield();
    }
}
 
void count_evens (YieldFunc yield)
{
    for (int i = 0; i < 100; i += 2)
    {
        std::cout << i << ", ";
        std::cout.flush();
        yield();
    }
}
 
int 
main (int argc, char** argv)
{
    InterleavedThreads(count_evens, count_odds);
    return 0;
}

I’m not sure how generally useful it is, but it is surprisingly clean, right? Not a lot of unreadable threading code in there.

(This might actually be far more efficient if fibers where used rather than threads since that would allow an explicit “swap” from one execution context to the other. However, I know very little about fibers other than vague reports that they are easy to misuse. Here’s a link that suggests fibers may not be the best idea.)

InterleavedThreads

If there’s any magic to the above, it’s all wrapped up in the InterleavedThreads implementation. Without much commentary, here it is:

typedef std::function<void()>  YieldFunc;
 
struct InterleavedThreads
{
    InterleavedThreads (std::function<void(YieldFunc)> f0, std::function<void(YieldFunc)> f1)
    {
        volatile int start0 = 0;
        volatile int start1 = 0;
        boost::mutex    active;
        boost::mutex    waiting;
 
        auto yield = [&]() 
        {
            active.unlock();
            waiting.lock();
            active.lock();
            waiting.unlock();
        };
 
        boost::thread_group group;
        group.create_thread( [&]() {
 
            active.lock();
            start0 = 1;
            while (!start1) {}
 
            f0(yield);
 
            active.unlock();
        });
        group.create_thread(  [&]() {
 
            while (!start0) {}
 
            waiting.lock();
            start1 = 1;
 
            active.lock();
            waiting.unlock();
 
            f1(yield);
 
            active.unlock();
        });
        group.join_all();
    }
};

There’s a tiny, little busy wait to ensure thread 0 starts before thread 1 (that’s part of the implicit contract I had in mind when throwing this example together). Feel free to substitute a “real” threading primitive in there to accomplish the same if you are allergic to busy waits.

Then the basic mechanism is accomplished via handshaking of an “active” mutex (held by the thread that’s actively running) and a “waiting” mutex (held by the thread waiting for to be active). Using the initial busy wait we ensure an initial state where thread 0 has the “active” mutex while thread 1 has the “waiting” mutex. The exchange of the ownership of the two mutexes is handled by the yield lambda. We two mutexes to accomplish the handshake, otherwise active.unlock() followed directly by active.lock() would likely mean the active thread would reacquire the mutex immediately without the waiting thread having a chance to acquire it.

Conclusion

I’m not sure there’s a conclusion. I’d love to hear some feedback if you find this at all interesting or can imagine a potential use for it.

Another item from the Great Bucket of Random Things:

The below Javascript will add a simple macro to TiddlyWiki to that substitutes <<define yourwordhere>> for a link to google searching for that term. Put the phrase in quotes if you want to include spaces (e.g. <<define “your phrase here”>>). Custom links like this are trivial, but can be a very useful shorthand.

Just search for “// Macro definitions” in the TiddlyWiki HTML to find where to inline your macro.

// ---------------------------------------------------------------------------------
// Macro definitions
// ---------------------------------------------------------------------------------
 
config.macros.define = {};
config.macros.define.handler = function(place,macroName,params)
{
    var link = createTiddlyElement(place, "a",null,null,params[0]);
    link.setAttribute("href", "http://www.google.com/search?q=define+" + params[0]);
    link.setAttribute("target", "_blank")
}

Note that TiddlyWiki is one of the six things on the Internet not using JQuery, so you do have to use the HTML DOM API directly to set up your inserted element.

From the Great Bucket of Random Things, here’s a script for generating a random numeric filename for each JPG in a folder using DOS Batch scripting:

Note: The below does not work properly for filenames with spaces, sorry. I’m sure it wouldn’t be too hard to fix it, but as this script has already served its purpose for me, alas, it will remain as it is…

@echo off
setlocal
 
for /f %%i in ('dir /b *.jpg') do call :resize %%i
 
:resize
:resize_1
    if "%1"=="" goto :EOF
    set filename=%random%.jpg
    if EXIST %filename% goto resize_1 
    rename %1 %filename%
    echo Renamed '%1' to '%filename%'
goto :EOF

I posted this on the wiki as well since I need to write something using a DOS batch with the exact same frequency with which I completely forget how to do so.

Update (2011.05.25): Native CMake support

In a frustrating case of less-than-intuitive usability, hard to locate documentation, and/or user error, the original post below on using Python to modify the Visual Studio solution to organize projects in folders is not necessary: CMake 2.8.3 does natively support solution folders.

The syntax is simple.  Here’s what I appended to the outer-most CMakeLists.txt in the Lx0 project to organize the sub-projects into folders (note – I have not tested this on any compiler other than VS2010 Professional):

#
# Organize projects into folders
#
SET_PROPERTY(GLOBAL PROPERTY USE_FOLDERS ON)

SET_PROPERTY(TARGET lxengine                PROPERTY FOLDER "Libs/LxEngine")
SET_PROPERTY(TARGET glgeom_benchmark        PROPERTY FOLDER "Libs/GLGeom")
SET_PROPERTY(TARGET glgeom_unittest         PROPERTY FOLDER "Libs/GLGeom")

SET_PROPERTY(TARGET sm_lx_cube_rain         PROPERTY FOLDER "Samples/LxEngine")
SET_PROPERTY(TARGET sm_lx_cube_asteriods    PROPERTY FOLDER "Samples/LxEngine")
SET_PROPERTY(TARGET sm_lxcanvas             PROPERTY FOLDER "Samples/LxEngine")
SET_PROPERTY(TARGET sm_terrain              PROPERTY FOLDER "Samples/LxEngine")
SET_PROPERTY(TARGET sm_raytracer            PROPERTY FOLDER "Samples/LxEngine")
SET_PROPERTY(TARGET sm_lxcraft              PROPERTY FOLDER "Samples/LxEngine")
SET_PROPERTY(TARGET sm_rasterizer           PROPERTY FOLDER "Samples/LxEngine")
SET_PROPERTY(TARGET sm_ogre_minimal         PROPERTY FOLDER "Samples/ThirdPartyLibs")
SET_PROPERTY(TARGET sm_v8_basic             PROPERTY FOLDER "Samples/ThirdPartyLibs")
SET_PROPERTY(TARGET cpp_smartptr            PROPERTY FOLDER "Samples/Cpp")

SET_PROPERTY(TARGET bm_lxvar                PROPERTY FOLDER "Benchmarks")
SET_PROPERTY(TARGET blendload               PROPERTY FOLDER "Sandbox")
SET_PROPERTY(TARGET elm_reference           PROPERTY FOLDER "Sandbox")
SET_PROPERTY(TARGET elm_function            PROPERTY FOLDER "Sandbox")
SET_PROPERTY(TARGET sandbox_shadergraph     PROPERTY FOLDER "Sandbox")
SET_PROPERTY(TARGET sb_fixedpoint           PROPERTY FOLDER "Sandbox")
SET_PROPERTY(TARGET ut_jsonparser           PROPERTY FOLDER "UnitTest")
SET_PROPERTY(TARGET ut_lx_vector            PROPERTY FOLDER "UnitTest")

Three parts worth noting:

1. SET_PROPERTY(GLOBAL PROPERTY USE_FOLDERS ON) – for some reason you need to turn this feature on before you can use it (shouldn’t simply not using it be equivalent to it being off??)
2. TARGET target name – as you might guess, after the name after the TARGET keyword should be the name of your executable or library as you define it elsewhere in the CMake project setup
3. PROPERTY FOLDER “Folder/SubFolder” – use a forward slash to denote sub-folders

I’m hesitant to complain about tools I’ve done nothing to contribute to, but it is frustrating that CMake is an incredibly useful and powerful tool but is lacking in the usability department (e.g. unnecessarily unique syntax – why is this a TARGET PROPERTY and not a PROJECT_FOLDER() macro?, hard to track down / poorly organized documentation – I found this via a Google search that pointed me to a diff in the CMake git server such that I realized my earlier efforts were not also calling SET_PROPERTY(GLOBAL PROPERTY USE_FOLDERS ON)).

CMake already supported exactly what I needed but it took non-trivial effort to just track down that it did!

Original Post (2010.12.16)

CMake supports the source_group() command for grouping files within a project. I like this feature as it maps well to how I organize projects conceptually.  CMake, however, appears to lack a means to create solution folders, i.e. folders for actually categorizing the projects themselves into a hierarchy.   With LxEngine, which has a lot of sub-projects – this would be a quite helpful feature for use with Visual Studio.  In other words, what is really desired (by me) is a project_group() CMake function.

In an effort to learn a tiny bit more Python – and to get the behavior I wanted (via a little less work than properly hacking the CMake sources to add project_group() functionality; though others have expressed interest and even possibly worked on such a feature, but as far as I can tell, it’s not yet a feature as of CMake 2.8.3), I put together a version 0.0 script for post-processing the Visual Studio 2010 solution file and inject the folder nesting that I want.  The result ends up with a project structure looking like this:

Solution Folders after post-processing

The script, as I mentioned, is definitely a version 0.0:

• It’s not yet reusable
• It’s hard-coded to the LxEngine project layout (but that’s easy to change)
• It’s hard-coded to the Visual Studio 2010 file format
• It’s low-quality Python code (apologies, I’m new to the language and just wanted to “get it to work”)
• It’s a post-process rather than something CMake could invoke directly

However, it does work correctly – so if anyone wants to hack together their own temporary workaround to get solution folders/ project_group() functionality from CMake, feel free to use this code as a template.

The source is located here on github.

Update:

Despite the closed bug report on vtk.org, CMake 2.8.3 for windows does not appear to have a project_group() or project_folder() command.  Nor does setting a manual property via ‘set_property(TARGET mproject PROPERTY FOLDER “MyFolder”)’ seem to have any effect.  This is true even for the nightly build 2.8.3.20101215-g4bf09.  I am assuming from the comments in the bug that the changes must only exist in a branch and not the main line?

A simple render from experimenting with multi-texturing in Blender 2.5:

Blender 2.5 Quick Terrain Experiment

The above was created by roughly following the tutorial posted on the Blender 3D Noob to Pro wikibook.

Testing out heightmaps in Blender

I experimented a bit with heightmaps in Blender 2.5 as well – there isn’t native support for image-based heightmaps, but it’s fairly easy to quick preview one: (1) create a dense 3d grid, (2) add a material/texture with the heightmap, (3) set the heightmap “RGB to Intensity” flag, turn off the diffuse color influence, and enable the Displace influence.

C++ is still predominantly the best, or at least most used, language for performance or graphics intensive applications aimed at the consumer. I consider myself a C++ programmer and very much admire the practical and flexible language design. Yet, still I am beginning to wonder if the software development community could hugely benefit from a new primary language.   But if I think highly of the language, why do I suggest a new one might be necessary?

It’s the tool chain and the process more than the language itself.

–

First of all, no matter how good a language is, it’s useless if the tools are not available to use that language to accomplish your goals. (I think this statement can safely be made without explicit justification.)

Now consider if your goal is easy, rapid, globally collaborative development.  Are the best tools really there to suit this problem?  Yes, there are tools, but do they fit that goal as well as they could?

What if your goal is to eliminate as much build engineering overhead as possible before software engineering on a particular problem can begin?

Old isn’t necessarily bad, but the current tool chain for C++ development effectively is very similar to the days of early Unix development.  That’s fine, but does that tool chain really fit the ideal process of global development community working to continually contribute to open source software development?  Consider how those tools would do under a serious usability study of global collaborative development.

–

To save typing, I’ll assume the reader is at least a bit familar with web development (e.g. Javascript) and C++ development. Rather than write out a formal argument for my point, I’ll simply say this: consider how much overhead it takes a novice to throw together a JQuery-based webpage versus the overhead in building a large open source project.  Or, as another example, head to the discussion forums on any major third-party C++ SDK and count the percentage of posts about missing headers, linker errors, dependencies, library versions, or other configuration questions that have absolutely nothing to do with the purpose of the SDK itself – and then compare that to a Javascript-based library.  I know this is not  a fair comparison, but bear with me for a moment.

Now, just for fun, imagine that building that large open source project were as easy as viewing a webpage.  I’m not talking about development on that large project, let’s simply focus on the process of building the project.   Seriously consider it.   Overlook the size / initial download problem for a moment and just compared the steps involved in those two tasks.   A webpage usually “just works” if you have a modern browser and the web page was written by a decent developer.  It’s a zero step process: merely providing the name of the webpage is viewing it.  On the other hand, a large open source project build – well, the steps to do that could be almost anything…it very rarely “just works” and instead often learning new skills with every project.

Maybe I’m wrong about it, but imagine the influx of casual contributions to open source projects if large application builds were effectively zero-overhead to get up and run from source…but I’m getting ahead of myself.   For now, focus simply on the notion of building (rather than developing) a large open source project as being a zero-step process.

-

The next question: is there any technical reason why it couldn’t just work in the majority of cases (or at least to the same percentage as a well developed web site works)?

I don’t think there is.

-

I’m not going to pretend that there haven’t been lots of attempts to solve the broadly defined “write once, run anywhere” problem.  Java, of course.  .NET jumps to mind as a bit closer to “build anywhere” notion discussed here given it’s multi-language support.  I think .NET is conceptually fairly right on, but in practice it hasn’t happened.   Full virtual machines and runtime environments aside, tools like CMake exist to alleviate the build problem – but how effective are they?  CMake itself is yet another “overhead” item that needs to be learned which likely has nothing to do with the actual problem the developer is trying to solve.  It may be better, but it’s still another build engineering hurdle requiring knowledge from the user.

Or to invert issue completely, consider how HTML5 and technologies like WebGL are in a way approaching this fundamental “building C++ applications is hard” problem: they aim to make it easier to bring application graphics development to the simplicity of browser-based technology development rather than bring the simplicity of browser-based technology development to application graphics development. (I do realize the last statement has quite a few embedded assumptions, but I generally think it’s a true statement – or at least true enough to convey the crux of the point.)  Better yet, isn’t the development of Chrome OS implying the same general trend of pushing traditional low-level development closer to the convenience of browser-based apps rather than vice-versa?

Ok, so one more way of looking at this…

How difficult would it be to allow browsers to support compiled languages?  Technically, not very.   Chrome already compiles Javascript into native machine code – and caches the Javascript files associated with pages.   What technical difference is there really if that source file is C++?  Yes, C++ pointers, etc. would throw some wrenches into security and verifiability but otherwise, it’s just a different compiler component implementation.   Why not extend that embed the make system into ‘browser’?   And why not embed the actual source control system implementations (at least the ‘get’ aspect) into the browser?  The user would end up with a ‘browser’ that effectively views project files, can pull the relevant source to local caches, compile it, and run the application.   There’s still the first-run (i.e. priming the cache) problem – but that’s solvable too (how about pre-compiled caches of popular configurations could be hosted on the application’s website?).  Why not hide the entire build tool chain in an application ‘browser’ of sorts?

Java essentially attempted (and still attempts) to solve much of this.  Java certainly didn’t eliminate the problem, but that fact does not undo the theory behind the idea.   As noted previously, in some manners, Chrome OS effectively is hiding the application “build” in the browser.   But what about approaching the problem by making the existing build processes more like a browser rather than making more apps work from a browser?

-

Ok, so what of this problem?  Why is this more than just a rant that build engineering in C++ should be easier and just work like a browser?

I suspect that a very real source of the problem is that many experienced developers dismiss the “tool chain” issue simply because they are already so used to it. It’s not a lack of technical knowledge out there.  It’s a lack of motivation.

Sure – learning CMake may teach you about multi-platform programming and all other sorts of useful information – but to a novice programmer who wants to experiment with 3D graphics, does it really make sense that he needs at least some degree of expertise in build tools first?  No, it doesn’t.  Software should be about tackling the problem you are interested in primarily, and then secondarily be about understanding the periphery of the problem so you can improve your initial solution (i.e. in this case, learning more about multi-platform programming via CMake after the novice programmer has his experiment working on his own machine).

Encapsulating the compiler tool-chain in a browser doesn’t eliminate or ‘solve’ build engineering forever; instead, it would need to evolve by a standard like HTML.   What would be needed is a standard for project builds to deliver content flexibly, but reliably.   The issue isn’t so much with C++; it’s with the build tool-chain that’s become associated with the delivery of C++ content.

-

I generally try to avoid complaining about a problem without proposing a solution: the first step toward the solution might be for serious C++ developers to no longer be content with build engineering skills being a prerequisite to software engineering.

My gut feeling remains stuck on what seems obvious: on modern, powerful workstations, there’s no technical reason that the build process for large applications needs to be so complex.  And if there’s no technical reason, what is really preventing the solution?

I attempted to start XAMPP, which includes Apache, and it failed.  It used to work correctly.  This was bad.

Sharing the solution might be useful to others, so here it is:

First, this large post “Apache Won’t Start on XAMPP” over on www.netshinesoftware.com has a lot of information about how to fix the problem.  There are a lot of comments that provide useful addition information; in fact, too many comments as it would take too long to read them all just to find your particular potential issue.  So, here’s what a summary of what I found…

Problem: a common cause of Apache failing to start is a port conflict with another running program.
Solution: track down which program is conflicting with Apache and get it to stop using that port*.

(*Or, of course, you could change the port Apache is using.  However, the assumption being made here is that some other program has started using one of the Apache ports and “shouldn’t” be.)

Steps to fix the issue:

(1)  Track down the port in conflict that Apache needs.

Of course, a port conflict might not be your issue: but it’s common, so that’s what this post will describe how to fix.

XAMPP unfortunately fails silently when Apache fails to start.  However, Apache keeps an error log at c:/xampp/apache/logs/error.log.  If it’s a port conflict, check the last line and likely you’ll see something like the following:

(OS 10048)Only one usage of each socket address (protocol/network address/port) is normally permitted.  : make_sock: could not bind to address 0.0.0.0:443

no listening sockets available, shutting down

Note: I’ve added emphasis to the port number in conflict.  In my case, the port was 443.

(2) Track down the program or service using that port.

From the comments on the Netshine blog post, there’s a link to CurrPorts, a standalone freeware tool that lists was programs and services are using which ports.  Sure, you can use netstat, the Windows Administrative Tools, and  Task Manager to figure this out yourself, but CurrPorts does that all for you: why make it harder for yourself?  CurrPorts is literally a standalone EXE – it doesn’t have an installer, won’t throw trash in the Windows registry, etc. so there’s no real reason not to simply use it – and delete it when you’re done if you like.

(3) Disable the program or service using that port

For many people, based on the original post, Skype was the offending application.   See the original post for details on how to fix Skype’s port usage, if that’s your problem.

In my case, it was Pando Media Booster.  I will neglect to elaborate on the idea of how the words “Media” and “Booster” in the same product title make me highly distrustful of that product, but rather simple confine my frustration to the fact that something installed it on my system without notifying me (or at least without notifying me clearly enough at it was installing something that takes exclusive control of well-defined ports whenever my computer is running).

Since I don’t install much, it was faily easy to track down that Lord of the Rings Online (which had a free trail period a while back) was the offending application that installed Pando Media Booster.   Since I played the game about twice and never again, the  fix was easy: uninstall Lord of the Rings Online.   Oh wait,…

…the Lord of the Rings Online installer doesn’t actually uninstall all the software that installed on my system.  Step 2: go to Uninstall Programs and also uninstall Pando Media Booster.

(4) Solved: Apache starts correctly

The main takeaway from the experience?

Don’t install software on your main system until you know you actually want to use it.   Simply based it’s a popular commercial product doesn’t mean it’s good, well-behaved software.  I’m sure this isn’t news anyone.

The second takeaway is another reminder that most software has many flaws.  Developers make many assumptions about what they are free to do on the users’ systems, resulting in poorly behaved programs once they have to run in the real world.   It’s disappointing, but it’s a reality.

After creating the blender model, I started looking into what an ideal asset management system might look like.  I used this as an excuse to spend some time with Python 3.1 and JQuery.   A python script runs on the server to generate some JSON indices for the website, then client side uses JQuery to present some user-friendly browsing and searching.

jQuery.fn.makeThumb = function(dim) {
  // Minimal sanity checking
  if (!this.is("img"))
    throw "Expected makeThumb call on an <img> element";

  // Provide a default of 192 px if no dimension is given
  if (typeof dim != "number")
    dim = 192;

  // Hide the image until the resizing is done to avoid
  // resizing flicker
  this.hide();

  // Wait for the image to load (so the width & height will
  // be accurate) then resize using CSS.  Of course the full
  // image is in memory and the rescaling algorithm is up to
  // the browser - but this is a decent way to generate thumbs
  // without any cached files.
  //
  this.load(function() {
    var w = parseInt( $(this).width() );
    var h = parseInt( $(this).height() );
    if (w > dim || h > dim) {
      var scale = (w > h) ? (dim / w) : (dim / h);
      $(this).css("width", (w * scale) + "px");
      $(this).css("height", (h * scale) + "px");
    }
    $(this).show();
  });
  return this;
}

Subsequently if to make all <img /> elements with the CSS class “thumb” into 192×192 thumbnails, the simple JQuery method call $(".thumb").makeThumb(192) can be made.

(Note: please feel free to use the above trivial code snippet as if it were your own; there is no need for permission, attribution, etc.)