A Lexical Mistake

Like many gamedev enthusiasts, I’ve only finished a handful of computer games and, perhaps not surprisingly, most of those were never released even on my homepage either. Usually the point of game development at an enthusiast or hobbyist level is not to create gaming masterpieces anyway, but rather to simply learn about the process and have fun working the code (or graphics, design, et cetera), and maybe even develop a few reasonably simple games.

Over the years I’ve spend quite a bit of time learning the gamedev concepts and recently I was able to apply some of them in a project, intended to test Haskell’s gamedev capabilities for hobbyist or small team game development (related post). Specifically I was interested to see how the difference between Haskell and Object-Oriented languages changed the design and implementation of the game code and engine, hopefully lending itself to the eventual development of languages tailored towards game development.

I set out to develop one of my favorite arcade remakes–yet another variation of Geometry Wars. The premise would be to keep the game as simple as possible, only the basic gameplay would be present with a couple of levels of progression. Enough to test all of the simple elements of gameplay ready for a more substantial project. As you may have guessed from the absence of the obligatory screenshot it didn’t work out as well as I expected.

Luckily I did pick up a couple of things while working on the game, and better yet I may even be able to rewrite the code and find that screenshot yet. It all comes down to a couple of quirks in Haskell’s syntax which didn’t blend well with my usual game architecture. Obviously the architecture would need to be modified to suite the idiomatic style of Haskell, but I was reasonably certain (and still are) that the architecture is the simplest implementation for both Object-Oriented languages as well as functional languages, based almost solely on the requirements of the high level game data structures.

The Haskell code for these structures was essentially based directly on records. The records originally contained only functional values, but I quickly realized how crazy that was, which lead me to the current design which uses records full of IORefs, at least for the mutable slots. (If its not immediately obvious why the IORefs are needed, it might help to know that game objects quite often need to reference each other simply because the operations are related between the two…bullets from the player ship for example.)

Finally, after working with this approach for a while it eventually sunk in that I could probably write the same code in Python in not only half the time, but also quicker for the same number of overall bugs. Not because the bugs are easier to catch in Python…they really, really aren’t. But because Python allows me to test many more variations of the game. At least more variations than Haskell seems to, all of which makes me wonder if I haven’t quite tapped into the idiomatic code still.

So I’m not quite out of ideas yet. I believe I can tweak my accessors to reduce the namespace damage caused by them (at the expense of some verbosity, however…why the hell hasn’t this been fixed yet anyway). And there’s even some clever usage of typeclasses which could simplify the use of common algorithms between game objects. If all goes to plan I’ll have some code to post next time…

– Lorenz

Recently I decided to pick up an apparently fantastic book on Erlang, written by Joe Armstrong (Programming Erlang). Now personally, I’ve never been particularly impressed by the man himself…too much of that functional programmer arrogance if you ask me. For some reason however, the hype got the better of me, especially after stumbling on a conversation between Armstrong and a couple of other functional programming ‘nuts’…uh…enthusiasts.

Perhaps of interest, is that I’m also quite a fan of functional programming myself, but I’d rather think of myself as avoiding a particularly insidious case of accidental complexity rather than using a fundamentally better language. I should explain… While functional programming languages does reduce a lot of the complexity accrued due to side effects and shared memory; they don’t deal with some of the other almost equally dangerous forms of accidental complexity.

Note: The actual problem I have with these functional language guys is the assumption that it’s a natural way of thinking about problems. Its not…it’s just, plain and simple, another way of learning how to solve problems. And its generally not one of the easiest to learn (basic anyone?)

I’m talking about compatibility of course, which brings me to the point of this post.

Cygwin…Mingw…’.a’…WTH!!?

After learning Erlang, and believing (perhaps even jumping the gun slightly), that I would be up to a real challenge, I realized that one thing was missing. A decent flexible, portable, Erlang code only FFI interface.

My challenge…write a real-time, computer game using Erlang. The likely candidate would be a space combat sim, within the vain of a Freespace2 style game (graphics and gameplay wise…hopefully the story will actually give our hero an actual name this time:)

Unfortunately the existing FFI in Erlang is either to use a network port to a C program linked to the candidate library, or to use a “linked-in driver”, attached to the VM and talking the VM’s data-structures in order to facilitate communication. Damn.

Talking to VM’s in case you’ve never tried it (although that seems unlikely if your reading my blog ;), is always a major pita. You effectively skewered between two of the most awkward types of interfaces known to programming kind, and worse yet, your only weapon of choice is what C gives you…or worse…a code generator (SWIG be damned).

No… What I need is something like the Pythoneers, imo brilliant, library…CTypes. And for C interfacing, the ctypes library uses a C level library called libffi (sort of anyway).

While this libffi based library of mine will no doubt be the topic of a future post, it does bring up a rather gruesome topic that I’ve been avoiding for quite some type now…Cygwin and Mingw.

After many hours of searching the web I’ve finally found what appears to be the solution.

Cygwin, as we all know is just a *nix/POSIX subsystem running over the windows COFF binaries (afaik), which allows us to blend windows and bash tasks during our development process (like using Windows compatible GNU/Make makefiles).

Mingw is where I got confused. You can find it in the installer for Cygwin, which put me under the assumption that it was more or less the default for this stuff…it isn’t.

Mingw is simply a cross-compiling target for GCC, which targets COFF binaries and the win32 C library (I can’t remember which one…I believe its the one in C:\windows however).

This means that to compile a binary for windows…one that doesn’t depend on cygwin.dll, or infact any of the cygwin dependent libraries you simply need to tell GCC to use the Mingw target…uh…sure…that sounds easy.

Actually, it turns out that this process is so involved sometimes (cross-compiling) that the mingw guys (or is that the Cygwin guys?) decided to make it easy. All you have to do, as mere mortals, is pass –mno-cygwin to GCC (“CFLAGS=’-mno-cygwin’ ./configure;make), and that’s it!

Fantastic…now back to the real challenge…using libffi to create my “Erlang C Types” library…

– Lorenz

One of the most important qualities of a good programming language is the first class function. This critical language feature makes possible a whole class of behavioural and data abstraction techniques. Without it we could not incorporate many modern language like continuations, generators, or callbacks.

There are of course many workarounds to any of these features. These workarounds range from articles on simulating continuations in Object-Oriented languages and callbacks can be replaced in most cases by the, somewhat more complicated, event or messaging passing mechanisms. Generators however, really are just un-useful without proper first class functions.

A generator is simply a function that sends a number, possibly infinite, of values to an independent language construct, which can then take advantage of these values, one at a time. The proverbial construct associated with generators is the for loop. The loop invokes the generator, a value function within the generator produces each value, and the value function itself calls a function parameter containing the code body of the loop itself. There are usually some extra mechanisms to handle premature termination of the loop (among other things).

I digress however, since what I really want to discuss here is the language elements left behind. Features such as environment management, the function dispatch mechanisms, exception handling, memory management, concrete syntax, and so on. What I really want to see is language designers moving towards a more unified system, providing programmers, or at least library implementers, with abstractions for these features, rather than fixed semantics that limit the expressivity of the language.

See despite so many languages taking great pride in providing first class functions, most actually just inherited these features due to the popularisation of functional programming techniques. The real hidden gem here, I feel, is reifying language semantics however. This can allow programmers to selectively enhance or extend their favourite language, without losing the built-in compiler or interpreter, possibly at the cost of optimisations complexity.

My favourite example here is the environment, an element of language design that has historically encountered limitless debate. Yet, for reasons that I cannot comprehend, even languages promoting expressivity over implementation consistently refuse to provide language level abstractions over these features.

I’m not saying that most programming requires such levels of expressivity, and quite the opposite, most programming tasks will never require it. But there are some tasks that would benefit greatly from the presence of such features. Rather than re-implementing these, it should be possible to just tap into the abstractions provided by the programming language.

One style of programming that would definitely benefit from these extensions is programming language research. It would be much simpler to test out new environment models, construct prototype virtual machines and interpreters, and even mess around with less common things like dispatch algorithms or meta object protocols, if instead of building entirely new environment abstractions, the existing ones could be utilized and re-engineered in a controlled way. This requires both abstractions limiting the direct manipulation of the host language implementation, otherwise you really just re-implementing the host language, and by providing a preferable more powerful interface than the one likely used in the host languages compilers or interpreter.

Using Scheme as an example, it would be trivial to add some intermediate abstractions for managing the environment. Scheme already implements something very close to first class environments, and in-fact many lisps do actually provide procedures for inspecting and controlling where evaluation takes place, but it’s really not quite there yet.

The reason I singled out Scheme in this case, even comparatively to other dialects of Lisp, is because the larger dialects typically include an impressive continuations facility. This combines with the hygienic macro feature, producing extremely impressive language extension capabilities. Unfortunately, you still need to generate raw code, when writing domain specific languages if you wish to allow fragments of host language code to be embedded into the DSL. In my case these fragments correspond to actions and predicates in a parsing language I’ve been implementing.

Perhaps I should attempt a Scheme dialect with such features, and just maybe I’ll even end up with something new?

– Lorenz

Creating a programming language is hard. I’m not talking about the kind of hard that’s involved in math problems or even getting pwned with that rather expensive new internet connection (no, its not the mouse either… you’re really just not that good;). No, I’m talking about the sheer amount of effort required to first come up with your amazing, world changing programming language, and then realizing that you actually need to implement the whole thing a few times over, using variations of that language that are in-fact worse than what you started with (unless your using Java, in which case, jump right ahead).

As you might have guessed I’m mainly writing this out of mere frustration, that even the dumbed down, simplified variation of Mention that I’m attempting to bootstrap is going, well, slowly. Very slowly.

So far I’ve come to the conclusion that there may even be some general rules of thumb, which to the enlightened few, might not actually seem all that obvious. Firstly, the current attempt to bootstrap Mention is utilizing the Squeak programming language/tool-chain, in the hopes that it would help to speed up the development process. It didn’t. Actually, I’m even wondering if its actually slowing me down… Which brings me to the obvious question. Why?

The most useful languages that spring to mind after asking this question, probably actually come from the Lisp family, along the lines of some dialect of Scheme + a parsing language to complete the picture. But what would make a language like Lisp, outgun its successors like Smalltalk or perhaps even python. Well, actually, I think python might even work, but Smalltalk has issues… Namespace issues.

The issues with Smalltalk I’m referring to are the conventions surrounding the naming of global variables, or at least the lack-there-of. Many scripting languages and most Lisps, allow the programmer to assign to any old variable name, and its simply there, for the entire world to see. But of course, when you have to think in objects, you can’t expect to have globals. After all, good Object-Oriented software should be based on the interaction of object, and not glorified globals, singletons, or otherwise. Although you could be mistaken for thinking that the Application object is actually a global, but of course its not. Why, well I’m not sure really, its just not.

I’d rather not go on any further with this, but the point is simple. We know that software systems evolve, and don’t simply get designed and then implemented. And we know that software is not actually re-used until at least some refactoring has been performed (generally more than some). So why can’t we just use our globals while we understand the problem we’re working on, and if they do the job we can just keep them, and if they really are flakey code smells bend on silicon armageddon then we can simply refactor them once we understand how the problem breaks down into the individual problem level (application, subsystem, et cetera) objects.

– Lorenz (feeling slightly better ;)

Well I finally bit the bullet and decided to learn the Haskell programming language. For those in the know, Haskell is a very strictly typed language (like some kind of draconian Java dialect), which uses a type inferencing technique to allow the programmer to write code without excessively typing… well… the types really.

What makes the language interesting however, is that is does this kind of strict typing from within a pure functional syntax (one that is very close to the mathematical lambda notation), and quite literally everything except the bindings themselves are first class values (objects really), and they all have types, which are themselves of some type and all can be fully reasoned with using the full capabilities of the syntax.

What all this means is that unlike Scheme (Lisp), instead of allowing objects of any type as values, all values and expressions must be the the correct type in any given piece of code (or they must be inferred transitively from their surrounding/contained code). Any mismatches are not allowed by the languages semantics and will cause compilation (or interpretation) errors.

This brings me to the point of this post, is a strictly typed language actually less or more capable than a weakly typed language. Also, if this is the case, does is it make the language better, or are other considerations more important (i.e. architectural concerns)?

I find this discussion interesting since it quite effectively contrasts the type system I designed, for Mention. While Mention’s type-system is effectively not just weakly typed but in many cases not checked at all, it allows arbitrary objects to be used in most situations, as long as they conform to the interfaces requested (in the type annotations or implicitly like Smalltalk).

Already I’m finding that I miss the more flexible type system when I find the only error messages cryptic type incompatibilities (what do you mean the type ‘a -> [a] -> a’ isn’t compatible with ‘Num a’… Hell where is that number anyway?). It is slowly getting easier as I understand the impressive type reasoning capabilities of the language, not to mention the ominous Monad magic (think C++ templates on steroids… although probably very much easier to use effectively)

Presumably as I continue to work with both Haskell, Smalltalk (Squeak), and Mention I can come form a better opinion on the effects of strong vs. weak type systems (in languages with first class function and type objects). Oh… and they are right… Haskell does seem to be a language well worth the effort to learn effectively.

– Lorenz