Category: Games

“Size is the Enemy” is a Jeff Atwood (Coding Horror) article that has made the rounds lately. It’s a response to a rambling rant by Steve Yegge about how large code bases are unwieldy, and how language choice affects the viability of software projects beyond a certain size. Early on, Jeff calls out a quote that I, too, find really amusing:

I happen to hold a hard-won minority opinion about code bases. In particular I believe, quite staunchly I might add, that the worst thing that can happen to a code base is size.

I don’t think I’ve ever encountered a programmer who truly believed that “bigger is better” with regards to codebase size. Granted, the crowd I tend to run with isn’t exactly a bunch of Perl golfers either, but no reasonable person would doubt that complexity scales superlinearly with lines of code. So, at least with regards to the sentiment that codebase size should be minimized, I find myself in agreement with both — if you can reduce the size of your code while maintaining readability, then it’s an easy win. For example, on Neverwinter Nights 2, we drastically reduced the number of scripts required by adding a parameter passing mechanism to NWScript — I actually ran the numbers and discovered that 75% of the script calls in conversations used our new, parameterized scripts rather than the one-off style of scripts from NWN1. This is a great example of reducing code size without reducing functionality, and it made things much more manageable.

I take issue with the second conclusion of Yegge’s article, though. While it’s true that functional languages can often express algorithms more eloquently than imperative languages, I don’t think this magically translates into huge maintainability wins in a monster codebase. Sure, the trivial example of reading lines out of a file makes other languages look great in comparison to hoary old C++. (I’m not sure if I’m allowed to call Java “hoary old Java” yet — after all, it’s only 12 years old or so.) But to hold up this example as proof of language superiority is missing the bigger picture — what makes an application distinctive is not how it reads lines out of a text file, but rather all the other stuff it does that no other piece of software does. The design of that “stuff” probably has more to do with maintainability than language choice ever could.

The biggest factor in the complexity of a code base, in my opinion, is the complexity of its internal interfaces. I hate to trot out the quixotic concept of the “software IC”, but thinking about an interface in terms of building it onto a chip is a decent analogy in this case. Once you get beyond a certain number of “pins,” or command multiplexers, or what have you, things get complicated. When you have 500,000 lines of this kind of “complicated,” all interacting with each other in mysterious ways, you have big trouble.

Now, I’m not advocating going gonzo with componentization, either, in spite of how delicious ravioli code sounds. I’ve seen and heard about way too many over-engineered, CORBA-gone-wild projects to make any kind of blanket statements in support of component architectures and stuff like that. But by keeping interfaces (whether internal to the code, or part of a component system) minimalist in nature, and designing them so that doing the right thing is easy, and doing the wrong thing is hard, you can maintain a good level of understandability in a codebase. I see no reason why this advantage does not scale with project size. Dividing a codebase into easily testable, well-defined components with simple interfaces is key, particularly for lone-wolf developers.

I find it very interesting that the project in Yegge’s article (Wyvern) is not just a game, but a role-playing game. I know first-hand that RPG rules systems can, by their nature, necessitate a kind of code design that leads to massive complexity. RPGs tend to carry a massive amount of state around, and have rules systems that interact with that state in often arbitrary ways. (An example of this would be a character trait that changes the order of combat resolution, or some kind of “luck” trait that allows re-rolls of certain types of skill checks. If you have many rule-changers like this, it’s almost impossible to write a clean system to handle it.) This kind of complexity is design complexity, and has nothing to do with programming languages and the features they support. For this reason, I think that even if he succeeds in his goal of removing 33% of Wyvern’s lines of code, I don’t think that the resulting code base will be any easier to maintain. (His game is a 2D Java RPG — I don’t think that it’s a stretch to say that the majority of his code is going to be related to game rules and game content.)

I think that both authors are missing a key point — namely, that if you can simplify your application or problem, you should do so. Wyvern should look more to Magic: The Gathering’s rules than D&D. (Granted, Magic has its own mind-benders, but I think that it’s fundamentally simpler than even the newer versions of D&D. Coming from a math professor, that’s what I would expect.) Granted, sometimes it’s not possible to simplify a problem any further, but in this case Wyvern appears to be a self-inflicted wound. And, finally, I am still in agreement that code size reduction is a wonderful thing, but I am less enthused about the opinions rendered in the language wars…

The gaming blog Rock, Paper, Shotgun recently ran an article mentioning the next project of the developers of the 2005 PC game Boiling Point. That upcoming game, however, is not the real focus of the article (nor of this one). There’s a bit of pointing and snickering at part of a bug fix list for the first patch for Boiling Point — I’m sure that the bugs in the list seem pretty crazy to most people. I thought it would be fun and illuminating for non-game developers to give my best guesses as to what these bugs actually represent under-the-hood, and how they may have happened, solely based on the descriptions. Note that I haven’t played or ever even seen Boiling Point in action before, so my guesses may be totally incorrect.

The aforementioned article had the following bugs highlighted:

• fixed: the snake wasn’t able to bite you while you were crawling

• I have two guesses as to how this may have happened. One would be that the player’s “hit box” and the player’s “personal space” (a space surrounding an object into which other objects can’t move — I’ve borrowed this term from Bioware‘s earlier games) are not always the same. When the player crawls, perhaps his personal space grows large enough that the snake’s weapon range isn’t long enough to reach from the edge of the player’s personal space to the player’s hit box.

  Another guess would be that the targeting AI for the snake (governing where to move in order to attack, and things like that) somehow got messed up when the player was prone, or had an early-out return.

  Odds are good that this slipped through the cracks because either it wasn’t tested, or that crawling was tested for other animals (but not the snake). Not uncommon.

• fixed: size of the moon

• My guess would be that, if there was actually a separate “moon” object (i.e. it was not something done entirely in code), that it was created as a copy of another object, and the size was never correctly adjusted.

  Alternately, if the moon was created in code, it’s possible that: a) nobody ever really looked for it, b) it was working at some point, and later broke, or c) if the game simulates lunar cycles, that testers explained away the missing moon by assuming that it was a new moon.

• fixed: posters in bar vanish as you turn away from them

• The culling code is buggy for that specific type of object, or the bounding boxes generated for them are incorrect. If the posters are built as three-dimensional solids with a very small depth, it’s also possible that it may be caused by numerical precision or other issues.

  Stuff like this tends to be pretty easy to notice, as long as it’s in an area of the game where players will commonly visit — I’m guessing that the developers knew about it, but it was not determined to be a “stop ship” bug.

• fixed: dog does not cast shadows

• For performance and artistic reasons, it’s common to have per-object flags to turn off certain effects or processing. The “cast shadows” flag was set wrong on the dog.

  Likewise, this is not a “stop ship” bug, so it’s a toss-up as to whether or not the developers were aware of it.

• fixed: a metal clanking sound plays, if the user’s character stabs the curtains

• As with the dog, this is an issue with incorrect data being set on an object.

  This is also not a fatal bug, but it’s one that I can easily imagine slipping through the cracks, since it is not only object-specific but weapon-specific as well.

• fixed: jaguar floats across screen at treetop level

• This is a tough one to hypothesize about without having seen the game. One possibility is that creatures in the game can be set as “flying” (unaffected by gravity), and this jaguar had that flag set on it for whatever reason. If the jaguar was initially placed on a slope, but then walked away, it would seem to be floating in mid-air.

  Another possibility is that it is somehow set to collide with coarse LOD bounding box representations of, say, a village, and is just “walking” on top of it.

  If this bug only occurs in one area, or with one jaguar, then I can see this potentially being missed during testing. If it doesn’t cause problems with the game, either they figured it was okay to ship, or ran out of time to fix non-fatal bugs.

• fixed: npc die on contact with grenades, and not from the actual explosion

• It’s not clear from the description if this is limited to friendly NPCs or not, but it’s possible that the damage resolution code for area-effect weapons has an (erroneous) early out for friendlies, while the damage resolution code for other types of attacks does not.

  This might be a case of a regression failure, where functionality that was working at one time later breaks due to other changes. Alternately, it may be a case of testers selectively testing ways of killing friendly NPCs. (Hey, it’s an anti-social thing to do in the first place, so I don’t blame them for not trying out every weapon on their buddies.)

Like I said before, I might be totally wrong on some of these, but it’s fun to speculate and to try and understand how games are doing things under the hood…

Having constructed the Potemkin villages that are game environments, game developers just don’t see games the same way that the general public sees them. Every game developer I know has, at some point (and usually, often) sat around with co-workers, playing the latest and greatest game and critically ripping it apart for problems that are both easily seen and those that are more subtle. “You can totally see a seam on that texture.” “Oh, great, another FedEx quest.” “This UI is terrible.” “That cutscene sequence was dumb.” “Weapon X is totally worthless.” “This game is so unbalanced.” There are probably a million more complaints that you could come up with. Knowing what goes on behind the curtain makes us understand limitations of game engines, where corners have been cut, and seemingly inexplicable design decisions better than most.

When the Playstation 2 was released in the US, I sat in on our company demo of the new console. Partly due to the underwhelming launch titles, and partly just because programmers tend to be grouchy, we had plenty of fodder for complaint and discussion. But one of the more excited and forgiving souls in the room rolled his eyes and pronounced that, “Man, listening to you guys is like going to see a movie with film majors.” Truer words were never spoken.

I recently read this great blog post about Mass Effect (a game I have not yet played), in which another game developer points out some of the tell-tale signs of triage, content cutting, technical challenges, and design changes. In addition, there’s also some discussion of how these situations come into being. Having been a big part of a large, ambitious game project myself, I can vouch for a lot of the bullet points discussed, and wanted to comment a little bit further on things.

It is extremely rare for games to ship with the full feature set envisioned early on in the project — so typically, features and content are classified as either vital or non-vital. If, during development, a feature or piece of content is not working out, it can be cut if it is non-vital. The need to classify work in this manner early in the project’s life span, however, also begets a lot of the problems mentioned in that blog post.

• “Undue or odd simplicity.” Game mechanics that sound great in design discussions may not work out so well once they are later implemented. (For example, the boss battles in Guitar Hero III have been pretty much universally panned.) Since a lot of resources may have been poured into these ideas, the preference (rightly or wrongly) is usually to keep trying to make them work somehow, rather than just cutting them. These triage efforts may take the form of simplification, or providing alternate methods to “get around” the balky mechanic. The latter category includes the “you’ve failed this part 5 times, do you want to just skip it?” crutch that has become popular in recent games.
  
• “Significant differences between demos and the actual game.” Even if you do decide to cut a feature or some content, if you’ve ever showed it to the public or talked about it in the press, people are going to be wondering where it went in the finished product. And if you leave those vestiges on the shipped disc, you never know what might happen…
  
• “Sudden narrative leaps and loose ends.” Cutting non-vital story content and dialogue can still leave awkward gaps in a story, even if it doesn’t totally compromise it. Characters that simply show up or disappear without a trace, deus ex machina, and sudden jumps in location or disposition all tend to be signs that some non-vital content was cut and patched up in a low-risk fashion.

I also have to throw in some additional RPG-specific commentary on the “lots of bugs” category. RPGs tend to be games of large scope. They try to provide an experience that is more customizable (in terms of choices having an impact on gameplay) than almost any other genre, an engrossing and lengthy story, and interesting gameplay on top. If you imagine the gameplay experience translated into a directed graph of a player’s actions, RPGs tend to have very high branching factors compared to other game genres. This variability drastically increases the amount of test paths that must be checked, and even if you have a massive army of testers, it is extremely difficult to ensure 100% test coverage. I would go so far as to say that no modern RPG can achieve this. (The continual refinement and patching process of MMORPGs is a testament to this.)

An anecdote of the complexity of RPG games was related to me by a former co-worker at Obsidian. He told me that, several years ago at Black Isle, there was some discussion of adopting the then-new Dungeons and Dragons 3rd edition rules (widely viewed as very successful, and which simplified many arcane aspects of earlier editions) into a game that was already in development and using an existing engine. As part of the feasibility study, effort was put into determining how the combat system would need to be changed. One of the programmers diagrammed the combat resolution system on whiteboards, and wound up using something like 54 square feet of whiteboard space on an incomplete flowchart of the system, before the decision was made that it not feasible to adopt the 3rd edition rule set. The vast number of feats, spells, and items that exist in modern D&D take a massive amount of effort to implement, and oftentimes have subtle, unspecified, or ill-defined interactions with each other. Granted, as far as complexity goes, D&D is fairly high on the scale, but even simpler rules systems are tricky to get right.

Mass Effect is, by all accounts, not a short game, so I’m not surprised that it may seem buggy. But if game developers and publishers want to continue producing epic, complex RPGs, I believe that the tools for producing and testing gameplay content must become much better, and automated testing of gameplay scripting must somehow become a reality. Ensuring that changes and additions do not break progression in an RPG can sometimes be a very time-consuming process, because a bug may only affect certain character types, later parts of the game, or (worst of all) be randomly-occurring. To put this in perspective, for NWN2, we had testers who were “sprinters,” whose job was to simply go through the game as quickly as possible, to ensure that the “critical path” of the story was working and viable. If I recall correctly, “sprinting” through the game took about 20 hours. When you need a multiple-day test to really ensure that a change hasn’t caused another problem somewhere else, your iteration time and desire to make polish changes suffer.

One source of direction and inspiration is Inform, the interactive fiction authoring system whose latest incarnation uses a natural language system to construct gameplay. I’ll probably have more to write about Inform later — I think it’s fascinating and a tantalizing glimpse of the possibilities of higher-level gameplay tools.

It can be found here. The biggest new features include integration with all SKUs of Visual Studio 2005, and support for Live on both Windows and the 360. The Visual Studio integration is huge, in my opinion, if only because it enables source control and add-in support for developers. I guess now you’re also able to use mixed-mode debugging for XNA projects, but I believe that is only useful for Windows-based projects — I can’t imagine that they’d ever allow unsigned code execution on the 360 (for security reasons). One can always hope, though. 😉

They claim that the final release isn’t far off, either. With Visual Studio 2008 having just been released to manufacturing as well, I would imagine that there will be a refresh for XNA Game Studio that will include support for 2008. (The 2.0 beta doesn’t seem to integrate with 2008, or at least Beta 2 of 2008.)