Home About EIS →

Designer Intent vs Emergence: Nissan Edition

Nissan Datsun 510: Fastest Car on the Planet

Forza Motorsport 3 is, as described by one of the designers, “car porn.” And, well, that beauty above? That delicious, glistening, throbbing piece of machinery? That’s a Nissan Datsun 510 from 1970. You’re free to take a moment right now, if you need one… imagining yourself astride the seat, hand gripped on the gear shift, thrusting forwards through the forest…

At least, that’s what Forza 3 would have you think.

Once you’re back from your “moment,” please continue over the jump to see exactly what I’m talking about.

The video linked above is aptly titled “Forza Motorsport 3 Drag Race WTF.” It shows a very fast race car being soundly beaten by one of those beautiful Datsuns. This driver was not the first person to notice. The problem is that the Nissan is able to be upgraded with the engine from a Skyline, amongst other upgrades, which boosts the car’s horsepower output to a phenomenal degree.

What’s happened here? Who’s to blame? Is anyone at fault? Here be the dragon of emergence.

Dominant strategies are one of the first aspects of a game to be communicated around the Internet. As soon as they are discovered, the viral potential is enormous. Some members of the community will revel in them, some will be disgusted, and many will be ambivalent until they’re upset by them directly.

In order to talk about the problem effectively, we need to enforce the boundaries of the problem (not doing this can lead you to arguing with a psychologist until you’re blue in the face about the possibility of solving of the Turing Test1 )

  1. This is not a bug in the implementation (unlike Modern Warfare 2’s javelin glitch), but a bug in the game design itself.
  2. The bug is a direct result of the emergence from having a very large number of combinations of cars and upgrades.

Doing some very rough mathematics, let’s assume there are 400 cars, each with a possible 25 upgrades that could be applied. This would give us:

400 * 2^25 = 13,421,772,800

This is, undoubtedly, a big number. It’s very, very rough: 25 upgrades is something I’ve pulled out of the air, and you can’t apply all upgrades at the same time (ie. you can’t apply two different types of exhausts). A more realistic number might be more along the order of:

400 * 2^15 = 13,107,200

The number gets very big due to the number of different upgrade combinations, which causes the tree of the possibilities to blow-up very quickly. Scientists dramatically call this an “explosion.”

That said, it is something we can enumerate, and 13 billion isn’t that many possibilities for a computer. However, for a human game designer, we’ve escaped the realms of human processing possibility, and there’s no way one person could ever understand such a state space.

The 'S' is for Software!

This is one of those problems where my ego imagines myself donning some sort of cape, adorned with fine circuit boards and transistors, and allowing myself to exclaim “software engineering to the rescue!” We could have detected the Nissan Datsun problem before the release of the game by encoding a set of designer expectations that we can test for. These expectations we’ll more formally call “assertions.”

A sample expectation, in this case, would be “An E-class car can’t win drag races in less than 25 seconds” or “An E-class car can’t beat upgraded B-class cars in a drag race.” We add them to a software program, benchmark all the possible cars, and come back and see which of our assertions failed. Perfect!

Such a program wouldn’t take long to create, so obviously there are hidden problems here. Here’s a few:

  1. Encoding assertions, while not hard, exercises a part of the brain we don’t often use. What’s important to assert, and what isn’t? Programmers wrestle with this problem every day: which part of my software do I need to test the most? It’s not an easy mindset to be in, but it is possible.
  2. Forza 3 takes about 3 seconds to benchmark the top speed of a car. If we ran that test across our low-ball estimate 13 million cars, it would take 1.2 CPU years. Against our conservative estimate of 13 billion cars, it would take 1276 CPU years. Oh dear. But! This just means that a brute force approach to testing isn’t possible. If we just look at the characteristics of the problem, maybe we can do better. Winning a 1 mile drag race is all about horsepower… what if we just calculate the statistics for each car and compare the HP? With a completely made up number of 0.1 seconds to calculate, we bring the numbers down to 15 days and 42 years, respectively. Some intelligence of searching (like testing the most expensive parts first) should improve the time it takes to get useful results.

As you can see, the struggle with designing correct emergent systems is a fight that designers are doomed to lose.

On the one hand, emergence is the key to making great games, experimentation and choice are a linchpin of the form. We can’t get rid of it, nor try to artificially constrain it. On the other hand, emergence creates systems that encourages abuse, being poked and prodded by millions every day. All it takes is one to communicate their results, and it’s all over. Gamers don’t want to lose in races to Datsuns. Testing against the emergent properties is going to become very important. I’m currently exploring this problem, and I hope to have something to show you all soon.

Until then, I’ll spend some time making the back of my Datsun look pretty. It’s all you’re ever going to see.


About the author:  Chris Lewis is a British PhD student researching the intersection of software engineering and video game development. Read more from this author


  1. The Turing Test is well within our grasp if you understand the boundaries of the test: convincing someone else you’re human with just textual responses does not require sentience! []
This entry was posted in Authoring, Games, Gaming Culture and tagged , , , , , . Bookmark the permalink. Both comments and trackbacks are currently closed.

11 Comments

  1. Til
    Posted January 5, 2010 at 10:21 AM | Permalink

    If you have 400 cars and do not want to make the Datsun faster than the race car, would it not be easier to simply state that in the program code?
    E.g. you have a max speed a car can have and from that number you substract a number determined by aerodynamics of the Datsun, stability of its pieces, etc. The reason a Datsun can not drive faster than the race car is physics after all, isn’t it? I am not a car engineer, but I bet there is an easier solution than searching all possible combinations.

  2. Posted January 5, 2010 at 10:52 AM | Permalink

    If I’m understanding you correctly, what you’re proposing is having some weighted value that scales with class, and then do something like “top speed – e^class”?

    This is a good idea!

    The only problem that I would have with this is that you create a system where the pieces in there do not create the same results for each car. Adding the a 500hp engine to the Porsche 911 would result in a higher top speed than on a Datsun, even if every other component is the same. This may confuse some gamers, where the upgrade doesn’t have perceivable results.

    While I agree with you that aerodynamics provides a perfectly reasonable explanation for this, my understanding is that Forza doesn’t actually expose that as a value (unlike HP, weight, lateral G-force, etc). If it did, I would expect that the developers would be wary of creating some magic value, as every other one has been meticulously researched and modeled. Perhaps having a magic value of “component quality” would work. The reason you expect a race car to beat a Datsun is not just its aerodynamics, but the quality of all the components in the car should work at a higher performance than the Datsun.

    So, to conclude, yes, I like your idea! Unfortunately, there are no easy answers, and there are trade-offs with both mine and yours. However, knowing the trade-offs means one can choose the right approach for one’s specific game!

  3. Til
    Posted January 6, 2010 at 8:02 AM | Permalink

    Thank you for your detailed answer. First of all I am sorry for writing a comment that might be hard to understand and furthermore I have to admit I have never played Forza, but was interested in your article because I currently am studying search strategies and have been working in game design before.

    A value like “top speed – class” might indeed work and was among my original ideas, but as I now understand the high standard of Forza it would be too much of a ‘cheat’. “Component quality” might be a more appropriate value as it can both be communicated to the player and scale with the class of a car (and each single car).

    I do not see the problem you point out in the second paragraph. As you can tell from the video, the expectation people have is that a Datsun can never be as fast as a Porsche 911. The reason is, I suppose, that we expect things like “component quality” and “aerodynamic limitations” restricing what you can make with a car. Thus I believe it would make sense to a player that a car with bad aerodynamics and old components can not convert the HP as well into speed as a Porsche (with the same HP).

    In general I found this topic interesting because it is testing vs. modeling (or in game producing the question how much of your time/money you put into design before you start to implement – obviously carefull design will reduce testing/repairing in the end). The main problem might be that both the designer or the one who controls the assertions you supposed would actually have to expect that the expectation “An E-class car can’t beat upgraded B-class cars in a drag race.” has to be considered. Plus you need time in your budget to check it.

  4. Posted January 6, 2010 at 1:53 PM | Permalink

    I suppose my issue is that we’re assigning a fiction to a game that presents itself as a scientific model. The narrative reason needs to be consistent in the world we present, and so while “aerodynamics” vs “component quality” appears to be cutting hairs (they’re both just a magic scaling factor), it is important in the context of this game.

    The difficulty with “aerodynamics” is that this a real, testable property that one could simulate in Forza. Forza appears to go to great pains to create realistic values, and so we can’t assign a quantifiable name to some magic unquantifiable scaling factor. That’s why I suggested “component quality.”

    The testing vs modeling argument is one that my personal research wrestles with, but what happens is that games developers are constantly pushing on the boundaries of what is accurately modeled… but solving a modeling problem then creates its own problems! The Havok physics engine is a great engine that couldn’t have been modeled 15 years ago. Now that a lot of games don’t have physics problems, the problem space moves to something else, such as players being able to block doorways by moving crates, which could well be outside the game designer’s wishes.

    I see testing vs modeling as complementary. I think that the problem here is that it seems that modeling simple systems like Newtonian motion is going to lead to very complex emergent systems (their relationship is that making small improvements to those simple systems can have large effects on the emergent properties), and that modeling will outpace the quality of testing available. Testing needs to keep up with the emergence better modeling will bring. However, right now, software engineering testing methods are designed to anticipate finite, testable state spaces, so games publishers simply employ thousands of games testers to manually go through the game. That’s not scalable (and arguably was out-scaled a long time ago).

  5. VRBones
    Posted January 7, 2010 at 10:20 AM | Permalink

    Well, the Datsun 510 chassis is pretty light in real life. Here’s a 510 doing 10 second 1/4 mile with only ~400bhp so going ~9 secs with >900bhp and 4wd wouldn’t be totally fantasy. Here’s another that looks even faster.

    Stuffing that engine into an elise or any other lighter body would produce even better results, so the problem comes down to where do you draw the line? What combo is legitimate? In real life there’s nothing to really worry about because even the 510 would get flogged by a Willys ’41 funny car. Looks has nothing to do with what’s under the bonnet.

    Since there is a line drawn in the game about what combinations are possible, I agree that it’s the responsibility of the game designers to at least check what combinations are going to make it to .. say .. the top 10. Even a simple power to weight calculation should be sufficient to give ballpark estimates as to what cars will make it up there. You could safely ignore every engine or component but the largest (or the top couple if they have rotaries that are themselves lighter), so that would give say 400 x ~2 = 800 calculations to give you an idea on what the top 10 cars could be. Top 10 would be easily digestible by designers working with the game day in day out to see if it’s reasonable.

    But back to the 510; would they have minded even if they saw the 510 in a top 10 list? It’s an iconic car. It could definitely do that sort of speed if given the engine. Maybe they meant it to be up there with the best of them? Maybe. I doubt they would’ve wanted it stripping everything else though because it just doesn’t feel right to the average punter. Why did I buy this Veyron?

    On the other hand, emergence creates systems that encourages abuse, being poked and prodded by millions every day. All it takes is one to communicate their results, and it’s all over.

    To me this is the kicker. It’s not emergence itself that’s the problem, it’s that there is only one solution. Once someone cottons on to the best formula (conventiently transmitted by the high score table) there is no point using any other formula if you’re attempting to compete. It’s drag racing itself that’s the problem because it is one-dimensional, there is no recourse to the optimised formula.

    In other games with emergent properties like “Magic the Gathering” and “Pokemon” there is more than one dimension for the problem space to be satisfied and there are pre-built counters to one-dimensional strategies. If a green creature deck is technically the most optimal deck for resource use, it cannot gain 100% following for it then is exposed to an anti-green strategy. There is no counter to someone going faster than you, you either play the same strategy or put yourself at a disadvantage.

  6. Posted January 7, 2010 at 11:09 AM | Permalink

    “It’s not emergence itself that’s the problem, it’s that there is only one solution. Once someone cottons on to the best formula (conventiently transmitted by the high score table) there is no point using any other formula if you’re attempting to compete. It’s drag racing itself that’s the problem because it is one-dimensional, there is no recourse to the optimised formula.”

    Ah, an interesting point. I won’t claim I thought of it when I was writing, but you are right that it is a big part of the issue with Forza. I can’t make a Lancia Integrale that goes that fast (trust me, I tried), so having the single dominant strategy is Not Cool. The drag race format also exacerbates the problem as it’s almost skill-free, but the problem does spill over… watching a Datsun fly past an R3 class car on Le Mans is disheartening!

    And in case it’s in doubt: I love emergence. As I said in my piece, it really is what makes the medium so special, it’s just so tricky to get right! Pokemon/MtG gets away with it by building their rock/paper/scissors cycle of x is better than y is better than x is better than x, ensuring no-one gets ahead. Unfortunately for Turn 10, it is possible to have a truly great car, given enough money!

    P.S. Those videos were amazing… I wonder if I can change the blog into “ridiculous drag race videos from YouTube.”

  7. Til
    Posted January 7, 2010 at 12:26 PM | Permalink

    I agree with your last comment and I can see your points. You did however awake my interest in automatic testing, so some remarks:

    What exactly do you mean by “employing thousands of game testers is not scalable”?

    Maybe useful: I can’t exactly recall how they did testing with Civ 4, but I think it was partly automatic and i found it interesting when I heard about it.

    BTW., as I discussed with a couple of fellow students how one could use neuroinformatics and reinforcement learning in the game industry, I would be interested if you knew whether such approaches have been used in automatic testing of games…? I could imagine that testing games might be an interesting field to apply respective methods…

    I have been testing games myself, I believe the big advantage of human testers, especially in a complex system, is that they might do things a programmer, who is writing a testing routine, did not think of and that for a very long time human testers will be ahead of machines (and often ahead of programmers :) ) in understanding human expectations (+ fun etc.).
    But it definitly would be great if humans did no longer have to test whether e.g. in a story with a lot of branches there are branches where things won’t work out anymore (no terminal state is reached). Such techniques would actually give designers/writers a lot of freedom to make better games.

    Do you know a good paper to get started with the topic of automatic testing?

  8. Posted January 8, 2010 at 1:54 AM | Permalink

    @Til:
    We hit the limit of reasonable threading, so I’ll go down here :)

    “What exactly do you mean by “employing thousands of game testers is not scalable”?”

    What happens when games get inevitably bigger? Hire tens of thousands? Even if it was feasibly possible, it’s economically not. If I was a conspiracy theorist, I would say this is why many games are having beta periods, and it’s not just to test the network code :)

    “Maybe useful: I can’t exactly recall how they did testing with Civ 4, but I think it was partly automatic and i found it interesting when I heard about it.”

    I think someone else may have mentioned this in passing to me as well. I need to look it up! I think it was something to do with simulating players, which I have seen before. Instead of exercising the entire code base, you simply try and model what players are likely to do, and see what happens. Player modeling is a rich area which other people in EIS would be able to talk about better than I, but I tend to shy away from big AI problems. The joining of large parts of Soft Eng and AI are inevitable (particularly in testing), but right now the AI is a bit too far out for the pragmatist in me! My thesis is being written in the next three years, not a decade 😉

    “BTW., as I discussed with a couple of fellow students how one could use neuroinformatics and reinforcement learning in the game industry, I would be interested if you knew whether such approaches have been used in automatic testing of games…? I could imagine that testing games might be an interesting field to apply respective methods…”

    You’d have to tell me more about what respective methods are :) Reinforcement learning has been employed successfully in AI-controlled characters in games. Coincidentally, the driver AI in Forza has reinforcement learning in there to reflect how players drive.

    “I have been testing games myself, I believe the big advantage of human testers, especially in a complex system, is that they might do things a programmer, who is writing a testing routine, did not think of and that for a very long time human testers will be ahead of machines (and often ahead of programmers :) ) in understanding human expectations (+ fun etc.).”

    Well, doing things you didn’t anticipate is the difficult aspect of any testing methodology, that’s why we have patches and updates :) My feeling is that as programmers get more experienced, they learn what they should and shouldn’t be testing (often learning from mistakes). My hope is that game designers will be able to do the same. I’m also not advocating removal of the human tester completely, but a reigning in to more managable team sizes and testing “what is fun, what could be improved, what needs polish”, as opposed to “please find the bugs that break the game so we can ship”

  9. Posted January 8, 2010 at 2:10 PM | Permalink

    A highly modified Datsun beating a Le Mans prototype in a drag race seems realistic to me. A Le Mans prototype is designed to generate downforce, improving handling to the detriment of acceleration. It seems like a highly modified Datsun ought to win in a straight line. If it was up against a top-fuel dragster that would be another story.

    I understand that there’s something of a psychological hurdle, i.e., “that little car shouldn’t be beating this race car!” But Forza’s all about modifying and improving cars. Imposing arbitrary performance limits just to insure that classes remain stratified is not what the game’s about.

    That’s not to say that problems can’t arise from the interplay of large numbers of elements; I’m sure we could find lots of good examples. But this isn’t one of them.

  10. Posted January 8, 2010 at 9:44 PM | Permalink

    I think then we come back to VRBone’s point that we still have a dominant strategy here: I can’t put a Ferrari engine in a Fiat, nor can I slap an Audi V10 engine into a Volkswagon. The Datsun, by hook or crook, happens to be the a black sheep, and the lack of any other cars that are like this makes it appear like an oversight. I would be surprised if Turn 10 jump on Kotaku and shout “this is how it was meant to be!”

    The other bummer about the Datsun is that the Performance Indicator that puts it in a class is totally broken. I’m fairly sure the video from above is a same class one, and I’ve done R3 and R2 races where the Datsun is amazingly quick in comparison… I need to properly test, but I reckon a souped up Datsun could win those races with little concern (last I played my friend was just excited about hitting 240mph down the LeMans straight before hitting the tire wall).

    Maybe if we change my assertion to “a car within 50 PI of another cannot beat the other car by > 5 seconds in a drag race” we have something more acceptable? :)

    However, we’re just arguing about the framing of my point. I tend to frame arguments about the games I’m playing :) Next up: why the Assassin’s Creed II villa system is terribly broken too!

  11. Theory test site
    Posted September 24, 2010 at 9:41 AM | Permalink

    hmm,
    I’m not familiar with the forza 3 game engine, it looks like it should have also employed some form of a physics engine. The datsun 510 does not look very aerodynamic so should have a high drag and require a higher output engine then the other car.