Sometimes, emulation is not enough

A tale illustrating why testing on a real machine is important when developing homebrew software.

Introduction

Two weeks ago, I participated to the A2FP, a convention of Apple II French users. For the occasion, I wanted to bring my machine and show case the “rogue-like” game I am coding. Unfortunately, the random level generation hanged the computer… whereas all was working perfectly on two different emulators! How is that even possible?

Starting a new game on AppleWin: the level is generated correctly.

Starting a new game on a real Apple IIe: the level generation hangs during "PLACING ACTORS"

What is the purpose of an emulator?

Most emulators seek to replicate the behavior of a machine closely enough to run its programs without any noticeable difference. Those programs reside inside a binary file representing the whole content of there mass storage, a “rom” file, an “iso” image or, in the case of the Apple II, a .DSK.

If the behavior of an emulator is accurate enough, all the programs will be able to run without any trouble. Some less perfect emulators require specific code-path to run some applications that have stricter accuracy requirement than the emulator can provide. But from an external point of view, the program runs as it would do on the real stuff.

Most emulators seek to be accurate enough to run the entire library of the machine; they do not try to be 100% accurate to their model. Why is that? Because beyond a certain point, increasing accuracy is hard, and costly in CPU. In an article published on Arstechnica 10 years ago, the famous byuu, author of the most accurate SNES emulator in existence, stated that, while SNES games could run on a 25MHz PC in the late 90s, a 3GHz modern machine is required to accurately emulate it. Their goal was not only to run the games, but also to document the machine for posterity when no original hardware will remain.

Back to my case.
The two emulators I ran my program on, AppleWin and Accurapple, are both reliable and can run most, if not all, the Apple II library at ease. So what could go wrong with them?

Why was the behavior on the emulators inaccurate?

The real machine is always right. So if my program hang it and ran perfectly on the emulators it meant two things: my program was buggy and the emulators were not accurate enough to trigger the bug. Although it is written in assembly, my game does not rely on any smart trick requiring precision in timings. Even the disk access use the OS’ API.

I was in big trouble: if on the Apple II you can break a program at any time using the instruction BRK then inspect the content of the registers and memory location, debugging on it would be a very tedious task.

The monitor can help to debug, but it is limited and crude

After a few attempts, I noticed something interesting: if I launched a “New Game”, the game would inevitably hang. But I could load a saved game. More interesting: if I then rebooted the machine and launched a “New Game”, it worked!

That could only lead to one thing: I was reading an uninitialized memory location. Indeed, soft resetting the Apple II won’t wipe its memory. So a memory location was not correctly set before usage, but written with the correct value when loading the game.

So why did it work on the emulators even without loading a game? Easy: the two emulators set the emulated memory pool to zero just after allocating it. On the contrary, when turning on the real machine, its memory is random.

Conclusion

Finding the bug was now easy. I modified an AppleWin to fill the memory with random numbers just after after its allocation, which allowed to reproduce my bug. I then compared the execution flow in a “normal” AppleWin and in the “modified” one, and I quickly converged to the source of the bug.

Difference between running on a pristine AppleWin and one modified to randomize the memory as on the real machine

It was indeed a memory location that was not initialized before being used. As it is often the case, 0 was the correct initial value. That’s why my bug did not happen when running in emulators that set the whole memory to zero during their own initialization!

; global vars
    lda #0
    sta CurrentLevel
    sta NextLevel    ; <== The initialization that was missing!
    lda #FALSE
    sta ExitLevel

This small misadventure shows that is important to regularly test homebrew software on a real machine during its development. Even if emulators provide much useful debuggers, their primary goal is often to run the existing library, not to help homebrew development by being perfectly accurate to their model.

The source code of the game is available on Github, under a GPLv3 license.