How I learned to play The Incident and other GTROM favorites sans the cartridge

GTROMTitle

A lot of excellent new NES homebrew has been published on GTROM based cartridges over the past few years. Most other homebrew is distributed using boards compatible with various classic or existing mappers meaning if you have the know-how and the tools to dump the ROM yourself then you can play them on any number of compatible emulators. But, for GTROM emulator support is still uncommon (although improving recently) so a ROM image for a GTROM based game doesn’t do you much good unless you take more drastic measures.

A long time coming

Arching and preserving games is something that has always been important to me and GTROM presents challenges in this area. I knew this was a topic that I wanted to explore and discuss even before I became at all active in the NES community or started this blog. This blog is now well over a year old but I’ve yet to touch on this topic because of the size and scope of the composition along with fears around how it would be received by the wider community.

But now, I feel like the relevancy is slipping away. This is because there has been a recent shift in the community towards ROM releases alongside or even in place of cartridge copies and with growing emulator support I suspect digital GTROM releases probably won’t be far off either. So, in short, it’s now or never. Case-in-point, Kevin from KHAN Games announced the discontinuation of the cartridge version of the Incident the day after I started work on this article.

KHANTweet

No ROM for you

Having paid in full, I feel entirely within my rights to dump these ROMs, modify them, and play them however, whenever, and wherever I choose. However, I don’t think those rights extend in any way beyond what affects me directly. For that reason, while I would be happy to share my patches–eager even–I would not do so at this point without the permission of each individual title’s creator; so don’t ask. Beyond that, maybe in some far-flung future where the developer has long since vanished from the community I’d consider releasing the ROM simply to ensure preservation of the title.

The Incident

The Incident was one of the very first homebrew cartridges I purchased and was the first GTROM game I owned or ever attempted to patch. The desire to have a backup copy and also to be able to play this game on mobile was really the driving force behind this entire endeavor.

Developing this game’s patch took the longest and I learned the most doing it as it was my very first attempt. That probably comes as no surprise but actually this game threw up a unique challenge involving a visual glitch from a timing bug that I have never fully resolved.

TheIncidentRev1

KHAN Games 4-in-1 Retro Gamepak

KHAN Games 4-in-1 Retro Gamepak was the next one I tried after my first mostly successful patch for The Incident. Again, I wanted to patch this one because I feel like the fishing game in it is just perfect to play on-the-go. It’s relaxing and doesn’t require a lot of effort when you are some place where you’re bored but really don’t want to have to think or do anything. What could be better?

As a whole this cartridge presented several new challenges. One was that (because it uses a larger percentage of the GTROM’s flash memory than The Incident) patching it required keeping the ROM size at 512 KB which complicated the MMC1 programming needed to switch banks in the oversized ROM. It also actually makes pretty significant use of the page-flipping CHR RAM capabilities provided by GTROM. The latter created an interested visual quirk in Sneak ‘n Peak which, again, I have never fully resolved.

SNPMMC1

Spook-o-tron

Spook-o-tron

For Spook-o-tron I got lazy. I originally tried to use the MMC1 mapper that I finally used with The Incident and the 4-in-1 pak so that emulators could save the high score list. However, the MMC1 patch kept crashing because, I think, it was getting interrupted in the middle of a bank switch.

I ended up just falling back on the simpler UxROM patch like my original attempt for The Incident. This works well but the high score list doesn’t save between power cycles. It doesn’t matter to me because I suck at this game and can never beat the lowest score on the default high score list anyway. If that sounds familiar then try the Game Genie codes I posted last year.

Scramble

Scramble2

Scramble received a lukewarm reception (or so I hear) in the homebrew community but I can’t really imagine why since I really enjoyed it. This one actually plays better on the AVS for the online scoreboard support. Plus, it doesn’t play well on a phone unless you have a gamepad with physical buttons (or at least that’s true for me). So, patching this really fell more under the “just because” umbrella than the others. All the same, I’m glad I did patch this since the patched version is absolutely perfect with no outstanding issues thus giving me a warm and fuzzy feeling.

By this point I had really hit my stride and the entire process became routine. The only really new challenge was the need to reinitialize the default high score list when the ROM was run without an existing .sav file accompanying it. I took the opportunity to really improve my overall approach and the sophistication of the patch and, of course, to give credit where credit is due:

Scramble

Swords and Runes

SwordsRunes

Swords and Runes is distributed on a GTROM board but I’ve found that it’s actually an NROM game and so it didn’t really require any patching. The GTROM cartridge contains a startup stub program which first copies CHR data from PRG into CHR RAM then just switches to another PRG page which contains the entire 32 KB program and finally jumps to its entry point address. From there the game behaves exactly like any NROM game with a CHR ROM.

All that was necessary to get this run in the emulator was to dump the GTROM, copy and paste the PRG and CHR data into a ROM file with an iNES header, and finally fix the “reset” vector in the PRG ROM to point directly to the game’s entry point address. There’s nothing else to patch since the game has no bank switching logic that needs to be updated because it all fits within a single 32 KB bank plus the 8 KB of fixed CHR data.

Dumping GTROM

I needed a copy of the ROM for the original version before I could really even seriously consider whether I wanted to try and patch any of these titles or how I might go about doing that. As I am working on this article, Paul over at Infinite NES Lives has been busy adding support for writing ROMs to GTROM boards and so I can only assume dump support won’t be far behind if it isn’t there already. However, two years ago when I first started this project there was no out-of-the-box solution for dumping GTROM games.

I had no knowledge of or means to download the ROM of a GTROM or any other NES game at that point and, indeed, whether I could weighed heavily on my decision on if I really wanted to even get involved much with NES homebrew at all. I even sent a feeler out to Kevin to see if he’d be willing to share the ROM (he politely reminded me that it wouldn’t work in my emulator). Finally, after a lot of agonizing and research I concluded I could probably write an Anago WX script to extract the ROM via Kazzo (the ancestor of the INL Retro) so I ordered one and the rest, as they say, is history.

Choosing a replacement mapper

Mappers are really extensions of the NES system; they add new features that the base console doesn’t have. In principle, a given game that needs a set of specific added features can be run on any mapper which is capable of providing all of those features in some form. This is what makes mapper hacks or patches like this one possible at all. Each mapper could work differently though and provide the same basic functionality in different ways. A big part of developing a patch is just figuring out what the game was telling the old mapper to do and then changing it just enough to tell it do the same thing with the new mapper.

Despite being a simple and inexpensive mapper, GTROM actually packs a punch and incorporates at least two features which are more advanced when compared with classic NES cartridges. Those features are a 4-screen nametable and also extra on-board general purpose RAM. Fortunately for us, although maybe a disappointment for Memblers, is that none of the games I’ve patched so far have made use of these features and so they can be ignored.

The main feature used by all these games is just simple bank switching in the PRG ROM and also non-volatile storage for saving games or statistics between play sessions. They also use CHR RAM which I treated as a requirement but in retrospect it could have actually been worked around. I didn’t really fully understand any of this coming into this project though. I muddled through experimenting with several different mappers trying to get things to work perfectly the way I wanted but generally falling short of perfection.

Page rearrangement

Nearly all mappers provide bank switching in some form but it doesn’t just make any two mappers interchangeable simply because they both incorporate this feature. The reason is that there is more than one way to “map” the memory of a larger storage chip into the smaller space provided by the NES’ 16-bit addressing. In order for two mappers to be interchangeable they both must do this mapping the same way or at least in a way that can be made compatible.

GTROM bank switching works in the simplest way possible by splitting its 512 KB into sixteen individual pages of 32 KB and allowing any one to be selected at time. Perhaps unexpectedly though is that none of the games in this article actually work this way. All four use a paging scheme strongly resembling a UxROM style mapper where a single 16 KB page is fixed and always visible to the CPU while a second 16 KB bank can switch between one of multiple possible pages. But since GTROM doesn’t do this, all these games actually accomplish it by just duplicating the same information in the second half of each page to create a faux fixed bank. The original GTROM literature from Memblers describes this process as “interleaving” and he later released a general purpose tool which helps duplicate data for this purpose.

Bank Compare Diagram

One practical implication of this is that it means the raw data on the GTROM’s flash chip, and therefore in the ROM file, is arranged differently than would be expected by another otherwise compatible mapper. So before the ROM’s code can even be patched, the data itself must be rearranged to put the pages in the correct sequence and to also eliminate the duplicate data. In my original attempt I did this manually by copying each 16 KB page separately using a hex editor. Later on, I automated this processing using a build script.

Rearrange Diagram

UxROM Patch

Once I had dumped the ROM and rearranged the data the next step was to actually update the game’s code to communicate with the replacement mapper. There are at least several mappers that can arrange their banks in the way these games expect but probably UxROM is the simplest to program so it’s the one I tried first. What this involves is finding all the spots where the game is programmed to change the selected ROM page and replace the GTROM specific code with UxROM code that does the same thing.

This is actually pretty simple since the debugger feature in FCEUX makes it easy by allowing a breakpoint to be set for whenever there is a write to an address which contains the GTROM’s control register. Once the code is located then it’s just a matter of writing new code to replace it, assembling it into binary code, and then pasting it over the old code in the ROM. This is especially easy for UxROM since the way the pages are switched is by just writing a byte to a register. That’s actually very similar to how GTROM already works but UxROM is slightly more complex due to the bus conflicts it has but not by very much.

UxROM Bank Switch

This was my first attempt at a patch for The Incident and it worked flawlessly. This story would have ended right here if it wasn’t for the fact that UxROM doesn’t have any form of non-volatile storage. Saving the game progression in NVRAM is very much something that I wanted to support and so had to continue to try other mappers.

MMC1 Patch, First Attempt

While the UxROM patch plays very well the mapper doesn’t have non-volatile storage so there’s no way to implement a replacement for GTROM’s flash saving capabilities. This set me on a new search to find another mapper which supported the same bank mapping scheme plus both CHR RAM and NVRAM. MMC1 (or SxROM) supports all of these things and it seemed to be the logical and obvious choice.

The patching process is basically the same as UxROM which is just to rewrite the two places in The Incident‘s code were it tells GTROM to switch pages to instead tell MMC1 to switch pages. Sounded easy enough but MMC1 throws an added complication since it uses a serial interface. What that means is that instead of writing a full byte to the mapper’s control register, like normal, only a single bit can be written at a time (apparently this saved on manufacturing costs back in the day). The practical implication is that the new code takes more instructions which use more ROM space and more CPU time to accomplish the same thing.

MMC1 Bank Switch Rev1

For my first attempt at writing this code I did what most programmers would probably do when faced with doing the same thing over and over again: I wrote a loop. I didn’t really pay any attention to the efficiency of the code or consider how long it would take to execute. The end result of my lack of attention to optimization was to introduce a timing bug which created a highly noticeable visual issue where the screen jumps around a lot during game play.

TheIncidentRev1

The reason this occurs is because the use of the PPUADDR register has a side effect of throwing off the scrolling offset when it is used which causes it to be at the wrong position at the start of the next frame. Normally, this would get reset by some writes to the PPUSCROLL register but these writes need to occur before the end of the VBLANK period. With all the added time to perform the bank switches this simply doesn’t happen fast enough and the effect is the incorrect scrolling that we see.

MMC4 Patch

Not understanding the reason why my MMC1 patch had this visual problem, I simply said “well, that didn’t work” and continued my search for another mapper that would. By this point the list of potential options is pretty well exhausted but MMC4 (or FxROM) at least looked promising. UxROM is exclusively CHR RAM based like GTROM and MMC1 games exist with CHR RAM or CHR ROM but no CHR RAM games exist for MMC4.

I assumed that this didn’t matter and that I could still just set 8 KB of CHR RAM in the iNES header for an MMC4 ROM and have it work as expected. However, the emulator didn’t like this at all and just gave me oddly garbled displays instead. This probably happens because the emulator actually needed to implement CHR RAM on the mapper separately from CHR ROM and the former was never done since no games ever used it.

IncidentTitleScreenMMC4.gif

My suspicion is that CHR RAM was never used with this mapper because it has an elaborate bank switching mechanism that allows it to switch banks in the middle of a frame. Making practical use of this feature required ROM back in day when RAM sizes were small and expensive. In any case, I strongly suspect this would work if a RAM chip were to be put on an actual MMC4 board.

MMC1 Patch, Second Attempt

Having completely failed at a MMC4 patch, I continued to look at other mappers but didn’t find anything else I thought would work. This lead me to back to reexamine why the MMC1 patch didn’t work correctly and I began to suspect that it was a timing issue. The direct cause of which is the significant increase in the total number of CPU cycles that must be used in order to switch the selected bank page.

For this second attempt I largely started over from scratch and this time focused on getting the number of instructions executed by the bank switch subroutine down to the absolute minimum. This was accomplished using assembler macros which generated separate instructions to handle each specific bit being written. This avoided looping logic and other overhead and executes in less time but actually consumes more space in the ROM due to mostly duplicate code.

SerialWriteFast

The rewrite did the trick and eliminated the issues in virtually all cases. It wasn’t until I had completely finished the development and was actually enjoying the patched version of the ROM that I noticed the one outstanding instance. It occurs during the stage completed screen but only in the case where a solution point is being awarded. Apparently, there’s just enough extra going on there to put us over the red line.

TheIncidentSolvedCompare

By this point I had moved on to other things so wasn’t keen on going back and revisiting it and so it remains in my final version of the patch to this day. At one point I considered diving back in and seeing if I could somehow optimize or adjust the game code to fix this but an easier solution would probably just be to try MMC4 again and just switch over to CHR ROM.

Making Space

What I’ve glossed over up to this point is that, speed optimized or not, the new code needed by the MMC1 patch is a pretty large volume. The existing startup code used by the games and the bank switching subroutine also live in the fixed bank of the ROM which is limited to just 16 KB. For all the others there’s enough free space left unused by the game to fit the patch code directly into the fixed bank so it’s not terribly difficult to insert it. The Incident though has filled the fixed bank to the brim so there’s just not enough space left over.

To make space, I relocated a portion of the block of code that is executed by The Incident immediately upon power-on to a page in the swappable bank where there was a lot more unused space. My MMC1 startup code and bank switching code live in the space freed up by this. When the game starts it begins to execute my replacement startup code which does the necessary MMC1 initialization then switches to the page with the old startup code and jumps to it. The original code then runs from the new page and when it reaches the end of the relocated block there’s another jump back to the first instruction from the original code that wasn’t moved. Everything proceeds normally from there.

StartupCode

CHR ROM

One thing that I’ve tried to not change with all of these patches is the use of CHR RAM. All these games used it originally since that’s the only way that GTROM works. This is basically a “get it for free” situation because the procedure for loading data into CHR RAM is generally exactly the same regardless of the mapper used. GTROM complicates it only slightly by allowing one of two separate 8 KB pages to be selected and swapped between.

For The Incident, Spook-o-tron, and Scramble this seems perfect since they already do all the loading work and don’t make use of any page flipping. So, everything should and does work perfectly with UxROM or MMC1 without needing any modification. While this is great in theory it means that I’ve limited myself to only mappers that support CHR RAM and I’ve already documented my struggles with those specific mappers thus far.

In hindsight, I suspect greatly that a better solution might be to try other mappers, such as MMC4, that can more easily support PRG page flipping and NVRAM but require CHR ROM. This would involve creating a CHR ROM section in the ROM file and copying the character data from PRG while building the ROM file rather than at runtime. It might also require that I remove the code that does the RAM load (if it turned out to cause problems) but since none of these games do anything after the data is loaded it means that, in principle at least, CHR ROM would work just as well.

CHR RAM

Unlike the others, KHAN’s 4-in-1 Gamepak makes more significant use of CHR RAM. There’s at least one different set of CHR data per each of the games and so the information gets updated dynamically after selecting the game to play from the menu.

Additionally, Sneak ‘n Peak makes use of the page flipping feature. Since the metasprites for the player character are so large there isn’t enough space for the facing up, down, left, and right versions in a single page. So, Sneak ‘n Peak loads a different version in each page and flips between as needed when the player turns.

On GTROM, when Sneak ‘n Peak is first run the game selects the first CHR RAM page and loads the left and right sprites and then it immediately selects the second page and loads the up and down ones. From there is can instantly swap by just changing the page using the GTROM’s register.

SNPGTROM

On MMC1 there’s only a single page and I removed the code that handled setting the page but I didn’t otherwise modify the code for Sneak ‘n Peak. What happens is that it selects the page, loads the left and right graphics, then tries to set the second page, and then loads the up and down graphics. Since no page switching occurs the first upload just gets overwritten and we’re left with only the second version.

The practical impact of this is that the player simply can’t face left or right. When GTROM page flips, the facing right and facing down graphics are swapped so on MMC1 facing right is just facing down. The same thing happens for facing up and facing left.

SNPMMC1

It’s a purely visual issue and doesn’t otherwise interfere with the game in anyway. One way to fix this would be to find another mapper that supported pagable CHR RAM. Another option would be to switch to a mapper with pagable CHR ROM but that would require finding and copying all the CHR data into various CHR ROM pages and then finding and updating all the code which currently uploads CHR RAM and changing it to instead select the correct CHR ROM page. Certainly possible but I wasn’t motivated to fix this so I’ve left it as is.

Nametables

Another lucky development was that none of these games made use of GTROM’s four screen nametable mirroring. Spook-o-tron and the 4-in-1 Gamepak don’t use scrolling at all so it’s completely a non-issue. Scramble and The Incident do scroll but scroll only left-to-right meaning the target mapper must just support vertical mirroring to allow the screen to scroll horizontally.

If you don’t know, the NES has four screens worth of titles stacked into a 2×2 grid. A view the size of one screen can be set anywhere in this big square and can be moved around. Moving the view’s position over multiple frames causes different sections to be seen on the actual display and creates the scrolling effect.

Normally, the NES provides only enough RAM for 2 of these screens and so the other two are just copies of the first. The developer can choose whether the top and bottom or the left and right are “mirrors” (or copies) of one another and depending on which one is chosen the game can scroll up and down or left and right.

MMC1 Nametable

GTROM on the other hand adds extra RAM right on the cartridge so all four screens have their own set of data and can be filled with tiles independently of the other two. This kind of configuration would make fluid scrolling in any of the 8 possible directions possible but since none of these games needed that it wasn’t something I needed to worry about. I just set all of them to use vertical mirroring for horizontal scrolling in the ROM header. Had any of them used this feature it would have complicated things since I would have needed to use a mapper with memory for all four screens. This is less common and none of the three mappers I’ve tried so far support it.

GTROM Nametable

Saved Games

In the heyday of the NES, games with “save” features that stored data when the console was off used a battery-backed RAM chip to accomplish this. Other solutions, like the flash memory saving used by GTROM, either hadn’t been invented yet or were simply cost prohibitive 30 years ago. Today, emulators support this by simply saving the information in a file (usually .sav) when the emulator is closed.

All four of these games use the flash saving feature in some capacity whether it’s for saving progression or just the high-score list. Since the method of saving is completely different the GTROM saving doesn’t work on any classic mapper. Loading works because the saved data is read like any other PRG data but it’s read-only and so reverts as soon as the emulator is restarted.

Fortunately, implementing NVRAM saving is really easy. All it requires is a mapper that has some NVRAM and the procedure is literally just to read and write it just like the NES’ built-in RAM. All these games saved data in GTROM by just copying a 256-byte block of memory from RAM to the flash chip. When the game was restarted it was just read from PRG directly back into RAM. This code is easily replaced with code which just copies between the console’s RAM and the cartridge’s RAM.

SaveNVRAM.png

When the emulator starts a game with NVRAM but has no save file to read from it just initializes the save RAM to all zero bytes. The Incident and the 4-in-1 pak were happy to start from a blank save so I did nothing special other than to just copy the data in both directions.

Scramble on the other hand assumes that the first time it is run the default high score list is already saved. So, if I leave it blank then there’s no highscore list and it breaks. To fix this I used what most old games did which was to store a checksum for the data in the save and verify it upon loading. If it matches then we use the saved data. Otherwise the data is assumed to be uninitialized or corrupted and a separate subroutine is called to generate the default highscore list instead.

LoadNVRAM

Build Script

I use ASM6 to assemble all my patch code into a small binary file. Then the next step is to copy each new block of code from a section of this file to the appropriate location in the ROM file while overwriting the old code. My original approach was to perform this second step manually by copying each block one-by-one using a hex editor. The procedure is recorded in the source file simply using comments. This approach works but its tedious, error prone, and requires a lot of repeated work as the patch changes.

PatchSteps

In later patches I replaced this with a script that always started from a clean and unmodified copy of the GTROM dump. The script first copies all fixed and non-fixed pages from their original positions in the GTROM file to the new corresponding locations in the output file. Then it runs ASM6 to build the patch code from source and finally copies each block of patched code and data from the assembly output to the ROM. This creates a single build procedure that can be easily tweaked, is repeatable, and more clearly documents the steps of the entire patching process.

PatchScript.png

Conclusion

It was quite an adventure, a lot of work, a lot of fun, and a lot to learn to actually get these patches working. While I don’t feel like I can share the patches at this time I really wanted to at least share the experience with this article which was also quite a bit of work to put together. I hope you found that it flowed pretty easily since I tried to keep the sections manageable, break it up, and keep it interesting with lots of illustrations. If you managed to read this far then you have my external gratitude. Cheers!