Solution to a particular BSOD on startup on Windows 7

Posted by Fatbag on February 23, 2014
Posted in: Discussion, Uncategorized. Leave a Comment

A few weeks ago, while I was hacking around with SafeDisc copy protection on my laptop with Windows 7, after a few hours, I shut it down for the night and went to bed; the next morning I faced a constant BSOD on startup. “Oh, so Secsrv.sys decided to screw me over.” I assumed that there was code in the SafeDisc driver that was segfaulting because by running the game in this unusual way trying to patch nanomites in order to unpack it, I probably got it to screw up some configuration somewhere. Fair enough, as long as I could fix it.

The details of the BSOD were this: “Check your hard drive to make sure it is properly configured and terminated. Run CHKDSK /F to check for hard drive corruption, and then restart your computer. *** STOP: 0x0000007B (0xFFFFF880009A97E8, 0xFFFFFFFFC0000034, 0x0000000000000000, 0x0000000000000000)”.

I received this each time I tried to start up Windows. The driver’s name itself did not appear on the BSOD. Just a generic useless error message if you did not happen to know the exact cause of the issue. I tried to boot into Safe Mode, but F8 would not open the Advanced Boot Options menu because Windows wanted me to either “launch startup repair (recommended)” or “start Windows normally”, but nothing else.

So, sure. I was curious to see if Windows and its startup repair tool were smart enough to fix the issue automatically, since that would hopefully be the safest route. They weren’t. After a long, long time on “attempting repairs” (about 30 minutes), I received: “Startup Repair cannot repair this computer automatically” with “NoRootCause”. The BSOD was not fixed.

So I just booted into Linux, verified that all my files appeared to be safe, backed some valuable stuff on to my phone, and deleted Secdrv.sys from the Windows partition and rebooted. Except that didn’t fix the BSOD either. I ended up putting that file back.

So, because I happen to keep the Windows 7 ISO on my computer, I put it on my flash drive (separate from my backed up data), booted from it, and tried its repair tool. This time I got: “Checking for disk errors. This might take more than an hour to complete.” Like before, it took about 30 minutes, and it asked me to reboot. Sure, maybe SafeDisc damaged some file and Windows considers “disk errors” to be in the same class as corrupt OS data. It makes sense. When I rebooted, the BSOD was still there.

Okay, so this time, I booted back on to the flash drive, went into the repair tool again, and after it gave me NoRootCause, I tried to perform “sfc /scannow”. This time, I got: “There is a system repair pending which requires a reboot to complete. Restart Windows and try again.” What? But it’s not doing anything… After googling the issue, it appeared you could delete C:\Windows\winsxs\pending.xml to force sfc /scannow to go through… except that the command line tool could not locate the directory. So I rebooted into Windows, let it give the BSOD again, rebooted so it would ask me if I want to enter the repair tool from the hard drive again, let it try and fail to repair, and then opened the command line, and did “del X:\Windows\winsxs\pending.xml” followed by “sfc /scannow”. Guess what? I got: “Windows Resource Protection did not find any integrity violations.”

I repeated the “enter the repair tool and let it fail and restart” procedure so that F8 would give me the Advanced Boot Options menu and tried entering Safe Mode to see if at least that would work. Except it didn’t. The last driver to load before crashing was classpnp.sys. This was a Microsoft driver. The general solution on the internet appeared to be to delete the file and run the repair tool, and if that did not work, reinstall. Tried it, and it didn’t work, and I put the file back. (I did not take the advice to reinstall.) Last Known Good Configuration did not work. Kernel debugging mode did not work (or give me any new messages). Boot logging, which is supposed to log the “real” last loaded kernel module to ntbtlog.txt, also did not create the file when I went back into Linux.

So I was thinking that SafeDisc probably screwed up a single registry entry somewhere that is required from very early on in the Windows boot process, just because it could (because I was the one trying to break it anyway). If it were a corrupt or missing driver, you would think the repair tool would have fixed that, but hell, the repair tool was not saying anything useful about what it was doing. I had been googling the error codes like “0xFFFFF880009A97E8, 0xFFFFFFFFC0000034″ during all of the waiting it had me do: this just returned forum posts in which people resorted to reinstalling Windows. I was seriously not wanting to reinstall Windows because *one* configuration setting somewhere screwed up. “Repair Windows registry from Linux” did not bring up anything useful because no automated tools for repairing Windows (better than Microsoft’s repair tool can) actually exist. The advice is always just to back up and reinstall, since that’s guaranteed to work even if it wastes your time.

I read that 0x0000007B corresponded to “INACCESSIBLE_BOOT_DEVICE”, and I confirmed this was true from the bug check code reference on Microsoft’s website. They also provide a description: “This bug check indicates that the Microsoft Windows operating system has lost access to the system partition during startup.”

Then I remembered that I disabled my DVD drive in the device manager earlier when I had the virtual drive of the game disc working (using WinCDEmu). I realized (since it told me I would need to reboot before I could see the changes) that I most likely saw “disk drive” and disabled my hard drive instead by accident. Oops. But this is something that you can definitely fix with a single registry tweak from Linux, right? At this point, I was definitely sure the problem was simply that I disabled my hard drive in the device manager.

So I started off by assuming that the device manager stores “enable/disable this device” settings in the registry. But everything you would think of googling to find this information came up with nothing helpful at all. I spent 20 minutes on this. Unlike with Linux where you can ask the community, and essentially the developers themselves, where information about how your distribution enables and disables its kernel modules is stored, and you’ll eventually find a helpful response, there was not actually any way to just ask Microsoft for this information. They’re a corporation. They offer “paid support” if you sign a business contract with them, but absolutely nothing to the regular people that use it other than: “reinstall everything or go to a repair shop to have them do it”. That’s just insane. We’re talking about a single configuration setting.

Right from the start (or rather, right when I backed up my data), I knew in the end I could resort to that, and it’s really not a big deal or a huge time-waster; but if I did need to do this, I wanted to understand WHY.

So I installed Windows 7 in a virtual machine, using virt-manager with KVM/QEMU on Linux, giving it 32GB of hard drive space. I disabled all unnecessary services (especially the search indexer and task scheduler) and background processes to speed things up. I tried to run the 32-bit version of the device manager in OllyDbg (which does not offer debugging support for 64-bit processes yet, although it’s getting close to completion), except that the 32-bit version (which you can only get to run by deleting the 64-bit version of mmc.exe) does not actually disable or enable hardware when you tell it to.

Okay then. That just meant another fun coding opportunity. I wanted a tool to track the device manager’s calls to kernel32.dll, advapi32.dll, and shlwapi.dll. Since my tool could not itself create the device manager process (mmc.exe) without giving it the right arguments, and I didn’t feel like enabling the search indexer service and rebooting the VM to put the device manager shortcut back on the search results of the start menu to get the arguments, the tool would attach to an existing process rather than create a new one. My tool could either hook all functions provided by these dlls in the process’s address space or use the Microsoft debugger API, though if I still wanted the tool to attach to an existing process, it would have to hook the functions by installing hotpatches and performing 64-bit relative jumps. The debugger API would be easier, and I believe I’ve already had enough practice with hooking in my SafeDisc work.

I finished the debugger in about 2 hours (I did all my coding and API googling inside the VM), and it worked; it showed various registry functions being called when I refreshed the list of devices or went into a device’s properties, but oddly, not when I simply enabled or disabled the DVD drive (not the hard drive, since I didn’t want to break the VM installation too), which was the only thing that mattered. Instead, CreateFile was being called, first on “\\.\PIPE\srvsvc”, and then on “C:\Windows\INF\setupapi.app.log”. Sadly, if the device manager performed its configuration changes by performing system calls, I would have no practical way of capturing that. In the worst case, if the configuration was somehow not stored in the registry, I could still look at the raw data on the hard drive and look at the bytes that changed, discover which files (if any) were changing, and dump them by mounting the image. But look, a log file. I peeked inside and discovered at the bottom: “DIF_PROPERTYCHANGE – IDE\CDROMQEMU_QEMU_DVD-ROM_______________________1.2.____\5&3A2A5854&0&1.0.0″. I opened regedit and saw keys under CurrentControlSet, ControlSet001, and ControlSet002: “Enum\IDE\CDROMQEMU_QEMU_DVD-ROM_______________________1.2.____\5&3A2A5854&0&1.0.0\Properties”. Unfortunately, regedit would not even let me view it: “An error is preventing this key from being opened. Details: Access is denied.” Trying to give the “Administrators” group permission failed: “You do not have permission to view the current permission settings for Properties, but you can make permission changes.” / “Unable to set new owner on Properties. Access is denied.” / “Registry Editor could not set security in the key currently selected, or some of its subkeys.”

No problem. I set the cache for the hard drive in virt-manager to “None” and mounted the file system using $ mount -o loop,ro,sync,offset=105906176 “/var/lib/libvirt/images/Windows.img” /mnt/vm: “loop” for loopback from a hard drive image, “ro” for read-only access, “sync” for disabling cache (at least partly), and “offset” for the offset in bytes from the beginning of the file to the NTFS partition where Windows 7 was located in the hard drive image (obtained using parted). Then, I could read the registry from the command line on Linux by loading /mnt/vm/Windows/System32/config/SYSTEM in chntpw.

As it turns out, “CurrentControlSet” is not a registry key of its own, but rather a pointer to either ControlSet001 or ControlSet002, so that eliminated one out of three places to search (assuming that the enabled/disabled flag was really located here). The Properties key under both versions of the IDE\CDROMQEMU_QEMU_DVD-ROM… key had about a dozen strings listing certain information about the device (such as its full name, “QEMU QEMU DVD-ROM ATA Device”, and its driver, “cdrom.inf:cdrom_device.NTamd64:cdrom_install:6.17601.17514:gencdrom”) and a dozen pieces of small binary data about 150 bytes each. There was nothing simply standing out as the enable/disable property. It didn’t matter though. I deleted both keys from chntpw, and when I restarted the VM, I got a “found new hardware” tooltip, and when the device was added, I went to the device manager and the DVD drive was re-enabled. Phew. But I was curious to see if I could figure out exactly which subkey is responsible.

I took a snapshot of SYSTEM while the DVD drive was enabled, and then disabled the drive, waited a few seconds, and closed the device manager, and took another snapshot. Sadly, the SHA-256 sum of the files were the same, so the file did not change. Maybe it had to do with caching, or maybe Windows stores this information somewhere else, or maybe Windows only stores it in memory and you have to go to Start -> Shutdown before it’s saved. I figured that at any rate, it was probably worth being absolutely sure that the data I would end up looking at really included the change, even if it meant weeding through a lot more data in the process; so I took a snapshot of the entire C:\Windows directory (I assumed it wouldn’t make any sense for it to be located anywhere else, and yes, I checked C:\ as well) after booting the VM with the device enabled, and again after disabling the device and shutting down the VM with Start -> Shutdown, and then I ran “git init” on the first version of the folder (which took about 45 minutes), moved the .git folder to the second folder, and ran “git status > git_status.txt” (which took about 20 minutes). There were *31,946* new files in C:\Windows\winsxs, and a grand total of 4 deleted files in C:\Windows\ServiceProfiles\LocalService\AppData\Local and C:\Windows\System32\LogFiles\WMI\RtBackup\, from simply starting Windows, disabling the DVD drive, and shutting down. And I had Windows Backup, System Restore, Remote Assistance, Windows Defender, Windows Firewall, and all of this stuff disabled. Does this happen all the time or does Windows prune old stuff in that folder? I’ll assume they prune.

There were also 36 files modified, four of which were DEFAULT, SECURITY, SOFTWARE, and SYSTEM, so I decided I’d look inside SYSTEM. First, I ran VBinDiff on the old and new versions of SYSTEM. There were over 200 binary changes, and it was a little time consuming to locate the QEMU keys. chntpw lets you dump keys into a plaintext format, so I went to that. diff showed that there were 38 differences between chntpw’s dumps of the QEMU keys under ControlSet001 and ControlSet002 in the “enabled” state, exactly one difference between ControlSet001 of the enabled state and ControlSet001 of the disabled state, and zero differences between ControlSet002 of the enabled state and ControlSet002 of the disabled state.

The key that changed in ControlSet001 was this: < “ConfigFlags”=dword:00000000, > “ConfigFlags”=dword:00000001.

0 for enabled, 1 for disabled. And it was NOT under Properties; it was under  5&3a2a5854&0&1.0.0 itself. How misleading. I backed up C:\Windows\System32\config\SYSTEM on my own Windows partition, loaded it up in chntpw, and yes, my hard drive (IDE\DiskTOSHIBA_MQ01ABD075______________________AX002M__) had its ConfigFlags set to 1 in ControlSet001. I believe Last Known Good Configuration is supposed to swap the two control sets back and forth, but maybe the time I ran it was a fluke, or maybe for whatever reason it thinks ControlSet001 was in use for the last good startup and it always reverts to that one even though it’s incorrect.

After changing it back to 0, the BSOD was resolved.

All of this was caused by disabling my hard drive in the device manager. Useless error message after useless error message, in combination with grossly insufficient documentation, caused the whole process of figuring out what was wrong and what needed to be changed quite involved. And this is something that can *easily* happen to anyone.

Finished SafeDisc unpacker, and arrival of a new present

Posted by Fatbag on February 18, 2014
Posted in: Client, Discussion, Uncategorized. 2 comments

Using documentation provided by w4kfu and haggar, I decided to give a shot at making my own open source SafeDisc v3 unpacker in C, so that I could begin studying The Sims Complete Collection in OllyDbg. I finally completed this, and I think I did a decent job commenting it. (I admit that my use of memcpy is probably “ugly”, but the calls formally do what they are supposed to.)

To compile it, you need to also obtain, compile, and link against libudis86, as that’s what I chose to use for disassembling data in the .text section of the game process. Using it is straightforward: put the game disc in your disc drive, and when it has loaded up and it appears in Computer, run: unpack.exe “C:\Program Files (x86)\Maxis\Sims.exe” Sims_unpacked.exe. The original game disc is needed for obtaining the decryption key. I have only tested the unpacker on the US version of The Sims Complete Collection. The unpacked version of the exe, produced by this tool, is 2.90MB, and interestingly, even at this small size, it provides RTTI, which can be read with rtti-reader.

Of course, I could have downloaded a cracked version of The Sims 1, but I knew this was a fun opportunity that would mark my first time removing a real commercial game’s copy protection myself.

Additionally, since Sims.exe showed Edith dialogs in its .rsrc section, I thought that the complete Edith developer tool might exist in The Sims 1. After unpacking, there are a number of strings, functions, and classes that are clearly used only for Edith, but there are no Windows API calls to CreateDialogParam, GetDlgItem, or SendMessage, the only call to CreateWindowEx occurs for the game window, and (unlike EA-Land) the game is not linked to MFC. So currently, there is not enough evidence (from I have obtained in the last 2 days) to say whether or not the backend for Edith completely exists in The Sims 1, but if it exists, it will require a fair bit of reconstruction of code to interface with the Windows API, which is not something that I had to do to start Edith in EA-Land.

Here are some interesting things. SIM::Simulate (so named according to the debugging string referenced at 0x004D1918) is located at 0x0046D9D6. If you patch it to just “ret”, then the SimAntics simulator remains paused and, even in play mode, the time remains at “AM 7:00″. If you do it at a random time, the sims will stop moving but the sounds and music will continue functioning as normal. The HIT analogue of SIM::Simulate (with no obvious name attached to it) is located at 0x0057D7CD; if you patch it to just “ret”, then the HIT simulator remains paused, causing only looped sounds to continue playing, for instance, birds chirping in the background. Music in Buy and Build Mode continue to work with the HIT simulator disabled, and when you resume Live Mode, the looped sounds resume. Even with these functions patched, you can leave the lot, enter the neighborhood, enter another lot, and so on. The neighborhood screen works without the SimAntics simulator running. If you disable the HIT simulator as well, then the neighborhood screen does not play background music, instead acting like Live Mode by playing the sounds that were previously looping.

A present: “Pre-Alpha” internal tester version

Somebody who wishes to stay anonymous sent me this in the mail:

The Sims Online Pre-Alpha

The complete gallery: http://niotso.org/files/pictures/pre-alpha/

I am currently undecided if I want to share the files with the public. Consider this demo CD, which goes for about $1000 on Ebay. Since it was never going to be released to the public, and the record label was never going to make any money off of it anyway, the people who obtained it ripped it and shared all the files with the public for free. But music is different from games: since the disc in our case actually contains some of the exact same assets used for The Sims 1, it is more questionable whether or not that argument applies for this. It is not unreasonable to believe I will share these files one way or another, by snail mail, meet-and-greet, or whatever we think is “equivalent”.

Officially, this batch was a test of their manufacturing process and was distributed to paid testers at Electronic Arts in May 2002. I speculate, though, that just because it says “Pre-Alpha” does not mean that this is the buggiest thing ever produced. They could have equally likely named it “tester version”, “[Month]-[Day] version”, or “Build xxxx”—the choice of name didn’t really matter.

There are two game discs totalling to about 1GB. Currently, you need to actually run the installer for this: the installer uses InstallShield format .cab files, as this is before they switched over to the Microsoft .cab format, so 7-zip and cabextract can’t extract the files, and yet unshield also reports an error when trying to extract them.

The “version” file in the installation reads “major: 1, minor: 3, patch: 1, build: 56″, or “1.3.1.56”, and the files date between 2002-03-08 and 2002-03-09. The protocol changed between this version and the Play Test version of the game, which means I have a few more reverse engineering steps before we get past the log-in screen in this. As you would anticipate, the game comes with a fully functional copy of Edith, in TSOEdithEditorD.dll (the same as in EA-Land), as well as two never-before-seen developer tools: GZ Performance Analyzer and Production Database Editor. (The dialogs for these were quickly found with Anolis Resourcer; I am currently assuming that the code that runs them exists as well, as we saw with Edith, but I will let you know in an update.)

What it does not come with, however, is the source code to the game, or the complete Hitlab developer tool.

Additionally, I separately obtained the September 2001 issue of PCGamer magazine from eBay, which contains the oldest known preview content of The Sims Online, over a year before the game came out. Pictures of the pages are in the link, and they are moderately readable; I will have to get around to scanning the pages, if I ever buy a quality CCD scanner. I have the CD that came with the magazine as well, but sadly it does not contain anything related to The Sims Online. Regardless, you should enjoy reading this.

Select-A-Sim and Create-A-Sim cracked

Posted by Fatbag on June 10, 2013
Posted in: Client. 45 comments

A fundamental event occurred today. But it requires a backstory.

On February 24, 2013, I created a project called TSO-SE, with the following README.md:

# TSO-SE

TSO-SE ("The Sims Online Server Emulator") is not a serious project, but a curious one attempting to see just how far the game's original protocol can be cracked.

The project started with some PHP files and some documentation. I intended to upload this project to Github, but day after day inside OllyDbg I found myself repeating the phrase “Let me just finish figuring this thing out first so I can put something nice in for the initial commit”. On March 20, 2013, I successfully broke past the login screen on the official Maxis client for the TSO New & Improved trial (version 1.1097.1.0), into Select-A-Sim, and documented the results on the wiki. The process involves uploading a few PHP files on an HTTPS server, or patching the client to connect over HTTP rather than HTTPS, and specifying its hostname in the client’s ini files. The same server can allow any version of The Sims Online into Select-A-Sim, from Play Test (1.3.2.89) to the final version of EA-Land (2.1667.5.0). For the rest of the game, a new server is required, which must use the ARIES/Voltron protocol over TLS, which must be reverse engineered without the help of any decrypted packet logs.

Today, I have broken into Create-A-Sim. The implications are tremendous. I can make a pixel-perfect reconstruction of the original menus for Niotso Tech Preview 1. And I plan to finish it by the end of the summer.

The Sims Online Select-A-Sim

Select-A-Sim

The Sims Online Create-A-Sim

Create-A-Sim

Getting into Create-A-Sim was significantly more involved than Select-A-Sim. Sure, Select-A-Sim’s protocol was completely plaintext, allowing me to place a breakpoint on the cRZString::Compare() function and write down every XML tag it checked for. But more importantly, the networking code for Select-A-Sim was completely serial, and all the logic started and completed inside the same function. What has held back Create-A-Sim for so long is two issues, the first of which is fully resolved, and the second of which is ongoing.

The first issue was finding the logic that handles the server’s response. Select-A-Sim makes an HTTP request using wininet.dll’s HttpOpenRequest() function and then immediately parses the response. Aries, however, runs on its own thread. When data is discovered using Winsock, it’s decrypted into a buffer using OpenSSL, and the 12-byte header is run through an initial validation. After that, some flags are set somewhere, and the TSOClient thread eventually gets to handling the data. These unknown flags have to indicate that the data has a complete Voltron packet before the TSOClient thread handles the data, meaning hardware breakpoints did nothing until I got the header right, so it’s a chicken-and-egg problem unless you can either find the flags or get it right: in my case, the header was just three fields, so it wasn’t difficult to guess. Afterwards, back over in the TSOClient thread, the code splits into hundreds of branches: unlike Select-A-Sim, there is no single right or wrong response by the server at any given time. How do you guess which (one or more, in cascade) of these corresponds to the Server Hello message (or whatever the hell it’s waiting for)? This problem can be reworded like this: what input state will cause this output state? In theory, as cryptographic hash functions exploit, this problem may be impractical to solve. However, as the “general” logic has at least been found, the initial issue in question can be more or less checked off.

The second issue is actually equivalent to the sub-problem of the first. After finding the function that updates the progress bar (which was not easy, because the results are not drawn until far later), I discovered that I could enter Create-A-Sim by forging the HandleConnectionEstablished message ID (0x3BF7000A) onto the cTSOClientLoginRegulator message queue in place of HandleConnectionFailed (0xFBF7001B), which was already an incredible find, but I wanted more than that: I wanted to know which packets cause the game to make this step on its own. The one function in TSONetServiceSimClientD.dll that posts 0x3BF7000A (located at 0x10011320) does not have any static call references; that is, it’s looked up from a v-table. The function appears in three v-tables at the same offset of +0x74. The regular expression “call[ ]*DWORD PTR \[.*\+0x74\]” yielded 30 matches, and after removing those that supply the wrong number of arguments or an obvious non-pointer as arg1, that still left me with 22 potential candidates. And if the code uses the lea instruction or anything else to look up the function from the v-table, all 22 could be false positives. Resolving references to a function that are computed on the spot like this, in theory, reduces to the halting problem. And this could just keep going back, requiring more and more levels of analysis. What connects the path from state A to state B? This is the point where I decided to give up going backwards, and try to attack this from the other direction.

The run-time type information from EA-Land, giving a name to each Voltron packet type, and guesswork are the only things that saved me here. It turned out that sending a HostOnlinePDU was the answer to getting into Create-A-Sim.

How to do it yourself

  • Make a backup of sys/gameentry.ini. Then open it with a text editor and set the “Server” field to niotso.org. If your version of TSO also has a gamedata/sys/ folder, do the same thing with the copy of gameentry.ini in that folder.
  • Make a backup of sys/cityselector.ini. Then open it with a text editor and set the “ServerName” field to niotso.org. Keep “ServerPort” at 80. If your version of TSO also has a gamedata/sys/ folder, do the same thing with the copy of cityselector.ini in that folder.

Since I’d rather not pay for HTTPS hosting, you’ll have to make a binary patch to 3 DLLs using a hex editor (such as wxHexEditor, Ghex, or Frhed) to connect over HTTP. (If you decide to connect to a different server that supports HTTPS, you can skip these modifications.)

  • Make a backup of authlogin.dll. Then open it with a hex editor. Replace the 5 bytes “68 00 30 C0 84″ (in version 1.1097.1.0 these appear at offset 0x8325) with “68 00 00 40 84″. Also, replace the 5 bytes “68 BB 01 00 00″ (in version 1.1097.1.0 these appear at offset 0x82AC) with “68 50 00 00 00″.
  • Make a backup of TSOServiceClientD.dll. Then open it with a hex editor. Replace the 8 bytes “68 74 74 70 73 3A 2F 2F” (in version 1.1097.1.0 these appear at offset 0x923C0) with “68 74 74 70 3A 2F 2F 00″.
  • Make a backup of InternetServiceD.dll. Then open it with a hex editor. Replace the 7 bytes “74 07 68 BB 01 00 00″ (in version 1.1097.1.0 these appear at offset 0x17B2) with “EB 07 68 BB 01 00 00″. Not all versions of the game contain this code (play test and EA-Land do not), so for those versions, this file should remain untouched.

The above steps have been tested to work on all five versions of the game in our possession. After these modifications, if you also wish to enter Create-A-Sim, download the TSO-SE cityserver and run it in the background. Because the city server listens on a port, Windows Firewall (if you’ve enabled it) will ask you if you want to allow it to do so; click allow.

After you have done this, launch the game through TSOClient.exe (not the updater utility) and log in with “asdf” as your username and “hjkl” as your password. When you wish to create a Sim, click the button, and when you get to the progress bar, give the cityserver the command: send hostonlinepdu.dat . If you’re curious, communications will be logged in log.dat, so take a look.

The userdata/localavatarcache folder is still being studied. As far as I can tell, the game does not even ask the server for the up-to-date avatar data before you connect to the city. Without valid data in the localavatarcache folder to begin with, you will see the “proxy” Sim shown in this screenshot. This can be solved by placing this data in your localavatarcache folder alongside the “dont_delete.txt” file. (The avatar ID I’m having the City Selection server assign is 1337, which explains the naming.)

Where I’m going next

I’ve just been accepted into electrical engineering at UT Austin. Additionally this summer, I’m starting a basic ray tracer project, going through a digital signal processing textbook, and getting my license.

Edith cracked

Posted by Fatbag on December 23, 2012
Posted in: Codebase, Formats. 25 comments
Edith for The Sims Online

Here’s the box and arrow layout for the “Concessione DiPizza” Professional Kitchen. “Good job! A successful pizza!”

Pictured is Edith, the development tool for The Sims long aspired since the game’s release in 2000 and *accidentally* distributed as a DLL in the final version of EA-Land. I reverse engineered the controlling object’s interface (cEdithEditor) about a week after producing the tool that dumps the MSVC RTTI. Since Edith is built to work alongside an ingame lot, it requires access to the game’s cTSOFramework and the simulation state; these and other resources in the game environment can only be accessed if Edith is launched through a specially patched version of the game exe, rather than through a 2kB standalone injector I initially had plans for.

So after patching the exe, in place of the EAL splash screen and game window you’re presented with Edith. My only tool of the trade was OllyDbg… and a plugin I wrote for it, to label all of the MFC80.dll functions using its corresponding Visual Studio PDB file. The patch will likely be revised multiple times as the need arises. The patch can be downloaded here and applied to the final version of EA-Land (2.1667.5.0) using this tool.

As the developers programmed every object in the game with the sole use of this tool, Edith is the single most useful resource for helping us learn SimAntics. Edith was not officially released outside of Maxis but was demonstrated in short videos by Don Hopkins and documented for a college game design course. These resources will help us grasp the language much, much quicker.

So what else has been accomplished in the half year since the last status update?

  • Niotso switched to git and is now hosted on Github. Binaries and third-party libraries were purged from the history and are instead now uploaded to http://niotso.org/pub/. The Compile Guide has been revised with the necessary steps to obtain the files from their distinct locations.
  • Niotso Server was set up with a basic Linux daemon, and a mock RFC has been set up that will one day describe the protocol for Alpha and above.
  • The design for the Translate Tool has been conceived, but currently we’ve been debating whether it should be done with GTK+, Qt, wxWidgets, phoenix, Python, or a combination of one of the above and the Windows API.
  • The Current Tasks page has been set up. This page describes in detail the toil that must be offered to complete each task relevant to the current development phase.

My time for this project has without a doubt become shy. But the time-invariantly welcoming nature of this project allows me to say the sometimes dubious message: I’m still working.

HIT discoveries

Posted by Fatbag on July 28, 2012
Posted in: Formats. 73 comments

Yesterday marks an interesting day: the HIT disassembler has just been completed and can correctly disassemble every HIT file in both games, The Sims 1 and The Sims Online. And interestingly, 47 of the 96 HIT instructions are unused, entailing us with a lot less research.

Here is the disassembly for the “sims_loadloop” function, exported in newmain.hit:

BINARY
[
sims_loadloop
        loadb vol #0
        loadb v1 #4
        loadb v2 #2
        loadl v3 #1020
        note_on v1
        set_loop
        wait v2
        add vol v1
        cmp vol v3
        ifge #_sims_loadloop
        end
_sims_loadloop
        wait_samp
        end
]

The Event ID for this “track” (as they’re called), sims_loadloop, is 0xC7D88A0E. The Sound IDs for sims_loadloop and _sims_loadloop are 0x000001E3 and -1E4, which refer to two Hitlists; these supply lists of sounds which the VM should select at random. The Patch table describes attributes for each sound, like whether it’s looped by default or preloaded. Still, the context of this function is unclear, as I haven’t located the real load loop’s sound ID of 00004f85 anywhere yet.

I made a syntax highlighter for this code for Notepad++, which will help make figuring out the instructions more pleasant, because as it turns out, we need HIT for the game menus, making it another task for Tech Preview 1.