Euler walks

This is the first 10000 digits of e (the base of the natural logarithm), as interpreted by a spiral walk determined by each successive digit:

Zoom:

And here is a similar interpretation for γ (the Euler-Mascheroni constant):

Zoom:

Pi walk

This is the first 10000 digits of π, as interpreted by a spiral walk, with each step of the walk determined by each digit. In other words, if the first digits are “3.1415…” then we walk up 3 pixels, then left 1 pixel, then down 4 pixels, then right 1 pixel, then up 5 pixels, and so on, while painting each step of the walk with a different random color.

Zoom:

Ulam’s spiral in your browser

My day-to-day work is focused mostly on Android and Windows development, so I often find myself a bit disconnected from web development. I thought I’d go through a few random exercises in JavaScript, and simultaneously bring some of my oldie-but-goodie projects “up to date,” as it were. A long time ago I made a Windows application that displays the Ulam prime number spiral, but there’s no reason it can’t be done in the browser today, so here we go:

Zoom:
Highlight twin primes
Highlight Mersenne primes

The above picture is dynamically generated in your browser. Go ahead and interact with it: you can use your mouse scroll wheel to zoom in and out, and use the checkboxes to highlight certain special types of primes.

Send me all your floppies! They nourish me.

Recently I had another data recovery case that involved a comically large number of floppy disks, as in… more than five hundred (split evenly between 3.5” and 5.25” disks). We’re talking several large USPS boxes packed to the brim with floppies.

Of the numerous 3.5” floppies, only about 10% had one or more bad sectors, and none of them were completely unreadable.  The same was true for the 5.25” floppies, even though some of them were physically bent or warped, to the point where I had to cut them open and transplant the disk itself into a new container.  Some of the oldest files on these disks dated all the way back to 1986!

The recovery was performed using two older PCs, each of which have both 3.5” and 5.25” internal floppy drives, allowing the reading to be done somewhat in parallel.

There are actually plenty of cheapo floppy drives that connect over USB that can be purchased even now for as little as $15, but these drives are not, I repeat not suitable for recovering data from actual old floppy disks.  They must be read by a proper original floppy drive, preferably from the same era as the disks themselves.

Anyway, when floppy disks were in widespread use in the 1980s and 1990s, they weren’t really intended or marketed as a long-term storage solution, but they’re proving to be quite resilient as time goes by.  I’m not nearly as optimistic that today’s USB flash drives or SD cards will be readable in 30 years.

To be fair, these old disks have a much lower data density than modern storage media, so it makes sense that they would be more resilient to wear and tear. But still, it’s impressive that even what seems like mediocre-quality floppy disks still hold up to this day.

Despite these excellent outcomes, this still underscores how important it is to recover this data now, rather than waiting any longer and risking these disks developing any more bad sectors. So, let this be a call to action: if you have any old floppies lying around (or old tapes, Zip disks, Jaz disks, or anything else!), contact me for details on how to send them over, and I’ll recover the data from them for a fraction of the cost of other companies.

Reverse-engineering the QICStream tape backup format

TLDR: I developed an open-source tool to read tape backup images that were made using the QICStream tool, and extract the original files from them.

During a recent data recovery contract, I needed to recover files from some old QIC tapes. However, after reading the raw data from the tapes, I couldn’t recognize the format in which the backup was encoded, and none of the usual software I use to read the backups seemed to be compatible with it.

After briefly examining the backup in a hex editor, it was evident that the backup was fortunately not compressed or encrypted, and there were signs that the backup was made using a tool called QICStream.  There doesn’t seem to be any documentation regarding this utility (or the format of the backup it saves) on the web. It’s easy enough to find the tool itself on ancient DOS download sites, and it may have been an interesting project to create an emulated DOS environment where the QICStream tool reads the backup from an emulated tape media, but it turned out to be much easier to reverse-engineer the backup structure and decode the files from the actual raw data.

The binary format of the backup is very simple, once you realize one important thing:  every block of 0x8000 bytes ends with 0x402 bytes of extra data (that’s right, 0x402 bytes, not 0x400). In other words, for every successive block of 0x8000 bytes, only the first 0x7BFE bytes are useful data, and the last 0x402 bytes are some kind of additional data, possibly for parity checking or some other form of error-correcting logic. (I did not reverse-engineer the true purpose of these bytes; they did not turn out to be important in the end.)

Other than that, the format is very straightforward, and basically consists of a sequence of files and directories arranged one after the other, with a short header in front of each file, and “control codes” that determine whether to descend into a subdirectory or to navigate back out of it.

Anyway, I put all of these findings into a small open-source utility that we can now use to extract the original files from QICStream backups. Feel free to look through my code for additional details of the structure of the file headers, and how the large-scale structure of the backup is handled.