I’ve got another recent data recovery job that is worth mentioning! (Quick reminder: I offer a service to read data from super-old media such as tapes and floppies) This particular client had a number of QIC-80 tapes from the mid-1990s that contained backups of a desktop workstation.
To read QIC-80 tapes, I have an ancient Colorado 250MB drive which I acquired a while ago, but here’s the problem: this drive connects to the floppy disk controller on the motherboard, which means that it must need very specific drivers or software to communicate with the drive.
In Linux, the way to communicate with such drives is using the ftape driver, which used to be included with the Linux kernel. However, ftape was removed from the kernel in 2006, citing too many bugs and too few users.
Since the tape’s label indicated that it came from a PC system, I decided to try keep it simple, and to recreate an actual DOS PC with this drive attached to the floppy controller. All I would need is to find the software that might communicate with this drive properly. After some serious dredging of old internet forums, I found a download site that contains the oldest versions of the Colorado backup software.
Setting up the boot disk
As I’ve done in the past, I used bochs to create a generously-sized disk image of 1 GB, and installed MS-DOS 6.22 onto it within the emulated environment of bochs. Then I copied over the installation files for the Colorado backup software. (I planned to do the actual installing of the Colorado software on the live PC instead of the emulated PC, since it might do some of the hardware detection during setup.)
Once the emulated disk image was complete, I wrote it onto a USB flash drive, plugged the drive into the old PC with the tape drive attached, and booted from the USB drive. (Thankfully the old PC has a very versatile motherboard that can boot from pretty much anything.)
The old PC booted successfully into MS-DOS, and I proceeded to set up the Colorado backup software. This did not present any issues, and the setup completed successfully without any non-default configuration.
Restoring
When I ran the backup software, it went through a first-time setup process where the first sign of hope appeared: the software said that the tape drive was detected!
The next step was to rebuild the backup catalog from the tape, while praying that the catalog is compatible with this version of the backup software. And what do you know – the catalog rebuilt successfully, and I could see the directory tree of the backup. The final step is to perform the actual restoring of the files, which I did directly onto the current boot disk.
There was not a single hiccup during the actual reading of the tape. I continue to be amazed by the resiliency of tape backup media, as well as the durability of the drive hardware, which still works flawlessly after 25 years.
This tape was a full backup of a PC workstation in 1996, so the final “bonus” step is to boot into the backup within an emulated environment, and see this workstation running in all its glory:
…who were born before there was a single artificial satellite in space.
…who were born before there was commercial air travel.
…who were born before plastic was invented.
…who were born before DNA was discovered.
…who were born before a single bit of digital information was stored anywhere.
We’ve made such a staggering amount of progress in a single human lifetime, and this progress continues to accelerate. This includes technologies with which we haven’t even evolved to cope, such as social media. Are we sure that we should continue progressing at this pace without taking a breath and taking the time to understand how these things are impacting our society, our psyche, and our future?
Rampant piracy is pretty much a “feature” of publishing software on the internet, and any author of a semi-popular app will be well-acquainted with it. Not a day goes by when I don’t see pirated copies of my app shared via Google Drive, Dropbox, and all kinds of shadier file sharing sites.
But now I’ve discovered an even more disturbing dimension to this seediness. The problem with Android apps (and any Java-based software, really) is that it’s very easy to reverse-engineer, even when the code is obfuscated. Given enough time and manpower, it’s possible to recreate nearly the original source code from the compiled app. It’s analogous to reconstructing a shredded piece of paper – it simply takes some time to find all the strips and glue them together.
So I recently received a communication from a Chinese user who alerted me that this is precisely what’s happening in China. My app is being deconstructed and repackaged under a different name. But worst of all, they have inserted their own payment mechanism into the app, which requires the user to submit a payment before the app can be used! According to my Chinese whistle blower, this counterfeit version of DiskDigger is being used by tech support departments that send the app to the user, make them install it, and then wait for payment before assisting the user further.
I generally turn a blind eye to most of the piracy of my apps, since it’s humanly impossible to continually track down violators, but this new kind of perversion makes me feel truly powerless. Without having any legal representation in China, there is literally nothing I can do to combat these bad actors. Perhaps it’s time to research how to establish a legal presence in China. If you have any experience with this, feel free to contact me. And in the meantime, when looking for DiskDigger, insist on the original!
Whenever I get too caught up in reading the news, or sucked into hopelessly unproductive political discussions, I like to relax with some images from the Hubble Space Telescope and similar telescopic wonders, which give us a glimpse into the wider universe outside the minuscule mote of dust on which we live.
Take, for example, the Ultra Deep Field image, which contains roughly ten thousand galaxies within an area of the sky that is about 2×2 arcminutes. Do you know how small a portion of the sky that is? If you take a 1×1 millimeter square, hold it at arm’s length, and peer through it, that’s the area of the sky captured in the Ultra Deep Field. The UDF was taken in a deliberately ordinary and uninteresting portion of the sky, which implies that for every 1x1mm portion of the sky, we could get a similarly breathtaking image of thousands of galaxies. Nearly every bright spot in the image is a galaxy, and some of the galaxies are over 12 billion years old.
How about some images that are impressive not in their depth, but in their resolution? Here is the sharpest ever image of the Andromeda galaxy (a composite of numerous other images), where we can zoom in to see literally every individual star in a galaxy outside of our own. If you look closely, you can also find globular clusters and nebulas, all in a different galaxy. There is a similarly high-resolution image of the Triangulum galaxy, as well as a super high-resolution image of the Orion nebula.
Or how about some images of gravitational lenses, where gargantuan clusters of galaxies create such strong curvature in spacetime that the light from more distant galaxies bends around them and becomes distorted, or even splits into multiple images of the same galaxies in different spots.
And of course there’s the recent image of an actual black hole (or rather an image of matter accreting around it) at the center of Messier 87, an extremely massive galaxy with a central black hole of 5 billion solar masses, making it easier to observe than the black hole in our own galaxy.
All of this provides a pleasant counterbalance to our daily political bickering, which all seems laughably “local” by contrast. To channel Christopher Hitchens for a moment, take some time to let these awe-inspiring images sink in, and then compare them to the story of Moses and his “burning bush.”
It’s not often that I come across a data recovery story in my own personal life, but recently I came across just such a story, and a rather unusual one at that.
You see, my mother-in-law has several video recordings of my wife from her middle school and high school years, which I naturally couldn’t wait to watch, much to her embarrassment. These recordings are saved on a number of DVD-R discs. I’m guessing that my mother-in-law recorded the videos on a camcorder (onto compact tapes), and then hooked up the camcorder to a DVD recorder and burned DVDs from the contents of the tapes. (In the early-ish days of DVDs, there were standalone DVD recording devices into which you could plug in a video input, and it will continuously write the video to the DVD.)
But, to my disappointment, when I inserted these discs into the DVD drive in my computer, they appeared completely blank. One after another, the same thing: the disc contains no files, and the system reports it having a capacity of 0 MB (with no errors or warnings), even though it was visually apparent by the burn marks that the discs had data on them. I tried reading them on a different computer, with the same result.
Since the problem seemed to be affecting all of the discs, we can conjecture that it was the DVD recorder’s fault, where it might have somehow recorded the data incorrectly, or failed to close the recording session, etc. But can there be a way to access the data that was written to the disc?
The standard way to get the total size of a disk (using the Windows API) is to call the DeviceIoControl function and get a DISK_GEOMETRY_EX structure that contains the dimensions of the disk. Calling this function on these discs was returning a size of just 2048 bytes, or just one sector, since optical discs usually have a sector size of 2048 bytes.
But just because the OS is telling us that the disk is a certain size doesn’t mean we can’t attempt to explicitly read beyond that limit. We can use the ReadFile function to brute-force the system into reading the disk at any location we specify. It may simply be that the driver is reporting an incorrect total size for the disk, while other areas of the disk might still be accessible.
So, I attempted to read the disc beyond the first sector. Reading the second, third, fourth, etc. sectors was returning errors, as might be expected, but I continued reading, and at around the 16000th sector, it started returning data! It’s almost as if the disc’s contents didn’t start until sector 16384. Onward from that point, the data could be read successfully all the way to the end of the disc.
Now, in order to actually recover the files present on the disc, we could potentially use DiskDigger to scan and carve any recoverable files from the raw data. But I wanted to go a step further: up until this point, DiskDigger did not support any optical disc file systems, and since I haven’t dealt with a lot of CD/DVD recovery cases, I admittedly wasn’t totally familiar with the file systems used in optical discs, which presents a perfect opportunity to learn.
The most basic and original file system used in these discs is ISO 9660, otherwise known as ECMA-119. This is a very simple file system without any special affordances like journaling, access control, etc., which is perfectly adequate for read-only media where the data is written once, and will not need to be modified again.
Later, Microsoft developed the Joliet extensions to the ISO 9660 file system, which basically added support for Unicode file names, while remaining backwards-compatible by introducing a supplementary volume descriptor. This way, systems that support only the original ISO 9660 would continue to use the original volume descriptor, and systems that support Joliet will know to look for the new volume descriptor. So basically, Joliet-formatted disks have two directory trees (one Unicode, the other non-Unicode), with the same file entries in each tree pointing to the same content on the disk.
And finally, by the time DVDs came around, the UDF file system (Universal Disk Format), also known as ECMA-167, was standardized. It’s not backwards-compatible with ISO 9660, but discs that are formatted with UDF usually also have a stub ISO 9660 volume that tells the reader to look for a UDF volume on the same disk. UDF is quite a bit more sophisticated, since it’s intended to be suitable for re-writable media, as well as multiple sessions on the same disk, but it’s still not nearly as complex as NTFS or ext4.
By the way, UDF can also be used on regular disks, not just optical disks. Here’s a little-known trick: it’s actually possible to format any disk as UDF by executing this command in an elevated command prompt: format <drive>: /fs:UDF
So, after poring over the ECMAspecifications (real page-turners, I assure you), I implemented support for these file systems in DiskDigger, as well as in my FileSystemAnalyzer tool.
When you use DiskDigger to scan a CD or DVD (or an .ISO image), it will simply dump the contents of the disk and make all the files available for you to save. The ISO 9660 and UDF file systems don’t really have a concept of “deleted” files, so DiskDigger will present all the files for recovery, even if they are still accessible by normal means. The benefit of this is that DiskDigger can now also scan the disk beyond the size reported by the OS (which was the issue I detailed above), and find these file systems in the space of the disk that is not accessible by normal means. You can do this by launching DiskDigger, going to the Advanced tab, and selecting the “Detect disk size manually” checkbox.
And in FileSystemAnalyzer, you can now examine these file systems in great detail. When you open an optical disk (or disk image), if it contains an ISO 9660 file system, it lets you examine and navigate it. If the disk contains a Joliet file system, it lets you examine it as either Joliet or ISO 9660, by letting you select which volume descriptor to use. And if the disk contains a UDF file system, it lets you examine it or the stub ISO 9660 volume that usually comes along with it.
This rounds out support for optical disk file systems in DiskDigger and FileSystemAnalyzer! It’s admittedly a bit late, and also a bit overkill, since it’s true that data recovery cases involving optical disks are few and far between, but it’s good to know that even these sorts of cases can now be handled easily and smoothly.
