I’ve got another recent data recovery job that is worth mentioning! (Quick reminder: I offer a service to read data from super-old media such as tapes and floppies) This particular client had a number of QIC-80 tapes from the mid-1990s that contained backups of a desktop workstation.
To read QIC-80 tapes, I have an ancient Colorado 250MB drive which I acquired a while ago, but here’s the problem: this drive connects to the floppy disk controller on the motherboard, which means that it must need very specific drivers or software to communicate with the drive.
In Linux, the way to communicate with such drives is using the ftape driver, which used to be included with the Linux kernel. However, ftape was removed from the kernel in 2006, citing too many bugs and too few users.
Since the tape’s label indicated that it came from a PC system, I decided to try keep it simple, and to recreate an actual DOS PC with this drive attached to the floppy controller. All I would need is to find the software that might communicate with this drive properly. After some serious dredging of old internet forums, I found a download site that contains the oldest versions of the Colorado backup software.
Setting up the boot disk
As I’ve done in the past, I used bochs to create a generously-sized disk image of 1 GB, and installed MS-DOS 6.22 onto it within the emulated environment of bochs. Then I copied over the installation files for the Colorado backup software. (I planned to do the actual installing of the Colorado software on the live PC instead of the emulated PC, since it might do some of the hardware detection during setup.)
Once the emulated disk image was complete, I wrote it onto a USB flash drive, plugged the drive into the old PC with the tape drive attached, and booted from the USB drive. (Thankfully the old PC has a very versatile motherboard that can boot from pretty much anything.)
The old PC booted successfully into MS-DOS, and I proceeded to set up the Colorado backup software. This did not present any issues, and the setup completed successfully without any non-default configuration.
Restoring
When I ran the backup software, it went through a first-time setup process where the first sign of hope appeared: the software said that the tape drive was detected!
The next step was to rebuild the backup catalog from the tape, while praying that the catalog is compatible with this version of the backup software. And what do you know – the catalog rebuilt successfully, and I could see the directory tree of the backup. The final step is to perform the actual restoring of the files, which I did directly onto the current boot disk.
There was not a single hiccup during the actual reading of the tape. I continue to be amazed by the resiliency of tape backup media, as well as the durability of the drive hardware, which still works flawlessly after 25 years.
This tape was a full backup of a PC workstation in 1996, so the final “bonus” step is to boot into the backup within an emulated environment, and see this workstation running in all its glory:
…who were born before there was a single artificial satellite in space.
…who were born before there was commercial air travel.
…who were born before plastic was invented.
…who were born before DNA was discovered.
…who were born before a single bit of digital information was stored anywhere.
We’ve made such a staggering amount of progress in a single human lifetime, and this progress continues to accelerate. This includes technologies with which we haven’t even evolved to cope, such as social media. Are we sure that we should continue progressing at this pace without taking a breath and taking the time to understand how these things are impacting our society, our psyche, and our future?
Rampant piracy is pretty much a “feature” of publishing software on the internet, and any author of a semi-popular app will be well-acquainted with it. Not a day goes by when I don’t see pirated copies of my app shared via Google Drive, Dropbox, and all kinds of shadier file sharing sites.
But now I’ve discovered an even more disturbing dimension to this seediness. The problem with Android apps (and any Java-based software, really) is that it’s very easy to reverse-engineer, even when the code is obfuscated. Given enough time and manpower, it’s possible to recreate nearly the original source code from the compiled app. It’s analogous to reconstructing a shredded piece of paper – it simply takes some time to find all the strips and glue them together.
So I recently received a communication from a Chinese user who alerted me that this is precisely what’s happening in China. My app is being deconstructed and repackaged under a different name. But worst of all, they have inserted their own payment mechanism into the app, which requires the user to submit a payment before the app can be used! According to my Chinese whistle blower, this counterfeit version of DiskDigger is being used by tech support departments that send the app to the user, make them install it, and then wait for payment before assisting the user further.
I generally turn a blind eye to most of the piracy of my apps, since it’s humanly impossible to continually track down violators, but this new kind of perversion makes me feel truly powerless. Without having any legal representation in China, there is literally nothing I can do to combat these bad actors. Perhaps it’s time to research how to establish a legal presence in China. If you have any experience with this, feel free to contact me. And in the meantime, when looking for DiskDigger, insist on the original!
It’s not often that I come across a data recovery story in my own personal life, but recently I came across just such a story, and a rather unusual one at that.
You see, my mother-in-law has several video recordings of my wife from her middle school and high school years, which I naturally couldn’t wait to watch, much to her embarrassment. These recordings are saved on a number of DVD-R discs. I’m guessing that my mother-in-law recorded the videos on a camcorder (onto compact tapes), and then hooked up the camcorder to a DVD recorder and burned DVDs from the contents of the tapes. (In the early-ish days of DVDs, there were standalone DVD recording devices into which you could plug in a video input, and it will continuously write the video to the DVD.)
But, to my disappointment, when I inserted these discs into the DVD drive in my computer, they appeared completely blank. One after another, the same thing: the disc contains no files, and the system reports it having a capacity of 0 MB (with no errors or warnings), even though it was visually apparent by the burn marks that the discs had data on them. I tried reading them on a different computer, with the same result.
Since the problem seemed to be affecting all of the discs, we can conjecture that it was the DVD recorder’s fault, where it might have somehow recorded the data incorrectly, or failed to close the recording session, etc. But can there be a way to access the data that was written to the disc?
The standard way to get the total size of a disk (using the Windows API) is to call the DeviceIoControl function and get a DISK_GEOMETRY_EX structure that contains the dimensions of the disk. Calling this function on these discs was returning a size of just 2048 bytes, or just one sector, since optical discs usually have a sector size of 2048 bytes.
But just because the OS is telling us that the disk is a certain size doesn’t mean we can’t attempt to explicitly read beyond that limit. We can use the ReadFile function to brute-force the system into reading the disk at any location we specify. It may simply be that the driver is reporting an incorrect total size for the disk, while other areas of the disk might still be accessible.
So, I attempted to read the disc beyond the first sector. Reading the second, third, fourth, etc. sectors was returning errors, as might be expected, but I continued reading, and at around the 16000th sector, it started returning data! It’s almost as if the disc’s contents didn’t start until sector 16384. Onward from that point, the data could be read successfully all the way to the end of the disc.
Now, in order to actually recover the files present on the disc, we could potentially use DiskDigger to scan and carve any recoverable files from the raw data. But I wanted to go a step further: up until this point, DiskDigger did not support any optical disc file systems, and since I haven’t dealt with a lot of CD/DVD recovery cases, I admittedly wasn’t totally familiar with the file systems used in optical discs, which presents a perfect opportunity to learn.
The most basic and original file system used in these discs is ISO 9660, otherwise known as ECMA-119. This is a very simple file system without any special affordances like journaling, access control, etc., which is perfectly adequate for read-only media where the data is written once, and will not need to be modified again.
Later, Microsoft developed the Joliet extensions to the ISO 9660 file system, which basically added support for Unicode file names, while remaining backwards-compatible by introducing a supplementary volume descriptor. This way, systems that support only the original ISO 9660 would continue to use the original volume descriptor, and systems that support Joliet will know to look for the new volume descriptor. So basically, Joliet-formatted disks have two directory trees (one Unicode, the other non-Unicode), with the same file entries in each tree pointing to the same content on the disk.
And finally, by the time DVDs came around, the UDF file system (Universal Disk Format), also known as ECMA-167, was standardized. It’s not backwards-compatible with ISO 9660, but discs that are formatted with UDF usually also have a stub ISO 9660 volume that tells the reader to look for a UDF volume on the same disk. UDF is quite a bit more sophisticated, since it’s intended to be suitable for re-writable media, as well as multiple sessions on the same disk, but it’s still not nearly as complex as NTFS or ext4.
By the way, UDF can also be used on regular disks, not just optical disks. Here’s a little-known trick: it’s actually possible to format any disk as UDF by executing this command in an elevated command prompt: format <drive>: /fs:UDF
So, after poring over the ECMAspecifications (real page-turners, I assure you), I implemented support for these file systems in DiskDigger, as well as in my FileSystemAnalyzer tool.
When you use DiskDigger to scan a CD or DVD (or an .ISO image), it will simply dump the contents of the disk and make all the files available for you to save. The ISO 9660 and UDF file systems don’t really have a concept of “deleted” files, so DiskDigger will present all the files for recovery, even if they are still accessible by normal means. The benefit of this is that DiskDigger can now also scan the disk beyond the size reported by the OS (which was the issue I detailed above), and find these file systems in the space of the disk that is not accessible by normal means. You can do this by launching DiskDigger, going to the Advanced tab, and selecting the “Detect disk size manually” checkbox.
And in FileSystemAnalyzer, you can now examine these file systems in great detail. When you open an optical disk (or disk image), if it contains an ISO 9660 file system, it lets you examine and navigate it. If the disk contains a Joliet file system, it lets you examine it as either Joliet or ISO 9660, by letting you select which volume descriptor to use. And if the disk contains a UDF file system, it lets you examine it or the stub ISO 9660 volume that usually comes along with it.
This rounds out support for optical disk file systems in DiskDigger and FileSystemAnalyzer! It’s admittedly a bit late, and also a bit overkill, since it’s true that data recovery cases involving optical disks are few and far between, but it’s good to know that even these sorts of cases can now be handled easily and smoothly.
This post is mostly tongue-in-cheek, but it’s something that I’ve been wanting to get off my chest for some time. The theory of evolution (or the fact of evolution, as Richard Dawkins refers to it) is still not as accepted as it should be, and remains “controversial” in some circles. According to a Pew Research Center study, as many as 34% of Americans do not believe that humans and other species evolved from earlier shared ancestors, and instead believe that humans have always existed in their current form, since the moment of creation.
Evolution is not a particularly difficult theory to grasp, but unfortunately its teaching is often encumbered by certain obstacles. The greatest such obstacle is religious dogma, which I don’t need to expand upon here. But there’s another obstacle that is nearly as harmful as religion towards a proper understanding of evolution: popular media. There is a great number of movies, TV shows, and video games that have “evolution” as their central theme, but do a very poor job of representing the reality of evolution, and in fact perpetuate falsehoods about evolution that might take even more effort to unlearn.
Here’s a selected list of popular media where evolution plays a role, but is presented in a frustratingly incorrect way. These are, of course, my personal gripes. Yours may be different, and if they are, I’d like to know about them. Enjoy!
X-Men
One the grossest misrepresentations of evolution occurs in the X-Men universe. Evolution is integral to the entire premise of the series: it is claimed that the X-Men have special abilities because of mutations in their genome.
The X-Men themselves believe that their mutations make them superior to ordinary humans. The ordinary humans, in turn, treat the mutants as outcasts and freaks because of, essentially, fear and jealousy. The point to be drawn here is that no single mutation per se makes an individual superior to any other. The only metric of “superiority” in evolution is reproductive success. If the X-Men series was about a mutant who could impregnate all the women in the world with a snap of his fingers, now that would be an advantageous mutation.
The sheer variety and randomness of “powers” possessed by the mutants begins to border on ridiculous. We have one mutant who can shoot energy out of his eyes, another who can control the weather, another whose skin turns into bulletproof metal, and another who can change the channel on the TV by blinking his eyes. Pray tell, what kind of mutation would enable a person to emit infrared light at the precise frequency and the right modulation to change the channel on a TV?
The powers of the X-Men are completely at odds with fundamental physical laws, including conservation of energy. But even if we dismiss the fact that there’s virtually no physical basis for any of the X-Men’s powers, there’s even less scientific basis for the idea that a genetic mutation is what causes them.
Pokémon
Alright, this is a bit of a stretch. But come on, Pokémon appeals to a substantial number of young children, and they, more than anyone, should be spared from misconceptions about evolution.
In the Pokémon world, a certain species of Pokémon can “evolve” into a different species by the mere act of gaining a sufficient amount of experience from battling other Pokémon. This perpetuates the falsehood that evolution is an instant “quantum leap” from one species to another, or that one species can instantly transform into a different, more advanced species.
Of course Pokémon are not real animals. Perhaps they are supposed to be “magical,” which might give them the ability to transform into different species. But whatever you call this transformation, don’t call it evolution.
Star Trek
As much as I like Star Trek, I’m afraid it’s one of the bigger offenders when it comes to properly depicting evolution, genetics, and DNA.
Firstly, in the Star Trek universe, it’s commonplace to see unions between two different species that produce offspring, for example a half-human half-Vulcan, or a half-Bajoran half-Cardassian, etc. The thing is, if two individuals are able to produce a child together, then they are by definition the same species. Just within the confines of the Earth, two individuals from different species cannot produce offspring, even if the species are very close in evolutionary time, e.g. humans and chimpanzees. So then, how could two species from different planets possibly be compatible?
Star Trek treats aliens from different planets as if they were different races of the same pseudo-species of “humanoids,” but this is very different from the idea of them being distinct species, with their own evolutionary lineage.
To its credit, Star Trek: The Next Generation attempts to explain the profusion of humanoid species on different planets in the episode The Chase, in which it’s discovered that life on habitable planets was inseminated billions of years ago by an “original” humanoid species that wanted to spread its likeness throughout the galaxy. This is a perfectly valid premise, but even if life was inseminated this way, the probability that the lineages on different planets would evolve into beings that look virtually identical billions of years later, and are able to interbreed, is essentially zero.
In sum
If you’re a Hollywood screen writer, or a video game designer, or a comic book author, or any other content creator, and are hoping to incorporate evolution into the premise of your production, consider presenting it as free as possible from misconceptions that might be absorbed by your audience. (Or hire me as a technical consultant!)
