Blog

Brain dump, December 2022

What’s happening with Twitter is rather transparent: a) Elon overpaid for Twitter, having had his bluff called to purchase it, and b) Having overpaid for Twitter, he now needs to inflate its value. And how do you inflate its value? By driving engagement. And how do you drive engagement? By performing a bunch of provocative stunts, including even more of the trolling, division, and outrage mongering that makes Twitter the cesspool that it already was.

Nothing about the “Twitter files” is interesting or noteworthy outside the context of creating a feedback loop of engagement on Twitter itself.

If there are indeed “files” that are as earth-shattering as was alleged, then why not publish the files? Why release them to just a few select journalists (Matt Taibbi, Bari Weiss, etc.), who just happen to be politically aligned with Elon? Why did these journalists publish their revelations by way of Twitter threads — possibly the clumsiest way of publishing long-form content?

The contents of the revelations themselves are amazingly banal: these are the inner workings of a social media company trying its best to do content moderation, imperfectly, as all social media companies are bound to do.

Elon is extremely good at generating hype around what he does, which is what has made him such a good businessman. With the release of the Twitter files, notice that he didn’t merely release the files; he built up to it over the course of many days, tweeting numerous times about how he’s going to release the files. He didn’t need to do that; he could have just released the files. But by building up to it, he drew more eyes to the platform, regardless of what the content of the “files” was going to be.

And what will happen with Twitter now? Aside from the easy argument that a privately-held company can literally, and by definition, never be a “public square,” what else could happen with it? Elon can twiddle whatever knobs he likes, such as imposing a pricing model for verified users, creating different levels of verification, making tweets editable, etc. But the fundamental engagement model of Twitter will remain the same: otherwise-serious people flinging verbal turds at each other, provoked and boosted by avalanches of bots and anonymous trolls.

I’ve never engaged that much on Twitter, and don’t plan to continue. I probably won’t delete my account outright, as Sam Harris was wise enough to do, since I’d like to log on occasionally and check the state of the platform, out of perverse curiosity, but I believe genuinely that Twitter continues to be a net negative for our society, our politics, and our mental health.

Brain dump, October 2022

I went down a deep rabbit hole on the history of Amiga computers, and specifically the types of image encodings used in old Amiga software. As an enthusiast of digital preservation, I want to make sure that any images saved using that software can still be decoded and converted using modern-day tools.

What I found is a rich history of software developers squeezing every last bit of performance out of underpowered hardware and limited storage, using techniques that blur the line between laughably hacky and downright ingenious.

A good example is the so-called Extended HAM (HAM-E) graphics mode, which is basically designed to enable a higher color depth, i.e. more color bit planes, at the expense of horizontal resolution. However, the native format of Amiga bitmap images didn’t support this kind of color encoding in the metadata of the bitmap file, so how do we encode our color palette? The solution: put the palette information literally into the actual image!

That’s right, the color palette actually appears as several lines of seemingly “garbage” pixels at the top of the image, and a proper decoder needs to be aware of this hack. Below is an example image that is decoded incorrectly, i.e. using a decoder that reads the image literally, without taking the extra color information into account:

Incorrect

Notice in the above image that we can vaguely discern a stretched shape of a person’s face, and we can also discern a few lines of pixels at the top that don’t seem to go along with the rest of the image. Now here’s the same image decoded correctly, with the color palette lines consumed and blanked out:

Correct

But doesn’t that sound wrong? Even if a proper decoder recognizes and blanks out the garbage lines, won’t the bitmap always have unused lines at the top? Not to worry — those lines will fall into the overscan region of the monitor! (You know, because a CRT monitor has overscan margins.) Therefore the palette lines will be automatically hidden from view, unless the user explicitly adjusts the overscan on their monitor. Ah, the good old days.

There are a few modern software packages that handle these types of images and a plethora of others, including abydos, recoil, and grafx2, so the preservation of old Amiga image formats is in good shape. And in my own ImageFormats library for .NET, I implemented support for some even more obscure Amiga formats, notably ACBM (Amiga Contiguous BitMap), which seems to be unsupported by other decoders.

Brain dump, September 2022

As the happy owners of several chickens, we’ve been finding that non-chicken-owners often have a comically flawed understanding of chicken anatomy and chicken reproduction. Specifically, people seem to assume that if a hen lays an egg, it implies that the egg contains an embryo, and definitely means that a chick will eventually hatch from the egg, unless the egg is taken from the hen and eaten before the embryo grows to a visible size. This assumption is maintained even when people are told that there’s no rooster in our flock, to which the response is “What does a rooster have to do with it?”

I remind my non-chicken-owning friends that a female human routinely produces eggs that fall out of her uterus. These eggs will never develop into a human embryo, unless they are fertilized by the human equivalent of a rooster.


I worked part-time while attending college (a decently mid-tier state school) in the early 2000s, and paid for the entirety of my tuition with my salary. I never took out any loans to pay for my education, and finished college without any debt. This is a choice that I made, and this choice is not for everyone. Despite this fact, I am overjoyed that many people will now have an easier time paying for college than I did, and I hold no resentment towards those people for whom the $10,000 loan forgiveness package will make a life-altering difference.

Thinking about it in the most abstract terms, the whole point of civilization is for the current generation to improve the quality of life for the next generation. Parents make sacrifices all the time to improve their children’s future, and good parents don’t resent their children just because their children’s lives are easier than theirs were.

The only minor annoyance I feel is that there’s still no attempt to fix the underlying root cause of the issue, which is twofold: 1) Colleges are experiencing unprecedented levels of administrative bloat, which is driving up the cost of education to the point where college is a luxury product that a seventeen-year-old is expected to purchase without thinking twice, and 2) We’ve created a culture which insists that going to college is the only way to succeed in life, when in fact there are a hundred different trades that are looking desperately for more people, and provide a very comfortable living and life satisfaction.

A few guidelines for navigating social media

  • Virtually nothing is as bad as social media makes it seem.
  • There is almost always more context for everything shared on social media. Read the full article, watch the full video, and check other sources.
  • If you find yourself feeling outraged, someone is making money.
  • If you find yourself feeling afraid, someone is making money.
  • If a public figure tweets something so provocative that you feel compelled to reply, someone is making money.
  • Before submitting your reply, ask yourself, “Am I contributing to the good in the world?”
  • Before scrolling any further, ask yourself, “Am I enriching myself in a meaningful way?”
  • Before even opening your social media app, ask yourself, “Am I a better person by knowing the news right now, rather than next week?”
  • Remember that there are foreign governments who benefit from sowing division among us, and yes, they are on social media.
  • Remember that “influencers” on social media are acting out of self-interest, and will go to great lengths to drive engagement.
  • If you feel compelled to show solidarity with a cause, take an action that contributes to the cause, instead of adding an emoji to your bio.
  • Most people are much nicer in real life than on social media.

Brain dump, July 2022

(Reminder: I perform a service of recovering data from old media such as magnetic tapes, floppies, weird flash memory formats, etc. Get in touch with anything you’d like to recover.)

Performed a very interesting data recovery case:
I received a number of QIC-150 tapes, and one QIC-525 tape, from a client in Tasmania (!) who made backups onto these tapes from an IBM AS/400 system in 1991. This presented a couple of challenges:

  • The tapes were written with variable-length block sizes, so we need to configure the tape driver accordingly by running mt -f /dev/nst0 setblk 0.
  • And then, when using dd to dump the tape contents, specify a sufficiently large block size to read the largest of the variable block sizes, which turned out to be 32760, so the command was dd … bs=32k.
  • After the data was dumped successfully, I determined that the data was saved using the SAVLIB command on the AS/400, which serializes the data onto the tape in a proprietary format. Unfortunately I couldn’t find a lot of public documentation on this format, so I had to do a bit of reverse-engineering. My client specifically wanted to locate some COBOL source files in these archives, which turned out to be easy to pinpoint and extract. Most of the COBOL code was uncompressed, but some of the files were compressed using SCB (string control byte) encoding, which is basically a form of simple run-length compression.
  • Finally, since the data came from an AS/400 system, the text contents of the files were EBCDIC-encoded, which required a conversion from EBCDIC to ASCII.

One of my first professional programming contracts was done in 1997, while I was still in high school. I built an MS-DOS application that communicated over a serial port with an industrial iron powder machine (i.e. a large apparatus that melts scrap iron and turns it into powder, to be recycled for other uses). This program would display real-time telemetry from the machine, and then output it to a daily log file. It was a very simple and rudimentary program, but it worked well enough that the company continued to use it for years. It was also enough to land me a full-time job at the same company, but that’s a subject for a longer post.

Although I still have the original executable file that I provided, the unfortunate thing is that I’ve lost the source code for this program. The only thing about it that I remember is that it was written in C, and compiled with Borland C++ v3. Therefore I’ve had a mini-quest in the back of my mind to either find the lost source code, or decompile the executable file and reconstruct the source code.

Recently I’ve come the closest I’ve gotten so far to reconstructing the code, all thanks to the excellent Reko decompiler. I opened the executable in Reko; it detected the Borland C v3 runtime effortlessly; it showed a list of function calls (which I recognized! finally!), and I was up and running navigating the disassembled functions:

The Reko decompiler working with my MS-DOS app.

The hurdle now will be to rewrite the source code based on the disassembly, so that it compiles as closely as possible to the original executable. There’s still a good amount of work left to do, but it’s now much more manageable because of Reko.

In case anyone’s interested, here is the program actually running in DosBox, albeit stuck in an “error” state because it’s not receiving any data from the aforementioned iron powder machine:

My DOS app from 1997 running in DosBox.