Blog

ECC RAM should be a human right

I am now a staunch advocate for ECC RAM, after the events of last week. You see, over the last several weeks my main desktop workstation has been misbehaving, with occasional freezing and crashing. After some diagnostics I began to suspect a faulty RAM module, and sure enough, upon performing a quick run of memtest86, it lit up the screen with a multitude of bit flip errors, at numerous memory locations, indicating that something was seriously wrong with the RAM.

Within a day or two I scrambled to replace the RAM modules with new ones, and when this was done the problems resolved themselves and everything was stable again. However, there was another more sinister side effect that I discovered shortly afterwards: Some of my data was corrupted! That’s right, it was the worst-case scenario for RAM failure: bit flip errors that get written back to the disk. I discovered that several video files that I had been editing had corrupted bits, and were no longer usable. Fortunately I still have the original source materials for the videos which I can use to recreate the final videos. It’s an unfortunate waste of time, but it could have been a lot worse if I’d let the RAM failure go on even longer. There doesn’t seem to be any further corruption in any more of my personal data, and just to be on the safe side I performed a clean install of Windows, to ensure that no system files or program files are corrupted.

The point of the story is that the data corruption was completely preventable, if only my RAM had ECC built into it. But because it doesn’t, these kinds of bit flip events go completely undetected, and proceed to wreak havoc on the integrity of our data, right under our noses.

Memory manufacturers assure us that desktop RAM is so reliable that it doesn’t need ECC, that the probability of bit flip events is so low that it’s not worth the extra “cost” of ECC. Chip manufacturers (i.e. Intel) produce CPUs that don’t even support ECC memory. Users are expected to upgrade to server-grade components just to get access to ECC memory.

Let’s quickly review the reasons why server-class machines are deemed to be “deserving” of ECC memory, while desktop machines are not:

On a personal desktop computer, your data is stored permanently on a disk, whether it’s a spinning hard drive, SSD drive, memory card, and so on. When you want to do something with your data (e.g. write a document, edit a photo, etc), the data is loaded into RAM, and when you’re finished modifying your data, it’s written back to the disk.

On a server machine, however, the situation is different: since disk access is much slower than RAM access, the server must keep as much data as possible in RAM, so that the data is instantly available to clients who request it. This means that the data ends up sitting in RAM for extended periods of time. If the RAM were to experience bit-flip errors that went undetected, the server would serve incorrect data, or worse, would end up writing incorrect data back to the disk. Therefore, the server’s RAM has ECC, so that it will correct itself in case of an occasional bit flip.

This is oversimplifying a bit, but the difference between a server and a desktop, for this exercise, is simply the amount of time that data is made to sit in RAM. So then, are we supposed to accept that if our data doesn’t remain in RAM for very long, it doesn’t need ECC at all?!

By the way, you’d better believe that your disk(s) have all kinds of error correction schemes built into them, which work automatically and transparently. It’s completely normal for data written to a physical medium to be imperfect, and those imperfections will be corrected by the firmware of the disk.

Well guess what? RAM is a physical medium, and yet we’re simply asked to take the manufacturers’ word that our RAM is reliable enough to never need ECC for the use cases of a desktop workstation. Well, I’m here to say that these practices are reckless, and represent a ticking time bomb for anyone who uses non-ECC memory for anything nontrivial. And it seems I’m not the only one.

(discussion on HackerNews)

Share: TwitterFacebookPin ItWhatsAppLinkedInBuffer

Brain dump, December 2022

What’s happening with Twitter is rather transparent: a) Elon overpaid for Twitter, having had his bluff called to purchase it, and b) Having overpaid for Twitter, he now needs to inflate its value. And how do you inflate its value? By driving engagement. And how do you drive engagement? By performing a bunch of provocative stunts, including even more of the trolling, division, and outrage mongering that makes Twitter the cesspool that it already was.

Nothing about the “Twitter files” is interesting or noteworthy outside the context of creating a feedback loop of engagement on Twitter itself.

If there are indeed “files” that are as earth-shattering as was alleged, then why not publish the files? Why release them to just a few select journalists (Matt Taibbi, Bari Weiss, etc.), who just happen to be politically aligned with Elon? Why did these journalists publish their revelations by way of Twitter threads — possibly the clumsiest way of publishing long-form content?

The contents of the revelations themselves are amazingly banal: these are the inner workings of a social media company trying its best to do content moderation, imperfectly, as all social media companies are bound to do.

Elon is extremely good at generating hype around what he does, which is what has made him such a good businessman. With the release of the Twitter files, notice that he didn’t merely release the files; he built up to it over the course of many days, tweeting numerous times about how he’s going to release the files. He didn’t need to do that; he could have just released the files. But by building up to it, he drew more eyes to the platform, regardless of what the content of the “files” was going to be.

And what will happen with Twitter now? Aside from the easy argument that a privately-held company can literally, and by definition, never be a “public square,” what else could happen with it? Elon can twiddle whatever knobs he likes, such as imposing a pricing model for verified users, creating different levels of verification, making tweets editable, etc. But the fundamental engagement model of Twitter will remain the same: otherwise-serious people flinging verbal turds at each other, provoked and boosted by avalanches of bots and anonymous trolls.

I’ve never engaged that much on Twitter, and don’t plan to continue. I probably won’t delete my account outright, as Sam Harris was wise enough to do, since I’d like to log on occasionally and check the state of the platform, out of perverse curiosity, but I believe genuinely that Twitter continues to be a net negative for our society, our politics, and our mental health.

Share: TwitterFacebookPin ItWhatsAppLinkedInBuffer

Brain dump, October 2022

I went down a deep rabbit hole on the history of Amiga computers, and specifically the types of image encodings used in old Amiga software. As an enthusiast of digital preservation, I want to make sure that any images saved using that software can still be decoded and converted using modern-day tools.

What I found is a rich history of software developers squeezing every last bit of performance out of underpowered hardware and limited storage, using techniques that blur the line between laughably hacky and downright ingenious.

A good example is the so-called Extended HAM (HAM-E) graphics mode, which is basically designed to enable a higher color depth, i.e. more color bit planes, at the expense of horizontal resolution. However, the native format of Amiga bitmap images didn’t support this kind of color encoding in the metadata of the bitmap file, so how do we encode our color palette? The solution: put the palette information literally into the actual image!

That’s right, the color palette actually appears as several lines of seemingly “garbage” pixels at the top of the image, and a proper decoder needs to be aware of this hack. Below is an example image that is decoded incorrectly, i.e. using a decoder that reads the image literally, without taking the extra color information into account:

Incorrect

Notice in the above image that we can vaguely discern a stretched shape of a person’s face, and we can also discern a few lines of pixels at the top that don’t seem to go along with the rest of the image. Now here’s the same image decoded correctly, with the color palette lines consumed and blanked out:

Correct

But doesn’t that sound wrong? Even if a proper decoder recognizes and blanks out the garbage lines, won’t the bitmap always have unused lines at the top? Not to worry — those lines will fall into the overscan region of the monitor! (You know, because a CRT monitor has overscan margins.) Therefore the palette lines will be automatically hidden from view, unless the user explicitly adjusts the overscan on their monitor. Ah, the good old days.

There are a few modern software packages that handle these types of images and a plethora of others, including abydos, recoil, and grafx2, so the preservation of old Amiga image formats is in good shape. And in my own ImageFormats library for .NET, I implemented support for some even more obscure Amiga formats, notably ACBM (Amiga Contiguous BitMap), which seems to be unsupported by other decoders.

Share: TwitterFacebookPin ItWhatsAppLinkedInBuffer

Brain dump, September 2022

As the happy owners of several chickens, we’ve been finding that non-chicken-owners often have a comically flawed understanding of chicken anatomy and chicken reproduction. Specifically, people seem to assume that if a hen lays an egg, it implies that the egg contains an embryo, and definitely means that a chick will eventually hatch from the egg, unless the egg is taken from the hen and eaten before the embryo grows to a visible size. This assumption is maintained even when people are told that there’s no rooster in our flock, to which the response is “What does a rooster have to do with it?”

I remind my non-chicken-owning friends that a female human routinely produces eggs that fall out of her uterus. These eggs will never develop into a human embryo, unless they are fertilized by the human equivalent of a rooster.

I worked part-time while attending college (a decently mid-tier state school) in the early 2000s, and paid for the entirety of my tuition with my salary. I never took out any loans to pay for my education, and finished college without any debt. This is a choice that I made, and this choice is not for everyone. Despite this fact, I am overjoyed that many people will now have an easier time paying for college than I did, and I hold no resentment towards those people for whom the $10,000 loan forgiveness package will make a life-altering difference.

Thinking about it in the most abstract terms, the whole point of civilization is for the current generation to improve the quality of life for the next generation. Parents make sacrifices all the time to improve their children’s future, and good parents don’t resent their children just because their children’s lives are easier than theirs were.

The only minor annoyance I feel is that there’s still no attempt to fix the underlying root cause of the issue, which is twofold: 1) Colleges are experiencing unprecedented levels of administrative bloat, which is driving up the cost of education to the point where college is a luxury product that a seventeen-year-old is expected to purchase without thinking twice, and 2) We’ve created a culture which insists that going to college is the only way to succeed in life, when in fact there are a hundred different trades that are looking desperately for more people, and provide a very comfortable living and life satisfaction.

Share: TwitterFacebookPin ItWhatsAppLinkedInBuffer

A few guidelines for navigating social media

  • Virtually nothing is as bad as social media makes it seem.
  • There is almost always more context for everything shared on social media. Read the full article, watch the full video, and check other sources.
  • If you find yourself feeling outraged, someone is making money.
  • If you find yourself feeling afraid, someone is making money.
  • If a public figure tweets something so provocative that you feel compelled to reply, someone is making money.
  • Before submitting your reply, ask yourself, “Am I contributing to the good in the world?”
  • Before scrolling any further, ask yourself, “Am I enriching myself in a meaningful way?”
  • Before even opening your social media app, ask yourself, “Am I a better person by knowing the news right now, rather than next week?”
  • Remember that there are foreign governments who benefit from sowing division among us, and yes, they are on social media.
  • Remember that “influencers” on social media are acting out of self-interest, and will go to great lengths to drive engagement.
  • If you feel compelled to show solidarity with a cause, take an action that contributes to the cause, instead of adding an emoji to your bio.
  • Most people are much nicer in real life than on social media.
Share: TwitterFacebookPin ItWhatsAppLinkedInBuffer