Minimalist programming, Android edition

Suppose you open Android Studio and create a new project from a template, say, a blank Activity with a button. Then build the project, and look at the APK file that is produced. You’ll notice that the APK is over 3 MB in size.  These days we don’t bat an eye at these kinds of numbers, and indeed this size is pretty modest in the grand scheme of today’s software ecosystem. However, objectively speaking, 3 MB is a lot!  Let’s take a deep dive into what these 3 MB actually consist of, and see how much we can reduce that size while maintaining the same functionality.

The app I’ve built for this exercise is slightly more complex than “hello world”; it’s an app that could actually be minimally useful: a simple tip calculator.

It’s literally a single input field (an EditText component) where the user enters a number, followed by a few lines of text that tell the user different percentages of that number — 15%, 18%, and 20%, which are the most common tipping percentages in the U.S.

Once again, with the default project settings generated by Android Studio, this app comes out to about 3.2 MB when it’s built. Let’s examine the generated APK file and see what’s taking up all that space:

Right away we can see that the heaviest dependency by far is the AndroidX library, followed by the Kotlin standard library and the Material library (under the com.google package).  In fact the code that actually belongs to our package (com.dmitrybrant.tipcalc) is a mere 76 KB, dwarfed by the library dependencies that it’s referencing.

To be fair it’s possible to reduce the size of the APK by a good amount by using the minifyEnabled directive, which is not enabled by default. In fact it would probably optimize away most of the “kotlin” dependency and much of the “androidx” dependency.  However, even with minifyEnabled our APK size would still be on the order of megabytes. For the purpose of this exercise I left minifyEnabled off, so that we can see exactly which packages are contributing to the code sizes in our APK.

In any case, let’s start whittling away at this extra weight, and see how lean we can get.

The Kotlin tax

As we can see, merely using Kotlin in our app causes the Kotlin standard library to get bundled into the APK. If we don’t want this library to get bundled, we must no longer use Kotlin. (Although I’ll repeat that if we use minifyEnabled, then Kotlin would pretty much be optimized away, so this is more of an observation than a “tax.”)

After converting the code to plain Java and rebuilding the app, our APK is now 2.7 MB:

That’s a little better! But now can we remove the bulkiest dependency, namely the androidx library?

The AndroidX tax

AndroidX is a fabulous library that ensures your app will run consistently on a huge range of different devices (but not all of them!) and different versions of the Android OS. It makes perfect sense that AndroidX is used by default for new projects, and I’m not saying that you should reject it when building your next app. Buuuut… could we actually get away with not using it? How would our app look and run without it?  And would our app still run on the same range of devices?

Getting rid of AndroidX means that our app will rely solely on the SDK libraries that are part of the operating system on the user’s device itself.  To get rid of AndroidX in our project, we need to do the following:

  • Our Activity can no longer inherit from AppCompatActivity, and will now simply inherit from the standard Activity class from the SDK.
  • We can no longer use fancy things like ConstraintLayout, and will be limited to using basic components like LinearLayout.
  • Our theme definitions can no longer inherit from predefined Material themes. We will need to apply any color and style overrides ourselves.

After these modifications are all done, here’s what our APK looks like:

That’s right, you’re not dreaming, the app is now 88 KB. That’s kilobytes!  Now we’re getting somewhere. And if we look closely at those numbers, we see that the bulk of the size is now taken up by the resources that are bundled in the app. What are those resources, you ask?

Launcher icons

By default Android Studio generates a launcher icon for our app that takes several forms: a mipmap resource, which is a series of PNG files at different scales, which will be chosen by the launcher to match the pixel density and resolution of the device, and also a vector resource that will be used instead of the mipmap on newer devices (Android 8 and higher).

This is all very useful stuff if you need your launcher icon to appear pixel-perfect across all devices. But since our goal is minimalism, we can dispense with all of these things, and instead use just a single PNG file as our launcher icon. I created a 32×32 icon and saved it as a 4-bit PNG file, making it take up a total of 236 bytes.  It doesn’t look perfect, but it gets the point across:

So, after getting rid of all that extraneous baggage, how are we looking now?

We’ve arrived at 10.5 KB!  This is more like it.  It may be possible to squeeze it down even further, but that would necessitate doing even more hacky and inconvenient things, such as removing all XML resources and creating layouts programmatically in our code. While I’m going for minimalism, I do still want the app to be straightforward to develop further, so I’m happy to make this a good stopping point.

This is definitely closer to the size that I would “expect” a tip calculator app for a mobile device to be.  Speaking of mobile devices, which devices will this app be able to run on?

Compatibility

By default Android Studio sets our minimum SDK to 21, making our app compatible with Lollipop and above. There are plenty of good reasons to set your minimum SDK to 21, but now that we’ve removed our dependency on AndroidX, as well as our dependency on vector graphics, there’s nothing stopping us from reducing our minimum SDK even lower.  How much lower? How about… 1? That’s right, we can set our minimum SDK version to 1.  This would make our app compatible with literally every Android device ever made.

I don’t own any devices that actually run Android 1.0, but here is my Tip Calculator app running on the oldest device I own, a Samsung Galaxy Ace from 2011, running Android 2.3 (API 9):

And here is the same app running on my current personal phone in 2021, a Google Pixel 3 XL running Android 11 (API 30):

Takeaways

Aside from being an interesting random exercise, there’s a point I hope to convey here:

As time goes by, software seems to be getting more and more bloated. I believe this might be because developers aren’t always cognizant of the cost of the dependencies they’re using in their projects, whether it’s third-party libraries that provide some kind of convenience over standard functionality, or even the standard libraries of their chosen programming environment that the developer has gotten used to relying upon.

As with anything in life, there should be a balance here — a balance between convenience offered by libraries that might add bloat, and lower-level optimization and active reduction of bloat.  However, it feels like this balance is currently not in a healthy place.  The overwhelming emphasis seems to be on convenience and abstraction ad infinitum, and virtually no emphasis on stepping back and taking account of the costs that these conveniences incur.

Android is far from the worst offender in the world of bloat, and even though a 3 MB binary may be totally acceptable, it doesn’t have to be that way.  Even though bulky standard libraries should be used in the majority of cases, they don’t need to be used all the time, and there may even be cases where the app would benefit from not using them.  If only developers would maintain a better sense of how their dependencies are impacting the size of their apps, or indeed what dependencies they’re even using in the first place, we can begin to restore the balance of bloat in our lives.

The lost art of system-level thinking

Our house has steam heat — you know, those old cast iron radiators that are heated by a boiler that sits in the basement. When we first moved in, I was skeptical of how well steam heat actually works, and I began to dread the idea (and the cost!) of having to replace the whole thing with modern forced-air heat. And when we powered on the heat for the first time, my suspicions were confirmed: some of the radiators were making strange noises, and sometimes the pipes themselves would make loud bangs that reverberate through the whole house.

But now, after just a few months, nearly all of these issues are solved, and I am a believer in steam heat and its many benefits.  This is all thanks to a book called The Lost Art of Steam Heating by Dan Holohan.  The book allowed me to solve our heating issues after roughly 20 pages (the pressuretrol was set way too high, if you’re reading, Dan), but I couldn’t put it down and kept reading until the end, for an entirely different reason: the book does a great job of emphasizing system-level thinking, as opposed to narrow immediate problem solving. Even though the book is intended more for HVAC contractors than individual homeowners, I believe that any serious engineer would find the stories and advice relatable.

With every heating “war story” that Holohan recounts, the solution invariably relies on widening one’s perspective beyond the single component that seems to be broken. By the end, I had forgotten that I was even reading about steam heat, but was simply enthralled by Dan’s ability to re-frame the problem from the highest level down. I wish that more engineers in my own field would do the same.   

Problems with steam heat turn out to be easy to diagnose.  It’s simple physics, if at times slightly counterintuitive. The harder lesson is to always think about the big picture. Don’t just replace a faulty component with a new one; think about how the new component affects the rest of the system. Don’t remove a component that you think is unnecessary without understanding why it was there to begin with. And so on.

The lost art, it would seem, is not really steam heat; the lost art is big-picture thinking itself.

“Think about the system.”  Well said, Dan.

Discovering little worlds

Like so many other people during the COVID lockdown, I’ve been looking for additional hobbies that could be done from home, which would occupy my time and help keep my mind off the collapse of civilization as we know it, and maybe even ground my thoughts and keep them away from hyperbole and catastrophizing.

While cleaning and organizing my basement I came across this small USB device. It’s barely larger than a flash drive, but it’s no flash drive at all — it’s a software-defined radio (SDR), namely an RTL-SDR V3 dongle.

I don’t even recall how this device ended up in my possession;  I think it was probably from one of my previous jobs:  they were throwing away a bunch of equipment that was no longer useful, and allowed me to keep some of the items.  Regardless, I had never actually used the SDR, and hadn’t really thought about what the SDR would be useful for.  I had a vague understanding that the SDR can let me tune in to any random radio frequency, but how interesting can that be?  Well, it turns out that playing around with this device led me into a rabbit hole of epic proportions.

Once I found the right software to work with the SDR device (SDRSharp for Windows, and CubicSDR for Mac), I was up and running.  The first rather trivial thing to do was to tune in to the local FM radio stations. Here they are, as viewed through a spectrogram:

FM radio

But that’s kind of boring. I wonder what lies outside of the FM radio band? Well, the next obvious destination is the local police frequencies, which are around 460 MHz to 490 MHz in my area. These are narrow-band FM (NFM) stations, so we adjust our software settings accordingly. In mere moments, I’m listening to police dispatchers communicating with units and telling them about robberies, car accidents, and the like:

And of course there are a few local HAM radio repeaters nearby, which tells me that the HAM community is very much alive and well.  Since I can’t transmit anything using my tiny SDR device, I can only listen in on the HAM conversations, but that’s okay since the conversation wasn’t particularly scintillating anyway, and I’m not sure that getting into the little world of HAM radio is really my goal here. As much as I salute the enthusiasts who keep HAM radio going, they can party on without me.

Mind you, all of this was using the cheap tiny antenna that came with the SDR itself.  But then I discovered that the SDR can be used for something else entirely: receiving signals from satellites!

Arguably the easiest satellites to pick up signals are the NOAA 15/18/19 satellites, which are weather satellites that transmit images of cloud cover over the ground. By “easiest” I mean requiring the least amount of additional equipment:  it only requires a rabbit-ear (V-dipole) antenna connected to your SDR, and a cloud-free day to get a good signal. Here is the signal at 137.9 MHz, and the resulting image, which is produced by special software that demodulates the “audio” data that was recorded:

The downside is that these satellites are in a sun-synchronous orbit, and will only pass by your location for ~15-minute intervals at the most, and can only be caught at very early or late hours of the day. The other downside is that the NOAA satellites are aging, and will probably be decommissioned in the coming years. And anyway, the images they transmit are not the highest quality. Time to step it up to the next level, namely the GOES satellites!

The GOES-16 satellite is a newer weather satellite that is geostationary, and is positioned permanently above the Americas. In fact its longitude is almost exactly over the East coast, which is perfect for my purposes, and its inclination from my location is about 45 degrees (because it orbits around the equator, as all geostationary satellites must do).

But because it’s geostationary it’s also much farther away, and therefore its signal is much weaker, and requires additional equipment:

The setup consists of an old WiFi grid antenna, which feeds into a SAW filter and amplifier, which then feeds into the SDR that’s now connected to a Raspberry Pi (the total cost was about $100).  The Raspberry Pi is running a package called goestools which demodulates the signal from the satellite in real time, and translates the signal into images. The satellite transmits images of Earth in many different spectral bands, ranging from visible light to deep infrared.

And so, the final little world I discovered on this adventure is this one:

The full resolution of these images is 10K, which is mind-blowing, and my next step is to create animations from these images, which are sent by the satellite every 15 minutes.

I think I’ll leave this antenna setup as a permanent installation in my house, so that I can grab these signals anytime I like. Even though this imagery is available on the web if you know where to look, there is something profoundly awesome in knowing that you personally can receive selfies of our world, from 35,000 kilometers away, using about $100 worth of equipment. It’s been a very satisfying few weeks, in spite of everything else that’s happening this year.

Thoughts on the new Star Trek

I watched Star Trek: Picard recently, and… I am sad.

Gene Roddenberry had a vision: a future which is truly post-racial, post-war, post-poverty, etc. It’s a world for us to strive towards, to admire, to want to live in.  But the world of Star Trek: Picard seems to have all the same problems we have in the 21st century.  The Federation is a systemically racist organization that refuses to help an enemy in a time of desperate need. There is deep wealth inequality between different classes of people on Earth. People treat sentient androids like property. And all of “space” is a hostile battleground where one dares not venture without being armed to the teeth.

That’s not a world I look forward to living in, and it would be depressing to me if this is how humanity “turns out” in three hundred years.

Aside from pontificating about today’s political issues, the show’s plot is completely incoherent, and the writing is so lazy and unfocused. Remember Picard’s caretakers at his château who were former Tal Shiar? Those were some interesting characters, but will we ever see them again? Was there any point in having the Borg involved in the story at all? Does the Romulan Samurai kid (bet you can’t think of his name!) have any purpose than to chop people’s heads off? Will Agnes’s murder of Bruce Maddox be swept under the rug? Will the fact that Picard is now a synthetic golem ever be mentioned again?

One perfect example of lazy writing is embodied in the hand-held purple repair device that the Androids conveniently give to Captain Rios. This device basically grants wishes: you can wish it to repair your broken warp core! Or you can wish it to create a mirage of a hundred starships, complete with warp signatures that can fool Romulan sensors! What luck!

And think about how Old Trek and New Trek are different in terms of fandom. There are fans who transform their basement to look like the bridge of the Enterprise. There are fans who program their desktop computer to look like an LCARS interface. And of course there are countless fans who attend conventions dressed up like characters from the Original Series and Next Generation. But will there be any fans who’ll want to recreate the bridge of Captain Rios’s ship (bet you don’t know what it’s called)? Will there be any fans who will admire or want to emulate any of these new characters?

Is it possible anymore to have a show where the whole universe isn’t about to blow up all the time? Can we just have a show where the Enterprise goes to a planet, and Picard negotiates a peace accord, while Data and Geordi get into a wacky holodeck adventure?  I ask for so precious little!

Reverse engineering a 25-year-old Visual Basic app

Following up from last week’s misadventures with the Avant Stellar keyboard (trying and failing to extract macro information from the keyboard’s internal memory), there was another glimmer of hope:  my friend found a backup file that possibly contains all the macros that were saved to the keyboard.  If I could just reverse-engineer this backup, we could extract the macros directly from the file.  It is a 2 KB file with a .KBD extension, unrecognizable as any binary format I’ve seen to date. Here is a partial hex dump of the file:

It’s pretty clear that the file contains a key mapping, as evidenced by the list of incrementing 32-bit numbers at the beginning, up to offset 0x210.  There are roughly 120 increasing numbers, which is roughly the number of keys on the keyboard, so we can safely assume that this is the key mapping.  After the key mapping, I presume, comes the macro information, and this is where things get tricky, since there’s virtually no way to tell how the macros are encoded in the file. The data simply looks too general to make sense of.

An obvious possibility would be to “load” the backup file into the Avant software tool that came with the keyboard, and visually inspect the macro(s) assigned to each key.  But no matter what I tried, the software would not load the file.  Or rather, it loaded the key mapping, but not the macros.  Time to think about the nuclear option: disassemble the Avant software and see how it’s actually processing the backup file.

Looking at the folder contents of the Avant software tool, I immediately notice a dead giveaway: VBRUN300.DLL, which means this tool was written in Visual Basic 3.0.  This makes our job much easier, because there are actually ready-made tools for decompiling Visual Basic executables. (If you recall, Visual Basic compiles executables into p-code instead of native machine code, which makes them much more straightforward to decompile.)  All of this took me quite a while to remember, because I hadn’t used these tools since my early, early hacking days, and it took a little while longer to find them in my archives!  The go-to utility for performing this task was literally called VB3 Decompiler, and the way to find this tool on the web today is… outside the scope of this post.

The decompilation basically results in several Visual Basic source files, in which the original function names are intact, but the local and global variables are changed to generic identifiers, since those names are not stored in the compiled code. It takes a little bit of further massaging to get these files to actually build within Visual Basic, but after that, it’s almost as if you have the original source code of the program at your fingertips.

There was one other minor hurdle because the Avant software uses custom UI components (.VBX files) that don’t allow themselves to be used in Design mode (as part of a copy-protection or licensing mechanism), but this is bypassable using another utility in the decompiler suite that “fools” Visual Basic into loading the components anyway.

With the source code buildable and debuggable, we can now easily run the program and load the .KBD backup file, and trace through where it processes the data in the file:

Even though the variable names aren’t very descriptive in the above screenshot, it’s easy enough to spot the loop that deserializes the keyboard macros, and how each macro is composed.  Not only that, but we can determine what was preventing it from displaying the macros in the first place — it turned out that it expects the keyboard to be physically connected while running, and while I’m pretty sure that we tried loading the backup with the keyboard attached, it wasn’t working anyway, probably because the keyboard is malfunctioning and no longer able to communicate properly.  But at last, with this requirement bypassed, the macros that were loaded from the backup file finally reveal themselves: