Category: Programming

Correctly naming your photos

I seem to be very minimal in my strategy of organizing my digital photo collection. I have a single folder on my computer called “Pictures,” and subfolders that correspond to every year (2011, 2010, …) since the year I was born. Some of the years contain subfolders that correspond to noteworthy trips that I’ve taken.

This method makes it extremely easy to back up my entire photo collection by dragging the “Pictures” folder to a different drive. It also makes it easy to reference and review the photos in rough chronological order. This is why I’ve never understood the purpose of third-party “photo management” software, since most such software inevitably reorganizes the underlying directories in its own crazy way, or builds a proprietary index of photos that takes the user away from the actual directory structure. If you’re aware of the organization of your photos on your disk, then any additional management software becomes superfluous.

At any rate, there is one slight issue with this style of organizing photos: all of the various sources of photos (different cameras, scanners, cell phones, etc) give different file names to the photos! So, when all the photos are combined into a single directory, they often conflict with each other, or at the very least become a disjointed mess. For example, the file names can be in the form DSC_xxxx, IMG_xxxx, or something similar, which isn’t very meaningful. Photos taken will cell phones are a little better; they’re usually composed of the date and time the photo was taken, but the naming format is still not uniform across all cell phone manufacturers.

Thus, the optimal naming scheme for photos would be based on the date/time, but in a way that is common between all sources of photos. This would organize the photos in natural chronological order. The vast majority of cameras and cell phones encode the date and time into the EXIF block of each photo. If only there was a utility that would read each photo, and rename it based on the date/time stored within it. Well, now there is:

Download it now! (Or browse the source code on GitHub)

This is a very minimal utility that takes a folder full of photos and renames each one based on its date/time EXIF tag. As long as you set the time on your camera(s) correctly, this will ensure that all your photos will be named in a natural and uniform way.

The tool lets you select the “pattern” of the date and time that you’d like to apply as the file name. The default pattern will give you file names similar to “20111028201345.jpg” (for a photo taken on Oct 28 2011, 20:13:45), which means that you’ll be able to sort the photos chronologically just by sorting them by name!

The FujiFilm .MPO 3D photo format

A few weeks ago my dad, in his love for electronic gadgetry, purchased a FujiFilm FinePix REAL 3D camera. The concept is pretty simple: it’s basically two cameras in one, with the two sensors spaced as far apart as an average pair of human eyes. The coolest thing about the camera is its LCD display, which achieves autostereoscopy by using a lenticular lens (kind of like those novelty postcards that change from one picture to another when you look at them from different angles), so if it’s held at the right angle and distance from the eyes, the picture on the LCD display actually appears 3-dimensional without special glasses!

Anyway, I immediately started wondering about the file format that the camera uses to record its images (as well as movies, which it also records in 3D). In the case of videos, the camera actually uses the well-known AVI container format, with two synchronized streams of video (one for each eye). In the case of still photos, however, the camera saves files with a .MPO extension, which stands for Multiple Picture Object.

I was expecting a complex new image specification to reverse-engineer, but it turned out to be much simpler than that. A .MPO file is basically two JPG files, one after another, separated only by a few padding zeros (presumably to align the next image on a boundary of 256 bytes?). Technically, if you “open” one of these files in an image editing application, you would actually see the “first” image, because the MPO file looks identical to a regular JPG file at the beginning.

I proceeded to whip up a quick application in C# to view these files (that is, view both of the images in each file). This quick program also has the following features:

  • It has a “stereo” mode where it displays both images side by side. Using this feature you can achieve a 3D effect by looking at both images as either a cross-eyed stereogram (cross your eyes until the two images converge, and combine into one) or a relaxed-eye stereogram. You might have to strain your eyes a bit to focus on the combined image, but the effect truly appears 3-dimensional.
  • In “single” mode, the program allows you to automatically “cycle” between the two images (a wiggle-gram, if you will), which creates a cheap jittery pseudo-3D effect (see screen shots below).
  • Also in “single” mode, the program lets you save each of the frames as an individual JPEG file by right-clicking on the picture.
So, if you want a quick and not-so-dirty way of viewing your MPO files, download the program and let me know what you think! (Or browse the source code on GitHub)
Here’s a screenshot of the program in “stereo” mode:

And a screenshot of the program in “cycle” mode:

If you like, you can download the original .MPO file shown in the screenshots above.

Now for a bit of a more technical discussion…. Clearly it would be a great benefit to add support for the .MPO format to DiskDigger, the best file carving application in town.

However, from the perspective of a file carver, how would one differentiate between a .MPO file and a standard .JPG file, since they both have the same header? As it is now, DiskDigger will be able to recover the first frame of the .MPO file, since it believes that it found a .JPG file.

After the standard JPG header, the MPO file continues with a collection of TIFF/EXIF tags that contain meta-information about the image, but none of these tags seem to give a clue that this is one of two images in a stereoscopic picture (at least not the tags within the first sector’s worth of data in the file, which is what we’re really interested in).

One of the EXIF tags gives the model name of the camera, which identifies it as “FinePix REAL 3D W3.” Perhaps we can use the model name (the fact that it contains “3D”) to assume that this must be a .MPO file, but I’d rather not rely on the model name, for obvious reasons, although the FinePix is currently the only model that actually uses this format (to my knowledge).

The other option would be to change the algorithm for JPG carving, so that every time we find a JPG file, we would seek to the end of the JPG image, and check if there’s another JPG image immediately following this one. But then, what if the second JPG image is actually a separate JPG file, and not part of a MPO collection?

For the time being, DiskDigger will in fact use the model name of the camera to decide if it’s a .MPO file or just a regular .JPG file. The caveats of doing this would be:

  • It won’t identify .MPO files created by different manufacturers.
  • It might give false positive results for .JPG images shot with the camera in 2D mode.

As always, you can download DiskDigger for all your data recovery needs. And if anyone has any better ideas of how to identify .MPO files solely based on TIFF/EXIF tags, I’d love to hear them!

Update: DiskDigger now fully supports recovering .MPO files, based on deep processing of MP tags encoded in the file!

Thumbnail cache in Windows 7 / Vista – a rumination

(Note: read my newer post on this subject!)

Today I was thinking about the security implications of thumbnail caching systems on most PCs out there today. What I mean by that is this: whenever you use Windows Explorer to browse a directory that contains photos or other images, and you enable the “thumbnail view” feature, you would see a thumbnail of each of the images. By default, Windows caches these thumbnails, so that it doesn’t have to regenerate the thumbnails the next time you browse the same folder.

This has several implications in terms of privacy and security, since it means that a copy of each image is made elsewhere on the computer (albeit lower resolution), basically without the user’s knowledge. This is good news from a forensic examiner’s point of view, since the thumbnail cache can contain thumbnails of images that have long been deleted. However, from the user’s point of view, it can present a privacy/security issue, especially if the images in question are confidential or sensitive.

Windows XP caches thumbnails in the same folder as the original images. It creates a hidden file called “Thumbs.db” and stores all the thumbnails for the current folder in that file. So, even if the original images were deleted from the folder, the Thumbs.db file will still contain thumbnails that can be viewed at a later time.

However, in Windows 7 and Windows Vista, this is no longer the case. The thumbnails are now stored in a single centralized cache under the user’s profile directory: C:\Users\[username]\AppData\Local\Microsoft\Windows\Explorer\thumbcache*.db

The above directory contains multiple thumbnail cache files, each of which corresponds to a certain resolution of thumbnails: thumbcache_32.db, thumbcache_96.db, thumbcache_256.db, and thumbcache_1024.db.

So then, wouldn’t you like to find out what thumbnails your computer has cached in these files? Well, now you can! I’ve whipped up a small utility for the sole purpose of viewing the contents of these thumbnail caches:

Download ThumbCacheViewer now! (or see the  source code)

This is probably not the first utility that does this, but it’s definitely the simplest. It automatically detects the thumbnail caches present on your computer, and lets you view all the thumbnail images in each cache.

If you want to disable the thumbnail cache in Windows 7 or Vista, you can find instructions here.

Discovering the 3D Mandelbulb

There is some exciting news this week in the world of fractals. Daniel White, on his website, describes what is apparently a completely new type of fractal, and the closest analog so far to a true 3-dimensional Mandelbrot set!

Although White mentions that this is probably not the “true” 3D Mandelbrot, the new fractal is undoubtedly a sight to behold, especially considering the renderings he showcases on his webpage.

Unable to contain my enthusiasm, I quickly wrote up a small program that uses OpenGL to actually display this shape in 3D, in real time, to get a feel for what this beast looks like from all angles. Don’t get too excited; the program does not render the shape in real time, it just displays the points rendered so far in real time. The actual rendering process can take a minute or so.

Download the program using the link below, and read further for instructions and screen shots.

Download the program! (or download the source code)

download

The program basically renders the 3D shape by constructing a “point cloud” that approximates the edge of the fractal.

Everything in the program should be relatively self-explanatory, but here’s a brief overview of the features so far:

  • The program lets you click-and drag the rendered shape to rotate it in trackball fashion (left mouse button), as well as zooming in and out (right mouse button).
  • The program lets you select the “power” of the Mandelbulb formula, as well as the number of iterations to perform.
  • The program lets you select the resolution of the point cloud.
  • It gives you a “selection cube” with which you can select a subset of the shape to zoom in on (with the “zoom to cube” button).
  • It has a number of other minor features like fog and anti-aliasing.
  • It uses multiple threads to render the shape, so it will take advantage of multiple cores/processors.

Here are some additional screen shots:

Manipulating the selection cube:

After zooming in on the cube:

Zooming in further:

Looking inside:

Colorized points:

The program was written in C# with WinForms, using the Open Toolkit Library (OpenTK) which provides an excellent OpenGL wrapper.

Of course, this program is very much in its early stages, so don’t expect it to be perfect. As always, suggestions are welcome!

The Math Book: Get it Now!

Cliff Pickover, the prolific author of more than forty popular science and mathematics books, has outdone himself with his latest compilation: The Math Book. This is a collection of 250 “milestones” of mathematics throughout history, complete with breathtaking glossy color illustrations for each entry (a first for his books), as well as insightful descriptions that explain the history and the significance of each of these marvels of mathematics.

This book is especially significant in one other way: it contains my artwork! The book’s entry on Knight’s Tours (p. 186) familiarizes the reader with the history of this problem, dating all the way back to Euler in 1759. And, alongside the article, Pickover displays a 30×30 knight’s tour that was solved by my neural network knight’s tour implementation. For the picture in the book, I used a modified version of the program that generated a sufficiently hi-res image. That particular knight’s tour took about 3 days for my computer to generate.

I’m deeply grateful to have one of my creations published in a book by someone as influential as Cliff Pickover. Of course, it’s all of the 250 entries in the book that make it an incredibly fascinating stroll through the history of mathematics. As mentioned elsewhere, this book definitely has bestseller potential, and could easily be one of Pickover’s best works. Buy the book now!