simonbrenner.org

Collecting Information

This is the english version. Look here for the german version.

Compendium of the organisation of digital images

--- Preamble ---

If you're the owner of some kind of digital camera you may find yourself confronted with the issue of adequate file and folder names as well as other useful features. One great advantage of digital cameras is the fact that you're able to take as many photos as you want to and to try out many other things. There's no longer the need to have your photos expensively developed on paper to finally see them. Of course, you can still have done that.
It shouldn't take much time to shoot hundreds or even thousands of photos. Perhaps, you find your memory card reaching its limits astonishingly quickly.
I myself was only recently (end of 2008) able to afford a Nikon D-60 and was facing exactly this kind of questions and problems. So I decided to think of solutions for the newly created problems as soon as I could so that no chaos could arise.

--- The camera assigns filenames ---

Everyone with the least bit of experience knows that every digital camera has its own naming scheme for its photos. One camera might call them "DSC_0001.jpg" and another camera might call them "IMG_0001.jpg" or "HPIM0001.jpg". The trailing numbers are increased with every photo taken.
So there's confusion among the different types and manufacturers!
The situation is even more confusing, if one camera has different naming schemes for various situations. My Nikon camera, for example, names its default photos like "DSC_0001.jpg". If you make use of the ability to directly edit photos on the camera, Nikon names the newly created photos "CSC_0002.jpg". There are even editing actions like resizing where the files are named like "SSC_0003.jpg". The good thing is that at least the trailing numbers keep increasing, so there's no "DSC_0001.jpg" and "CSC_0001.jpg" at the same time.
Nevertheless we even don't have a chronological order of our photos, if we see them together in one list! I'll come back to this point later.

Sadly, there don't seem to be a lot of people who realize that most cameras offer the setting of continuous increasing photo count numbers or of starting the count anew when the memory card is cleared.
My Nikon, for example, was set to start the count anew with every clearing of the memory card by default. So every time I formatted the SD card, the first photo was named "DSC_0001.jpg"!
You can surely imagine the outcome of such an inattention or even ignorance: Many photos obtain the same file name. If you're trying to copy some of those into some directory which already contains equally named photos, you're being asked whether you want to overwrite the existing file. Then your mistake is only a mouse click away and you'll loose precious photos immediately.

--> First conclusion: Always check the camera settings regarding the continuation of trailing sequence numbers and be aware of the present setting! <---

--- New filenames ---

I have thought a lot about filenames for photos. One could wonder why there's the need to rename photos at all, if the camera itself is able to manage the filenames, even with continuous numbering.
At first, a camera itself can already be confusing with its different filenames, as described above with my Nikon. Then, imagine you are collecting several photos from different photographers. There's a probability that two photos have got the same name - I already had that case!

The basic idea of a unique filenaming structure is simple: At least the date and time of the shooting moment have to be in the filename! Date and time are the best criteria for sorting, namely chronological.

The question is: How is it possible to have the timestamp written into the filename without the need to edit everything by hand?
Surely, there are innumerable mass file-renaming programs for all kinds of operating systems. I have tested some of them in order to find my personal favourite and to see their plus and minus sides.

Before we delve into batch file renaming, we have to learn about one fundamental thing: EXIF data.

--- EXIF = Exchangeable Image File Format ---

With EXIF, our digital cameras have got the possibility to store various information - meta data - within the photos, like

  • date and time
  • orientation
  • focal length
  • exposure time
  • aperture
  • GPS coordinates
  • etc...

The cool thing is that those meta data are automatically written INTO the picture file by the camera. The great advantage of that is that every photo carries its own precise information around and the user doesn't have to care about those data. They simply exist.
However, one disadvantage is that sensible or personal data can spread into the world though this was not your intention. Or who wants to allow everyone in the world to know his name, address or even geographical data of his house?! I'll come back to that later.
For now, it is our goal to write the timestamp of a photo into its filename. And for that, there's nothing better than to use the data written by the camera itself. One thing has to be said very clearly though: If date or time aren't set correctly in the camera, that will result in wrong filenames! This can mean a lot of work afterwards.

--> This leads us to a first small conclusion: ALWAYS make sure that date and time are set correctly in the camera. <--

Well, now we just have to find some software that is able to compose new filenames out of specific EXIF data fields.
As I said earlier, I've tested several of the possible tools.

-- IrfanView --

One quite easy and very useful program is IrfanView, see http://www.irfanview.de
IrfanView is a mighty instrument with little size for the viewing and editing of pictures of all kinds of formats.
I will however concentrate on the automatic renaming with the help of EXIF data.

Click "Batch Conversion/Rename..." in the "File" menu. Select in the upper left field ("Work as") the entry in the middle: "Batch rename". By the way, with the help of this window you could add several other tasks that you want to be done with a batch processing action, for example automatic scaling, changing quality, adjusting colors etc. But we just want to change the filenames now. The pictures themselves are not touched in any other way, so don't worry about that.

The most important setting is done in the field "Batch rename settings". Enter the following text into the text field "Name pattern":
"$E36867(%Y-%m-%d - %H-%M-%S) - $N"

Let me explain this wildly looking string. "$E36867" tells IrfanView that the following variables (the ones with leading %) are related to the EXIF tag "DateTimeOriginal". This EXIF tag is one of the standard fields written by (hopefully) every digital camera.

-- Excursion to the world of time specifications --

The EXIF standard offers furthermore the fields "DateTime" and "DateTimeDigitized". At least concerning my Nikon it's like that: I can edit photos directly from within the camera. There are quite a lot of such live editing functions. There are functions like resizing, cropping and several effects. The Nikon camera updates the EXIF field "DateTimeDigitized" to the current time stamp after every internal editing action. So when I shoot a photo and edit it directly in the camera afterwards, the EXIF fields "DateTimeOriginal" and "DateTimeDigitized" of the newly created edited photo (!) contain different timestamps. In my opinion, this newly created editing timestamp is almost of no importance for the chronological order of the photos. So we ignore the EXIF field "DateTimeDigitized" for now.
I wasn't able to find a difference between "DateTime" and "DateTimeOriginal" by now. But just to be sure I will always be using "DateTimeOriginal" for the renaming actions.
Then there's the general date and time of the picture files that are saved by the operating system. In Microsoft Windows you can see those if you right click on a file and select "Properties". They haven't got anything to do with the timestamp we need (they can be easily changed when copying files) so I will ignore them as well!

Back to the text "$E36867(%Y-%m-%d - %H-%M-%S) - $N".
After our excursion, the meaning of "$E36867" should be clear. However, this parameter alone means that the EXIF field is taken exactly as it is written by the camera. I can influence the order of the components of this field by some setting of my Nikon (e.g. "2009.02.21 20:00:12" or "21.02.2009 20:00:12"). So the order within this string is not reliably defined.
Because of that IrfanView enables us to tell which parts of the timestamp should stand at which places. This is done by the letters thereafter. I myself change my timestamp to the format "2009-02-21 - 20-00-12" for then the chronological order is assured. You can get more information on those date and time parts if you click the "Options" button to the right of the textfield and then "Help".

Now we've saved the chronological order of our photos - automatically done with certainly correct values! - As long as date and time are set correctly in the camera!!!

One remark remains to be said. Some of you may wonder whether I'm not exaggerating a bit with writing even seconds into the filenames of my photos...
The answer to that is simple: My Nikon, as well as many other models of digital cameras, supports a burst mode. That means I keep the release pressed and the camera shoots away as many photos as possible. My Nikon reaches up to 3 photos per second. So it's only logical that I also keep the seconds in my filenames. Otherwise I could have several files with the same name!

What's the solution to that?

This leads us to the last part of my newly created filename: the "$N".
The simple meaning of that is that the original filename is also kept in the renamed file. Now the problem of more files with the same name cannot occur any more because the camera creates continuous numbers for our photos.

Back to the "Batch Conversion" dialogue of IrfanView.
You have to enter a target directory for the renamed files to be stored in. That means that after the renaming we have got the original photos with original names and additionally the newly renamed photos. So at that moment you need twice the amount of disk space. However, if you did the last steps diligently, you can actually delete the originals because:

--> Conclusion: During the described renaming of the files the actual contents of the photos are never touched! Only the filenames are changed, nothing else. <--

In the end you have to tell IrfanView which photos you want to rename. Either click on "Add all" to add all files that are listed to the batch queue or select only specific photos you want to rename and click on "Add". Then the files to rename should appear in the lower box.
To finally start the whole business click "Start Batch".

One last hint: Many digital cameras are able to save their photos in a so called RAW format. To learn more about that please have a look on the internet. My Nikon can also save RAW and JPG at the same time. In this mode two files are created with every photo. The camera names both files equally, only with different extensions (NEF and JPG). By the way, the extension of a RAW file depends on the manufacturer.
These two files are actually no problem because with the process of renaming they remain in chronological order.

--> Conclusion: In our first step we made sure that our photos are sorted in a chronologically correct order. By using specific EXIF data this was even achieved mostly independent from the original filename This step is very important to obtain a well-sorted photo archive, even over many years. <--

--- A reasonable folder structure ---

Now we've got to look for a suitable place and structure for our photos. Here, I just want to point out some advantages and disadvantages of the structure I'm using myself. Ultimately every photographer has to think about his best fitting structure himself.
My general idea with sorting into folders is the following: There are specific events of which I took photos. Those event photos are stored in one folder then. For example, if the event was "Trip to London on 21.01.2009", my folder would be named "2009-01-21 - Trip to London". Here, of course, I'm also paying attention to the chronological order with the help of the already used year-month-day order.
In my case, I'm also taking lots of photos which cannot be sorted into a specific event. For example, this would be the case if I test something and try out many photos with different settings and I want to keep my excercises.
For those photos I create folders for every quarter of the year. Those folders would be named like "2009-Q1-Photos", "2009-Q2-Photos" etc.
But this is one of the points I meant when I said that everyone may have his own system here. When you take lots of photos that would have to be stored in those quarterly folders, you can also think about one folder per month.

I store all the event folders and all the quarterly folders together in one superior folder. So there's not a huge directory tree you have to delve into when looking for something. The quarter folders appear below the event folders of a year because Windows first has numbers and then characters.

Furthermore, I've got two other special folders: "Temp" and "Rename".
"Temp" is the folder where I temporarily copy the original photos directly from the SD card. This is the source directory for IrfanView to apply the renamings.
"Rename" is the folder where the renamed files are stored.

Before being copied into the final directories there's one last important step...

--- IPTC data and tags ---

We've already heard that with the help of EXIF data it's possible to store data in picture files. In the case of EXIF, the camera itself is doing this. There is however another possibility for us users to store a lot of our own data in the photos: The IPTC-NAA standard, or simply IPTC data. Quickly told, there are many predefined fields with no content by default. So we can fill in our own information there, if we want to use them. Usually (sadly), most users make no use of IPTC. A consequent usage of IPTC fields leads to a mighty and very useful photo management!
Here are some examples of existing IPTC fields:
Copyright, title, keywords (tags), name of the photographer, location (GPS tags), etc...
These are only some of the many IPTC fields. The cool thing about this is that (by means of appropriate software tools) we're able to make rapid and precise searches through our photo collection.
So the key to a practical usage of IPTC data is a software capable of dealing with them! Pay attention with the similar looking kind of data that you see when right clicking on a photo in Windows and selecting "Properties". They seem to be completely different fields from the real IPTC data.

As for myself, I'm using two different programs to deal with IPTC data: On one hand I use Google Picasa as a fast and easy-to-use viewer and unique cataloguer tool. With Picasa, I write my tags (or keywords) into the photos. On the other hand I use the command line tool "exiv2" to write the other IPTC data fields.

-- Google Picasa --

Picasa is a free photo management software which was bought by Google in 2004. The program is absolutely useful for handling even big amounts of pictures and writing specific tags.
After the installation you see two columns and one row below. On the left side there is the tree of the folders which are to be read by Picasa. On the right side there are the photos and below this area you've got quick access to several options and functions.
First, you have to tell Picasa which folders you want to manage and how this is to be done. This is accomplished with the help of the menu "File" -> "Add Folder to Picasa". In the "Folder Manager" you can then specify for each folder of your hard disk whether you want Picasa to scan it only once, to ignore it or to scan it constantly (when the program is running). I removed all automatically added folders in the beginning and then selected only those I really want to have monitored. Picasa treats the "Scan always" function recursively, so subfolders are automatically read as well. Of course you can change the option afterwards for subfolders.
In the field "Watched Folders" you see all currently added folders summarized.
After clicking "Ok" the reading of the photos can take some time. Then you see all folders in the tree on the left side where you can click on them to see the photos.

 

TO BE CONTINUED....