Data Recovery in Ubuntu · English

I deleted all the files in the home directory on my netbook. I deleted them through the console, so I couldn’t recover them from the trash. The most important things were probably the photos, my Emacs config, and a few documents of minor importance. I started looking for “undelete” tools and the like, and came across foremost, which seemed the easiest to use.

Let me say right away to anyone fond of saying “it’s all because you’re constantly running as root”: all the files in my home directory have my permissions, and I would have deleted them even if I’d been logged in as myself! If it had been another directory, I’d agree with them.

Background

Actually, I deleted them through a strange combination of circumstances. Here’s how it happened:
I launched mc, pressed Ctrl+O, switched to the console, and then:


cp -R /www/project /home/adw0rd
cd /home/adw0rd
find ./project -name ".svn" | xargs rm -rf

Now I wanted to leave the console and get back to mc:
while pressing Ctrl+O, I accidentally pressed Ctrl+P (just like our president with мое6). Ctrl+P registered first, bringing up the previous command:


find ./project -name ".svn" | xargs rm -rf

and immediately afterward Ctrl+O registered, returning me to mc.

I thought it was no big deal, since I was already in file manager mode (there was nothing in the console at the bottom), and I wanted to go into ./project.
I clicked the “project” directory… And that’s when everything went to hell: mc froze, the HDD started churning, and I tried to shut down the console, but apparently it was already too late… :)

Recovering Files

Let’s install the foremost utility:


sudo apt-get install foremost

This runs an audit and creates a report file:


foremost -w -i /dev/sda1 -o /path/to/recovery

This recovers all files:


foremost -i /dev/sda1 -o /path/to/recovery

This recovers files of a specific type:


foremost -t jpg -i /dev/sda1 -o /path/to/recovery

It’s best to put the recovery directory on another storage device mounted on your system, such as an external hard drive, etc. That way, the recovered data won’t overwrite anything you haven’t recovered yet.

A Script for Removing Duplicates

Since foremost recovers everything indiscriminately, you end up with duplicate files whose contents are completely identical.
So I decided to write a little script that would remove duplicates and leave only the originals.


#!/usr/bin/env python

import os, sys
from hashlib import md5

COLOR_RESET = "\033[0m"
COLOR_RED = "\033[1;31m"
COLOR_GREEN = "\033[1;32m"

def main(path):
    hashes = []
    filenames = []

    for root, dirs, files in os.walk(path):
        files_len = 0

        for file in files:
            files_len += 1
            path_to_file = path + file
            f = open(path_to_file).read()
            fhash = md5(f).hexdigest()
            try:
                i = hashes.index(fhash)
                #if f == open(path + filenames[i]).read():
                #    print COLOR_RED + path_to_file + ' --- remove!' + COLOR_RESET
                #    os.remove(path_to_file)
                print COLOR_RED + path_to_file + ' --- remove!' + COLOR_RESET
                os.remove(path_to_file)
            except ValueError:
                print COLOR_GREEN + path_to_file + ' --- UNIQ!' + COLOR_RESET
                hashes.append(fhash)
                filenames.append(file)
        hashes_len = len(hashes)
        print "--------------------------------"
        print "Files: " + str(files_len)
        print "Deleted: " + str(files_len - hashes_len)
        print "Rest: " + str(hashes_len)

if __name__ == "__main__":
    main(sys.argv[1])

It gets the MD5 hash of a file’s contents and puts it in a list. If that hash is already in the list, the file is deleted; otherwise, it moves on to the next file.
For anyone concerned about possible collisions, uncomment the block of code in the middle of the script that also checks the file’s contents if its hash is found in the list.

Here’s how to use it:


./remover.py /recovery/jpg/

Protecting Ourselves for the Future

Well, let’s start small: in ~/.bashrc (or the configuration for another shell, such as ~/.cshrc), we’ll add an alias that asks for confirmation before deleting:


alias rm="rm -i"

I’m not particularly fond of this, since it gets annoying…

The trash-cli Utility

An even better idea is moving data to the trash in the background, as Anatoly Larin recently described here, for example. For this, we’ll use the trash-cli utility, which is written in Python and consists of several commands, described below…

Each user on the system has their own trash directory, something like:


/home/<username>/.local/share/Trash/files
/root/.local/share/Trash/files

That’s where our files will be stored, so we can retrieve them if necessary or delete them permanently.

Let’s install the “trash-cli” utility:


sudo apt-get install trash-cli

Create an alias for deletion (in ~/.bashrc, and don’t forget to add it to /root/.bashrc as well):


alias rm='trash'

Deletion will still be done through rm, and you can view the list of “deleted” files (more precisely, in our case they won’t be deleted, but moved to the trash) like this:


list-trash

Emptying the trash:


empty-trash

For the curious: I tried creating a user, putting a bunch of files and directories in their account, and then deleting everything there, but it tripped over the trash directory and threw an exception, so the data will remain intact.

One Last Thing

After deleting something, try not to get too worked up, and you’ll be able to recover everything. The main thing is not to panic, or you’ll make an even bigger mess! :)