Why erase file with random patterns instead of all 0's or 1's?

Question

Computer files aren't actually removed when they are deleted on Windows. The address pointer is simply released and that space freed for overwrite later.

For sensitive information, it is recommended that you erase it with a scrubber that obscures the file first.

It turns out that these scrubbers write a specific sequence of 1's and 0's to the hard drive over several iterrations.

Why can't you just write all zeros or all ones to the file and then delete it normally?

How could someone manage to recover data from a straight wipe like that any more or less than using multiple iterations and patterns?

Solution 1

The short version: theoretically, the original pattern can still be read by certain sophisticated hardware and software. To ensure the security of your "erased" data, it must be wiped.

The long answer: http://en.wikipedia.org/wiki/Data_remanence

---

Edit: In fairness to those that have already voted, I'm leaving my answer as originally written; however, do read the comments for full disclosure.

Many people, myself included, feel that anything more than one pass with cryptographically-sound pseudorandom data is a waste of time and CPU cycles.

Solution 2

I think I remember reading something about the magnetic properties of the platters being such that any given random set of byte overwriting a block would not necessarily fully demagnetize or retun an area of disk to a fully neutral state, thus some information about the prior data was left behind. Granted I don't think it was much, but it sounded like enough that a determined forensic analysis could retrieve at least the nature of the wiped data.

The idea of the specific patterns of 1's and 0's is that they are such that they work with the hard drive 8/10B encoding (or whatever it is) that is used in order to return the overall magnetic storage block to a neutral state.

Have a look at Spinrite, it can apparently show you the various magnetic "levels" that data is stored at in order to recover and "refresh" the data on a drive, at least that's what it claims.

Solution 3

Normal software only recovery methods cannot recover data that is overwritten once by any pattern, it takes a big budget and sophisticated techniques to recover data that has been overwritten only once. One overwrite is good enough unless you have the FBI NSA, NASA ect., wanting your data. But if your paranoid overwrite it 35 times, then disassemble the hard drive and grind the platters into fine dust, then scatter that dust in the open ocean over a 100 mile voyage, hopefully you wont get stranded on an island in the process, ;-)

Of course, modern operating systems can leave copies of " deleted" files scattered in unallocated sectors, temporary directories, swap files,remapped bad blocks, etc, but Gutmann believes that an overwritten sector can be recovered under examination by a sophisticated microscope and this claim has been accepted uncritically by numerous observers. I don't think these observers have followed up on the references in Gutmann's paper, however. So I can say that Gutmann doesn't cite anyone who claims to be reading the under-data in overwritten sectors, nor does he cite any articles suggesting that ordinary wipe-disk programs wouldn't be completely effective.

http://www.nber.org/sys-admin/overwritten-data-guttman.html

.

Comments

[Admin]

Great question.

[Joe_Rose]

So for magnetic media, there is a legacy bit that could potentially be read?

[Adrien]

Not a legacy bit, but that, at a level separate from the filesystem, the previous, different levels of magnetic energy are not fully neutralized without several overwrites with varying patterns. It is definitely a forensic, high-dollar operation. (Megacorp and/or three-letter-agency type budget)...

[Jarvin]

This has been a longtime rumor, but I do not believe there is any evidence to support the idea that this is possible. I've read several people's journeys to try and find someone who can do this and they've all come up empty handed. They all came to the conclusion that it is a myth. That being said, there is really no knowing what goes on or doesn't within the walls of the three-letter-agencies.

[Adrien]

@Dan: First, I am firmly on the "Gutmann was on crack" side of the debate. The "some people disagree with this theory" was, I thought, implicit in my "theoretically", and is explicitly mentioned in the linked article. In the spirit of "don't duplicate effort", I was hoping to get people to read what is already a pretty good article rather than simply regurgitate it here. Perhaps I should have been more blunt, so here goes: Many people, myself included, feel that anything more than one pass with cryptographically-sound pseudorandom data is a waste of time and CPU cycles. YMWV

[supercat]

@Dan: If a track is written when the drive head is in one location, and overwritten when the head is in a slightly different place, then information may remain on one side or the other of the original track. The width of the residue will generally be small relative to the width of the track (potentially zero), but in some cases, especially with older drives, the residue could be recoverable.

[Fiasco Labs]

In the good old days of MFM encoding, low data density and stepper motors, drives had issues where the inter-track spacing could retain residual magnetization due to lax head positioning and temperature changes. Gutmann's paper was written back in 1996. His 35 pass, multiple fixed pattern overwrite was an attempt to wipe a drive with certainty without knowing anything about it. In the decades since then, he's stated that scrubbing with random data is sufficient for modern high data density drives.

[Fiasco Labs]

Why bother scrubbing the disk when simply heating the platters above the Curie point will completely erase them? Grind to dust if you must, but a good charcoal fire is cheap.