How do you limit the use of network storage in your company?

Question

I think this is one area that is a pain for every sysadmin.

Users don't really know how much storage space they are using, and don't take the time to manage what is stored there, or how long it stays around.

What policies does your company implement to keep networked disk space under control?

Solution 1

This is one of those areas where my views are contrary to the mainstream. We do not have disk quotas, in fact we encourage users to get data onto the servers and out of file cabinets. We have been doing this for a while, in preparation for implementing document management.

• Disk is inexpensive, and has been for a while.

• None of the alternatives is more palatable to letting home folders and shared folders grow. If users have disk quotas, they are either going to save data onto their local drives, find other places on the network to hide it, or print it and stick it in their drawers/files.

• We do regularly monitor space and usage, and work with users who are getting out of control. We can help them archive things to DVD and/or to triage when necessary.

• We take this approach with email as well. We have an archiving solution (old mail to an archive server) so the disk space on the email server is not out of control. Only a couple of users are really a concern anyway, and we can work with them.

EDIT: Reading other answers and comments, a question comes to mind .. should we force the business and the users to cater to our convenience? My view is that we should cater to the business as much as possible, which is why we operate a "quota-free" environment. Our role is to support the business, not the other way around.

Solution 2

I was a sysadmin at a school and all students had network accounts. I wanted to encourage the use of computers so I removed the quota limits on the home directories. For most students this went well but some filled up their directory quickly with gigs of video and mp3.

Contrary to what you expect this wasn't pirated stuff but their own video footage and music from the school band.

So in the end I only needed a script which made up some nice statistics of the file usage, if a home drive was filling up more then ten times the average I would have a look at it and suggest a better way of archiving. Mostly this was just a list of the biggest files and last access time. Really misbehaving students would be put back into quota but that situation has not risen during my employment there.

Solution 3

We have a chargeback model where the customer pays for what they consume. That puts us in the business of selling storage, so the more they use, the better it is for us. (Higher volume = lower unit cost, since staffing requirements don't scale up quickly with disk utilization) We define a couple of different categories of storage (remote, central, central-replicated), and don't use quotas or similar mechanisms.

The keys to making something like this work is:

• Be transparent
• Assist customers with paring down storage upon request
• Revisit your chargeback model from time to time to ensure that it is fair, and isn't subsidizing particular customers.

Solution 4

We use the default Microsoft quota system on the file servers, specifically when it comes to home drives.

Other than that, one of our admins down south of us has a tendency to let the users' project spaces fill up, then encourages them to run duplicate finders and large-file/last-accessed programs such as "Doublekiller", "Easy Duplicate Finder" and "JDisk Report", which is one of my personal favorites.

Another solution we're looking at implmenting is Symantec's "Enterprise Vault" especially if you're doing disk-disk-tape or have the tape storage available to shuffle files off to.

Solution 5

We split disks into two groups.

You get a limited amount of backed up storage, and, this is limited by us being able to actually backup that data.

You get a much much less limited amount of unbackedup storage.

This works in an environment where we don't have regulatory requirements to save and audit everything, and, we have users who can make the distinction between important and not so important storage.

If your environment isn't so relaxed you might want to look into Sun's SAM/QFS which lets you push snapshots offsite. I'm sure that other vendors have similar products. I have friends in heavily regulated environments who use this and are happy.

View more solutions

Comments

[Zoredache]

Disk space may be cheap, but is your backup space also cheap? How many backups to you keep? With a long retention cycle 1GB of data can mean many times that amount in real usage.

[tomjedrz]

Backup is at times a headache. Currently, LTO4 is of sufficient speed and capacity that we are OK. We have a makeshift hierarchical system for the email archive and the big users where old stuff is moved to a location only backed up on the weekends.

[TomOnTime]

Drives are cheap. Storage is expensive. The physical disks are about 20% of the cost of storage. You aren't providing "disk space", you are providing "the service of storing, maintaining, and protecting your data". Socialize that philosophy around. Permit people to choose between unprotected data on their desktop and "protected data" on the server... at a reasonable price.

[sleske]

A pity I can only upvote once. +++1 for remembering that computers should work for users, not vice versa.

[sleske]

+1 If you need to be restrictive, doing it fairly and transparently is best.

[warren]

good point - most students wouldn't put their pirated crud on the network share anyways (not all, mind you - I've seen the ones who do), they'll keep it on their laptops for easy conveyance.

[Phil]

+1 for "we work with users" instead of the all-too-common "we generate unilateral policies and enforce them until we have obliterated user productivity."

[korylprince]

Work at a district with a 1:1 Macbook deployment in grades 7-12 (total about 1000-1100 students). This summer I cleaned off roughly 1TB of pirated music, movies, and games from student home directories. Interestingly enough, a large chunk of it came from the lower grades.

[korylprince]

@Zoredache delta-style backups should be minimal overhead, especially with user files that change very little.