[mdlug] Quick Directory Removal?

Wojtak, Greg GregWojtak at quickenloans.com
Mon Aug 18 12:57:33 EDT 2008


I’ve got an application that generates on the order of a million files in one directory in one day.  This directory is NFS mounted.  Aside from the poor design of the application treating a directory as a database, I’ve put a script in place on the nfs server to clean it up every night, removing files that are a day old.  This is a GFS filesystem, which isn’t built for speed to begin with, and running find in this directory takes a long time, and so does a rm -rf.  So I started thinking of ways to remove this directory more quickly.

One thing I came up with and wanted to check with other UNIX smarty-pants about was this: suppose the filesystem I am mounting is /export, and the directory I want to remove is /export/appdata.  What I was thinking was to create a null device on /export, then:

# cd /export
# mv appdata null
# mkdir appdata

I haven’t tested this, but I believe this would be very fast for two reasons: I’d be moving the directory to the bit bucket without scanning the files in it, and it would be, as far as the filesystem subsystem is concerned, on the same filsystem, so it wouldn’t have to move any data on the disk.

So I have two questions – will it be as fast as I think it is?  Is this safe to do?  I’d like to hear opinions on this .  If this can save on time and IO overhead, it would make me very happy.  Of course, I’d still preach to the developers about how badly this is hurting the server. ☺

Greg




More information about the mdlug mailing list