UNIX Socket FAQ

A forum for questions and answers about network programming on Linux and all other Unix-like systems

You are not logged in.

#1 2008-02-28 07:46 AM

raviinroorkee.7
Member
Registered: 2007-04-25
Posts: 18

Re: Delete Files older than x hours

Can anyone suggest me a way for deleting files older than x hours.
Any command or shell script ?

Offline

#2 2008-02-28 01:37 PM

RobSeace
Administrator
From: Boston, MA
Registered: 2002-06-12
Posts: 3,847
Website

Re: Delete Files older than x hours

Just use something like tmpwatch...  That's exactly what it's designed for...

If you can't get tmpwatch (or similar) for whatever system you're on, then you could
roll your own with "find -mtime/-mmin/etc.", but it can get tricky...

Offline

#3 2008-03-13 09:05 AM

mlampkin
Administrator
From: Sol 3
Registered: 2002-06-12
Posts: 911
Website

Re: Delete Files older than x hours

Bored... and expanding on Rob's answer...

find . --min +N -name "*" -print | xargs /bin/rm -f

Where N is the number of minutes ( age ) minimum since the last modification time that must be matched...

Of course you could also use -cmin etc. instead... and the '.' part could be changed to the desired target directory... same with the "*" which current matches all files simply based on time ( ignoring name )...

As already warned... this can be VERY dangerous... a typo could result in a lot of files being deleted by accident... so changing the rm command to e.g. a mv so you could manually verify before deletion might be safer...


Michael


"The only difference between me and a madman is that I'm not mad."

Salvador Dali (1904-1989)

Offline

#4 2008-05-08 08:31 PM

jfriesne
Administrator
From: California
Registered: 2005-07-06
Posts: 348
Website

Re: Delete Files older than x hours

What I really want is to be able to set a "cachefile bit" on my file.  That bit would inform the filesystem that the file can be deleted if necessary to free up space on the drive.

With that feature in place, you'd no longer have to decide (e.g.) how many megabytes to devote to your web browser's cached-files directory, because it would be perfectly feasible for that directory to take up all the space on your drive.  The user would never notice, because he'd never see any "out of disk space" errors... instead, when the filesystem went to write to the disk and found that the disk was "out of space", it would simply find the oldest file with the cachefile bit set, delete it, and then try again.

If everybody's going to go around buying 1TB hard drives, we might as well use all of that extra space for something :^)

-Jeremy

Offline

#5 2008-05-08 09:17 PM

i3839
Oddministrator
From: Amsterdam
Registered: 2003-06-07
Posts: 2,239

Re: Delete Files older than x hours

That won't work out that well, because file placement is important too. Leaving
around old files could force the filesystem to use less optimal space for new files,
fragmentating them worse than necessary. Most filesystems perform rather badly
when the filesystem is almost full, so working on an always full disk would degrade
performance quite a lot. But I guess the above is mostly true for smaller filesystems.

Offline

#6 2008-05-08 09:26 PM

mlampkin
Administrator
From: Sol 3
Registered: 2002-06-12
Posts: 911
Website

Re: Delete Files older than x hours


"The only difference between me and a madman is that I'm not mad."

Salvador Dali (1904-1989)

Offline

#7 2008-05-08 10:38 PM

RobSeace
Administrator
From: Boston, MA
Registered: 2002-06-12
Posts: 3,847
Website

Re: Delete Files older than x hours

Plus, at a certain large enough size, things like local disk-based caches will become
inefficient, anyway...  Ie: it'll end up taking longer to seek through the whole disk cache
to find what you're looking for (if it ends up even being in there after all that!) than it
would to just hit the network and download it again...  Exactly what that size is will vary
greatly depending on particular disk and network access specs, of course...  But, you
certainly wouldn't want to just blindly always give your whole disk away for cache
usage...  (RAM, on the other hand, sure...  Until networks VASTLY increase in speed,
anyway...  But, disks are still relatively slow and can easily be beaten by the network
at a certain load point, I'm sure...  Unless maybe you're still on 1200 baud dial-up... ;-))

Offline

#8 2008-05-08 10:39 PM

jfriesne
Administrator
From: California
Registered: 2005-07-06
Posts: 348
Website

Re: Delete Files older than x hours

Offline

#9 2008-05-08 11:34 PM

mlampkin
Administrator
From: Sol 3
Registered: 2002-06-12
Posts: 911
Website

Re: Delete Files older than x hours

If there are lookup concerns...

The top N cache clusters e.g. keyed by site / domain could be kept in something like a skip list structure which would give very good lookup speeds for them...

The a secondary structure for them ( all entries ) could be kept in a hash or tree layout... more costly for time - especially if not perfectly optimized... but should still give a decent response time... it could also be dual keyed e.g. one set on site... the second on file age... the latter keying only existing if the site / files are allowed to be modified...

?

Of course its much easier writing this than it would be implementing it... lol

Michael


"The only difference between me and a madman is that I'm not mad."

Salvador Dali (1904-1989)

Offline

#10 2008-05-09 01:17 PM

i3839
Oddministrator
From: Amsterdam
Registered: 2003-06-07
Posts: 2,239

Re: Delete Files older than x hours

Jeremy, adding that functionality in the filesystem is only burdensome,
because it complicates them without adding any real advantage to do
it there instead of with a normal daemon. All the information you need
can be gathered by a daemon who can make intelligent decisions.
As for the cache bit, that can probably be done with extended attributes
or perhaps even easier, just keeping a list of directories which contain
cached data, then you don't even need application support.

Offline

#11 2008-05-09 03:22 PM

jfriesne
Administrator
From: California
Registered: 2005-07-06
Posts: 348
Website

Re: Delete Files older than x hours

int write(...)
{
   int ret = do_write(...);
   if ((ret < 0)&&(errno == ENOSPC))
   {
      DeleteSomeAutoDeleteableFiles();
      return do_write(...);
   }
   return ret;
}

Offline

#12 2008-05-09 04:53 PM

i3839
Oddministrator
From: Amsterdam
Registered: 2003-06-07
Posts: 2,239

Re: Delete Files older than x hours

True, except that harddisk are so damn slow that in practice it's not a problem,
because you will be able to delete files much quicker than filling your disk.
Harddisk also don't fill up very suddenly, so you can see it comming for a while.
So you can check free disk space, see that it has 5 Gb free, and check again in
a few minutes. If it has 50 Gb free you can check in a few hours. The closer the disk
comes to being full, the more often you need to check. How much free space is left
over in the end determines how often you need to check. Of course whacky
applications can seek around in files like crazy, expanding it beyond the end of
the file and increasing the amount of reserved disk space for it, for filesystems
that do the accounting immediately, but it's doubtful you're better off with it
succeeding.

In the end it's a matter how much information needs to be exchanged. The fs
knows when the disk is full, but everything else is known by user space. So an
optimal implementation would probably be a daemon or program that is notified
or run by the kernel when the disk is full and space is needed. But I suspect that
even just that would be overkill.

Edit: Checking the diskspace usage is only one statfs() call, so it's very cheap anyway.

Offline

Board footer

Powered by FluxBB