Tracking down disk space hogs

TygerTyger · 12th January 2010, 04:06 PM

I have a site reportedly using 400Mb of space, but I can only account for around a third of that. I assume there's a stack of large files somewhere, but I can't find them. Is support able to look more easily? Does CPanel have better facilities for seeing what is where?

VEAdas · 12th January 2010, 04:14 PM

I have the same problem. Ensim seems to be rubbish at it. I also hope its better in cpanel, I also hope that webmail and OS files are not included in our allocations as it is currently.

TygerTyger · 12th January 2010, 04:32 PM

You have to look at it from the perspective of the host, they can't exclude parts from allowances any more than you can exclude your OS from your own hard drive. Eventually you're simply going to run out of space. I had a very big loop which threw an error with every recursion. My error log ballooned, filled the rest of my disk allowance within minutes and the site crashed until support deleted the log. But that's just how it is.

UH-Matt · 12th January 2010, 05:17 PM

Quote:

Originally Posted by VEAdas

I have the same problem. Ensim seems to be rubbish at it. I also hope its better in cpanel, I also hope that webmail and OS files are not included in our allocations as it is currently.

The only thing currently included is around 8MB for some system files (nowhere near an OS) due to the fact that you have your own instances of things such as php. Squirrelmail also uses 10-15MB.

Cpanel does not utilise space for system files as you don't have your own instances of software like Ensim, but scripts such as webmail will still use space.

Unfortunately as TygerTyger said, its very difficult to descriminate with disk usage... Usage is usage and everything on your account has its size which in disk uses space.

UH-Matt · 12th January 2010, 05:17 PM

Regarding the initial question its a good idea to first check the usual suspects such as /var/log/httpd/ for large error_log files, and also users home dir's for large IMAP mail files or folders.

Support can definitely help you identify where that usage is coming from, feel free to open a ticket.

percepts · 13th January 2010, 08:35 PM

Quote:

Originally Posted by TygerTyger

I have a site reportedly using 400Mb of space, but I can only account for around a third of that. I assume there's a stack of large files somewhere, but I can't find them. Is support able to look more easily? Does CPanel have better facilities for seeing what is where?

your ftp program should tell you. I use smartftp.

TygerTyger · 13th January 2010, 09:03 PM

Once I knew it was definitely in the file system I just downloaded the whole lot and dealt with it in Windows. Time to upgrade my FTP software, probably. But a bug (or unexpected behaviour, if you prefer) was also a problem and I should probably explain in case anyone else gets caught out and searches the forum.

If you disable web log generation, log files are frozen as they are and are not deleted until you re-enable log generation and they start being rotated again. For whatever reason, it doesn't clear the cache and you have no way of knowing because it also removes the downloads (and therefore file sizes) from the administration panel. An old access log from October was still taking up 75Mb, a substantial proportion of the entire site.

You can either re-enable log generation and then catch it immediately after rotation for re-disabling or just leave it turned off and ask support to delete the files for you.