No space left on device

Got problems with your B2 or B3? Share and get helped!
Post Reply
lukas
Posts: 3
Joined: 22 Sep 2010, 06:26

No space left on device

Post by lukas » 11 Oct 2010, 10:22

I suddenly got a problem with my Bubba|Two. It say "No space left on device", but I am sure the disk is filled at less than 25 percent. I wasn't able to connect on the web interface due to php errors, but ssh works. I was able to solve the php errors by symlinking /var/lib/php5 to a folder inside the /home/myusername directory.

When I check the syslog in the web interface, I see lots of errors like "Oct 11 16:15:13 bubba postfix/postdrop[4179]: warning: uid=0: No space left on device".

The output of the df -h command yields this:

Code: Select all

Filsystem          Storlek Anvnt Tillg Anv% Monterat på
/dev/sda1             9,2G  9,2G     0 100% /
tmpfs                 125M     0  125M   0% /lib/init/rw
udev                   10M   24K   10M   1% /dev
tmpfs                 125M  4,0K  125M   1% /dev/shm
/dev/mapper/bubba-storage
                      907G  158G  704G  19% /home
In addition, I am not able to access the Bubba as a network drive anymore.
Best regards, Lukas

albert
Posts: 69
Joined: 09 Jul 2007, 14:35
Location: The Netherlands

Re: No space left on device

Post by albert » 11 Oct 2010, 17:18

Hi,

you mistake the space on your device used to store your data, and the space on the device that contains your root system directory (which is what this error is about).

One common reason for having the root filesystem (/ ) filled up is that it contains the /var/log directory as well. Have a look in there and just remove all *.gz log files. There might be a samba or httpd apache directory as well, so have a look in there as well. Log files could fill up quickly if something is wrongly configured.

Hopefully it is as simple as that. When it is, please check your log files (the ones which are the largest) to see the reason of all the loggin.

Good luck,

Albert

lukas
Posts: 3
Joined: 22 Sep 2010, 06:26

Re: No space left on device

Post by lukas » 12 Oct 2010, 04:59

Thanks a lot for the explanation, you saved my day!
I was able to solve it by investigating the root filesystem, and found that my crontabbed backup command had copied stuff to /mnt/backup without checking that an external hdd is actually mounted. Network sharing works great too now after that I disabled and then enabled windows file share in the web backend.

Best regards, Lukas

kastrom
Posts: 13
Joined: 29 Oct 2008, 17:56

Re: No space left on device

Post by kastrom » 14 Oct 2010, 17:39

I have the same problem, I have removed every qz log file and every other large log file I could find but I still have close to full disk:

Code: Select all

/dev/sda1             9.3G  9.0G     0 100% /
tmpfs                 125M     0  125M   0% /lib/init/rw
udev                   10M   24K   10M   1% /dev
tmpfs                 125M  4.0K  125M   1% /dev/shm
/dev/mapper/bubba-storage
                      914G  344G  525G  40% /home
192.168.0.196:/c/media
                      2.8T  2.4T  415G  86% /home/storage/nas
What can I do do make more space? Or is this normal?

/Kent

albert
Posts: 69
Joined: 09 Jul 2007, 14:35
Location: The Netherlands

Re: No space left on device

Post by albert » 15 Oct 2010, 05:55

The system will struggle with a filled up root filesystem.

To find where's the culprit try:

du -x -h / | more

the -x uption means du is not going to view other (mounted) filesystems, the -h is for a more human readable form (i.e. 1G for 1 GByte etc).

At least you can see which directories are containing the most data, have a look in there.

Albert

ryz
Posts: 183
Joined: 12 Feb 2009, 06:03

Re: No space left on device

Post by ryz » 16 Oct 2010, 08:04

An better command to run is

Code: Select all

du -Sx / | sort -nr | less
Note that you should be the root user when running this command to be sure that you have permission to read all the files.

The result from this command will be a sorted list with the biggest file/directory first. Note that I have removed the -h flag to the du command so the size is in bytes. The sort command can not handle the postfix. The -S flags tells that it should not include size of subdirectories.

kastrom
Posts: 13
Joined: 29 Oct 2008, 17:56

Re: No space left on device

Post by kastrom » 16 Oct 2010, 10:37

Thank you for your help!
It was /var/spool/postfix/maildrop that where full of strange files with names like E77F78BD17. Now one day after I removed all files there are 61 MB in the directory.
Is there a way to stop the files to filling my disk?

/Kent

Ubi
Posts: 1547
Joined: 17 Jul 2007, 09:01

Re: No space left on device

Post by Ubi » 16 Oct 2010, 13:23

Do not kill postfix mail files like this as it sill mess up your mail subsytem completely. Instead run "postsuper -D ALL". Better still is to check why your mails are being stuck by using "mailq" and looking at the mail logs

Still. You seem to need 5GB space and all you can find is 60MB of mail files? I'd say you missed the important bits

Ubi

kastrom
Posts: 13
Joined: 29 Oct 2008, 17:56

Re: No space left on device

Post by kastrom » 16 Oct 2010, 14:05

I don't have mail configured and I have 61 MB since yesterday when I erased 7 GB of these files. postsuper -D ALL doesn't work and mailq doesn't give me any clues:

Code: Select all

624958BC94   244096 Sat Oct 16 07:55:07  root
                                         root

A03848BB76   244096 Sat Oct 16 03:30:07  root
                                         root

BEC9D8BA94   244096 Sat Oct 16 00:25:11  root
                                         root

A1D5B8BA76   244096 Sat Oct 16 00:15:10  root
                                         root
/Kent

geertvw
Posts: 1
Joined: 29 Apr 2011, 03:21

Re: No space left on device

Post by geertvw » 29 Apr 2011, 03:26

Had the same problem yesterday. I don't postfix at all on my bubba 2, and still files are queuing up in the maildrop dir. Yesterday I removed 7GB, today 4MB is back already. Where is it comming from?

I know this is an old thread, but is there a solution now?

RandomUsername
Posts: 904
Joined: 09 Oct 2009, 18:49

Re: No space left on device

Post by RandomUsername » 29 Apr 2011, 05:34

Things like cron jobs etc. will email the root user if there's a problem. You could set up an alias for root to your email address. Can't type any more now, guests arriving for my royal wedding party!

Moloko
Posts: 51
Joined: 12 Feb 2010, 00:50

Re: No space left on device

Post by Moloko » 29 Apr 2011, 09:28

Can't type any more now, guests arriving for my royal wedding party!
So you are still here typing, aren't you invited to your royal wedding party?? :wink:

RandomUsername
Posts: 904
Joined: 09 Oct 2009, 18:49

Re: No space left on device

Post by RandomUsername » 29 Apr 2011, 09:56

I was invited but my wife was doing all the organising. I had to let people in the door, I'm back now.

To finish my post, I added this to my /etc/aliases file:

Code: Select all

root: darren
but that's because I have an email address set up under my own username in postfix. Not sure if you can do this:

Code: Select all

root: user@someotherdomain.com
But it's worth a try.

Then run (as root):

Code: Select all

newaliases

Cheeseboy
Posts: 789
Joined: 08 Apr 2007, 12:16

Re: No space left on device

Post by Cheeseboy » 29 Apr 2011, 14:41

Hi Darren,

Hope you all had a nice day off and celebration today (I spent most of the day playing games as all the UK customers were non-existent).
I have this in my /etc/aliases:

Code: Select all

root: admin@xyz.se
But the domain it is pointing to is the one handled by B3 itself...

Cheers,

Cheeseboy

Post Reply