Problem copying files to Bubba

[johannes]

Ok, thanks for this information, it helps. And don't worry about the 30 day return period, this issue is clearly a warranty issue. This behaviour is not acceptable for a server, we'll either fix it or offer replacement or return.

Thanks for your patience.

[jws]

Ok, thanks for this information, it helps. And don't worry about the 30 day return period, this issue is clearly a warranty issue. This behaviour is not acceptable for a server, we'll either fix it or offer replacement or return.

Thanks for your patience.

Some more data that might be useful:

Today I could copy the directory "MySoap" (8 GB) from a USB disk plugged into Bubba2 to /home/storage/video. The Bubba2 did not crash. Why not?

I did two things differently:

-- apt-get install hddtemp
-- during the copy process, every second or so did

Code: Select all

ls -al /home/storage/video/MySoap

and

Code: Select all

hddtemp /dev/sda

Maybe constantly looking up things like that slowed down the copying process sufficiently. Or hddtemp by itself has some influence. I'll test an un-interrupted transfer next.

During the copy, the temperature went up slowly from 38 degrees C to 40 degrees. It is now slowly creeping down.

BTW these temperatures are in the range of body temperatures (of humans with some fever). How about a small hole in the back of the Bubba where you could stick an infra-red "ear thermometer" in?

[jws]

Unfortunately it failed again after (uninterrupted) copying of 20 GB out of a 35 GB directory. By that time the disk temperature was 42 degrees C. Trying again after plugout/plugin and restarting was not successful:

-- immediate restart and retry: immediate crash
-- waiting for a while, then restart and retry: something is copied, then crash. By this method I have now copied 25 GB out of the total 35 GB using cp -uR.

The fact that "waiting" seems to have some effect makes me think that temperature is somehow involved. I do not have the heater running yet, and today was colder than yesterday, so maybe that's why the copy operation lasted longer. But maybe it is something else (capacitors recharging and draining, backup battery?) Anyway it looks more like a hardware problem than a software problem.

[johannes]

jws,

From/to what are you copying? From USB to internal disk? Or from network?

[jws]

jws,

From/to what are you copying? From USB to internal disk? Or from network?

In this case, from USB (external) disk plugged directly into bubba, to internal disk.

[jws]

A new fact (if it is a fact):

I now managed to crash the bubba2 many times during huge downloads. The problem is finding the cause of the crashes.

One thing I noticed: each time when a crash happened, there was a slight "click" sound from the external USB disk (Western Digital "Elements", 500 GB). Just the same sound which it makes when it moves its arm for a "seek". This would cause a delay in the read operation, while presumably the write operation to the bubba2 would go ahead. If the delay is too great, a buffer underflow would result. A "seek" could happen at any time, "randomly" (for instance, my own bubba2 runs a website, so disk activity is also influenced by the outside world).

If so, we have to find out which buffers are used by the cp program, and give them bigger default sizes. Or something.

Just speculation of course.

/jws

[Hape]

...This behaviour is not acceptable for a server...

We could ask 'super-nanny' to surpress this 'unacceptable behaviour' ? ;-)


Sudden death/stall could well be CPU core temp, or one of the supporting chips or even the hard disk itself.

What type of harddisk is used in the situation where the unit freezes up?
Low power (i.e. WD GP type) or standard?


As I only have just received the unit and just starting to tweak a bit, it's no big deal for me to restore to fact-default if things really go wrong, so I might stick a temp-sensor on the CPU and have a try with 50+ GB copy.

- haven't opened the unit yet, is the CPU easy to ID? (any images avail, pref. with pointing finger ;-)


Regards,

Hapé

Bubba/Two
Uptime 2 days 15:58:00 (grin)
Version 1.0.8-1
HD 1000GB Western Digital WD10EACS GP
LAN 100 Mbit

[Hape]

A new fact (if it is a fact):

I now managed to crash the bubba2 many times during huge downloads. The problem is finding the cause of the crashes.

One thing I noticed: each time when a crash happened, there was a slight "click" sound from the external USB disk (Western Digital "Elements", 500 GB). ..
/jws

I just wonder... is it (crash/hang....click) OR (click .... crash/hang)?

Modern HD's are mostly autoparking..move the head(s) to landingzone (or even lift them) when going offline. They tend to *click* at that moment. IMHO this is what you hear.

Suppose your USB HARDDISK is shortly going offline and online again midst a transfer? Bubba sees discon/connect on USB port and tries to 'identify' (auto-sense) the just attached device?

Did you test with a different USB HD?

I wonder what would happen if I just 'pull the plug' (USB plug in this case;-) during a transfer. Will try this after current test.

b.t.w. did you already post the configuration of your setup? Like what services are running (notably FireFly, etc. in view of auto scans of disk content while your copy is busy),
what is connected? (only LAN, or LAN + WAN?) and do you use Bubba as router/firewall between you and the *evil* internet?
Apart from that, do you have Win/XP HOME on the same network while copy from othe NFS system? (Some weird things happen from XP/Home ;-)


My now running test (110 GB transfered sofar at constant CPU temp of 54°C) from USB to internal HD runs flawless.

Rgds,

Hapé

[Hape]

Johannes,

Is there an utility available on the Bubba/Two to control the blue frontl LED state?

As the Bubba only shows it being alive by means of ethernet, a problem with ethernet would look as if Bubba is 'dead'.

If jws could background a process that blinks the LED every 30 seconds, even after trashing ethernet, the Bubba could indicate that the CPU is still alive...

Just a thought..

rgds

Hapé

[mad]

I've had similar problems with excessive torrent traffic. When downloading torrents from lots of peers the bubba has hung completely and then it reboots automagically.
The logs reveal nothing and there are no core files to be found. Haven't had time to research it more as throttling the downloads helped.

[jws]

I just wonder... is it (crash/hang....click) OR (click .... crash/hang)?

Don't know; it feels like click ..crash but it could just as well be the other way round, because the crash itself does not give a visible or audible sign. When I hear the click, I type in a login shell, and there is no reaction.

Modern HD's are mostly autoparking..move the head(s) to landingzone (or even lift them) when going offline. They tend to *click* at that moment. IMHO this is what you hear.

That could very well be it.

Suppose your USB HARDDISK is shortly going offline and online again midst a transfer? Bubba sees discon/connect on USB port and tries to 'identify' (auto-sense) the just attached device?

Did you test with a different USB HD?

No, but I also have crashes when copying large amounts of data from the main HD of my desktop computer (over an nfs connection). So I doubt that the external USB HD (which is fairly new and has never given problems) is the cause.

I wonder what would happen if I just 'pull the plug' (USB plug in this case;-) during a transfer. Will try this after current test.

what is connected? (only LAN, or LAN + WAN?) and do you use Bubba as router/firewall between you and the *evil* internet?
Apart from that, do you have Win/XP HOME on the same network while copy from othe NFS system? (Some weird things happen from XP/Home

WAN and LAN are connected; the Bubba2 runs a small website which has traffic maybe once every 2 minutes; Bubba2 is router/firewall for two desktops, one running Debian Sid, and one running Ubuntu. No XP. I installed shorewall on Bubba2, blocking any outside traffic to Bubba2 itself apart from 80. It also does some port-forwarding for a bittorrent client on one of the desktops. But I also had a crash during a test when the ADSL modem was switched off (i.e. no WAN traffic).

My now running test (110 GB transfered sofar at constant CPU temp of 54°C) from USB to internal HD runs flawless.

You mean HD temp? Or is there really a utility for measuring the CPU temp?
This is encouraging in showing that the problem is not universal. But according to this thread, I am not the only one having trouble with it. BTW do you just use the cp command for the copying?

As to your suggestion in your other message, a "life tester" using the LED as indicator can be made as follows:

Code: Select all

#!/bin/bash
while [ 1 ]; do
     /etc/init.d/led_blink
     sleep 1
     /etc/init.d/led_on
     sleep 9
done

for a blink every ten seconds. Am going to test it now. But I think it will show that the Bubba2 really crashes (not just an ethernet problem) because one of the symptoms of a crash is that the reset button does not work anymore.

Oh, and before you ask: I could find nothing useful in /var/log/syslog or /var/log/messages. The crash must happen very suddenly.

Regards, Jan

[Hape]

Good morning!

Jan, thanks for extensive reply.
My biggest problem sofar is time. I only received the unit wednesday, so I didn't have time to study it in detail. Thanks for pointing out the led_on/off in /etc/, saves time looking for it.

I agree that the problem is not isolated to USB. I now read the whole thread and read that you already had the same problem on NFS / network.

The outcome of my testing yesterday:

Some factors during test:

- Ambient (room) temp: 22 °C
- CPU Temp measured by means of digital probe, stuck (with tape!) on top of the CPU package. (Extech 430 multimeter)
- FireFly service NOT running.
- WAN NOT connected.
- Attached USB HD: WD 500Gb, Toshiba USB Casing (IDE to USB2) with 143 Gb MP3 files in hundreds of dirs, FAT32, greatest filesize 25 Mb

The CPU ran at 43 °C when idle (no LAN activity, no webinterface/SSH opened) for ca. 1 hour.

After that I connected to Bubba with SSH, and did

Code: Select all

#su
# cd /home/storage/extern/disk_0/
# cp -r * /home/storage/music/test/ [enter] date [enter]

The second 'date' so I would get the time after finish.

Fired up second SSH to sens aliveness:

Code: Select all

# while true; date; do sleep 30;done

Minimal impact on performance, indicates time the unit would stop responding.

Code: Select all

lap time/temp°C/ remark
01:00   43  start copy
01:08   48  approx 3 GB/500 files done
01:30   50
02:00   52

interrupted SSH session 2 (the 30sec loop) and counted progress:

# du -sm /home/storage/music/test/
 
(cont. loop) 
03:00   55   53940   /home/storage/music/test/  (so approx 54 GB transfered, sofar 18 GB/hr)
04:30   54   80692   /home/storage/music/test/ (still very close to 18 GB/hr)
05:30   54
06:30   54
07:00   54   122261  /home/storage/music/test/  (17.4/hr, dropped a little)
08:12   54   142890  /home/storage/music/test/
08:15   57   !found that I covered the top of the unit with the multimeter *ahem*
08:17   54   but it cooled down quickly again

approx 143 GB transferred in 8hrs 12min

This morning the temp was back at 48 °C after night running idle

Hafway the test I started to make some connections to the Bubba with a Win2k system, mounting networkdrive //Bubba/ and looking at the copied dirs. Counting (find *) files and refreshing (F5) the dirlist.

No real impact was found.

I concluded that
- very unlikely a thermal problem on the systemboard (well designed!)
- hang does not occur with USB copy of a lot of 'small' files


* * I've been told that Johannes indeed confirmed a complete hang now, NOT only ethernet connection.


In the past I have been experimenting with a MU5000 from Ovislink, a small embedded linux Samba server. The biggest problem I had was ...... Unit froze up during transfer of BIG files.


So, the next test I'll conduct is transfer BIG files to/from the Bubba via network.

Jan, just a hunch, but could you test your previous situation (with hang) but now with FireFly service OFF?
Just curious.

Will keep you posted.

Regards,

Hapé

[jws]

If jws could background a process that blinks the LED every 30 seconds, even after trashing ethernet, the Bubba could indicate that the CPU is still alive...

I just did a test, trying to copy a 50GB directory to the Bubba2, while running the "life indicator" described in my previous post.

After 12 minutes and about 8 GB copied (i.e. a speed of about 10 MB / 100 Mbits per second), the by now inevitable crash occurred. The blue LED was constantly on (did not blink anymore every 10 seconds). So the Bubba2 was really, truly dead..

Johannes, please, solve this problem soon.

[jws]

Jan, just a hunch, but could you test your previous situation (with hang) but now with FireFly service OFF?

I don't know what you mean by FireFly; nothing like that shows up in ps aux. But from Googling I gather that it has something to do with daap. I am not interested in daap, so I took some drastic measures; by means of

Code: Select all

update-rc.d -f <file> remove

where <file> is some file in /etc/init.d.

I disabled many Bubba-specific things:

Code: Select all

mdadm
mdadm-raid
mediatomb
mysql
mysql-ndb
mysql-ndb-mgm
mt-daapd
ftd

But the &@$%%!! crash still occurred. When experiments are done on the same day, it seems each new experiment has the crash occurring sooner. The first test today lasted 12 minutes; the last only 2 minutes. I am beginning to think that my Bubba2 has a hardware problem.

Regards, Jan

[Hape]

The first test today lasted 12 minutes; the last only 2 minutes. I am beginning to think that my Bubba2 has a hardware problem.

I understand your frustration.

Indeed mt-daapd (webinterface 'DAAP streaming') http://www.fireflymediaserver.org/ is the one I ment.

I just noted that Excito even follow this discussion on sundays and I am totally confident they will do anything needed to keep you as a satisfied customer. (No I am not related to them;-)
Allowing this discussion to continue public and not behind hidden support-email also shows they are dedicated to solutions.

Keep our fingers crossed for a quick solution

Hapé