Write errors when backup to FTP target

[pa]

Sorry for not responding, but this is not something that is easily solved, and we do have lot of other things to do aswell.

As far as this matter goes, I am working on a setup that will display this error where I have access to the serverside as well since I do not know the reason for why the connection is lost. Our backup is based on duplicity and that is that program that in turn is running ncftp. There are updates to duplicity but none currently available in the debian repositories for etch that is newer that what we are running.

We are looking at the possibility to upgrade duplicity, but that requires a lot of effort from our side and is something that will have to come along with other major upgrades, nothing we can release on a shot term since it must be tested prior to release.

So in short, you are not forgotten and not ignored, but since I have not found the cause of the problem yet I can not guarantee you when this issue will be solved.

/PA

[RandomUsername]

Thanks for replying pa, sorry if I came across as a little rude but this is an advertised, out-of-the-box feature of your product that isn't working and it's one that is very important to me. I'm very disappointed because of this.

I appreciate you've other things to do but you did mention in your last post that you've managed to recreate the issue. This gave me a bit of hope that you might be able to solve the problem but it seems that perhaps this isn't the case.

As per previous posts, I've eliminated the possibility of it being the ftp server or ncftp itself. I've no experience with duplicity but perhaps you could offer some way of troubleshooting that outside of the backup.pl script?

Thanks.

[RandomUsername]

Fixed it. I added

Code: Select all

--timeout=180

to each duplicity command in the backup script. Duplicity apparently defaults to 30 seconds and this wasn't enough time for it to get a list of files from the ftp server for comparison. Not sure why it was working for other people though, my internet connection is pretty good so there's no reason for it to be timing out.

[RandomUsername]

I think I have got to the fundamental root of all the problems mentioned in this thread but I can't work out how to fix it.

Somewhere between logging on to the ftp server and starting to upload files this appears in the log:

Code: Select all

ftp command: 'set passive yes' 

The correct syntax to set passive mode with ncftp is

Code: Select all

set passive on

If you enter the command

Code: Select all

set passive yes

at an ncftp prompt it disables passive mode whatever it was set to previously.

Obviously running ftp in active mode when the ftp server is not on the LAN means the Bubba's firewall blocks the incoming traffic - hence all the timeouts.

Now I'm not sure why adding the

Code: Select all

--timeout=180

worked for me before (I guess something to do with the way the ftp server was configured) but I've switched to a different ftp provider (because the old on was too unreliable) and this fix no longer works.

The problem is, I can't see what part of the whole process is issuing this command. I don't think it's the backup.pl script, I actually think it's duplicity but I haven't worked out anything more specific than that yet.

Some help from the Excito folks would be most welcome.

Thanks.

[RandomUsername]

Fixed again.

There are two lines are in

Code: Select all

/usr/share/pycentral/duplicity/site-packages/duplicity/backends/ftpbackend.py

that need to be changed from

Code: Select all

 self.passive = "no"

and

Code: Select all

self.passive = "yes"

to

Code: Select all

 self.passive = "off"

and

Code: Select all

self.passive = "on"

My test backup just ran through successfully so: woot!

I've spent days on this problem so I hope it benefits somebody else as well.

[pa]

Hi,

I thought you left it with adding the timeout, but I am glad you got to the root of the problem.
You are right about being duplicity that issues the command, but we will look into this and see if it is either already fixed in newer versions of duplicity or else we need to push this "upstream" to duplicity.

/PA

[RandomUsername]

Hi Pa,

I did check out a newer version of duplicity and it doesn't seem to have these commands in it. Feel free to double check though.

I thought you left it with adding the timeout,

This fix stopped working when I changed ftp servers so I had to start investigating again.

Now that it's all fairly stable I'm going to have another look at getting newer versions of Duplicity working so I get the "resume failed backups" feature working.

Cheers.

[pa]

I have been looking at newer versions of duplicity and there are numerous fixes to ftp sessions.
However if I remember correctly the calls to duplicity have also changed somewhat making impact also on backup.pl so there is quite a lot to do to make it work with our userinterface.

But it is on our todo to update duplicity, there are many things there that we want to have....

/PA

[grushog]

I've spent days on this problem so I hope it benefits somebody else as well.

I have had the same problem. I applied the changes you described and the problem appears to be solved. Thanks.

Excito: Any plans to fix this?

[RandomUsername]

I have had the same problem. I applied the changes you described and the problem appears to be solved. Thanks.

You're welcome, glad it came in useful.