Easyfind update: script and cron job for automation

[OlivierC]

I have read this topic:
http://forum.excito.net/viewtopic.php?f=9&t=5304

I have a different issue but someone could give me answer and tutorial to solve it

B3 network profile: Router + Firewall + Server

B3 <> Router DSL modem [PPPoA] <> outside world
--------------------- 192.168.10.x /24 -----[LAN____B3_192.168.10.1___WAN]------192.168.1.x --------------192.168.1.254/24 --[LAN DHCP pool SpeedTouch_510_v4_10.0.0.138 WAN]---81.x.x.x(public IP)

The Wan side of B3 local router (facing the modem) has a PUBLIC IP. It sends traffic to the LAN side of the modem, that has a PRIVATE IP.

The WAN-port is always getting a new IP-address after lease time (7 days) of ISP DHCP-Server

Question for wm.bubba or other expert who answered in "Easyfind down?" topic

I also run a script every hour from cron that checks if my IP address has changed and only then calls the easyfind script.

Please could you post script and explain how to run it from cron.
I'm not a Linux guy so detailled instructions would help me to configure myself my B3

Currently, i have to log on to B3 and update Easyfind from Web GUI each 7 days.

[wm.bubba]

Hi OlivierC,

Hopefully some simple instructions you can follow to do this. Replace wherever there is an xxxxx with your user-name on your B3.

Code: Select all

First install wget, nano & sudo (not sure if these are installed by default)

a.  login using ssh to bubba,

	  shh xxxxx@192.168.1.3

     and enter your password.

b.  su to root,
	  and enter root password (excito).

c.  install wget, nano & sudo -

	  apt-get install wget nano sudo

d.  set editor for use with visudo -

	  export EDITOR=nano

e.  edit "sudoers" file using -

	  visudo

f.  add the following line to the end of the file -

	  xxxxx ALL = NOPASSWD: /usr/lib/web-admin/easyfind.pl

     when done - Ctrl-X, select Y, then Enter.

g.  to restart service - 

	  /etc/init.d/sudo restart

h.  exit root user, type
    
	  exit

Code: Select all

Create scrip to run easyfind

	mkdir bin
	cd bin
	nano ip.sh

paste in the following text:

#!/bin/bash
#ip.sh

user="/home/xxxxx/bin"
ip_old=$(cat $user/ip.txt)
ip_new=$(wget --no-proxy -U firefox -O - -q "http://www.telize.com/ip")

if [ -n "$ip_new" ]; then
    if [ "$ip_old" != "$ip_new" ]; then
        date=$(date +"%d/%m/%Y, %H:%M:%S : ")
        easyfind=$(sudo /usr/lib/web-admin/easyfind.pl)
        echo $date $easyfind >> $user/easyfind.log
    fi
fi
exit 0

when done - Ctrl-X, select Y, then Enter.

now to make this script executable, type:

          chmod +x ip.sh

Code: Select all

Add entry to cron, type the following to edit cron for user:

	crontab - e

and paste the following line:

	00 *  *  *  *     /home/xxxxx/bin/ip.sh >/dev/null 2>&1

when done - Ctrl-X, select Y, then Enter.

That should be all. Any questions or issues, let me know.
Good Luck!

[Gordon]

I'm somewhat puzzled by that solution. Why use a secondary service to verify your IP address when the easyfind server does the exact same thing? What if that service stops or they change the output format? And I don't really get why you run the IP checker as a regular user instead of root - the latter which would eliminate the need for sudo.

I would also like to point out that the DHCP client has a hook script that will automatically update easyfind if the WAN port on the B3 receives a new IP address. So if you do actually have a public IP on that port - unlike OP as indicated by the ASCII art drawing - no special action would be required.

[wm.bubba]

This is how I understand the Easyfind service works:

The Easyfind "change-IP trigger" functionality uses three signals;
1) changing IP on WAN-port, or
2) UPnP-IDG broadcast of new WAN IP from your router, or
3) reboot or network re-start.

My B2 (I don't have a B3) is behind a router, and thus does not currently detect changes in WAN IP (WAN-port disconnected). Also I'm not sure my router has UPnP-IDG functionality (or I can't find the setting), so the only option was to do a manual update running from cron (it's actually every 15 minute, not every hour).

As I didn't want to swamp the easyfind server every 15 minutes, I came up with the above solution. Maybe it's overkill and there is an easier way to do, but it has been running for 2 years without any issues. Maybe it's time for a rethink!

[Gordon]

That's the old way.

Don't remember which software version introduced a change to that. timestamp on /usr/share/bubba-backend/bubba-easyfind.tac suggests this must have been the 2.6 version. This may however be an update version of that core component of the bubba easyfind service, as users found it would clogg their system log (and drive!).

As stated earlier, this new method does not ask for an IP address to be submitted to the easyfind server, but only the unique bubba ID and chosen name. The IP is taken from the (masked) return address that is in the request header. Just like telize.com does.

If you activate easyfind from the B2|3 web interface it will also start the bubba easyfind service which polls the easyfind server every minute. IMO that's a bit much, but some might like it. If you don't you can change the looptime in /usr/share/bubba-backend/bubba-easyfind.tac. Or you disable the service manually and use cron (as root) to periodically run /usr/lib/web-admin/easyfind.pl

[wm.bubba]

Thanks Gordon. I wasn't aware of that.

I have had a look at bubba-easyfind.tac, and can see the time is set to 60 seconds, and agree it is a bit much. Looking at the log files, In syslog I can see messages every 60 seconds, so it would appear the Easyfind services is running.

Earlier today I commented out my script, and forced an IP address change to see what would happen. I have copied the log messages at the time of the IP address change (I have masked out the full IP address):

Code: Select all

May 19 12:17:01 b2 twisted: [HTTP11ClientProtocol,client] Got new IP '87.114.###.###' which is the same as the last one '87.114.###.###'
May 19 12:18:01 b2 twisted: [HTTP11ClientProtocol,client] Got new IP '91.125.###.###' which is not the same as the last one '87.114.###.###'
May 19 12:19:01 b2 twisted: [HTTP11ClientProtocol,client] Got new IP '91.125.###.###' which is the same as the last one '91.125.###.###'

You can see my IP address changed between 12:17 & 12:18. However, running a query on the excito server shows it's still pointing to the previous IP address:

Code: Select all

$ nslookup ###.myownb2.com ns1.excito.org
Server:         ns1.excito.org
Address:        109.74.2.52#53

Name:   ###.myownb2.com
Address: 87.114.###.###

If I understand correctly, the Easyfind process should have updated this to point at my new IP address of '91.125.###.###' , but it has not done so. Am I missing something?

[Gordon]

That seems familiar. You should search the forum with that log line, this has come up before.

[wm.bubba]

I have found the relevant posting http://forum.excito.net/viewtopic.php?f ... ent#p23346.

It's too much to take in during my lunch break, so I'll try and go through it tonight and see where that gets me.

[Gordon]

I did some reading back on that item and essentially the way the easyfind service operates isn't that different from your method at all. IP checking is in fact done on a different server than the one used for updating the DNS. The difference is that you use a secondary provider to verify the IP address and I would advise against that if you don't know why they provide that service and whether you can rely on them to keep offering that service.

I don't know if that issue ever got solved for the person that submitted it (bIO) or why it failed in the first place. On my own B3 it appeared to do what it was supposed to do, but I'm on a fixed IP and therefore can't really tell if not receiving an error actually results in an updated IP. Ofcourse bIO did see an error being returned from the update server, that the supplied 'key' and 'mac0' parameters did not match the info in the Excito database.

[OlivierC]

I want B3 acting in "router+firewall+server mode"
Alcatel SpeedTouch ADSL modem had to be configured in Routed PPPoA mode.
modem dial PPPoA to get me to the internet.
B3 Wan side port obtain private IP address from modem dhcp pool.
in GUI: network settings > WAN
address shown is private address assigned by Speedtouch modem/router to B3 WAN port
not the ISP public IP

In my previous post, wording was wrong

The Wan side of B3 local router (facing the modem) has a PUBLIC IP. It sends traffic to the LAN side of the modem, that has a PRIVATE IP.


correct is
The Wan side of B3 local router (facing the modem) has a PRIVATE IP (dynamic from SpeedTouch dhcp pool). It sends traffic to the LAN side of the modem, that has a fixed PRIVATE IP.

When ISP renew my public IP address each seven days, does B3 really detect IP change
as the IP address on the WAN port is still the same (SpeedTouch assign always the same private IP address to B3).

That why i would like to initiate automatically with a cron job the Easyfind update.
not a specific script which check other service
but built-in B3 script (which act like GUI button Easyfind update click)
I would schedule it to run once a week.

[Gordon]

Hi Oliver,

No, the B3 cannot sense a change of IP in another device. To be able to actually catch that moment the other device would need to send some sort of notification of the event. While that is technically possible I've never seen it in any ISP supplied router I've received.

The way it usually works - and also in case of the B2|3 - is to go onto the internet and literally ask "what IP do you see me sending from?" and compare that with the IP address that is associated with your DNS entry. If they're the same then no action is required and otherwise you send a message to the dynamic DNS provider to update the entry with the now current IP address.

Now the way you describe your situation, you would not actually need to verify the IP address since you know your ISP assigns you a new one in a specific interval. You can simply take that same interval and run the update routine /usr/lib/web-admin/easyfind.pl. What you should be aware of though is that the update routine sometimes can fail and that a regular check isn't actually a bad thing to do. What I found some time ago, and noted in that other topic, is that the update routine updates the local record of the easyfind address regardless of whether the update actually succeeds. The cron driven routine as shown by wm.bubba in this topic in fact appears to do the same.

If you decide to adopt that script to run in large intervals (i.e. 1 hour or more) I'd therefore suggest that you do not simply verify against a local record of the IP you think is associated with your easyfind name, but in fact do a host lookup. Preferably do that lookup directly at the Excito DNS servers, as it can take some time for updates to be rolled to other DNS servers on the internet.

[wm.bubba]

Indeed that is how my script works, currently. I was going to look at rewriting it this weekend, but Easyfind breaking again scuppered that idea.

I'll probably not get round to this for a few weeks now, but the changes I plan to make include:

• using http://ef.excito.org/ip.json, http://easyfind.excito.org/extip.php (or similar), to verify my external IP address instead of a third party,
  using nslookup (or similar) to verify Easyfind has been correctly updated,
  still using cron to trigger the above checks.

Any ideas or improvements to the above would be gratefully received.

[Gordon]

There was of course this script in that other topic:

Code: Select all

#!/bin/bash
old_ip=$(grep '^ip' /etc/network/easyfind.conf | awk {'print $3'})
new_ip=$(wget -o /dev/null -O /dev/stdout http://ef.excito.org/ip.json| \
    grep '{"ip_address":"[0-9\.]*"}'| sed 's/{"ip_address":"\([0-9\.]*\)"}/\1/')
if [ ! -z "$new_ip" ] && [ "$old_ip" != "$new_ip" ]; then
    /usr/lib/web-admin/easyfind.pl
fi

This takes the old IP address from the easyfind config file rather than keeping a record of it in a separate file. The new IP is taken from ef.excito.org, similar to the easyfind twistd service. I did this because this server is obviously meant to take the load of the actual update server and will be better dimensioned towards a lot of people simultaneously requesting their IP - remember the easyfind service does it every 60 seconds.

Verifying the DNS entry could be done like this:

Code: Select all

ef_name=$(grep '^name' /etc/network/easyfind.conf | awk {'print $3'})
ef_ip=$(host -s ${ef_name} ns1.excito.org | grep ${ef_name} | awk {'print $4'})
ef_ip2=$(host -s ${ef_name} ns2.excito.org | grep ${ef_name} | awk {'print $4'})
if [ -n "${ef_ip}" ] && [ -n "${ef_ip2}" ] &&  [ "${ef_ip}" != "${ef_ip2}" ]; then
  echo "DNS is inconclusive - got two different responses"
fi
if [ -z "${ef_ip}" ] ; then ef_ip=${ef_ip2}; fi
if [ -n "${ef_ip}" ] && [ "${ef_ip}" != "$(grep '^ip' /etc/network/easyfind.conf | awk {'print $3'})" ]; then
  echo "DNS does not match local record"
fi

[OlivierC]

I tried to call script (1st listed by Gordon) saved with nano to checkip.sh
from command line with command:
bash checkip.sh
Unfortunately, it fails and i don't know how to debug
output is
:command not found

this one echo some output:

Code: Select all

#!/bin/bash
old_ip=$(grep '^ip' /etc/network/easyfind.conf | awk {'print $3'})
new_ip=$(wget -o /dev/null -O /dev/stdout http://ef.excito.org/ip.json| \
    grep '{"ip_address":"[0-9\.]*"}'| sed 's/{"ip_address":"\([0-9\.]*\)"}/\1/')
if [ ! -z "$new_ip" ] && [ "$old_ip" != "$new_ip" ]; then
    echo IP address has changed since last check
    echo Update Easyfind using new IP address $new_ip
    /usr/lib/web-admin/easyfind.pl
else
    echo No IP address change detected
fi

when compared into Windows utility Notepad++, i don't see any differences between both except extra echo lines and if then else fi

[Ubi]

My guess would be you do not have wget installed. If thats not so, then just enter an exit statement halfway the script, or put 'set -x' at the first line to echo all output.