Easyfind: what is the road map for this service?

[Gordon]

Never mind. Binkem already did it.

Reading the easyfind managing scripts on the B3 it does not only send the secret key, but the first mac address as well . Verification on the easyfind server therefore appears to be done by matching the secret key to the first mac address in the production database. However, since the provided key does in fact match it seems now that there must be a third variable being included that can cause a valid key/mac0 pair to be discarded as well.

Essentially you'd be looking for an SQL statement somewhere on the server in index.php (or index.pl or some included file) that says

Code: Select all

select * from production where secret='$_POST["key"]' and mac1='$_POST["mac0"]' and something=true

...or possibly "sold < $today"... just throwing some options.

PS B3 mac addresses all start with 00:22:02 and there's really no guessing involved if you can just read the key here on this forum, is there? May want to search the forum for other instances as well as I stumbled over one while searching the forum for other easyfind related issues. Did not make a note which message though.

[Ubi]

Binkem changed the source code on the mybubba server? Cool, considering he has no access to these machines!

As for the code, i know exactly which function is misbehaving, and its not working the way you think it is. Im not some 12 year old writing his first script here.

As for the mac, yes, b3 users can deduct this, but all other people on the interweb cannot by using google. I tried to be discreet, but alas.

Gordon, I dont understand why you are getting so cranky all of a sudden, Im just trying to help you out. If this is not appreciated then fine.

[Gordon]

Nope...

Binkem wiped the secret entry that you put inside the [code] ... [/code] block in your earlier post.

And I'm not being cranky. For me the easyfind server that you are hosting is a black box and it gives me an unexpected response for a very specific range of B3's I currently have in my possession. From your own statement we know that the two apparent inputs are not causing this issue and therefore there has to be some other reason. I'm also not saying that you don't know your business, but the fact is that you did not write the application that is returning the conflicting answers. I'm also assuming that you put the X's in your earlier post to conceal what are in fact known digits, but I guess if you didn't and this is the actual content of the database that would certainly explain why there is no match on my mac0 submission.

[Gordon]

Problem identified.

I was in fact thrown off track by the script returning the conf file content and only contacting the server if it can't find all the information in there. As it turns out there was a problem with the key extraction routine and I had a trailing newline character in the key I was sending. After trimming the value running the getname function now returns:

Code: Select all

No record_id name set (x485)
{"error":true,"opcode":18,"msg":"No record_id name set (x485)","operation":"Get record"}

And once a name has been set and easyfind is then disabled I get:

Code: Select all

Not a HASH reference at ./easyfind.pl line 223

So apparently there is a difference between no name being set (NULL value?) and a name that was cleared (''). Obviously I'm doing stuff here on the command line that the web interface wouldn't do, but the server is returning an incorrect response here. Dumping the hash it shows:

Code: Select all

$VAR1 = {
          'record' => bless( do{\(my $o = 0)}, 'JSON::PP::Boolean' ),
          'operation' => 'Get record',
          'opcode' => 18,
          'error' => $VAR1->{'record'}
        };

(The value for 'record' in this case reads 'false')

The correct response should however be something like:

Code: Select all

$VAR1 = {
          'opcode' => 18,
          'error' => bless( do{\(my $o = 1)}, 'JSON::PP::Boolean' ),
          'operation' => 'Get record',
          'msg' => 'No record_id name set (x485)'
        };

(from the response when name=NULL - unsure what the x485 refers to)

[Ubi]

ah ok so the lookup is not a server error as you concluded but a user error on your end. Thanks.

x485 is a reference to the line number that I stuck in as as pointer for myself. And yes, there is a difference between NULL and EMPTY, being that in the case of NULL the key is not transmitted in the POST, changing the number of key-value pairs in the POST, and this number of pairs determines a switchpoint in the server code.

Why do you insist of saying (again) that the server return is wrong?
the server returns a single string: "{"error":true,"opcode":18,"msg":"No record_id name set (x485)","operation":"Get record"}" which is exactly what it should be.

Then you say it returns " 'error' => $VAR1->{'record'} ". It just doesnt do that. The server code doesn't even contain $VAR1 anywhere. Maybe the client code does, but not the server.

[Gordon]

Try yourself (if you own a B3 that has never used easyfind before).

Set easyfind to enabled:

Code: Select all

echo "enable = yes">/etc/network/easyfind.conf

Request your registered name from the easyfind server:

Code: Select all

/usr/lib/web-admin/easyfind.pl getname

Return:

No record_id name set (x485)
{"opcode":18,"error":true,"operation":"Get record","msg":"No record_id name set (x485)"}

Add an easyfind name:

Code: Select all

/usr/lib/web-admin/easyfind.pl setname <yourname>.myownb3.com

Delete it:

Code: Select all

/usr/lib/web-admin/easyfind.pl disable

Set easyfind to enabled again:

Code: Select all

echo "enable = yes">/etc/network/easyfind.conf

Request your registered name from the easyfind server:

Code: Select all

/usr/lib/web-admin/easyfind.pl getname

Return:

Not a HASH reference at /usr/lib/web-admin/easyfind.pl line 227

You may note a different line number here than in the previous post. That is because this is the original unchanged script on a B3 still running bubba OS software version 2.6.0.1

About $VAR1 : this is from the perl "Dumper" function. If the server side script matches the client script the hash will most likely be called '%response'

[Ubi]

The server end is written in PHP (and some Ruby but thats rewritten in PHP as well). The buildup of the JSON string is completely different from the Perl decoding.
But I'll take your debugging output and run it through the server code to see whats going on.

[Ubi]

so the x485 error seems to be correct: you "get" a name while the box is not registered. There is no name to be gotten and the server wont give it.

as for the HASH issue: the server does never returns a hash upon request. This seems to be a client problem.
But even so: you deliberately break the client-server interaction by turning on easyfind after deleting the entry in the server. The server code is not designed for this creativity and it gives a funny result. Why is this a problem?

[Gordon]

Like I said: for me the easyfind server is a black box.

And it is not so much a problem as it is ugly. The point being that when running easyfind on anything other than the original Squeeze based system I cannot use the web interface because it isn't there (yet). This is work in progress.

I also think you're missing the point about the cause of the bad return. If the B3 never had an easyfind name, in which case I presume the database entry will be holding a NULL value, than the response is an error state with a message attached to it. If the B3 did have a name earlier and it was wiped the response is an empty record that causes an error in the client script. I'm assuming that in this case the value of the name field is a zero length string, but it could be the other way around.

About that client script:
Returning to the original code I noticed that the original return for the get_key function was prepended by a return that URL escapes the key. I also remember you making some comment about it a while ago, stating that some keys (including the one from your own B3 if I remember correctly) were having issues because they were not escaped correctly - or so you assumed. You also repeated that statement in this thread, but these values are posted and therefore do not need to be URL escaped. My bet is that you may in fact be experiencing the same "user" error I had. So here's a patch that you may try and see if it fixes that issue:

Code: Select all

--- a/easyfind.pl	2015-02-19 16:49:51.000000000 +0100
+++ b/easyfind.pl	2015-02-19 16:52:45.000000000 +0100
@@ -135,8 +135,9 @@
 	my $ba = parse_boot();
 
 	if($$ba{"key"}){	
-		return CGI::escape($$ba{"key"});
-		return ($$ba{"key"});
+		$key = ($$ba{"key"});
+		$key=~s/\s//g;
+		return $key;
 	}elsif(-e KEY){
 		open INFILE, "< ".KEY or die "Can't open ".KEY." : $!";
 		@a_key = <INFILE>;

[Ubi]

I also think you're missing the point about the cause of the bad return. If the B3 never had an easyfind name, in which case I presume the database entry will be holding a NULL value, than the response is an error state with a message attached to it.

Again, this is not how it works. The validation tables and the dns tables are not the same data system. THe production is just a lookup service.

were having issues because they were not escaped correctly - or so you assumed.

Yes, and in contrast to your assumption, I verified this checked afterwards. THe problem is that sometimes the B3 uses GET (which is encrpted) and sometime POST (which is not) but the server end does not realize this. It documented in the other thread. However this is completely unrelated to your code, as you are looking at CLIENT code and I was discussing SERVER code. The client does not get to decode the key from the server as it does not receive one.

Here's wat a successful return looks like:

Code: Select all

{"error":false,"opcode":1,"operation":"IPupdate","record":{"id":"1905","domain_id":"1","name":"ubi.myownb3.com","type":"A","content":"211.131.66.67"}}

See: there is no key

As you mentioned, here's indeed the reason why it could not find your key, as you entered a windows-style newline in the key:

Code: Select all

SELECT * FROM units WHERE mac1='00:22:02:00:##:22' and secret='##########bOkZbF1zlFXsOVOBU=\r\n'

after that it works without errors.
(yes: mac1 in the DB table is mac0 on the system. I didnt make this up).

As for units 15377, 16144 and 16742, there is no mention in the logs of these boxes ever announcing themselves.

yes the client code is ugly in returning inconsistent answers, but it is only ugly if you deliberately break the communication. Apart from your sense of aesthetics, why is this a problem?

I changed the error codes a bit to be more informative. I also added a unique identifier to the server code that is embedded in the server log as well in the JSON output. I propose that future discussions always contain these uids so all knwo we're talking about the same stuff without posting secrets.

[Ubi]

ah, fun fact: the client cashes the server response. If I stop the server the command "/usr/lib/web-admin/easyfind.pl getname" still works

Code: Select all

root@korona:~# curl http://easyfind.mybubba.org
curl: (7) couldn't connect to host
root@korona:~# /usr/lib/web-admin/easyfind.pl getname
{"ip":"82.161.206.19","name":"ubi7.mybubba.org","error":"false"}

[Gordon]

I also think you're missing the point about the cause of the bad return. If the B3 never had an easyfind name, in which case I presume the database entry will be holding a NULL value, than the response is an error state with a message attached to it.

Again, this is not how it works. The validation tables and the dns tables are not the same data system. THe production is just a lookup service.

I'm sensing you're trying to make a discussion out of this. It's not. As stated, I'm looking at a black box and this black box is asking me to provide a key and a mac address and feeds me information that states "production database". I'm not talking DNS entries, because in both described situations there is (or should be) no DNS entry. The difference is a correct return that no name is set versus an empty record that does not match what the client expects in return. And yes I'm being a user here and I'm doing things that the original programmer did not foresee. I don't think that is you, so I'm not getting why you're making such a fuzz about it and considering it like a personal attack. It's not. It's a bug. Or a feature, depending on whether you think that people using software in a different way than intended should be scared off with random non informative error messages.

were having issues because they were not escaped correctly - or so you assumed.

Yes, and in contrast to your assumption, I verified this checked afterwards. THe problem is that sometimes the B3 uses GET (which is encrpted) and sometime POST (which is not) but the server end does not realize this. It documented in the other thread. However this is completely unrelated to your code, as you are looking at CLIENT code and I was discussing SERVER code. The client does not get to decode the key from the server as it does not receive one.

I don't see this in the client code. It's POST only, both in the perl script called by web-admin and the DHCP hook, and in the python twistd service script. AFAIK the called methods do not include a fallback method to try GET if the original call using POST fails. Older software versions might have used GET method though, but I have no way of verifying that assumption.

As you mentioned, here's indeed the reason why it could not find your key, as you entered a windows-style newline in the key:

Code: Select all

SELECT * FROM units WHERE mac1='00:22:02:**:**:**' and secret='yadda-yadda=\r\n'

after that it works without errors.
(yes: mac1 in the DB table is mac0 on the system. I didnt make this up).

That was apparent from the first db entry you quoted. I'm quite amazed though to see that there is a 0x0D character in the submitted key as well. It's not like I'm running Windows on my B3. That stuff comes straight from the u-boot environment and I definitely do not expect any linux application that grabs that entry to attach anything other than a 0x0A character to it for displaying. Obviously I would not have wanted it to add anything, but then I would also never consider that any server could ever accept a newline character as valid input for a string that is supposed to contain printable characters only. That said: if I copy-paste the fixed length response key from my bank with a trailing space to transfer some money it will also state that the input is incorrect. It always amazes me how appropriate the saying "in the land of the blind, the one-eyed man is king" is for just about everything that is computer related.

ah, fun fact: the client cashes the server response. If I stop the server the command "/usr/lib/web-admin/easyfind.pl getname" still works

Code: Select all

root@korona:~# curl http://easyfind.mybubba.org
curl: (7) couldn't connect to host
root@korona:~# /usr/lib/web-admin/easyfind.pl getname
{"ip":"82.161.206.19","name":"ubi7.mybubba.org","error":"false"}

That's what I said put me on the wrong track. That information is fed from the easyfind conf file.

[Gordon]

Oh and that's no offence meant as well. I too make my living in IT. But there are a lot of idiots roaming this business.

[Ubi]

The thing is that I cant really get the code to filter out these \n\r characters as they could literally by "\n\r".

Now that we're making progress, can we recap what the remaining current problem is (there still is one as some clients still cannot seem to connect)

My proposal is that we
* create a reproducible test case
* list observations from server and client end
* fix it.

My preference would be to not hide any credentials or keys during this fix. If you can test the client side (as you have a box which we know misbehaves), I have no problem swapping the motherboard afterwards with a new replacement.

[Gordon]

Okay,

I just ran some verifications and it turns out that both the carriage return and newline are not in the u-boot environment, nor being produced by any of the tools to transfer them to userspace (through command line in the original kernel or u-boot-tools in sakaki's kernels). It is in fact perl running the easyfind script that adds these characters and then requires you to strip them. And the same happens in python as well, where I also noticed at least a newline being appended.

Currently all systems I have here and that contain disks verify correctly if I trim the key (I noticed you added some debugging lines on the server). However, except for one of them I can't get to the key using the original method of parsing the command line and I am using a fallback method that I injected into the script. I'll try and see if I can boot one of these machines using the original Bubba OS and verify if it still can connect.

The only issue I'm seeing at this moment is the "NOHASH" error. But this is no client misbehaviour. Both machines that are currently showing this error can create/verify/delete a myownb3 name.