New user's registration have been closed due to high spamming and low trafic on this forum. Please contact forum admins directly if you need an account. Thanks !

Debian jessie image 1.0 released for Bubba|2 / B3

Discuss development on Bubba
Post Reply
MouettE
Site admin
Posts: 341
Joined: 06 Oct 2011, 19:45

Debian jessie image 1.0 released for Bubba|2 / B3

Post by MouettE »

Hello everyone,

At long last I finally finished working on the minimal jessie image for both Excito Bubba|2 and B3. These images are constructed from 3 different projects : the jessie OS image itself, a new install/rescue system and a new installer script. More details on these further in this post. First the links and a quick howto:
Quick install howto

The image works in a very similar way from the classic excito images. Please note that there is no need to update U-Boot on the B3 and that created partitions will be well-aligned for SSD users. I've tried to test it the best that I could but everything is brand new so do not expect it to work flawlessly. BACKUP BACKUP BACKUP if you have data that you intend to keep on the server.
  • Unzip the downloaded file on a fat-formatted usb key with a partition table
  • By default the installer will wipe the entire disk. If you wish to keep the existing data of your device, edit the install.ini file inside the install directory and set wipe to false at the bottom of the file. The installer will then only format the first partition of the drive.
  • plug the usb key on the Bubba|2 or B3 and apply power while pushing the rear button.
  • The rescue/install system will start, format the drive, extract the jessie image and reboot the server.
  • By default the network is configured for dhcp on both network interface. It can be changed in the install.ini file (note that these settings only apply to the rescue/install system ; installed system will always use dhcp on both interfaces, see below).
  • If the LED turns red, something went wrong. Turn off the server and post the install.log file which is created in the install directory of the key so I can help you out.
  • When the LED is solid blue you can ssh into the box with user 'excito' (no quotes) and password 'excito' (no quotes). Then you can 'su' ; the root password is 'excito' (no quotes).
  • Once you're logged in I strongly recommend to set the locale and timezone of your server (its defaults are POSIX and UTC). Run the following commands :

    Code: Select all

    dpkg-reconfigure locales
    dpkg-reconfigure tzdata
    
OS image contents

The jessie OS is a very minimal debian install with only the core packages needed to start and ssh into the server. The image was bootstraped directly from standard debian utils (not built from upgrade of a previous version). There is a github repository which describes all the steps taken to build it.
  • The kernel used is Linux LTS 3.18.x series with Excito patches. I've choosen this version because it's LTS and close from the one debian uses in jessie (3.16). Sources are available on github
  • Excito-specific packages (kernel, led management) are available in a repository I maintain on the Excito servers. The image is pre-configured to use it so further upgrades can be installed with apt tools. Everything is on github too.
  • U-Boot tools are installed and configured so you can play with u-boot if you dare. WARNING this is risky business, use it at your own risk.
  • The root password is 'excito' (without quotes) and I've also created an excito user with the same password. Note that by default in jessie you can not ssh directly as root with a password.
  • SSH host keys are generated on the first boot and it takes a few seconds.
  • The network configuration is DHCP for both interfaces.
  • Should you be allergic to systemd, the excito packages _SHOULD_ work with systemV but I haven't tested them. The feedback is welcome (or not so I can remove all these complicated init scripts).
The install/rescue system

For this release I decided to build a new install/rescue system with a recent kernel and set of packages. It's based on the latest 2015.11 Buildroot and uses a 3.18.21 linux kernel. I've tried to include the tools I considered useful for both install and troubleshooting (partitioning/formatting, lvm/raid, ecryptfs, btrfs, debootstrap, etc.) while keeping it small (b3's install/rescue system is 15mb compressed). Full sources and rudimentary compilation instructions are available on github.

The new installer script

At this point there was no good reason not to write a new install script. Initially I tried to write a full-fledged installation system, with raid and lvm support, u-boot manipulation but it was too complicated and far too risky so I changed my mind and lowered the specifications. It works in two modes :
  • wipe mode: in this mode the script will wipe the disk and create two partitions : one for the system and one for the swap. The sizes are 10G/1G by defaults and can be customized.
  • non-wipe : in this mode the script will not change the partition table. It will try to use the first partition as system (if it's large enough) and an exisiting swap partition. The system and the swap partitions will be formatted leaving the rest untouched.
The installer is configured through the install.ini file on the USB install key. It's heavily commented so all the options can be understood and changed accordingly.
The network configuration can also be changed in the same file.
On the B3, in rescue-only or if reboot is disabled, the install script on exit will also animate the LED to dictate the first IP address of the box. It's bonus 8) and will hopefully make it easier to know the DHCP-attributed address of the server.

Special thanks

I would especially like to thank sakaki for her support and all her work on arch and gentoo images which was extremly useful and specifically allowed this image to be available on the Bubba|2.

Conclusion

I have tried to test the most of it and I think it's fairly good for public release. However this is still a 1.0 version so there will probably will be a few bugs and useful evolutions for the next versions. Every bug report or suggestion is welcomed either here or as issues in the github repositories.

And ask as many questions as you want I will be more than happy to answer them.

Charles
MouettE
Site admin
Posts: 341
Joined: 06 Oct 2011, 19:45

Re: Debian jessie image 1.0 released for Bubba|2 / B3

Post by MouettE »

Please note that I also wrote a small C easyfind client and it's available on the repository as well (sources on github). It's not included in the image but you can install it with the following commands:

Code: Select all

apt-get update
apt-get install easyfind-client
In order to use it :
  • Register the desired domain with ef as root:

    Code: Select all

    ef myspecialname.myownb3.com
    
  • If registering was successful, start the daemon:

    Code: Select all

    systemctl start easyfind-client
    
It's not very well documented nor perfect yet but it does the job.
Gordon
Posts: 1461
Joined: 10 Aug 2011, 03:18

Re: Debian jessie image 1.0 released for Bubba|2 / B3

Post by Gordon »

MouettE wrote:Please note that I also wrote a small C easyfind client and it's available on the repository as well. It's not included in the image but you can install it with the following commands:
...
It's not very well documented nor perfect yet but it does the job.
Sounds promising. Likely substantially more lightweight than the original python based daemon. Does it feature an unregister command as well? Does it also allow update without starting the daemon?
MouettE
Site admin
Posts: 341
Joined: 06 Oct 2011, 19:45

Re: Debian jessie image 1.0 released for Bubba|2 / B3

Post by MouettE »

Forget to mention that the easyfind-client source are available on github too.
Gordon wrote:Likely substantially more lightweight than the original python based daemon.
That what the main reason I went for C despite the significantly longer development time. Still wondering if it's worth it though...
Gordon wrote:Does it feature an unregister command as well
Yes. To unregister you must stop the daemon and run ef with -d switch :

Code: Select all

systemctl stop easyfind-client
ef -d
Gordon wrote:Does it also allow update without starting the daemon?
Not by design but trying to re-register the same name may trigger an update. I'll look into that.
Gordon
Posts: 1461
Joined: 10 Aug 2011, 03:18

Re: Debian jessie image 1.0 released for Bubba|2 / B3

Post by Gordon »

The original clients trigger an update by only sending key and (eth0) mac. This is the default action for the perl client script and the only action the python script performs. I guess if the default action for your version is to display a help page you could add a new switch to it. Or maybe it would work to run the daemon in single execution mode (i.e. not as a daemon)?

I do get your point about whether it's all worth it. I tend to overcomplicate stuff as well and frequently end up with that question myself. Every little step does count though and what matters is the end result: a durable B3.
beaufils
Posts: 16
Joined: 22 Feb 2012, 04:24

Re: Debian jessie image 1.0 released for Bubba|2 / B3

Post by beaufils »

Thank's for the good work: it worked on my b3 which is now running Debian Jessie :-)

Longer step was ... backup before reinstalling.

Some stuff however is not completely functional:
  • changing the network configuration in the install/install.ini file does not change anything. When booting the network was still configured to be set up via DHCP although I modified the file adding this section:

    Code: Select all

    [lan]
    proto = static
    ipaddr = 192.168.42.1
    netmask = 255.255.255.0
    gateway = 192.168.42.254
    nameservers = 192.168.42.254
  • the previous data partition (which was a LVM volume) is not added in the /etc/fstab file and thus is not mounted.
However my b3 is now running Debian Jessie and configuration time is now begginning :-)

Thank's again for the very nice work.
MouettE
Site admin
Posts: 341
Joined: 06 Oct 2011, 19:45

Re: Debian jessie image 1.0 released for Bubba|2 / B3

Post by MouettE »

beaufils wrote:changing the network configuration in the install/install.ini file does not change anything. When booting the network was still configured to be set up via DHCP although I modified the file adding this section:

Code: Select all

[lan]
proto = static
ipaddr = 192.168.42.1
netmask = 255.255.255.0
gateway = 192.168.42.254
nameservers = 192.168.42.254
I just tested it again and it seems to work here. Could you open an issue in the github installer projet providing your full install.ini so I can look into it ?
beaufils wrote:the previous data partition (which was a LVM volume) is not added in the /etc/fstab file and thus is not mounted.
That is expected behaviour. I won't generate the fstab because it would be complicated and I would have no idea where to mount the drives I find. Leave it to the admin ! In any case if you have difficulties don't hesitate to ask for help.
beaufils wrote:However my b3 is now running Debian Jessie and configuration time is now begginning :-)
Goooood
johannes
Posts: 1470
Joined: 31 Dec 2006, 07:12
Location: Sweden
Contact:

Re: Debian jessie image 1.0 released for Bubba|2 / B3

Post by johannes »

Wow, worked flawlessly. Excellent work, as always. Was confused for a while - I thought it exited the installer and booted regularly, but it was just a really fast installer, nice!

Thanks a lot!
/Johannes (Excito co-founder a long time ago, but now I'm just Johannes)
ahoff
Posts: 105
Joined: 01 Apr 2008, 20:50
Location: Swe

Re: Debian jessie image 1.0 released for Bubba|2 / B3

Post by ahoff »

Impressive work. Thank you very much for your work.
Åke Hoff
Örskogen
Sweden
NisseJ
Posts: 40
Joined: 09 Jan 2011, 07:55
Location: Sweden
Contact:

Re: Debian jessie image 1.0 released for Bubba|2 / B3

Post by NisseJ »

Now my Main B3 is running Jessie. Thanks for your hard work. Well done. :)
Stryker
Posts: 56
Joined: 17 Oct 2013, 11:03

Re: Debian jessie image 1.0 released for Bubba|2 / B3

Post by Stryker »

Does the unzip-tarball-to-partition-Method of installation still work?
I prefer to setup the disk with gparted on another system and then simply dump the image-files on it.

Since you mentioned there were no changes done to the U-Boot, I figure the B3 still requires an ext2-filesystem with the kernel located at sda1.
MouettE
Site admin
Posts: 341
Joined: 06 Oct 2011, 19:45

Re: Debian jessie image 1.0 released for Bubba|2 / B3

Post by MouettE »

Stryker wrote:Does the unzip-tarball-to-partition-Method of installation still work?
I prefer to setup the disk with gparted on another system and then simply dump the image-files on it.
Absolutely.
Stryker wrote:Since you mentioned there were no changes done to the U-Boot, I figure the B3 still requires an ext2-filesystem with the kernel located at sda1.
Yes but ext3 works fine too (careful on the Bubba|2, there are some restrictions on inode size and extended attributes). You can even create a small /boot partition if you like (you just need a boot link to . inside)
Stryker
Posts: 56
Joined: 17 Oct 2013, 11:03

Re: Debian jessie image 1.0 released for Bubba|2 / B3

Post by Stryker »

EDIT: Important: If you decide to install by dumping the files on the disk and do not use the install-script, remember to generate SSH-Host-Keys and place them on the disk aswell.
This is something the script usually does so there are none present if you just extract the files.
I just wasted 2 hours figuring out why my B3 simply refused connection.

Code: Select all

ssh-keygen -t ed25519 -f ssh_host_ed25519_key < /dev/null
ssh-keygen -t rsa -b 4096 -f ssh_host_rsa_key < /dev/null
This will generate 2 key-pairs in your current working directory, one for RSA- and one for ed25519-encryption. I prefer not to use the other two schemes available (DSA and ECDSA).
Place them at /etc/ssh/ and this will allow you to login via ssh.
MouettE wrote:You can even create a small /boot partition if you like (you just need a boot link to . inside)
That was always something I hoped to achieve as a first step to enable full-disk-encryption on my NAS.
But my guess is that this will severely cut down on performance, because the processor does not have adequate hardware-crypto onboard.
I remember there being a thread on this forum about enabling the hardware-acceleration, but it only supports outdated crypto-schemes (aes-128-cbc I think).

I was actually researching how to enable a headless device to use full-disk-encryption when you cannot enter the password locally.
The key is to include a small ssh-server in the initial ramdisk where you can then login via ssh, enter the password to decrypt the root-partition and continue booting.
Gordon
Posts: 1461
Joined: 10 Aug 2011, 03:18

Re: Debian jessie image 1.0 released for Bubba|2 / B3

Post by Gordon »

Strange... I thought ssh generates those key files by itself if either one is missing. Quite sure it did on my system (did not have the ed25519 key when I transferred to Gentoo)
Stryker wrote:I remember there being a thread on this forum about enabling the hardware-acceleration, but it only supports outdated crypto-schemes (aes-128-cbc I think).
That is correct. The hardware crypto enabled version of openssl that you need for it is part of my bubba overlay for Gentoo, as is the conclusion that the hardware crypto engine does aes-cbc. This cypher has been disabled with current versions of openssl, but I don't know for what reason. The cypher itself may be weak, or the method to use it on the wire may be vulnerable. In the latter case it would still do the job if you want to use it for disk encryption, because this does not include the use of wired transfer. If you think this feature would be useful, you should definitely investigate this.

As for having a small boot partition, do be aware that the kernel needs to be told what the root partition is and that the default u-boot says this is /dev/sda1, i.e. the same partition that is the boot partition. Sakaki solved this by hardcoding the kernel parameters in the kernel itself, setting the root partition to be /dev/sda3. The other option is to change the parameters in the u-boot environment, but this is in reality a lot more difficult than it should be in theory. You will need a serial connector if you want to go this way.
sakaki
Posts: 172
Joined: 15 Aug 2014, 11:20

Re: Debian jessie image 1.0 released for Bubba|2 / B3

Post by sakaki »

That is correct. The hardware crypto enabled version of openssl that you need for it is part of my bubba overlay for Gentoo, as is the conclusion that the hardware crypto engine does aes-cbc. This cypher has been disabled with current versions of openssl, but I don't know for what reason. The cypher itself may be weak, or the method to use it on the wire may be vulnerable. In the latter case it would still do the job if you want to use it for disk encryption, because this does not include the use of wired transfer. If you think this feature would be useful, you should definitely investigate this.
Unlikely possibility, but if you think someone might tamper with your encrypted disk you should steer clear of cipher-block-chaining (cbc) due to known-plaintext malleability attacks (see this paper, for example).
In reality, more likely to be a concern on e.g. a PC which is dual booting with Windows, or a traveller's laptop in an evil maid attack, than on a Bubba.

Best. sakaki
Post Reply