Gentoo live USB for B3 (with Linux 3.15.9) released

Discuss development on Bubba
Post Reply
sakaki
Posts: 171
Joined: 15 Aug 2014, 11:20

Gentoo live USB for B3 (with Linux 3.15.9) released

Post by sakaki » 21 Aug 2014, 09:56

Hello

I've just released a live-USB image of Gentoo Linux for the B3 on GitHub (please search there for "gentoo-on-b3", the board won't let me post the link directly). It uses kernel version 3.15.9 from the mainline, with the necessary code to switch off the L2 cache prepended, and the kirkwood-b3 device tree blob appended.

You can burn the supplied image to a USB key (>=4Gb), then boot your B3 from it, without affecting any installed system on your B3's hard drive. You can even boot a diskless B3! No soldering, compilation or U-Boot flashing is required.

It's a full (persistent) armv5tel Gentoo system, not simply a 'minimal install' or 'stage3', so you can run emerge operations (Gentoo's equivalent of apt-get) etc. immediately.

Full instructions are provided on the GitHub page (including how to specify initial network settings, so you can ssh in once booted).

If you get a chance to try it, please let me know how you get on ^-^

best,
sakaki

Gordon
Posts: 1312
Joined: 10 Aug 2011, 03:18

Re: Gentoo live USB for B3 (with Linux 3.15.9) released

Post by Gordon » 22 Aug 2014, 13:19

Nice job!

My favourite Linux ;) Not being a binary distribution, it would be nice though to have at least hostapd, samba, avahi, web- and ftp services pre built in the install image to offer a quick start. Compiling this all yourself will probably take days to complete.

sakaki
Posts: 171
Joined: 15 Aug 2014, 11:20

Re: Gentoo live USB for B3 (with Linux 3.15.9) released

Post by sakaki » 23 Aug 2014, 05:40

Hi Gordon

no problem - I'm happy to spin a v1.1 for those not using distcc!

So, here's a strawman for the additional @world packages to include - anyone else got any strong views about something they'd like precompiled in the image? (Please add your thoughts by Wednesday 27th Aug; I'll lock off the image then & aim to release next weekend):
  • net-ftp/proftpd
  • net-wireless/hostapd
  • net-fs/samba
  • net-dns/avahi
  • www-servers/apache
  • mail-mta/postfix
  • net-misc/openvpn
  • net-firewall/iptables
(all with default USE flags (except proftpd, which will have "openssl" in package.use); stable-branch arm versions, except hostapd, which is **; packages will be supplied in 'freshly emerged' state, up to the end user to configure)

best,
sakaki

MouettE
Site admin
Posts: 248
Joined: 06 Oct 2011, 19:45

Re: Gentoo live USB for B3 (with Linux 3.15.9) released

Post by MouettE » 23 Aug 2014, 15:42

sakaki wrote:No soldering, compilation or U-Boot flashing is required.
Where is the fun ;-) ?

This is an amazing work, extremly well documented. Congratulation ; I will certainly look on everything for inspiration...

RandomUsername
Posts: 904
Joined: 09 Oct 2009, 18:49

Re: Gentoo live USB for B3 (with Linux 3.15.9) released

Post by RandomUsername » 24 Aug 2014, 12:58

I haven't been very active on these forums lately but have been following all these "alternative OS" projects and just want to say well done and thanks to you and everyone else involved with these projects.

past_51
Posts: 22
Joined: 25 Aug 2014, 06:27

Re: Gentoo live USB for B3 (with Linux 3.15.9) released

Post by past_51 » 25 Aug 2014, 08:05

Hi,

The same for me.
I'm not very active, but keep an eye on all these projects and would like to thank you for all your great stuff and your time.

sakaki
Posts: 171
Joined: 15 Aug 2014, 11:20

Re: Gentoo live USB for B3 (with Linux 3.15.9) released

Post by sakaki » 26 Aug 2014, 06:57

Thank you all for your kind comments! And many thanks also to those of you who have tried out the image thus far ^-^

Based on feedback received by email (and Gordon's post), the current 'precompiled' package list for the v1.1 now stands as follows:
  • app-admin/logrotate-3.8.7
  • app-admin/syslog-ng-3.4.8
  • app-misc/screen-4.0.3-r6
  • app-portage/eix-0.29.3
  • app-portage/euses-2.5.9
  • app-portage/gentoolkit-0.3.0.8-r2
  • app-portage/mirrorselect-2.2.2
  • dev-embedded/u-boot-tools-2014.01
  • mail-mta/postfix-2.10.3
  • net-dns/avahi-0.6.31-r2
  • net-firewall/iptables-1.4.21-r1
  • net-fs/samba-3.6.23
  • net-ftp/proftpd-1.3.5
  • net-mail/dovecot-2.2.13-r1
  • net-misc/bridge-utils-1.5
  • net-misc/dhcpcd-6.4.3
  • net-misc/mediatomb-0.12.1-r8
  • net-misc/minidlna-1.1.3
  • net-misc/netifrc-0.2.2
  • net-misc/openvpn-2.3.3
  • net-wireless/hostapd-2.0-r1
  • net-wireless/iw-3.10
  • net-wireless/wireless-tools-30_pre9
  • sys-apps/mlocate-0.26-r1
  • sys-apps/pciutils-3.2.0
  • sys-fs/dosfstools-3.0.26
  • sys-libs/e2fsprogs-libs-1.42.10
  • sys-process/cronie-1.4.11-r1
  • www-servers/apache-2.2.27-r4
Plus of course the normal @system set of tools.

The version numbers are Gentoo ebuilds, but they generally map 1-to-1 onto upstream package versions.

I'm also planning to bump the kernel to 3.16.1 from gentoo-sources.

Please send any further requests (or add them to this thread) by close of play tomorrow (Wed 27 Aug) as I'll lock the image off then. It should be available on GitHub sometime during the weekend.

Best,
sakaki

Gordon
Posts: 1312
Joined: 10 Aug 2011, 03:18

Re: Gentoo live USB for B3 (with Linux 3.15.9) released

Post by Gordon » 27 Aug 2014, 12:03

Let's link that project

Second: I'm currently not in a position where I can run any experiments on my B3, but I have been reviewing the install procedure. Specifically partitioning drew my attention and I wondered whether that would make any difference, i.e. why you would override the root partition to be at sda3.

I then noticed that you have root formatted as ext4, which explains why you can't share it with /boot in sda1 as u-boot doesn't support ext4. I then ran a benchmark on my WD20EARS drive and noticed two things: first the drive never exceeds read speeds over ~81MB/s which appears to be a limitation enforced by the Kirkwood SoC since it keeps doing so up to ~1.4G. Second is that close to the end of the drive, where Bubba OS placed the swap partition, speed drops down to ~50MB/s, so it would appear that the default partitioning that is done by the Bubba OS install is in fact not optimal for performance.

What I also did notice is that you did not split /home from the root partition. Which, if you would, would make it more sensible to put swap in between root (sda2) and /home (sda4) as this will most likely result in better seek times from your drive (obviously only applies for mechanical drives). I guess the root partition will have to be significantly bigger that 10G from Bubba OS though. Checking disk usage on the portage tree I'm counting 16G already :shock:

Edit: One more thing. Since the regular disks in the B3 are GPT'd, you'll probably want to use gdisk rather than fdisk for partitioning.

sakaki
Posts: 171
Joined: 15 Aug 2014, 11:20

Re: Gentoo live USB for B3 (with Linux 3.15.9) released

Post by sakaki » 28 Aug 2014, 07:51

Hi Gordon

Thanks for taking the time to look over the documentation!

yes, I did think about reusing the existing /dev/sda1 as the Gentoo root (for those users who want to put Gentoo on their internal drive, rather than simply run it from the 'live USB'). However, 10G is quite tight for a Gentoo machine. As you note, /usr/portage can get quite large - the ebuild tree (uncompressed) takes about 1G, but by default Gentoo also keeps tarballs for every source package ever emerged (in /usr/portage/distfiles), so unless you run eclean regularly it can easily top 10G!

Otherwise, the setup is standard Gentoo handbook stuff - just trying to keep things simple ^-^ As such I've assumed the user is overwriting the disk with a DOS partition table (o in fdisk). However, the version of fdisk on the image can create GPT partition tables too, and there's nothing to prevent a user doing that if they like. Or creating LVM over LUKS etc!

Of course I guess most people will simply run it from the USB key to have a play with Gentoo, without altering their existing Excito system.

Version 1.1.0 of gentoo-on-b3 (GitHub)

This is now locked off - thanks to everyone who sent comments or tried out the 1.0.0 image. I'll release it within the next couple of days on GitHub, and post a message here when I do.

How to Cross-Compile a B3 Kernel on a Gentoo PC

Lastly, for anyone with Gentoo on their home machine, spinning a custom kernel for the B3 is easy, via Gentoo's crossdev (and a lot faster than doing it on the B3 itself! 5 minutes vs 4 hours, in my case). Should you want to build a kernel on your Gentoo PC (for your B3 that is), here's what you do.

First, since crossdev uses the first overlay it finds as home, if you use layman, then you should do the following:

Code: Select all

gentoo_pc ~ # mdkir /usr/local/portage
gentoo_pc ~ # echo 'PORTDIR_OVERLAY="/usr/local/portage $PORTDIR_OVERLAY"' >> \
  /etc/portage/make.conf
Next, if you haven't already done so, convert the relevant /etc/portage files (package.accept_keywords etc.) to directories. You can run the following script to do this automatically (credit: MobileAPES):

Code: Select all

#!/bin/bash
PROFILE_DIR="/etc/portage"

if [ ! -e ${PROFILE_DIR} ]; then
    mkdir ${PROFILE_DIR};
fi;

MYCHOST=$(gcc-config -c | sed -e 's/\-[0-9]*\.[0-9\.]*.*$//g')

for PACK_DIR in package.keywords package.use package.unmask package.mask \
    package.license package.accept_keywords; do
    CUR_DIR="${PROFILE_DIR}/${PACK_DIR}"
    if [ ! -e ${CUR_DIR} ]; then
        mkdir ${CUR_DIR}
    fi

    if [ -e ${CUR_DIR} -a ! -d ${CUR_DIR} ]; then
        mv ${CUR_DIR} ${CUR_DIR}.moving
        mkdir ${CUR_DIR}
        mv ${CUR_DIR}.moving ${CUR_DIR}/${MYCHOST}
    fi
done

echo "Completed!"
Install crossdev and the uboot tools (for mkimage) if you don't already have them:

Code: Select all

gentoo_pc ~ #  emerge -av sys-devel/crossdev dev-embedded/u-boot-tools sys-fs/dosfstools 
Next, let crossdev build a cross-toolchain for our target architecture, using 'stable branch' tools (gcc etc.)

Code: Select all

gentoo_pc ~ #  crossdev --stable --target armv5tel-softfloat-linux-gnueabi
(It may complain about '/usr/local/portage/metadata/layout.conf' not existing, but crossdev will create and populate the file for you.)

Now grab some kernel sources. Let's say you wanted 3.16.1. As of the time of writing, this is not stabilized for the arm architecture on Gentoo, so we'd need to issue:

Code: Select all

gentoo_pc ~ #  echo "sys-kernel/gentoo-sources ~arm" >> \
  /etc/portage/package.accept_keywords/cross-armv5tel-softfloat-linux-gnueabi
And you need the 'freedist' license:

Code: Select all

gentoo_pc ~ #  echo "sys-kernel/gentoo-sources freedist" >> \
  /etc/portage/package.license/cross-armv5tel-softfloat-linux-gnueabi
Grab the sources (note we use --nodeps here; we don't actually want to cross-emerge the perl etc. tools that'll be used by Gentoo to patch the sources, you already have them in your native arch)

Code: Select all

gentoo_pc ~ #  ARCH="arm" armv5tel-softfloat-linux-gnueabi-emerge \
  -av --nodeps =sys-kernel/gentoo-sources-3.16.1
That's all the one-off steps out of the way. Now, go into the source directory (crossdev automatically keeps things separate from your normal sysroot, to avoid pollution):

Code: Select all

gentoo_pc ~ #  cd /usr/armv5tel-softfloat-linux-gnueabi/usr/src/linux-3.16.1-gentoo
Grab a suitable starter config from somewhere, such as off of my GitHub project, save it to .config, and sanitize it:

Code: Select all

gentoo_pc linux-3.16.1-gentoo #  cp <your config file> .config
gentoo_pc linux-3.16.1-gentoo #  make ARCH="arm" \
  CROSS_COMPILE="armv5tel-softfloat-linux-gnueabi-" olddefconfig
(Don't forget the hyphen at the end of armv5tel-softfloat-linux-gnueabi-!) Now set any configuration options you like in there (such as built-in command line etc):

Code: Select all

gentoo_pc linux-3.16.1-gentoo #  make ARCH="arm" \
  CROSS_COMPILE="armv5tel-softfloat-linux-gnueabi-" menuconfig
To build it, copy the following script into /root/build_image.sh and make it executable:

Code: Select all

#!/bin/bash
# Prepare B3 image, switching off cache, and appending DTB; also
# builds the modules. Results go to deploy_root directory.
# Execute in top-level kernel directory.
# Copyright (c) 2014 sakaki
# License: GPL 3.0+
# NO WARRANTY
set -e
set -u
DR="deploy_root"
die() { cat <<< "$@" 1>&2; exit 1; }
if [ ! -d "${DR}" ]; then
    echo "Creating directory: ${DR}"
    mkdir "${DR}"
fi
echo "Cleaning prior ${DR}..."
if [ -d "${DR}/boot" ]; then
    rm -rf "${DR}/boot"
fi
if [ -d "${DR}/lib" ]; then
    rm -rf "${DR}/lib"
fi
if [ -s "${DR}.tar" ]; then
    rm -f "${DR}.tar"
fi

NUMTHREADS=$(( $(grep -E 'processor\s+:' /proc/cpuinfo | wc -l) + 1 ))
KNAME="$(basename "${PWD}")"
KNAME="${KNAME/linux-/Linux }"

echo "Compiling kernel (${KNAME})..."
make -j${NUMTHREADS} ARCH=arm CROSS_COMPILE=armv5tel-softfloat-linux-gnueabi- \
  zImage
echo "Compiling DTB..."
make ARCH=arm CROSS_COMPILE=armv5tel-softfloat-linux-gnueabi- kirkwood-b3.dtb
echo "Compiling modules..."
make -j${NUMTHREADS} ARCH=arm CROSS_COMPILE=armv5tel-softfloat-linux-gnueabi- \
  modules
echo "Installing modules to ${DR}..."
make ARCH=arm CROSS_COMPILE=armv5tel-softfloat-linux-gnueabi- \
  INSTALL_MOD_PATH="${DR}" modules_install
echo "Creating patched image for B3 (caches off, DTB appended)..."
pushd arch/arm/boot
# per Debian workaround #658904 (credit: Ian Campbell)
echo -n -e \\x11\\x3f\\x3f\\xee >  cache_head_patch
echo -n -e \\x01\\x35\\xc3\\xe3 >> cache_head_patch
echo -n -e \\x11\\x3f\\x2f\\xee >> cache_head_patch
echo -n -e \\x00\\x30\\xa0\\xe3 >> cache_head_patch
echo -n -e \\x17\\x3f\\x07\\xee >> cache_head_patch
cat cache_head_patch zImage dts/kirkwood-b3.dtb > zImage-dts-appended
rm cache_head_patch
mkimage -A arm -O linux -T kernel -C none -a 0x00008000 -e 0x00008000 \
  -n "Gentoo ARM: ${KNAME}" -d zImage-dts-appended ../../../uImage
popd
echo "Final uImage has been created, copying to ${DR}..."
mkdir -p "${DR}/boot"
cp -v uImage "${DR}/boot/uImage"
cp -v System.map "${DR}/boot/System.map"
cp -v .config "${DR}/boot/config"
find "${DR}/lib/modules" -type l -name 'build' -delete
find "${DR}/lib/modules" -type l -name 'source' -delete
echo "Creating tarball..."
tar cf "${DR}.tar" "${DR}"
echo "Syncing filesystems, please wait..."
sync
echo "All done!"
This script deals with all the unpleasant L2-caches-off code, device tree blob etc. Now run it! This does a parallel compile so on a modern PC, it shoudn't take long (5-10 mins):

Code: Select all

gentoo_pc linux-3.16.1-gentoo # /root/build_image.sh
When done, you'll have a new directory deploy_root, which will contain a directory boot (with the uImage, System.map and config in it), and lib (with the modules and firmware).

Deploy these to your (Gentoo) B3 and off you go! (You can repeat the menuconfig / build_image.sh steps as often as you need, to refine the kernel).

Obviously, proceed at your own risk: only do this if you have a reasonable idea of what you are doing! And make sure you have a rescue system on hand. Don't delete your old /boot/uImage (just rename it); and leave your old /lib/modules/<...> in place), in case you need to fall back.

Note also that this will only work with version >=3.15 of the kernel, as that is when the B3's device tree data was integrated into the mainline (thereby obviating the need for an Excito-patched kernel).

3.16.1 on a Debian Squeeze B3

BTW, and I mention this more as a curiosity than anything else, I've managed to boot the standard Debian 'squeeze' B3 (no Gentoo) from a 'stock' 3.16.1 kernel using this method (the only difference being, that I downloaded the sources using sys-kernel/vanilla-sources, so no Gentoo patch set was applied). Most things seem to work (e.g., the Excito interface web GUI comes up), but I'm not a Debian hacker. Of course, some things are broken. For example, the LEDs are now accessed via the /sys paths defined in the device tree (rather than via Excito's patches, which obviously aren't applied), so as things stand, the front light doesn't go blue when you boot, etc.)

If anyone is interested, I can post that kernel and module set on GitHub too, just let me know.

best
sakaki

Binkem
Posts: 388
Joined: 10 Jul 2008, 02:26

Re: Gentoo live USB for B3 (with Linux 3.15.9) released

Post by Binkem » 28 Aug 2014, 08:36

Hi Sakaki,

I think it would be good to post the kernel. It might help the people working on updating B3 to more recent Linux distributions. I wouldnt mind having my B3 run without the blue light :)

Martijn

sakaki
Posts: 171
Joined: 15 Aug 2014, 11:20

Re: Gentoo live USB for B3 (with Linux 3.15.9) released

Post by sakaki » 29 Aug 2014, 14:18

Hello,

just to let everyone know that v1.1.0 has now been released on GitHub. The new release has been upgraded to kernel 3.16.1, and has a bunch of precompiled apps provided too. As before, you can run it on a standard or diskless B3, and since it is a 'live USB', it will not affect your existing Excito system.

Please let me know if you have any problems. Best,

sakaki

Gordon
Posts: 1312
Joined: 10 Aug 2011, 03:18

Re: Gentoo live USB for B3 (with Linux 3.15.9) released

Post by Gordon » 30 Aug 2014, 05:28

Hi Sakaki

I'll check it out. I built a 3.16.1 gentoo kernel myself to run on versatilepb, so I can now run whatever anyone would want to throw in to this project in QEMU.

One thing I noticed when running the 1.0 image is that you made eth0 the main interface, which may be somewhat confusing as this is the WAN port on the B3. Obviously when using udev you can give any interface whatever name you like and thus let the LAN port that has the higher MAC address be eth0, but that does seem out of normal convention.

I don't suppose it is possible to only enforce a different root partition and let other cmdline parameters pass? It would be useful to run e.g. easyfind, although one could rewrite the original procedure to get the key value directly from mtd1 rather than cmdline.

sakaki
Posts: 171
Joined: 15 Aug 2014, 11:20

Re: Gentoo live USB for B3 (with Linux 3.15.9) released

Post by sakaki » 30 Aug 2014, 20:04

Hi Gordon,

the kirkwood-b3.dts device tree file cites the eth0/eth1 interfaces, so I guess that's where the phy binding comes from. Isn't eth0 (the lower MAC) the wan port on a stock/squeeze B3 as well though? Or is your point rather that using the wan port for initial login is potentially confusing to a user? (As it's easy to rc-update the net.eth1 interface (so it too comes up at boot), and modify /etc/conf.d/net to your liking, once logged in, I hadn't really thought about that - but I guess using the lan port would be a bit more natural.)

Regarding the kernel command line, I think the behaviour you want can be achieved by setting CONFIG_CMDLINE_EXTEND rather than CONFIG_CMDLINE_FORCE. Need to check this.

Also, I think it might be useful to do a spin with all the systemd pre-reqs turned on in the kernel (even though the image uses OpenRC init, some people may want to migrate)... unfortunately, looks like I'm going to be snowed with (non-Excito-related) work for the next week or so, but when I get some time free, I'll look into it!

And right now, I have to un-brick my husband's B2 ... which I appear to have just fried ><

Best,

sakaki

Gordon
Posts: 1312
Joined: 10 Aug 2011, 03:18

Re: Gentoo live USB for B3 (with Linux 3.15.9) released

Post by Gordon » 31 Aug 2014, 04:17

Yes. I don't really know why WAN and LAN are organised the way they are on the outside - it's probably related to aesthetics - but the regular Bubba system will use eth0 for WAN and eth1 for LAN. Using the rescue system however, you are supposed to connect to the B3 through WAN, which is essentially what your Live system does as well but IMO should be reversed when installing to disk.

Systemctl still seems somewhat unstable to me. It's quicker but doesn't always follow dependencies the way you'd expect. For instance on my desktop system I always need to restart samba if I want access from a (virtual) Windows system, because at boot it only gets a localhost IPV6 binding. Should actually reconfigure that system also, as I originally installed Sabayon on it and their installer put root on /dev/mapper, which is a total nuisance when upgrading the kernel because you also need an initrd. And I also think moving to btrfs for /home will be a great improvement over mdadm.

Post Reply