Debian 9 (stretch)
Debian 9 (stretch)
As some of you may already know, the new debian version "stretch" (9.0) is currently in the soft freezing phase. The release date is not yet known but it's likely to be released in a couple of months. I will begin working shortly on porting this new version on the B3.
Bubba|2 users should be aware that debian has dropped support for the powerpc architecture. That means that there will be no release of debian 9 for the Bubba|2. Upgrades will also not be possible. However you will still be able to use the gentoo images from sakaki as an alternative to the original excito squeeze images. Debian jessie will probably be maintained for another couple of years after the stretch release and I will continue to support the kernel during this time.
I will post updates here as work is progressing. If some of you would like beta releases let me know and I'll make them available. I can already tell you that :
- upgrading existing B3 running jessie will be possible, following the standard debian procedure that will be published on release
- The kernel will probably be a 4.10 kernel as stretch should use it and it would also be an official LTS kernel
- The install/rescue system will remain unchanged
- Unless needed (which I doubt) the install script will remain unchanged. Howver as it's been a problem with many of you, I'll add the option to copy the installer network settings into the new system if I have time.
Re: Debian 9 (stretch)
Re: Debian 9 (stretch)
![Very Happy :D](./images/smilies/icon_e_biggrin.gif)
We really appreciate this!
Puma
Re: Debian 9 (stretch)
Is there a way we could reward you for your work?
Re: Debian 9 (stretch)
Re: Debian 9 (stretch)
![Very Happy :D](./images/smilies/icon_e_biggrin.gif)
https://www.debian.org/releases/stretch/
Puma
Re: Debian 9 (stretch)
Thanks for the update. We appreciate your work on the B3!
Enjoy your vacation first
![Very Happy :D](./images/smilies/icon_e_biggrin.gif)
Puma
Re: Debian 9 stretch
Mouette, enjoy your vacation, and thanks for all the work on maintaining Debian releases for B3 & co!
Now, the actual stretch release uses a 4.9 kernel, not 4.10[0]. Will the B3 stretch release be base on 4.9?
I see that there's a official Debian kernel image for marvell cpu:s[1]. Can that kernel package be used on a B3 as is? It would be great to run a kernel that is maintained by Debian, to ensure rapid and timely security updates in a sustainable way. — No critique implied or intended!
Will the work on a B3 stretch release - install usb image, kernel, other patches - be available on github? I'm no kernel hacker, but if I happen to find a scratch to itch, I'd love to be able to dive in, build stuff myself, and perhaps even supply patches. — A man can dream.
![Smile :)](./images/smilies/icon_e_smile.gif)
Let us know if there is anything we can do to help your efforts!
[0] https://tracker.debian.org/pkg/linux
[1] https://packages.debian.org/stable/kern ... ge-marvell
Re: Debian 9 (stretch)
Yes, currenlty I'm working on the latest 4.9.34 from upstream.
None taken. Yes I agree it would have its advantages especially from a security perspective. sakaki has tried to run the debian kernel using the tools she uses on gentoo to run the official gentoo kernel. She actually managed to run it but the hack is somewhat tricky. There are also some advantages to run a custom built kernel so I'm not decided yet, we'll see. It's likely that the image will initially be using a custom kernel to limit delay. Afterwards maybe I'll try to make the debian kernel work and publish updates.fredrikj wrote: ↑23 Jun 2017, 06:38 I see that there's a official Debian kernel image for marvell cpu:s[1]. Can that kernel package be used on a B3 as is? It would be great to run a kernel that is maintained by Debian, to ensure rapid and timely security updates in a sustainable way. — No critique implied or intended!
Yes everything will be on github as for the previous jessie release. All the details will be provided in the announcement post here.fredrikj wrote: ↑23 Jun 2017, 06:38 Will the work on a B3 stretch release - install usb image, kernel, other patches - be available on github? I'm no kernel hacker, but if I happen to find a scratch to itch, I'd love to be able to dive in, build stuff myself, and perhaps even supply patches. — A man can dream.
Re: Debian 9 (stretch)
Hi fredrikj,
so, for fun I've taken a quick look at this. Good news on one level: I've managed to get the 'official' Debian kernel (4.9.0-3-marvell) you referenced to boot a B3 successfully. This was on a Gentoo B3 system (I'm afraid I don't know much about Debian) - essentially a modified version of my current 2.0.0 live-USB. Everything seems functional (HDD, USB, WiFi etc.), at least on a quick review, so that's a plus - however, the process to get this working was a bit... involved. It would be automatable though, should anyone care to do that - via a post-install hook or whatever - so just in case it helps someone, I'll lay out the steps I took below.
Update: there's now a working special edition of my Arch Linux live-USB image available, using the official Debian linux-image-4.9.0-3-marvell kernel and module set, created using the techniques described in this post. Scroll to the bottom of this message to download and try it out!
Issues with the Debian Kernel Package (linux-image-<...>-marvell)
There are three main issues with the supplied package (linux-image-4.9.0-3-marvell), that have to be resolved in order to use it:
- The kernel is provided as a zImage (a compressed version of the Linux kernel that is self-extracting). By default, U-boot on the B3 is set up to deal with uImages for HDD booting, and FIT images for USB booting. Most people won't want to risk bricking their B3 by reflashing the U-boot settings (or version), meaning that the supplied zImage cannot be directly booted in its supplied form.
- Debian have used xz compression on their zImage (the file vmlinuz-4.9.0-3-marvell), and xz doesn't seem to work properly on the B3 - coming up with "out of memory" or "data is corrupt" errors on (kexec) boot. It's unclear to me exactly what causes this - but I have been able to replicate the issue on a known-good Gentoo kernel - boots fine with CONFIG_KERNEL_GZIP=y, fails with CONFIG_KERNEL_XZ=y. So it appears the image can't be used in its xz-compressed form (archlinuxarm.org use gzip on their distribution kernels, incidentally, as do all the live-USBs).
- Finally, unlike MouettE's custom kernels for the B3 (and indeed the other kernels provided by the various live-USBs available for the B3 through this forum), the Debian kernel doesn't have sufficient drivers configured as built-ins to mount the root partition (no HDD or USB visible by default), so it can't simply be started on its own.
Issue 1: Booting zImages Directly on the B3, without Reflashing U-Boot
To boot a zImage, without upgrading the U-boot version (to get access to bootz, which would also require a variable reflash), we could package it up as a uImage for HDD boot (which can simply be renamed as a FIT for USB boot), using mkimage (it isn't hard to do this, see for example my buildkernel-b3 script). However, this approach does not provide the user access to the kernel command line without modifying U-boot nvram variables. So, the approach I used (mirroring that on the latest Gentoo and Arch live-USBs for the B3) was to utilise an "interstitial" kernel. This is a (uImage) kernel that is started by U-Boot, and whose job it is (via its built-in initramfs) to act as a kind of second stage bootloader - setting up the kernel command line, loading the DTB and initramfs (if any), loading and patching the ("real") zImage kernel, and then jumping to the new kernel, using kexec.
In terms of drawbacks, this approach has only two: there is an increase in boot time (around 20 seconds) and the MACs of the B3's ethernet adaptors get cleared and must be re-configured in the final system userland, before bringing up networking.
OK so how do you build an interstitial kernel for yourself? The main thing to sort out is the initramfs. I'm going to assume you're doing the following as root on a PC with a cross-compilation toolchain installed (I have instructions for setting this up for a Gentoo PC here; it is straightforward to do in other distros too).
First, download kernel sources. It's not important what kernel you use really for the interstitial, as long as it is reasonably modern - it will only be around for a short time during boot. Sticking with our Gentoo PC system (adapt for your own cross-development OS), let's download the most recent gentoo-patched kernel sources to use:
Code: Select all
gentoo_pc ~ # armv5tel-softfloat-linux-gnueabi-emerge --nodeps ~sys-kernel/gentoo-sources
Code: Select all
gentoo_pc ~ # ln -s /usr/armv5tel-softfloat-linux-gnueabi/usr/src/{linux-4.11.7-gentoo,interstitial-gentoo}
gentoo_pc ~ # cd /usr/armv5tel-softfloat-linux-gnueabi/usr/src/interstitial-gentoo
Code: Select all
CONFIG_KERNEL_GZIP=y
CONFIG_BLK_DEV_INITRD=y
CONFIG_INITRAMFS_SOURCE="/usr/armv5tel-softfloat-linux-gnueabi/usr/src/interstitial-gentoo/initramfs"
Code: Select all
CONFIG_KERNEL_GZIP=y
CONFIG_BLK_DEV_INITRD=y
CONFIG_INITRAMFS_SOURCE="/usr/armv5tel-softfloat-linux-gnueabi/usr/src/interstitial-gentoo/initramfs-sda"
Let's put together the first of those super-minimal file structures (the second will be a copy with only minor changes, in the init). Do:
Code: Select all
gentoo_pc interstitial-gentoo # mkdir -pv initramfs/{bin,boot,dev/pts,etc,lib,mnt/root,proc,root,sbin,sys}
Code: Select all
gentoo_pc ~ interstitial-gentoo # cd initramfs/dev
gentoo_pc dev # mknod -m 600 console c 5 1
gentoo_pc dev # mknod -m 666 null c 1 3
gentoo_pc dev # mknod -m 660 sda b 8 0
gentoo_pc dev # mknod -m 660 sda1 b 8 1
gentoo_pc dev # mknod -m 660 sda2 b 8 2
gentoo_pc dev # mknod -m 660 sda3 b 8 3
gentoo_pc dev # mknod -m 660 sdb b 8 16
gentoo_pc dev # mknod -m 660 sdb1 b 8 17
gentoo_pc dev # mknod -m 660 sdb2 b 8 18
gentoo_pc dev # mknod -m 660 sdb3 b 8 19
gentoo_pc dev # mknod -m 666 zero c 1 5
gentoo_pc dev # mknod -m 666 ptmx c 5 2
gentoo_pc dev # mknod -m 666 tty c 5 0
gentoo_pc dev # mknod -m 444 random c 1 8
gentoo_pc dev # mknod -m 444 urandom c 1 9
gentoo_pc dev # chown -v root:tty {console,ptmx,tty}
gentoo_pc dev # chown root:disk sd*
gentoo_pc dev # cd ..
To build busybox on Gentoo and install into the initramfs working tree:
Code: Select all
gentoo_pc initramfs # USE="static mdev" armv5tel-softfloat-linux-gnueabi-emerge busybox
gentoo_pc initramfs # cp -a /usr/armv5tel-softfloat-linux-gnueabi/bin/busybox bin/
You can easily thin this down if you like, but its not really necessary, the static busybox binary is only about 1.7MB.
Set up some minimal links so that busybox can mimic enough apps to get our init script up and running (it will perform a call to create the rest of the links, but we need a basic set to even launch a shell):
Code: Select all
gentoo_pc initramfs # cd bin
gentoo_pc bin # for B in ash cat chmod cut dmesg echo flock ls mktemp mount mountpoint readlink realpath sh stty umount uname; do ln -s busybox "${B}"; done
gentoo_pc bin # cd ..
Code: Select all
gentoo_pc initramfs # mkdir -pv /usr/armv5tel-softfloat-linux-gnueabi/etc/portage/package.accept_keywords
gentoo_pc initramfs # echo '=sys-apps/kexec-tools-9999 **' > /usr/armv5tel-softfloat-linux-gnueabi/etc/portage/package.accept_keywords/kexec-tools
gentoo_pc initramfs # USE="-zlib" LDFLAGS="-static" armv5tel-softfloat-linux-gnueabi-emerge kexec-tools
gentoo_pc initramfs # cp -a /usr/armv5tel-softfloat-linux-gnueabi/sbin/kexec sbin/
Code: Select all
gentoo_pc initramfs # touch init; chmod +x init; nano -w init
Code: Select all
#!/bin/busybox sh
# Attempt to mount boot partition and source KSCRIPT within it.
# Copyright (c) 2016 sakaki <sakaki@deciban.com>
# License: GPL 3.0+
# NO WARRANTY
# Simple init script to source /boot/kexec.sh
# (the sourced script should setup and then kexec -e the final
# target kernel)
# If the specified partition cannot be found, starts recovery console
BOOT_DESC="UUID=EED5-410D"
DELAY=5
# We will try and exec the following script on the boot partition
KSCRIPT="/boot/kexec.sh"
show_red_led() {
# turn off green and blue LEDs
echo -n 0 > /sys/class/leds/bubba3\:green\:programming/brightness
echo -n 0 > /sys/class/leds/bubba3\:blue\:active/brightness
# turn on the red (error) LED
cat /sys/class/leds/bubba3\:red\:error/max_brightness > \
/sys/class/leds/bubba3\:red\:error/brightness
}
start_mini_udev() {
echo "Starting mini udev..."
echo "/sbin/mdev" > /proc/sys/kernel/hotplug
mdev -s
}
error_exit() {
# light the red led
show_red_led
echo "Starting recovery shell on serial console..." >&2
exec /bin/busybox sh -i
}
# Mount the /proc and /sys filesystems.
/bin/busybox mount -t proc none /proc
/bin/busybox mount -t sysfs none /sys
/bin/busybox mount -t devpts none /dev/pts
# ensure all symlinks are present
/bin/busybox --install -s
# start udev
start_mini_udev
sleep "${DELAY}"
echo "Trying to locate partition ${BOOT_DESC}..."
if ! BOOT_PATH="$(findfs "${BOOT_DESC}")"; then
echo "Error - could not find ${BOOT_DESC}!" >&2
error_exit
fi
echo "Mounting boot filesystem (read only)..."
if ! mount -o ro "${BOOT_PATH}" /boot; then
echo "Error - failed to mount ${BOOT_PATH}!" >&2
error_exit
fi
if [ -f "${KSCRIPT}" ]; then
echo "Sourcing ${KSCRIPT}..."
source "${KSCRIPT}"
else
echo "Error - ${KSCRIPT} not found!" >&2
error_exit
fi
# should not get here, KSCRIPT should have kexec'd a new kernel
echo "Error - ${KSCRIPT} has returned!" >&2
error_exit
This script basically just does the minimum to be able to source another file, kexec.sh, which must be located at the top level of the specifed (boot) directory - here UUID=EED5-410D. It is that file that will actually load and start the "real" kernel... but by keeping it outside the initramfs, we make our bootloader's behaviour user-editable (so that e.g. the command line can be changed, without having to recompile the interstitial kernel).
OK, we're done now, your initramfs filesystem should look something like this now (don't worry about the exact datestamps or file sizes, since the below is taken from my working tree):
Code: Select all
gentoo_pc initramfs # ls -lR
.:
total 44
drwxr-xr-x 2 root root 4096 Jun 30 2014 bin
drwxr-xr-x 2 root root 4096 Apr 7 2016 boot
drwxr-xr-x 3 root root 4096 Apr 7 2016 dev
drwxr-xr-x 2 root root 4096 Jun 30 2014 etc
-rwxr-xr-x 1 root root 1966 Jun 17 00:49 init
drwxr-xr-x 2 root root 4096 Jun 30 2014 lib
drwxr-xr-x 3 root root 4096 Jun 30 2014 mnt
drwxr-xr-x 2 root root 4096 Jun 30 2014 proc
drwxr-xr-x 2 root root 4096 Jun 30 2014 root
drwxr-xr-x 2 root root 4096 Apr 7 2016 sbin
drwxr-xr-x 2 root root 4096 Jun 30 2014 sys
./bin:
total 1676
lrwxrwxrwx 1 root root 7 Jun 30 2014 ash -> busybox
-rwxr-xr-x 1 root root 1713784 Apr 7 2016 busybox
lrwxrwxrwx 1 root root 7 Jun 30 2014 cat -> busybox
lrwxrwxrwx 1 root root 7 Jun 30 2014 chmod -> busybox
lrwxrwxrwx 1 root root 7 Jun 30 2014 cut -> busybox
lrwxrwxrwx 1 root root 7 Jun 30 2014 dmesg -> busybox
lrwxrwxrwx 1 root root 7 Jun 30 2014 echo -> busybox
lrwxrwxrwx 1 root root 7 Jun 30 2014 flock -> busybox
lrwxrwxrwx 1 root root 7 Jun 30 2014 ls -> busybox
lrwxrwxrwx 1 root root 7 Jun 30 2014 mktemp -> busybox
lrwxrwxrwx 1 root root 7 Jun 30 2014 mount -> busybox
lrwxrwxrwx 1 root root 7 Jun 30 2014 mountpoint -> busybox
lrwxrwxrwx 1 root root 7 Jun 30 2014 readlink -> busybox
lrwxrwxrwx 1 root root 7 Jun 30 2014 realpath -> busybox
lrwxrwxrwx 1 root root 7 Jun 30 2014 sh -> busybox
lrwxrwxrwx 1 root root 7 Jun 30 2014 stty -> busybox
lrwxrwxrwx 1 root root 7 Jun 30 2014 umount -> busybox
lrwxrwxrwx 1 root root 7 Jun 30 2014 uname -> busybox
./boot:
total 0
./dev:
total 4
crw------- 1 root tty 5, 1 Jun 29 2014 console
crw-rw-rw- 1 root root 1, 3 Jun 29 2014 null
crw-rw-rw- 1 root tty 5, 2 Apr 7 2016 ptmx
drwxr-xr-x 2 root root 4096 Apr 7 2016 pts
cr--r--r-- 1 root root 1, 8 Apr 7 2016 random
brw-rw---- 1 root disk 8, 0 Jun 29 2014 sda
brw-rw---- 1 root disk 8, 1 Apr 7 2016 sda1
brw-rw---- 1 root disk 8, 2 Apr 7 2016 sda2
brw-rw---- 1 root disk 8, 3 Apr 7 2016 sda3
brw-rw---- 1 root disk 8, 16 Apr 7 2016 sdb
brw-rw---- 1 root disk 8, 17 Apr 7 2016 sdb1
brw-rw---- 1 root disk 8, 18 Apr 7 2016 sdb2
brw-rw---- 1 root disk 8, 19 Apr 7 2016 sdb3
crw-rw-rw- 1 root tty 5, 0 Jun 30 2014 tty
cr--r--r-- 1 root root 1, 9 Apr 7 2016 urandom
crw-rw-rw- 1 root root 1, 5 Apr 7 2016 zero
./dev/pts:
total 0
./etc:
total 0
./lib:
total 0
./mnt:
total 4
drwxr-xr-x 2 root root 4096 Jun 30 2014 root
./mnt/root:
total 0
./proc:
total 0
./root:
total 0
./sbin:
total 616
-rwxr-xr-x 1 root root 628020 Apr 7 2016 kexec
./sys:
total 0
Next, make a copy of this initramfs, for use when booting from HDD - this will be nearly identical:
Code: Select all
gentoo_pc initramfs # cd ..
gentoo_pc interstitial-gentoo # cp -ax initramfs{,-sda}
Code: Select all
gentoo_pc interstitial-gentoo # sed -i 's#UUID=EED5-410D#/dev/sda1#g' initramfs-sda/init
Fetch the USB-based interstitial kernel config, and build it (with integral initramfs):
Code: Select all
gentoo_pc interstitial-gentoo # wget https://github.com/sakaki-/gentoo-on-b3/raw/2.0.0/configs/usb_interstitial_config
gentoo_pc interstitial-gentoo # cp usb_interstitial_config .config
gentoo_pc interstitial-gentoo # wget https://github.com/sakaki-/gentoo-on-b3/raw/2.0.0/reference/kirkwood-b3-earlyled.dts
gentoo_pc interstitial-gentoo # make ARCH=arm CROSS_COMPILE=armv5tel-softfloat-linux-gnueabi- olddefconfig
gentoo_pc interstitial-gentoo # buildkernel-b3 --usb --custom-dts="${PWD}/kirkwood-b3-earlyled.dts"
gentoo_pc interstitial-gentoo # mv deploy_root{,_usb}
OK, after this, you'll have a "deploy_root_usb" directory containing boot and lib directories. You can ignore the lib - we just need to copy the contents of boot into the first partition of the appropriate USB (in this case, the gentoo live-USB, since we have specified the first partition by UUID). This (install/install.itb) is the kernel that (your unmodified) U-boot is going to start (when your B3 is booted with the USB key inserted, and the rear button depressed).
Next, build a version for deployment on HDD (no --usb option):
Code: Select all
gentoo_pc interstitial-gentoo # wget https://github.com/sakaki-/gentoo-on-b3/raw/2.0.0/configs/b3_interstitial_config
gentoo_pc interstitial-gentoo # cp b3_interstitial_config .config
gentoo_pc interstitial-gentoo # make ARCH=arm CROSS_COMPILE=armv5tel-softfloat-linux-gnueabi- olddefconfig
gentoo_pc interstitial-gentoo # buildkernel-b3 --custom-dts="${PWD}/kirkwood-b3-earlyled.dts"
gentoo_pc interstitial-gentoo # mv deploy_root{,_sda}
We're nearly done - all we need now is a kexec.sh script to actually do the kexec chainloading. Put the following at the top level of the first partition of the USB, in kexec.sh:
Code: Select all
# Copyright (c) 2015-7 sakaki <sakaki@deciban.com>
# License: GPL 3.0+
# NO WARRANTY
# This script fragment will be sourced by the initial (boot) kernel's
# init script; it must in turn load the DTB and final (target) kernel,
# setup the kernel command line, and finally pass control over to the
# new kernel with kexec -e.
# Remember that this script is running in a fairly minimal busybox
# environment, and that the shell is ash, not bash.
# On entry, /boot is already mounted (read-only).
# adjust the following to suit your system...
ROOT="UUID=232f3fd4-6b37-406d-bb5c-c96674ceefac" # third partition on Gentoo live-USB (see blkid)
DELAY=5
ROOTSPEC="rootfstype=ext4"
CONSOLE="console=ttyS0,115200n8 earlyprintk"
INITRAMFS="/boot/initramfs-linux.img"
DTB="/boot/kirkwood-b3.dtb"
VMLINUZ="/boot/zImage"
# use the patched DTB, if provided
if [ -f "/boot/kirkwood-b3-earlyled.dtb" ]; then
DTB="/boot/kirkwood-b3-earlyled.dtb"
fi
echo "Creating patched zImage from ${VMLINUZ}..."
cat /boot/cache_head_patch "${VMLINUZ}" "${DTB}" > zImage
echo "Loading patched kernel and setting command line..."
# if an initramfs is present, use this
if [ -f "${INITRAMFS}" ]; then
echo "Booting WITH initramfs..."
kexec --type=zImage --dtb="${DTB}" \
--initrd="${INITRAMFS}" \
--append="root=/dev/ram0 real_root=${ROOT} ${ROOTSPEC} rootdelay=${DELAY} ${CONSOLE}" \
--load zImage
else
kexec --type=zImage --dtb="${DTB}" \
--append="root=${ROOT} ${ROOTSPEC} rootdelay=${DELAY} ${CONSOLE}" \
--load zImage
fi
umount /boot
echo "Booting patched kernel with kexec..."
kexec -e
If using HDD, copy the same file to the HDD's first partition top level, but set ROOT="/dev/sda3" in the file, in place of ROOT="UUID=232f3fd4-6b37-406d-bb5c-c96674ceefac".
You'll need the file cache_head_patch in the top level directory of the first partition of the boot device too. This temporarily turns off the L2 cache in early boot (otherwise, it will hang). You can download the patch from here.
You should also place the kirkwood-b3-earlyled.dtb in the same top level directory (unless you want to work with, and are supplying, the file kirkwood-b3.dtb in this directory). You can download the earlyled DTB here.
That's it - when booted, U-Boot will start the interstitial kernel (install/install.itb if working from USB, boot/uImage if booting from HDD). The interstitial kernel will unpack and start the integral initramfs we just put together, run its init, which will mount the first partition of the boot device (USB or HDD) and then source kexec.sh (the above script). This will load the "real" zImage (and it can be a zImage, so we did solve our first problem ^-^), patch it, load its initramfs if any, and then start it.
The good news is you only have to do this work once - just like U-Boot, the "interstitial" kernel does not need constant updating, it is just there as a bootloader. You can modify the script kexec.sh (to modify the kernel command line for example) without having to recompile (or reflash).
There's just one more thing to take care of - the use of an interstitial kernel will clear the MACs of the B3's ethernet adaptors (normally set up by U-Boot) preventing them from starting. To work around this, put a script in your "real" root - for example, the third partition of the live-USB. The Gentoo and Arch live-USBs, which use interstitial booting already, have an appropriate file in place. Here's what is the Gentoo system (which uses OpenRC init) has, in /etc/init.d/setethermac (enabled for the default runlevel):
Code: Select all
#!/sbin/openrc-run
# Ensure that the B3's ethernet interfaces have the correct MACs set
# (because, if the kernel was kexec-booted, the MACs will initally
# both be be 0, and networking will not start unless this is corrected)
# Copyright (c) 2017 sakaki <sakaki@deciban.com>
# License: GPL 3.0+
# NO WARRANTY
depend() {
need localmount
before net
}
start() {
ebegin "Starting setethermac on eth0 and eth1"
ip link set dev eth0 address $(fw_printenv ethaddr|rev|cut -c -17|rev)
ip link set dev eth1 address $(fw_printenv eth1addr|rev|cut -c -17|rev)
eend $?
}
Code: Select all
[Unit]
Description=Set MAC address of %I
Wants=network-pre.target
Before=network-pre.target
BindsTo=sys-subsystem-net-devices-%i.device
After=sys-subsystem-net-devices-%i.device
[Service]
Type=oneshot
ExecStart=/bin/bash -c "if [[ %i -eq 'eth0' ]]; then ip link set dev eth0 address $(fw_printenv ethaddr|rev|cut -c -17|rev); fi"
ExecStart=/bin/bash -c "if [[ %i -eq 'eth1' ]]; then ip link set dev eth1 address $(fw_printenv eth1addr|rev|cut -c -17|rev); fi"
[Install]
WantedBy=multi-user.target
Issue 2: Switching the Debian Kernel to use gzip Compression, without Recompiling it
If the Debian kernel vmlinuz-4.9.0-3-marvell was a gzip compressed zimage (as archlinuxarm.org's linux-kirkwood-dt kernels are, for example), we'd be done (modulo problem 3!); we could just let the Debian kernel package installer write the vmlinuz-... file into /boot (the mounted first partition's filesystem), then rename it in a post-install hook to /boot/zImage. The interstitial kernel's init-sourced kexec.sh would then pick it up and boot it (this is what happens in the Arch live-USB, where the process works fine and allows the user to upgrade kernels during their standard pacman -Syu).
However, as noted, the Debian kernel is a self-extracting xz archive (that unfortunately won't self-extract at boot on a B3, due to an apparent bug in the decompression algorithm, when kexec'd at any event). Accordingly, although we have now bought ourselves the ability to deal with zImages, we still can't use Debian's vmlinuz-...marvell zImage (under kexec on a B3).
Yet, all is not lost. Recall that the zImage format is an executable, within which is embedded a blob of compressed "piggy_data": the actual kernel image we are interested in. The "head" of the zImage executable is just a bit of hardware init, followed by the "decompress-piggy_data-then-jump-into-its-payload-kernel" code.
As such, a strategy suggests itself: extract piggy_data (the xz-compressed payload kernel image) from the Debian vmlinuz, decompress it (in userland) using xzcat (which should work fine), then recompress that using a known-good algorithm (gzip), and finally repack the new piggy_data back into a self-extracting zImage.... somehow. Ideally, do this without having to compile anything, so the process (in principle) could be productionized to be carried out on an end-user B3 (e.g. during a package post-install hook), even one where no kernel sources or build infrastructure was present (the eventual target being a Debian userland, not Gentoo, after all ^-^).
Let's tackle this step-by-step. First, we need the base Debian kernel zImage . Obviously, on a production system we could just install this and then post-process the zImage in /boot, but in this (proof of concept) I'm running Gentoo, so I'll have to do it manually.
Begin by fetching the Debian kernel package:
Code: Select all
gentoo_pc interstitial-gentoo # wget -c http://security.debian.org/debian-security/pool/updates/main/l/linux/linux-image-4.9.0-3-marvell_4.9.30-2+deb9u2_armel.deb
Code: Select all
gentoo_pc interstitial-gentoo # emerge deb2targz
gentoo_pc interstitial-gentoo # deb2targz linux-image-4.9.0-3-marvell_4.9.30-2+deb9u2_armel.deb
Code: Select all
gentoo_pc interstitial-gentoo # mkdir debkernel
gentoo_pc interstitial-gentoo # tar -xJf linux-image-4.9.0-3-marvell_4.9.30-2+deb9u2_armel.tar.xz -C debkernel/
gentoo_pc interstitial-gentoo # ls debkernel
boot lib usr
gentoo_pc interstitial-gentoo # ls debkernel/boot
System.map-4.9.0-3-marvell vmlinuz-4.9.0-3-marvell
config-4.9.0-3-marvell
gentoo_pc interstitial-gentoo # cd debkernel/boot
gentoo_pc boot # file vmlinuz-4.9.0-3-marvell
vmlinuz-4.9.0-3-marvell: Linux kernel ARM boot executable zImage (little-endian)
Code: Select all
gentoo_pc boot # emerge binwalk bgrep
gentoo_pc boot # binwalk vmlinuz-4.9.0-3-marvell
DECIMAL HEXADECIMAL DESCRIPTION
--------------------------------------------------------------------------------
36 0x24 Linux kernel ARM boot executable zImage (little-endian), load address: "0x00000000", end address: "0x001F3A68"
22315 0x572B xz compressed data
22649 0x5879 xz compressed data
gentoo_pc boot # bgrep fd377a585a00 vmlinuz-4.9.0-3-marvell
vmlinuz-4.9.0-3-marvell: 0000572b
vmlinuz-4.9.0-3-marvell: 00005879
Code: Select all
gentoo_pc boot # dd if=vmlinuz-4.9.0-3-marvell bs=1 skip=$((0x5879)) | xzcat > Image-4.9.0-3-marvell
Code: Select all
gentoo_pc boot # cat Image-4.9.0-3-marvell | gzip -n -f -9 > piggy_data-4.9.0-3-marvell
gentoo_pc boot # ls -l
total 12368
-rw-r--r-- 1 root root 5844072 Jun 29 18:21 Image-4.9.0-3-marvell
-rw-r--r-- 1 root root 1759271 Jun 18 09:14 System.map-4.9.0-3-marvell
-rw-r--r-- 1 root root 157820 Jun 18 09:14 config-4.9.0-3-marvell
-rw-r--r-- 1 root root 2849335 Jun 29 18:28 piggy_data-4.9.0-3-marvell
-rw-r--r-- 1 root root 2046568 Jun 18 09:14 vmlinuz-4.9.0-3-marvell
Let's fetch a sensible config and build a "surrogate" zImage (remember, we can't use the Debian one; it's self-decompressor is of the wrong type).
Code: Select all
gentoo_pc boot # cd ../..
gentoo_pc interstitial-gentoo # wget https://github.com/sakaki-/gentoo-on-b3/raw/2.0.0/configs/b3_baseline_config
gentoo_pc interstitial-gentoo # cp b3_baseline_config .config
gentoo_pc interstitial-gentoo # make ARCH=arm CROSS_COMPILE=armv5tel-softfloat-linux-gnueabi- olddefconfig
gentoo_pc interstitial-gentoo # buildkernel-b3 --custom-dts="${PWD}/kirkwood-b3-earlyled.dts" --zimage
gentoo_pc interstitial-gentoo # mv deploy_root{,_surrogate}
Code: Select all
gentoo_pc interstitial-gentoo # ls -l arch/arm/boot/compressed/piggy_data
-rw-r--r-- 1 root root 3952543 Jun 29 18:46 arch/arm/boot/compressed/piggy_data
Now let's inject our new piggy_data. We take a copy of the surrogate - it doesn't need to be recompiled each time:
Code: Select all
gentoo_pc interstitial-gentoo # cd debtest/boot
gentoo_pc boot # cp ../../deploy_root_surrogate/boot/zImage .
Code: Select all
gentoo_pc boot # binwalk zImage
DECIMAL HEXADECIMAL DESCRIPTION
--------------------------------------------------------------------------------
36 0x24 Linux kernel ARM boot executable zImage (little-endian), load address: "0x00000000", end address: "0x003C9520"
17736 0x4548 gzip compressed data, maximum compression, from Unix, NULL date (1970-01-01 00:00:00)
gentoo_pc boot # bgrep 1f8b08 zImage
zImage: 00004548
Code: Select all
gentoo_pc boot # dd if=piggy_data-4.9.0-3-marvell of=zImage seek=$((0x4548)) bs=1 conv=notrunc
BTW, the reason this works is because the gzip decompressor will simply stop once it has extracted a single valid member - which we have just injected. It doesn't attempt to account for the additional bytes inside piggy_data, nor to decode them.
OK, nearly done! As you can appreciate, the repetitive parts of the above could easily be automated into a post-install hook to reformat the vmlinuz...marvell zImages: the same surrogate kernel can be reused each time, it does not need recompilation.
So, let's review: we can now boot zImages using kexec, and we have a post-processed Debian zImage that will boot under kexec. What we don't have is a resulting kernel with the wherewithal to mount our "real" root so that we can bring up our userland OS. Let's fix that next.
Issue 3: Enabling the Debian Kernel to Mount Root on the B3
To solve this problem, we need to create another initramfs; this time, for use by the "real" kernel (the payload injected into the surrogate). This initramfs will contain (and its init, modprobe) the 4.9.0-3-marvell kernel modules needed to allow the (rather minimalist) Debian kernel to at least access the HDD and USB storage on the B3, so the main root filesystem can be brought up. Once done, its init script will switch_root to the "proper" init (OpenRC, systemd or whatever) on the now-visible real root filesystem.
Now, I don't know much about Debian, but I see it has a number of initramfs tools cited in the linux-image-4.9.0-3-marvell dependencies; so maybe this is a non-issue and can be dealt with automagically.
But since we're here, let's do it the old school way, and roll our own (external) initramfs. We can base it on the one created earlier, so we'll start by making a copy of that, and then removing the kexec binary which we don't need in this context:
Code: Select all
gentoo_pc boot # cd ../..
gentoo_pc interstitial-gentoo # cp -ax initramfs initramfs-debian
gentoo_pc interstitial-gentoo # cd initramfs-debian
gentoo_pc initramfs-debian # rm sbin/kexec
Code: Select all
gentoo_pc initramfs-debian # mkdir -p lib/modules
gentoo_pc initramfs-debian # cp -ax ../debtest/lib/modules/4.9.0-3-marvell lib/modules/
Code: Select all
gentoo_pc initramfs-debian # mkdir -p etc/conf.d
gentoo_pc initramfs-debian # nano -w etc/conf.d/modules
Code: Select all
# minimal module set to boot B3
# this file is processed by init directly (fn: load_baseline_kernel_modules)
# HDD
scsi_mod
libata
sata_mv
# USB
usb-common
usbcore
ehci-hcd
ehci_orion
usb_storage
# Ethernet
libphy
fixed_phy
of_mdio
mv643xx_eth
# /dev/mtd (for MACs etc)
spi-nor
m25p80
# WiFi
rfkill
cfg80211
mac80211
ath
ath9k_hw
ath9k_common
ath9k
# executable file formats
binfmt_misc
# gpio
gpio_keys
Code: Select all
gentoo_pc initramfs-debian # > init
gentoo_pc initramfs-debian # nano -w init
Code: Select all
#!/bin/busybox sh
# Trivial initramfs init (just enough to allow early module load etc., so main
# rootfs can be brought up)
#
# Simply looks up the "real_root=<...>" from /proc/cmdline to find
# the root filesystem (resolving UUID=... if necessary), and then
# mounts and switches root to /sbin/init in that new filesystem
#
# Copyright (c) 2017 sakaki <sakaki@deciban.com>
# License: GPL 3.0+
# NO WARRANTY
ROOT_DEV="/dev/sdb3" # we'll try to replace this
ROOT_MOUNT="/mnt/.root"
ROOT_FS_OPT="" # let mount figure it out unless we are told explicitly
DELAY=5
show_early_startup_led() {
modprobe leds_gpio
# turn on green and blue LEDs
cat /sys/class/leds/bubba3\:green\:programming/max_brightness > \
/sys/class/leds/bubba3\:green\:programming/brightness
cat /sys/class/leds/bubba3\:blue\:active/max_brightness > \
/sys/class/leds/bubba3\:blue\:active/brightness
# turn off the red (error) LED
echo -n 0 > /sys/class/leds/bubba3\:red\:error/brightness
}
show_red_led() {
# turn off green and blue LEDs
echo -n 0 > /sys/class/leds/bubba3\:green\:programming/brightness
echo -n 0 > /sys/class/leds/bubba3\:blue\:active/brightness
# turn on the red (error) LED
cat /sys/class/leds/bubba3\:red\:error/max_brightness > \
/sys/class/leds/bubba3\:red\:error/brightness
}
load_baseline_kernel_modules() {
# load specified modules
grep "^\w*[a-zA-Z]" "/etc/conf.d/modules" | xargs -n1 modprobe
}
start_mini_udev() {
echo "Starting mini udev..."
[ -e "/proc/sys/kernel/hotplug" ] && echo "/sbin/mdev" > /proc/sys/kernel/hotplug
mdev -s
}
error_exit() {
# light the red led
show_red_led
echo "Starting recovery shell on serial console..." >&2
exec /bin/busybox sh -i
}
find_root_from_kernel_cmdline() {
echo "Looking up real_root in kernel command line..."
if ! ROOT_DEV="$(cat /proc/cmdline | grep -o 'real_root=[^ ]*')"; then
echo "Could not find real_root=<...> in kernel command line!" >&2
error_exit
fi
# extract just the target part of last cited entry
ROOT_DEV="${ROOT_DEV##*real_root=}"
}
find_rootfstype_from_kernel_commandline() {
local FST
echo "Looking up rootfstype in kernel command line..."
if FST="$(cat /proc/cmdline | grep -o 'rootfstype=[^ ]*')"; then
ROOT_FS_OPT="-t ${FST##*rootfstype=}"
fi
}
# Mount the /proc and /sys filesystems.
/bin/busybox mount -t proc none /proc
/bin/busybox mount -t sysfs none /sys
/bin/busybox mount -t devpts none /dev/pts
# ensure all symlinks are present
/bin/busybox --install -s
# show cyan LED to indicate the point we are at
show_early_startup_led
# load core kernel modules
load_baseline_kernel_modules
# start udev
start_mini_udev
sleep "${DELAY}"
# now we have a functioning system, let's find the root=<...>
# passed to the kernel command line, and set in ROOT_DEV
find_root_from_kernel_cmdline
find_rootfstype_from_kernel_commandline
# look up what we were given, in case it is a UUID based reference
echo "Trying to locate partition ${ROOT_DEV}..."
if ! ROOT_PATH="$(findfs "${ROOT_DEV}")"; then
echo "Error - could not find ${ROOT_DEV}!" >&2
error_exit
fi
# looks good, try to mount it (on a temporary mountpoint, read-only)
echo "Mounting root filesystem (read only, initially)..."
[ -d "${ROOT_MOUNT}" ] || mkdir -p "${ROOT_MOUNT}"
if ! mount -o ro ${ROOT_FS_OPT} "${ROOT_PATH}" "${ROOT_MOUNT}"; then
echo "Error - failed to mount ${ROOT_PATH}!" >&2
error_exit
fi
# now we can switch to the new rootfs, and run _its_ init; sayonara...
exec switch_root "${ROOT_MOUNT}" "/sbin/init"
# should not get here - that was an exec we just did!
echo "Error failed to switch_root!" >&2
error_exit
That's it for the new initramfs. Next, we have to create the compressed image of this ourselves, as it is going to be passed to the (Debian) kernel via the kexec call, it isn't getting built in. To create the image, do:
Code: Select all
gentoo_pc initramfs-debian # find . | cpio -o --quiet -R 0:0 -H newc | gzip > ../initramfs-linux.img
That's it! As I say there may be a "blessed", automated way to handle initramfs construction in Debian, and if so, obviously you'd want to use that. Note that the initramfs (with the appropriate-version module set in lib/modules) has to be created each time a new kernel package is installed, since the modules are kernel-specific. But it wouldn't be an onerous thing to script.
So, now we have the ability to boot zImages (without changing u-boot) under kexec, a process to create a kexec-bootable zImage from the Debian distributed vmlinuz-...-marvell file, and the necessary mechanism to provide the (relatively minimalistic) Debian kernel sufficient modules to be able to mount the "real" root, and switch_root to the proper init. After that, everything is vanilla ^-^
Additional Considerations
Of course, it is necessary to have the /lib/modules/4.9.0-3-marvell (or whatever) installed into the "real" root filesystem too, and "depmod -a" should be run on that - but presumably, the normal Debian install process for linux-image-...-marvell would do that for you (I haven't checked).
Bonus Extra: Demo Arch Live-USB with the Debian Kernel & Modules from linux-image-4.9.0-3-marvell
To show this all running, and just to prove you don't need a Gentoo userland, I have grafted the above kernel and zImage into my Arch Linux v1.5.1 live-USB image (which already has an interstitial boot) and pushed this as a "special edition" image to GitHub. You can download the image, write it to a USB stick, and then try booting your B3 from it (it won't harm any existing Debian system on your hard drive), to see that the above process does indeed work.
To download, on your Linux box (as root) issue:
Code: Select all
# wget -c https://github.com/sakaki-/archlinux-on-b3/releases/download/1.5.1/archb3seimg.xz
Now, insert a spare USB stick of at least 4GB capacity (SanDisk and Lexar seem the most reliable), note it's device path, and write the image to it:
Code: Select all
# xzcat archb3seimg.xz > /dev/sdX && sync
This will take between 5-20 minutes to complete. It should exit cleanly when done - if you get a message saying 'No space left on device', then your USB key is too small for the image, and you should try again with a larger capacity one.
To boot, begin with your B3 powered off and the power cable removed. Insert the USB key into either of the USB slots on the back of the B3, and make sure the other USB slot is unoccupied. Connect your B3 into your local network (or directly to your ADSL router, cable modem etc., if you wish) using the wan Ethernet port. Then, while holding down the button on the back of the B3, apply power (insert the power cable). After five seconds or so, release the button. If all is well, the B3 should boot the interstitial kernel off of the USB key (rather than the internal drive), then patch, load and kexec the Debian kernel, load the necessary modules from its initramfs, then proceed to mount the "real" root partition (also on the USB key) and start Arch Linux. This will all take about 90 seconds or so. The LED on the front of the B3 should:
- first, turn green, for about 20 seconds, and then briefly purple, as the interstitial kernel loads; then,
- turn off for about 10 seconds, and the 'real' (Debian) kernel is patched and loaded; then
- turn cyan for about 30 seconds, as the initramfs init loads the necessary 4.9.0-3-marvell modules, and mounts and starts the "real" init (systemd, in this case); then
- turn green as Arch Linux comes up.
First, connect your client PC (or Mac etc.) to the lan Ethernet port of your B3 (you can use a regular Ethernet cable for this, the B3's ports are autosensing). Alternatively, if you have a WiFi enabled B3, you can connect to the "b3" WiFi network which should now be visible (the passphrase is changeme).
Then, on your client PC, issue:
Code: Select all
$ ssh root@archb3
Code: Select all
$ ssh root@192.168.50.1
And you're in:
Code: Select all
==============================================================================
| Welcome to Arch Linux for the B3 miniserver (with Debian kernel)! |
| Live-USB maintainer: sakaki <sakaki@deciban.com> |
==============================================================================
| There is ABSOLUTELY NO WARRANTY, to the extent permitted by applicable law |
==============================================================================
| (Edit, or delete, /etc/motd to modify this message) |
==============================================================================
[root@archb3 ~]# uname -a
Linux archb3 4.9.0-3-marvell #1 Debian 4.9.30-2+deb9u1 (2017-06-18) armv5tel GNU/Linux
Feel free to play around with the image and please post below any issues you find with the Debian kernel (missing drivers, issues with performance etc.).
Not sure if this is of use to anyone but it was fun to get it running ^-^, best
sakaki
Re: Debian 9 (stretch)
![Very Happy :D](./images/smilies/icon_e_biggrin.gif)
If someone is feeling confident and crazy enough to create a debian package which would build a patched up kernel, include the interstitial stuff and everything I will be more than happy to add it to the stretch repo. In the meantime I will provide a very classical (and boring) custom kernel for now.
Now a bit of hype
Code: Select all
excito@b3:~$ uname -a
Linux b3 4.9.34-1 #1 Thu Jun 29 16:35:21 EDT 2017 armv5tel GNU/Linux
excito@b3:~$ lsb_release -a 2>/dev/null
Distributor ID: Debian
Description: Debian GNU/Linux 9.0 (stretch)
Release: 9.0
Codename: stretch
![Mr. Green :mrgreen:](./images/smilies/icon_mrgreen.gif)