ZFS on B3?

Discuss development on Bubba
stasheck
Posts: 107
Joined: 15 Jan 2014, 13:13

ZFS on B3?

Post by stasheck » 03 May 2015, 12:39

Given the new ArchLinux build, did anyone test the performance of running ZFS on B3? Is it even possible?

MouettE
Site admin
Posts: 257
Joined: 06 Oct 2011, 19:45

Re: ZFS on B3?

Post by MouettE » 04 May 2015, 00:29

I'm not familiar with ZFS on Linux, but on Solaris it requires an great amount of RAM to run properly (at the very least a few gigs). I would not use it on the B3 given the low RAM amount and limited CPU power. Maybe btrfs would be better choice, or stick with lvm.

stasheck
Posts: 107
Joined: 15 Jan 2014, 13:13

Re: ZFS on B3?

Post by stasheck » 04 May 2015, 04:40

Hm, I was asking because after a few years I started to discover some bit-rot on my photo archive. ZFS offers quite nice protection against that, LVM is just a container system.

Need to check if btrfs does anything about this.

Gordon
Posts: 1327
Joined: 10 Aug 2011, 03:18

Re: ZFS on B3?

Post by Gordon » 04 May 2015, 07:13

Did you consider that it may be the internal memory that is starting to fail?

MouettE
Site admin
Posts: 257
Joined: 06 Oct 2011, 19:45

Re: ZFS on B3?

Post by MouettE » 04 May 2015, 14:19

stasheck wrote:Hm, I was asking because after a few years I started to discover some bit-rot on my photo archive. ZFS offers quite nice protection against that, LVM is just a container system..
Wouldn't backups be a good answer here ?

Ubi
Posts: 1547
Joined: 17 Jul 2007, 09:01

Re: ZFS on B3?

Post by Ubi » 05 May 2015, 07:41

indeed, if your drive is going into failure, (especially with a single-disk setup), no file system is going to protect you.

stasheck
Posts: 107
Joined: 15 Jan 2014, 13:13

Re: ZFS on B3?

Post by stasheck » 08 May 2015, 08:54

Guys, I have a high esteem of your skills, but you seem to be missing the point here. Do you know what bit-rot is?

http://arstechnica.com/information-tech ... lesystems/

So, a standard drive will degrade data at rate of about 10^-14 bits (meaning 1 bit out of 10^14 will be flipped. The figure is 10^-15 for server drives). Sounds huge, right? Let's see:
10^1 b = 1 B (actually, 8b=1B, but we're doing back-of-napkin math)
10^3 B = 1 kB (yes, 1024, but is doesn't change calculation much)
10^3 kB = 1 MB
10^3 MB = 1 GB
10^3 GB = 1 TB
=====
10^13 b = 1 TB

This means that on average, writing 10 TB of data to drive will result in a single bit flipped, and nothing will prevent that with older filesystems. During backup, you might notice if you're hashing every backed up file prior to backup.

I already lost a number of photos this way (well... small number, compared to total amount).

Using ZFS, for example, adds a few addition 10^-x (can't remember the number now, sorry), and due to design running RAID-1 on ZFS allows for transparent fixing of bad blocks.

Need to investigate BTRFS now :D

Gordon
Posts: 1327
Joined: 10 Aug 2011, 03:18

Re: ZFS on B3?

Post by Gordon » 08 May 2015, 11:13

Yes I know what it is, but I still say you might want to try take the disk out and put it in a different system to verify that the file you think is corrupted is in fact corrupted on disk. Because bitrot can happen to RAM as well.

Ubi
Posts: 1547
Joined: 17 Jul 2007, 09:01

Re: ZFS on B3?

Post by Ubi » 08 May 2015, 12:04

I understand bitrot would affect executable software, but a photo archive? This will just create a small corruption in the jpg file which is easily fixable and worst case will alter the color of a single pixel. For TIFFs it wont even corrupt it. Losing multiple photos due to such a rare process (considering there are other processes that give the same corruption but are much more common, such as previously mentioned RAM issues or just drive failures) this whole story seems a little far-fetched to me, and the solution a little over-the-top. But hey, go ahead and see if you can get zfs or btrfs on the B3, it'll be an interesting hack in the least.

stasheck
Posts: 107
Joined: 15 Jan 2014, 13:13

Re: ZFS on B3?

Post by stasheck » 08 May 2015, 13:12

Ubi, you seem not to know much about JPEGs :-) One bit will make (in best case) half of you photo in render with funny colors, and althought I have a couple ideas floating on how to fix that, I have yet to find a ready program that fixes this. TIFF, more true.

Also, this is a _real_ problem that I encountered, I'm not saying that the cause was the drive actually inserted to B3 - might have been flash card, might have been USB stick... It's not like I'm reviewing all my photos each time I copy them :D

I just wanted to secure at least one part of backup scheme.

Since my B3s disk died on me a few days back, I'm now going to go Arch way (wish me luck, never used Arch before :D) - I'll try to install the disk as BTRFS.

Ubi
Posts: 1547
Joined: 17 Jul 2007, 09:01

Re: ZFS on B3?

Post by Ubi » 08 May 2015, 13:41

you have heard of incremental backups right? Or checksums?

stasheck
Posts: 107
Joined: 15 Jan 2014, 13:13

Re: ZFS on B3?

Post by stasheck » 09 May 2015, 05:26

Ubi, I really appreciate your services to community, and general willingness to help, but sometimes you're really getting on my nerves. Your technical knowledge seems to come from 15-20 years of experience, but the way you write sometimes sounds like you're 15 yourself.

I have a friend that our bunch is calling "1-up" - because whatever any of us says, this guy is always a level higher - either did it earlier, or better, or found something more interesting. Please don't be that guy :-)

And to address your question: yes, I did. Some of the photos I refer to are 10-15 years old, and a) as I mentioned, it's hard to tell when the degradation happened, b) as you may imagin, I'm through at least 5 backup schemes by this time :-)

Ubi
Posts: 1547
Joined: 17 Jul 2007, 09:01

Re: ZFS on B3?

Post by Ubi » 09 May 2015, 06:23

Contrary to popular believe I'm not trying to be an ass.
I'm just trying to make you think about what it is you want to achieve. Installing ZFS solves a very unlikely problem but gives you performance and stability issues that you need to deal with every day. It's just not a good trade-off. We see a lot of people here that want something very complicated (because they read about it on the internet usually) to solve a problem that can be addressed much more elegantly. I ask annoying (and in your view immature) questions to make you think about what you really want to gain.
Apparantly you have heard of checksums and incremental backups. Many people have not, as this is not a forum for experts. Instead of posting a completely prepared alternative example I generally post a few vague pointers, so people are required to do a bit of investigation themselves, or ignore it altogether as they please.

But again: go ahead with your proposal zfs install. Its another cool hack that is a nice addition to the wiki (you are going to post your results there right?). Being able to do these things is whats nice about the platform.

stasheck
Posts: 107
Joined: 15 Jan 2014, 13:13

Re: ZFS on B3?

Post by stasheck » 09 May 2015, 07:22

:D Ok, you made mi smile here.

Anyway, believe it or not but I usually try to create neat, simple solutions. I have some prior experience with running ZFS on Solaris and remember reading some time ago that ZFS can now be used not only via FUSE. Turns out that running it on B3 might not be such a good idea after all, so I used BTRFS instead (has pretty much same feature set), just for /home right now.

Now, fun fact: when creating GPT partitions, you can select type 0x8302 - Linux /home. When you do that, don't add '/home' line into fstab because for some reason B3 will not boot (and since I don't have serial cable, I've no idea why ATM).

However, if you don't add '/home', it will be mounted regardless, and boot just fine.

For reference:

Code: Select all

# <fs>                  <mountpoint>    <type>          <opts>          <dump/pass>

# NOTE: If your BOOT partition is ReiserFS, add the notail option to opts.
/dev/sda1               /boot           ext3            noatime         0 2
/dev/sda2               none            swap            sw              0 0
/dev/sda3               /               ext4            noatime         0 1
#/dev/sda4               /home           btrfs           noatime,autodefrag,spacecache   0 1

Code: Select all

]# mount
/dev/sda3 on / type ext4 (rw,noatime,data=ordered)
devtmpfs on /dev type devtmpfs (rw,relatime,size=255124k,nr_inodes=63781,mode=755)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls,net_prio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/bfqio type cgroup (rw,nosuid,nodev,noexec,relatime,bfqio)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpu,cpuacct)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=22,pgrp=1,timeout=300,minproto=5,maxproto=5,direct)
tmpfs on /tmp type tmpfs (rw)
mqueue on /dev/mqueue type mqueue (rw,relatime)
fusectl on /sys/fs/fuse/connections type fusectl (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
/dev/sda1 on /boot type ext3 (rw,noatime,stripe=4,data=ordered)
/dev/sda4 on /home type btrfs (rw,relatime,space_cache)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=51052k,mode=700)

Gordon
Posts: 1327
Joined: 10 Aug 2011, 03:18

Re: ZFS on B3?

Post by Gordon » 09 May 2015, 16:59

Add "nofail" option to the sda4 line. Debians mountall will then not halt the system so you can check dmesg what error was returned.

Post Reply