Has anyone tried converting a large B3 LVM system to ext4?

[Cheeseboy]

I am a hoarder.
I can't seem to get rid of stuff. So I have a SATA port multiplier, and now 3x3TB WD Green disks attached to my B3. I have done some modifications. I have made the partitions aligned (as they are not out of the box). I have partitioned my disks in multiple partitions to have more PVs to play around with.

I got my latest S1 just the other day. It works very well after repartitioning and LVM trickery to get it all working. The wonders of having an external SATA port multiplier is that you can always attach another disk temporarily in order to have the space to arrange things the way you like. I can grow my file systems while they are on-line. Excellent.

Now regarding ext4:
Has anyone tried converting an existing large LVM system on a B3 to ext4 without losing data?
My main reason for wanting to do this is faster fsck.

I guess the boot-stick doesn't care about /dev/sda2 when you use it in rescue mode, but what if I had to reinstall the system (keeping /home)? Will it accept an ext4 /home?

The fact that you cannot use the boot/rescue-stick to check LVM volumes is a pain. Today I grew an ext3 file-system while it was on-line, and all seems fine, but I would still like to be able to do a proper off-line forced fsck every now and again. This currently takes many hours as it is ext3, and the procedure to do it is so tedious it's barely comprehensible:

1. create temporary home directories on /dev/sda1
2. copy needed stuff from your and admin's home directories to the temporary ones on /dev/sda1
3. disable services that will write to /home (dovecot etc) so you don't have to sync your temporary home back with the real one once you are done.
4. reboot with bootstick
5. mount /dev/sda1
6. create symlinks in /home on /dev/sda to point to your temporary home dirs
7. edit /etc/fstab on /dev/sda1 so it doesn't mount the home dirs from the LVM on the next boot
7. reboot normally
8. run the fsck on your now unmounted logical volumes
9. reboot with bootstick
10. reset everything
11. reboot normally
12. bite your finger-nails in anticipation

Sorry for rambling. To simplify:
1. Has anyone converted to ext4, and if so, how did it go?
2. Excito: Could we please have a boot-stick with a busybox that handles LVM2?

Many thanks!

/Cheeseboy

EDIT: Re-wrote most of it as it was gibberish, and also wrong

[Cheeseboy]

OK, I have now tried this on a small scale on the B3 - preparing for the big jump!

I set up a new Logical Volume consisting of several Physical Volumes, formatted them to ext3 and copied 100 GB of files to the file system.
File sizes varied from a couple of bytes to 10 GB.
I looked at several guides, and several things went wrong.

I think this one is the most reliable:
https://wiki.archlinux.org/index.php/Ext4

Mainly because it specifies that you should use chattr +e on files before directories after the switch is done.
Before that I just did chattr -R on the directory structure, and it caused havoc.

When it comes to performance, I can only comment on fsck -f, which took 90 minutes on the 100 GB FS when it was ext3, and 7 minutes when it was ext4...
I did not do any before/after read/write tests.

Next step is going to be for real on my 7 TB file system on the B3.
I'll get back with a progress report (or perhaps not get back at all if my system dies).

Cheers,

Cheeseboy

[Ubi]

i read this 3 times and it made no sense at all

* several physical volumes ina B3??
* why first make this thing in ext3, load it with data, then think about upgrading?
* why ext4? Its not really gonna matter methinks on a b3

maybe it helps explain what your post is about instread of saying "the big jump"

[Cheeseboy]

Hi Ubi!

Thanks for your interest (and also your always present and amusing scepticism )

Well, I have multiple HDDs (in S1 enclusures) connected to my B3 via a eSATA port multiplier.
All bundled up in the "storage" logical volume as separate Physical Volumes (several partitions/PVS per disk actually).
Can't afford redundancy in form of LVM striping or RAID, so always trying to be careful.
I use smartd to get basic notifications on how the disks are doing, and once it told me well in advance when it was time to replace a disk (lucky that, and it makes me feel relatively safe with my unsafe solution).
The fact that I had the port multiplier (and LVM) didn't make it very hard to replace the disk before it died.

But I DO want to run fsck every now and then on the FS itself, not the disks. On a file system that large with ext3, it will mean several days with the system off-line. As the FS seems to always grow, and never shrink, I thought I'd had to convert to ext4 eventually just because of the fsck performance.

Hence the original post: Has anyone done it?
Obviously not. But I'm going to try it anyway.

And having just recovered from what I thought was a loss of my system partition, I realize that that would have been so much worse than the 7 TB of data I might lose in this attempt. The system partition represents many years of undocumented tinkering, whilst the 7 TB is just downloaded data (it can be replaced - even if it will cost time and money)

EDIT:
Forgot to answer question number 2:
I did it as a dry run / test of course!
Using some old crappy disks lying around.

Cheers,

/Cheeseboy

[Cheeseboy]

OK, it is done now.
And it seems to work just fine! I left the /dev/sda1 partition alone of course, but /dev/mapper/bubba-storage is now ext4 and using extents. It took forever, and several fscks were needed during the operation.

I will let it run for a while before tweaking reserved blocks etc.

[Ubi]

good job! Interesting to know this works so well

[Cheeseboy]

A little update on the ext4 performance:

I recently bought a 3rd S1 disk cabinet, and today I installed another 3TB WD Green into it.
I divided it into several LVM partitions and added 3 of them to the LV:

Code: Select all

sudo lvextend bubba/storage /dev/sdb1 /dev/sdb2 /dev/sdb3

Each of the above partitions are 500 GiB. It happened while the FS was live, and in use.
It took about an hour, and I noticed no lack of performance during the operation (I was watching a movie streamed from the same FS during the time).

Being a bit paranoid, I still took it off-line afterwards and did a fsck on it:

Code: Select all

root@b3:/temphome/niklas# time fsck -t ext4 -V -C /dev/mapper/bubba-storage -fDp
fsck from util-linux-ng 2.17.2

Bubba_home: 28252/562102272 files (28.6% non-contiguous), 1379395006/2248406016 blocks
time fsck -t ext4 -V -C /dev/mapper/bubba-storage -fDp
real    56m29.750s
user    29m47.090s
sys     4m20.390s

The file system is currently at 8.3 TiB (there is plenty more to go around at the moment, but as I'm experimenting, I'm taking it it slowly)
I think this same check would have taken a week on ext3.

I'll keep you posted, and will let you know when my system dies completely because of my experiments (if I still have internet access by then)

Cheers,

/Cheeseboy