small silent secure systems

BtrfsThe Nascent GNU/Linux Filesystem

An Introduction

byRichard Melville

Floss UK Spring Conference 2015 York

Building a Btrfs RAID 1 System

For use as a root file system

Why Btrfs and how does it differ from other file systems?

● Pooling file system

● Own built-in version of RAID

● Built-in support for SSDs

● Obviates the need for traditional partitions

● Can enable subvolumes and quotas instead of partitions

● Allows snapshots of subvolumes

● Use snapshots for backups

● Use snapshots to roll-back to an earlier state

● Once set up almost everything can be done on the fly

Pooling file system – my definition

A pooling file system creates an area of common storage where

data is stored. The pool can be increased in size by adding more

devices, or decreased by removing them, and this can be achieved

on the fly. This is different from the traditional device partitioning

scheme where fixed areas of storage, or partitions, are rigidly

allocated to various parts of the operating system tree.

Similarities to LVM

BUT

Btrfs works at the file system level rather than a layer above it

RAID

Btrfs has its own version of RAID built-in

BUT

It differs from MDADM RAID

IMPORTANT!

Btrfs RAID 1 keeps only two copies of data and metadata no matter how many devices you have

in the pool!!!

And those two copies may be held on any two devices

Built-in support for SSDs

Your /etc/fstab mount options could include

ssd,discard

Obviates need for traditional partitioning...

run

mkfs.btrfs /dev/sdx

to format a bare device with the Btrfs file system

...or format a device partition

mkfs.btrfs /dev/sdxx

You can convert an existing partition

1) Unmount the system

umount /dev/sdxx

2) Run a file system check

fsck.extN -f /dev/sdxx

3) Do the conversion

btrfs-convert /dev/sdxx

Change your fstab settings

/etc/fstab

new UUID /mount/point btrfs options 0 0

Reboot and if all is well delete ext2_saved

ext2_saved is the subvolume saved for you by the btrfs-convert tool

Example

btrfs subvolume delete ext2_saved

If there is a problem you canroll-back

So don't delete but rather mount ext2_saved which has been saved as a subvolume

IMPORTANT!

You should NOT continue using your new viable Btrfs system without deleting ext2_saved

Using a bootable USB flash drive to boot into a Btrfs RAID 1 rootfs

The boobtable USB flash looks like this from gdisk

Number Start (sector) End (sector) Size Code Name 1 2048 206847 100.0 MiB 8300 Boot 2 206848 19081215 9.0 GiB 8200 Swap 3 19081216 30282974 5.9 GiB 8300 Resc

Syslinux can be found here

If anybody is interested

https://www.kernel.org/pub/linux/utils/boot/syslinux/

It's worth it, believe me

Download the latest kernel and btrfs-progs

The latest stable kernel is 3.19.2 from kernel.org

For the latest btrfs-progs run

git clone git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-progs.git

If you don't have git install it first

I guarantee the distro repositories won't have anything like the latest versions.

Let's format our two-device Btrfs RAID 1 rootfs

mkfs.btrfs -m raid1 -d raid1 /dev/sda /dev/sdb

Clearly, you need to do this from a rescue system, or externally connect the two devices, via USB, to a build system, or another system

Change your fstab settings again

/etc/fstab

UUID / btrfs device=/dev/sda,device=/dev/sdb 0 0

The mount options where the devices are listed would be added to with a comma separated list

of your other options. The full list of possible mount options can be found here

https://btrfs.wiki.kernel.org/index.php/Mount_options

First create some subvolumes

Create a subvolume for root

btrfs sub create root

Creating further subvolumes

btrfs sub create homebtrfs sub create varbtrfs sub create opt

etc...

Mount your additional subvolumes

/etc/fstab

UUID / btrfs subvolume=root... 0 0UUID /home btrfs subvolume=home... 0 0UUID /var btrfs subvolume=var... 0 0UUID /opt btrfs subvolume=opt... 0 0

etc...

Default subvolume

The default subvolume created automatically is 0/5 into which the system boots if no other

subvolume is passed to mount

This can never be deleted

The default subvolume can be toggled with

btrfs subvolume set-default <id> <path>

Quotas

Quotas can be applied to subvolumes

Run btrfs quota enable /

Before creating your subvolumes

Applying limits to quotas

Example

Run btrfs qgroup limit 1g /var

To limit the size of /var to 1GB

Snapshots

Snapshots are just special subvolumes

Basic example for snapshotting

btrfs subvolume snapshot /home/ /home/snapshot

Snapshot and backup tools using Btrfs send/receive

Choose one of these tools if you wish to automate your backups

https://github.com/masc3d/btrfs-sxbackup # written in pythonhttp://www.digint.ch/btrbk/doc/readme.html # written in perlhttps://github.com/ruediste/btrbck # written in java

My favourite is the second written in perl

Kernel patch for booting without an initrd

If you are booting without an initrd you may find that you need to hack the kernel to stop the

rootfs attempting to load too fast. You can find a patch here

https://gist.github.com/the8lack8ox/2706705

This will not work with every kernel version, so you may need to add the appropriate lines of code manually. The good news is there's not

much to change in the file

Testing the file system

After the system boots successfully run

btrfs fi show /

it may return

Label: 'raid 1' uuid: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx Total devices 2 FS bytes used 6.63GiBdevid 2 size 29.82GiB used 8.03GiB path /dev/sda1 *** Some devices missing

Getting rid of an annoying message

Run: btrfs device scan

You will find that the message has disappeared

btrfs fi show /

Label: 'raid 1' uuid: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx Total devices 2 FS bytes used 6.63GiBdevid 1 size 29.82GiB used 8.03GiB path /dev/sdb1devid 2 size 29.82GiB used 8.03GiB path /dev/sda1

You can add btrfs dev scan to a boot script

Scrub your file system

Make sure you run this periodically

btrfs scrub start /

You can test the progress with

btrfs scrub status /

You can run this as a cron job

Booting a corrupted root file system

Boot with

rootflags=degraded

as a kernel boot parameter on whichever boot loader you use

Replacing the failed device with another device

btrfs device add /dev/sdx(x) /

Device missing?

If you receive a “device missing” warning when you run

btrfs fi show /

Then run

btrfs device delete missing /

Summary of steps necessary to replace a failed device

1) Mount in degraded mode

2) Add a new device

3) Remove the missing device

4) Rebalance the data and metadata with

btrfs balance start -dconvert=raid1 -mconvert=raid1 /

Checking for free disk space

Use this invocation

btrfs fi df <btrfs mountpoint>

And not this

df -h <btrfs mountpoint>

You can test each and see the difference

Abbreviation of commands

Example

sub=subvolume and fi=filesystem

There is no definitive list of shortened forms You can make them up as you go along

A “shortened-form-failed” example

If instead of btrfs subvolume list we attempted

btrfs s l

The parser would return

: ambiguous token 's'

Did you mean one of these ? subvolume scrub send

Shortened form corrected

So if we run

btrfs su l

The command completes as expected

The parser operates on a contextual basis

Help from the Web

There are many howtos on the webHere are links to three

https://btrfs.wiki.kernel.org/index.php/Main_Page

https://wiki.archlinux.org/index.php/Btrfs

https://www.howtoforge.com/a-beginners-guide-to-btrfs

Acknowledgements

Gerard Beekmans – The man behind Linux From Scratch who made it possible initially for us to build our battery-powered solid-state servers

#btrfs on freenode – a great source of help and advice

Chris Mason – lead developer of Btrfs

Thank you

Richard Melville

Cellularity Limited

richardm@cellularity.co.uk

@6tricky9

Any questions?