meta data for this page
  •  

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision		Previous revision
linux:fs:btrfs_as_root [2015/07/17 06:25] – external edit 127.0.0.1linux:fs:btrfs_as_root [2017/01/18 21:41] (current) – niziak
Line 1: Line 1:
-=== System overview ===+====== BTRFS as root filesystem ====== 
 + 
 +===== System overview =====
   - /dev/sda1 4.08GB ext4 "/" used 1.56GB, free 2.53GB   - /dev/sda1 4.08GB ext4 "/" used 1.56GB, free 2.53GB
   - /dev/sda2 10.91GB extended   - /dev/sda2 10.91GB extended
Line 5: Line 7:
     - /dev/sda6 10.40GB ext4 "/home" 321MB used, 10.09GB free     - /dev/sda6 10.40GB ext4 "/home" 321MB used, 10.09GB free
  
-=== Boot from live-cd ===+===== Boot from live-cd =====
 Btrfs support is heavily developed in Linux Kernel. So do not use old kernel. Btrfs support is heavily developed in Linux Kernel. So do not use old kernel.
  
-I was using gparted-live-0.22.0-1-amd64.iso +I was using gparted-live-0.22.0-1-amd64.iso
 (NOTE: to match your existing system architecture (i586, amd64, etc)) (NOTE: to match your existing system architecture (i586, amd64, etc))
 Which contains: Which contains:
   - Kernel 3.16.0-4-586   - Kernel 3.16.0-4-586
   - btrfs utils version 3.17   - btrfs utils version 3.17
-   
-   
-=== Conversion === 
-fsck -f /dev/sda1 
-fsck -f /dev/sda6 
  
-btrfs-convert /dev/sda1 
  
-Disk usage before conversion: used 1.56GB, free 2.53GB +===== Update to kernel 4 =====
-after conversion: used 1.65GB, free 2.43GB +
- +
-btrfs check /dev/sda1 +
- +
-=== Update system === +
-Mount new root filesystem: +
- +
-mount /dev/sda1 /mnt +
-for i in dev dev/pts proc sys ; do mount --bind /$i /mnt/$i ; done +
-chroot /mnt +
- +
-get new UUID of /dev/sda1 using "blkid" +
-mount /dev/sda1 somewhere, go to etc/fstab and update to new UUID: +
- +
-UUID=a74f5787-aee1-4981-b7e6-fbd3cb6ac919 /               btrfs    defaults 0       1 +
- +
-update-grub +
-grub-install /dev/sda +
- +
-exit +
- +
-reboot +
-and remove Live CD with Gparted +
- +
-Rollback to ext2 is possible. There is subvolume ext2_saved on filesystem +
-# btrfs subvolume list / +
-To delete backup +
-# btrfs subvolume delete /ext2_saved +
- +
- +
-=== Redundancy === +
- +
-=== Prepare 2nd disc === +
-Lets assume that we have 2nd disc for other than system stuff, but we want  +
-to use it as backup solution for system disc. System should be able to boot if one  +
-disc fails. +
- +
-"By default, metadata will be mirrored across two devices and data will be striped across all of the devices present. +
-If only one device is present, metadata will be duplicated on that one device. " +
- +
-Make second disc bootable: +
-grub-install /dev/sdb +
- +
-Now we have running and bootable system from BTRFS at /dev/sda1 +
-Add second device to / volume. Device is unformatted, only partitioned. +
- +
-btrfs device add /dev/sdb1 / +
- +
-# btrfs filesystem show +
-Label: none  uuid: 2a3f41d6-571c-4ed5-91f1-ca5f5c1d3b5e +
-    Total devices 2 FS bytes used 1.37GiB +
-    devid    1 size 4.08GiB used 4.08GiB path /dev/sda1 +
-    devid    2 size 5.00GiB used 0.00B path /dev/sdb1 +
- +
-Btrfs v3.17 +
- +
-Replicate data to second volume. +
-General command: +
- +
-btrfs balance start -d <filter> -m <filter> / +
- +
-Starts balance action on data (-d) and metadata (-m). +
-For details please look into man btrfs balance +
-Full command: +
- +
-btrfs balance start -dconvert=<profile> -mconvert=<profile> / +
- +
-Possible profiles are: raid0, raid1, raid10, raid5, raid6, dup, single +
- +
-single - is to put single copy of data (not stripping) and duplicated metadata +
- +
-dup - duplicate metadata - but it is not guaranteed that metadata is duplicated on separate devices +
- +
-"For now, RAID-1 means 'one copy of what's important exists on two of the drives  +
-in the array no matter how many drives there may be in it'." +
- +
-It is possible to crate raid1 profile in degraded state, +
-to add real 2nd hard disc later. +
-As 2nd disc use small loop device and after creation remove loop device and remount +
-in degraded mode. +
- +
- +
-# btrfs fi df / +
-Data, single: total=2.69GiB, used=948.35MiB +
-System, single: total=32.00MiB, used=4.00KiB +
-Metadata, single: total=1.36GiB, used=456.89MiB +
-GlobalReserve, single: total=32.00MiB, used=0.00B +
- +
- +
-btrfs balance start -dconvert=dup -mconvert=dup / +
- +
-=== Error: unable to start balance === +
-ERROR in dmesg: +
-[ 2687.757913] BTRFS error (device sda1): unable to start balance with target data profile 32 +
- +
-Update to kernel 4.0.2 and btrfs-tools v4 doesn't help. +
-Probably dup (profile 32) cannot be used for data. +
- +
-It works: +
-btrfs balance start -dconvert=raid1 -mconvert=raid1 / +
- +
-=== Drive fail === +
-Now shutdown system and disconnect 2nd drive  +
- +
-Boot fail +
-BTRFS: failed to read the system array on sda1 +
-BTRFS: open_cree failed +
- +
-And initramfs console appear. +
- +
-mkdir /mnt +
-mount /dev/sda1 /mnt  +
- +
-the same error +
- +
-mount /dev/sda1 /mnt -odegraded +
- +
-WORKS! +
- +
-To boot automatically kernel with degraded btrfs, please add "rootflags=degraded" to kernel commandline. +
-/etc/default/grub +
-GRUB_CMDLINE_LINUX_DEFAULT="quiet rootflags=degraded" +
-Add to grub and fstab. Then update grub: +
-update-grub +
-grub-install /dev/sda +
-grub-install /dev/sdb +
- +
-Remove missing device : +
-btrfs device delete missing / +
- +
-Add new disc: +
-btrfs device add /dev/sdb / +
- +
-Remove missing device - word 'missing' is special device name for this command +
-btrfs device delete missing / +
- +
-ERROR: error removing the device 'missing' - No space left on device +
- +
-There is command +
-btrfs replace +
- +
-btrfs replace start -r 2 /dev/sdb1 / +
- +
-Because device is removed from system followin parameters are used: +
--r - use raid redundancy mechanism +
-2  - dev id  +
-/dev/sdb1 - new free device (not assigned to any btrfs) +
-/ - mount point +
- +
-Device needs to be same or greater size, but I made ideal clone of /dev/sda disc using sfdisk and  +
-above command complains that: +
-BTRFS error (device sda1): target device is smaller than source device! +
- +
-So I created bigger device and replace process starts. With some errors in dmesg: +
- +
-[ 3185.709933] BTRFS: checksum error at logical 6796906496 on dev /dev/sda1, sector 5898312, root 5, inode 163698, offset 244748288, length 4096, links 1 (path: zero) +
-[ 3185.711283] BTRFS: bdev /dev/sda1 errs: wr 0, rd 0, flush 0, corrupt 10, gen 0 +
-[ 3185.712447] BTRFS: unable to fixup (regular) error at logical 6796906496 on dev /dev/sda1 +
-[ 3190.557027] scrub_handle_errored_block: 4678 callbacks suppressed +
- +
-and finally +
- +
-[ 3305.403916] BTRFS: dev_replace from <missing disk> (devid 2) to /dev/sdb1 finished +
- +
-btrfs replace status / +
-Started on 15.Jul 22:38:55, finished on 15.Jul 22:40:55, 0 write errs, 0 uncorr. read errs +
- +
- +
- +
-Grub installed to /dev/sda /dev/sdb +
-Test - remove sda,  +
- +
-In grub menu - additional option to boot debian 8 from sdb1 shows. +
-Booting from sdb1 stops an initramfs, because option to mount degraded is missing. +
- +
-Booting from original options, stuck on  +
-"A start job is running for dev-disk-by\x2duuid-fb8d9... 55s / 1min 30s" +
-After timeout 1:30, system boots into emergency mode. +
-"Give root password for maintenance" +
-"(or type Control+D to continue) +
- +
-journalctl -a +
-Jul 16 22:22:17 debian kernel: work still pending +
-Jul 16 22:23:43 debian systemd[1]: Job dev-disk-by\x2duuid-fb8d97df\x2d625b\x2d440d\x2db63b\x2dcb529586df24.device/start timed out. +
-Jul 16 22:23:43 debian systemd[1]: Timed out waiting for device dev-disk-by\x2duuid-fb8d97df\x2d625b\x2d440d\x2db63b\x2dcb529586df24.device. +
-Jul 16 22:23:43 debian systemd[1]: Dependency failed for /home. +
-Jul 16 22:23:43 debian systemd[1]: Dependency failed for Local File Systems. +
-Jul 16 22:23:43 debian systemd[1]: Dependency failed for File System Check on /dev/disk/by-uuid/fb8d97df-625b-440d-b63b-cb529586df24. +
-Jul 16 22:23:43 debian systemd-journal[177]: Runtime journal is using 4.0M (max allowed 9.8M, trying to leave 14.8M free of 94.4M available → current limit 9. +
- +
-/lib/systemd/system/local-fs.target +
- +
- +
-This is because /home directory is missing (it was located on /dev/sda2)  +
-but whole /dev/sda removed. +
-This bad behaviour, because I want system to be bootable even without additional disc. +
- +
- +
-=== TODO === +
-Network in initramfs with SSH to repair remotely +
-Network with SSH in maintenance mode +
- +
-=== Bulletproof system === +
-Run dropbear on alternative SSH port +
-apt-get install dropbear +
-/etc/default/dropbear +
- +
- +
- +
-=== Update to kernel 4 ===+
 Edit /etc/apt/sources and add sid release Edit /etc/apt/sources and add sid release
 +<code bash>
 apt-get update apt-get update
 apt-get install linux-image-amd64 apt-get install linux-image-amd64
 apt-get install btrfs-tools apt-get install btrfs-tools
 update-initramfs -u update-initramfs -u
 +</code>
 +
 +  * ERROR: Warning: /sbin/fsck.btrfs doesn't exist, can't install to initramfs, ignoring.
 +  * EXPLANATION: "This is going to be changed back in the next version of btrfs-tools."
 +  * SOLUTION: ln -s /bin/fsck.btrfs /sbin/fsck.btrfs
 +
 +Check btrfs-tools version
 +<code bash>/bin/btrfs version</code>
  
-ERROR: Warning: /sbin/fsck.btrfs doesn't exist, can't install to initramfs, ignoring. 
-EXPLANATION: "This is going to be changed back in the next version of btrfs-tools." 
-SOLUTION: ln -s /bin/fsck.btrfs /sbin/fsck.btrfs 
  
-/bin/btrfs version+===== Converting dirs into subvolumes  ===== 
 +To make backups, snapshots, moving much easier and faster, every directory you want to backup by snapshot should be a subvolume. Also root filesystem should be created in separate subvolume not in btrfs root.
  
 +To distinguish volumes from directories we will use @ character at beginning of subvolume name. This is Ubuntu naming convention for subvolumes.
  
 +Proposed subvolumes structure (where / is BTRFS root)
 +  * /@
 +  * /@home
 +  * /@var
  
 +Mount BTRFS filesystem (BTRFS root) to /mnt/btrfs.
  
 +==== root ====
 +  * take a snapshot of root volume. Name of snapshot is '@'
 +    * ''btrfs subvolume snapshot /mnt/btrfs @''
 +  * delete files on BTRFS root (/mnt/btrfs)
 +  * make @ subvolume a default root
 +      * update fstab:<file | /etc/fstab>UUID=739e6086-d925-4bdb-94f5-26d8c10dc171 / btrfs defaults,subvol=@</file>
 +    * OR 
 +      * get @ subvolume id ''btrfs subvolume list /mnt/btrfs''
 +      * set default subvolume to use when system is mounted ''btrfs subvolume set-default 256 /mnt/btrfs''
  
  
 +==== subdirectories ====
 +  * On BTRFS root create a new subvolume
 +    * ''btrfs subvolume create /mnt/btrfs/@home''
 +  * move files into the new subvolume as if it were a directory
 +    * ''mv /mnt/btrs/@/home/* /mnt/btrfs/@home/''
 +    * ''cp --reflink'' is much faster on BTRFS
 +  * remove old directory
 +    * ''rmdir /mnt/btrfs/@/home''
 +  * update /etc/fstab to mount @home as /home
  
 +===== fstab ====
 +Do not use ''errors=remount-ro'' option which is not valid for BTRFS.
 +Example working fstab file:
 +<file | /etc/fstab>
 +UUID=ed730508-753e-4a7f-ac1e-bf8646f7bd63 / btrfs defaults,subvol=@,compress=no 0 0
 +UUID=ed730508-753e-4a7f-ac1e-bf8646f7bd63 /var btrfs defaults,subvol=@var,compress=no 0 0
 +UUID=ed730508-753e-4a7f-ac1e-bf8646f7bd63 /var/log btrfs defaults,subvol=@varlog,compress=no 0 0
 +UUID=ed730508-753e-4a7f-ac1e-bf8646f7bd63 /home btrfs defaults,subvol=@home,compress=no 0 0
 +UUID=ed730508-753e-4a7f-ac1e-bf8646f7bd63 /mnt/btrfs btrfs noauto,defaults,subvolid=5,compress=no 0 0
 +</file>
  
 +To prevent systemd to halt system during startup if device is not found (nofail and x-systemd.device-timeout):
 +<file | /etc/fstab>
 +UUID=<the_device_uuid> /mount/point btrfs nofail,x-systemd.device-timeout=10,noatime,compress=lzo,autodefrag 0 0
 +</file>
 +NOTE: do not use autodefrag - it consumes lots of cpu