meta data for this page
  •  

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
linux:fs:btrfs:send [2024/11/20 15:07] – niziaklinux:fs:btrfs:send [2024/11/21 17:32] (current) – niziak
Line 1: Line 1:
 ====== send / receive ====== ====== send / receive ======
 +
 +grub-install --target=x86_64-efi --efi-directory=/target/@rootfs/boot/efi --boot-directory=/target/@rootfs/boot --bootloader-id=debian --recheck --no-floppy
 +update-grub 
 +
 +mount --rbind /dev  /target/@rootfs/dev
 +mount --rbind /proc /target/@rootfs/proc
 +mount --rbind /sys  /target/@rootfs/sys
 +chroot /target/@rootfs /bin/bash --login
 +
 +grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=debian --recheck --no-floppy
 +Installing for x86_64-efi platform.
 +grub-install: error: cannot find a device for /boot/grub (is /dev mounted?).
 +
  
 ===== migrate to new storage ===== ===== migrate to new storage =====
 +
 +Use Clonezilla live USB with latest 6.11 kernel.
 +
 +There is a hole between Windows partitions and Windows recovery partitions:
 +<code>
 +parted /dev/nvme0n1
 +print free
 +Number  Start   End     Size    File system  Name                          Flags
 +        17.4kB  1049kB  1031kB  Free Space
 + 1      1049kB  106MB   105MB   fat32        EFI system partition          boot, esp, no_automount
 + 2      106MB   123MB   16.8MB               Microsoft reserved partition  msftres, no_automount
 + 3      123MB   60.3GB  60.2GB  ntfs         Basic data partition          msftdata
 +        60.3GB  511GB   451GB   Free Space
 + 4      511GB   512GB   944MB   ntfs         Basic data partition          hidden, diag, no_automount
 +        512GB   512GB   335kB   Free Space
 +</code>
 +
 +So create 16GB swap at end and BTRFS for root part.
 +
 +<code bash>
 +parted /dev/nvme0n1
 +
 +mkpart swap sw 495G 511G
 +mkpart root btrfs 60.3GB  495GB
 +</code>
 +
 +<code bash>
 +mkfs.btrfs /dev/nvme0n1p6
 +mkdir /target
 +mount /dev/nvme0n1p6 /target
 +</code>
  
 <code bash> <code bash>
 sudo apt install netcat-openbsd pv zstd sudo apt install netcat-openbsd pv zstd
 </code> </code>
 +
 +==== rsync ====
 +
 +Rsync send is not related to BTRFS, but here it can help if FS is ''ro'' state due to errors.
 +
 +Recreate destination volumes
 +
 +<code bash>
 +cd /target
 +btrfs sub create @rootfs
 +btrfs sub create @rootfs/home
 +btrfs sub create @rootfs/var
 +btrfs sub create @rootfs/var/log
 +btrfs sub create @rootfs/var/cache
 +btrfs sub create @rootfs/var/spool
 +btrfs sub create @rootfs/var/lib
 +btrfs sub create @rootfs/var/lib/docker
 +btrfs sub create @rootfs/var/snap
 +btrfs sub create @rootfs/opt
 +btrfs sub create @rootfs/usr
 +btrfs sub create @rootfs/usr/local
 +
 +btrfs sub create @rootfs/home/user
 +btrfs sub create @rootfs/home/user/prj
 +btrfs sub create @rootfs/home/user/Documents
 +</code>
 +
 +Rsync option explanation:
 +  * ''a'' --archive, -a archive mode is -rlptgoD (no -A,-X,-U,-N,-H)
 +    *        --recursive, -r          recurse into directories
 +    *        --links, -l
 +              When  symlinks are encountered, recreate the symlink on the des‐
 +              tination.
 +    *     --perms, -p              preserve permissions
 +    *        --times, -t              preserve modification times
 +    *        --group, -g              preserve group
 +    *        --owner, -o              preserve owner (super-user only)
 +    *        -D                       same as --devices --specials
 +      *        --devices                preserve device files (super-user only)
 +      *        --specials               preserve special files
 +  * ''v'' --verbose, -v increase verbosity
 +  * ''x'' --one-file-system, -x don't cross filesystem boundaries
 +
 +  * ''H''       --hard-links, -H         preserve hard links
 +  * ''A''       --acls, -A               preserve ACLs (implies --perms)
 +  * ''X'' --xattrs, -X  preserve extended attributes
 +  * ''P'' -P                       same as --partial --progress
 +  * --numeric-ids            don't map uid/gid values by user/group name
 +  *       --whole-file, -W         copy files whole (w/o delta-xfer algorithm)
 +
 +<file ini /etc/rsyncd.conf>
 +[rootfs]
 +        path = /target/@rootfs
 +</file>
 +rsync -v --daemon
 +
 +on client, list rsync modules:
 +
 +rsync 192.168.177.161::
 +rootfs         
 +
 +
 +Now rsync but do not limit to one filesystem, to copy all subvolumes into its places (already created on dst):
 +
 +<code bash>
 +rsync -avHAXPW --exclude=".snapshots" 192.168.177.161::rootfs /target/@rootfs
 +</code>
 +
 +Try ssh to correct permission errors
 +<code bash>
 +rsync -avHAXPW --exclude=".snapshots" ssh://user@192.168.177.161/target/@rootfs/ /target/@rootfs
 +</code>
 +
 +No need to add on not live FS:
 +<code>
 +--exclude='/dev' --exclude='/proc' --exclude='/sys'
 +</code>
 +
 +Even if rsync daemon is started from root account (on src host) it cannot read some files according to normal permissions:
 +<code>
 +rsync: [sender] send_files failed to open "/var/spool/postfix/pid/unix.local" (in rootfs): Permission denied (13)
 +rsync: [sender] send_files failed to open "/var/spool/postfix/pid/unix.retry" (in rootfs): Permission denied (13)
 +rsync: [sender] send_files failed to open "/var/spool/postfix/pid/unix.smtp" (in rootfs): Permission denied (13)
 +</code>
 +
 +workaround is to start rsync daemon on dst host with the same rsyncd.conf config and start rsync from src host:
 +
 +<code bash>
 +rsync -avxHAXPW --exclude=".snapshots" /target/@rootfs 192.168.177.171::rootfs 
 +</code>
 +
 +
 +tar:
 +  * ''-c'' create
 +  * ''-p'', ''--preserve-permissions'', ''--same-permissions'' extract information about file permissions  (default  for  superuser)
 +  * ''-s'', '' --preserve-order'', ''--same-order'' Sort names to extract to match archive
 +  * ''-f'', ''--file=ARCHIVE'' Use  archive  file  or  device  ARCHIVE.   If this option is not given, tar will first examine the enviro
 +            
 +             
 +Sender:
 +<code bash>
 +#tar -czpsf - /target/@rootfs | pv -b | nc -l -p 44444
 +cd /target/@rootfs
 +tar --acls --xattrs -cpsf - . | zstd --fast -T0 | pv | nc -w 2 -N 192.168.177.171 44444
 +</code>
 +
 +Receiver:
 +<code bash>
 +#nc 192.168.1.4 3333 | pv -b | tar --acls --xattrs -xzpsf -
 +
 +cd /target/@rootfs
 +nc -vl 44444 | zstd -d | pv | tar -xpsf -
 +
 +</code>
 +
 +==== btrfs send ====
 +
 +It can send only ''ro'' volumes. If FS is broken and forced ''ro'' you cannot set volumes to ''ro'' :)
 +Only possible to send  ''ro'' snapshots.
  
 Sender: Sender:
 <code bash> <code bash>
-btrfs send | zstd --fast -T0 | pv | nc -w 2 -N <remote_ip> 44444+btrfs property set -ts /target/@roots ro true 
 +btrfs send /target/@rootfs | zstd --fast -T0 | pv | nc -w 2 -N 192.168.177.171 44444
 </code> </code>
  
 Receiver: Receiver:
 <code bash> <code bash>
-nc -vl 44444 | zstd -d | pv | btrfs receive+nc -vl 44444 | zstd -d | pv | btrfs receive /target
 </code> </code>