The methods detailed in the previous post work just fine, but can take a lot of time and lack certain flexibility.
Examples:
1. Cloning from a rw mounted filesystem with dd is really not recommended.
2. Getting access to a non-live filesystem is not easy if you want to clone from an existing system. (Sure, you could take the HDD out and image it on a 3rd machine...)
This means you have to choose between an online clone that might get corrupted, or an off-line clone which requires taking a machine down and using another machine (or partition) to take an image, etc.
Also, once you do take an image, if you want to restore it to a smaller partition, well that is very risky indeed, or requires special tools. A bigger partition is usually okay, as it seems fsck will expand the partition for you (at least for ext4).
So why didn't I use cp, tar or rsync? Well basically, because there lots of issues that can happen, and lots of stuff to break:
1. Security permissions.
3. Extended attributes
4. Hard links/soft links
5. Device magic files
6. All sorts of other stuff.
Nonetheless, the main problem I have with using dd is that if it gets stopped in the middle for some reason, you have to start again from the beginning. Therefore I wanted to give rsync a go.
Besides that, for recently setup machines, the setup I am using looks like this:
a. u-boot is set to boot from USB
b. USB contains BPI kernel I compiled
c. Root filesystem is ext4 or btrfs on the first partition of the HDD/SSD.
c. Home partition (where most everything is stored) is on the second partition of the HDD/SSD
What this means is that if I have a 64gb root filesystem, then instead of 1TB, I have only 960gb. This is worse in reality because of the space required for overhead. Look at it this way, if you have 10 machines, you are using 640GB for just the OS. Granted, the Linux I am using will easily fit into 32GB, which would mean it "only" uses 320GB per 10 machines, but I want room to grow.
I could use btrfs volumes, but I trust partitions more for my purposes. (which is to be able to easily nuke the OS and reinstall it without touching the data if need-be).
I decided that I would really prefer if the root fs would be on USB or sd-card. I went with USB for now because it was simpler and cheaper. Amazon has Sandisk 64GB USB3 memory "thumb drives" for about $10 now.
Since the entire OS can be installed on the USB disk, then it is also easier to backup, modify or troubleshoot simply by removing the USB disk.
So, to get started, first log into an existing QuaStation which you would like to copy:
1. Mount the existing USB boot drive (with the kernel) and copy its contents for later use
mkdir /mnt/sda1
mount /dev/sda1 /mnt/sda1
mkdir /console/usb-backup
cp -R /mnt/sda1 /console/usb-backup
umount /mnt/sda1
2. Insert the new USB flash disk, partition, format, and mount
Use cfdisk to resize the existing VFAT partition to 1GB or so, and add a new second partition with the remainder of the space.
mkfs.vfat /dev/sda1
mount /dev/sda1 /mnt/sda1
mkfs.btrfs /dev/sda2
mkdir /mnt/sda2
mount /dev/sda2 /mnt/sda2
3. Copy the backed up kernel/boot files to the new USB flash drive.
cp -R /console/usb-backup /mnt/sda1
4. Copy the root filesystem to the new USB flash drive.
root@quastation0:/# rsync -avPAHXx --numeric-ids / /mnt/sda2/
...
sent 7,152,742,959 bytes received 1,839,424 bytes 13,360,564.67 bytes/sec
total size is 7,974,496,643 speedup is 1.11
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1207) [sender=3.1.3]
root@quastation0:/#
Note: Given the error message, I always run it a second time to see which files were actually in error. In this case, it was the /mnt subfolders, which is fine. (Bear in mind, running it a second time will be very fast).
The whole copy operation only took a few moments, and bear in mind that was using the USB2 port since I had the USB3 port occupied with the ethernet dongle. (Yes I know I could buy a USB 3 hub, but that would probably cost more than I paid for these machines!).
If your system is still up, then there will be some logs, etc. constantly changing, so there will be some files to transfer.
Run 2 results:
sent 85,709,194 bytes received 26,687 bytes 2,559,280.03 bytes/sec
total size is 7,974,983,669 speedup is 93.02
5. Unmount the USB drive and insert it into the new machine.
umount /dev/sda1
6. Boot up the new machine with the serial console, and change the settings appropriately to tell it to boot from USB and use the second USB partition for the root drive.
In my case, that looks like this:
Realtek> env set bootcmd "usb start;fatload usb 0 0x01f00000 /bpi/bpi-w2.dtb;fatload usb 0 0x03000000 /bpi/uimage;fatload usb 0 0x02200000 /bpi/uinitrd; env set bootargs earlycon=uart8250,mmio32,0x98007800 console=ttyS0,115200n8 initrd=0x02200000,0x7F0000 root=/dev/sda2 rw rootwait rootfstype=btrfs init=/sbin/init selinux=0 nmi_watchdog=1 devtmpfs.mount=1;env set bootcmd 'booti 0x03000000 - 0x01f00000';b2ndbc"
Realtek> env save
Saving Environment to FACTORY...
[ENV] Writing to Factory...
[FAC] factory_save: MMC
[FAC] Save to eMMC (blk#:0x1100, buf:0x07000000, len:0x21e00)
[FAC] Save to eMMC (seq#:0x64, pp:0)
done
Realtek>
Once you are up and running, you will need to:
a. Change the hostname
root@quastation0:/home# nano /etc/hostname
root@quastation0:/home# bash
root@quastation12:/home#
b. delete the old SSH keys or move them our of the way
console@quastation12:/etc/ssh$ sudo /etc/init.d/ssh stop
Stopping ssh (via systemctl): ssh.service.
root@quastation12:/etc/ssh# mkdir oldkeys
root@quastation12:/etc/ssh# mv *host* oldkeys/
console@quastation12:/etc/ssh$ sudo dpkg-reconfigure openssh-server
Creating SSH2 RSA key; this may take some time ...
3072 SHA256:WPADbbZiw/1sesR7hUEOGw+LAH41W3o3t3bTTmSNWog root@quastation12.local (RSA)
Creating SSH2 ECDSA key; this may take some time ...
256 SHA256:T6SG1kklH2VNimQSsSO+LRf4xNFdNzFq6brGrr0EUaM root@quastation12.local (ECDSA)
Creating SSH2 ED25519 key; this may take some time ...
256 SHA256:/MzcpoBNlnoB9L8EUOzmGZpgnzehlHfTRTnQ5oS3c68 root@quastation12.local (ED25519)
console@quastation12:/etc/ssh$ sudo /etc/init.d/ssh start
Starting ssh (via systemctl): ssh.service.
c. Uninstall Resilio sync and delete all the config (especially the unique ID) - if you are using it.
root@quastation12:/home# apt-get remove -y resilio-sync
root@quastation12:/etc/ssh# time rm -R /var/lib/resilio-sync
real 0m0.296s
user 0m0.000s
sys 0m0.296s
root@quastation12:/etc/ssh#
d. Set up the networking (since you will use a new dongle, the device name will change, and hence /etc/networking/config or whatever you are using will change).
root@quastation12:/etc/ssh# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
link/ether a0:64:8f:bf:12:78 brd ff:ff:ff:ff:ff:ff
3: enx18ece796f85f: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
link/ether 18:ec:e7:96:f8:5f brd ff:ff:ff:ff:ff:ff
root@quastation12:/etc/ssh#
root@quastation12:/# nano /etc/network/interfaces
root@quastation12:/# ifconfig enx18ece796f85f up
root@quastation12:/# ifconfig
enx18ece796f85f: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 240d:1a:cc:5d00:1aec:e7ff:fe96:f85f prefixlen 64 scopeid 0x0<global>
inet6 fe80::1aec:e7ff:fe96:f85f prefixlen 64 scopeid 0x20<link>
ether 18:ec:e7:96:f8:5f txqueuelen 1000 (Ethernet)
RX packets 9 bytes 2189 (2.1 KB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 9 bytes 965 (965.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1 (Local Loopback)
RX packets 3397 bytes 600027 (600.0 KB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 3397 bytes 600027 (600.0 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
root@quastation12:/# dhclient
[ 1395.980368] r8169 98016000.gmac eth0: rtl_csiar_cond == 0 (loop: 100, delay: 10).
[ 1395.989496] r8169 98016000.gmac eth0: rtl_csiar_cond == 1 (loop: 100, delay: 10).
root@quastation12:/#
root@quastation12:/# ping www.google.com
PING www.google.com(nrt12s28-in-x04.1e100.net (2404:6800:4004:810::2004)) 56 data bytes
64 bytes from nrt12s28-in-x04.1e100.net (2404:6800:4004:810::2004): icmp_seq=1 ttl=116 time=3.97 ms
64 bytes from nrt12s28-in-x04.1e100.net (2404:6800:4004:810::2004): icmp_seq=2 ttl=116 time=2.86 ms
e. Delete /tmp/* and /var/log/* as appropriate
rm -R /tmp/*
rm /var/log/*.gz
f. Partition, format, and mount the internal HDD. (don't forget to update /etc/fstab).
>cfdisk /dev/satab
Write and Quit.
Next, make the filesystem. (note the -F option).
Technically, you don't even need to partition with BTRFS... but it makes it less likely that something else will clobber your disk later.
nano /etc/fstab
root@quastation0:/home# mount /home
root@quastation0:/home# mkdir /home/console
root@quastation0:/home# chown console /home/console/
Reboot to test. If /home is mounted properly, and the machine is accessible over the network, then you can close it up and reinstall resilio-sync if necessary.
root@quastation12:/etc/ssh# shutdown -r now
The network should show up during boot similar to below:
Check from remote and set up password-less login:
console@quastation0:~$ ssh-copy-id console@quastation12.local
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/home/console/.ssh/id_rsa.pub"
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
console@quastation12.local's password:
Number of key(s) added: 1
Now try logging into the machine, with: "ssh 'console@quastation12.local'"
and check to make sure that only the key(s) you wanted were added.
console@quastation0:~$ ssh-copy-id console@quastation12.local
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/home/console/.ssh/id_rsa.pub"
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
console@quastation12.local's password:
Number of key(s) added: 1
Now try logging into the machine, with: "ssh 'console@quastation12.local'"
and check to make sure that only the key(s) you wanted were added.
console@quastation0:~$ ssh quastation12.local
Welcome to Ubuntu 20.04.2 LTS (GNU/Linux 4.9.119-BPI-W2-Kernel aarch64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
System information as of Sun Sep 5 12:38:22 UTC 2021
System load: 1.06
Usage of /home: unknown
Memory usage: 11%
Swap usage: 0%
Temperature: 59.3 C
Processes: 220
Users logged in: 1
IPv4 address for enx18ece796f85f: 192.168.1.96
IPv6 address for enx18ece796f85f: 240d:1a:cc:5d00:1aec:e7ff:fe96:f85f
40 updates can be applied immediately.
To see these additional updates run: apt list --upgradable
Web console: https://quastation12.local:9090/ or https://192.168.1.96:9090/
Last login: Sun Sep 5 12:35:13 2021 from 192.168.1.67
console@quastation12:~$
The whole process took under an hour, and most of that time was spent cleaning the new system after it was booted. This time could be further reduced with a script.
Moreover, from now on, you should never need to open the machine again, since you can keep an emergency USB boot drive just for debugging purposes and won't need access to the console to boot it.
Note on gzip: If you do want to transfer an image of the entire USB stick or root filesystem over the network to or from a qua station, recommend you use:
root@quastation12:/home/console# gzip -v --fast --rsyncable ./64gb-usb.img
The -v option lets you know what's going on (since it can take a while)
The --fast option makes the compression fast(er) since the quastation can take a long time to compress a large image.
The --rsyncable option generates a file such that if you transfer an image and then later want to transfer an updated version, rsync can efficiently find the differences. This should help with ResilioSync as well.
Comments
Post a Comment