au Qua Station - Part 5 - Cloning a Working Qua Station
Once you have one machine up and running, you may want to copy the existing configuration to additional machines instead of setting them up one by one. This helps ensure consistency and saves time.
There are two main parts to this:
Copy the eMMC from the existing Qua Station:
In this case, we already have an eMMC with the device tree and kernel installed, and the boot environment set up. Rather than doing all that work again, we just want to clone the entire eMMC as-is. While this is cloning more data, there is no calculation involved and less chance for human error.
Copy the root filesystem from one machine to the other:
This can be done via an intermediate device like an SD card, or by copying directly to the destination device using an external USB enclosure.
Step 1: Dump the ROM from the source machine
On the machine to be cloned (the old machine):
Unmount anything on the eMMC first if there is anything mounted. (The mounted portion would not the the part we care about, but it could affect checksums).
root@QuaStation1:~# md5sum /dev/mmcblk0
bb05643eab6165c5057f44de1c188341 /dev/mmcblk0
nohup dd status=progress bs=512 if=/dev/mmcblk0 of=./mmcblk0.img &
tail -f nohup.out
7803832320 bytes (7.8 GB, 7.3 GiB) copied, 260 s, 30.0 MB/s
15269888+0 records in
15269888+0 records out
7818182656 bytes (7.8 GB, 7.3 GiB) copied, 260.55 s, 30.0 MB/s
ls -l
-rwxr-xr-x 1 root root 7818182656 Jun 30 18:40 mmcblk0
md5sum ./mmcblk0.img
bb05643eab6165c5057f44de1c188341 /home/console/mmcblk0.img
gzip ./mmcblk0.img
Note:
1. Make sure the md5sums match, as if this file is corrupted, you could brick your new machine. Keep this number for future reference
2. The device may be mmcblk0 or mmcblk1 depending on the kernel. You can confirm this by looking at the partition map with cfdisk, or (easier) just make sure there is no SD card inserted. You should confirm the exact size to be sure. Also, if it's the correct disk, the first 95MB and the last 160MB should be free space in cfdisk.
3. We could technically copy just the first 95 and last 160MB, but to keep things simple, we just copy the entire eMMC.
Step 2: Flash the ROM on the destination machine
Method 1: (Can be used if there is no usable system on the new machine yet)
On the old machine, insert a USB disk with sufficient space, formatted as FAT32, and copy the mmcblk0.gz file to it. Another option is to put the file onto a SATA disk which you will be using in the new machine. (Note that FAT32 doesn't support files over 4GB, so you won't be able to copy the file uncompressed).
EXFAT supports larger files but may not be readable by u-boot.
(You can compress the image, put it on the USB stick, insert an HDD/SSD, mount that, copy it to there, and then decompress it there instead).
Method 2: (Can be used if there is a system up and running with networking on the new machine)
Copy the memory dump to the new machine via SCP
scp console@quastation1.local:~/mmcblk0.gz .
mmcblk0.gz 100% 396MB 7.7MB/s 00:51
Whichever method you use, the following section will be the same:
Recheck the checksum on the new machine/disk to make sure there were no issues
md5sum ./mmcblk0.img
bb05643eab6165c5057f44de1c188341 /home/console/mmcblk0.img
Reboot into the rescue system on the new machine if you're not already there. (and don't already have a system up and running).
(You can use the ZIDOO rescue disk for this, boot to u-boot, clear bootcmd and bootargs, insert the USB, and type "goru").
If you are at all interested in keeping a backup copy of the original eMMC, then you should perform the same process as in step1 on the destination machine, but call this image something like mmcblk0-backup.img.
Confirm the eMMC device on your destination machine. Again it may be /dev/mmcblk0 or /dev/mmcblk1.
Reconfirm the md5sum of the transferred file just to be sure everything is as it should be, then perform the dd command again, adjusting the output side to match your eMMC device as necessary. Next, run the md5sum command on the eMMC one final time to be sure. If any of the checksums along the way don't match, repeat the previous step until they do. Note that while unlikely, a power outage or crash while writing to the eMMC could leave you with an unbootable system.
root@QuaStation:~# md5sum ./mmcblk0-qs1-linux.img
bb05643eab6165c5057f44de1c188341 ./mmcblk0.img
root@QuaStation:~# dd bs=512 if=./mmcblk0.img of=/dev/block/mmcblk0
root@QuaStation:~# md5sum /dev/mmcblk0
7818182656 bytes (7.8 GB, 7.3 GiB) copied, 1541.28 s, 5.1 MB/s
sync
Note that the eMMC write speed is much slower than the read speed, so it will take a while this to complete.
Step 3: Back up the root filesystem from the source machine
There are two methods:
Method 1: The SD Card Method
Prepare an SD card at least as large than your root file system.
Check the device with the root file system, and the device with the SD card before proceeding.
(Make sure not to overwrite the eMMC with the root filesystem by accident!)
On the source machine:
cfdisk /dev/mmcblk1
(Create one partition spanning the whole drive)
Write and exit.
sync
dd status=progress if=/dev/sda2 of=/dev/mmcblk1p1
sync
(Note: If using Zidoo rescue, the devices will be under /dev/block/sdx1)
This should ideally be done while booted from a rescue system, but it can technically be done from a live system - although I would recommend re-mounting as read-only first to avoid inconsistencies.
In any case, check the file system after copying:
fsck.ext4 /dev/mmcblk1p1
e2fsck 1.45.5 (07-Jan-2020)
rootfs: clean, 120363/977280 files, 2398654/3902720 blocks
sync
Method 2: The external drive method
cfdisk /dev/sdb
dd status=progress if=/dev/sda2 of=/dev/sdb2
Step 4: Restore the root filesystem in the destination machine.
Assuming there is no problem, you can go to the destination system and boot up a recovery disk via USB:
Realtek>env set bootcmd=
Realtek>env set bootargs=
Realtek>end set bootdelay=5
confirm which devices are the SD Card and HDD
/ # dmesg | grep mmc
[ 8.809454] mmc2: new ultra high speed SDR104 SDHC card at address 0001
[ 8.809824] mmcblk1: mmc2:0001 SD16G 14.8 GiB
[ 8.810805] mmcblk1: p1
/ # dmesg | grep sdb
[ 8.444249] sd 1:0:0:0: [sdb] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB)
[ 8.444448] sd 1:0:0:0: [sdb] Write Protect is off
If you are using the Zidoo image, the devices will show up under /dev/block. Once you have identified the devices, you can use the dd command to copy the whole device or just the partition if you have already partitioned the destination device. (Note that the "status" option does not work for the busybox dd command in the Zidoo rescue image). Although this takes a while, you'll know it's working because the HDD access LED will be flashing.
/ # dd if=/dev/block/mmcblk1 of=/dev/block/sdb
31223808+0 records in
31223808+0 records out
sync
fsck.ext4 /dev/block/sdb1
sync
If you are using the Zidoo rescue, there is no shutdown command (and the power button doesn't work), so you can just cut the power after syncing. (Note: Sadly, there is also no fsck command on the Zidoo image, though you can take out the drive and run fsck on another machine before continuing).
Re-inserting the power should cause the machine to boot up to the new root filesystem if the root filesystem and eMMC have been copied properly.
The login and passwords will be the same as the system you copied the root filesystem from.
You may want to reinitialize SSH to get a new key (otherwise it will be a duplicate of your source system's SSH key)
First delete the existing keys, then regenerate them with dpkg-reconfigure. Select to keep the existing configuration files, the configuration script will automatically generate new keys. (Obviously, if you aren't using a Debian based system, then this part will be different).
These changes will take effect upon reboot. Follow this with sudo systemctl restart ssh or /etc/init.d/ssh restart if you want to apply the changes right away.
Next, you'll want to adjust the partitions. In order to find out what partition root is now set to, try:
root@QuaStation:~# mount | grep /dev/s
/dev/sda1 on / type ext4 (rw,noatime,data=ordered)
(adjust /etc/fstab is necessary)
cfdisk /dev/sda
If you disk is larger than the root filesystem you copied, there will be a "Free Space" section to which you can add a new partition for /home, etc.
(Technically, you might want to change the disk UUID as well, but since each machine will only have one disk, it doesn't really matter).
You may need to restart the system before the system will recognize the new partition.
Once this is done, you should be able to use mkfs to make a new filesystem in the previously empty space.
root@QuaStation:~# mkfs.btrfs /dev/sda2
btrfs-progs v4.4
See http://btrfs.wiki.kernel.org for more information.
Detected a SSD, turning off metadata duplication. Mkfs with -m dup if you want to force metadata duplication.
Performing full device TRIM (916.62GiB) ...
Label: (null)
UUID: c12303b2-327b-483a-bdec-094b356d9042
Node size: 16384
Sector size: 4096
Filesystem size: 916.62GiB
Block group profiles:
Data: single 8.00MiB
Metadata: single 8.00MiB
System: single 4.00MiB
SSD detected: yes
Incompat features: extref, skinny-metadata
Number of devices: 1
Devices:
ID SIZE PATH
1 916.62GiB /dev/sda2
root@QuaStation:~#
You an edit /etc/fstab again to add this new partition to be mounted automatically upon boot.
You probably want to update the hostname too:
nano /etc/hostname
My base root system has no users since I always add /home later, so I add my users now with adduser.
Reboot and test the boot process and logging in locally as well as over the network - The main thing is to make sure you can get into the machine before you close it back up!
Testing via Local serial connection:
Seems we're good to go, and can close her up!
Final Thoughts:
1. If you save images of the eMMC and root filesystem, then step 1 and 3 can be skipped, saving lots of time.
2. If you already have linux up and running, then you can update the eMMC without even needing to open up the machine.
3. If you are really devious, you can even update the root filesystem online. (hint: have two root filesystem partitions and two ROM images which point to different ones).
4. Much of what I am doing here is less than proper, and this kind of "dirty" method would not be recommended on a production server - but since the system we are writing to has no data on it anyway, in the worst case if we corrupt the root filesystem we can just try again.
5. Note that I am very paranoid about the eMMC - just remember, this contains the u-Boot firmware: if you trash this there is probably no way to get back into the machine.
Comments
Post a Comment