FreeBSD Root on ZFS HDD ディスク交換

前提環境

raidz2、UEFI/GPTブート、swapはgmirror

Partitioning > Auto (ZFS) でインストール

ZFS Configuration
Pool Type/Disks:  → raidz2: 8 disks
PoolName zroot
Force 4K Sectors? YES
Encrypt Disks? NO
Partition Scheme GPT (BIOS+UEFI)
Swap Size 8g
Mirror Swap? NO → YES
Encrypt Swap? NO

パーティション

$ gpart show ada0
=>        40  1953525088  ada1  GPT  (932G)
          40      409600     1  efi  (200M)
      409640        1024     2  freebsd-boot  (512K)
      410664         984        - free -  (492K)
      411648    16777216     3  freebsd-swap  (8.0G)
    17188864  1936334848     4  freebsd-zfs  (923G)
  1953523712        1416        - free -  (708K)

デグレード

ada1 がトラブルで detached > zpool REMOVED

$ zpool status
  pool: zroot
 state: DEGRADED
status: One or more devices has been removed by the administrator.
	Sufficient replicas exist for the pool to continue functioning in a
	degraded state.
action: Online the device using 'zpool online' or replace the device with
	'zpool replace'.
  scan: none requested
config:

	NAME                    STATE     READ WRITE CKSUM
	zroot                   DEGRADED     0     0     0
	  raidz2-0              DEGRADED     0     0     0
	    ada0p4              ONLINE       0     0     0
	    753047756040194058  REMOVED      0     0     0  was /dev/ada1p4
	    ada2p4              ONLINE       0     0     0
	    ada3p4              ONLINE       0     0     0
	    ada4p4              ONLINE       0     0     0

errors: No known data errors

復旧

ada1 のディスクを物理交換

パーティション設定
(空なのを確認)
$ gpart show ada1
gpart: No such geom: ada1.

(パーティションを複製)
$ gpart backup ada0 > gpart.txt
$ cat gpart.txt
GPT 128
1            efi         40     409600 efiboot0 
2   freebsd-boot     409640       1024 gptboot0 
3   freebsd-swap     411648   16777216 swap0 
4    freebsd-zfs   17188864 1936334848 zfs0 

# cat gpart.txt | gpart restore ada1

(確認)
$ gpart show ada1
=>        40  1953525088  ada1  GPT  (932G)
          40      409600     1  efi  (200M)
      409640        1024     2  freebsd-boot  (512K)
      410664         984        - free -  (492K)
      411648    16777216     3  freebsd-swap  (8.0G)
    17188864  1936334848     4  freebsd-zfs  (923G)
  1953523712        1416        - free -  (708K)
zpool リプレース
# zpool replace zroot 753047756040194058 ada1p4
Make sure to wait until resilver is done before rebooting.

If you boot from pool 'zroot', you may need to update
boot code on newly attached disk 'ada1p4'.

Assuming you use GPT partitioning and 'da0' is your new boot disk
you may use the following command:

	gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da0

(確認)
$ zpool status
  pool: zroot
 state: DEGRADED
status: One or more devices is currently being resilvered.  The pool will
	continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
  scan: resilver in progress since Fri Mar  8 12:53:58 2024
	1.80T scanned at 1.15G/s, 238G issued at 2.74G/s, 1.80T total
	4.84G resilvered, 12.96% done, 0 days 00:09:44 to go
config:

	NAME                      STATE     READ WRITE CKSUM
	zroot                     DEGRADED     0     0     0
	  raidz2-0                DEGRADED     0     0     0
	    ada0p4                ONLINE       0     0     0
	    replacing-1           REMOVED      0     0     0
	      753047756040194058  REMOVED      0     0     0  was /dev/ada1p4/old
	      ada1p4              ONLINE       0     0     0
	    ada2p4                ONLINE       0     0     0
	    ada3p4                ONLINE       0     0     0
	    ada4p4                ONLINE       0     0     0

errors: No known data errors
ブートコード更新

2 freebsd-boot なので -i 2

# gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 2 ada1  
partcode written to ada1p2
bootcode written to ada1
ESP(EFI System Partition) 更新

1 efi なので ada1p1

# newfs_msdos -F 16 -L EFISYS /dev/ada1p1

# mkdir /tmp/esp
# mount -t msdosfs /dev/ada1p1 /tmp/esp

# mkdir -p /tmp/esp/efi/boot
# cp /boot/loader.efi /tmp/esp/efi/boot/BOOTx64.efi
# echo 'BOOTx64.efi' >> /tmp/esp/efi/boot/startup.nsh

(確認)
# cd /tmp/esp/
# find .
.
./efi
./efi/boot
./efi/boot/BOOTx64.efi
./efi/boot/startup.nsh

(アンマウント)
# umount /tmp/esp

※ (既存の)ESPのファイルシステム確認

# file -s /dev/ada0p1
/dev/ada0p1: DOS/MBR boot sector, code offset 0x3c+2, OEM-ID "BSD4.4  ", sectors/cluster 16, root entries 512, sectors/FAT 100, sectors/track 63, heads 16, sectors 409600 (volumes > 32 MB), serial number 0xc7190f1d, label: "EFISYS     ", FAT (16 bit)
swap (gmirror) 復旧
(確認)
$ gmirror status
        Name    Status  Components
 mirror/swap  DEGRADED  ada0p3 (ACTIVE)
...

# gmirror forget swap

(確認)
$ gmirror status
        Name    Status  Components
 mirror/swap  COMPLETE  ada0p3 (ACTIVE)
...

# gmirror insert -v swap ada1p3
Done.

(確認)
$ gmirror status
        Name    Status  Components
 mirror/swap  DEGRADED  ada0p3 (ACTIVE)
                        ada1p3 (SYNCHRONIZING, 17%)
...
↓
$ gmirror status
        Name    Status  Components
 mirror/swap  COMPLETE  ada0p3 (ACTIVE)
                        ada1p3 (ACTIVE)
...

参考

UEFI+BIOS両用のGPTディスクの場合、freebsd-bootはada0p2になっている。もし実行例のとおり-i 1のままだと、EFIパーティションが破壊されてUEFIからブートできなくなる。またfreebsd-bootはレガシーBIOS経由のとき使われるブートコードなので、UEFIからブートする場合に使われるEFIパーティションも更新しなければいけない。現状でレガシーBIOS経由で起動していると気付かないまま放置してしまい、あとでUEFI経由で起動しようとしてハマる可能性がある。

ZFS poolのアップグレード #FreeBSD - Qiita 2020

ESPの作成
...
容量はお好みで。参考までにFreeBSDインストーラは200MBで作る模様。
BIOS/GPTブートのFreeBSD ZFS環境をUEFI/GPTブートに移行する [のふ処|NOFUDOKORO]

↑とても参考になる

startup.nsh はなくてもOK

By default UEFI will look for a EFI/BOOT/BOOTX64.EFI on the efi partition (which is just a FAT filesystem, you can mount it with msdosfs(5)). Unless those variables have been changed, maybe by a previously installed Windows or Linux installation. That's where efibootmgr(8) comes in handy, you can add/remove/change what gets booted and the order.

In any case:
Code:

mount -t msdosfs /dev/ad0p1 /mnt # assuming p1 is the efi partition
mkdir -p /mnt/EFI/BOOT
cp /boot/loader.efi /mnt/EFI/BOOT/BOOTX64.efi

If you do have a startup.nsh file already in EFI/BOOT, just make sure it has BOOTx64.efi in there, nothing else.

Upper/Lower-case doesn't matter, FAT is not case-sensitive.

https://forums.freebsd.org/threads/startup-nsh.79973/

$ sudo gmirror insert -v swap ada1p3
gmirror: Not all disks connected.

forget が必要

gmirror forget gm0

https://forums.freebsd.org/threads/gmirror-lost-disk-howto-proceed.2828/

参考 手動インストール

FreeBSD 13.1 にて検証したものになります。
オレオレ式、FreeBSDインストール手順 · GitHub

↑とても参考になる

FreeBSD/FreeBSD13-install - NORK's "HOW TO..." Wiki, It's Know How Wiki!