# Mount zroot/usr/home on another disc.



## judd (Sep 13, 2020)

Good afternoon.

I want to mount zroot/usr/home to the second 500 GB HHD disk of the same laptop where FBSD 12.1 is installed on a 30 GB SSD:


```
# lsblk
DEVICE         MAJ:MIN SIZE TYPE                          LABEL MOUNT
ada0             0:97  466G GPT                               - -
  <FREE>         -:-   1.0M -                                 - -
  ada0p1         0:99  466G linux-data                        - -
  <FREE>         -:-   7.5K -                                 - -
ada1             0:98   30G GPT                               - -
  ada1p1         0:100 200M efi                    gpt/efiboot0 -
  ada1p2         0:101 512K freebsd-boot           gpt/gptboot0 -
  <FREE>         -:-   492K -                                 - -
  ada1p3         0:102 8.0G freebsd-swap              gpt/swap0 SWAP
  ada1p4         0:103  22G freebsd-zfs                gpt/zfs0 <ZFS>
  ada1p4.eli     0:109  22G zfs                               - -
  <FREE>         -:-   836K -                                 - -
```

First I looked for defective parts of the disc (ada0) with dd:


```
# dd if=/dev/ada0 of=/dev/null  bs=1M
476940+1 records in
476940+1 records out
500107862016 bytes transferred in 5517.928814 secs (90633257 bytes/sec)
```

I then proceeded to delete everything:

`# gpart destroy -F ada0
ada0 destroyed`

And so it is now:

`# lsblk ada0
DEVICE         MAJ:MIN SIZE TYPE                          LABEL MOUNT
ada0             0:97  466G GPT                               - -
<FREE> -:- 466G - - -`

What I'm trying to do is have zroot/usr/home (now 2% occupied), take care of my data like photos, videos, etc. as any /home partition on the HDD and thus free up the SSD for the root and its sub-directories. since the SSD where the system is installed has very little storage.


```
# df -hT
Filesystem          Type     Size    Used   Avail Capacity  Mounted on
zroot/ROOT/default  zfs       21G    7,4G     13G    36%    /
devfs               devfs    1,0K    1,0K      0B   100%    /dev
zroot/var/log       zfs       13G    436K     13G     0%    /var/log
zroot/usr/home      zfs       13G    249M     13G     2%    /usr/home
zroot               zfs       13G     88K     13G     0%    /zroot
zroot/var/crash     zfs       13G     88K     13G     0%    /var/crash
zroot/usr/ports     zfs       13G     88K     13G     0%    /usr/ports
zroot/var/mail      zfs       13G    112K     13G     0%    /var/mail
zroot/var/audit     zfs       13G     88K     13G     0%    /var/audit
zroot/tmp           zfs       13G    6,9M     13G     0%    /tmp
zroot/var/tmp       zfs       13G     88K     13G     0%    /var/tmp
zroot/usr/src       zfs       13G     88K     13G     0%    /usr/src
```

`# cat /etc/fstab
# Device                Mountpoint      FStype  Options         Dump    Pass#
/dev/ada1p3 none swap sw 0 0`


I have honestly never done this and if you help me, I would like to make it happen. Then I'd have FBSD running totally, like a single disk, and it's already running at peak performance!

Thanks as always.


----------



## T-Daemon (Sep 13, 2020)

Proceed as follows:

```
gpart create -s gpt ada0
gpart add -t freebsd-zfs ada0

zfs rename  zroot/usr/home zroot/usr/home0
zpool create -O compression=lz4 -O atime=off -m /usr/home 500G ada0

( 500G is the pool name, deviated from the disk size, eventually set other name )

Check mountpoints:
df -h

cp -a /usr/home0/* /usr/home

Log in as a user, check for error messages, if there are none, proceed:

zfs destroy zroot/usr/home0
```

If you are planing to trust valuable data on that laptop, make yourself familiar with _zfs send snapshot_ to external disks for backups. In addition make also backups you are used to make until now.


----------



## judd (Sep 14, 2020)

T-Daemon 

First, thank you for your time !

There were errors:

```
root@solo:/usr/home/solo # gpart create -s gpt ada0
gpart: geom 'ada0': File exists
root@solo:/usr/home/solo # gpart add -t freebsd-zfs ada0
gpart: table 'ada0' is corrupt: Operation not permitted
root@solo:/usr/home/solo # zfs rename  zroot/usr/home zroot/usr/home0
cannot unmount '/usr/home': Device busy
root@solo:/usr/home/solo # zpool create -O compression=lz4 -O atime=off -m /usr/home 500G ada0
mountpoint '/usr/home' exists and is not empty
use '-m' option to provide a different default
root@solo:/usr/home/solo # df -h
Filesystem            Size    Used   Avail Capacity  Mounted on
zroot/ROOT/default     21G    7,4G     13G    36%    /
devfs                 1,0K    1,0K      0B   100%    /dev
zroot/tmp              13G    6,8M     13G     0%    /tmp
zroot/usr/home         13G    254M     13G     2%    /usr/home
zroot/var/crash        13G     88K     13G     0%    /var/crash
zroot/usr/src          13G     88K     13G     0%    /usr/src
zroot                  13G     88K     13G     0%    /zroot
zroot/var/audit        13G     88K     13G     0%    /var/audit
zroot/var/log          13G    440K     13G     0%    /var/log
zroot/usr/ports        13G     88K     13G     0%    /usr/ports
zroot/var/mail         13G    112K     13G     0%    /var/mail
zroot/var/tmp          13G     96K     13G     0%    /var/tmp
root@solo:/usr/home/solo # cp -a /usr/home0/* /usr/home
cp: No match.
root@solo:/usr/home/solo #
```


----------



## judd (Sep 14, 2020)

Right now I'm deleting again *ada0* but this one you see with dd.


```
# dd if=/dev/zero of=/dev/ada0 bs=1m
dd: /dev/ada0: short write on character device
dd: /dev/ada0: end of device
476941+0 records in
476940+1 records out
500107862016 bytes transferred in 5517.884153 secs (90633991 bytes/sec)
```

With the dd parameter it worked:


```
lsblk
DEVICE         MAJ:MIN SIZE TYPE                          LABEL MOUNT
ada0             0:97  466G GPT                               - -
  ada0p1         0:99  466G freebsd-zfs                       - <ZFS>
ada1             0:98   30G GPT                               - -
  ada1p1         0:100 200M efi                    gpt/efiboot0 -
  ada1p2         0:101 512K freebsd-boot           gpt/gptboot0 -
  <FREE>         -:-   492K -                                 - -
  ada1p3         0:102 8.0G freebsd-swap              gpt/swap0 SWAP
  ada1p4         0:103  22G freebsd-zfs                gpt/zfs0 <ZFS>
  ada1p4.eli     0:108  22G -                                 - -
  <FREE>         -:-   836K -                                 - -
```


```
df -h
Filesystem            Size    Used   Avail Capacity  Mounted on
zroot/ROOT/default     21G    7,4G     13G    36%    /
devfs                 1,0K    1,0K      0B   100%    /dev
zroot/usr/home         13G    257M     13G     2%    /usr/home  >>>>>is still here.
zroot/tmp              13G    6,7M     13G     0%    /tmp
zroot/var/tmp          13G     88K     13G     0%    /var/tmp
zroot/var/crash        13G     88K     13G     0%    /var/crash
zroot/var/audit        13G     88K     13G     0%    /var/audit
zroot/usr/ports        13G     88K     13G     0%    /usr/ports
zroot/var/log          13G    448K     13G     0%    /var/log
zroot                  13G     88K     13G     0%    /zroot
zroot/usr/src          13G     88K     13G     0%    /usr/src
zroot/var/mail         13G    112K     13G     0%    /var/mail
```


```
# fdisk ada0
******* Working on device /dev/ada0 *******
parameters extracted from in-core disklabel are:
cylinders=969021 heads=16 sectors/track=63 (1008 blks/cyl)

Figures below won't work with BIOS for partitions not in cyl 1
parameters to be used for BIOS calculations are:
cylinders=969021 heads=16 sectors/track=63 (1008 blks/cyl)

Media sector size is 512
Warning: BIOS sector numbering starts with sector 1
Information from DOS bootblock is:
The data for partition 1 is:
sysid 238 (0xee),(EFI GPT)
    start 1, size 976773167 (476940 Meg), flag 0
        beg: cyl 0/ head 0/ sector 2;
        end: cyl 1023/ head 255/ sector 63
The data for partition 2 is:
<UNUSED>
The data for partition 3 is:
<UNUSED>
The data for partition 4 is:
<UNUSED>
```

`# zfs destroy zroot/usr/home0
cannot open 'zroot/usr/home0': dataset does not exist.  ???`


Is the fstab as it is or should a line be added ?
# cat /etc/fstab

```
# Device                Mountpoint      FStype  Options         Dump    Pass#
/dev/ada1p3             none    swap    sw              0       0
```


----------



## T-Daemon (Sep 14, 2020)

judd said:


> root@solo:/usr/home/solo # gpart create -s gpt ada0
> gpart: geom 'ada0': File exists
> root@solo:/usr/home/solo # gpart add -t freebsd-zfs ada0
> gpart: table 'ada0' is corrupt: Operation not permitted
> ...



I expected after your `gpart destroy -F ada0` ada0 with no partition table, also `gpart: table 'ada0' is corrupt: Operation not permitted` needs to be investigated.

Furthermore I expected you where executing those commands from /root not /usr/home/solo, this can't work from there, zroot/usr/home can't be renamed as long you are in that directory, as consequence no pool with mountpoint /usr/home on ada0 can be created when there is a second one on zroot .

On top of that I overlooked you have full disk encryption, sorry for that, it was late when I responded to your posting.

There are two options, first we could keep trying to create on ada0 a encrypted/unencrypted /usr/home, or second option install a encrypted root-on-ZFS system on ada0, and copy the data from ada1 over ( zfs send / receive system configuration, packages and their configuration and databases, etc. ). 

The second option is cleaner, no scattered file system over two disks, makes in case of a system failure a full disk restoration easier.
I will be gone for a day or two, don't be alarmed when not receiving a response. In that time others can join in to help.


----------



## judd (Sep 14, 2020)

T-Daemon said:


> Furthermore I expected you where executing those commands from /root not /usr/home/solo, this can't work from there, zroot/usr/home can't be renamed as long you are in that directory, as consequence no pool with mountpoint /usr/home on ada0 can be created when there is a second one on zroot .



I understand, now I understand my failure. Of course I didn't leave the directory.

I have no problem removing an installation and reinstalling so I can have /usr/home on ada0.
Although it is true, as you say, that /usr/home is not on another disk in case something needs to be fixed.


----------



## Mjölnir (Sep 14, 2020)

It is essential that you give distinctive names to the two zpool(8)s.
You can use PEFS pefs-kmod and only encrypt sensible data instead of GELI encryption.  pefs(8) is very mature & works on top of any filesystem.  Then you can easily switch to ZFS native encryption later, once OpenZFS has stabilized (around FreeBSD 13.3 or 14.x).
Setting `vfs.zfs.mdcomp_disable="1"` in loader.conf(5) will speed up file operations on the SSD.
Since a laptop has an UPS built-in (the battery), you can optionally set `vfs.zfs.cache_flush_disable=1` & `vfs.zfs.vdev.bio_flush_disable=1` in sysctl.conf(5).  This is _non-standard -- do at your own risk --_ but I never had any bad experience. Do not do this if your HDD is connected external (e.g. via USB), because if it gets disconnected accidently, the filesystem can get corrupted & result in irrecoverable data loss.
Set `zfs set atime=off` on both pools and `zfs inherit` that for all datasets, and only have it on for /var/mail.
Concerning overwiting the partition table see this post & the one I posted before that.


----------



## judd (Sep 14, 2020)

[QUOTE = "mjollnir, publicación: 478153, miembro: 37644"]
Es esencial que asigne nombres distintivos a los dos zpool (8) s.
[/CITAR]

Lo escribo y luego lo leo. : +1:

[QUOTE = "mjollnir, publicación: 478153, miembro: 37644"]
You can use PEFS pefs-kmod and only encrypt sensible data instead of GELI encryption. pefs(8) is very mature & works on top of any filesystem. Then you can easily switch to ZFS native encryption later, once OpenZFS has stabilized (around FreeBSD 13.3 or 14.x).
[/QUOTE]

Ok, I have not yet experienced PEFS ...
Another thing to read, I'm running out of eyes ...



mjollnir said:


> Setting  vfs.zfs.mdcomp_disable="1" in loader.conf(5) will speed up file operations on the SSD.



Well, I deleted the SSD and installed FBSD on the HDD.

```
lsblk
DEVICE         MAJ:MIN SIZE TYPE                          LABEL MOUNT
ada0             0:96  466G GPT                               - -
  ada0p1         0:98  200M efi                    gpt/efiboot0 -
  ada0p2         0:99  512K freebsd-boot           gpt/gptboot0 -
  <FREE>         -:-   492K -                                 - -
  ada0p3         0:100 8.0G freebsd-swap              gpt/swap0 SWAP
  ada0p4         0:101 458G freebsd-zfs                gpt/zfs0 <ZFS>
  ada0p4.eli     0:106 458G -                                 - -
  <FREE>         -:-   4.0K -                                 - -
ada1             0:97     - -                                 - -
```

I'll write it down and leave it for the next installation on the SSD.



mjollnir said:


> Since a laptop has an UPS built-in (the battery), you can optionally set  vfs.zfs.cache_flush_disable=1 &  vfs.zfs.vdev.bio_flush_disable=1 in sysctl.conf(5). This is _non-standard -- do at your own risk --_ but I never had any bad experience. Do not do this if your HDD is connected external (e.g. via USB), because if it gets disconnected accidently, the filesystem can get corrupted & result in irrecoverable data loss.



I proceed and then comment.
I never use an external hard drive in the laptop, I use it in the desktop computer.




mjollnir said:


> Set  zfs set atime=off on both pools and  zfs inherit that for all datasets, and only have it on for /var/mail.



I don't understand it.



mjollnir said:


> Concerning overwiting the partition table see this post & the one I posted before that.



If I have read them, they are interesting!

~~

Actually, my head and my eyes are about to explode, so I'm going to take it one step at a time. It's too much reading, although necessary, of course.


----------

