# ZFS on an usb stick



## click (Mar 7, 2011)

Hello,

I`m interested if anyone has tried installing ZFS on USB sticks (not the root or any part of the system itself)? What I have in mind is using USB sticks for daily backup - create zfs snapshot and send it to the zfs of the USB stick. 

Will I be able to boot in this configuration if the USB stick is removed? If I remove the USB stick and put it in, will I be able to retrieve the pool? Will I be able to use the USB stick pool on other system running using zfs? Would I need to label the usb GPT partition using glabel? 

The system in subject is FreeBSD 8.2-RELEASE  

Thanks


----------



## ian-nai (Mar 7, 2011)

click said:
			
		

> I`m interested if anyone has tried installing ZFS on USB sticks(not the root or any part of the system itself)?
> What I have in mind is using USB sticks for daily backup - create zfs snapshot and send it to the zfs of the USB stick.


Yes, zpool create usbstick /dev/da1 creates a zpool on whatever's at /dev/da1 (usb stick, for instance)



			
				click said:
			
		

> Will I be able to boot in this configuration if the USB stick is removed?


I don't know, if you mirror a bootable zpool onto the USB stick, probably.  Has anybody else tried this already?



			
				click said:
			
		

> If I remove the USB stick and put it in, will I be able to retrieve the pool?


Yes, all you need to do is tell zpool to export the pool contained on the usb drive *before* you remove it.

[CMD=""]zpool export usbstick[/CMD]



			
				click said:
			
		

> Will I be able to use the USB stick pool on other system running using zfs?


Yes, just connect the drive and import it.
[CMD=""]zpool import
zpool import usbstick
[/CMD]
"zpool import" tells you what is available for import and the second command imports a pool named usbstick



			
				click said:
			
		

> Would I need to label the usb GPT partition using glabel?


No, ZFS labels drives itself.  Though, I think it's "okay" to glabel a drive before passing the label to ZFS (just be sure to do it in that order...I think.)

Also, the relevant backup commands to copy a dataset (filesystem, snapshot, pool, etc) are zpool send and zpool receive.  An example would be something like:

[CMD=""]zpool send pool2send | zpool receive usbstick [/CMD]

I hope that helps!


----------



## danbi (Mar 8, 2011)

I have found that because of the write aggregation, USB sticks perform better with ZFS, than with UFS for "normal" filesystem operation. The "normal" usage is different from writing backup data as large files on the stick.

You should however test the booting process of the USB stick, because controllers are wildly different, both on the USB stick and the host USB interfaces. Some sticks don't boot on some motherboards and that's it. Also make sure you have something like 


```
kern.cam.boot_delay=10000
```

in /boot/loader.conf of the USB stick before trying to boot from it, as FreeBSD after 8.0 has some trouble with USB sticks/interfaces that are slow to discover the "drive".

If you need to boot the USB stick, you need to use either MBR or GPT label. The various ZFS-only-on-GPT guides work for USB sticks.

I am still unclean as to what your question really is. Here is what I understood:

- USB and ZFS -- good combination, yes!
- Booting USB stick with ZFS - works as expected.
- Using USB stick with ZFS filesystem in it as backup device - works as explained already.

- Using the USB stick as backup ZFS filesystem. This would work if your original ZFS filesystem is fairly small, because USB sticks are small in size, compared to other drives. There should not be trouble to send updates to the USB stick resident ZFS filesystem and it still boot. But if your data set it large, USB connected external hard drive is probably better idea.


----------



## danbi (Mar 8, 2011)

I have found that because of the write aggregation, USB sticks perform better with ZFS, than with UFS for "normal" filesystem operation. The "normal" usage is different from writing backup data as large files on the stick.

You should however test the booting process of the USB stick, because controllers are wildly different, both on the USB stick and the host USB interfaces. Some sticks don't boot on some motherboards and that's it. Also make sure you have something like 


```
kern.cam.boot_delay=10000
```

in /boot/loader.conf of the USB stick before trying to boot from it, as FreeBSD after 8.0 has some trouble with USB sticks/interfaces that are slow to discover the "drive".

If you need to boot the USB stick, you need to use either MBR or GPT label. The various ZFS-only-on-GPT guides work for USB sticks.

I am still unclean as to what your question really is. Here is what I understood:

- USB and ZFS -- good combination, yes!
- Booting USB stick with ZFS - works as expected.
- Using USB stick with ZFS filesystem in it as backup device - works as explained already.

- Using the USB stick as backup ZFS filesystem. This would work if your original ZFS filesystem is fairly small, because USB sticks are small in size, compared to other drives. There should not be trouble to send updates to the USB stick resident ZFS filesystem and it still boot. But if your data set it large, USB connected external hard drive is probably better idea.


----------



## click (Mar 9, 2011)

*Solved*

Thanks a lot ian-nai and danbi. 
I didn`t explain my goal correctly at first place, but will try now.

We had one gmirror made out of two hard drives and one single disk used for temporary storage. Until now we had daily backup on usb sticks using tar and weekly backups using dump. Now
we`ve migrated the system to zfs with the same configuration - 1 zpool in mirror with two disks and one zpool with one disk. My goal is to make snapshot and send it to the usb stick - not dump it to file. The flash drive dosen`t needs to boot, I was just wondering if in the middle of the backup the system reboots and the usb stick dies will the system be able to came up again with the broken zpool still imported.

So I`ve test it and yes the system boots up without the usb stick.

And the solution is:

Destroy the old MBR layout and create new GPT one

```
gpart destroy -F da0
gpart create -s gpt /dev/da0
```

Add partition for the zfs and label it so it can work even if it change its device name

```
gpart add -t freebsd-zfs -l weekly-backup da0
```

Create the zpool

```
zpool create weekly-backup /dev/gpt/weekly-backup
```

Exporting the zpool so we can remove the flash drive in a hurry without damaging the zpool

```
zpool export weekly-backup
```

The actual backup will be some script that does:

```
zpool import weekly-backup
zfs snapshot zstore/samba/storage@`date +%Y%m%d`
zfs send zstore/samba/storage@`date +%Y%m%d` | zfs receive weekly-backup/storage@`date +%Y%m%d`
zpool export weekly-backup
```


----------



## danbi (Mar 9, 2011)

By the way, there is not much point in labeling the USB stick, if it will be for single-drive zpool. ZFS will figure out the devices that comprise the pool all by itself -- you will inport/export the pool frequently with the USB stick anyway.

By not labeling the USB stick, it will be usage in order operating systems.


----------



## Galactic_Dominator (Mar 10, 2011)

danbi said:
			
		

> By not labeling the USB stick, it will be usage in order operating systems.


He's using GPT labels, not GEOM labels meaning every OS that supports GPT will read that label just fine.


----------



## hput (May 20, 2022)

ian-nai said:


> Yes, zpool create usbstick /dev/da1 creates a zpool on whatever's at /dev/da1 (usb stick, for instance)


A noob (hput) responds:
Please excuse the serious depth of ignorance in this reply/question.:

What about mount/unmount and accessing a zfs fs on zpool usbstick /dev/da1?
Doesn't whatever zfs fs is put on there need mounting before a zfs send/recv can happen?


----------



## grahamperrin@ (May 28, 2022)

hput said:


> Doesn't whatever zfs fs is put on there need mounting before a zfs send/recv can happen?



Welcome to FreeBSD Forums. 

When a pool is created, its root dataset is mounted. 






						zpool-create.8 — OpenZFS  documentation
					






					openzfs.github.io
				




An example: 



Spoiler: zpool create angular /dev/da4p1





```
root@mowa219-gjp4-8570p-freebsd:~ # service zfsd stop
Stopping zfsd.
Waiting for PIDS: 2426.
root@mowa219-gjp4-8570p-freebsd:~ # gdisk /dev/da4
GPT fdisk (gdisk) version 1.0.9

Partition table scan:
  MBR: not present
  BSD: not present
  APM: not present
  GPT: not present

Creating new GPT entries in memory.

Command (? for help): n
Partition number (1-128, default 1):
First sector (34-15646686, default = 2048) or {+-}size{KMGTP}:
Last sector (2048-15646686, default = 15644671) or {+-}size{KMGTP}:
Current type is A503 (FreeBSD UFS)
Hex code or GUID (L to show codes, Enter = A503): a504
Changed type of partition to 'FreeBSD ZFS'

Command (? for help): w

Final checks complete. About to write GPT data. THIS WILL OVERWRITE EXISTING
PARTITIONS!!

Do you want to proceed? (Y/N): y
OK; writing new GUID partition table (GPT) to /dev/da4.
Warning: The kernel may continue to use old or deleted partitions.
You should reboot or remove the drive.
The operation has completed successfully.
root@mowa219-gjp4-8570p-freebsd:~ # zpool create angular /dev/da4p1
root@mowa219-gjp4-8570p-freebsd:~ # zfs list angular
NAME      USED  AVAIL  REFER  MOUNTPOINT
angular   360K  6.78G    96K  /angular
root@mowa219-gjp4-8570p-freebsd:~ # zpool destroy angular
root@mowa219-gjp4-8570p-freebsd:~ # zpool labelclear /dev/da4p1
root@mowa219-gjp4-8570p-freebsd:~ # gdisk /dev/da4
GPT fdisk (gdisk) version 1.0.9

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.

Command (? for help): x

Expert command (? for help): z
About to wipe out GPT on /dev/da4. Proceed? (Y/N): y
Warning: The kernel may continue to use old or deleted partitions.
You should reboot or remove the drive.
GPT data structures destroyed! You may now partition the disk using fdisk or
other utilities.
Blank out MBR? (Y/N): y
root@mowa219-gjp4-8570p-freebsd:~ # service zfsd start
Starting zfsd.
root@mowa219-gjp4-8570p-freebsd:~ # uname -aKU
FreeBSD mowa219-gjp4-8570p-freebsd 14.0-CURRENT FreeBSD 14.0-CURRENT #12 main-n255769-f16e38162c7-dirty: Tue May 24 11:48:57 BST 2022     root@mowa219-gjp4-8570p-freebsd:/usr/obj/usr/src/amd64.amd64/sys/GENERIC-NODEBUG amd64 1400059 1400059
root@mowa219-gjp4-8570p-freebsd:~ # zfs --version
zfs-2.1.99-FreeBSD_gc0cf6ed67
zfs-kmod-2.1.99-FreeBSD_gc0cf6ed67
root@mowa219-gjp4-8570p-freebsd:~ #
```


----------

