# zfs - reinstall - totally lost!



## BeautifulFish (Dec 1, 2009)

i am very new to zfs so i am very lost..

i have a desktop pc with 2GB RAM

i am used to having root "/" on one partition and home "/home" on another partition, that is when i use linux
i do that to keep my data protected, if problems occur i can easily format "/" and install a new system

now to FreeBSD, what i need to do now is have "/" on one partition and "/usr/home" on another partition, correct me if i am wrong?
that is easy to do with sysinstall..

but with zfs, i am very lost
here is the guide i followed:
http://www.ish.com.au/solutions/articles/freebsdzfs

i am not sure i totally understand the guide, but i think the root partition "/" created at first will be only used as "/boot" later???
also through the guide, i created only one partition, i tried creating more than 1 partition and put them in the same zpool, but now i think the data will be shared between both partitions, so i can't just format one of them
also i tried creating a second zpool for "/usr" but the boot process failed..


so anyway, my questions are:
how can i achieve "/" on one partition and "/usr/home" on another partitions? or how can i do it through zpools?

how can i reinstall FreeBSD without messing with my zpools and mountpoints and "/usr/home"


thanks in advance for your help


----------



## Blueprint (Dec 2, 2009)

No offence, but I think you should first research and understand how pooled storage works before going any further.


----------



## BeautifulFish (Dec 2, 2009)

none taken 
i have tried to understand it, it is too hard!

the good news is that i have successfully followed the guide in the link in my previous post
but now i have a boot partition, a swap partition and a zfs partition
the zfs partition contains everything, so lets say i am going to reinstall FreeBSD now, is there a way to just delete everything except "/usr/home"?

EDIT:
i think i solved the main part of my problem now, i created another additional partition, then created another zpool especially for /usr/home 

so my current question is:
how can i reinstall FreeBSD?


----------



## graudeejs (Dec 2, 2009)

No you do it wrong..... you need only 1 pool

Before anything Backup your data on some media, that is guaranteed you won't destroy



First thing you need to boot from fixit cd or dvd and enter fixit mode.
Do this:

```
# ln -s /dist/boot/kernel /boot/kernel
# ln -s /dist/lib /lib
# kldload zfs
```

create pool let's name it coolpool
Set options for coolpool, that you think will be default for most zfs partitions....

_NOTE: I haven't made any system that boots straight from zfs, so this is undiscovered field for me. I use HDD encryption, so that's not even possible... I boot from flash._

No I won't be using coolpool as root...., but instead will use coolpool/root as root

```
# zfs set mountpoint=none coolpool
# zpool set altroot=/mnt coolpool
# zfs create coolpool/root
# zfs create coolpool/home
# zfs create coolpool/tmp
# zfs set mountpoint=/home coolpool/home
# zfs set mountpoint=/tmp coolpool/tmp
# zfs set mountpoint=/ coolpool/root
# zfs create coolpool/root/boot
[B]# zpool set bootfs=/coolpool/root/boot[/B]
# zfs set compression=off coolpool/root/boot
```
Note: boot partition can't be compressed

as I understand zpool set bootfs is like active flag in MBR 

now we need to mount eveything needed to install base system

Check that coolpool/root and coolpool/root/boot is mounted to /mnt
If not mount then manually

and now fallow This tread to install system without sysinstal 

After that `# cp -Rf /boot/zfs /mnt/boot/`

*WARNING: Do not export coolpool. Simply reboot.*
If all goes well you will boot into shiny new FreeBSD.
*WARNING: something will go wrong.*

NOTE: I wrote this from my head... It may not work, there may be mistakes/errors, but if you have access to working PC, I can help you....
WARNING: I have never configured zfs only setup. In this guide I was improvising from what I know about zfs. I don't know if everything works.... But since you made backups  you can try ....
NOTE: If it wouldn't work for me I could fix it.... Hope you have enough skills


----------



## BeautifulFish (Dec 2, 2009)

thanks very much for your time  

i couldn't set altroot after pool creation, i searched for it:

```
# zpool create -R/mnt coolpool /dev/ad0s1d
```
also a little modification:

```
# zpool set bootfs=coolpool/root/boot coolpool
```

so i got to the point where i will have to follow the FreeBSD installation without sysinstall and i got lost
for example, i dont have the iso image available, but i have the dvd...  also it says there to use fdisk, but i already have partitions as i have created the pool 

oh i am still not sure where zfs information is kept, i don't know what export is lol  last night i tried to delete the partitions, but zfs was still there  i thought it was like a directory /coolpool but it turned out to be like a virus :e

so now i:
have the pool
have no installation
don't know how my /home will be safe
don't know how to reinstall FreeBSD

i have a working pc as i am doing this now on virtualbox before trying it on my real hard drive

thanks again


----------



## graudeejs (Dec 2, 2009)

OK, stick to default FreeBSD installation... Install ufs....
Learn FreeBSD command line etc.....

Once you feel little experienced then start thinking about zfs.... and stuff....

I really recommend that you start with basics.


----------



## BeautifulFish (Dec 2, 2009)

but i am so close to make it 

i mean for re-installation, if i have 2 zpools, can't i just destroy the zpool for FreeBSD installation and create another one, then import the 2nd zpool for "/usr/home"?


----------



## wonslung (Dec 2, 2009)

All you need is one pool.  The beauty of ZFS is that is is a pooled filesystem, meaning you have a single large pool of space, which your various filesystems can use.

let's say you have a pool called tank
and you want to have several filesystem

you might have

```
tank/var
tank/usr
tank/usr/home
tank/usr/ports
tank/usr/ports/distfiles
```

all of these filesystems will be sepearate but they will all share space from the entire pool

you can also set different settings for each, like you may set compression=gzip-9 for tank/usr/ports because it's highly compressable data but you would probably not set this for tank/usr/ports/distfiles because these files are already compressed.

having said that, if you can't figure out how to follow the guide, there is nothing wrong having a ufs based install and then a ZFS based pool for storage.

also, you said you didn't understand where the zfs information is kept.

It is kept in the filesystem...and the filesystems are kept in the pool.  The filesystems can have children (for example, tank/usr/ports would be the child of tank/usr/ in our previous example)  and you can mount the filesystems anywhere you choose.  by default, when you create a Zpool, it will be mounted on a directory of the same name, so if you create zpool tank, it would be mounted on /tank and any filesystems would be mounted within

You can easily change the mount point of a zfs filesystem simply by using the command:


```
zfs set mountpoint
```
for example, if you made a zfs filesystem tank and you wanted to mount tank/home at /home instead of /tank/home you could do something like this:

```
zfs set mountpoint=/home tank/home
```


----------



## graudeejs (Dec 2, 2009)

BeautifulFish said:
			
		

> so now i:
> have the pool
> have no installation
> don't know how my /home will be safe
> don't know how to reinstall FreeBSD



You don't have almost nothing... creating pool is peace of cake...
Did you create backups???

What does this "have no installation" mean?


In my 1st post I wrote pretty much everything needed to install FreeBSD on zfs... I don't know if it works.. but It's there


----------



## BeautifulFish (Dec 2, 2009)

wonslung said:
			
		

> All you need is one pool.  The beauty of ZFS is that is is a pooled filesystem, meaning you have a single large pool of space, which your various filesystems can use.
> 
> let's say you have a pool called tank
> and you want to have several filesystem
> ...



thanks for your help 
i have read about zfs so i know the idea of pools, but as you said they all share the same space, so it is not like having a partition for root and a partition for home, so when things go wrong i can format root partition without touching my data at home
if i have one pool for FreeBSD installation and i destroyed it then i will destroy my data too at tank/usr/home
and still i don't know how to destroy a pool on which i installed FreeBSD




			
				wonslung said:
			
		

> also, you said you didn't understand where the zfs information is kept.
> 
> It is kept in the filesystem...and the filesystems are kept in the pool.  The filesystems can have children (for example, tank/usr/ports would be the child of tank/usr/ in our previous example)  and you can mount the filesystems anywhere you choose.  by default, when you create a Zpool, it will be mounted on a directory of the same name, so if you create zpool tank, it would be mounted on /tank and any filesystems would be mounted within
> 
> ...



so zpools are actually directories under "/"?
then we activate zfs in kernel
then tell the system to boot from that directory and convince it that "/tank" is "/"

i hope i am making any sense 




			
				killasmurf86 said:
			
		

> You don't have almost nothing... creating pool is peace of cake...
> Did you create backups???
> 
> What does this "have no installation" mean?
> ...



yeah i know it is easy to create a pool, i am using virtualbox so i am not worrying about my data now  btw great guide for backups 

i have no installation as i couldn't follow the guide for installing FreeBSD without sysinstall, i mentioned why i couldn't follow it


----------



## wonslung (Dec 2, 2009)

BeautifulFish said:
			
		

> thanks for your help
> i have read about zfs so i know the idea of pools, but as you said they all share the same space, so it is not like having a partition for root and a partition for home, so when things go wrong i can format root partition without touching my data at home
> if i have one pool for FreeBSD installation and i destroyed it then i will destroy my data too at tank/usr/home
> and still i don't know how to destroy a pool on which i installed FreeBSD


you wouldn't ever destroy an entire pool unless you had backups or didn't need the data.  On the otherhand, you can easily destory a FILESYSTEM.  honestly man, you need to READ up on this stuff...this is VERY BASIC knowledge when it comes to zfs.  Read http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide and http://flux.org.uk/howto/solaris/zfs_tutorial_01  those are for solaris, but the zfs information transends.





> yeah i know it is easy to create a pool, i am using virtualbox so i am not worrying about my data now  btw great guide for backups
> 
> i have no installation as i couldn't follow the guide for installing FreeBSD without sysinstall, i mentioned why i couldn't follow it



if you have virtualbox then you should have no problem learning this stuff.

this is the first guide i used for ZFS back in the day, but honestly, you don't even have to go this far..just do this:

install minimal freebsd to a hard drive/partiton

make a zpool with a harddrive/group of hard drive, lets call it tank
create some filesystems, let's say tank/var tank/usr tank/usr/ports tank/usr/local and tank/usr/home

use cpio or tar to copy the data out of the actual locations into the new zfs filesystems but leave /etc and / alone

reset the zfs mountpoints and you'll have a system with ZFS which should boot easily....once you can do this much, the rest should be simple.


----------



## BeautifulFish (Dec 2, 2009)

thanks for the links, i have read some of the info in them earlier, they are huge though :e

i successfully followed the guide in my first post, it is very straightforward and i installed FreeBSD which boots from zfs

but the point is... i have 1 pool, it includes all my filesystems, i keep my data in /usr/home
i know i will break the FreeBSD installation at some point, so what will i do then? i can't destroy the pool as:
1- the system boots from it, i couldn't destroy it, maybe i can destroy it by booting from the dvd and importing it then destroy it?
2- i will lose /usr/home, that is why i created a second pool for it

from my edited previous post:


> so zpools are actually directories under "/"?
> then we activate zfs in kernel
> then tell the system to boot from that directory and convince it that "/tank" is "/"
> 
> i hope i am making any sense


----------



## graudeejs (Dec 2, 2009)

BeautifulFish said:
			
		

> i have no installation as i couldn't follow the guide for installing FreeBSD without sysinstall, i mentioned why i couldn't follow it



dude, use logic and improvise.... it's zfs you don't need fdisk...
What was important in that guide is how to extract archives containing base freebsd


----------



## graudeejs (Dec 2, 2009)

I guess I will write guide for this... when I have more time


----------



## BeautifulFish (Dec 2, 2009)

ok :e

i think it is solved now, i think i need 2 zpools, 2 partitions ( other than root partition where FreeBSD is installed on first and a swap partition )
*so i have*
ad0s1a root "to install minimal FreeBSD at first"
ad0s1b swap
ad0s1d zfs "where minimal FreeBSD is copied, then complete the installation"
ad0s1e zfs "for /usr/home"

*assuming i want to reinstall FreeBSD:*
i booted from the dvd
choose fixit from sysinstall menu
import the 1st zpool "which is first partition", destroy it
then back to sysinstall, custom installation, install FreeBSD on the 1st partition
then continue as the guide in my first post and have FreeBSD installed in a new zpool
then import my 2nd zpool "which is 2nd partition" where my precious /usr/home is!

i hope the idea of 2 zpools makes sense, it is the only way i can think of to protect /usr/home


----------



## graudeejs (Dec 2, 2009)

if you want to reinstall freebsd, you can create new zfs partition... install freebsd on it, and modify /boot/loader.conf and tell it to boot from coolpool/newroot for example 

How do you think I reinstall my system?

http://forums.freebsd.org/showthread.php?t=6230
again put layer of abstraction here, and think how you can apply this to your system


----------



## BeautifulFish (Dec 2, 2009)

oh but i don't have an additional space for a new partition 
but then how can you get your data from the old pool to the new one?

please see my previous edited post


----------



## graudeejs (Dec 2, 2009)

You create 1 pool, on few disks (or one... ) depends on how many you have... copy files to that pool, and later add disks with ufs to pool.

I haven't added any disk to pool yet, but I bet you can find lots of info on internet


----------



## phoenix (Dec 2, 2009)

Think of "pool" like 1 big giant harddrive.

In the old days, you slice up the harddrive and create partitions for each filesystem.

In ZFS, you just create the filessystem.  That is your "partition".

If you need to re-install, you just delete the filesystem(s) you no longer want, and leave the ones you want to keep.  Just like in the old days, when you leave partitions around.

The easiest way to keep things in sync in your head is to stop thinking about partitions and slices and just think about filesystems.  Then everything is the same between non-ZFS and ZFS systems.

ie:  1 filesystem for /, 1 filesystem for /home, 1 filesystem for /var

How you create that filesystem depends on the storage system used (disk+slice+partition, or pool), but it's still all about filesystems.


----------



## phoenix (Dec 2, 2009)

This is also why I still like to use UFS for the OS (/, /usr), and put everything else (/usr/ports, /usr/src, /usr/obj, /usr/local, /home, /var, /tmp) onto ZFS.  That way, you can re-install the OS as many times as you need to, without ever touching the ZFS pool.  

Use a separate HD for the OS, or put it on a USB stick, or CompactFlash, etc.  Then you can dedicate entire drives to ZFS instead of just partitions.  And, if things ever go bad, you can still boot to single-user mode and have access to all your normal OS tools.  (Someday, when sysinstall comes with support for ZFS, I may look into using a separate mirrored pool for the OS.)


----------



## graudeejs (Dec 2, 2009)

phoenix, I now you're probably nr1 here in terms of zfs, but I wonder have you read:
http://forums.freebsd.org/showthread.php?t=6230
?


----------



## BeautifulFish (Dec 3, 2009)

thanks very much phoenix, i think many things are starting to make sense to me now 

so a separate pool for the os isn't a bad idea? :e

i followed this guide and finally has a working ZFS FreeBSD installation :e
http://wiki.freebsd.org/RootOnZFS/GPTZFSBoot

i kept taking snapshots of the virtual machine so that kernel panics don't happen 

will i still need freebsd-boot partition for booting after sysinstall supports ZFS?


----------



## graudeejs (Dec 3, 2009)

Yes


----------



## Blueprint (Dec 3, 2009)

phoenix said:
			
		

> This is also why I still like to use UFS for the OS (/, /usr), and put everything else (/usr/ports, /usr/src, /usr/obj, /usr/local, /home, /var, /tmp) onto ZFS.  That way, you can re-install the OS as many times as you need to, without ever touching the ZFS pool.
> 
> Use a separate HD for the OS, or put it on a USB stick, or CompactFlash, etc.  Then you can dedicate entire drives to ZFS instead of just partitions.  And, if things ever go bad, you can still boot to single-user mode and have access to all your normal OS tools.  (Someday, when sysinstall comes with support for ZFS, I may look into using a separate mirrored pool for the OS.)



Hi Phoenix, im new to freebsd and this community. I want to thank you for your good zfs tips in other threads. It has helped me alot with my setup.

Coming from an AIX background we always run at least 2 volume groups so thats what I did with my freebsd setup. Theres so much benifit to being able to export your data pool/vg while keeping the working system in tact. 

I have a mirrored zroot pool and a raidz zstorage pool. I dont see the benifits of still running some of the system filesystems on ufs. If i want to reinstall my OS, ill just export my zstorage pool, reboot into fixit and import the zroot pool and change the root dataset mountpoint (which will cascade down to the rest). Then create new datasets and proceed to do a normal install as per the wiki. 

I think if our concern is pool corruption it will be more from some hardware malfunction then doing a new OS install.


----------



## phoenix (Dec 3, 2009)

killasmurf86 said:
			
		

> phoenix, I now you're probably nr1 here in terms of zfs, but I wonder have you read:
> http://forums.freebsd.org/showthread.php?t=6230
> ?



Yeah, I've glanced over it a few times.  Same with your howto for ZFS on GELI.  Haven't read either of them in detail yet.


----------



## phoenix (Dec 3, 2009)

Blueprint said:
			
		

> Hi Phoenix, im new to freebsd and this community. I want to thank you for your good zfs tips in other threads. It has helped me alot with my setup.
> 
> Coming from an AIX background we always run at least 2 volume groups so thats what I did with my freebsd setup. Theres so much benifit to being able to export your data pool/vg while keeping the working system in tact.
> 
> ...



The reason I keep a UFS partition around is that I started with ZFS in the 7.0-STABLE days.    Back before you could boot into ZFS, and before the "/boot on UFS" hacks appeared.  I initially started with just / on UFS (/bin, /lib, /sbin, etc) with everything else in ZFS (/usr, /var, /tmp, /home, etc).  But, we ran into a few bumps that required booting to single-user mode to fix ... and it's very hard to do that when half the OS is inaccessible in ZFS.  

I'll consider moving over to all-ZFS once there are fixit CDs and LiveCDs that support ZFS repairs.    And once I have more experience with fixing broken ZFS pools and filesystems.

However, once I do move to an all-ZFS system, I will keep two pools:  one for the OS (/ and /usr) and one for everything else (/usr/local, /home, /var, /tmp, and so on).  The former will be a mirrored pool (2 disks), the latter will be some form of raidz


----------



## graudeejs (Dec 3, 2009)

fixit disk and dvd does support zfs.... you just need to

```
# mkdir /boot/zfs
# ln -s /dist/boot/kernel /boot/kernel
# ln -s /dist/lib /lib
# kldload zfs
```

I wrote this from my memory, so there might be minor bugs  but you get the idea.


----------



## jb_fvwm2 (Dec 3, 2009)

In a fixit CD yesterday, I had to preface 
paths with /dev (mount ... /dev). Just in
case someone is writing down the post above
and it does not work...
(was fstab stuff though, not zfs )


----------



## BeautifulFish (Dec 3, 2009)

jb_fvwm2 said:
			
		

> In a fixit CD yesterday, I had to preface
> paths with /dev (mount ... /dev). Just in
> case someone is writing down the post above
> and it does not work...
> (was fstab stuff though, not zfs )



oh it does work as killasmurf86 wrote it..

btw should i wait for 8.0-STABLE before formatting because of this:
http://groups.google.com/group/mail...ad/5b6ff6729353f6a9/fef81f4daae88fdc?lnk=raot


----------



## graudeejs (Dec 3, 2009)

BeautifulFish said:
			
		

> oh it does work as killasmurf86 wrote it..
> 
> btw should i wait for 8.0-STABLE before formatting because of this:
> http://groups.google.com/group/mail...ad/5b6ff6729353f6a9/fef81f4daae88fdc?lnk=raot



Are you going to import zfs from OpenSolaris? I don't think so....

EDIT:
[I admin I only read title]


----------



## BeautifulFish (Dec 3, 2009)

i am not going to, but please read the last post in the link..


----------



## graudeejs (Dec 3, 2009)

Uhhh, interesting... does this mean, that people who use gpt, will have to repartition their drives?

Well, I fall out of this category... since I started using zfs I don't use gpt on my HDD's


----------



## BeautifulFish (Dec 5, 2009)

killasmurf86 said:
			
		

> Uhhh, interesting... does this mean, that people who use gpt, will have to repartition their drives?



i am really looking forward to an answer to your question since i am going to use gpt


----------

