# Manual partitioning



## Tony62w (Oct 31, 2021)

Hi, I'm novice with FreeBSD but have some experience with Linux.

My Hardware:

Threadripper 1900
64 GB RAM
2 TB nvme 4k PBS/LBS (physical/logical block size) = 1863 GiB

My Partitions on twin nvme
`sgdisk -n 1:1M:+1G -t 1:ef00 -c 1:EFI-0002 $DISK

sgdisk -n 2:0:+1766G -t 2:8300 -c 2:ARTIX-0002 $DISK

sgdisk -n 3:0:+96G -t 3:8200 -c 3:SWAP-0002 $DISK`
Note: Full GiB are used 16.07 MiB at end not used

My wished partitioning:

Partition Type               Size                   Mountpoint                 Label

freebsd-boot                1GiB
freebsd-zfs                   1766GiB            /?                                BSDZ-001?
freebsd-swap               96GiB                      
*Questions:*

How are structured the disk-layers on ZFS? Disk-controller (PBS/LBS) structure at base, partitions boundaries on disk-structure, and formatting on top for at least boot and swap?
Can I use 'manual partitioning' even for ZFS just choosing `freebsd-zfs` or have I to make many other "adjustment" for using ZFS?
Can I use 1GiB boot partition or is there a size limitation?
Can I make the partitions boundaries like shown, or must swap be absolutely on second place?
Do you recommend an encrypted swap?
Is inside the "dvd1.iso" an installation script there, e.g. for "Auto (ZFS)" where I can modify at least the partitions-size?
Do ZFS span on all the partitions in this case or only on / (root)? And if swap is encrypted, too?
Do I need an L2ARC and/or ZIL for such amount of RAM and SWAP on NVMe?

Sorry for so much, and maybe silly questions, to you. Thanks anyway if you can help me.


----------



## drhowarddrfine (Oct 31, 2021)

While you're waiting, some of those questions have already been answered on this forum and you can search for them.


----------



## T-Daemon (Nov 1, 2021)

Tony62w said:


> My wished partitioning:
> 
> freebsd-boot 1GiB


1 GiB for a FreeBSD boot partition is exaggerated. These are the sizes in a GPT BIOS+UEFI partition scheme for 'efi' and 'freebsd-boot' partitions created by bsdinstall(8) guided installations:

```
# gpart show -p
=>      40  30001072    ada0  GPT  (14G)
        40    532480  ada0p1  efi  (260M)
    532520      1024  ada0p2  freebsd-boot  (512K)
    ...
```



Tony62w said:


> freebsd-swap 96GiB


See thread New freebsd-swap space with 400GB but htop shows 66.0G



Tony62w said:


> 2. Can I use 'manual partitioning' even for ZFS just choosing `freebsd-zfs` or have I to make many other "adjustment" for using ZFS?


Choosing 'freebsd-zfs' is sufficient.



Tony62w said:


> 3. Can I use 1GiB boot partition or is there a size limitation?


No size limitation, but see above response.



Tony62w said:


> 6. Is inside the "dvd1.iso" an installation script there, e.g. for "Auto (ZFS)" where I can modify at least the partitions-size?


That installation script on the installation image is /usr/libexec/bsdinstall/zfsboot. It creates the 'ZFS Configuration' menu and is also used for scripted installations (see bsdinstall(8), keyword /etc/installerconfig). But it doesn't allow to set a 'freebsd-zfs' partition size, except on 14.0-CURRENT, and only for scripted Root-on-ZFS.

To set the 'freebsd-zfs' partition size you need to open a shell at the "Partitioning" menu and set the size manually, or use a modified zfsboot script [1] [2], to get a menu with the option to set the zroot pool size.


----------



## SirDice (Nov 1, 2021)

T-Daemon said:


> 1 GiB for a FreeBSD boot partition is exaggerated.


Not only that, freebsd-boot can't be larger than 545 KB or it will fail to load. The default size of 512 KB is more than enough. 

Contrary to Linux freebsd-boot does NOT contain /boot, that directory is simply part of the root filesystem. The freebsd-boot partition contains code from gptzfsboot(8) or gptboot(8), and depends on if you boot to ZFS or UFS. It is not a filesystem, that partition is read as-is into memory and executed. 



T-Daemon said:


> No size limitation, but see above response.


freebsd-boot does have a size limitation. 


```
The freebsd-boot partition should be smaller than 545 KB.
```
See gpart(8).


----------



## eternal_noob (Nov 1, 2021)

SirDice said:


> should


RFC2119 says that "SHOULD" means


> This word, or the adjective "RECOMMENDED", mean that there may exist valid reasons in particular circumstances to ignore a particular item, but the full implications must be understood and carefully weighed before choosing a different course.


As opposed to "MUST" which means


> This word, or the terms "REQUIRED" or "SHALL", mean that the definition is an absolute requirement of the specification.


I conclude that the 545 KB limit is not mandatory.


----------



## SirDice (Nov 1, 2021)

eternal_noob said:


> RFC2119 says that "SHOULD" means


The word 'SHOULD' would have been capitalized. 


eternal_noob said:


> I conclude that the 545 KB limit is not mandatory.


I suggest you test your theory and find out.


----------



## eternal_noob (Nov 1, 2021)

I consider this a bug in the documentation if "should" doesn't mean "SHOULD".


----------



## Emrion (Nov 1, 2021)

eternal_noob said:


> RFC2119 says that "SHOULD" means
> 
> As opposed to "MUST" which means
> 
> I conclude that the 545 KB limit is not mandatory.


The code inside pmbr doesn't agree with that. It's really mandatory because freebsd-boot partition is loaded in x86 real-mode and you have less than 1 MiB of memory...


```
next_boot:    incl (%si)             # Next LBA
              adcl $0,4(%si)
              mov %es,%ax            # Adjust segment for next
              addw $SECSIZE/16,%ax   #  sector
              cmp $0x9000,%ax        # Don't load past 0x90000,
              jae err_big            #  545k should be enough for
              mov %ax,%es            #  any boot code. :)
              jmp load_boot
```


----------



## SirDice (Nov 1, 2021)

eternal_noob said:


> I consider this a bug in the documentation if "should" doesn't mean "SHOULD".


Manual pages are not RFCs.


----------



## zirias@ (Nov 1, 2021)

The RFCs just insist on exact english semantics here. So yes, "should" isn't the best word if not following that advice will just break. But instead of discussing this here, how about opening a PR for changing the wording of that manpage?


----------



## eternal_noob (Nov 1, 2021)

Zirias said:


> opening a PR for changing the wording of that manpage?


In an ideal world I would open a PR but I know it'll get ignored. So it's not worth the hassle.
I just took a note that man pages are not reliable when it comes to wording.


----------



## Tony62w (Nov 1, 2021)

T-Daemon said:


> Code:
> 
> ```
> # gpart show -p
> ...


My GPT partitions (under Linux) start at sector 256 (by 4k PBS/LBS) not at sector 40, is there a reason for that? Or a special need from BSD?
40*4k=160k is enough space for GPT-information? Or is sector 40 "only for MBR? Or 512k/4k=128, 255-128 =127 hence start on 120~127 sector?
If I used a disk for FreeBSD-OS and later-on want to use it for ZFS-Data-Storage, have I at least to erase the first MiB on the disk with `dd` and afterward make a new GPT-table?


T-Daemon said:


> See thread New freebsd-swap space with 400GB but htop shows 66.0G


Some application like Firefox take so much RAM making the whole machine slow or freezing if swap is not oversized, this phenomena stop (by me) as I set 1.5*RAM as swap, to be tested or replace web-browser in freebsd.


SirDice said:


> Not only that, freebsd-boot can't be larger than 545 KB or it will fail to load. The default size of 512 KB is more than enough.
> 
> Contrary to Linux freebsd-boot does NOT contain /boot, that directory is simply part of the root filesystem. The freebsd-boot partition contains code from gptzfsboot(8) or gptboot(8), and depends on if you boot to ZFS or UFS. It is not a filesystem, that partition is read as-is into memory and executed.



I read some instructions assigning also 800k to freebsd-boot but these instructions confirm yours.
Does anyone know where zbm (zfs-boot-menu) (in case) will take place? or is this not supported by free-bsd?


SirDice said:


> See gpart(8).


Is this code making FAT32 or VFAT EFI?

```
gpart add -a 4k -s 800K -t efi ada0
 # prior to FreeBSD 12.x
 gpart bootcode -p /boot/boot1.efifat -i 1 ada0
 # for FreeBSD 12.x and up, create a FAT32 partition
 newfs_msdos -F 32 -c 1 /dev/ada0p1
 mount -t msdosfs -o longnames /dev/ada0p1 /mnt
 mkdir -p /mnt/EFI/BOOT
 cp /boot/loader.efi /mnt/EFI/BOOT/BOOTX64.efi
```
I ask this because, if I want format efi-partition, I have to use vfat instead, see:

```
mkfs.vfat -F32 -s 2 -S 4096 -v /dev/nvme0n1p1
```
fat32 don't work at all, this code (at least under Linux not):

```
mkfs.fat -F 32 -n EFI /dev/nvme0n1p1
```

How do FreeBSD installer handle swap?
if freebsd-udf is chosen, then make a freebsd-swap partition and...
if freebsd-zfs is chosen, then make a "swap" volume or  subvolume on the (remaining) disk?
Is necessary or recommended encrypting swap choosing ZFS?

How do FreeBSD installer handle encryption?
I read full-disk-encryption with GELI, "only" `zroot` encryption with OpenZFS (apparently not at all recommended, in Ubuntu 21.10 also not working, I read in forum).
It's possible to choose (using the installer) the type/kind of encryption or succeed automatically, depending on FS (udf, zfs) and/or swap (encrypted or not)?


T-Daemon said:


> But it doesn't allow to set a 'freebsd-zfs' partition size, except on 14.0-CURRENT, and only for scripted Root-on-ZFS.


Partition size or empty space at end of disk (if you want to replace a raid-disk) is very important and not only for that.
Well, if 14.0-CURRENT allow it, allow it in the installer like with swap or script modifying is still necessary?

Thank you guy for all the given answers, that's good for my knowledge and... please don't misunderstand me, I just would understand well how FreeBSD works.


----------



## astyle (Nov 1, 2021)

Tony62w said:


> *Questions:*
> 
> How are structured the disk-layers on ZFS? Disk-controller (PBS/LBS) structure at base, partitions boundaries on disk-structure, and formatting on top for at least boot and swap?
> Can I use 'manual partitioning' even for ZFS just choosing `freebsd-zfs` or have I to make many other "adjustment" for using ZFS?
> ...



For starters, ZFS is a *very different beast*. You can configure the datasets to be roughly equivalent to "partitions" of UFS - and even do it post-install, and adjust it later - but there are reasons to leave it alone. A big reason is _Boot Environments_ - that stuff works best with the defaults offered at installation.
Not impossible, but it's more sensible to stick with defaults if you choose ZFS at install.
Per SirDice post earlier, you only need 545 KB or less, and yes, that is a limitation. Fortunately, ZFS install will take care of that for you.
Swap? no such animal in ZFS. Take a look here.
See answer #4.
See answer #1.
Not impossible, but mixing of different filesystems is not recommended on a single disk. Once again, no swap needed.
No idea, but your system's specs suggest that you don't need any special messing around with compression to successfully run ZFS. Once again, no swap even needed.


----------



## SirDice (Nov 1, 2021)

Tony62w said:


> Is this code making FAT32 or VFAT EFI?


FAT32. See newfs_msdos(8):

```
-F FAT-type
             FAT type (one of 12, 16, or 32).
```


----------



## zirias@ (Nov 1, 2021)

Actually, VFAT and FAT32 are orthogonal. Any FAT filesystem can be used as "VFAT" (hiding "long" filenames in extra directory entries that have attributes considered invalid by original FAT and therefore are just invisble when VFAT ist not supported), it's just a question of how it is mounted. It's totally irrelevant when creating a FAT filesystem.


----------



## Tony62w (Nov 2, 2021)

Well, I should use the standard bsd-installer and I will do it.

Now, the Linux standard installers (Ubiquity, Calamares) don't recognize my native 4k PBS & LBS (4096 physical and logical block size) of my 2TB NVMe at all, so I'm forced to make partitioning and formatting manually and advice the installer to use part 1 for EFI,  part 2 for ROOT and part 3 for SWAP.
See here (1st code-block partitioning, 2nd code-block formatting):

```
sgdisk -n 1:1M:+1G -t 1:ef00 -c 1:EFI-0002 $DISK

sgdisk -n 2:0:+1766G -t 2:8300 -c 2:ARTIX-0002 $DISK

sgdisk -n 3:0:+96G -t 3:8200 -c 3:SWAP-0002 $DISK

mkfs.vfat -F32 -s 2 -S 4096 -v $DISK-part1

mkfs.ext4 -F -b 4096 -F $DISK-part2

mkswap -f -p 4096 $DISK-part3
```
I can understand if you are not familiar with Linux, hence I put my question as simple as possible:
Will the bsd-installer recognize the native 4k sector of my NVMe 100%? or have I to make tricks?
Note: Native 4k (no 512 and no 512e (fake 4k)) on NVMe.



SirDice said:


> FAT32. See newfs_msdos(8):
> 
> ```
> -F FAT-type
> ...


In this precise case: Will newfs_msdos of bsd-installer recognize native 4k and use automatically the -S switch/option or have I to force it anyway?

```
-S    sector-size
         Number of bytes per sector.  Acceptable values are    powers of 2 in
         the range 512 through 32768, inclusive.
```



astyle said:


> Swap? no such animal in ZFS. Take a look here.


Bad news (in any sense), Swap is there, the size can be increased and the option to encrypt it is present too.
Maybe not under `zroot`, maybe in an additional partition, maybe as swap-file, BUT... if the bsd-installer (§ 2.6.4 Guided Partitioning Using Root-on-ZFS) say, is there, then should it be somewhere and reported in fstab.
I don't ask "Where is Bill" (like in Tarantino film) but "Where is Swap" and which kind of swap is, e.g. frebsd-swap or something else.


astyle said:


> Not impossible, but mixing of different filesystems is not recommended on a single disk. Once again, no swap needed.


I don't want mix anything, nor convert Freebsd to act as a Linux, I just want to install a rock-solid FreeBSD and would understand some particularity. Swap is there by default and is needed too.


astyle said:


> No idea, but your system's specs suggest that you don't need any special messing around with compression to successfully run ZFS. Once again, no swap even needed.


L2ARC and/or ZIL are not compression but "data-buffers", lz4 compression is recommended even in BSD-forum but not present as option in bsd-installer. I don't understand this, but I don't care too because of 2TB NVMe. I don't intent to use FreeBSD as "Server" but as OS without 'SystemD' but with perfect integrated ZFS.

The need or not of swap will I not comment again or anymore. It's true, I don't know FreeBSD but I know e.g. Firefox, KDE5, etc., so... I don't know if BSD modified those packages and really swap is not needed, but I have plenty of place (2TB), bsd-installer offer it and I will definitely have on BSD.


----------



## astyle (Nov 2, 2021)

Tony62w said:


> Bad news (in any sense), Swap is there, the size can be increased and the option to encrypt it is present too.
> Maybe not under `zroot`, maybe in an additional partition, maybe as swap-file, BUT... if the bsd-installer (§ 2.6.4 Guided Partitioning Using Root-on-ZFS) say, is there, then should it be somewhere and reported in fstab.
> I don't ask "Where is Bill" (like in Tarantino film) but "Where is Swap" and which kind of swap is, e.g. frebsd-swap or something else.


Yeah, the FreeBSD installer screenie that you link to does mention swap even when selecting the ZFS install. However no, it's *not reported in* /etc/fstab. You can safely ignore that option, leave it on whatever default value is supplied, and continue the installation. If you select UFS (as opposed to ZFS) at install - then yeah, you gotta worry about swap. But ZFS is a totally different beast, it frees you from that. I haven't tried Linux on ZFS (no time for that for me), but I keep seeing info that this is where FreeBSD is WAY ahead of Linux 

FWIW, my rig has a Ryzen 5 1400, 32 GB of RAM, and a 256 GB SSD. I'm on 13-RELEASE, but was messing around with ZFS-based FreeBSD installs since 2017, when my processor was just released. And - no issues with ZFS, no messing with options to fine-tune my install, just plain a bit of exploring to squeeze interesting juice out of that.


----------



## zirias@ (Nov 2, 2021)

astyle said:


> If you select UFS (as opposed to ZFS) at install - then yeah, you gotta worry about swap. But ZFS is a totally different beast, it frees you from that.


Are you suggesting to use ZFS volumes for swap? 

If so, don't. Always use classic swap partitions. The reason is, ZFS itself can need dynamically allocated memory. If that allocation triggers a page fault and the pager must "swap out" pages to solve it, it would need ZFS to do so, creating a nice deadlock.


----------



## T-Daemon (Nov 2, 2021)

Tony62w said:


> Partition size or empty space at end of disk (if you want to replace a raid-disk) is very important and not only for that.
> *Well, if 14.0-CURRENT allow it, allow it in the installer like with swap or script modifying is still necessary?*


The zfsboot script from 14.0-CURRENT doesn't provide a menu option in the "ZFS Configuration" dialog. It provides a environment variable which can be used to determine the pool size in a scripted installation only. As example how to use, taking from the bsdinstall(8) manual, chapter "SCRIPTING":

/etc/installerconfig (file not a part of the official installer media, needs to be created):

```
DISTRIBUTIONS="kernel.txz base.txz"
export ZFSBOOT_VDEV_TYPE=stripe
export ZFSBOOT_DISKS="ada0 ada1"

export ZFSBOOT_POOL_SIZE=1766g

export nonInteractive="YES"
```
If you want a menu option in the "ZFS Configuration" dialog you can replace the original script with the modified zfsboot I linked to at the end of post #3


----------



## astyle (Nov 2, 2021)

Zirias : Did you see OP's HW specs? several steps up from mine. I never had to worry about page faults, so I see no reason why OP should, either.


----------



## zirias@ (Nov 2, 2021)

If you don't, why add swap at all? Without swap, if you ever hit such a situation, OOM-killer will take care of it (at the expense of a killed process). IMHO still better than running into a deadlock.


----------



## Vull (Nov 2, 2021)

I don't know about version 14.0-CURRENT, but, using FreeBSD-13.0-RELEASE-amd64-memstick.img with the automatic ZFS install option gave me a freebsd-swap partition on partition 3, and this swap partition is specifed in the /etc/fstab file in exactly the same way as in a UFS install.

The freebsd-boot and freebsd-zfs partitions are not in /etc/fstab, but the efi and the freebsd-swap partitions are.


----------



## Tony62w (Nov 4, 2021)

Thank you all for your contributions.


SirDice said:


> FAT32. See newfs_msdos(8):


FAT32 will not work at all with native 4k PBS/LBS, at least don't work in Linux. Before I start such thing, I would be sure FreeBSD is 100% able to handle native 4k properly. Anyone knows it?



T-Daemon said:


> As example how to use, taking from the bsdinstall(8) manual, chapter "SCRIPTING":





T-Daemon said:


> The zfsboot script from 14.0-CURRENT doesn't provide a menu option in the "ZFS Configuration" dialog. It provides a environment variable which can be used to determine the pool size in a scripted installation only. As example how to use, taking from the bsdinstall(8) manual, chapter "SCRIPTING":



From § *scriptedpart* _parameters/size__ 
[/I]
specifies the partition size to create in bytes
               (K, M, and G suffixes can be appended to    specify    kilo-
               bytes, megabytes, and gigabytes respectively)
[I]_
To assure 4k boundary the size must be given in KiB, MiB and GiB (Kibibyte, Mebibyte, Gibibyte) and not KB, MB, GB that not assure anything but boundary fragmentation.



Vull said:


> The freebsd-boot and freebsd-zfs partitions are not in /etc/fstab, but the efi and the freebsd-swap partitions are.


Thanks for your confirmation. Is the FEI-partition the one starts at sector 40 on first 1 MiB of disk? Or it's something different?


I'm a little bit surprised, frustrated and dismayed in discovering the need of such long (and sometimes controversial) discussion just talking/asking about partitioning only.

I hope, some developers and wiki-maintainers read this thread and can provide some clarification and/or modify the options of bsdinstall/pre.

For me is disk-handling in an OS the most important thing, I also expect a "GUI"-installer detect/recognize if disk-block-size is switchable and ask for or execute the switch e.g. from 512 to 4096 automatically.

Please don't hesitate to contact me as soon you have some (for me) important information.

Thank you all again for your effort, bye!


----------



## Vull (Nov 4, 2021)

Tony62w said:


> Is the FEI-partition the one starts at sector 40 on first 1 MiB of disk? Or it's something different?


It's not necessary to start at sector 40.
The designation is not FEI, it's EFI, or UEFI. Yes, it's something different. You can read all about it here:









						EFI system partition - Wikipedia
					






					en.wikipedia.org


----------



## Tony62w (Nov 5, 2021)

Vull said:


> It's not necessary to start at sector 40.
> The designation is not FEI, it's EFI, or UEFI. Yes, it's something different. You can read all about it here:


Sorry, that was a writing mistake (FEI), naturally I mean EFI/UEFI.

And no, your link don't clarify anything because there are only dare description about EFI on Linux, Windows and Mac-OS but not in BSD.
Beside, this don't clarify how the "GUI-installer" handle the EFI-partition, where put it exactly and if recognize the native 4k-sectors properly.

Wikipedia is not a reliable source in this case but the experts of FreeBSD and hence I ask here and not wikipedia to include an article for EFI on FreeBSD.

Pointing me (or anybody else) to wikipedia article is a very bad idea, I don't like it at all, is not polite (to send client to pastry to ask how is the pork-tenderloin), sorry!
And I prove it using the ext4 file-system as an example:

If you have a disk with native (already 10 years old) 4k-sectors, the Ext4 file-system don't recognize it at all, but relay on what is listening from Kernel, if the kernel is well/right configured or loaded.
This mean ext4-fs is blind but hear/listen from kernel, here the prove:
`mkfs.ext4 -F -b 4096 -F $DISK-part2`
as you can see, the use of switch/option -F (that mean FORCE) is needed twice to oblige ext4 to recognize the 4k-sector on themselves and not only listening from kernel.

The same is valid for vfat and linux-swap,,, fat32 don't work at all but, these are other stories.
The conclusion about Linux-file-systems on 4k disks is:
- The partitioning ==> setting partitions-boundaries (partitions-delimitations) in GPT is OK and contains FAT32 (EFI, code: ef00), ext4 (`/` called also root, code: 8300) and linux-swap (code: 8200).
- The formatting is NOT OK ==> EFI: force `vfat` and 4k-bs (block-size), ROOT: force 4k-bs twice, SWAP: set ps (page-size= block-size) to 4096=4k

Now you can read the wikipedia article on ext4 and tell me (afterwards) if any of this topics are handled or at least mentioned with one word in the article.

And, how will you feel if you ask me about ext4 and I post a wikipedia-article instead?  It's not shaming to say "I don't know"! And, yes, I (myself) don't know anything about Freebsd and hence ask here.

Listen: Screens have controllers with EDID-information, disks have controllers telling how big is a data-carrier, the sectors-size and so on, GPUs have controllers, each piece of hardware have a controller sending information about itself, but... data-carriers/disks are the most important piece of HW where all the information and data of an OS are stored on them, that allow the OS to work at all / boot.

It's not like reload EDID in xorg or install GPU-driver after the installation. If disks are not handled well/properly, you cannot boot at all, and you have no possibility to remedy to it at all.

If you (or anybody else here) cannot tell me how to remove efi-boot-loader (safely) from first MiB of disk without damaging the disk same for using this same disk as file-storage or to use it with another OS (installing another OS on it), I will not risk wasting a 2TB NVMe costs 250-350$ only for experimenting with FreeBSD. I will not even waste a USB-stick for that, for it are VMs (virtual machines) that don't tell almost anything about a real (bare-metal) OS.

We/I still talking about just normal/standard FreeBSD with own "GUI-installer" hence "Auto-ZFS", we/I not talking now (anymore) about special partitioning or "rEFInd + zbm (zfs-boot-manager) + FreeBSD + Artix-Linux + Debian" like here.


----------



## astyle (Nov 5, 2021)

You can always get a cheap $25 SSD with a 256 GB capacity to swap in and play around.  Or are you not confident in your ability to do a simple hardware swap? 
And 2TB stuff is falling in price - it's $175 on Amazon if you bother to look. FreeBSD doesn't even need that much (2TB) room, it will be happy in a 64 GB SSD.

You can do your homework all you want, it's your hardware and your money.  But - 


Tony62w said:


> Pointing me (or anybody else) to wikipedia article is a very bad idea, I don't like it at all, is not polite (to send client to pastry to ask how is the pork-tenderloin), sorry!


Pointing people to wikipedia is the standard thing to do on these forums. We do it all the time, newbie, there's a tremendous upside to that that you're missing out on.


Tony62w said:


> If you (or anybody else here) cannot tell me how to remove efi-boot-loader (safely) from first MiB of disk without damaging the disk same for using this same disk as file-storage or to use it with another OS (installing another OS on it),


If you wanna clean up the 2TB disk for FreeBSD, you can always use your Linux skills to boot a rescue disk (there's even a specialty distro called GParted for that chore  ). Frankly, no need for that, FreeBSD 13-RELEASE makes very good use of the EFI bootloader.


----------



## Tony62w (Nov 6, 2021)

astyle said:


> You can always get a cheap $25 SSD with a 256 GB capacity to swap in and play around. Or are you not confident in your ability to do a simple hardware swap?
> And 2TB stuff is falling in price - it's $175 on Amazon if you bother to look. FreeBSD doesn't even need that much (2TB) room, it will be happy in a 64 GB SSD.


Have you, this forum or wikipedia a list of manufacturer and models of NVMe using native 4k-sectors or at least let you switch the disk (the controller on those disks) to 4k?
10 years after all manufacturer sing 4k specifications... is this not strange?
I know at least one manufacturer that have such disks and one that don't have such disks but, I will not make any advertisement like you.


astyle said:


> Pointing people to wikipedia is the standard thing to do on these forums. We do it all the time, newbie, there's a tremendous upside to that that you're missing out on.


This is an insult to anyone's intelligence that this forum is reading and, I will not comment on it further. Especially not according to all the evidence I have provided.


astyle said:


> If you wanna clean up the 2TB disk for FreeBSD, you can always use your Linux skills to boot a rescue disk (there's even a specialty distro called GParted for that chore  ). Frankly, no need for that, FreeBSD 13-RELEASE makes very good use of the EFI bootloader.


Parted, with GUI or not, don't understand 4k at all. All those, parted apps/program, state beginning of disk at 1024 or 2048 sector and not at 256 even by reading 4k disks.
Here and now you have proved to yourself the origin and quality of your knowledge. There is no need for further comment or proof on my part.


----------



## Alain De Vos (Nov 6, 2021)

The freebsd installer for zfs is not friendly. But nor are linux installers for zfs.
My five cent, Install FreeBSD on ufs, experiment with zpool&zfs, build knowledge. Google,reddit,whatever.
PS: don't be too fixated on 4K.


----------



## astyle (Nov 6, 2021)

Tony62w said:


> Have you, this forum or wikipedia a list of manufacturer and models of NVMe using native 4k-sectors or at least let you switch the disk (the controller on those disks) to 4k?


I meant pull the 2TB SSD out of the case, and put in a different one. Or are you the kind who's scared of a simple hardware swap like that? I bet you can't jack up your RAM, either, and would drive for 5 hours to a Best Buy just for that chore, and blow $200 on a 4GB stick installed by a Best Buy tech who will also fish porn out of your iphone.




Tony62w said:


> 10 years after all manufacturer sing 4k specifications...


Any partitioning software worth its salt will let you specify 4k boundaries in software. Never gave a toot about 4k boundaries being in hardware, and got better results than you. If you're so smart, you'll know how to do the homework yourself. On these forums, we help each other do exactly that. In my case, I would have never found klarasystems.com and its treasure trove of FreeBSD-related info if somebody on these forums didn't give me a link.


----------



## Tony62w (Nov 9, 2021)

Alain De Vos said:


> The freebsd installer for zfs is not friendly. But nor are linux installers for zfs.


I was hoping to find out the opposite but, you're right.


Alain De Vos said:


> My five cent, Install FreeBSD on ufs, experiment with zpool&zfs, build knowledge. Google,reddit,whatever.


I wanted to install the OS on native ZFS and not just experiment a bit with ZFS. For small experiments I can install FreNAS on USB-stick and there I have not to flickering around with the CLI.
The post #23 and wiki/manpage of bsdinstaller demonstrate the absent ability to make proper partitions-boundaries/-delimitation (KB, MB, GB instead of KiB, MiB, GiB).
This have impact on 512, 512e & 4096 and I'm stunned (as FreeBSD-newbie) to discover it.

Of course, I can make proper partitions-boundaries with `sgdisk` under Linux (see code-table and if it's possible at all) and modify or create BSD-installer-script but...
at this point, I can make a script, with which I am somewhat familiar, and install Linux on ZFS not starting from scratch with BSD or writing my own OS.

*sgdisk partitions-code for BSD-ZFS*

`a501 FreeBSD boot
a504 FreeBSD ZFS
a502 FreeBSD swap
# other BSD-partitions
a503 FreeBSD UFS`


Alain De Vos said:


> PS: don't be too fixated on 4K.


Do you propose to switch my 4k-native (that has existed for 10 years) NVMe back to 30-35 years old 512?
Note: The speed of all data-carrier is specified at 4k.

Please don't get me wrong, I have nothing against you or this forum (except wikipedia links), well/only I have had greater expectations from BSD.

I thought I could do a perfect partitioning (of boot, zfs and swap) and a formatting of the boot and swap and, from this point on, the bsd-installer could take over (as with the GUI installer of Linux).

I was definitely wrong and I'm sorry to have stressed you out.

I also understand that you don't make a (real) difference between partition delimitation and formatting, but that doesn't help any of us.

I have less understanding for the use of marketing gags ("sizes" or "dimensions") such as KB, MB and GB, but that doesn't help us either.

But... i think I can guess why (the upcoming) "TrueNAS Scale" is based on Debian/Devuan. It is (apparently) easier to implement a Linux in ZFS than to eliminate all the old camels (with every kernel release).

P.S.: I need an OS to work and enjoy without SystemD with ZFS, and not to experiment or repair an OS. Not to mention the reactions of programs or desktop environments such as Dolphin or KDE5. For this I was willing to make and accept a few small changes. But, as I said, I was wrong.

I would like to thank all those who tried to help me.


----------



## Alain De Vos (Nov 9, 2021)

Alpine linux is not bad. Or void linux. Give things a try.


----------



## astyle (Nov 9, 2021)

Tony62w said:


> I thought I could do a perfect partitioning (of boot, zfs and swap) and a formatting of the boot and swap and, from this point on, the bsd-installer could take over (as with the GUI installer of Linux).


That's not quite how ZFS really works. It's much easier to just let the installer use ZFS for everything. The (Boot, filesystem, swap area) idea is a Linux thing. Nothing wrong with it, but with with FreeBSD, the very design is different. There's nothing wrong with trying to swap Ext4 filesystem for ZFS, and a lot of Linux distros have put in the effort to include ZFS as an option. 

FWIW, you can do the math, and specify the size of ZFS datasets to be exactly a multiple of 4k. The nice thing about ZFS is that you can do that post-install, and adjust that at any time. I'm not aware of any other filesystem that can do THAT. ZFS is a very different beast than any other filesystem. If you get a good handle on ZFS, you can recover from mistakes as easy as collapsing a fishing rod.

FreeBSD has no SystemD, it does have ZFS (In fact it was there before any Linux distro even caught wind of ZFS), and it's possible to do the whole install on just plain ZFS, no swap needed.


----------



## _martin (Nov 12, 2021)

Emrion said:


> The code inside pmbr doesn't agree with that. It's really mandatory because freebsd-boot partition is loaded in x86 real-mode and you have less than 1 MiB of memory...
> 
> 
> ```
> ...



I remember I had 64k reserved for legacy boot, then 128k, and for 13.x it seems we need to have 128k+ (I've created 256k for a good measure).

But this code made me realize (yeah, I know this information is not something new) system won't legacy boot on native 4Kn disk (i.e. excluding 512e). I know I could use EFI or, if HW doesn't support it, other means of booting (usb,pxe..).
But still, I found it interesting, however irrelevant for this case.


----------



## grahamperrin@ (Nov 13, 2021)

Tony62w said:


> … some experience with Linux. …



I see you using sgdisk(8).

Probably overlapping with this topic: GPT fdisk (gdisk(8)) and gpart(8) | The FreeBSD Forums tl;dr I find things such as gdisk easier to use than gpart.



Tony62w said:


> Do you recommend an encrypted swap?



Yes.



Tony62w said:


> Do I need an L2ARC and/or ZIL for such amount of RAM and SWAP on NVMe?



Without knowing your use case: it's hard to tell, however I imagine that with so much fast primary storage, benefits from L2ARC (on a separate device?) might be negligible.

I _usually_ get *great* benefits from low-end L2ARC (two or three old USB flash drives) however my hardware is vastly inferior to yours.

OpenZFS: L2ARC: CACHE vdev devices : freebsd

My hit ratio at the time of writing is _unusually_ low:


```
% date ; uptime
Sat 13 Nov 2021 07:47:58 GMT
 7:47a.m.  up 2 days,  3:13, 7 users, load averages: 0.98, 0.98, 0.86
% zpool iostat -v
                         capacity     operations     bandwidth
pool                   alloc   free   read  write   read  write
---------------------  -----  -----  -----  -----  -----  -----
Transcend               290G   174G     23      4  3.02M   536K
  gpt/Transcend         290G   174G     23      4  3.02M   536K
cache                      -      -      -      -      -      -
  gpt/cache-transcend  14.3G   198M      4      2   308K   237K
---------------------  -----  -----  -----  -----  -----  -----
august                  290G   622G     29     17  1.11M   486K
  ada0p3.eli            290G   622G     29     17  1.11M   486K
cache                      -      -      -      -      -      -
  gpt/cache-august      805M  28.0G      3      0   149K  41.0K
  gpt/duracell         2.51G  12.9G     17      0   541K  62.1K
---------------------  -----  -----  -----  -----  -----  -----
% zfs-stats -L

------------------------------------------------------------------------
ZFS Subsystem Report                            Sat Nov 13 07:48:09 2021
------------------------------------------------------------------------

L2 ARC Summary: (DEGRADED)
        Low Memory Aborts:                      3.17    k
        Free on Write:                          68.35   k
        R/W Clashes:                            32
        Bad Checksums:                          83.81   k
        IO Errors:                              3

L2 ARC Size: (Adaptive)                         17.54   GiB
        Decompressed Data Size:                 34.55   GiB
        Compression Factor:                     1.97
        Header Size:                    0.10%   36.95   MiB

L2 ARC Evicts:
        Lock Retries:                           27
        Upon Reading:                           7

L2 ARC Breakdown:                               10.75   m
        Hit Ratio:                      37.57%  4.04    m
        Miss Ratio:                     62.43%  6.71    m
        Feeds:                                  150.30  k

L2 ARC Writes:
        Writes Sent:                    100.00% 31.43   k

------------------------------------------------------------------------

% zpool status -x
all pools are healthy
% zpool status
  pool: Transcend
 state: ONLINE
  scan: scrub repaired 0B in 04:01:49 with 0 errors on Thu Sep 16 07:05:21 2021
config:

        NAME                   STATE     READ WRITE CKSUM
        Transcend              ONLINE       0     0     0
          gpt/Transcend        ONLINE       0     0     0
        cache
          gpt/cache-transcend  ONLINE       0     0     0

errors: No known data errors

  pool: august
 state: ONLINE
  scan: scrub repaired 0B in 02:45:23 with 0 errors on Thu Sep 16 05:48:48 2021
config:

        NAME                STATE     READ WRITE CKSUM
        august              ONLINE       0     0     0
          ada0p3.eli        ONLINE       0     0     0
        cache
          gpt/cache-august  ONLINE       0     0     0
          gpt/duracell      ONLINE       0     0     0

errors: No known data errors
%
```

Sometimes: I have no idea why an amount used drops so low, for example (above) 805 M of 28.0 G; this particular cache device was brought online more than six hours ago.

Other times: I reckon, it's simply FreeBSD not working as well as I'd like with USB. It can be quite pernickety, compared to other OSes.


```
% lsblk
DEVICE         MAJ:MIN SIZE TYPE                                          LABEL MOUNT
ada0             0:124 932G GPT                                               - -
  ada0p1         0:126 260M efi                                    gpt/efiboot0 -
  <FREE>         -:-   1.0M -                                                 - -
  ada0p2         0:128  16G freebsd-swap                              gpt/swap0 SWAP
  ada0p2.eli     2:68   16G freebsd-swap                                      - SWAP
  ada0p3         0:130 915G freebsd-zfs                                gpt/zfs0 <ZFS>
  ada0p3.eli     0:136 915G -                                                 - -
  <FREE>         -:-   708K -                                                 - -
da0              2:37   14G GPT                                               - -
  <FREE>         -:-   1.0M -                                                 - -
  da0p1          2:74   14G freebsd-zfs                     gpt/cache-transcend <ZFS>
da1              2:97   15G GPT                                               - -
  <FREE>         -:-   1.0M -                                                 - -
  da1p1          2:98   15G freebsd-zfs                            gpt/duracell <ZFS>
da2              2:95   29G GPT                                               - -
  da2p1          2:144  29G freebsd-zfs                        gpt/cache-august <ZFS>
da3              2:183 466G GPT                                               - -
  <FREE>         -:-   1.0M -                                                 - -
  da3p1          2:186 466G freebsd-zfs                           gpt/Transcend <ZFS>
% geom disk list da1
Geom name: da1
Providers:
1. Name: da1
   Mediasize: 16567500800 (15G)
   Sectorsize: 512
   Mode: r1w1e3
   descr: Generic Flash Disk
   ident: YTAI0012080818233298
   rotationrate: unknown
   fwsectors: 63
   fwheads: 255

% geom disk list da2
Geom name: da2
Providers:
1. Name: da2
   Mediasize: 30943995904 (29G)
   Sectorsize: 512
   Mode: r1w1e3
   descr: Kingston DataTraveler 3.0
   ident: E0D55EA1C84FF390A9500FDA
   rotationrate: unknown
   fwsectors: 63
   fwheads: 255

%
```

At the office, this installation of FreeBSD seems to be entirely reliable with the Kingston device in side ports of an HP 2012 120W Advanced Docking Station. With an identical dock at home, I learnt to not use the Kingston device in those ports; instead I habitually use the left rearmost port of the EliteBook 8570p.


```
root@mowa219-gjp4-8570p-freebsd:~ # date ; uptime ; hw-probe -all -upload
Sat Nov 13 08:07:10 GMT 2021
 8:07AM  up 2 days,  3:32, 6 users, load averages: 0.31, 0.63, 0.72
Probe for hardware ... Ok
Reading logs ... Ok
Uploaded to DB, Thank you!

Probe URL: https://bsd-hardware.info/?probe=ea51e03be6
root@mowa219-gjp4-8570p-freebsd:~ #
```

▶ <https://bsd-hardware.info/?probe=ea51e03be6>



Tony62w said:


> … I set 1.5*RAM as swap …



Smart. I chose 16 GB swap, just rarely I wish that I had chosen more.


```
% sysctl hw.physmem hw.usermem vm.swap_total
hw.physmem: 17055141888
hw.usermem: 11022512128
vm.swap_total: 17179869184
%
```

deskutils/sysctlview may be of interest.

More generally, looking ahead:









						Technology Roadmap
					

https://freebsdfoundation.org/blog/technology-roadmap/  Enjoy.




					forums.freebsd.org
				












						FreeBSD 14.0 planning
					

… Do you have an idea when version 14.0 will be available?   14.0-RELEASE some time in 2023, I guess.  https://github.com/bsdjhb/devsummit/blob/main/14.0/planning.md  June 2021 FreeBSD Developer Summit: 14 0 Planning - YouTube




					forums.freebsd.org


----------

