# Trimming NVMe disks under FreeBSD 13 ZFS?



## kalleboy (Oct 25, 2021)

I've read some posts telling "we're in 2021 and we don't need manual trimming of flash devices anymore", just confused and wanted to ask here, the best place to ask&learn. 

How often would you suggest doing "zpool trim zroot" under a web/sql server with tons of little files (images and so on) and lots of read/write?
I'm planning to manually trim my ZFS pool (I heard setting it auto has a bad impact on performance?), add the command "zpool trim zroot" to the crontab, just trying to find a best interval.

I got 2x512GB Samsung NVMe drives in single stripe;


```
root@mybox:~ # zpool list
NAME    SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
zroot   944G   245G   699G        -         -     1%    25%  1.00x    ONLINE  -
```



```
root@mybox:~ # gpart show
=>        40  1000215136  nvd0  GPT  (477G)
          40        1024     1  freebsd-boot  (512K)
        1064         984        - free -  (492K)
        2048     4194304     2  freebsd-swap  (2.0G)
     4196352   996018176     3  freebsd-zfs  (475G)
  1000214528         648        - free -  (324K)


=>        40  1000215136  nvd1  GPT  (477G)
          40        1024     1  freebsd-boot  (512K)
        1064         984        - free -  (492K)
        2048     4194304     2  freebsd-swap  (2.0G)
     4196352   996018176     3  freebsd-zfs  (475G)
  1000214528         648        - free -  (324K)
```


```
# zpool get all | grep trim
zroot  autotrim                       off                            default
```

Manually executed "*zpool trim zroot*" first time. The result;


```
root@mybox:~ # zpool status -t
  pool: zroot
 state: ONLINE
  scan: scrub repaired 0B in 00:01:00 with 0 errors on Sat Oct 23 03:02:23 2021
config:


        NAME        STATE     READ WRITE CKSUM
        zroot       ONLINE       0     0     0
          nvd0p3    ONLINE       0     0     0  (100% trimmed, completed at Mon Oct 25 18:13:55 2021)
          nvd1p3    ONLINE       0     0     0  (100% trimmed, completed at Mon Oct 25 18:13:56 2021)
```

So how often should I do this?
I frequently run scrub but never did trim. Any suggestion would be much appreciated.

Thanks!


----------



## kalleboy (Oct 29, 2021)

Anyone with any idea? Much welcome


----------



## gpw928 (Oct 29, 2021)

Allan Jude hosts the BSD Now podcast, and is active in ZFS development.  They take questions, and may give answers.


----------



## Sevendogsbsd (Oct 29, 2021)

I am just interested in the statement “web/sql” server. Hackers dream


----------



## grahamperrin@ (Oct 30, 2021)

kalleboy said:


> … (I heard setting it auto has a bad impact on performance?), …



From zpoolprops(7): 



> … automatic trimming of recently freed data blocks can put significant stress on the underlying storage devices. This will vary depending of how well the specific device handles these commands. …



Side note: FreeBSD bug 259537 – The online manual page for zpoolprops(7) should be found in the FreeBSD area


----------



## kalleboy (Nov 9, 2021)

Any idea to my main question?


----------



## Eric A. Borisch (Nov 10, 2021)

If you have a period of downtime / less important performance time (e.g. overnight) just set a nightly cron job to run zpool trim.

That said, unless you are benchmarking or really hitting the disks hard consistently, you likely won't notice a performance hit with autotrim=on.


----------



## mark_j (Nov 10, 2021)

Autotrim is a best approach to trim'ing ssd but it's not foolproof. I have not heard of or experienced any performance issues using autotrim as it's an opportunistic algorithm when there's low disk i/o. (I missed this line, I hate using tablets for this forum) _*The algorithm is designed, as I understand it, to collate small blocks then run the trim*_. This means that when it runs 'autotrim' it's looking for little blocks that make big blocks that can be trimmed as one big 32k (IIRC?) block.
Consequently, it can miss some. So then it's best to place a cron job to manually trim your disks. How often you do this depends on your workload.
You can gain some insight with zpool iostat of the pool(s).


----------



## kalleboy (Nov 30, 2021)

Thank you all, for your replies.

Is it ok to run trim/scrub on a VPS (obviously, using virtual disk images), or trim and scrub tools are effective only on real harddisks (ssd flash/sata)? I mean, would it help on virtual disks as well?


----------



## VladiBG (Nov 30, 2021)

Virtual machines which runs on top of SSD also benefit of TRIM as long end-to-end TRIM is supported by the virtualization.





						[base] Revision 365702
					






					svnweb.freebsd.org
				





__ https://twitter.com/i/web/status/1283849621786316801_View: https://twitter.com/klarainc/status/1283849621786316801_


----------



## grahamperrin@ (Dec 1, 2021)

kalleboy said:


> … ok to run trim/scrub on … virtual disk images), …



Essentially: *zpool-scrub(8)* examines _data_. It's as effective with pools in a virtual machine as it is with pools on physical devices.

<https://openzfs.github.io/openzfs-docs/man/8/zpool-scrub.8.html>


----------

