# 10 Do's and Don't for FreeBSD



## ShelLuser (Apr 24, 2018)

*Editorial*

Hi gang!

FreeBSD is plain and simple my all time favorite Unix operating system. And yes: technically speaking it's Unix-_like_ but that's mostly due to licensing concerns. If you look at the backwards compatibility aspects (see /usr/src/sys/amd64/conf/GENERIC, the COMPAT_FREEBSDxx options), the slow pace of changes (we don't get "exciting" and "OS changing" features with every update) and the display of high respect for current standards (this is just me, but I think that pkg_add (old) vs. `pkg add` (current) is a good example and also extremely well thought off)... so if you look at all that then I think you can agree that when solely looking at the environment then FreeBSD has everything it needs to be considered a true Unix operating system.

Another thing is that the system is like elastic. You can bend and shape it in any way you want. In both good and bad ways. So I figured.. why not a small collection of very general things which you might want to pay attention to while working with this system?

*#1 Don't mix ports and binary packages*

This is an aspect which has confused _many_ people and I'm sure it will continue to do so for many more in the future. In short: either install your software through the ports collection (so building it using /usr/ports, either using the regular build commands or a utility of some sort) _or_ install your software using merely pkg. So, for example: `pkg install mystuff`.

Don't do both!

Why? Because it has the potential to seriously disrupt your entire system. If you upgrade your system and suddenly see packages which you wanted to keep get automatically deinstalled then this could be a clear sign of trouble. If you have upgraded your system and a program you're using suddenly spits out error messages about missing libraries or missing functions, library symbols or whatever more. That too could be a clear signal that you broke things. The reason this is happening is because binary packages were build with default settings and the expectation of the same thing for all their dependencies.

What do you think could happen if you installed a notepad system using the package manager while you installed a required database library using the ports collection, while _also_ exchanging MySQL for PostgreSQL? The application itself might still work. But all automated backups are likely to fail miserably because chances are high that it would depend on mysqldump for that, a program which is no longer available on your system. So the next time you experience a crash and figure that you came fully prepared then you might be in for a very nasty surprise.

*#2 Don't edit 'default' files*

Have you ever wondered which processes FreeBSD would start on its own? Or maybe what about the boot stage: would FreeBSD automatically load any kernel modules? These questions can be easily answered by taking a closer look at /etc/defaults/rc.conf and /boot/defaults/loader.conf.

Whatever you do: do not edit these files!

First, the most obvious reason: this is a file which is a part of the base system. Therefor it could get updated at any given time, fully overwriting whatever changes you made. If something suddenly stopped starting after you upgraded your system then... maybe this is why.

But the other reason is because there is no need. Any changes which you want to make should be applied to the main official file instead. So /etc/rc.conf and /boot/loader.conf respectfully. The default files are at their best as research material. It's not merely to tell FreeBSD what to do by default, it can even help us out!

Example: maybe you don't enjoy Sendmail and wish to use another MTA. I don't know, maybe mail/postfix? As you probably know Sendmail is a part of the base system, so it would also have set up several settings to cater for that. But which ones....

Well, easy:

```
peter@unicron:/etc/defaults $ grep sendmail rc.conf
mta_start_script="/etc/rc.sendmail"
# Settings for /etc/rc.sendmail and /etc/rc.d/sendmail:
sendmail_enable="NO"    # Run the sendmail inbound daemon (YES/NO).
sendmail_pidfile="/var/run/sendmail.pid"        # sendmail pid file
sendmail_procname="/usr/sbin/sendmail"          # sendmail process name
sendmail_flags="-L sm-mta -bd -q30m" # Flags to sendmail (as a server)
sendmail_cert_create="YES"      # Create a server certificate if none (YES/NO)
#sendmail_cert_cn="CN"          # CN of the generate certificate
sendmail_submit_enable="YES"    # Start a localhost-only MTA for mail submission
sendmail_submit_flags="-L sm-mta -bd -q30m -ODaemonPortOptions=Addr=localhost"
sendmail_outbound_enable="YES"  # Dequeue stuck mail (YES/NO).
sendmail_outbound_flags="-L sm-queue -q30m" # Flags to sendmail (outbound only)
sendmail_msp_queue_enable="YES" # Dequeue stuck clientmqueue mail (YES/NO).
sendmail_msp_queue_flags="-L sm-msp-queue -Ac -q30m"
                                # Flags for sendmail_msp_queue daemon.
sendmail_rebuild_aliases="NO"   # Run newaliases if necessary (YES/NO).
```
Everything you might wanted to know about FreeBSD's default Sendmail usage but were too afraid to ask  Of course an easier way to grep only the entries we need would be: `grep -e "^sendmail.*=\"YES" rc.conf`.

So basically: these default files should only be used to look up specific settings and/or behavior. And they can truly be an invaluable source for that. When I set up my own private SSL certificates for Sendmail I didn't want it to continuesly try to build its own useless stuff in /etc/mail/certs. And this is how I found out to stop that behavior.

*#3 Don't mess with /etc/crontab*

As you might know /etc/crontab is the main Cron configuration file so it would be the obvious place to set up our own entries for it, right? Well... not so much. Although not necessarily wrong there are some risks involved. Just like with most other files in /etc you're basically messing with a system file which could undergo changes during upgrades.

But the most compelling reason is that any changes you make would not be immediately picked up by the system. You'd have to reload the crontab as well. Why all the extra bother if you don't have to?

Instead it is advised to use the crontab command. Do you need some jobs run as the bind user? Just use `# crontab -u bind -e` and done. Do you need some specific system settings? Install those as root: `# crontab -e` and done. You only need to worry about the 5 time fields (m h dom m dow) and the command(s) to run and that's it.

After you're done editing the crontab will be automatically installed and activated. No need to manually mess with service reloading. And as an added bonus: this way you also don't have to worry about accidentally removing important system entries whenever you no longer need certain jobs and want to remove those.

*#4 Don't mess with /etc/passwd and /etc/groups either!*

Since we're focusing ourselves on /etc a little here is another one  I'll be perfectly honest: I sometimes also do this myself, but fortunately very seldomly.

If you insist on editing these files manually at the very least rely on `vipw` and/or vigr. There are many advantages for that. First it will apply a lock on the file so that you can't accidentally mess things up (for example: if another process would try to change a password and you finish making your changes after that you'd effectively undo the password change, which could lead to very nasty results). And another aspect is that any made changes will be automatically processed if they have to. Better yet: vipw will also apply some syntax checking so if you're at risk of trashing your entire login process because of a messed up edit then this would save you from that.

What's that? You don't like using vipw because it always uses vi? So change that! All it does is honor the EDITOR environment variable. Just point that to something else and you'll be home free.

But there are more reasons not to do this:

```
operator:*:2:5::0:0:System &:/:/usr/sbin/nologin
```
Which entry was the home directory again? Wait, I know! I think it was the empty entry between those two colons, let's change that...

vs:

`# chpass operator` (note: chsh also works):

```
#Changing user information for operator.
Login: operator
Password: *
Uid [#]: 2
Gid [# or name]: 5
Change [month day year]:
Expire [month day year]:
Class:
Home directory: /
Shell: /usr/sbin/nologin
Full Name: System &
Office Location:
Office Phone:
Home Phone:
Other information:
```
Everything you might want to edit in an impossible to miss way. This is how you edit /etc/passwd _without_ the risks of making any accidentally dumb mistakes (like the one I hinted at above). For more information check out chpass(1). This is also the best way to change your own settings, even as a regular user. Don't like your current shell? `chsh` to the rescue!

And there's something even better: pw. I'll admit: the sheer complexity of this command also set me back many times. I mean, just look at the _massive_ sea of entries when you check pw(8). Honestly: I have opened that manual page many times in the past only to glance at it, quickly close it and then run either vipw and/or vigr and left it at that.

If you're new to pw it is very easy to get totally overwhelmed and/or confused at first, and that is definitely not your fault.

But despite its difficult appearance it's actually extremely easy to use. Want to delete a user? `# pw userdel unwanteduser`, and done. Or what about a group? Fortunately logic is heavily applied here as well, so you can simply swap out userdel for groupdel.

Need to add someone to the wheel group? This means you need to modify a group: groupmod. So: `pw groupmod wheel -d currentmember -m newmember`. This would delete currentmember and add newmember.

Bottom line: there really is no need to manually edit /etc/passwd and/or /etc/group at all. At the very least rely on vipwd.

*#5 Reconsider the removal of any options from your customized kernel configuration*

Before you wonder... I'm doing the same thing myself, even made a sport out of this. Here is the top of /root/kernels/unicron.conf which is the symlink target for /usr/src/sys/amd64/conf/UNICRON:


```
include GENERIC
ident OMICRON

nooptions       QUOTA
nooptions       GEOM_RAID
nooptions       INCLUDE_CONFIG_FILE

# Floppy
nodevice        fdc

# ATA
nodevice        ahci, mvs, siis

# SCSI
nodevice        ahc, ahd, esp, hptiop, ispfw, mps, mpr, sym
nodevice        trm, adv, adw, aic, bt, isci
nooptions       AHC_REG_PRETTY_PRINT
nooptions       AHD_REG_PRETTY_PRINT
#nodevice       ispfw, ncr
```
(I know that I should rename the kernel as well, but.. that's for a later time)

This was manually edited against GENERIC, manually kept updated and before every major upgrade I carefully compare the new GENERIC with this file.

So here's the problem: What if my current hardware would suddenly fail and as an emergency replacement I got hold of a Tekram DC395U/UW/F DC315U adapter?   Well, simple: I can install the hardware but as you can see above my current kernel would totally ignore it because I manually removed support for that (notice the trm entry above?).

Basically: in order to use new hardware I'd have to rebuild a whole new kernel. Sounds like the perfect strategy for a server which hundreds of users depend on! 

Now, don't get me wrong here. These things can be fun to do. I also enjoy tinkering with my system (see above) and it also gives me a good sense of satisfaction that my kernel is pretty streamlined. Not to mention the fact that setting all of this up has taught me _a lot_ about the kernel and the source tree itself.

But always heed the risks.

And in case you wonder... I definitely enjoy tinkering but obviously I also came prepared   On my system I maintain /boot/kernel.gen as well which contains a copy of a GENERIC kernel with all the available modules (I usually rely on the latest kernel.txz). And my /boot/loader.conf contains this entry: kernels="kernel kernel.old kernel.gen". So if things do go dramatically wrong then I always have a GENERIC failsave available.

_End of part I_ (the entire message was too long)


----------



## ShelLuser (Apr 24, 2018)

*Part II*

*#6 Don't change the root shell to something else*

/bin/csh is the shell for the root user while everyone else usually gets /bin/sh, and this is done for a very good reason. Although many people use the root user and its privileges extremely casually you should never underestimate the amount of damage which a small mistake as root can do to your system. Don't treat root sessions as something casual!

This is one of the reasons why using a different shell can be a very good idea: because if you don't often use this then it will force you to think about your actions:

```
peter@unicron:/home/peter $ for a in a b c; do mkdir -p test/$a; done
peter@unicron:/home/peter $ file test
test: directory
peter@unicron:/home/peter $ csh
% for a in a b c; do mkdir -p test/$a; done
for: Command not found.
a: Undefined variable.
```
Of course there's more. The C shell also has a strong emphasis on interaction. That 'for' command I showed above? The C shell is perfectly capable of that as well, but in a much more interactive way:

```
% foreach a ( a b c )
foreach? mkdir -p test2/$a
foreach? end
% file test2
test2: directory
```
See what I mean? Instead of simply typing out one huge command on a single line this allows you to cut the whole thing up into smaller pieces, also allowing you to check for any typoes in the single command(s) and to (re)think about each of those as well.

Sure there's also a downside here: if you have to repeat this set of commands a few times then this is obviously not the best of options. But why not write a temporary shell script and use that instead?

*#7 Don't use the root user all the time*

Many people claim that Unix is a very secure operating system which can hardly be affected by any nastiness such as viruses. Although there is definitely some truth to that the underlying reasons for this don't necessarily have much to do with Unix. See, under normal circumstances you wouldn't be logged on as root and as a direct result of that....

```
peter@unicron:/home/peter $ rm /boot/kernel/*
override r-xr-xr-x  root/wheel uarch for /boot/kernel/agp.ko? y
rm: /boot/kernel/agp.ko: Permission denied
override r-xr-xr-x  root/wheel uarch for /boot/kernel/coretemp.ko? y
rm: /boot/kernel/coretemp.ko: Permission denied
peter@unicron:/home/peter $ rm -rf /boot/kernel/kernel
rm: /boot/kernel/kernel: Permission denied
peter@unicron:/home/peter $ echo "rm -rf ~/*" >> /etc/profile
/usr/local/bin/ksh: cannot create /etc/profile: Permission denied
peter@unicron:/home/peter $ echo "/home/peter/bin/passwordh4x >> /home/peter/ >> /root/.profile
/usr/local/bin/ksh: cannot create /root/.profile: Permission denied
```
Slightly offtopic, but seriously: if you want to you could achieve the exact same thing on Windows as well:






'Toegang geweigerd' aka: 'Permission denied'

I'm actually using Windows as a regular (non-administrator) user and as a result I don't have access to most system environments. Some ransomware trying to encrypt c:\windows\system32 on my system? Best of luck with that! 

Back to FreeBSD: part of keeping your system secure is not to use excessive options if you don't have to. And root is an extremely powerful user because it basically has little to no restrictions. A mistake as root can have much bigger consequences than those of a regular user. Stay safe...

*#8 /var/backups is a thing*

Now, sure, you should not rely too much on automated "out of the box" backups, but if certain problems do occur then definitely check out this location first. Did something trash your package database? You might find the solution here. Were you too quickly with that pw command and removed an important account? This might be a quick way out too.

Of course you shouldn't overly depend on this either but this system very slick. It is powered by the Periodic system and actually quite intelligent too. For example: although it can keep a "roulation backup" of your files (with this I mean that it keeps several retention copies) it won't if there is no need. For example: /var/backups/aliases.bak dates from October last year on my system. And there's also an aliases.bak2 dating from July last year. Guess what? Last time I applied changes to the /etc/aliases file were.. you might have guessed it: around July and October last year 

*#9 Check system integrity using /etc/mtree*

Did you know that FreeBSD includes a full fledged intrusion detection system out of the box called mtree? Now you do 

Keep in mind that the specifications in /etc/mtree only track directory permissions and not those of the individual files. So if you want to use the files in /etc/mtree then you also might want to use the -e parameter to exclude any file checks:

```
peter@unicron:/# mtree -e < /etc/mtree/BSD.root.dist
./etc/autofs missing
./etc/bluetooth missing
./etc/ppp missing
```
Guess what? It's correct about that. If I wanted to I could tell mtree to fix these issues automatically using the -u option.

This can be the perfect way to check if no one messed with some directory permissions:


```
peter@unicron:/var# mtree -e < /etc/mtree/BSD.var.dist
db/ports:
        permissions (0755, 0775)
```
Another very well spotted change.

I actually set this up as an extension of #7; very often I don't build ports as root but using my normal user account. This permission allows any user within the wheel group to modify the options of a port without having to elevate their permissions.

Want to learn more about mtree (it can do tons more)? Then you might enjoy my small mtree guide.

*#10 What works for me doesn't have to work for you!*

All these items are basically best practices but definitely not mandatory rules. Always keep in mind that every situation is most often unique and when it comes to security, configuration and an operating system there is no such thing as a "one size fits all" solution.

Sure there are risks involved when manually firing up vi to edit /etc/passwd, but not so much if you're in single user mode. Running a fully customized kernel definitely has its quirks if you need to replace hardware, but how often would you do that on a laptop anyway? It's cool to be able and check permission defaults, but what if you applied better permission settings specific for your own environment? Then you probably don't care about /etc/mtree at all.

"_Your milage may vary_" as the saying goes.

And there you have it, 10 regular do's and dont's for FreeBSD. Hope this was useful for some of you.


----------



## robroy (Apr 24, 2018)

ShelLuser, this is probably my favorite of all of your posts and was an excellent idea; thanks for this.


----------



## gkontos (Apr 25, 2018)

Dang! I never imagined that `whoami` ever existed in Windows


----------



## Gumnos (May 22, 2018)

Curious on #1 (mixing packages & ports) since [FONT=Courier New]portmaster()[/FONT] has explicit "[FONT=Courier New]--packages[/FONT]*" options for installing dependencies from packages instead of from ports. It a lot faster if you just want to tweak some top-level package's build-options but not spend hours rebuilding dependency ports. I also understand that part of the make-a-port process actually creates a package that then gets installed via the normal pkg()-style machinery.


----------



## ShelLuser (May 22, 2018)

Gumnos said:


> Curious on #1 (mixing packages & ports) since [FONT=Courier New]portmaster()[/FONT] has explicit "[FONT=Courier New]--packages[/FONT]*" options for installing dependencies from packages instead of from ports.


I'm familiar with the option(s) and I personally consider it a bad one. It's all about how you use it though...  For example: --packages-build could be useful and not too much destructive.

I wrote a more in-depth guide about the issues with mixing ports and packages, might be a good read:

https://forums.freebsd.org/threads/guide-about-ports-and-binary-packages.62126/

The problem is that it's not always too obvious that something is very wrong. The main problem is when you change build options and then install packages which rely on those changed packages. Because they rely on a package with default build options, and if you changed something which the other packages actually use... then this could turn into a disaster. Or something totally unnoticed.


----------



## zirias@ (May 22, 2018)

I'm not sure I get the reasoning behind your advice #6 ... not wanting to revive an old flame here (about whether you should use a C shell _at all_), but there are well-known arguments against it.

What I don't understand from your text is -- what bad could actually happen if you chose a different shell for root? On my system, I even build the base system `WITHOUT_TCSH=yes` and use zsh for everyone, including root. I didn't have any problems with this setup? Is your point that with the same shell, you could risk to forget you're logged in as root? This is easily solved e.g. by a prompt looking _very _different.

And about #7 -- that's reasonable of course! But I'd qualify it a bit, thinking about my usecase on a server with jails for services, many of them only having root and some system users: You'll always log on as root on these to do administrative tasks and that's fine  It's  a different story on a workstation of course.


----------



## sidetone (May 22, 2018)

I always combine ports and packages. If you keep track of it, and don't make it too complex, you can do it. It depends on the set of ports of course, and that they are both from the same release: quarterly or head. Certain software makes good use for package installation, like programming language dependencies, and build only dependencies if they are up to date, especially from the Vuxml database. The next time, I rebuild a port over an old package, some dependencies get stripped out from the make config menu options.

Most of the programs I use can be `grep`-ed from my window manager start up configuration file. That way, I install those, and not treat a former dependency as an independent package or port needing to be installed.


----------



## sidetone (May 22, 2018)

On #5, I rename one GENERIC kernconf to TRIM or something like that, that is stripped down and has no additions to it. That file is included from a CUSTOM kernconf that references much of what is displayed by typing `kldstat`.


----------



## xavi (May 22, 2018)

Zirias said:


> What I don't understand from your text is -- what bad could actually happen if you chose a different shell for root?



It is recommended that the root user's default shell remain unchanged since shells which are not included in the base distribution are installed to /usr/local/bin. In the event of a problem, the file system where /usr/local/bin is located may not be mounted. In this case, root would not have access to its default shell, preventing root from logging in and fixing the problem.


----------



## DutchDaemon (May 22, 2018)

To use a different shell for root, the toor account  is available. Just use `su - toor` after giving toor its own password and shell through `chsh`. Leave root alone.


----------



## zirias@ (May 23, 2018)

So, not really a reason to use a C shell for root -- after all, /bin/sh will be used anyways if you built `WITHOUT_TCSH=yes`.

*But*, a reason to only use shells in /bin. Yes, this _might _be important, but it depends on the configuration of your system. I don't really see risks here when you have a single zpool, or you're inside a jail that's constructed using null-mounts, but of  course this is important on a classical setup with individual filesystems on different partitions. In the original posts, it reads like root *must *use the C shell -- without really giving a reason for that


----------



## dch (May 23, 2018)

I'll gladly throw in 10 things from me, should that go here, or in a separate post?


----------



## mod3777 (Mar 31, 2019)

xavi said:


> It is recommended that the root user's default shell remain unchanged since shells which are not included in the base distribution are installed to /usr/local/bin. In the event of a problem, the file system where /usr/local/bin is located may not be mounted. In this case, root would not have access to its default shell, preventing root from logging in and fixing the problem.



`zpool import -fR zroot && zfs mount zroot/ROOT/default` and rollback/chroot from Live USB stick. Unless there is a hard disk failure, nothing can break a properly snapshot_ed FreeBSD system.


----------



## linux->bsd (Mar 31, 2019)

Great list, except for the "OMG root is so powerful -- don't use it!" advise that permeates the *nix community. Forgive the analogy, but it's like telling people guns are dangerous so they shouldn't ever touch one unless absolutely necessary. So instead of learning how to _safely_ use root by continual practice, users end up not having the skills necessary to use it well when they have to.

I get telling new users to not use it by default, and I get telling users not to run around in an X11 session logged in as root, but the rest is just fearmongering by people that should know better.

I once worked with people that belonged not-within-100-miles of root access, but because their position required its occasional use, they would inevitably wreck entire servers. But instead of learning how to use it well, they shied away from it even more and never got better at its use. Practice makes perfect because it forces you to learn _good habits_ and _technique_, and it's those good habits that prevent you from breaking the system. It's like those images you see of Special Forces: the safety is _off_ on their weapons; it's their discipline and training (trigger finger *off* the trigger) that prevents accidents.


----------



## rootbert (Apr 1, 2019)

dch said:


> I'll gladly throw in 10 things from me, should that go here, or in a separate post?


I am always keen on learning so go ahead here ;-)


----------



## unitrunker (Apr 1, 2019)

I've seen *#1 Don't mix ports and binary packages* repeated over and over with no good advice on how to cope with the common situation of one or two ports that must be build from /usr/ports instead of pkg.

In my case, I needed SASL support in svnserver. The pkg version did not have it. Everything else installed came from pkg.

What to do?

Go to Jail. Create a jail for the one or two odd ports that must be built by hand. Install all dependencies there.

I've also run into dependency problems across packages - like different versions of pgsql. Running different versions in separate jails solves this.


----------



## aimeec1995 (Apr 5, 2019)

I usually mix ports and binaries anyways because I don't want to wait weeks for a package to build but I also need the latest version of some things and/or something is not available in the ports tree as a binary


----------



## SirDice (Apr 5, 2019)

aimeec1995 said:


> I usually mix ports and binaries anyways because I don't want to wait weeks for a package to build


Switch to the latest package branch. Builds happen fairly frequently nowadays. In the past it could take some time but it never took weeks. I'm betting you are looking at the quarterly packages, those are updated once every three months and only receive security and stability updates. 



			https://pkg-status.freebsd.org/


----------



## Deleted member 30996 (Apr 5, 2019)

linux->bsd said:


> Great list, except for the "OMG root is so powerful -- don't use it!" advise that permeates the *nix community. Forgive the analogy, but it's like telling people guns are dangerous so they shouldn't ever touch one unless absolutely necessary. So instead of learning how to _safely_ use root by continual practice, users end up not having the skills necessary to use it well when they have to.



I have never used anything but `su` to become root. I do a lot of file transfers that require root access to mount a USB stick so I become root in the terminal then open x11-fm/xfe to work from as root. When I'm done I unmount the media, close my file manager (which free's up the terminal if I need to do anything else) and log out from the root account back to my usr account.

When I compile ports I exit x11-wm/fluxbox to the login terminal and `su` to work so I can tell at a glance what's going on.

When I'm working as root I never open a browser from my usr account or do anything that might compromise the root account, much less open a browser as root. I keep 2 laptops running so I can use another if need be.

I've been advised it's not good practice but am very comfortable working as root and consider myself competent to do so. Knowing full well I won't be asked if I really want to do something I did not intend to and the possible consequences thereof, but that doesn't happen.


----------



## linux->bsd (Apr 6, 2019)

Trihexagonal said:


> I have never used anything but `su` to become root.
> ...
> When I'm working as root I never open a browser from my usr account or do anything that might compromise the root account, much less open a browser as root.
> ...
> I've been advised it's not good practice but am very comfortable working as root and consider myself competent to do so. Knowing full well I won't be asked if I really want to do something I did not intend to and the possible consequences thereof, but that doesn't happen.



That's pretty much my workflow, too, except I always have a tmux session running as root in a different Terminal tab. When I need root access, I do my work, then back to my non-privileged user for daily tasks. I use $PS1 to indicate which user I am: 'root' in the prompt in screaming red. Before doing any deletions or overwriting files, my pinky freezes over the Enter key for at least two seconds instinctively. good habits == good sysadmin.


----------



## msplsh (Apr 8, 2019)

unitrunker said:


> I've seen *#1 Don't mix ports and binary packages* repeated over and over with no good advice on how to cope with the common situation of one or two ports that must be build from /usr/ports instead of pkg.



The common advice is to build your own packages in your own repo on your own server using Poudriere.  Presumably because if you make a mistake with options and dependencies, it blows up there and not in your install.

If you don't want to do that, people have developed other "wrong" ways of manually managing this, like mine.


----------



## hukadan (Apr 8, 2019)

msplsh said:


> people have developed other "wrong" ways of manually managing this


Someone also developed a "right way(TM)" of automatically managing this : the *Fetch prebuilt packages* option of synth(1) . It seems a lot simpler than the method you describe in your post.


----------



## msplsh (Apr 8, 2019)

I'll have to say I've never heard anyone suggest this specific option or knew it existed.  I'll try it out.


----------



## hukadan (Apr 8, 2019)

msplsh said:


> I'll try it out.


I would be interested by the results, namely if it builds more packages than with your method. Just be sure to set your repository to _latest_ (sorry if it is obvious to you, it is in the FAQ of synth so it is not obvious to everyone).


----------



## kpedersen (Apr 14, 2019)

aimeec1995 said:


> I usually mix ports and binaries anyways because I don't want to wait weeks for a package to build but I also need the latest version of some things and/or something is not available in the ports tree as a binary



Me too and I find that if you only use ports for applications that are not dependencies, you can never break anything.

So if I need SDL2, I will grab a package (because I don't really care about being the latest version), but if I want the latest version of Gimp or Blender, that the packages are lagging behind, I will simply build it as a port (only a port, I will use packages for the dependencies).

Since I know that nothing has a dependence on Gimp of Blender, I am 100% guaranteed that my system will not break.

This is because ports are flexible and will target any version of a libary you have installed; whereas packages often have hardcoded version numbers.


----------



## zader (Jan 7, 2020)

by mixing ports/packages..

would this also apply if you installed portmaster on the host and installed ports ... but had a jail or two that used packages? I have one jail with plex in it that was installed via pkg .. I just crontabed a package update command ..  its brand new so if its best to just nuke it and use portmaster is that a better option?

is there an option to globally build/upgrade all packages on hosts and in jails with port master? 

thanks


----------



## Lamia (Jan 8, 2020)

zader said:


> would this also apply if you installed portmaster on the host and installed ports ... but had a jail or two that used packages? I have one jail with plex in it that was installed via pkg .. I just crontabed a package update command .. its brand new so if its best to just nuke it and use portmaster is that a better option?


They are different systems - host and jail(s). You can choose to use pkgs in a host and ports in jails and vice versa. 


> I just crontabed a package update command .. its brand new so if its best to just nuke it and use portmaster is that a better option?


If it works well, you don't have to. It is not everytime that ports successfully builds a pkg. Take for instance Firefox; many people report success building it via ports. But a large number of people also report otherwise. The latter will have to fall on pkgs. 



> is there an option to globally build/upgrade all packages on hosts and in jails with port master?


I can't think of one. That's why we have poudriere and others. Another unrelated option is flavor in (ez)jails.


----------



## Alain De Vos (Jan 8, 2020)

As root shell I use mksh, installed in /bin. It is only dependent on /lib/libc.so.7
/etc/make.conf
.if ${.CURDIR:M*/shells/mksh}
PREFIX=/
.endif


----------



## CraigHB (Jan 9, 2020)

Right now I only use packages, but I do build my own machine specific kernels (because it's just cool and makes things more efficient).  At some point I may need to use a port of something and would like to know how to address the issue.  So is using `synth` to build your own packages a good way to avoid the dependency problem of mixing ports and packages?  That seems like a pretty easy way to deal with it.


----------



## Sevendogsbsd (Jan 9, 2020)

Easier than ports-mgmt/poudriere but I am used to the latter so am working on re-setting up my build box again. Not having luck with 2 ports though so fortunately I have not switched my main machine's repository yet. Not sure I am going to use this as a final solution because it's a bit of a pain. Not a big deal but I have to updated ports, then run poudriere to build, which normally takes a few hours, then update everything on my workstation from the new repository. It's not like I am run a hurry though - just my personal workstation at home.

"Latest" packages works very well for me so far.


----------



## SirDice (Jan 10, 2020)

Sevendogsbsd said:


> Not a big deal but I have to updated ports, then run poudriere to build, which normally takes a few hours, then update everything on my workstation from the new repository.


Once you have everything set up just create a small script that runs the update and starts a build. Turn it into a cronjob and let it do it's thing while you sleep. 

Here's what I've been using for some time now:

```
#!/bin/sh

POUDRIERE=/usr/local/bin/poudriere

BASEDIR=/usr/local/etc/poudriere.d

${POUDRIERE} ports -u -p desktop
${POUDRIERE} ports -u -p server

for j in 12-stable 121-release; do
  ${POUDRIERE} bulk -j ${j} -p server -f ${BASEDIR}/${j}-server-package.lst
done

for j in 12-stable; do
  ${POUDRIERE} bulk -j ${j} -p desktop -f ${BASEDIR}/${j}-desktop-package.lst
done
```


```
# ll *-package.lst
-rw-r--r--  1 root  wheel  1426 Dec 26 21:16 12-stable-desktop-package.lst
-rw-r--r--  1 root  wheel  1392 Dec 26 16:12 12-stable-server-package.lst
-rw-r--r--  1 root  wheel  1459 Dec 29 23:10 121-release-server-package.lst
```

I just let this run during the night. I used to have a 113-release jail too but I recently upgraded everything and only have 12-STABLE and 12.1-RELEASE now.


----------



## driesm (Jan 10, 2020)

SirDice said:


> Once you have everything set up just create a small script that runs the update and starts a build. Turn it into a cronjob and let it do it's thing while you sleep.



I have used both and I think Poudriere is the better tool but what I don't like about it is the jails.
Synth uses the host system with null_fs mounts and chroot to mimic a jail, there is an open PR to add this to Poudriere.
I always wonder, do you have to update the poudriere jail each time you do a source upgrade of the host system?
When runing STABLE-12 that can be quite frequent, and each time a poudriere jail is updated it rebuilds all packages.

SirDice, do you have any experience with this, or ever observed a breakage? Or should you only update the poudriere jail when the host system has gotten a major version upgrade?

TBH, using the base system as a jail is the only real feature I miss in Poudriere that is a bit of a blocker for me to switch.


----------



## SirDice (Jan 10, 2020)

Duffyx said:


> I always wonder, do you have to update the poudriere jail each time you do a source upgrade of the host system?
> When runing STABLE-12 that can be quite frequent, and each time a poudriere jail is updated it rebuilds all packages.


I update my 12-STABLE rather irregularly. Usually only when I feel like it or if there's some security issue. I have one machine I use for reference, I just follow the usual buildworld/installworld etc. there. Then I run a `make release` which gives me the images and an FTP tree. I use that FTP tree to update the poudriere jail:

```
JAILNAME    VERSION         ARCH  METHOD                                    TIMESTAMP           PATH
121-release 12.1-RELEASE-p1 amd64 ftp                                       2019-11-16 16:28:09 /usr/local/poudriere/jails/121-release
12-stable   12.1-STABLE     amd64 url=file:///storage/release/12-stable/ftp 2019-12-17 00:10:11 /usr/local/poudriere/jails/12-stable
```
Now the 12-stable can be updated fairly quickly with `poudriere jail -u -j 12-stable`. 



Duffyx said:


> @SirDice, do you have any experience with this, or ever observed a breakage?


Do things ever break? Sure. Some ports sometimes fail to build for whatever reason. The poudriere web interface is very useful and I check it regularly to see if anything broke. You can easily check the failed build logs there too.


----------



## Sevendogsbsd (Jan 10, 2020)

Now I feel stupid: there is a poudriere web interface?  That would be handy because I have 2 ports that won't build and I can't figure out why.


----------



## SirDice (Jan 10, 2020)

Sevendogsbsd said:


> there is a poudriere web interface?


There is, yes. Look in /usr/local/share/examples/poudriere/ for nginx and Apache configuration examples.


----------



## zader (Jan 10, 2020)

thanks SirDice, that's good info .. I have been struggling a bit installing poudriere in a jail.. (vnet+loopback is a pita and for some reason when I setup a zfs filesystem in the jail, iocage can no longer unmount it for restarts/shutdowns)

q: ... is it worth going through the extra effort to create a zfs file-system within the poudriere jail? or just use the iocage defaults..  my thought was it could be good for snapshots.. but at the same time if I'm going to roll back  .. just rolling back from the iocage dataset is just as easy..

after digging into this. I really would LOVE to see a default install option to create a local repository by default ..


----------



## Sevendogsbsd (Jan 10, 2020)

So there is a sticky in this forum on how to install and set up poudriere. Very straight forward. You don't have to do anything special about the jails, they get created dynamically. I use zfs on my build box so didn't worry about what was "in"
 the jail. Here is a link to the tutorial: Thread 38859


----------



## forgiven_noob (Jan 11, 2020)

i mix ports and pkg, it is kind of a mess
but i do not want to wait days for things like llvm or xorg to build
i only need a few programs to be as recent as possible and to be built with specific things (VLC with ASS, nvidia-driver with acpi power management and a few other things)
and from what i can see most of the more refined package management solutions involve building things from source one way or the other
my pc is not very good it can take a very long time to build something, i have just 2 cores


----------

