# Confused: how does pkg, ports, and freebsd-update work?



## shirozuki (Mar 16, 2015)

Hello Everybody!

Just recently I migrated to FreeBSD from Debian Linux and I'd like to ask a couple of noobish question about things that confuse me in FreeBSD OS. I'm sure that information concerning things I want to ask about are there somewhere but I just can't seem to find straightforward answers. Please go easy on me 

The first thing: the 'freebsd-update' command. When I use it to download and apply 'patches' I'm applying them to what exactly? Only kernel, applications included in vanilla installation of FreeBSD? And how does this relate to 'pkg' command? Is pkg able to update the same things freebsd-update updates? I just can't understand this, because in Debian there was no division - all I had to do was 'apt-get upgrade' and the system updates everything from kernel to web browser.

The second thing: how ports relate to the package system? When I install something from ports will the information concerning this application be available to pkg? Can I update an application which I installed from the ports collection with pkg? And if so - how would the update process treat custom-build application?


----------



## drhowarddrfine (Mar 16, 2015)

Packages are pre-compiled ports. Ports will always be ahead of packages because someone has to compile and test them. When you install from ports, a package is made and installed.

Installing from packages will be faster since compiling is not necessary.

`freebsd-update` adds patches to the kernel and all the supporting systems but not applications.

https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/ports.html
https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/updating-upgrading-freebsdupdate.html


----------



## shirozuki (Mar 16, 2015)

Thank You!
And, just to be sure: the packages are pre-compiled ports WITH default compilation options selected?


----------



## protocelt (Mar 16, 2015)

shirozuki said:


> Thank You!
> And, just to be sure: the packages are pre-compiled ports WITH default compilation options selected?


That is correct.


----------



## chrbr (Mar 16, 2015)

shirozuki said:


> And, just to be sure: the packages are pre-compiled ports WITH default compilation options selected?


Nevertheless this does not mean that mixing ports and packages can never lead into any problems .


----------



## shirozuki (Mar 16, 2015)

chrbr said:


> Nevertheless this does not mean that mixing ports and packages can never lead into any problems .


Could You please elaborate?  What should I be aware of and pay attention to when mixing them?


----------



## drhowarddrfine (Mar 16, 2015)

I've forgotten the list but one problem is due to what I mentioned before, that packages are later than ports and, if a package requests an older dependency that was updated and installed by using ports, there could be issues.

I _think _most people choose one or the other. I have a fast system so I use ports for everything. Someone on a slow system would probably be best to stick with packages. An experienced user knows how to deal with the potential problem I mentioned.

However, I believe portmaster(8) and pkg(8) now warn of inconsistencies but someone else will have to verify that.


----------



## tankist02 (Mar 17, 2015)

I am trying to find how to determine what ports revision was used to build latest official packages. I remember somebody even wrote a little script to automate it, but I can't find it. Can somebody help?

The idea is to use exactly the same ports as were used for building packages, so potential inconsistencies will be avoided.


----------



## kpa (Mar 17, 2015)

tankist02 said:


> I am trying to find how to determine what ports revision was used to build latest official packages. I remember somebody even wrote a little script to automate it, but I can't find it. Can somebody help?
> 
> The idea is to use exactly the same ports as were used for building packages, so potential inconsistencies will be avoided.



The SVN revision of the ports tree used to build the packages is not included in the package metadata (the ports are also built from portsnap(8) trees where the SVN revision is not even available) so I don't see how this could be even achieved? If it was it would be very trivial to match the right revision of the ports tree, you just checkout the correct revision and use that.


----------



## tankist02 (Mar 17, 2015)

IIRC there was a way to extract ports revision used to build official packages. This info is not in a package meta-data, but, I think, somewhere in the build cluster history, log or something.


----------



## jrm@ (Mar 17, 2015)

shirozuki, no one has mentioned ports-mgmt/poudriere. ADDED: Although you'll find information in the Handbook linked to above.  If you have a multi-core system, preferably with ZFS, then poudriere can be the best of both worlds.  You can customize the options ports are compiled with, ports can be compiled in parallel and you only have to compile once if you have more than one system.  Here's a tutorial.  It's more involved to get things set up than simply `pkg install some_package`, so if you're trying to ease into things it might not be what you're looking for.


----------



## chrbr (Mar 17, 2015)

shirozuki said:


> Could You please elaborate?  What should I be aware of and pay attention to when mixing them?


When I understand the experts in this forum correctly it is difficult to make sure that the dependencies are on track. As far as I know ports and packages finally use pkg as back end but the source and hence the dependencies can differ. Therefore it might be better not to mix ports and packages unless the user knows exactly what is going on. At least this is my understanding and conclusion.


----------



## ANOKNUSA (Mar 18, 2015)

chrbr said:


> When I understand the experts in this forum correctly it is difficult to make sure that the dependencies are on track. As far as I know ports and packages finally use pkg as back end but the source and hence the dependencies can differ. Therefore it might be better not to mix ports and packages unless the user knows exactly what is going on. At least this is my understanding and conclusion.



The ports tree is rolling---port maintainers can update their ports at their leisure, and users can grab a snapshot via portsnap or svn-lite at their leisure. The package repository is only updated once per week, though. So without knowing exactly which revision of the ports tree is used to update the repository there's a potential for conflicting dependencies. I typically install a package from the repository if I want to try it out. If I like it, I add it to my list of packages to be built with ports-mgmt/poudriere. I run Poudriere once a week, so a package could sit on my system for days before being rebuilt from source. For what it's worth, I've yet to run into problems doing this. Dumb luck, maybe.


----------



## sidetone (Mar 18, 2015)

`freebsd-update` updates base files, plus the generic kernel. Base files would be most of what is initially in (but not limited to) /bin, /etc, /sbin boot and other files that are used before any packages or ports are added. The kernel can be installed independently of freebsd-update, too, so while it is part of the base, it is sort of different.



tankist02 said:


> IIRC there was a way to extract ports revision used to build official packages. This info is not in a package meta-data, but, I think, somewhere in the build cluster history, log or something.


 I don't know, but I do know that port builds use the "HEAD" revision, which means the latest. This seems to imply another revision of ports, but not necessarily. If possible, it'd have to be a snapshot.


----------



## tankist02 (Mar 18, 2015)

I found what I was looking for:

To get ports revision that was used to build packages for FreeBSD 10 amd64:


```
svn up -r `fetch -qo - http://beefy2.isc.freebsd.org/data/10amd64-default/.data.json | jq -r '.builds[.builds.latest].svn_url | split("@")[1]'`
```

Is it a good way to keep ports and packages in sync?


----------



## sidetone (Mar 18, 2015)

Use svnup(1), net/svnup, to update kernel source files. Use portsnap(8) to update your ports, and use freebsd-update(8) to update the base system.

svnup is much easier to use than other svn programs. All you have to do is uncomment which server you choose, and reference which section you want to update from /usr/local/etc/svnup.conf.

Something like this, svnup is one word in this example: `svnup release`


----------



## shirozuki (Mar 19, 2015)

Big thanks to all of you guys for sharing your knowledge! I'd very much like to learn more about FreeBSD. The official FreeBSD handbook is probably the best piece of documentation I've ever seen, but the way I see it - it's more about getting certain things done allowing one to easily accomplish specific tasks. Does anyone know any good tutorial or even a book that could help a noob such as myself understand how FreeBSD functions and get a little bit more familiar with its structure?


----------



## teo (Mar 19, 2015)

sidetone said:


> svnup is much easier to use than other svn programs. All you have to do is uncomment which server you choose, and reference which section you want to update from /usr/local/etc/svnup.conf.


Hi sidetone,  according to the  FreeBSD Handbook, he has to install the port devel/subversion and check the sources remotely and then update. Is it possible to update the sources with svnup? Must I install the port svnup to update the  sources on FreeBSD?  what  line of the file must uncomment?  Can I update all the sources without removing any reference?


----------



## sidetone (Mar 19, 2015)

teo said:


> Hi sidetone,  according to the  FreeBSD Handbook, he has to install the port svn and check the sources remotely and then update. Is it possible to update the sources with svnup? Must I install the port svnup to update the  sources on FreeBSD?  what  line of the file must uncomment?  Can I update all the sources without removing any reference?


net/svnup can replace svn directly for the purpose of updating sources, for common use. To configure and use it, edit /usr/local/etc/svnup.conf
uncomment one of the host lines according to which server is closer to you, and check that the [release] section matches your version:

```
host=svn0.<choice of server>.freebsd.org
[release]
branch=base/releng/10.1
```
Then all you'd have to do is type svnup and the section of the options you want.
	
	



```
svnup release
```

You can use either svnup or svn, you don't need both, but svnup is made to be easier to use for those who don't need development options.

svnup(1), svnup.conf(5)

I didn't understand your last question  about removing references, but svnup works independently svn, uses a different configuration file, and if you meant uncommenting, svnup.conf's configuration is simple. I believe this answers your last question too, or is easier to figure out now.


----------



## sidetone (Mar 19, 2015)

shirozuki said:


> Does anyone know any good tutorial or even a book that could help a noob such as myself understand how FreeBSD functions and get a little bit more familiar with its structure?



There are several books but (almost all of) the latest ones were made for FreeBSD 6 and 7 releases. Much is still the same, but they covered csup and cvup which aren't used for updating FreeBSD's base sources anymore. There are some books listed in https://www.freebsd.org/publish.html, and there are a few not listed there. FreeBSD 6 Unleashed, Network Administration with FreeBSD 7, Absolute FreeBSD (2nd edition), the Best of FreeBSD basics, are a few books that I want or already own. There are some more recent books too, these are made for specific tasks, are in other languages, or are made with information from wikis. https://wiki.freebsd.org/ is a wiki for developers, not for common tasks; http://www.freebsdwiki.net/index.php/Main_Page is a wiki, but a lot of it's latest edits were from 2007. http://www.bsdnow.tv/  and  http://www.bsdguides.org are two websites. There are a couple more websites and many manpages to fill in on what can't be found or was updated.


----------



## protocelt (Mar 19, 2015)

sidetone said:


> net/svnup can replace svn directly for the purpose of updating sources, for common use. To configure and use it, edit /usr/local/etc/svnup.conf
> uncomment one of the host lines according to which server is closer to you, and check that the [release] section matches your version:
> 
> ```
> ...


If you are using a version FreeBSD 10.0-RELEASE or greater and all you want to do is sync from the latest available source , you can just use svnlite(1) which is included in the base install and will accomplish the same thing as using net/svnup or devel/subversion(svn). For example:

Initial full checkout:
`# svnlite checkout https://svn0.us-west.freebsd.org/base/releng/10 /usr/src`

Updating local source:
`# svnlite up /usr/src`


----------



## sidetone (Mar 19, 2015)

Does `svnlite` use a config file that is simple like svnup.conf? Does it use svn.conf? I looked into `svnliteadmin`, and didn't see options for it. svnlite's manpage didn't say anything on it, and the link from that manpage was about svn in general and was from 2011. There's nothing about svn in /etc either.


----------



## kpa (Mar 19, 2015)

sidetone said:


> Does svnlite use a config file that is simple like svnup.conf? Does it use svn.conf? I looked into svnliteadmin, and didn't see options for it. svnlite's manpage didn't say anything on it, and the link from that manpage was about svn in general and was from 2011.



It's the full blown Subversion revision control system that is designed to do a whole lot more than just checking out a copy from a repository. There is no "simple" configuration file for it. It does say "lite" but that's just because it has the more advanced repository management features stripped away so it can be built as part of the base system without too many dependencies.


----------



## protocelt (Mar 19, 2015)

This revision also explains it a little more; https://svnweb.freebsd.org/base?view=revision&revision=251886


----------



## sidetone (Mar 19, 2015)

`svn` from devel/subversion uses svn.conf. Does that mean svnlite uses svn.conf too?


----------



## kpa (Mar 19, 2015)

sidetone said:


> `svn` from devel/subversion uses svn.conf. Does that mean svnlite uses svn.conf too?



It probably does but the configuration file is not for configuring repositories or anything like what you can do with the net/svnup configuration file. Instead it's for configuring global settings for the Subversion client. Checked out copies in Subversion are self-contained, recognized by the existence of a .svn metadata directory, and the user is expected to either change the current directory to a working copy first or give the full path to the working copy on the command line.


----------



## teo (Mar 20, 2015)

protocelt said:


> If your using a version FreeBSD 10.0-RELEASE or greater and all you want to do is sync from the latest available source , you can just use svnlite(1) which is included in the base install


Always and installed the sources from the port devel/subversion  and verified the sources remotely and then update. As I remove the tree ports or sources if they are problems to check remotely? According to the FreeBSD Handbook, to update ports and sources is `svn update /usr/ports` and `svn update /usr/src`?  After updating ports and sources, is it necessary the  reconstruct  world for  system of  desktop?

sidetone, I have nothing in the file /usr/local/etc/svnup.conf, necessarily I have to install the net/svnup port and then edit the file /usr/local/etc/svnup.conf? I do not understand it of section, net/svnup release, is command?


----------



## sidetone (Mar 20, 2015)

teo
After net/svnup is installed, /usr/local/etc/svnup.conf and /usr/local/etc/svnup.conf.sample show up, and the command `svnup` becomes available.

Svnup is only an alternative; you only need 1 of a choice of svnlite, svnup, or devel/subversion


----------



## ANOKNUSA (Mar 20, 2015)

shirozuki said:


> The official FreeBSD handbook is probably the best piece of documentation I've ever seen, but the way I see it - it's more about getting certain things done allowing one to easily accomplish specific tasks.



Well, you need to remember that the software which actually constitutes the FreeBSD operating system---the "base system"---is pretty slim. It's ~800 megabytes or so for a default installation (I think---I've trimmed some stuff out of my install, so I don't recall exactly). The Handbook covers how to get those integral parts of FreeBSD working the way you want them to, sort of a comprehensive Beginner's Guide. There are some parts of the system not covered in the Handbook that are pretty awesome, things I've just stumbled across since I started using FreeBSD. If you wanna see a few of the things you can do with and to the system itself, check out things like nanobsd(8), release(7), build(7), make.conf(5) and src.conf(5) for starters.



shirozuki said:


> Does anyone know any good tutorial or even a book that could help a noob such as myself understand how FreeBSD functions and get a little bit more familiar with its structure?



Absolute FreeBSD, 2nd Edition is pretty well-written. As sidetone mentioned a little bit of it is outdated---how to update software, for example---but most of it is still relevant. It's also written for people new to *nix in general, so as a Debian user some of it will be old hat to you (You probably know what /etc and cron are by now, for example  ) My local public library had a copy on-hand.


----------



## ronaldlees (Mar 21, 2015)

I've used FreeBSD for quite a while, but still have what probably ranks as a newbie question.  I think it is on topic for the thread. Here it is:

I know I can use portsnap to update the entire ports tree, but how do I update a single port?  (preferably without using svn directly).


----------



## ANOKNUSA (Mar 21, 2015)

ronaldlees said:


> I know I can use portsnap to update the entire ports tree, but how do I update a single port?



This might be possible, but it's a very bad idea. Update one port, and if its dependencies have changed you'll have to update them as well. Update those dependencies, and you'll have to update any ports that share those dependencies. Failure to do so will inevitably break something, and the longer it takes for something to break the less likely you are to know what went wrong. You can't snatch a link from the middle of a chain without breaking the whole chain.

You could of course just keep the ports tree up-to-date and upgrade a single port with `make reinstall`; if anything breaks in such an instance, you would (at least ideally) just have to run a full update of all your ports to get everything working again. It's still a risk, though a much smaller one that's easier to resolve. I believe ports-mgmt/portmaster can be used to upgrade individual packages without much hassle, but again, you should always keep the entire ports tree in sync.


----------



## ronaldlees (Mar 21, 2015)

Sometimes I just like to do the cowboy coder sort of thing, and take my chances, just cuz it's faster


----------



## shirozuki (Mar 27, 2015)

One more question 
I've built a custom kernel for my hardware. Now when I'll run `freebsd-update install` what will happen with it? Will patches apply to the default kernel located in /boot/kernel.old?


----------



## getopt (Mar 27, 2015)

`freebsd-update` requires a GENERIC kernel.


----------



## jrm@ (Mar 27, 2015)

getopt said:


> `freebsd-update` requires a GENERIC kernel.


To clarify, you can still run `freebsd-update` with a custom kernel, but last time I tried, which was several years ago, it prompted/warned you to compile your kernel.


----------

