# pkg build lag/schedule?



## spork (Sep 8, 2018)

Curious how long in general it should take before security updates on a port make their way to the official pkg repo?

For example, two days ago a curl vulnerability was fixed:

https://vuxml.freebsd.org/freebsd/f4d638b9-e6e5-4dbe-8c70-571dbc116174.html

and the port was updated:

https://www.freshports.org/ftp/curl

What's the lag from that port being updated and the build cluster building that for all supported FreeBSD branches?  As of now, I'm not seeing the above package being available, but I have no idea what the underlying process/schedule for building all this entails.


----------



## spork (Sep 11, 2018)

Anyone?  I kind of thought this would be Handbook material, but I'm having a really hard time tracking down any info on the pkg build cluster and what the timing of things looks like.

I found this, but it's kind of inscrutable, and I don't see a way to drill down into the successful build list (and then the version built): https://pkg-status.freebsd.org/

There's also this, but it appears to have been broken for the last year or so:  http://portsmon.freebsd.org/portoverview.py


----------



## xtaz (Sep 12, 2018)

I think the packages are rebuilt on an almost continuously running basis. When one package build ends another one begins. But how long it takes depends on how many ports have changed and thus which packages actually need rebuilding. Things like Chromium or Libreoffice can take a whole day to rebuild. The new packages will only show up in the repository when the entire package build has completed. So to answer your question, it could be hours, it could be days. In extreme circumstances it could be over a week.

The best way to see what's going on is to look at https://pkg-status.freebsd.org/ and scroll down to the Package Builds section. Find the jail that corresponds to your architecture and release. Note that the builders use the oldest supported release which means 11.1 and not 11.2. So for me that's 111amd64. Then pick either quarterly or default (latest) depending on what you use. Find this in the list.

If you then scroll right over to the right you see the package builder machine that builds this. For 111amd64 default that is beefy9. There is a little icon next to it that looks like a cannonball with a fuse. Click it. This takes you to http://beefy9.nyi.freebsd.org/ which will show you the current build that's going on. Click the build number. Then you see how far it's got and can work out how long is left to go.


----------



## rihad (Sep 17, 2019)

Releases fixing security vulnerabilities (pkg-audit(8)) should probably be put high on their lists.


----------



## rihad (Sep 17, 2019)

xtaz said:


> The new packages will only show up in the repository when the entire package build has completed.


Thanks for the detailed explanation. Why aren't successfully-built packages made available right away? It's been three days since ftp/curl got updated, and its builder beefy9 has got over 6000 more packages to go.


----------



## rigoletto@ (Sep 17, 2019)

First thing you have to look is in what repository you are. The default is quarterly which is updated err... quarterly, and then latest which is updated regularly.



rihad said:


> Why aren't successfully-built packages made available right away?



Because the repository needs to be recreated after the build process with all metadata of the packages in it.


----------



## rihad (Sep 17, 2019)

Yup, beefy9 is for building 11.2 default (latest) which I believe is the earliest supported minor release for 11.x. Which is cold comfort because of the considerable lag (


----------



## SirDice (Sep 17, 2019)

rihad said:


> Why aren't successfully-built packages made available right away?


Because the repository as a whole needs to be all the same or else you can get weird dependency issues half way through the builds.


----------



## spork (Sep 17, 2019)

rihad said:


> Releases fixing security vulnerabilities (pkg-audit(8)) should probably be put high on their lists.



I was kind of freaked out about the last mysql/mariadb vulnerability. I have nagios warning me when 'pkg audit' finds something amiss and mariadb 10.2 linger for over a month (July 22 to September 1) for a commit then it took a few more days for the package to appear, and even then I had to pull from current instead of quarterly.


----------



## pgauret (Sep 17, 2019)

Just realising how this works after investigating why ftp/curl isn't updating on my systems. In the case of a more serious vulnerability (think remotely exploitable), being left exposed for days is not ideal. 

Short of setting up a private build system for the packages one uses, how could this be mitigated ? Is this something already reognized as an issue and being worked on?


----------



## SirDice (Sep 18, 2019)

How would you suggest this could be sped up? Keep in mind that there are 34000+ packages to be built for a couple of different architectures. There's only a finite amount of packages you could build within a certain time frame.


----------



## pgauret (Sep 18, 2019)

Indeed not an easy task given *all* packages need to be rebuilt, not just those affected by a security issue. 

Based on what I learned here (btw thanks to all previous posters for explaining) I've set up a nightly ports-mgmt/synth build run for the handful of my packages exposed to the internet and their dependencies.

Oh and also I finally got around to send a donation to the FreeBSD Foundation. Funds for faster hardware probably can't hurt


----------



## Michał (Sep 19, 2019)

This security updates delay is the only pressing issue I'm having with FreeBSD, too bad this problem is not being worked on.


----------

