# FreeBSD package mirror infrastructure



## redstreams (Dec 17, 2021)

Is there a particular reason why FreeBSD has only few package mirrors? For major Linux distros there are dozens of mirrors, same for OpenBSD. It looks like in the last few years a few mirrors were added. 
I did not test it but when I tried it last time mirroring was rather slow and I think no mirror provides rsync or some way to check for new packages.


----------



## rafael_grether (Dec 17, 2021)

I don't see need to increase the amount of mirrors. On Linux you can have dozens of mirrors, but there are also so many more hundreds of users.
See: http://pkg.freebsd.org
The pkg mirror is switched based on your GeoIP Location. You can force it to use another mirror.
See /etc/pkg/FreeBSD.conf

But at your risk, no guarantees of any particular mirror.

I didn't understand your last question about check for new packages. You dont need rsync for this. "pkg update && pkg upgrade" don't satisfies you?


----------



## redstreams (Dec 17, 2021)

The background of the question was to create a local mirror for packages. Having more official mirrors would reduce the load on the current ones.
For mirroring all packages something like rsync would be useful but the current mirrors don't support this. iirc you can't use wget either because the mirror does not support timestamping. Also overall performance of the mirrors seemed to be slow last time I tested it. You can't even browse the package list anymore.


----------



## SirDice (Dec 17, 2021)

FreeBSD Quarterly Status Report 3rd Quarter 2021
					

FreeBSD is an operating system used to power modern servers, desktops, and embedded platforms.




					www.freebsd.org


----------



## redstreams (Dec 18, 2021)

Mirroring FreeBSD
					

The all in one guide for mirroring the FreeBSD website, FTP servers, and more




					docs.freebsd.org
				





> Due to very high requirements of bandwidth, storage and adminstration the FreeBSD Project has decided not to allow public mirrors of packages


Wouldn't using more mirrors actually reduce bandwidth requirements? In particular if other mirrors are synced preferably instead of normal FreeBSD end users?
Storage requirements are about 1TB which seems rather cheap. I don't know about the administration overhead but I *think* for some mirror operator it is mostly running something like rsync to the master mirrors every few hours. Looking at just a few examples like https://mirror.yandex.ru https://ftp.halifax.rwth-aachen.de https://mirror.bytemark.co.uk/ they store everything and the kitchen sink including FreeBSD (minus packages).


----------



## grahamperrin@ (Dec 20, 2021)

Parallel to rootbert https://forums.freebsd.org/profile-posts/3507/



> Due to very high requirements of bandwidth, storage and adminstration



_Blame_ can take us to a point in time <https://github.com/freebsd/freebsd-...ion/content/en/articles/hubs/_index.adoc#L165> preceding which it might be unreasonably difficult to discover *when, or why, the phrase was added*.

Not specific to packages, the note at the head of <https://docs.freebsd.org/en/articles/hubs/> might be more relevant. With added emphasis:



> We are not accepting new mirrors _at this time_.



Why not? I'm not a member of the team, but I guess that its *work in progress* <https://www.freebsd.org/status/report-2021-07-2021-09/#_cluster_administration_team> is enough for members to *not* want attention diverted to anything else _at this time_. 

Glancing at some progress points, another guess: 

the fewer the mirrors, the easier it will be to coordinate when things change.



redstreams said:


> The background of the question was to create a local mirror for packages. …











						How to make a local mirror for FreeBSD.
					

I need a local mirror server, because I need an FreeBSD cluster (run ZFS and PostgreSQL).  For Debian system, the local mirror can easily be created with the apt-mirror tool.  Is there some tools to create a local mirror for FreeBSD to run freebsd-update and pkg commands, so the entire cluster...




					forums.freebsd.org
				






fender0107401 said:


> I will find a way to work around it.



Performance



redstreams said:


> … overall performance of the mirrors seemed to be slow last time I tested 𣀦…











						Slow package fetching
					

Hello, I've trying to install some packages with pkg install, but the process is too slow today. How can I change the source? I've installed fastest_pkg from ports. But this util did not respond after start? Or Should I wait for it to finish?




					forums.freebsd.org
				




For anyone who's not already aware of this test script: 

ports-mgmt/fastest_pkg

– kudos to anyone who's willing to draft a pull request for this enhancement: 

If /usr/local/bin/uclcmd exists, consider using uclcmd to append to FreeBSD.conf · Issue #3 · ehaupt/fastest_pkg
Two related FreeBSD bugs (ERs): 

260517 – ports-mgmt/fastest_pkg exemplify latest only if latest is already preferred
260518 – ports-mgmt/fastest_pkg consider using a pkg+http:// or pkg+https:// (etc.) URL scheme


----------

