# Ports Tree Philosophy Upgrade Needed



## fossette (Oct 8, 2015)

The computer world is becoming more complex every day, and resources to take care of FreeBSD are finite. With that combination, the grab of Murphey's Law is just a release away…  And Oko's quote "_most FreeBSD developers use MAC_" doesn't help either.

The short version:
The FreeBSD Ports Tree should be able to handle older port releases for improved desktop stability.

The solution inspiration:
I had some issues with www/firefox, so I decided to upgrade it.  I upgraded it once before, so what's the big deal, right?  Well, this time, www/firefox requires a new version of multimedia/ffmpeg which fails to install due to missing doc files.  Wtf?  This multimedia/ffmpeg port mess also affect ports that require multimedia/ffmpeg like multimedia/vlc.  How useful is my FreeBSD desktop now?

Before committing to a new port release, the port must be thoroughly tested.  Perhaps that was easy in the good'ol days, but it's now wishful thinking.  Systems are infinitely more complex, and they have a huge variety of peripherals.  So many things to test, so little time.  Bugs/issues will eventually slip through, and the end-user ends up resuming the port's test phase unknowingly.

Here's a solution to minimize breaking users' FreeBSD systems.  Let older versions of a port coexist in the Ports Tree until no other port requires them.  For example, you want to upgrade multimedia/ffmpeg to 2.8? Fine!  Go for it!  But keep 2.7 as well until multimedia/vlc, www/firefox et al.,  have been expressively tested using the newest 2.8 version.  So, if multimedia/ffmpeg 2.8 has issues, other ports that are still based on the multimedia/ffmeg 2.7 will still be able to run.  THAT'S DESIRED STABILITY FOR USERS.

In order to have several versions of the same library in the same space, the version number must be part of the file name or the file path, just like tarballs are named.  Obviously, Makefile files will have to link to the specific library version that the port has been thoroughly tested with.  Besides, some ports of the Ports Tree are already version specific.  Why not extend this feature to all libraries?  That being said, there's no need to change the entire Ports Tree to implement this solution right away.  Ports can gradually be adjusted when a new version is ready to be committed in the Ports Tree.  VOILA!

Needless to say that the maintainers' sanity will be spared with this solution.  Possible damage will be limited to one port, and the few users that have decided to upgrade in the meantime.  That's less people affected by a bad port release, less stress for the maintainers, and 'relatively' more time for them to address the issue (because everybody got offline lives).  THAT'S A WIN-WIN SITUATION.

Looking forward for your input and suggestions!  Hopefully, the FreeBSD Developers will find this solution appropriate.

Dominique.


----------



## Peter2121 (Oct 8, 2015)

*fossette*, thanks a lot for putting attention on this problem.

I'm a user of FreeBSD desktop (PCBSD) since 9.1 version. A have the problem of software stability during all this time. When one port or package needs to be updated,  I must update almost ALL ports/packages, and very often this update repairs a problem of one software but introduces another problem in other software. The last example - I needed to update Firefox, I finished by updating all packages and now my LibreOffice does not work anymore - I need to update my OS from 10.1 to 10.2 to get it working... But there are some other software I'm using which does not work on 10.2!! So updating my OS I will have other stuff not working anymore. And so on... So the problem exists, it concerns (mainly) desktop users, and a solution is really needed.

As about possible solutions - I think that coexistence of several ports in the ports tree is a possible solution, but definitely NOT the best one. Another solution should be used, based on isolation of software.

At the time of PCBSD 9.1 it was using PBI system, where every software was coming with all necessary dependencies in one package. The packages were installed in separate folders, and symlinks were created in main tree for libs. Sure, such system was buggy and difficult to maintain (for example, any external plugin must be integrated at the moment of PBI creation). But globally, PBI system was able to solve the problem of coexistence of several versions of software, I could delete the actual version of software and reinstall the previous one. And it was working! Finally, PCBSD guys changed the PBI system and finished by using the 'classic' packages approach. So the initial realization of PBI is not available anymore in PCBSD.

I've searched a little bit for another solutions and it seems that two other systems exist actually - 0install and GNU Guix. So with the old PBI I've just explained there are three systems of software isolation that potentially can be used on FreeBSD. Some time ago 0install was even present in ports tree, I don't know why it was deleted.

Sorry for long post, but this problem is really important for me, I would like to see some answers of FreeBSD devs about the subject.


----------



## Deleted member 45312 (Oct 8, 2015)

Since I am building my packages with ports-mgmt/poudriere, updating my FreeBSD Desktop is far easier. ports-mgmt/poudriere consistently rebuilds ports with all the needed dependencies.
My packages repository is created at the end of process only if there hasn't been any errors during the built of all my ports.
I can set my own build options for all the ports I need, and upgrade is as easy as :
`# pkg upgrade`
All my FreeBSD workstation are using the same validated package repository which is shared with NFS (can also be done with http server).


----------



## Peter2121 (Oct 8, 2015)

*dlegrand*,
And what do you do if the users detect some problems with a package *after* the upgrade to the last version?


----------



## SirDice (Oct 8, 2015)

> January 2014 saw the release of the first quaterly branch, intended at providing a stable and high-quality ports tree. Those stable branches are a snapshot of the head ports tree taken every 3 months and currently supported for three months, during which they receive security fixes as well as build and runtime fixes.



https://lists.freebsd.org/pipermail/freebsd-ports-announce/2014-April/000079.html


----------



## Deleted member 45312 (Oct 8, 2015)

Peter2121 said:


> *dlegrand*,
> And what do you do if the users detect some problems with a package *after* the upgrade to the last version?


You can keep old release of packages repository for rollback.
This can be set in /usr/local/etc/poudriere.conf

```
# Keep older package repositories. This can be used to rollback a system
# or to bisect issues by changing the repository to one of the older
# versions and reinstalling everything with `pkg upgrade -f`
# ATOMIC_PACKAGE_REPOSITORY is required for this.
# Default: no
#KEEP_OLD_PACKAGES=no

# How many old package repositories to keep with KEEP_OLD_PACKAGES
# Default: 5
#KEEP_OLD_PACKAGES_COUNT=5
```


----------



## abishai (Oct 8, 2015)

Peter2121 said:


> *dlegrand*,
> And what do you do if the users detect some problems with a package *after* the upgrade to the last version?


Software like ports-mgmt/portmaster can save old packages as well. Also, you can ports-mgmt/portdowngrade broken port.


----------



## Peter2121 (Oct 8, 2015)

*SirDice*,
Using this 'stable' tree means that:
- one should not use packages
- one would stay on old versions of ALL ports
Once again, the problem is that on desktop system we often need to update one package to the last version and to downgrade another one to a previous one. Using ports on desktop system is hard, and the problem of coexistence of different versions of packages is NOT solved by using the 'stable' version of ports tree.

*dlegrand*,
As I understand, old packages are in old versions of repository. So again, we can downgrade ALL packages or upgrade ALL packages. This is not a feature I'm searching for.

*abishai*,
Sometimes, old packages can help. Sometimes they are in conflict with another packages (same dependencies of different versions). So it is not a good solution.
ports-mgmt/portdowngrade seems to be the best solution today, but there are two problems with it:
- difficult to use (one needs to search commits to revert)
- only ports can be used (no packages)
So in real life of desktop user ports-mgmt/portdowngrade is not an option...


----------



## fossette (Oct 8, 2015)

Peter2121 said:


> As about possible solutions - I think that coexistence of several ports in the ports tree is a possible solution, but definitely NOT the best one. Another solution should be used, based on isolation of software.


Isolation of software is definitely an interesting concept Peter, but I have no idea how this could be implemented/integrated easily in the existing FreeBSD structure.  Some kind of lite virtual machine or lite jail?

I believe that there's a Linux flavor which stores every dependent tools & libraries in a sub-directory of an application that requires them.  That's interesting too, but not very convenient if you want to keep the source code as well.

PS: Thanks to the kind soul who edited my port tags.  I used lots of them just to add a little color, but forgot that they also serve a purpose...  

Dominique.


----------



## formateur_fou (Oct 9, 2015)

Peter2121 said:


> Using this 'stable' tree means that:
> - one should not use packages
> - one would stay on old versions of ALL ports



In the quarterly branch there are packages available, even for security updates:
http://pkg.freebsd.org/freebsd:10:x86:64/quarterly/All/

This is true you will stay with "old" versions, but you will face the same situation with most Linux or BSD desktop systems. And, as this branch gets updated every 3 months, the packages are still pretty new.

Fossette, don't forget that one should not update packages blindly:
http://svnweb.freebsd.org/ports/head/UPDATING?view=markup

Maybe, you could have solved your problem by installing another browser instead of trying to update Firefox? But again, reading the link above is mandatory, especially if you're tracking the last packages branch.

I stopped using FreeBSD as a desktop because of the then packages rolling release model. Too many of them were broken for days or weeks. There was no quarterly branch at this time which the aim is to avoid this kind of problem and which became the default repository lastly.

I am curious to have any feedback about it.


----------



## SirDice (Oct 9, 2015)

formateur_fou said:


> This is true you will stay with "old" versions, but you will face the same situation with most Linux or BSD desktop systems.


Yep, like on RHEL/CentOS 7. It's using Perl 5.16. Which was deprecated more than a year before RHEL7 was released.


----------



## fossette (Oct 9, 2015)

formateur_fou said:


> I stopped using FreeBSD as a desktop because of the then packages rolling release model.


or


> I stopped using FreeBSD


is the fundamental problem to address.  Migrants fleeing to other OS.  I have another problem with my FreeBSD web browsers (a new forum post later), so I use firefox in my almost-1-year-old Windows XP VirtualBox guest.  How sick is that?

My ports handling suggestion is not perfect and requires discipline, but I believe it's good for the road, it's simple to implement, and it eases the pain of both users and maintainers.  It deserves some consideration at the next FreeBSD meeting.

Dominique.


----------



## cpm@ (Oct 9, 2015)

Maintainers try to do it best here. I don't have any problem to use FreeBSD as a desktop. Currently, I'm using GNOME 3 and it works almost flawlessly. So I encourage people to try FreeBSD as a desktop, we have a lot of DEs to try.

Long Live to FreeBSD


----------



## formateur_fou (Oct 9, 2015)

fossette said:


> Migrants fleeing to other OS


It was not my intention to discourage you.

I kept FreeBSD a long time on my desktop and I found it a very good system for my needs. I only had the feeling that sometimes, packages were not working as predictably as I would have wanted them to but that was not a big deal.

Over time, the new pkg tools made life much easier, and I guess the new repository will bring its own improvements too. This is why I am interested in getting feedback from people using it.


----------



## ANOKNUSA (Oct 11, 2015)

Peter2121 said:


> *SirDice*,
> Using this 'stable' tree means that:
> - one should not use packages
> - one would stay on old versions of ALL ports
> Once again, the problem is that on desktop system we often need to update one package to the last version and to downgrade another one to a previous one. Using ports on desktop system is hard, and the problem of coexistence of different versions of packages is NOT solved by using the 'stable' version of ports tree.




There's a quarterly repository available for packages. You just need to edit /etc/pkg/FreeBSD.conf and edit the repository URL to read "quarterly" rather than "latest." [1]
While a desktop user is likely to have more ports/packages installed than a server user, and so would certainly have more ports/packages to update, I have to wonder just how often you think these updates really need to take place. Is an absolute maximum of three months really too long to go without getting the latest version of a port? That still gets you the latest version of a package (though not necessarily a port) much sooner than OpenBSD gets it, not to mention just about any Linux distribution you can name. FreeBSD ports tend to be fresher than Gentoo packages; the Arch Linux devs have a much, much smaller number of packages to deal with, as do Exerbo and Alpine; CRUX will get you the latest version as soon as it comes out, in a manner of speaking: it has no official repository to speak of, as users make their own ports. In short, with the FreeBSD quarterly tree/repo, the inconvenience of waiting is negligible in my opinion.
The quarterly ports tree and package repository still gets security updates and critical bug fixes. That hasn't been overlooked.
Personally, I switched to the quarterly ports tree because I build lots of ports with custom options using ports-mgmt/poudriere, but don't want to have to rebuild lots of them (or larger ports like editors/libreoffice) more often than necessary. At the same time, I don"t want to have to deal with keeping track of ignored/locked ports/packages, and don't want to eat up disk space with lots of backup packages I may never, ever need. There's no need to frequently build the latest ports available, and frankly, there's no way I or anyone else is missing out on something absolutely vital by having to wait a few weeks to a month before getting the latest release of something. I know that, because that's what the vast majority of *nix users do anyway. (I've also gathered in my time here that most FreeBSD users tend to build ports when first installing a system, then update at some lengthy interval--say when a bug fix is released, or manual intervention is needed, or maybe on a monthly basis.)

[1]: Sorry, I'm not at my machine right now, and I can't recall the key for that line.


----------



## fossette (Oct 11, 2015)

*ANOKNUSA*, personally, I tend to adhere to the if-it-aint-broke-dont-fix-it philosophy.  My WindowsNT guest VM has *never* been upgraded, and I'm quite ok with that.  Lol!!!  I like stability!  An upgrade to one of the compilers? Not convinced it will have a significant impact on the ports already installed and running.  My Firefox 38.0 had issues, but I kept it 4 months nonetheless.  I don't have USB connection in VirtualBox, but I'll probably keep this 4.3.28 version until the next major release.  I can do without the USB devices for now.  I just don't want to risk it with upgrades, and spare me the wasted time should problems occur.

Dominique.


----------



## Peter2121 (Oct 12, 2015)

Guys,

Probably, there is a real problem of communication here. Look at the beginning of the thread. Two users (fossette and me) explained a problem of using FreeBSD as a desktop because of impossibility to install a previous version of software, different from the current one (present in ports/packages). We proposed to discuss a possible solution of this problem.

The ONLY post with a solution proposed was from abishai, who proposed to use ports-mgmt/portdowngrade. All others proposals are about using another repositories, another port trees or about update policy.

OK, guys, I'm really happy to get all these answers, but the question was definitely NOT about it! Using another repo/tree means synchro all ports/packages with the repo/tree, and the initial question was about the coexisting of different versions in the same repo/tree...


----------



## cpm@ (Oct 13, 2015)

Indeed, you can downgrade ports but it's dangerous due the old versions contain bugs or lack some implementations as well.

Are you completely sure that you need/want this? In this case, you can still use an old release and you can downgrade the ports tree to its corresponding version, download it from FreeBSD archive site. Of course, all this at your own risk


----------



## drhowarddrfine (Oct 14, 2015)

cpm said:


> I don't have any problem to use FreeBSD as a desktop. Currently, I'm using GNOME 3 and it works almost flawlessly. So I encourage people to try FreeBSD as a desktop, we have a lot of DEs to try.


Ditto.

I don't get it. Sure, I occasionally have a hiccup when I upgrade a port but, otherwise, I've been running a FreeBSD desktop for 11 years! What's the big deal? It's not hard. Everything works. And I work on my workstation possibly longer in a day than anyone else.

I just don't get it.


----------



## SirDice (Oct 14, 2015)

Just to get back at the issue, I would recommend building your ports using the aforementioned ports-mgmt/poudriere. If you configure poudriere to use SVN to update the ports tree it's fairly easy to downgrade a single port by simply reverting that specific port to a previous commit. If you then build everything again poudriere will make sure the dependencies stay intact. But, use at your own risk of course. Some ports are updated due to security issues and by keeping that port on a vulnerable version you are opening the door for abuse.


----------



## chrbr (Oct 14, 2015)

Peter2121 said:


> ports-mgmt/portdowngrade seems to be the best solution today, but there are two problems with it:
> - difficult to use (one needs to search commits to revert)
> - only ports can be used (no packages)
> So in real life of desktop user ports-mgmt/portdowngrade is not an option...


It is better than nothing and it works. I have used that in the past. Currently I follow a different solution. I have a second HDD in my PC for some stuff. From time to time I restore the latest backup to one partition and edit /etc/fstab. And of course I try to boot from that HDD. This should work and it gives the good feeling that recovery works - just in case. But additionally I have an old version of /usr/ports/ from the time of the last recovery. In the very seldom case that a port compiles well but does not work I simply copy the /usr/ports/whatever/it/is/ stuff from the backup HDD and re-compile that port.

This works with ports only but avoids to have to dig the correct SVN version. But basically this is just a side effect of the idea to verify the backups from time to time. Or it is an additional reason to do that.


----------



## protocelt (Oct 14, 2015)

I'd like to mention if there is a problem in an updated port, filing a problem report is the way to get these types of problems fixed faster. If you absolutely can't wait for the port to be fixed and ZFS is an option, another option you have is setting up your FreeBSD installation with ZFS to use boot environments. You can then just create a new boot environment, update your ports, and test. If there is a problem or regression, you can boot into the previous boot environment and delete the testing environment bringing you back where you started. Of course this assumes you are somewhat familiar with ZFS are or willing to become familiar with it. This is how I update when using portmaster(8) to update ports. When using ports-mgmt/poudriere as SirDice suggested, I run into much fewer issues with ports on other machines to begin with.


----------



## Peter2121 (Oct 16, 2015)

Ok, probably one more example would be nice here, just to help understanding the issue I often have with the current pkg system.
Today I need to add freerdp to my laptop. So `pkg install freerdp` should do it:

```
[root@pcbsd-peter] ~# pkg install freerdp
Updating pcbsd-major repository catalogue...
pcbsd-major repository is up-to-date.
All repositories are up-to-date.
The following 50 package(s) will be affected (of 0 checked):
Installed packages to be REMOVED:
  libreoffice-4.3.7
  mupdf-1.7,1
New packages to be INSTALLED:
  freerdp: 1.2.0_5
  jpeg-turbo: 1.4.1
  xf86-video-scfb: 0.0.4_2
Installed packages to be UPGRADED:
  xorg-server: 1.14.7_5,1 -> 1.14.7_6,1
  libressl: 2.2.1 -> 2.3.0
  kdelibs: 4.14.3 -> 4.14.3_1
  curl: 7.43.0_2 -> 7.44.0
  openldap-client: 2.4.41 -> 2.4.42_2
  cyrus-sasl: 2.1.26_9 -> 2.1.26_12
  sylpheed: 3.4.3 -> 3.4.3_1
  ruby: 2.0.0.645,1 -> 2.0.0.647,1
  efl: 1.13.2 -> 1.15.1
  libarchive: 3.1.2_2,1 -> 3.1.2_4,1
  openvpn: 2.3.7 -> 2.3.8
  libtorrent-rasterbar: 1.0.4 -> 1.0.6
  py27-libtorrent-rasterbar: 1.0.4 -> 1.0.6
  gnome-vfs: 2.24.4_3 -> 2.24.4_4
  git: 2.4.6 -> 2.6.0
  p5-Net-SSLeay: 1.70 -> 1.72
  libssh2: 1.4.3_5,2 -> 1.6.0_1,2
  unrar: 5.21_1,5 -> 5.30,5
Installed packages to be DOWNGRADED:
  stunnel: 5.19 -> 5.14
Installed packages to be REINSTALLED:
  python27-2.7.10
  ptlib-2.10.10_3
  pcbsd-base-1425064224 (direct dependency changed: xf86-video-scfb)
  heimdal-1.5.3_4 (needed shared library changed)
  py27-cryptography-0.8.2 (needed shared library changed)
  serf-1.3.8 (needed shared library changed)
  apr-1.5.2.1.5.4 (needed shared library changed)
  nginx-1.8.0_3,2 (options changed)
  wget-1.16.3 (needed shared library changed)
  opusfile-0.6_1 (needed shared library changed)
  py27-curl-7.19.5.1 (needed shared library changed)
  pulseaudio-6.0_2
  trousers-tddl-0.3.10_7 (needed shared library changed)
  virtualbox-ose-4.3.30 (needed shared library changed)
  qca-2.1.0_3 (needed shared library changed)
  libvncserver-0.9.9_11 (direct dependency changed: jpeg-turbo)
  libevent2-2.0.22_1
  tor-0.2.6.9 (needed shared library changed)
  clamav-0.98.7 (needed shared library changed)
  postgresql93-client-9.3.9 (needed shared library changed)
  gkrellm2-2.3.5_6 (needed shared library changed)
  librtmp-2.4.20130923 (needed shared library changed)
  x11vnc-0.9.13_2 (direct dependency changed: libXrender)
  rdesktop-1.8.3 (needed shared library changed)
  socat-1.7.3.0_2 (needed shared library changed)
  p5-Crypt-OpenSSL-Random-0.10 (needed shared library changed)

The operation will free 399 MiB.
139 MiB to be downloaded.
```

Do I want to remove libreoffice-4.3.7?? NO!!!
Do I want to update xorg-server?? NO!!!
I just want to install freerdp because I need it NOW!
But the current pkg system does not allow me to install the freerdp package without updating some important parts of my system. This is a hell, I don't want to do all the updates now, I have no time, I've just asked to install freerdp!


----------



## drhowarddrfine (Oct 16, 2015)

Well, you're supposed to keep your system up to date for these reasons. Some updates are required for security and compatibility. It's like complaining about running FreeBSD 5.0 and not being able to install something until you update to version 10.2. That's not the fault of the ports tree (the subject of this matter, not packages but I haven't re-read through this whole thing).


----------



## Peter2121 (Oct 16, 2015)

*drhowarddrfine*,
My system is almost up-to-date, I've updated all packages three or four month ago approximately.
Please, explain me why must I update or remove LibreOffice (BTW, the last version does not work correctly on 10.1) when I just need to install freerdp? Why does my Mac OSX not ask me the things like this? Why does my Windows not ask me neither?


----------



## hukadan (Oct 16, 2015)

Peter2121 said:


> Please, explain me why must I update or remove LibreOffice


For the removing part, it might be related to this bug report. If it is the case, it should be taken care of quickly I suppose.



Peter2121 said:


> Some time ago 0install was even present in ports tree, I don't know why it was delete


Yes that's true : devel/zeroinstall-injector. And the reason it was remove is no more valid if we consider PBI abandoned. If you feel the need of it, you could make a port of the software. At the moment, it seems that there is a problem with FreeBSD. But as you can see, a fix is available.

For a more general perspective, I never had any issue while updating my ports(7) but I update everything not some software only. As far as I know, the ports has to be taken as a whole and I have hard time to find a simple solution to your problem where you want to upgrade some software and not some others (I am not the more qualified though).

Hopping you will find what you seek with 0install.


----------



## ANOKNUSA (Oct 17, 2015)

Peter2121 said:


> Do I want to remove libreoffice-4.3.7?? NO!!!
> Do I want to update xorg-server?? NO!!!
> I just want to install freerdp because I need it NOW!
> But the current pkg system does not allow me to install the freerdp package without updating some important parts of my system. This is a hell, I don't want to do all the updates now, I have no time, I've just asked to install freerdp!



This is because the _main package repository_ is built from the HEAD ports tree. It's a rolling-release distribution system--update one thing, and you need to update everything else along with it. *There is not and cannot be any alternative.* Updating a rolling-release system piece-by-piece is absolutely guaranteed to break something eventually--not one rolling-release Unix-like system I know of supports this, with the exception of NixOS, which is built from the ground up specifically for this one feature. The only alternatives FreeBSD offers you are to build from ports (since you have to manually update the ports tree) or use the quarterly package repository. That's it.[1] I don't want to speak for the developers, but it seems to me the chances of this changing any time in the foreseeable future is slight, since pkg(8) is inherently dependent on the structure of the ports system. If this is too inconvenient for you, that's fair enough, but FreeBSD may not be for you.



Peter2121 said:


> Please, explain me why must I update or remove LibreOffice (BTW, the last version does not work correctly on 10.1) when I just need to install freerdp??



Shared dependencies. Package _a_ depends on version _1_ of package _x_, while package _b_ requires version _2_ of package _x_.



Peter2121 said:


> Why does my Mac OSX not ask me the things like this?



If you used MacPorts or Homebrew, it would.



Peter2121 said:


> Why does my Windows not ask me neither?



Why doesn't my electric car take diesel?

[1]: This isn't strictly true: as a third alternative, you could also use `pkg lock <package_name>` to prevent a package and its dependencies from being upgraded, but this raises the maintenance burden rather than making things more convenient.


----------



## fossette (Oct 17, 2015)

*Peter2121, I think we need another FreeBSD** fork** again.  ;-)*


----------



## Beastie7 (Oct 17, 2015)

fossette said:


> *Peter2121, I think we need another FreeBSD fork again.  ;-)*



I think you should fork it, or quit the complaining and help out with PC-BSD development. That way the Project can continue to thrive without any more desktop related doom and gloom non-sense, complaints, and FUD.

It's amazing how some people (mostly Linux refugees) think FreeBSDs future sustainability is dependent on whether or not it's a viable desktop.


----------



## kpa (Oct 17, 2015)

We are still stuck with the ancient ports system that should have been rewritten ages ago. It has few major flaws. Fixed dependencies that prevent ports depending on abstract entities such as "any perl version 5.16 or higher". The other one is that it's not possible to mix parts of the ports tree with the older contents of ports tree, essentially downgrading to earlier versions easily. There is no proper API in /usr/ports/Mk/* that would enforce compatiblity forwards or backwards, the only guaranteed way that anything works is that the all of the tree is from same point in time. The one other thing that the portmgr@ people have expressed dissatisfaction is the format used to express the ports themselves, Makefiles are very bad for expressing declarations because of the arcane parsing rules and other wierdness, there's also the problem that Makefiles allow port maintainers to write make(1) or sh(1) code in their Makefiles that nobody else understands when they try to fix the port. Something like UCL (that is used in pkg as the configuration file format) would be much better for the task.


----------



## Peter2121 (Oct 17, 2015)

ANOKNUSA said:


> Updating a rolling-release system piece-by-piece is absolutely guaranteed to break something eventually--not one rolling-release Unix-like system I know of supports this, with the exception of NixOS, which is built from the ground up specifically for this one feature.


Why does the approach of NixOS cannot be used in FreeBSD software management?


----------



## kpa (Oct 17, 2015)

Peter2121 said:


> Why does the approach of NixOS cannot be used in FreeBSD software management?



Patches accepted.


----------



## drhowarddrfine (Oct 17, 2015)

Peter2121 said:


> *drhowarddrfine*,
> My system is almost up-to-date, I've updated all packages three or four month ago approximately.


My system gets updated daily. You forget that packages and ports, such as LibreOffice, are written by third-parties and not FreeBSD. Any complaints about updates should be brought up with those third parties, not FreeBSD which only passes them on to you.


> Why does my Mac OSX not ask me the things like this? Why does my Windows not ask me neither?


They aren't as up to date. Internet Explorer only updates when they feel like it, years apart, while Firefox and Chrome updated on a six-week schedule. IE is the worst browser on the planet while Firefox and Chrome are light years ahead. And no one is complaining about that.


----------



## ANOKNUSA (Oct 17, 2015)

Peter2121 said:


> Why does the approach of NixOS cannot be used in FreeBSD software management?





ANOKNUSA said:


> ...NixOS ... is built from the ground up specifically for this one feature.



To really emphasize this: the NixOS Linux distribution was created for one purpose only--as an experiment for the founder's PhD thesis. The entire OS is built around a package management system that absolutely nobody else uses, and maintaining the distribution means having a group of people dedicated to building and maintaining several versions of every single package available, while requiring users to frequently tweak and pick at various configurations and individual parts of the system (including the "base system") to maintain system integrity. NixOS has been around for roughly a decade, and virtually nobody has heard of it. It isn't used in any critical/professional setting--while I don't doubt that it's possible to use NixOS as a personal production system, and that some of its dedicated users do so, it quite simply has always been a novelty.

I've got no problem with the ports and package management systems undergoing some refinement, and I'd bet no-one else really does either. But it would have to be something that is both manageable and arguably and demonstrably better than what already exists. I think there's one thing that really, _really_ needs to be stated here: there are indeed other Unix-like systems (namely, Linux distributions such as Gentoo, Arch, NixOS, and CRUX) that allow users to install and individually update multiple versions of a single port/package. But these package management systems require a greater degree of hands-on system maintenance. They do not make anything more convenient, and _they absolutely do not make anything more "stable"_. They increase system complexity, making things less convenient and (at least arguably) less stable. There's a weird trade-off involved, whereby you don't merely always have access to the latest packages, but must in fact frequently update to avoid problems. Update too frequently, and you risk running into previously undiscovered bugs (since you're among the first users to install the new package); don't update frequently enough, and one large update later on is more likely to break your installation. As a former Arch Linux user, I can say that:

if every time you install a new package in Arch, you're expected to completely update every other package simultaneously;
if you break your Arch system by updating individual packages you won't get any sympathy or help from the community; and 

the common wisdom in the Arch community is that if you go more than six weeks or so without completely updating every package on your system, it's recommended to just perform a clean install.
The fact that FreeBSD has managed to deftly avoid this kind of conundrum and find something in the middle--access to the latest software, but only when you want it--is the primary reason I switched to it from Arch. Unfortunately, this means either tracking one package repository and waiting for version bumps, or building everything from source. Personally, I can live with either.


----------



## fossette (Oct 17, 2015)

Peter2121 said:


> Why does the approach of NixOS cannot be used in FreeBSD software management?





kpa said:


> Patches accepted.


LOL!!!

I like the Nix approach.  At this time, my FreeBSD system is ALMOST perfect.  It's just that when some upgrade is needed, anxiety goes to the roof.  With my FreeBSD experience so far, it's well within reason.  I'd love to help contributing more to FreeBSD, but the learning curve is overwelming.  I only have a few small success stories so far, all documented in this forum.

I must admit that I'm very tempted by the NixOS solution.  I'll try it first in a VM if it works for me.  And install the QVWM WM that I'm curently using on FreeBSD.

Dominique.


----------



## kpa (Oct 17, 2015)

fossette said:


> LOL!!!



I'm not kidding much. That's how everything got started. You want something in ports, you do the legwork yourself and prepare port and PR. You want an update to a port, same thing. The basic workflow hasn't changed a bit.


----------



## cpm@ (Oct 18, 2015)

drhowarddrfine said:


> Ditto.
> 
> I don't get it. Sure, I occasionally have a hiccup when I upgrade a port but, otherwise, I've been running a FreeBSD desktop for 11 years! What's the big deal? It's not hard. Everything works. And I work on my workstation possibly longer in a day than anyone else.
> 
> I just don't get it.



Sorry, if I was not clear enough. I meant that FreeBSD is good as desktop and people should try it before speak so quickly. We have a good OS that cover our necessities and that's the most important for me besides the great FreeBSD Community.

Hope that you can see now my viewpoint


----------



## Peter2121 (Oct 19, 2015)

drhowarddrfine said:


> Any complaints about updates should be brought up with those third parties, not FreeBSD which only passes them on to you.


No problems with software updates, the problem (once again) is that I must install ALL updates or NOTHING.



drhowarddrfine said:


> IE is the worst browser on the planet while Firefox and Chrome are light years ahead. And no one is complaining about that.


Few people are using IE now, even on Windows. But Windows let me choose - what version of Chrome or Firefox could I install. It does not force me to update Chrome when I want to update Firefox.



ANOKNUSA said:


> maintaining the distribution means having a group of people dedicated to building and maintaining several versions of every single package available


So the small team of unknown distrib can accomplish this task, too complex and impossible for FreeBSD community? 



ANOKNUSA said:


> there are indeed other Unix-like systems (namely, Linux distributions such as Gentoo, Arch, NixOS, and CRUX) that allow users to install and individually update multiple versions of a single port/package. But these package management systems require a greater degree of hands-on system maintenance. They do not make anything more convenient, and _they absolutely do not make anything more "stable"_.


I don't think that the approach of Arch or Gentoo is great. That's why I'm using FreeBSD desktop 
Probably, the best path is in the middle - define a "base FreeBSD desktop system", larger than simply a FreeBSD base system. Inside this base system one need to update all or nothing (like today). And additionally we need a way to install applications with all their dependencies like in NixOS, but it should mainly concern large desktop applications with tons of dependencies (like LibreOffice).


----------



## drhowarddrfine (Oct 19, 2015)

Peter2121 Then you must not understand how these things work. As pointed out earlier, some updates are necessary for software to run. If you don't update the dependencies, then the interfaces won't work or the security is bad or any number of other reasons.



> Few people are using IE now, even on Windows.


I'm a web developer and that's not true at all but I don't want to go off on that tangent.

FreeBSD does NOT force you to update Chrome, or ANY program, any more than Windows does. Nor does Windows let you choose. In your case, it's the software you are trying to install that's telling you that you must update or the software won't run cause due to incompatibilities! Windows and Linux are the same way. You have to keep your dependencies, drivers, etc. up to date or they won't run!



> Probably, the best path is in the middle - define a "base FreeBSD desktop system"


Then you are stepping outside the bounds of what FreeBSD's goals and purpose. If you want that, that's what PC-BSD is for.



> Inside this base system one need to update all or nothing


But that's what you're here complaining about.



> And additionally we need a way to install applications with all their dependencies like in NixOS


FreeBSD DOES install all the dependencies. How NixOS does it, I don't know, but then someone would come along and complain we need to do it "just like XXX ... cause theirs is better".


----------



## sidetone (Oct 25, 2015)

FreeBSD doesn't [necessarily] need to be forked. What can be forked is the ports system to something that is more efficient. Something like NetBSD's ports. I tried that OS, but couldn't continue with it, because of its lack of wireless network compatibility.

I will learn over the next few years, and contribute more than documentation.


----------



## junovitch@ (Oct 26, 2015)

sidetone said:


> ...
> I will *learn* over the next few years, and *contribute* more than documentation.



I highlighted the words I like to see.  More folks throwing in quality assistance is what we need and not forks that will stagnate.  Start learning.  Dig into a problem that concerns you and contribute back a quality fix one issue at a time.


----------



## jrm@ (Oct 26, 2015)

Peter2121 said:


> ```
> [root@pcbsd-peter] ~# pkg install freerdp
> Updating pcbsd-major repository catalogue...
> pcbsd-major repository is up-to-date.
> ...





Peter2121 said:


> Do I want to remove libreoffice-4.3.7?? NO!!!


I think we can agree this isn't ideal, but removing a package without a replacement shouldn't happen often.  What I guess occurred is that when editors/libreoffice and common dependencies between editors/libreoffice and net/freerdp were built in PC-BSD's  ports-mgmt/poudriere run, a common dependency of editors/libreoffice and print/mupdf failed and you happened to update before the problem was fixed and the new, missing packages became available.   As others have explained, for the most part, package dependencies are version specific.  It's difficult to allow more flexibility.

Maybe this situation could be prevented if an updated repository didn't go live whenever any dependent ports fail in the net-mgmt/poudriere run.


----------



## sidetone (Oct 26, 2015)

I'll point out that something in ports is redundant in a useless way and unnecessary. (For some situations redundancy is good.) Afterwards, coincidentally or not it seems to be fixed. Then soon, things go back to being bulky, or broken, but at least features were added because that ability was added for what the developer probably dreamed of the program being.

Only by using FreeBSD did I realize that Linuxism based dependencies are a mess. Tons of not needed features have to get compiled to get something a feature to work, such as 1 file from a library, or a port that re-installs other software, whereas the package already has what's needed to do it is already in the install process. It's like a car having a second gasoline engine and transmission mounted on top of the hood just to run the windshield wipers, and another one to adjust the electric mirrors. It makes me wonder if people don't understand what the existing code does, but keep adding on to it. I don't either, but from a non-programmer's view I believe that there's enough evidence of a lot of redundant but unused code installed. Running into compile choices that lead to circular dependencies is another kind of evidence of inefficiency and redundancy. Packages would reduce install time, but still the packages become vulnerable to broken installs, stalling and crashing, because of their unnecessary complexity. Don't get me wrong, Linux capabilities and FreeBSD together are great.

I don't want to promote many forks, I only think 1 fork would do. A NetBSD package compatibility layer would also do, but that is probably outside of FreeBSD's interests. People have different ideas of what ports should be. I think there should be efficient code, and Linuxisms can be used on top of it to add functionality, but not for the sake of none other than being a Linuxism which brings in vast amounts of unused code. I only suggested a ports fork, because people don't want to abandon it, which is their right. That will require fixing the GNU/Linux community too, which is outside of BSD interests. Maybe my theory about dependency complexity is not completely accurate.


----------



## kpa (Oct 26, 2015)

sidetone said:


> Only by using FreeBSD did I realize that Linuxism based dependencies are a mess.



You're incorrect here. The dependency model used by FreeBSD packages was not invented on nor copied from Linux but is very much homegrown out the technical features/limitations of the ELF file format, ld.so(1) style dynamic linking of shared libraries and the fact that no one could envision that someone would want to use binary packages from multiple sources. The dependencies are now handled essentially the same way as they were in the very first versions of the old pkg_* tools, they are just translated to SQLITE database entries.


----------



## sidetone (Oct 26, 2015)

They are a mess. It's not just the libraries, it's the ports that packages are built off of. Compare how smooth the base system of any BSD with little else runs to most Linux distributions. I've tried compiling on Linux systems in the past, and the option choices go on forever. I had to study the choices and write it down taking hours at a time to do just that. It fits their reasons to have many options: different priorities for different distributions. FreeBSD shows that this can be avoided, then run better.

People port programs the only way they know how, but without replacing Linux dependencies with what exists in the base install or minimal installations of BSD.



kpa said:


> style dynamic linking of shared libraries and the fact that no one could envision that someone would want to use binary packages from multiple sources.



Isn't that a FreeBSD ports/package problem that is very similar to Linux problems?


----------



## kpa (Oct 26, 2015)

sidetone said:


> They are a mess. It's not just the libraries, it's the ports that packages are built off of. Compare how smooth the base system of any BSD with little else runs to most Linux distributions. I've tried compiling on Linux systems in the past, and the option choices go on forever. I had to study the choices and write it down taking hours at a time to do just that. It fits their reasons to have many options: different priorities for different distributions. FreeBSD shows that this can be avoided, then run better.
> 
> People port programs the only way they know how, but without replacing Linux dependencies with what exists in the base install or minimal installations of BSD.
> 
> ...



It's the same problem and both sides have come up with various attempts to solve it independently but none of them are really satisfactory. So far the only way to guarantee smooth updates is to force a single source for the binary packages that are built from a source that known to be good. The latter condition "known to be good" is where FreeBSD ports stumbles quite often because it's rolling release with commits coming in at unpredictable times from various committers.


----------



## sidetone (Oct 27, 2015)

That happens too. One example of it being a rolling release that breaks certain features, is when installing an application that runs solely on one toolkit like qt4. That's expected, and normal.

One example of complicated redundancies is: Installing a program with gtk3, then some added features in the options goes back and pulls in gtk2. For that one program compile, I just want either gtk2 or gtk3, while I can have both on my system. I will admit that, gtk2 has been cleaned up somewhat.

Other times, it will be a qt4 dependent program. Usually it will work fine, but a newer feature will be available that a qt4 solution doesn't exist for that feature. It will pull in gtk, AND it will install tons of codecs, hardware access programs, when all but few of these functionalities are already included in the qt4 _install_ queue.

It will need 1 audio codec, and it will install an additional set of every type of codec it can find. I'll want a solution from 1 tool, not 2 or more.

In short, 1 toolkit solution can pull in toolkit solutions that don't function with that program, except 1 (or few) minor dependency. 3 other duplicate bundles aren't needed, when they aren't meant to work with that toolkit dependency solution.

Another odd problem, is when gtk dependencies get pulled in for a non-graphical program install. It doesn't happen as often, but it does happen.

The packages built this way slow down the computer a lot, and cause a lot of potential for breakage.

This one, I don't expect anyone to fix, because I figure this is how they like doing it. When installing gtk3, it will pull in Jadetex, or texmf, for documentation/manpage translation from Gnu/Linux. It's ridiculous, that the whole compile has to stop to download 1 gb, at a very slow speed _at times 4kb/s, not representative of my internet connection_. Installing the package doesn't always bypass that inconvenience. I typically disable this on my own when I can. It's not manpages that I need to get that program working, but just for bloat. There was a bsd _alternate ports/package_ solution for it, but it's port description says to use something better.

* edit
It may be necessary to install another solution from another toolkit, but it should be limited to that feature/codec. Not repeat an alternate bundle of codecs for what's not used/needed for that program.


----------



## sidetone (Oct 27, 2015)

This is me just dreaming, so don't take this one personally. I'd like to see a low-level purely BSD solution for libraries, dependencies, graphics and applications.

For instance purely BSD solution of Wayland, Xaw3d, Fluxbox, i3 metaports that are ready to go, and eventually to be installed by one command from the base-system of FreeBSD through ports/packages. BSD codecs too. Then other open-source solutions, including xorg, can build on top of that.


----------



## wblock@ (Oct 27, 2015)

sidetone said:


> I will learn over the next few years, and contribute more than documentation.



More is better, but this makes it sound like documentation is a lesser part.  A lot of people think that, but the applications and operating systems that succeed often do it on the strength of their documentation.


----------



## sidetone (Oct 27, 2015)

Documentation is important. I'm limited by what I don't know to document or otherwise contribute.



wblock@ said:


> operating systems that succeed often do it on the strength of their documentation.


 That's NetBSD's philosophy. Their documentation is neat, and their ports system is efficient.


----------



## fossette (Oct 30, 2015)

Here's another idea!  How about adding a dependencies build flag option to ports?  By default, even if not specified, the build option is active and the build process behaves as it always worked.  If specified, for example:

```
# cd /usr/ports/x11-fm/xfe
# make dependencies=false install clean
```
would attempt to build xfe without compiling its dependencies.  Obviously, it's not the safest option.  But if it works, good!  You may save lots of building time, and prevent risks of poisoning already installed ports.  With this option, you have the ability to temporarily ignore minor changes to X or Perl if you wish so.  It's a *Use at your own risk* option.

If it doesn't work, you simply revert to the good old:
`make install clean`
as it's always been done before.

Dominique.


----------



## kpa (Oct 30, 2015)

fossette said:


> Here's another idea!  How about adding a dependencies build flag option to ports?  By default, even if not specified, the build option is active and the build process behaves as it always worked.  If specified, for example:
> `cd  /usr/ports/x11-fm/xfe
> make dependencies=false install clean`
> would attempt to build xfe without compiling its dependencies.  Obviously, it's not the safest option.  But if it works, good!  You may save lots of building time, and prevent risks of poisoning already installed ports.  With this option, you have the ability to temporarily ignore minor changes to X or Perl if you wish so.  It's a *Use at your own risk* option.
> ...



What would this accomplish? When you're building from a completely clean state you absolutely must build all the dependencies first or your build will fail. You can't play tricks with the dependencies and start guessing if they are already installed or not. When you build a package of a port the end result can not be different depending on if a dependency was already installed or if it had to be built first.


----------



## uzsolt (Nov 3, 2015)

fossette said:


> But if it works, good!


If it works you should report a bug - the port isn't okay if has a false dependency.


----------



## sidetone (Nov 6, 2015)

fossette said:


> would attempt to build xfe without compiling its dependencies.  Obviously, it's not the safest option.  But if it works, good!  You may save lots of building time, and prevent risks of poisoning already installed ports.  With this option, you have the ability to temporarily ignore minor changes to X or Perl if you wish so.  It's a *Use at your own risk* option.



You would have to test a port's build on your own computer/computer's jail from a clean port's tree, or know of a valid replacement for a port to then suggest it. Then it moves on to developer's territory, usually shell scripting and C programming language which is another thread and forum.


----------



## initd (Nov 18, 2015)

Not sure if I can still add anything useful to this post. Hope you won't blame for arriving late. 
I think the endemic problem of dependency hell will somehow probe to be a limiting factor for growth in the *NIX world. Linux is also ill with this epidemic situation.

I think we need to retake the PBI model. FreeBSD shines because base system and ports are separated, so there are no reasons to not try again, and this time with success. I am more than happy to help to rearchitect the ports system based on nPBI (new PBI) model, if somebody has the patience to sit with me and explain the roadblocks we faced with PBI.


----------



## sidetone (Nov 18, 2015)

FreeBSD's port's dependency problem has made major improvements this year.

One thing that can be done, is make ports depend primarily depend on BSD solutions like audio/oss or sqlite. If there is a feature these lack, let it pull down a version of a program such as audio/alsa, that is usually trimmed to only include the features that are missing, such as from OSS or a database. Doing this dependency-wise would be most practical. Doing more, would require major changes to sourcecode, and that would be incompatible with how the ports are downloaded, or it would take the compilation process knowing what to omit to achieve this.

For a while I was eager to see sysutils/hal replaced with devd(8) in the ports tree as the default option, except for newly ported programs, but they seem to coexist together without conflicting. These two programs coexisting seem ok.


----------

