# Lost in library land



## PacketMan (Nov 13, 2015)

So, during my rebuild of all ports on my GNOME3 machine, (after going from 10.1-RELEASE to 10.2-RELEASE) I made a mistake. I can't remember if it was portmaster or pkg but I did a 'remove' only to realize I was at a 'parent' folder, and I think it was a library folder.  Good way to learn right?   Thank goodness this is a home machine and not a machine running the national bank.

After rebuilding all the ports practically most of the very many missing library files were put back, at least that's what `pkg_libchk` tells me.  But a few remain missing stubbornly.


```
apache-openoffice-4.1.2: /usr/local/openoffice-4.1.2/openoffice4/program/libofficebean.so misses libjawt.so
lwjgl-2.9.1: /usr/local/lib/lwjgl2.9.1/liblwjgl64.so misses libjawt.so
plexhometheater-1.4.1_1: /usr/local/lib/plexhometheater/system/players/dvdplayer/avcodec-53-x86_64-freebsd.so misses libavutil.so.51
plexhometheater-1.4.1_1: /usr/local/lib/plexhometheater/system/players/dvdplayer/avfilter-2-x86_64-freebsd.so misses libavcodec.so.53
plexhometheater-1.4.1_1: /usr/local/lib/plexhometheater/system/players/dvdplayer/avfilter-2-x86_64-freebsd.so misses libavformat.so.53
plexhometheater-1.4.1_1: /usr/local/lib/plexhometheater/system/players/dvdplayer/avfilter-2-x86_64-freebsd.so misses libswresample.so.0
plexhometheater-1.4.1_1: /usr/local/lib/plexhometheater/system/players/dvdplayer/avfilter-2-x86_64-freebsd.so misses libswscale.so.2
plexhometheater-1.4.1_1: /usr/local/lib/plexhometheater/system/players/dvdplayer/avfilter-2-x86_64-freebsd.so misses libavutil.so.51
plexhometheater-1.4.1_1: /usr/local/lib/plexhometheater/system/players/dvdplayer/avformat-53-x86_64-freebsd.so misses libavcodec.so.53
plexhometheater-1.4.1_1: /usr/local/lib/plexhometheater/system/players/dvdplayer/avformat-53-x86_64-freebsd.so misses libavutil.so.51
plexhometheater-1.4.1_1: /usr/local/lib/plexhometheater/system/players/dvdplayer/postproc-52-x86_64-freebsd.so misses libavutil.so.51
plexhometheater-1.4.1_1: /usr/local/lib/plexhometheater/system/players/dvdplayer/swresample-0-x86_64-freebsd.so misses libavutil.so.51
plexhometheater-1.4.1_1: /usr/local/lib/plexhometheater/system/players/dvdplayer/swscale-2-x86_64-freebsd.so misses libavutil.so.51
```

I have reinstalled games/lwjgl and multimedia/plexhometheater a couple times to no avail.

But it gets worse, GNONE3 desktop doesn't boot up.  I'll try to find the messages for you but I did see some ELF messages and a whole bunch of /usr/src/lib path messages I do believe.

I have failed you my masters, and come to you with humble heavy heart.  Cast me out to the wild animals if ye will but I will come back.      I could simply reinstall the whole thing from scratch but I refuse to do so yet. I want to learn and this has gotta be a darn good way.  So, in general what should I be doing now? And also, where can I read some good documentation on the libraries?


----------



## wblock@ (Nov 13, 2015)

The output shows that these libraries are needed, but not what provides them.  libjawt.so is easy, that is Java.  Either the JRE or JDK, java/openjdk7-jre or java/openjdk7.  libavutil.so is from multimedia/ffmpeg.  Install those, then re-run `pkg_libchk`.


----------



## PacketMan (Nov 13, 2015)

I will try again but I'm pretty sure I have reinstalled multimedia/ffmpeg already but will try the others.  Where do I read and learn that stuff you just said? Surely its documented somewhere; some kind of library map?

Thanks again wblock@.


----------



## wblock@ (Nov 13, 2015)

Finding out which packages provide which libraries can be challenging.  I just used `locate libavutil.so` and then `pkg which`, but that does not help if it is not installed.  find(1) can be used to grep through all the port pkg-plist files.  However, some plists are dynamically generated, so it might not find the library filename.


----------



## PacketMan (Nov 13, 2015)

wblock@ said:


> Finding out which packages provide which libraries can be challenging.



So its not documented? There no set of web pages dedicated to detailing the library system, and which port/pkg installs what? 
So I will have to simply learn as I go and lean on awesome people like you from time to time?


----------



## PacketMan (Nov 13, 2015)

wblock@ said:


> The output shows that these libraries are needed, but not what provides them.  libjawt.so is easy, that is Java.  Either the JRE or JDK, java/openjdk7-jre or java/openjdk7.  libavutil.so is from multimedia/ffmpeg.  Install those, then re-run `pkg_libchk`.



Did all three, no difference.


----------



## wblock@ (Nov 13, 2015)

PacketMan said:


> So its not documented? There no set of web pages dedicated to detailing the library system, and which port/pkg installs what?


Not really, no.  There is no clear way to do this.  It only worked for me because those libraries were already installed on my system.

You might need to rebuild the ports that need those libraries.


----------



## PacketMan (Nov 15, 2015)

I've decided to kick off another complete rebuild, i,e `portmaster -a -f` with some other flags. So far so good its still building away. I'll let you know the outcome of that.  Thanks again wblock@.

So lets say hypothetically I were to start a company and build products using FreeBSD OS. How would we know which libraries are for which purpose, and in some cases should we use libraryABC.a.2 instead of libraryABC.b.1 for example?  Maybe you just hire experienced folks?


----------



## wblock@ (Nov 15, 2015)

Libraries are normally pulled in as dependencies by the package system.  So if some application needs a library, whatever provides that library will be installed.


----------



## PacketMan (Nov 17, 2015)

Well after days at it, this still remains:


```
apache-openoffice-4.1.2: /usr/local/openoffice-4.1.2/openoffice4/program/libofficebean.so misses libjawt.so
lwjgl-2.9.1: /usr/local/lib/lwjgl2.9.1/liblwjgl64.so misses libjawt.so
plexhometheater-1.4.1_1: /usr/local/lib/plexhometheater/system/players/dvdplayer/avcodec-53-x86_64-freebsd.so misses libavutil.so.51
plexhometheater-1.4.1_1: /usr/local/lib/plexhometheater/system/players/dvdplayer/avfilter-2-x86_64-freebsd.so misses libavcodec.so.53
plexhometheater-1.4.1_1: /usr/local/lib/plexhometheater/system/players/dvdplayer/avfilter-2-x86_64-freebsd.so misses libavformat.so.53
plexhometheater-1.4.1_1: /usr/local/lib/plexhometheater/system/players/dvdplayer/avfilter-2-x86_64-freebsd.so misses libswresample.so.0
plexhometheater-1.4.1_1: /usr/local/lib/plexhometheater/system/players/dvdplayer/avfilter-2-x86_64-freebsd.so misses libswscale.so.2
plexhometheater-1.4.1_1: /usr/local/lib/plexhometheater/system/players/dvdplayer/avfilter-2-x86_64-freebsd.so misses libavutil.so.51
plexhometheater-1.4.1_1: /usr/local/lib/plexhometheater/system/players/dvdplayer/avformat-53-x86_64-freebsd.so misses libavcodec.so.53
plexhometheater-1.4.1_1: /usr/local/lib/plexhometheater/system/players/dvdplayer/avformat-53-x86_64-freebsd.so misses libavutil.so.51
plexhometheater-1.4.1_1: /usr/local/lib/plexhometheater/system/players/dvdplayer/postproc-52-x86_64-freebsd.so misses libavutil.so.51
plexhometheater-1.4.1_1: /usr/local/lib/plexhometheater/system/players/dvdplayer/swresample-0-x86_64-freebsd.so misses libavutil.so.51
plexhometheater-1.4.1_1: /usr/local/lib/plexhometheater/system/players/dvdplayer/swscale-2-x86_64-freebsd.so misses libavutil.so.51
```

I have removed and reinstalled the three ports listed above multiple times, as well as x11/gnome3 and then did multimedia/plexhometheater again after the GNOME3 reinstall, ensured my ports tree is latest-greatest every day I start, and I did a second `portmaster -a -f`. When I deinstall I use `make deinstall clean`, and when I reinstall I use `make install clean`.  Portmaster says all is up to date, but command `pkg_libchk` burps back with the above.  I am pretty sure I have removed and reinstalled multimedia/ffmpeg and multimedia/ffmpeg0, and the JAVA ports listed earlier. Note my saying "removed and reinstalled". I am not rebuilding something already install, though I have tried that too. I am removing and then installing using ports.

Is there some sort of command option that basically says when you install this port, reinstall all libraries and their dependencies, even if all looks to be in order?  Should I consider using `-r` with portmaster to rebuild up the tree instead of just rebuilding down the tree?


----------



## wblock@ (Nov 17, 2015)

Every time you do `portmaster -af`, it (slowly) rebuilds everything.  So there is no point to adding other options there.  Doing any other sort of update before that is just a waste of time that will be wiped out when those ports are rebuilt.  Having benchmarked -af versus the update procedure shown in Thread rebuilding-all-ports-with-portmaster.51210, I don't use -af at all.

One possibility is that the files shown here are outdated leftovers from some earlier mistake, and not actually needed or used by current ports.  What does `pkg which /usr/local/lib/plexhometheater/system/players/dvdplayer/avcodec-53-x86_64-freebsd.so` say?

Rather than continue to fight with that, I would consider deinstalling multimedia/plexhometheater, manually deleting that library directory, then installing the port again.


----------



## kpa (Nov 17, 2015)

I would also look at /usr/local/lib and see if there are any old versions of libavutil.so* (and other missing libraries) present and manually delete them if found. The ports system tries its best to avoid a situation where a port accidentally links to an older version of a library but it's really on the mercy of the upstream vendor to get the compilation and linking right.


----------



## wblock@ (Nov 18, 2015)

Oh, and maybe delete /usr/local/lib/compat/pkg, where portmaster(8) puts all the outdated libraries.


----------



## PacketMan (Nov 18, 2015)

wblock@ said:


> Every time you do `portmaster -af`, it (slowly) rebuilds everything.  So there is no point to adding other options there.  Doing any other sort of update before that is just a waste of time that will be wiped out when those ports are rebuilt.  Having benchmarked -af versus the update procedure shown in Thread rebuilding-all-ports-with-portmaster.51210, I don't use -af at all.
> 
> One possibility is that the files shown here are outdated leftovers from some earlier mistake, and not actually needed or used by current ports.  What does `pkg which /usr/local/lib/plexhometheater/system/players/dvdplayer/avcodec-53-x86_64-freebsd.so` say?
> 
> Rather than continue to fight with that, I would consider deinstalling multimedia/plexhometheater, manually deleting that library directory, then installing the port again.



`pkg which /usr/local/lib/plexhometheater/system/players/dvdplayer/avcodec-53-x86_64-freebsd.so`
/usr/local/lib/plexhometheater/system/players/dvdplayer/avcodec-53-x86_64-freebsd.so was installed by package plexhometheater-1.4.1_1

So just to be clear you mean for me to `cd /usr/local/lib` and then `rm -fr /plexhometheater` ?



wblock@ said:


> Oh, and maybe delete /usr/local/lib/compat/pkg, where portmaster(8) puts all the outdated libraries.


 My /compat is already empty, does not contain anything.


----------



## wblock@ (Nov 18, 2015)

PacketMan said:


> So just to be clear you mean for me to  cd /usr/local/lib and then  rm -fr /plexhometheater ?


One step is enough: `rm -rf /usr/local/lib/plexhometheater`.


----------



## PacketMan (Nov 18, 2015)

wblock@ said:


> One step is enough: `rm -rf /usr/local/lib/plexhometheater`.



yeah yeah I know that, sorry I worded that poorly. I meant to say you want me to delete the actual folder /plexhometheater and not just the files inside leaving said folder intact. 

Stay tuned.


----------



## PacketMan (Nov 19, 2015)

The folder /usr/local/lib/plexhometheater is automatically deleted when I do the deinstall the port.

At /usr/local/lib If I tab out a file name I see something like this, and then a long listing shows me the linking:

```
$ ls -l libavutil.
libavutil.a  libavutil.so  libavutil.so.54  libavutil.so.54.31.100
$ ls -l libavutil.so
lrwxr-xr-x  1 root  wheel  22 Nov 15 21:15 libavutil.so -> libavutil.so.54.31.100
```
The needed library file(s) is not there.


----------



## PacketMan (Nov 21, 2015)

Well I've been still searching and reading and no joy. Something is not adding up.  I am now using ldd() to see what multimedia/plexhometheater wants to use. This port net/avahi-app got me curious, but regretfully deinstall/reinstall made no difference. I even did a `make rmconfig-recursive` followed by `make config-recursive` on another reinstall of Plex Home Theatre.  I am really starting to think there is something not right with this port. But I am likely wrong and will keep looking.


----------



## wblock@ (Nov 21, 2015)

Beats me.  A problem with the port, or possibly something weird that `pkg_libchk` does not correctly interpret.  Does multimedia/plexhometheater work?


----------



## protocelt (Nov 21, 2015)

I think the multimedia/plexhometheater port may need to be updated. It is likely built against a specific version of multimedia/ffmpeg and the multimedia/ffmpeg port has since been updated which would explain the missing libraries. PacketMan, what is the output of `ldd /usr/local/bin/ffmpeg` on your box?


----------



## PacketMan (Nov 21, 2015)

wblock@ said:


> Beats me.  A problem with the port, or possibly something weird that `pkg_libchk` does not correctly interpret.  Does multimedia/plexhometheater work?



Yep seems to work fine.  I hadn't booted GNOME3 and PLEX in a while as I was focused on trying to resolve these library issues. Playing a movie for the kids right now. But maybe there is a piece of functionality that is broke that I didn't discover yet.



protocelt said:


> I think the multimedia/plexhometheater port may need to be updated. It is likely built against a specific version of multimedia/ffmpeg and the multimedia/ffmpeg port has since been updated which would explain the missing libraries. PacketMan, what is the output of `ldd /usr/local/bin/ffmpeg` on your box?




```
libavdevice.so.56 => /usr/local/lib/libavdevice.so.56 (0x800850000)
   libavfilter.so.5 => /usr/local/lib/libavfilter.so.5 (0x800a58000)
   libavformat.so.56 => /usr/local/lib/libavformat.so.56 (0x800dc6000)
   libavcodec.so.56 => /usr/local/lib/libavcodec.so.56 (0x801200000)
   libavresample.so.2 => /usr/local/lib/libavresample.so.2 (0x80251d000)
   libpostproc.so.53 => /usr/local/lib/libpostproc.so.53 (0x80273b000)
   libswresample.so.1 => /usr/local/lib/libswresample.so.1 (0x80295a000)
   libswscale.so.3 => /usr/local/lib/libswscale.so.3 (0x802b75000)
   libavutil.so.54 => /usr/local/lib/libavutil.so.54 (0x802e07000)
   libX11.so.6 => /usr/local/lib/libX11.so.6 (0x80306c000)
   libm.so.5 => /lib/libm.so.5 (0x8033a5000)
   libthr.so.3 => /lib/libthr.so.3 (0x8035ce000)
   libvdpau.so.1 => /usr/local/lib/libvdpau.so.1 (0x8037f2000)
   libc.so.7 => /lib/libc.so.7 (0x8039f5000)
   libopencv_core.so.2 => /usr/local/lib/libopencv_core.so.2 (0x803e00000)
   libopencv_imgproc.so.2 => /usr/local/lib/libopencv_imgproc.so.2 (0x80422e000)
   libfreetype.so.6 => /usr/local/lib/libfreetype.so.6 (0x8046bf000)
   libfontconfig.so.1 => /usr/local/lib/libfontconfig.so.1 (0x80495b000)
   libgmp.so.10 => /usr/local/lib/libgmp.so.10 (0x804b9b000)
   libgnutls.so.28 => /usr/local/lib/libgnutls.so.28 (0x804e11000)
   libbz2.so.4 => /usr/lib/libbz2.so.4 (0x805136000)
   libz.so.6 => /lib/libz.so.6 (0x805348000)
   libxvidcore.so.4 => /usr/local/lib/libxvidcore.so.4 (0x80555e000)
   libx265.so.59 => /usr/local/lib/libx265.so.59 (0x805a00000)
   libx264.so.144 => /usr/local/lib/libx264.so.144 (0x805f1d000)
   libvpx.so.3 => /usr/local/lib/libvpx.so.3 (0x80627f000)
   libvorbisenc.so.2 => /usr/local/lib/libvorbisenc.so.2 (0x806614000)
   libvorbis.so.0 => /usr/local/lib/libvorbis.so.0 (0x8068b6000)
   libtheoraenc.so.1 => /usr/local/lib/libtheoraenc.so.1 (0x806ae3000)
   libtheoradec.so.1 => /usr/local/lib/libtheoradec.so.1 (0x806d10000)
   libschroedinger-1.0.so.0 => /usr/local/lib/libschroedinger-1.0.so.0 (0x806f20000)
   liblzma.so.5 => /usr/lib/liblzma.so.5 (0x8071d8000)
   libxcb.so.1 => /usr/local/lib/libxcb.so.1 (0x8073fd000)
   librpcsvc.so.5 => /usr/lib/librpcsvc.so.5 (0x80761c000)
   libXext.so.6 => /usr/local/lib/libXext.so.6 (0x807825000)
   libc++.so.1 => /usr/lib/libc++.so.1 (0x807a36000)
   libcxxrt.so.1 => /lib/libcxxrt.so.1 (0x807cf5000)
   libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x807f11000)
   libexpat.so.1 => /usr/local/lib/libexpat.so.1 (0x80811f000)
   libp11-kit.so.0 => /usr/local/lib/libp11-kit.so.0 (0x808345000)
   libtspi.so.1 => /usr/local/lib/libtspi.so.1 (0x8085a1000)
   libtasn1.so.6 => /usr/local/lib/libtasn1.so.6 (0x80880c000)
   libnettle.so.4 => /usr/local/lib/libnettle.so.4 (0x808a1e000)
   libhogweed.so.2 => /usr/local/lib/libhogweed.so.2 (0x808c4c000)
   libintl.so.8 => /usr/local/lib/libintl.so.8 (0x808e7a000)
   librt.so.1 => /usr/lib/librt.so.1 (0x809085000)
   libogg.so.0 => /usr/local/lib/libogg.so.0 (0x80928b000)
   liborc-0.4.so.0 => /usr/local/lib/liborc-0.4.so.0 (0x809491000)
   libXau.so.6 => /usr/local/lib/libXau.so.6 (0x809723000)
   libpthread-stubs.so.0 => /usr/local/lib/libpthread-stubs.so.0 (0x809925000)
   libXdmcp.so.6 => /usr/local/lib/libXdmcp.so.6 (0x809b26000)
   libffi.so.6 => /usr/local/lib/libffi.so.6 (0x809d2b000)
   libcrypto.so.7 => /lib/libcrypto.so.7 (0x809f32000)
```

Looks to me that multimedia/ffmpeg is newer, it has replaced older libraries that multimedia/plexhometheater is still looking for. Normally older ones wouldn't get deleted if a program depended on them right? I'm guessing my screwup earlier (where I think I deleted a bunch of library stuff) did this. Guessing that is.


----------



## protocelt (Nov 21, 2015)

I just installed multimedia/plexhometheater since I've been curious about the port anyway. On my system, after installation, I get no output when running `pkg check` and `pkg_libchk` so I don't think there is an issue with the port.

I know you've reinstalled a lot of ports already, but maybe running `# portmaster -r plexhometheater` may clear things up.


----------



## PacketMan (Nov 22, 2015)

You need multimedia/plexmediaserver up and running in order to use multimedia/plexhometheater.  I am running `portmaster -r plexhometheater` now. But what is interesting I have another box that I have done zero ports updating on. After doing an OS update and then a `portmaster -a -f` I get the same missing libraries messages.

I'll let you know the outcome but didn't my previous `portmaster -a -f` rebuild *every* port, so how is using -r option (by itself) different?


----------



## protocelt (Nov 22, 2015)

PacketMan said:


> You need multimedia/plexmediaserver up and running in order to use multimedia/plexhometheater.


I'm aware. I've been using multimedia/plexmediaserver using a Playstation 3 as the client. 


PacketMan said:


> I am running  portmaster -r plexhometheater now. But what is interesting I have another box that I have done zero ports updating on. After doing an OS update and then a  `# portmaster -a -f` I get the same missing libraries messages.
> 
> I'll let you know the outcome but didn't my previous  `# portmaster -a -f` rebuild *every* port, so how is using -r option (by itself) different?


It should have, but for whatever reason, that isn't the case here. I don't know why. Running `# portmaster -r plexhometheater` will just rebuild multimedia/plexhometheater and everything it depends on so at worst it just wastes a few minutes of time and at best, it fixes the missing libraries.


----------



## PacketMan (Nov 22, 2015)

Nothing fixed. Same missing libraries. Same result on a 2nd box but I made sure my ports tree was up to date, and I got that result before I went to 10.2-RELEASE.  I'm going to ignore it for a few days to give my mind a rest.  I'm tempted to cave in, and just below away the complete system and reinstall the whole darn thing from scratch. *Shrug* you know the way the other OS folks do it. Or I just might find the missing hole and get it plugged.

Thanks for all your help everyone.


----------



## protocelt (Nov 23, 2015)

One other thing you could try is running `# pkg check -Ba` to regenerate all library data for pkg(8). It is odd pkg(8) is listing missing libraries even though the port works.


----------



## kpa (Nov 23, 2015)

protocelt said:


> One other thing you could try is running `# pkg check -Ba` to regenerate all library data for pkg(8). It is odd pkg(8) is listing missing libraries even though the port works.



It's the same problem as discussed a while ago. FreeBSD ports and pkg seems to insist that shared libraries have an SONAME elf tag in their headers in order of them to be recognized as shared libraries. The tag is completely optional though and the programs link and run just fine without it. OpenBSD for example doesn't make any such requirement for shared libraries.


----------



## protocelt (Nov 23, 2015)

Ah, that's right. The reply on the mailing list to this was "Fix the port(s)".

It seems counterproductive to me for pkg(8) to require a SONAME tag when the tag is optional. Especially when many port maintainers are already overloaded with work. I just don't see every port with no tag getting fixed until there are more maintainers to spread the load so unless I'm not getting the whole picture, users will continue to run into this type of issue in the future.


----------



## PacketMan (Nov 24, 2015)

protocelt said:


> One other thing you could try is running `# pkg check -Ba` to regenerate all library data for pkg(8). It is odd pkg(8) is listing missing libraries even though the port works.



No change. When it was running it made no mention of the Plex libraries. And when I rerun `pkg_libchk` I get the same results.



protocelt said:


> .....users will continue to run into this type of issue in the future.



So I'm not half the screw-up I thought I was? This is someone elses doing?  So if this is because a tag is missing, shouldn't you too also have the same issue?


----------



## protocelt (Nov 24, 2015)

PacketMan said:


> So if this is because a tag is missing, shouldn't you too also have the same issue?


I don't know. I didn't update the port though. It was a fresh install. You may have something else going on related to upgrading as well but I don't know what it is if that's the case. If you really have no problems with the port, personally I would just ignore it for now.


----------



## PacketMan (Nov 25, 2015)

protocelt said:


> If you really have no problems with the port, personally I would just ignore it for now.



Righto, thanks. I learned a lot (mostly my breaking stuff), and got a lot of confidence by fixing most if not all of what I broke. Or at least I think I fixed it.   I'll keep scratching at it once in a while, and if I figure it out I'll let you all know.

Thanks again everyone.


----------

