# Portmaster: skip already built ports & check everything first



## Caliante (Oct 16, 2010)

Hi all,

After my debacle of upgrading FreeBSD from 8.0 -> 8.1 I found out I have around a zillion ports that needed upgrading. There is this extremely kind person (an understatement, and this person even refuses the beer I am offering as a way of saying 'thank you':beer ) that advised me to use portmaster for it, more specifically the last section of the portmaster man: delete all ports and have them reinstalled all.

This proves to be a (cu)cumbersome e) path; it seems Mr. Murphy decided to come into my box and demonstrate his law. So, for the last 7 days, I have started portmaster every night before going to bed (portmaster -D `cat ~/installed-port-list`)(see man) only to find it to have halted every morning with yet another error. The extremely kind person has helped me solve each error, yet right now it is starting to get annoying; not only for me, but I could suspect for him also. However, just inserting the 8.1 DVD is the easy way and you don't learn from it, so I'll try to sit it out 'till the end.

I have two questions:

1. What surprises me is that on every new run I start at night I see portmaster starting to rebuild all the ports that it had built already the night before. I was suspecting portmaster -R to be avoiding this, but this seems not to be the case. The man isn't too clear about this: on the last page it says you need to use -R, on the first pages it is not too clear what -R is supposed to do.

Would anybody know what the correct syntax is?

2. I thought it better to have it run through everything first without making/compiling (as to see upfront what will go wrong later so I can fix that first), and thought portmaster -n would do this, but this also isn't doing it.

Again, would anybody know what the correct syntax is?

Thank you in advance,

Bye,


----------



## Crivens (Oct 16, 2010)

Hello Caliante,

the -R option is, if I get the manpage right, only for updates and not for new installations.
So all ports you list on the command line will be build. However, I may also be missing something here.

Usually I split these new installs in smaller steps, like, first download all needed files (-F), then run the config for all ports (--force-config). Only then i start the build process. Little is more annoying than to look at the progress in the morning and to see it stopped with some "unable to fetch" error or a config screen _right_ when you turned around to go to bed. When the build terminates, i mark all ports which were built with a marker character in the text file and use a "grep -v" in the pipeline to skip these in later runs.


----------



## jb_fvwm2 (Oct 16, 2010)

I'd try something like: 
	
	



```
cd /usr/ports/audio/yell && make install
```
then

```
portmaster -d -B /var/db/pkg/this-# /var/db/pkg/this-# /var/db/pkg/that-# /var/db/pkg/this-# && yell
```
 (maybe?? do the largest ones first)... when using portmaster.  But I do not try to update a lot of ports at once. Why not just the ones you use then the remainder really slowly...


----------



## Caliante (Oct 16, 2010)

Thanks to the both of you :e

Since the list of ports to update was huge, I was doing a full reinstall, which according to the portmaster man should work. Like this:



> Using portmaster to do a complete reinstallation of all your ports:
> 1. portmaster --list-origins > ~/installed-port-list
> 2. Update your ports tree
> 3. portmaster --clean-distfiles-all
> ...



Going through the ports one by one according to the above text shouldn't be necessary. And the process does work to a certain extend. According to pkg_info > 560 ports have been installed. It is just that on every error the process halts, and on fixing that error and restarting the `cat`-list (see above) the process starts rebuilding everything from port # 1, although that has been built already.


----------



## Caliante (Oct 16, 2010)

jb_fvwm2 said:
			
		

> I'd try something like:
> 
> 
> 
> ...



Why would you want to clean the distfiles in a multi-port upgrade?


----------



## Caliante (Oct 16, 2010)

Crivens said:
			
		

> Hello Caliante,
> 
> the -R option is, if I get the manpage right, only for updates and not for new installations.



Could you perhaps point out where you've read that? Because all I am seeing is the explanation at the bottom, in the big example I am following, where -R is supposed to skip already installed (by portmaster) ports. Then again, the man page isn't consistent in this aspect as far as I my small brain can comprehense :e



			
				Crivens said:
			
		

> So all ports you list on the command line will be build. However, I may also be missing something here.
> 
> Usually I split these new installs in smaller steps, like, first download all needed files (-F), then run the config for all ports (--force-config). Only then i start the build process. Little is more annoying than to look at the progress in the morning and to see it stopped with some "unable to fetch" error or a config screen _right_ when you turned around to go to bed. When the build terminates, i mark all ports which were built with a marker character in the text file and use a "grep -v" in the pipeline to skip these in later runs.



The -F is a usefull option, thanks for the tip 

From my experience, using the example from the man page that I am following all the config happens upfront.

Isn't marking ports in a textfile manually a huge job when, like in my case, it concerns more than 560 ports (the ones currently already installed)? From my simple noob-perception this is FreeBSD, this is about automation; it should be possible to do it without having to mark 560 lines on a paper manually?


----------



## jb_fvwm2 (Oct 17, 2010)

The -d option only cleans older versions of  distfiles afaik. The most recent one used is kept for each port. /edit/ Howsoever, it probably cleans all distfiles from dependencies or maybe even globbed-similar ports, that were or were not or both rebuilt, (I suspect an unintended consequence of the coding somewhere).  /end edit/


----------



## wblock@ (Oct 17, 2010)

jb_fvwm2 said:
			
		

> ```
> portmaster -d -B /var/db/pkg/this-# /var/db/pkg/this-# /var/db/pkg/that-# /var/db/pkg/this-# && yell
> ```



/var/db/pkg is not needed, just use the name of the port, or the origin (category/portname):

`# portmaster firefox ghostscript editors/nano`


----------



## jb_fvwm2 (Oct 17, 2010)

Correct, however in the past if I run without the "/var/dg/pkg/" prefixed, it will (would) (did) (sometimes without asking) rebuild multiple ports: "di" can become dillo, diffuse, dirdiff... quicker sometimes to prepend /var/db/pkg/ than lookup the port category.


----------



## Crivens (Oct 18, 2010)

Caliante said:
			
		

> Could you perhaps point out where you've read that?


Sorry, currently I am faced with something that is not FreeBSD (you may guess as to what OS it is), so this has to be answered later. If I remember correctly, I only found the -R option mentioned when updates were the topic, but not when reinstalls were mentioned. From that I cleverly deducted that the option has no sense in new installs. 



> The -F is a usefull option, thanks for the tip


You are welcome.



> Isn't marking ports in a textfile manually a huge job when, like in my case, it concerns more than 560 ports (the ones currently already installed)?


You do not need to tag or remember all ports you have. When listing the ports with portmaster, you can see some of these are leaf ports, root ports, ...
The ports which have dependencies upon them, no matter if they depend on something else, then you do not need them in that text file. Installing the XFCE main port will pull in all the dependencies so there is no need to list them again. These dependencies may also vary, so you might end up installing ports which were used in the past, but due to changes is the application or different config settings, you no longer would need them.

My laptop has some 700+ installed packages but the file I track all things before I start the build has only a fraction of these. As long as there is a backup of that list I can reinstall all ports pretty quick.


----------



## dougb@ (Oct 31, 2010)

*The portmaster -R option*



			
				Crivens said:
			
		

> Hello Caliante,
> 
> the -R option is, if I get the manpage right, only for updates and not for new installations.



Quoting from the man page:


> -R  used with the -r or -f options to skip ports updated on a previous run.



I fail to understand how that could be hard to understand.


----------



## dougb@ (Oct 31, 2010)

Caliante said:
			
		

> Hi all,
> 
> After my debacle of upgrading FreeBSD from 8.0 -> 8.1 I found out I have around a zillion ports that needed upgrading. There is this extremely kind person (an understatement, and this person even refuses the beer I am offering as a way of saying 'thank you':beer ) that advised me to use portmaster for it, more specifically the last section of the portmaster man: delete all ports and have them reinstalled all.



This process is only needed when you are doing a major upgrade of the FreeBSD version or architecture. Such as 8.x -> 9.x, or i386 -> amd64. You would have been better off with [cmd=]portmaster -a[/cmd]



			
				Caliante said:
			
		

> This proves to be a (cu)cumbersome e) path; it seems Mr. Murphy decided to come into my box and demonstrate his law. So, for the last 7 days, I have started portmaster every night before going to bed (portmaster -D `cat ~/installed-port-list`)(see man) only to find it to have halted every morning with yet another error. The extremely kind person has helped me solve each error, yet right now it is starting to get annoying; not only for me, but I could suspect for him also. However, just inserting the 8.1 DVD is the easy way and you don't learn from it, so I'll try to sit it out 'till the end.
> 
> I have two questions:
> 
> 1. What surprises me is that on every new run I start at night I see portmaster starting to rebuild all the ports that it had built already the night before.



I don't know why that's surprising, it's doing exactly what you told it to do.   Portmaster has no way to know why you want to do what you're telling it to do, it just does it. Therefore, you have to be careful to tell it to do only what you actually want it to do. If there are ports on your list that you've already reinstalled and you don't wish to re-re-install them, remove them from the list.  


hth,

Doug


----------



## Crivens (Nov 1, 2010)

dougb@ said:
			
		

> Quoting from the man page:
> 
> 
> I fail to understand how that could be hard to understand.



I have learnt to be careful with documentation. Since it is pretty clear what these lines say, they still do not tell you when/if exceptions apply. Or if they simply are not connected with reality at all.  
So without trying it and confirming it myself I have only the man page to rely on - and those have lied to me often enough. But I must say the man pages of FreeBSD are right almost all the time, keeping out the "user experience" with other documentations (work related) is the hard part...


----------

