# Port dependencies as packages rather than building more ports recursively



## kpedersen (Nov 25, 2010)

Hello,

Is there a feature like OpenBSD's FETCH_PACKAGES (in /etc/Mk.conf) which instead of recursively compiling ports dependencies, instead it simply adds the respective packages?

On the local network, we have a mirror of the FreeBSD packages and would like to use this rather than wasting bandwidth downloading countless distfiles.

So when I type `make` in openoffice.org-3, instead of compiling shedloads of software, it just pkg_add's the required packages from the local mirror and then compiles OpenOffice?

Any advice would be greatly appreciated 

Karsten


----------



## SirDice (Nov 25, 2010)

Why don't you just use pkg_add(1)? Set up one server as a "Build" server. Build all your needed packages there. Export that filesystem and on other machines just mount the remote filesystem, cd to it and run pkg_add from there.

Also have a look at sysutils/bsdadminscripts.


----------



## kpedersen (Nov 25, 2010)

Hello thank you for the suggestion,

However, seeing as I already have the majority of packages, I just don't really want to have to build them all again especially since our internet is slow and the distfiles will take a long time to download.

I will have a tinker with bsdadminscripts, but it looks as though I may have to knock up my own system to do exactly what I need. I just wanted to make sure I wasn't duplicating something that is already available.

Since there doesn't seem to be anything similar, I can post the script here if anyone is interested in the same venture.

Best Regards,


----------



## SirDice (Nov 25, 2010)

kpedersen said:
			
		

> However, seeing as I already have the majority of packages, I just don't really want to have to build them all again especially since our internet is slow and the distfiles will take a long time to download.


And you don't have to. If that server contains all the packages you need simply (nfs) export /usr/ports/packages/ or even the full /usr/ports/ directory and mount it on all your other servers. 

That's how I do it. I have one server with a jail that does all my building. Everything gets packaged during this process and I use those packages to install all my other machines.

Build once, install many :e


----------



## wblock@ (Nov 25, 2010)

Something like

```
#!/bin/sh
for portname in `make missing`; do
  pkgname=`make -C /usr/ports/$portname -V PKGNAME`
  pkg_add $pkgname
done
```

But it seems like reinventing the -P option from portinstall(1) (ports-mgmt/portupgrade-devel).


----------



## DutchDaemon (Nov 25, 2010)

... and portmaster(8) (*-P* and *-PP*).


----------



## kpedersen (Nov 25, 2010)

SirDice said:
			
		

> And you don't have to. If that server contains all the packages you need simply (nfs) export /usr/ports/packages/ or even the full /usr/ports/ directory and mount it on all your other servers.



So if I put all my packages in /usr/ports/packages/All and then on that same machine go into the openoffice.org-3 port directory and type *make* I find that it starts trying to fetch the gmake distfile... Why? When I already have gmake-3.81_4.tbz in my /usr/ports/packages/All folder? Why can it not just install and use that?

It is going to do this for a hell of a lot of packages, which can easily be avoided by just getting it to install the packages waiting in /usr/ports/packages/All



			
				wblock said:
			
		

> Something like
> 
> ```
> #!/bin/sh
> ...



Something like this is ideal, however using *make missing* to list the packages I need will cause me to have a lot of unneeded packages (ones that would have been used to build dependencies, which I am not doing (I just want to install them))
Is there something like *make missing* but...

1) lists build dependencies of the port
2) lists run dependencies of the port
3) lists *only* run dependencies of the port's dependencies recursively

*make missing* in the openoffice.org-3 port directory gives a massive list but only because it is listing the stuff to build everything (as opposed to just the stuff required to build the port itself)

I will have a look at portmaster,

Thanks for the suggestions so far guys!


----------



## SirDice (Nov 25, 2010)

kpedersen said:
			
		

> So if I put all my packages in /usr/ports/packages/All and then on that same machine go into the openoffice.org-3 port directory and type `make` I find that it starts trying to fetch the gmake distfile... Why?


Because that's what ports do 

In any case, that's not what I meant. What I meant was to take a clean installation (or a clean jail) and build openoffice there. Use *make package-recursive* (or use any of the port tools) so it will create packages for all it's dependencies too. So you're downloading a distfile and building a port just once. After that's done you mount your exported /usr/ports/packages on each machine that needs it and do a `# pkg_add openoffice.org-3.tbz` to install it.

Even once in a while I pre-build everything, something like this:

```
# cd /usr/ports/ports-mgmt/portmaster
# make package clean
# rehash
# cd ../../
# portmaster -dg x11/xorg-minimal
# portmaster -dg x11/gnome2-lite
# portmaster -dg multimedia/mplayer
etc. etc.
```

Obviously this will take a while but once it's done you'll have a neat package tree with everything you need. You have built it once so the distfiles are also only downloaded just once. The resulting packages in /usr/ports/packages can be saved to CD, exported with NFS or FTP or whatever you want.

For any new machine you just do a basic install, set that up and install your saved packages. Hey, presto, an up to date machine without having to download anything from the internet. And you can do this a zillion times. Just as long as you only install from your own packages you won't have any dependency issues or version differences either because all the packages you've made were done on a clean system.


----------



## ProFTP (Nov 25, 2010)

portupgrade:

*port upgrade:*


```
man portupgrade
```


```
portupgrade -f
```


```
portupgrade -fRr
```
 (force portupgrade)

-N - new port

=============

*all ports upgrade:*
  1) 


```
cd /usr/ports/sysutils/portupgrade && make && make install && make clean
```


```
pkgdb -aF
```
or  
	
	



```
pkgdb -fu
```

  2) 
	
	



```
portupgrade -arRf
```
2,a) or 


```
#!/usr/bin/perl

$nn = 0;

while (1) {

    $nn++;

    open( OPERN, "portversion |" );

    @all = <OPERN>;

    if ( $nn > 6 ) {
    print "while 6 exit";
    exit;
    }

    foreach (@all) {

        my ( $pp, $st ) = split( / /, $_, 2 );

        if ( $st =~ '<' ) {

            print "UPDATE: $pp\n";

            system("portupgrade -f $pp");  


        } else {
            print "ok UPDATE";
            exit;
        }
    }

}
```

2,b) or SUPER SCRIPT:
(*Recommended*)

```
#!/usr/bin/perl

$nn = 0;

while (1) {

    $nn++;

    open( OPERN, "portversion |" );

    my @all2 = <OPERN>;

    close OPERN;

    my @all;

    for ( $i = 0 ; $i < @all2 ; $i++ ) {

        my ( $pp, $st ) = split( / /, $all2[$i], 2 );
        if ( $st =~ '<' ) {

            push @all, $pp;    # $all[$i] = $pp;

        }

    }

    exit if ( !$all[0] || $nn > 2 );

    while (1) {

        last if !$all[0];

        print "$all[0]\n";

        logsave( get_time(), $all[0] );

        system("portupgrade -rf $all[0]");  
      # system("portupgrade -Rf $all[0]");

      # First port to try to update the length and breadth
      # (Above the system need to comment)

      # if ($nn == 1) {
      #  system("portupgrade -rRf $all[0]");
      #  } else {
      #  system("portupgrade -rf $all[0]");
      #  }
   # Experimental:
   # You can add a script to automatically pressed the Enter or (portupgrade -y)
   #    use IO::Select;  
   #     my $select = IO::Select->new;
   #    for(@array) 
   #     open my $pipe, "|$_";
   #     $select->add($pipe);
   #    }
   #    my @waiters = $select->can_write($timeout);
   #    print $_ "\x0a" for @waiters; 


        logsave( get_time(), $all[0] );

        shift @all;

        my @all = old(@all);

    }

}

sub old {

    my @all = @_;

    open( OPEN2, "portversion |" );

    my @all_all = <OPEN2>;

    close OPEN2;

    my @old;
    my @no_old;

    foreach my $p (@all_all) {

        my ( $pname, $status ) = split( / /, $p, 2 );

        if ( $status =~ '<' ) {

            push @old, $pname;

        }
        else {
            push @no_old, $pname;
        }
    }

    my %seen;
    @seen{@all} = ();
    delete @seen{@no_old};
    return keys %seen;

}

sub get_time {
    my ( $sec, $min, $hour, $mday, $mon, $year, $wday, $yday, $isdst ) =
      localtime(time);
    $mon++;
    $year += 1900;
    if ( $mday < 10 ) { $mday = "0$mday"; }
    if ( $mon < 10 )  { $mon  = "0$mon"; }
    if ( $min < 10 )  { $min  = "0$min"; }
    my $date        = "$mday $mon $year";
    my $time        = "$hour:$min:$sec";
    my $cur_all_day = $mday + $mon * 30 + $year * 365;
    my $radate      = "$year-$mon-$mday $hour:$min:$sec";
    return $radate;
}


sub logsave {
    my ( $time, $ports ) = @_;
    my $logfile;
    $logfile = "\n time:  $time \n  ports: $ports \n\n";
    system("touch /var/log/portupgrade.log");
    open( DB2, "/var/log/portupgrade.log" ) || die "Cannot open file: $!";
    my @base = <DB2>;
    close(DB2);
    open( DB, ">/var/log/portupgrade.log" ) || die "Cannot open file : $!";
    print DB @base;
    print DB $logfile;
    close(DB);
}
```


----------



## kpedersen (Nov 25, 2010)

SirDice said:
			
		

> Because that's what ports do



Lol, agreed and I am very thankful that they do or it would be a heck of a lot harder to compile from source 

ProFTP,

OK, I will try all your commands out... I just hope you havn't hidden a `rm -r -f /` in there somewhere... (Or even worse `rm -r -f ~/Porn`) 

Thanks


----------

