# Using "make fetch" without FTP access.



## krivenok (Jan 18, 2010)

Hello Gurus!

I have a very basic question about FreeBSD ports.
Unfortunately, I didn't find answer in documentation and google.

Is there a way to completely avoid using FTP protocol during 
"make fetch" command and use HTTP instead?

The problem is that all FTP connections are dropped on the firewall
of my company and it's almost impossible to change firewall rules 
to allow FTP (it's not a technical problem).
This applies to both active and passive FTP modes.

To demonstrate the problem lets try to fetch pcre tarball in my environment:

```
# make fetch
===>  Vulnerability check disabled, database not found
=> pcre-8.00.tar.bz2 doesn't seem to exist in /usr/ports/distfiles/.
=> Attempting to fetch from ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/.
fetch: transfer timed out
=> Attempting to fetch from ftp://ftp.fu-berlin.de/unix/misc/pcre/.
fetch: transfer timed out
=> Attempting to fetch from ftp://ftp.tin.org/pub/libs/pcre/.
fetch: transfer timed out
=> Attempting to fetch from ftp://ftp.FreeBSD.org/pub/FreeBSD/ports/distfiles/.
fetch: transfer timed out
=> Couldn't fetch it - please try to retrieve this
=> port manually into /usr/ports/distfiles/ and try again.
*** Error code 1

Stop in /usr/ports/devel/pcre.
#
```

As you can see the command tried 4 different URLs.
If I understood correctly, the first 3 URLs are hard-coded in 
Makefile of pcre:


```
# grep ftp Makefile 
MASTER_SITES=   ftp://ftp.csx.cam.ac.uk/pub/software/programming/%SUBDIR%/ \
                ftp://ftp.fu-berlin.de/unix/misc/%SUBDIR%/ \
                ftp://ftp.tin.org/pub/libs/%SUBDIR%/
#
```

and the fourth URL points to main FreeBSD site.

So what I need is
1) Http URL pointing to the site which contains all tarballs for current versions of all ports.
2) The way to force "make fetch" to use this URL instead of 
those listed in Makefile and default one.

Any ideas?

Thank you beforehand!


----------



## SirDice (Jan 18, 2010)

krivenok said:
			
		

> 1) Http URL pointing to the site which contains all tarballs for current versions of all ports.


http://ftp.freebsd.org/pub/FreeBSD/ports/



> 2) The way to force "make fetch" to use this URL instead of
> those listed in Makefile and default one.


Use http URLs.


----------



## DutchDaemon (Jan 18, 2010)

Or if your network has a proxy, try using that for FTP.


----------



## krivenok (Jan 18, 2010)

Thanks for your answers!
I found the following solution:

```
# diff -u /usr/ports/Mk/bsd.port.mk.orig /usr/ports/Mk/bsd.port.mk 
--- /usr/ports/Mk/bsd.port.mk.orig      2010-01-14 17:02:43.000000000 +0000
+++ /usr/ports/Mk/bsd.port.mk   2010-01-18 14:29:47.000000000 +0000
@@ -2618,7 +2618,7 @@
 
 # The primary backup site.
 MASTER_SITE_BACKUP?=   \
-       ftp://ftp.FreeBSD.org/pub/FreeBSD/ports/distfiles/${DIST_SUBDIR}/
+       http://ftp.FreeBSD.org/pub/FreeBSD/ports/distfiles/${DIST_SUBDIR}/
 MASTER_SITE_BACKUP:=   ${MASTER_SITE_BACKUP:S^\${DIST_SUBDIR}/^^}
 
 # If the user has MASTER_SITE_FREEBSD set, go to the FreeBSD repository
#
```

If tarball cannot be downloaded using URLs specified in Makefile, then "make fetch" will also try URL specified in MASTER_SITE_BACKUP. 

The only problem is that it still attempts to download tarball via FTP at first. But you can set FTP_TIMEOUT=1 to speed up the 
process.


----------



## crsd (Jan 18, 2010)

You could also try defining _MASTER_SORT_REGEX=^http_ for ports which have both http and ftp links in MASTER_SITES.


----------

