# ports: parallel fetch



## Beju (Aug 15, 2009)

Well, this is just an idea that might be helpful to all users who don't have access to a broadband Internet connection. I don't even know if it is "implementable", but I think It's worth of trying 
The idea is simple:
When You are installing more than one app from ports (i.e. the dependencies), it's a sequence of downloading and building alternately. Because some ports have very long build time especially on slower machines, that's an unnecessary waste of time. So why don't fetch the next required package while building the current? This feature is implemented i.e. in Gentoo portage. I think It'd be useful.


----------



## avilla@ (Aug 16, 2009)

of course it would be. actually, it's already in the wiki: http://wiki.freebsd.org/PortsToDo#head-0f9c53b6934169a564bd772bbee63aab6b1dfa71


----------



## MG (Aug 16, 2009)

Then there must be some sort of mechanism that watches the fetching proces to pass information to the build proces about the downloads that are in progress. Otherwise make sees a partially downloaded distfile and tries to complete it while the other fetching process is still running. It might result in corrupted distfiles.


----------



## MG (Aug 16, 2009)

xzhayon said:
			
		

> of course it would be. actually, it's already in the wiki: http://wiki.freebsd.org/PortsToDo#head-0f9c53b6934169a564bd772bbee63aab6b1dfa71



Nice, haven't seen this doc before.


----------



## avilla@ (Aug 16, 2009)

MG said:
			
		

> Then there must be some sort of mechanism that watches the fetching proces to pass information to the build proces about the downloads that are in progress. Otherwise make sees a partially downloaded distfile and tries to complete it while the other fetching process is still running. It might result in corrupted distfiles.



or, you can set a "fetching" status (creating work/.fetching._port_._usr_local... ports are already using those files to mark the steps of the build), and then make make(1) poll the pending ports and wait for one of them to complete and be ready for the build
and there are probably plenty of better options than this 

and yes, that's a pretty interesting document... i hope some of those ideas get implemented in the future


----------



## jb_fvwm2 (Aug 16, 2009)

```
wget -c -nd --limit-rate=1k http://...file...
```
if you have a 4k dial up connection, you can download
4 at once, each 1/4 the speed, in seperate, for 
example, xterms, and if only downloading 3 at once, that
leaves 1k for browsing, (pages will load slower so you
can load them in the tabs -- background with the right
browser(s). )


----------



## VictorM (Dec 29, 2009)

this could be interesting when deploying VMs based on NetBSD - thanks given


----------



## lme@ (Dec 30, 2009)

Thread moved to Ports section.


----------



## 0mp (Dec 10, 2021)

There is now an experimental tool called parfetch which can fetch FreeBSD ports in parallel and with connection pipelining. See https://github.com/t6/parfetch.


----------



## grahamperrin@ (Dec 20, 2021)

avilla@ said:


> http://wiki.freebsd.org/PortsToDo#head-0f9c53b6934169a564bd772bbee63aab6b1dfa71



_does not exist yet_


----------

