# Can we make a perl script to download everything from a mirror and make a mirror?



## fender0107401 (Thursday at 1:27 PM)

This is the mirror I want to copy: https://mirrors.ustc.edu.cn/freebsd-pkg/

I hope I can make a private mirror for FreeBSD 13 or 14.

So, can I just download everything and make a private mirror?


----------



## SirDice (Thursday at 1:29 PM)

fender0107401 said:


> So, can I just download everything and make a private mirror?


Yes, technically possible. You don't even have to download _everything_, just download what you need (and their dependencies). You can generate a catalog for use on your own server using pkg-repo(8).


----------



## fender0107401 (Friday at 3:46 PM)

SirDice said:


> Yes, technically possible. You don't even have to download _everything_, just download what you need (and their dependencies). You can generate a catalog for use on your own server using pkg-repo(8).



Can I config an Nginx Server to share this private mirror?

I am not sure whether the pkg(8) tool can access to this private mirror with Nginx.

Here is the plan:

Step 1 is to download everthing and make a private mirror wike pkg-repo(8).
Step 2 is to share this private mirror with other FreeBSD system by using an Nginx Server.


----------



## SirDice (Friday at 4:21 PM)

fender0107401 said:


> Can I config an Nginx Server to share this private mirror?


Yes, I have my private repository (built with ports-mgmt/poudriere) shared with nginx. 



fender0107401 said:


> I am not sure whether the pkg(8) tool can access to this private mirror with Nginx.


See pkg.conf(5)

```
For a MIRROR_TYPE of NONE, any of the URL schemes supported by
     libfetch(3) can be used, including: http://, https://, ftp://, or
     file://.  In addition a ssh:// URL scheme is also supported.  Where
     MIRROR_TYPE is SRV, you should use a pkg+http:// or pkg+https:// (etc.)
     URL scheme.  Using an http:// URL implies that the hostname part is a
     simple hostname according to RFC 2616, and is no longer accepted.
```
It's still mentioned but I do believe ftp:// has been removed, or will be removed sometime in the near future.


----------



## BobSlacker (Friday at 6:24 PM)

SirDice said:


> ftp:// has been removed, or will be removed sometime in the near future


What is the rationale behind this?


----------



## fender0107401 (Yesterday at 1:14 AM)

SirDice said:


> Yes, I have my private repository (built with ports-mgmt/poudriere) shared with nginx.
> 
> 
> See pkg.conf(5)
> ...



I got enough information.


----------



## Alain De Vos (Yesterday at 1:21 AM)

```
wget -r -np "https://mirrors.ustc.edu.cn/freebsd-pkg/FreeBSD%3A13%3Aamd64/quarterly/"
```


----------



## fender0107401 (Yesterday at 1:40 AM)

SirDice said:


> Yes, I have my private repository (built with ports-mgmt/poudriere) shared with nginx.
> 
> 
> See pkg.conf(5)
> ...



One more question, can I do some similar things to create a private mirror to run freebsd-update.

I known freebsd-update may only need to download and update a few files, bu I still hope can make a private mirror to update the base system.


----------

