# Quaterly ports fetch failure for devel/isl & multimedia/svt-av1



## Alain De Vos (Oct 4, 2021)

Building quarterly ports fails for fetch for the ports:
devel/isl

```
[00:00:02] => isl-0.24.tar.xz doesn't seem to exist in /portdistfiles/.
[00:00:02] => Attempting to fetch http://isl.gforge.inria.fr/isl-0.24.tar.xz
[00:01:17] fetch: http://isl.gforge.inria.fr/isl-0.24.tar.xz: Operation timed out
[00:01:17] => Attempting to fetch http://distcache.FreeBSD.org/ports-distfiles/isl-0.24.tar.xz
[00:01:18] fetch: http://distcache.FreeBSD.org/ports-distfiles/isl-0.24.tar.xz: Not Found
[00:01:18] => Couldn't fetch it - please try to retrieve this
[00:01:18] => port manually into /portdistfiles/ and try again.
[00:01:18] *** Error code 1
[00:01:18]
[00:01:18] Stop.
```

multimedia/svt-av1.

```
[00:00:02] => AOMediaCodec-SVT-AV1-3971c982311d49f9355dc8dccdcf8d21b70fa624_GL0.tar.gz doesn't seem to exist in /portdistfiles/.
[00:00:02] => Attempting to fetch https://gitlab.com/AOMediaCodec/SVT-AV1/-/archive/3971c982311d49f9355dc8dccdcf8d21b70fa624.tar.gz?dummy=/AOMediaCodec-SVT-AV1-3971c982311d49f9355dc8dccdcf8d21b70fa624_GL0.tar.gz
[00:00:02] fetch: https://gitlab.com/AOMediaCodec/SVT-AV1/-/archive/3971c982311d49f9355dc8dccdcf8d21b70fa624.tar.gz?dummy=/AOMediaCodec-SVT-AV1-3971c982311d49f9355dc8dccdcf8d21b70fa624_GL0.tar.gz: size unknown
[00:00:02] fetch: https://gitlab.com/AOMediaCodec/SVT-AV1/-/archive/3971c982311d49f9355dc8dccdcf8d21b70fa624.tar.gz?dummy=/AOMediaCodec-SVT-AV1-3971c982311d49f9355dc8dccdcf8d21b70fa624_GL0.tar.gz: size of remote file is not known
[00:00:02] AOMediaCodec-SVT-AV1-3971c982311d49f9355dc8dccdcf8d21b70fa624_G        6251 kB 6453 kBps    01s
[00:00:03] => 284ef885f85f.patch doesn't seem to exist in /portdistfiles/.
[00:00:03] => Attempting to fetch https://gitlab.com/AOMediaCodec/SVT-AV1/-/commit/284ef885f85f.patch
[00:00:04] fetch: https://gitlab.com/AOMediaCodec/SVT-AV1/-/commit/284ef885f85f.patch: size unknown
[00:00:04] fetch: https://gitlab.com/AOMediaCodec/SVT-AV1/-/commit/284ef885f85f.patch: size of remote file is not known
[00:00:04] 284ef885f85f.patch                                    5526  B 9145 kBps    00s
[00:00:04] => Fetched file size mismatch (expected 5534, actual 5526)
[00:00:04] => Trying next site
[00:00:04] => Attempting to fetch http://distcache.FreeBSD.org/ports-distfiles/284ef885f85f.patch
[00:00:04] fetch: http://distcache.FreeBSD.org/ports-distfiles/284ef885f85f.patch: Not Found
[00:00:04] => Couldn't fetch it - please try to retrieve this
[00:00:04] => port manually into /portdistfiles/ and try again.
[00:00:04] *** Error code 1
[00:00:04]
[00:00:04] Stop.
[00:00:04] make: stopped in /usr/ports/multimedia/svt-av1
[00:00:04] =>> Cleaning up wrkdir
```


----------



## SirDice (Oct 4, 2021)

Upstream site seems to have problems. Not a lot you can do about that.


----------



## astyle (Oct 4, 2021)

I usually try not to build everything at once... My use case actually requires that in order to avoid circular deps.  So, Alain De Vos , just wait an hour, and try again. My experience is that when big repos are doing the quarterly or annual churn with a new version release, that's when things kind of clog up and slow down. This happened to me once when I was trying to install KDE right around the time they released a new version of KF5 this past summer.  Compilation went fine, but fetching was a pain - mirrors were all tied up. I had to wait a day for things to unclog.


----------



## diizzy (Oct 4, 2021)

Unfortunately GitLab tends to change the way on how patches are generated which causes mismatches like the one above


----------



## astyle (Oct 4, 2021)

diizzy said:


> Unfortunately GitLab tends to change the way on how patches are generated which causes mismatches like the one above


I never had an issue with Gitlab specifically. If there's a checksum mismatch, I just re-fetch, and that solves the issue.


----------



## SirDice (Oct 4, 2021)

Problem in this case seems to be that http://isl.gforge.inria.fr is offline or really, really slow to respond. It doesn't want to open for me too.


----------



## zirias@ (Oct 4, 2021)

astyle said:


> I never had an issue with Gitlab specifically. If there's a checksum mismatch, I just re-fetch, and that solves the issue.


I've seen exactly that before. Gitlab (like github) offers a way to create a patch from a commit. Unlike github, the exact output changes sometimes.

I don't remember which port was affected when I've seen it, but I *do* remember the maintainer had enough of it after it happened twice and instead pulled the patch into files.



SirDice said:


> Problem in this case seems to be that http://isl.gforge.inria.fr is offline or really, really slow to respond. It doesn't want to open for me too.


The first case, yes. Really not much you can do about it. But the second clearly is the annoying habit of gitlab to slightly change the patch format…


----------



## astyle (Oct 4, 2021)

Zirias said:


> I've seen exactly that before. Gitlab (like github) offers a way to create a patch from a commit. Unlike github, the exact output changes sometimes.
> 
> I don't remember which port was affected when I've seen it, but I *do* remember the maintainer had enough of it after it happened twice and instead pulled the patch into files.
> 
> ...


Yet another reason to not over-rely on automation - it does make it much more difficult to isolate the issue and deal with it.


----------



## T-Daemon (Oct 4, 2021)

Alain De Vos said:


> `[00:00:02] => isl-0.24.tar.xz doesn't seem to exist in /portdistfiles/.`


You can manually download and put the file in /portdistfiles: https://src.esselfe.ca/isl-0.24.tar.xz


----------



## Alain De Vos (Oct 4, 2021)

T-Daemon said:


> You can manually download and put the file in /portdistfiles: https://src.esselfe.ca/isl-0.24.tar.xz


I'm using poudriere. So in which full-directory to put this file ?


----------



## astyle (Oct 4, 2021)

/usr/ports/distfiles, or wherever you pointed Poudriere to.


----------



## Alain De Vos (Oct 5, 2021)

mail/alpine fetch is also failing,

```
[00:00:01] --End resource limits--
[00:00:01] =======================<phase: check-sanity   >============================
[00:00:01] ===========================================================================
[00:00:01] =======================<phase: pkg-depends    >============================
[00:00:01] ===>   alpine-2.25 depends on file: /usr/local/sbin/pkg - not found
[00:00:01] ===>   Installing existing package /packages/All/pkg-1.17.2.pkg
[00:00:02] [ap-ports-job-04] Installing pkg-1.17.2...
[00:00:02] [ap-ports-job-04] Extracting pkg-1.17.2: .......... done
[00:00:03] ===>   alpine-2.25 depends on file: /usr/local/sbin/pkg - found
[00:00:03] ===>   Returning to build of alpine-2.25
[00:00:03] ===========================================================================
[00:00:03] =======================<phase: fetch-depends  >============================
[00:00:03] ===========================================================================
[00:00:03] =======================<phase: fetch          >============================
[00:00:03] => alpine-2.25.tar.gz doesn't seem to exist in /portdistfiles/.
[00:00:03] => Attempting to fetch https://repo.or.cz/alpine.git/snapshot/465e23b898e5a3393b9ae52fc7620002541b2e75.tar.gz?dummy=/alpine-2.25.tar.gz
[00:00:11] fetch: https://repo.or.cz/alpine.git/snapshot/465e23b898e5a3393b9ae52fc7620002541b2e75.tar.gz?dummy=/alpine-2.25.tar.gz: size unknown
[00:00:11] fetch: https://repo.or.cz/alpine.git/snapshot/465e23b898e5a3393b9ae52fc7620002541b2e75.tar.gz?dummy=/alpine-2.25.tar.gz: size of remote file is not known
[00:00:11] alpine-2.25.tar.gz                                      22 MB 3264 kBps    06s
[00:00:18] => maildir.patch.gz doesn't seem to exist in /portdistfiles/.
[00:00:18] => Attempting to fetch http://alpine.x10host.com/alpine/patches/alpine-2.25/maildir.patch.gz
[00:00:18] fetch: http://alpine.x10host.com/alpine/patches/alpine-2.25/maildir.patch.gz: size mismatch: expected 33849, actual 34808
[00:00:18] => Attempting to fetch http://distcache.FreeBSD.org/ports-distfiles/maildir.patch.gz
[00:00:19] fetch: http://distcache.FreeBSD.org/ports-distfiles/maildir.patch.gz: Not Found
[00:00:19] => Couldn't fetch it - please try to retrieve this
[00:00:19] => port manually into /portdistfiles/ and try again.
[00:00:19] *** Error code 1
[00:00:19]
[00:00:19] Stop.
[00:00:19] make: stopped in /usr/ports/mail/alpine
[00:00:19] =>> Cleaning up wrkdir
[00:00:19] ===>  Cleaning for alpine-2.25
[00:00:19] build of mail/alpine | alpine-2.25 ended at Tue Oct  5 19:01:28 UTC 2021
[00:00:19] build time: 00:00:18
```


----------



## Alexander88207 (Oct 5, 2021)

Yep.

The MAILDIR option causes this.






						258675 – mail/alpine: distinfo out of date for maildir.patch.gz
					






					bugs.freebsd.org
				





```
===>  License APACHE20 accepted by the user
===>   alpine-2.25 depends on file: /usr/local/sbin/pkg - found
=> maildir.patch.gz doesn't seem to exist in /usr/ports/distfiles/.
=> Attempting to fetch http://alpine.x10host.com/alpine/patches/alpine-2.25/maildir.patch.gz
fetch: http://alpine.x10host.com/alpine/patches/alpine-2.25/maildir.patch.gz: size mismatch: expected 33849, actual 34808
=> Attempting to fetch http://distcache.FreeBSD.org/ports-distfiles/maildir.patch.gz
fetch: http://distcache.FreeBSD.org/ports-distfiles/maildir.patch.gz: Not Found
=> Couldn't fetch it - please try to retrieve this
=> port manually into /usr/ports/distfiles/ and try again.
```


----------



## grahamperrin@ (Oct 5, 2021)

Alain De Vos said:


> I'm using poudriere. So in which full-directory to put this file ?



My preference: 


```
root@mowa219-gjp4-8570p-freebsd:~ # grep -i distfiles /usr/local/etc/poudriere.conf
# If set the given directory will be used for the distfiles
# This allows to share the distfiles between jails and ports tree
# the required distfiles.
DISTFILES_CACHE=/usr/ports/distfiles
# DISTFILES_CACHE=/usr/local/poudriere/ports/default/distfiles
root@mowa219-gjp4-8570p-freebsd:~ #
```

Hint: `sudo mkdir -p /usr/ports/distfiles`


----------

