# Questions when using distfile groups in a port - MASTER_SITE_SUBDIR not being applied?



## neogeo (Dec 29, 2021)

Hello, all,

I'm working on trying to update the earlier *lang/rubinius* port, beginning at the last port revision available from before the port was deleted from the active ports tree, updating that to v5.0 from *rubinius at github*.

When using all of the upstream build files, it seems that the port would need multiple distfiles. It would need one distfile for the main code bundle from GitHub, plus six more for three bundled archives and three archive checksum files used under rubinius' *configure.rb*. Each of the archive and checksum files is distributed under a similar site URL and distdir. Short of any additional patching, both are used in the configure scripting.

Each of those three bundles is distributed under a known subdirectory of the base URL `https://rubinius-binaries-rubinius-com.s3.amazonaws.com/`. For a list of bundles named "codedb", "codetools", and "stdlib" e.g the first bundle would be available under the following URL, with its corresponding checksum file for the upstream configure scripting:


```
https://rubinius-binaries-rubinius-com.s3.amazonaws.com/codedb/rubinius-codedb-cache.bz2
https://rubinius-binaries-rubinius-com.s3.amazonaws.com/codedb/rubinius-codedb-cache.bz2.sha512
```

Although I believe that the sha512 files may be redundant here, beside the distfile checksums provided normally in a port `distinfo` file, each checksum file is used in Rubinius' configure.rb, to verify the integrity of the archive data file.

For this part of the port distfiles section, I was trying to use the following. It seems that the `MASTER_SITE_SUBDIR` value is not being applied for any of these files


```
PORTNAME=       rubinius
PORTVERSION=    5.0
PORTREVISION=   1
CATEGORIES=     lang ruby

USE_GITHUB=     yes
GH_TUPLE=       rubinius:rubinius:v${PORTVERSION}
MASTER_SITES=   GH
MASTER_SITES+=  https://rubinius-binaries-rubinius-com.s3.amazonaws.com/:codedb,codetools,stdlib

RUBINIUS_DISTCACHES=    codedb codetools stdlib
.for .M. in ${RUBINIUS_DISTCACHES}
MASTER_SITE_SUBDIR+=    ${.M.}/:${.M.}
DISTFILES+=             rubinius-${.M.}-cache.bz2:${.M.} rubinius-${.M.}-cache.bz2.sha512:${.M.}
.endfor
DIST_SUBDIR=    rubinius
EXTRACT_ONLY=   ${DISTFILES:M*\:DEFAULT:C/:DEFAULT//}
# ...
```

The variable has a value that can be reviewed with the following:

`[u1000@riparian ~/.../ports_devo/lang/rubinius ]$ make -V MASTER_SITE_SUBDIR`

This produces the output

```
codedb/:codedb codetools/:codetools stdlib/:stdlib
```

However, under `make makesum` and `make fetch-list` the MASTER_SITE_SUBDIR declarations are not showing up. in the URLs. Instead of the expected value, it produces a URL e.g `https://rubinius-binaries-rubinius-com.s3.amazonaws.com/rubinius-codedb-cache.bz2`

Looking at Section 5.4.9.2. of the Porters Handbook (Detailed Information) I believe I may have the syntax right for what's shown there. I don't know why the MASTER_SITE_SUBDIR entries are not being applied.

I'll attach the current Makefile for the port update, onto this thread item. There are some temporary annotations/comments for point of reference, in the Makefile.

Outside of a patch for how the upstream codebase is resolving Ruby gemspec names of a specific version - when there is another way to do this -  hopefully it may be fairly straightforward bit of time to update the port, after working out the distfiles part.

I'll take a look at what's bundled in those *.bz2 archives. For a ports build, maybe there's a third way to do this ...

*Update*: Orthogonal to the question about distfile URLs and after looking at some of the files in the Rubinius source tree, it seems that two of the cache.bz2 files contain bundled source files, for a set of software components distributed separately under some specific version for each. Separately, the codedb file would be ... something else, roughly, maybe not a build dependency though distributed as something like an upstream distfile. The other two files can probably be provided from FreeBSD ports, though I'd like to test the build first with what the maintainers have published for it.

I could leave out the MASTER_SITE_SUBDIR part and just use three separate MASTER_SITES entries for each separate cache file. I believe it seems less succinct, however. I'm not certain why the MASTER_SITE_SUBDIR section is not working out

*Update:* Noticing that any change in the upstream *.bz2 files may break some builds, as the upstream files would then differ to what would correspond to the checksums in the distinfo file, this approach of using those upstream bundle files could only be of some temporary use for a port build. It could not be of use for a port for the ports tree.

There must be a way to approach all of those intrinsic build dependencies, to use each thing as available in ports, and to provide it in some way matching any assumptions in the upstream codebase.

Although there is a workaround for it, I wonder why the MASTER_SITE_SUBDIR expressions are not working out, in the port definition?


----------

