# Poudriere: Creating repository in /tmp/packages



## poorandunlucky (Dec 21, 2017)

Poudriere does this after it's done building:


```
[00:18:46] Creating pkg repository
Creating repository in /tmp/packages: 100%
Packing files for repository: 100%
[00:35:44] Committing packages to repository
```

As you can see, it takes forever...  What is that?  Is that normal?


----------



## rigoletto@ (Dec 21, 2017)

This is not normal and I do not know what is wrong in there. It take just a couple seconds at maximum in here.


----------



## talsamon (Dec 21, 2017)

Maybe, other services or cronjobs run at this time, check this.


----------



## poorandunlucky (Dec 21, 2017)

lebarondemerde : It used to be pretty fast here, too...  I just figured it was getting longer as the repository was getting bigger, but now it's ridiculous...


----------



## rigoletto@ (Dec 21, 2017)

Running out of space, maybe?

You could run `poudriere pkgclean`.


----------



## SirDice (Dec 22, 2017)

The only thing that's being done at this stage is a pkg-repo(8), even with a 600+ packages repository it shouldn't take more than a minute. Is there anything else running that might be using a lot of I/O perhaps? That would certainly slow things down.


----------



## xtaz (Dec 22, 2017)

I have 179 packages in my repo and that stage takes about 1.5 seconds.


----------



## poorandunlucky (Jan 19, 2018)

lebarondemerde SirDice xtaz : idk if you guys remember this, but maybe a week ago I was trying to watch a 4K stream that kept getting (visually) "buffered" by the TV, and I checked a bunch of things, and it wasn't being transcoded, it wasn't the network, it wasn't ... anything, so I decided to deal with it, but after pausing the movie I saw that the HDD was being accessed like crazy, and since the stream wasn't being transcoded, and the TV was on pause, I ran top to figure out what was going on, and it was pkg.

I wasn't building anything, nothing was being installed, and it was 2 or 3 AM, so I figure there's a monthly maintenance that involves pkg and a database or a cache of some sort, and since my installation is still fairly new, I think maybe that stage takes forever because I have a pretty big repo (for a laptop, anyway), and pkg doesn't have the hints it needs to be able to deal with it efficiently...

I killed it to watch the movie, though, so I'll know during the month, i guess.


----------



## talsamon (Jan 20, 2018)

I did not write, but that is was I guessed in the post above --> `pkg check -qsa` the checksum check......

Edit: exactly `/usr/local/etc/periodic/security/460.pkg-checksum` and it starts (seems default) at 3:30. Maybe, an hour difference between the timezones.


----------



## Chris_H (Jan 20, 2018)

poorandunlucky ,
I can't help but notice that both of the times indicated in your posts, were suspiciously close to cron(8)'s daily tasks ( periodic(8) ); mail at midnight, and your administrative system informational reports at 3am. Not saying that must be it. Just saying it might be a possibility. Really, more information _is_ required for any conclusive deduction/solution. Out of curiosity; how much RAM/SWAP? What CPU/SPEED? Lastly; are you running any other heavy services than ports-mgmt/poudriere; like a web server, or something?

--Chris


----------



## talsamon (Jan 20, 2018)

But it is not clear, why it runs: all my `periodic.conf` shows:

```
# 460.chkportsum
security_status_chkportsum_enable="NO"          # Check ports w/ wrong checksum
security_status_chkportsum_period="daily"
```

and it is very likely it is, it is slow down `poudriere`.


----------



## Chris_H (Jan 20, 2018)

Well, what shows up in your administrative mail every day? You know. Whoever you aliased as root. So that the system mail gets delivered. 

--Chris


----------



## talsamon (Jan 20, 2018)

```
mail
No mail for root
```

I never looked  at this process.  If I wanted to know I ran it per hand. I have  not seen it until it locks the port database, when I wanted install a port.
It is a senseless running process, results are nowhere in any log.  How to stop this or log it?


----------



## Chris_H (Jan 20, 2018)

Um, OK. So I guess the _real_ question is whats the output of `cat /etc/periodic`.
Next. What's the contents of /etc/defaults/periodic.conf. The last one will better indicate exactly _what_ is enabled in your system.

HTH

--Chris


----------



## talsamon (Jan 20, 2018)

`cat /etc/periodic` does not work it is a directory.


----------



## talsamon (Jan 20, 2018)

Ah, I see it:
If I want stop it, I had to change

```
security_status_chkportsum_period="daily"
to
security_status_chkportsum_period="NO"
```
.

but how to log it?


----------



## Chris_H (Jan 20, 2018)

Ugh. Sorry it's late, and I'm tired. I meant /etc/*crontab*
crontab(1).
This will indicate if, and what you're running (daily,weekly,monthly) which is the default. fe;

```
#minute hour    mday    month   wday    who     command
#
# Save some entropy so that /dev/random can re-seed on boot.
*/11    *       *       *       *       operator /usr/libexec/save-entropy
#
# Rotate log files every hour, if necessary.
0       *       *       *       *       root    newsyslog
#
# Perform daily/weekly/monthly maintenance.
1       3       *       *       *       root    periodic daily
15      4       *       *       6       root    periodic weekly
30      5       1       *       *       root    periodic monthly
#
# Adjust the time zone if the CMOS clock keeps local time, as opposed to
# UTC time.  See adjkerntz(8) for details.
1,31    0-5     *       *       *       root    adjkerntz -a
```
the output from mine.
and also the contents of /etc/defaults/periodic.conf
periodic.conf(5)

--Chris


----------



## talsamon (Jan 20, 2018)

Thanks, /etc/crontab looks the same.
=> how to log:
I only have to read .....

```
#$daily_output might be set to /var/log/daily.log if you
# wish to log the daily output.....
```

this should solve it.


----------



## Chris_H (Jan 20, 2018)

I think you misunderstand. If you read your well commented periodic.conf, and correlate that with your crontab. You will understand that you're running several jobs; 1) at 12am, and several at 3am. This really shouldn't be a problem. For me, it's easy to plan any heavy tasks between 12:30am - 2:59am, and 3:15am - 11:59pm. If I feel that the heavy tasks will be too burdensome to run during periodic(8). As to logging; if you haven't done so, edit /etc/mail/aliases ( aliases(5) ), and change the entry

```
root:   me@my.host
```
to your administrative account. fe;

```
root:    chris_h
```
then
`cd /etc/mail]`
`make aliases`
then you'll be able to see what is actually going on, on your system. This will also help you better determine which logs are important to you, and which ones you can disable. Make a little more sense?

HTH

--Chris


----------



## poorandunlucky (Jan 22, 2018)

Chris_H said:


> poorandunlucky ,
> I can't help but notice that both of the times indicated in your posts, were suspiciously close to cron(8)'s daily tasks ( periodic(8) ); mail at midnight, and your administrative system informational reports at 3am. Not saying that must be it. Just saying it might be a possibility. Really, more information _is_ required for any conclusive deduction/solution. Out of curiosity; how much RAM/SWAP? What CPU/SPEED? Lastly; are you running any other heavy services than ports-mgmt/poudriere; like a web server, or something?
> 
> --Chris



I'm the only user on this system, nothing was running at the time other than pkg...  Firefox, though idle, was taking more CPU than minidlna...  top was showing pkg to be using 300% CPU or something...

The problem is really after building a package, poudriere says it's building a repository in /tmp/packages, and for most people this step takes a few seconds, while it's several minutes on my system...

It's a Intel Core i7 1.73-2.36 GHz (4 cores, 8 threads) with 16 GB of RAM, and 4/5 GB of SWAP...  Memory's never really been an issue...  CPU often is, but not in normal use...

I'm just not sure if both situations are related (the long time after building, and the one time pkg was using 300% CPU for an hour at 3AM).


----------



## Chris_H (Jan 22, 2018)

You have more than enough resources available for running ports-mrmt/poudriere as you are. If it were a memory issue, and you had /tmp mounted as tmpfs(5), which is essentially an md(4), I could see a _possibility_. But in your case, it's swamping the CPU. Which _shouldn't_ be an issue in your case. FWIW I'm _frequently_ plagued with issues, when Firefox is running -- it has always leaked memory like a sieve, and _often_ doesn't handle the compiled javascrips delivered these days. Maybe it's worth trying it w/o Firefox running, next time. 

Good luck!

--Chris


----------

