# Bash script to automate installing bunch of packages



## Sergei_Shablovsky (May 23, 2022)

Dear FreeBSD Gurus!
Please share *bash script to automate installing bunch of packages without any user interaction in a process.*

Let me write a little bit of details what I need exactly:

*pkg_autoinstaller_list.txt* - text file that contain name of packages that I would to install. Each name in separate row. Full name of package (“zsh-5.2.4”), or short version (“zsh”), case-sensitive.
Looks like this below:
—
zsh
zsh-highlighting
py38-speedtest-cli
librespeed
fping
bpytop
webmin
...
—

*pkg-autoinstaller.sh* - bash script that I asking for.

*pkg_autoinstaller_mmddyyyy_hhmmss.log* - log file, that also contain copy of screen output of each pkg installation process for review later.
Name of the file contain data&time of creation.
Each string of log with time stamp.
Summary with Start time and End time, number of possible errors/warnings during the process at the start of this log file, looks like this:
—
Installations summary:
TOTAL packages to install         54
                      EXIST IN REPO   54
                               ABORTED   1
SUCCESSFULLY INSTALLED    53
                            WARNINGS    2
                    REBOOT REQUIRED


TOTAL packages to install - calculated from pkg_to_install.txt.
EXIST IN REPO - calculated from pkg_to_install.txt that successfully founded in pkg repo.
ABORTED - calculated if in output words “error”, “aborted”, case-sensitive
SUCCESSFULLY INSTALLED = EXIST IN REPO - ABORTED
WARNINGS - calculated if in output word “warning”, case-sensitive
REBOOT REQUIRED  - persist if in output word “reboot”, case-sensitive

Script parameters:
-fpkg #force update pkg system
-f #force reinstall package even installed already
-c #ignore case-sensitivity when search in pkg repo
-e #not open log-file in default system editor at the end of installation
-rst #restart system after installation complete if no errors (mean no “Error”, “Failed” in every package from list)
-s #play short beeps (if possible by system mb beeper) on complete after log-file was opened in editor successfully.

*Logic*: (you may correct or add something):
- clean pkg system cache
- update pkg system before installation
- check/reserve 300kb space for log file
- read each package name from pkg_to_install.txt, install it, write screen output to pkg_autoinstaller.log log file
- [OPT] open log file in editor on tty0 (or where from script started), play short beeps to get attention
- [OPT] standard restart server

Thanks you all another one time!

P.S. I more that sure that this is trivial script, that may be exist in Your scripts library...  I still thinking it very useful for SysAdmins that installing BSD frequently.


----------



## Sergei_Shablovsky (May 23, 2022)

P.P.S.
I know that another one option would be installing some tool both on source server and replica like “pkg manager”, and using pkg list export/import to automate installation.
Anyway, keeping a set of this pkg_list.txt not so different from keeping a set of files to import to this “pkg manager”.

May be a You suggest me really great useful tool.


----------



## Bobi B. (May 23, 2022)

Don't use bash(1), as you'll have to install it 1st. Perhaps you can base it on this

```
#!/bin/sh
env ASSUME_ALWAYS_YES=yes pkg-static bootstrap
while read pkgname; do
    env ASSUME_ALWAYS_YES=yes pkg install ${pkgname}
done < pkg_list.txt | tee pkg_autoinstaller_$(date +%Y%m%dT%H%M%S).log
```

Or maybe even

```
env ASSUME_ALWAYS_YES=yes pkg-static bootstrap
env ASSUME_ALWAYS_YES=yes xargs pkg install < pkg_list.txt | tee pkg_autoinstaller_$(date +%Y%m%dT%H%M%S).log
```


----------



## sko (May 23, 2022)

If this is for (mass) deployment, you should either take a look at the scripting capabilities of bsdinstall(8) to create a custom, automated installer image, or consider using something like sysutlis/ansible where you can define a set of packages + configuration for a given type of host (e.g. a webserver) and run that against multiple hosts.
Depending on your use case both may have their advantages and you could also combine them.


----------



## Geezer (May 23, 2022)

ports-mgmt/synth is used to manage ports, and can be configured to use only packages.


----------



## ondra_knezour (May 23, 2022)

sysutils/firstboot-pkgs may be source of some inspiration


----------



## Sergei_Shablovsky (May 23, 2022)

ondra_knezour said:


> sysutils/firstboot-pkgs may be source of some inspiration



Thank You for suggestion!

From Description:

_When the system first boots, install the pkg(8) tools (if not already installed) and packages listed in the $firstboot_pkgs_list rc.conf variable.  If the installed packages added new rc.d scripts, request a reboot. 
Obviously, this port is not useful after a system is already running; it is intended to be included as part of the installation or disk image building process._

I start this topic from “*script*” because needs to easy keep “stack” of packages for each type of server (balancer, public web server, fw, etc...).
Adding extra + one pkg to stack are easy - just add pkg name to list, and run script on each server of that type.

No needs to pull off server from working environment, reinstall, then put it back...


----------



## Sergei_Shablovsky (May 23, 2022)

Geezer said:


> ports-mgmt/synth is used to manage ports, and can be configured to use only packages.



Thank You for suggestions!

From Description:

Synth is a custom packge repository builder for FreeBSD and DragonFly. It is intended to replace Portmaster, portupgrade, and poudriere for the average user.  It is simple to learn (the powerful options are limited in number) and user-friendly, but it is extremely fast due to its parallel building capability.  It will "drop-in" on any system as it leverages the stock pkg(8) facilities.  All ports are built in a clean environment, so it is finally safe to build ports as needed on a live system.  The default profile is the system itself, not a new jail, which can be a valuable feature for some environments. To bring a system up-to-date only requires one command after the ports tree is updated:  > synth upgrade-system During the building process, a curses-based display will show the status of all the builders and the entire bulk run process. A dynamic and searchable web-based report is generated simultaneously. Synth is intended to be grasped and utilized by novice users within minutes, but offers most of the same powerful features as Poudriere for the power users. Synth requires no preparation; it works immediately upon installation. WWW: https://github.com/jrmarino/synth

Looks like VERY close to what I need. Needed to read more it's docs...


----------



## Sergei_Shablovsky (May 23, 2022)

sko said:


> If this is for (mass) deployment, you should either take a look at the scripting capabilities of bsdinstall(8) to create a custom, automated installer image, or consider using something like sysutlis/ansible where you can define a set of packages + configuration for a given type of host (e.g. a webserver) and run that against multiple hosts.
> Depending on your use case both may have their advantages and you could also combine them.



Thank You for suggestions so much!

A few questions:

1.
I am a little bit newbie in BSD. Could You (or anyone else here  be so please to explain in short main difference in pro/cons between Ansible, bsdinstall, synth?

2.
From management point of view no difference between needs to keep bunch of BSD install images (one per each type of server), OR needs to keep bunch of “pkg-lists” + pkg management system.

Right now I see only one disadvantage of bsdinstall: each time when FreeBSD rollout new STABLE, I need to re-creating ALL of custom installer images.

Where am I wrong?

3.
Which ansible2 or ansible I need for FreeBSD 13 ?

4. 
From RedHat web I see that ansible is paid software ? 
I am a little bit disappointed, is it free (as I see it on freshports) or paid (as I see on RH web) ?

Thank You!


----------



## Sergei_Shablovsky (May 23, 2022)

Bobi B. said:


> Don't use bash(1), as you'll have to install it 1st. Perhaps you can base it on this
> 
> ```
> #!/bin/sh
> ...



Thank You so much!!!
2 perfect lines of code! Impressive! 

Do I need to put in pkg_list.txt full name of package “py38-speedtest-cli-2.4.7”, or shorted version “py38-speedtest-cli” would be enough ?


----------



## Bobi B. (May 23, 2022)

Sergei_Shablovsky said:


> Do I need to put in pkg_list.txt full name of package “py38-speedtest-cli-2.4.7”, or shorted version “py38-speedtest-cli” would be enough ?


Write the name the same way you would normally do if you called pkg(8) manually: `pkg install py38-speedtest-cli`, hence `py38-speedtest-cli`.


----------



## SirDice (May 23, 2022)

Bobi B. said:


> Write the name the same way you would normally do if you called pkg(8) manually: `pkg install py38-speedtest-cli`, hence `py38-speedtest-cli`.


This is going to pose a problem when the default Python version changes. Use the port origin in this case; `pkg install net/py-speedtest-cli`. Then it will always be installed, regardless of the Python version at that time.


----------



## bakul (May 23, 2022)

Sergei_Shablovsky said:


> Please share *bash script to automate installing bunch of packages without any user interaction in a process.*


Can't you just use `pkg install -y $(cat pkg_to_install)`? Why make it more complicated? In your pkg list file put _category_/_package_-_name_ as SirDice suggests. If you want to clear the cache just do `rm -rf /var/cache/pkg` once in a while.


----------



## SirDice (May 23, 2022)

bakul said:


> If you want to clear the cache just do `rm -rf /var/cache/pkg` once in a while.


pkg-clean(8)


----------



## sko (May 24, 2022)

Sergei_Shablovsky said:


> Right now I see only one disadvantage of bsdinstall: each time when FreeBSD rollout new STABLE, I need to re-creating ALL of custom installer images.


No
1. why would you want to run stable on servers? Use -RELEASE and quarterly packages until you have a very good reason not to. It makes EVERYTHING much more sane and easier to maintain if you don't have several updates every other day and can just run identical packet versions on all hosts.
2. The 'custom installer image' pretty much is just the stock image + your bsdinstall script. Everything else is pulled from the repositories and configs or additional scripts can also be pulled from e.g. a git repo.
To add the script you just create an md device from the image using mdconfig(8), mount the appropriate partition/slice, add everything you want and you're done:

```
# mdconfig FreeBSD-12.3-RELEASE-amd64-memstick.img
# # gpart show md0
=>      1  2099728  md0  MBR  (1.0G)
        1     1600    1  efi  (800K)
     1601  2098128    2  freebsd  [active]  (1.0G)
# mount /dev/md0s2a /mnt
# ls /mnt
.cshrc          COPYRIGHT       boot/           etc/            libexec/        mnt/            proc/           root/           tmp/            var/
.profile        bin/            dev/            lib/            media/          net/            rescue/         sbin/           usr/

<add / modify anything you want>

# umount /mnt
# mdconfig -d -u md0
```

If this still is too much work, it can be very easily scripted. But if you use -RELEASE for your servers (as you should), this task is only necessary every few months or even just once a year.

On top of that you can (and probably should, depending on the number of systems) run an orchestration framework like ansible/chef/puppet. Keep the bsdinstall routine as minimal and generic as possible - e.g. only necessary network config; ssh keys, custom pkg repos and hooks to integrate to your config management. Everything on top of that which might be host-specific is then handled by the config management.
There's absolutely no need to keep dozens of install images and the horrors of keeping them all up to date. With something like ansible you just add a new role to a server, maybe write a new config file template and you're done.


Regarding ansible: just use sysutils/ansible if you deploy a new setup. The older versions only make sense if you have to keep compatibility.


----------



## Sergei_Shablovsky (May 24, 2022)

Bobi B. said:


> Don't use bash(1), as you'll have to install it 1st. Perhaps you can base it on this
> 
> ```
> #!/bin/sh
> ...


So, if I need use that SirDice wrote, is this scripts working with string in pkg_list.txt ?
—
net/py38-speedtest-cli
net/fping
...
—

Could You be so please to modify a Your script to be able work with?
Thank You so much!


----------



## SirDice (May 24, 2022)

The port is net/py-speedtest-cli, that's called the "origin" in FreeBSD's port/package lingo. Depending on the (default) version of Python the name of the resulting package is py${PYTHON_VERSION}-speedtest-cli. If the Python version changes to 3.9 for example the package would be named py39-speedtest-cli. See how that Python version is reflected in the _package name_? If you try to install py38-speedtest-cli it will fail, because the package is now called py39-speedtest-cli. If you run `pkg install net/py-speedtest-cli`, it will always be installed, regardless of what the (default) Python version happens to be.


```
pkg install [--{automatic,force,no-scripts,ignore-missing}]
                 [--{dry-run,fetch-only,quiet,recursive,no-repo-update,yes}]
                 [--repository reponame]
                 [--{case-sensitive,glob,case-insensitive,regex}]
                 <pkg-origin|pkg-name|pkg-name-version> ...
```
In this case `pkg-origin` is `net/py-speedtest-cli`; `pkg-name` is py38-speedtest-cli and `pkg-name-version` is py38-speedtest-cli-2.1.3.


----------



## Sergei_Shablovsky (May 24, 2022)

bakul said:


> Can't you just use `pkg install -y $(cat pkg_to_install)`? Why make it more complicated? In your pkg list file put _category_/_package_-_name_ as SirDice suggests. If you want to clear the cache just do `rm -rf /var/cache/pkg` once in a while.


Thank You for suggestion!

Is this code doing the same as Bobi B.'s script doing? 

And what about pkg_autoinstaller.log ?


----------



## Sergei_Shablovsky (May 24, 2022)

SirDice said:


> The port is net/py-speedtest-cli, that's called the "origin" in FreeBSD's port/package lingo. Depending on the (default) version of Python the name of the resulting package is py38-speedtest-cli. If the Python version changes to 3.9 for example the package would be named py39-speedtest-cli. See how that Python version is reflected in the _package name_? If you try to install py38-speedtest-cli it will fail, because the package is now called py39-speedtest-cli. If you run `pkg install net/py-speedtest-cli`, it will always be installed, regardless of what the (default) Python version happens to be.


Thank You for detailed explaining!

So, following this logic, the Bobi B.' script must using as input “origin” name. 
—
net/py-speedtest-cli
net/fping 
net/smokeping
...
—

And this would be best variant?


----------



## SirDice (May 24, 2022)

If you look at an existing server, to get a list of packages to install, look at the output from `pkg prime-list`, or `pkg prime-origins`. You don't want to install _everything_, you only want to install the packages you need. Dependencies can change over time and they will automatically be installed, so you don't need to install those yourself.


----------



## bakul (May 24, 2022)

Sergei_Shablovsky said:


> And what about pkg_autoinstaller.log ?


`sudo pkg install -y $(cat pkg_to_install.txt) >& pkg_autoinstaller_$(date +%m%d%Y_%H%m%S).log`? I assume this is only for new installs? For regular updates you'd need something else.

In zsh I usually have these aliases for date related stuff:

`alias   ymd='date +%Y%m%d'
alias   ymdhms='date -u +%Y%m%d%H%M%S' # for UTC timestamp`


----------



## Sergei_Shablovsky (May 24, 2022)

SirDice said:


> If you look at an existing server, to get a list of packages to install, look at the output from `pkg prime-list`, or `pkg prime-origins`. You don't want to install _everything_, you only want to install the packages you need. Dependencies can change over time and they will automatically be installed, so you don't need to install those yourself.


Thank You!

Do I need to add to Bobi. B's script at the end some commands to delete old/unused/unlinked dependencies? Or BSD doing that on regular base (cron) by themselve?


----------



## Sergei_Shablovsky (May 24, 2022)

bakul said:


> `sudo pkg install -y $(cat pkg_to_install.txt) >& pkg_autoinstaller_$(date +%m%d%Y_%H%m%S).log`? I assume this is only for new installs? For regular updates you'd need something else.



Yes, for new installs. 



bakul said:


> In zsh I usually have these aliases for date related stuff:
> 
> `alias   ymd='date +%Y%m%d'
> alias   ymdhms='date -u +%Y%m%d%H%M%S' # for UTC timestamp`


Thanks! 

Which main difference in Your and Bobi. B script in real work? Which is better ?


----------



## bakul (May 24, 2022)

Sergei_Shablovsky said:


> Which main difference in Your and Bobi. B script in real work? Which is better ?


You will have to decide!


----------



## astyle (May 24, 2022)

OP should just keep it simple.  Have something to practice all those ideas on - not the production server, but something that is OK to mess up and delete. Once the results of practice are acceptable, take down some good notes, reproduce the steps for good measure, and only then touch the production stuff.


----------



## SirDice (May 24, 2022)

astyle said:


> Have something to practice all those ideas on - not the production server, but something that is OK to mess up and delete.


Practice makes perfect 

sko already mentioned it, if you're looking for ways to automate server installation and configuration you should invest time in learning chef/puppet/ansible. I learned Puppet a long time ago[*] and have grown to like it (it has a rather steep learning curve in my opinion). Ansible is a bit easier to get started with. It's also quite popular at the moment and many companies are migrating their Puppet infrastructure to Ansible. Learning Puppet or Ansible is also quite beneficial for other operating systems and will definitely increase your position on the unix admin job market. The "traditional" admin jobs are dying out, it's all devops and infrastucture-as-code nowadays.

[*] In the past 5-6 years I've been writing puppet code for most of my working days.


----------



## Sergei_Shablovsky (May 24, 2022)

bakul said:


> You will have to decide!


Ok, I’ll do. 

But is any technical difference in both, or just endothermic style of code writing?


----------



## Sergei_Shablovsky (May 24, 2022)

astyle said:


> OP should just keep it simple.  Have something to practice all those ideas on - not the production server, but something that is OK to mess up and delete. Once the results of practice are acceptable, take down some good notes, reproduce the steps for good measure, and only then touch the production stuff.


Thank You! Totally agree with this methodology.


----------



## SirDice (May 24, 2022)

Sergei_Shablovsky said:


> But is any technical difference in both, or just endothermic style of code writing?


All roads lead to Rome. Something you will find out when dealing with developers and admins alike. If you ask 10 admins to write a script to do something you get 10 different scripts


----------



## bakul (May 24, 2022)

Sergei_Shablovsky said:


> Ok, I’ll do.
> 
> But is any technical difference in both, or just endothermic style of code writing?


What I am trying to tell you is, it doesn't matter. Stop reading the menus pasted outside restaurants; go inside one and eat! Find out from your own experience what works and what doesn't. in Patanjali's Yoga Sutra, the seventh Sutra says "right knowledge comes from direct experience, inference or testimony (of authority -- wise people or the scriptures)". Notice how direct experience is the first recommendation! And do not treat random people as authority! Even test what so called "authoritative" sources tell you!


----------



## astyle (May 24, 2022)

bakul said:


> Stop reading the menus pasted outside restaurants; go inside one and eat!


I'd still want to see what's on offer, what looks appealing, and how much that costs. Computing-wise, that means, don't just grab the first thing you see, take the time to think about it.


----------



## bakul (May 25, 2022)

astyle said:


> I'd still want to see what's on offer, what looks appealing,


How do you know they are not lying to you or giving you the complete picture? My daughter, when she was 5, demanded a favorite dish of hers which was *not* on the menu of this particular upscale Chinese restaurant. The waiter said, no problem, it is his favorite too but they just didn't put on the menu! The direct experience can often be _pleasantly_ surprising so don't bash it!


----------



## sidetone (May 25, 2022)

`pkg install `cat pkglist.txt`` will do it. You could incorporate that into your script. Also, `portmaster `cat portlist.txt``: ports-mgmt/portmaster. I knew that the port category could also be included in a package list, but from reading this, there's more details that are useful.

` is on the keyboard below the Esc key; it's different from the regular single quote. This makes it so the output of the file from the `cat` command is directed into the main command.

I'm not sure if you meant something like this in your second post.

I would be interested in a script in sh(1), the basic Bourne shell.


----------



## astyle (May 25, 2022)

bakul said:


> How do you know they are not lying to you or giving you the complete picture? My daughter, when she was 5, demanded a favorite dish of hers which was *not* on the menu of this particular upscale Chinese restaurant. The waiter said, no problem, it is his favorite too but they just didn't put on the menu! The direct experience can often be _pleasantly_ surprising so don't bash it!


Most places don't post the complete menu on the doorstep anyway. Just enough to grab you by the nose.  Software is like that, too. You read a little of the description, play with it a little, and if you like it, you go all in. But you gotta have a good idea of what you're getting into.


----------



## grahamperrin@ (May 25, 2022)

bakul said:


> … In your pkg list file put _category_/_package_-_name_ …



Beware of doing so where, for example, there are flavours of a port.


```
% pkg info -x falk
falkon-22.04.1
% sudo pkg install --dry-run www/falkon
grahamperrin's password:
Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
Updating poudriere repository catalogue...
poudriere repository is up to date.
All repositories are up to date.
The following 1 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
        falkon-qtonly: 22.04.1 [FreeBSD]

Number of packages to be installed: 1

The process will require 11 MiB more space.
2 MiB to be downloaded.
%
```

I already have full-flavoured Falkon
I don't want the inferior flavour.
<https://www.freshports.org/www/falkon/#flavors>


----------



## sidetone (May 25, 2022)

grahamperrin said:


> Beware of doing so where, for example, there are flavours of a port.


I'm wondering about port category along with `pkg origin` for flavors for this. Could the desired flavor be selected this way?


SirDice said:


> This is going to pose a problem when the default Python version changes. Use the port origin in this case; `pkg install net/py-speedtest-cli`. Then it will always be installed, regardless of the Python version at that time.





SirDice said:


> The port is net/py-speedtest-cli, that's called the "origin" in FreeBSD's port/package lingo. Depending on the (default) version of Python the name of the resulting package is py${PYTHON_VERSION}-speedtest-cli. If the Python version changes to 3.9 for example the package would be named py39-speedtest-cli. See how that Python version is reflected in the _package name_? If you try to install py38-speedtest-cli it will fail, because the package is now called py39-speedtest-cli. If you run `pkg install net/py-speedtest-cli`, it will always be installed, regardless of what the (default) Python version happens to be.


It looks like `pkg origin` only works on packages which are installed. I tried it on a package I had installed, and it showed the origin. When I tried it on the above example, also based on the output of `pkg search`, it didn't work.

`pkg search electrum; pkg origin py38-electrum`

```
py38-electrum-4.2.1_1          Easy to use Bitcoin client
finance/electrum
```
I tried it on falkon-qtonly as well. `pkg origin` only worked for it after I installed it.


----------



## bakul (May 25, 2022)

grahamperrin said:


> I already have full-flavoured Falkon
> I don't want the inferior flavour.


I guess you'll need a more complicated script! Put flavored packages on lines by themselves?
`$ cd /usr/ports/www/falkon
$ make package-name
falkon-22.04.1
$ make FLAVOR=qtonly package-name
falkon-qtonly-22.04.1`
Ideally language version dependency would be done differently to allow for the simple version....


----------



## sidetone (May 25, 2022)

How about with packages only, without make, building ports or port options?

Edit: in response to


bakul said:


> $ cd /usr/ports/www/falkon
> $ make package-name





> $ make FLAVOR=qtonly package-name


----------



## Sergei_Shablovsky (May 25, 2022)

SirDice said:


> If you look at an existing server, to get a list of packages to install, look at the output from `pkg prime-list`, or `pkg prime-origins`. You don't want to install _everything_, you only want to install the packages you need. Dependencies can change over time and they will automatically be installed, so you don't need to install those yourself.


Extra one tough come to me: about *quarterly vs latest*.

There are always a lot of holy wars on any forums about “*old but stable, with several bugs that I already have workaround*”
vs
“*latest with some improvement and new features, annoying bugs fixed and a few little new bugs*”.

Life in IT always going speed up, developers always have a short time to fix bugs, and making new features. For a various of reasons (including SaaS, DevOps paradigm, sharing code platform like GitHub, Stackexchange, Stackoverflow, etc) that we not need to discuss here.

As a result *we always constantly have software with some amount of bugs*, so difference between “quarterly” and “latest” strategies *for systems with big community and deployment base* (like FreeBSD are) just become to nothing.

So, may be correcting the /usr/local/etc/pkg/repos/FreeBSD.conf as described
Change between `quarterly` and `latest` package set used by `pkg` tool in FreeBSD and ​How to install the latest version of software in FreeBSD? would be also great?​


----------



## Sergei_Shablovsky (May 25, 2022)

sidetone said:


> How about with packages only, without make, building ports or port options?


This is what we starting at the first of this tread: automated script. To manually install what a You need, BUT all modifications of system config files (if they needed), You may doing yourself manually.
So this is “semi-automation”, and may be good for home users with a lot of free time to manual operations on one machine.

As next step the same script may be used in Ansible with addition to another scripts to doing needed modifications of  system files/configs (like loader.conf, rc.conf, etc....) for each of installed packages.
So, in this case we have “fully automated” system.

P.S. Because this certain case is about installing on *bare metal servers* with *numbers up to 200* (no virtual dynamically created), and we also be happy to use the same system to operate with *active network applience*, we not need complexity with a reactions on environment/inputs and IFTTT logic realization.
So in this certain case powerful orchestrate system like Chef, CFengine, Salt, StackSorm not needed.


----------



## sidetone (May 25, 2022)

Sergei_Shablovsky said:


> This is what we starting at the first of this tread.


That was in response to bakul.


> $ cd /usr/ports/www/falkon
> $ make package-name





> $ make FLAVOR=qtonly package-name


----------



## bakul (May 25, 2022)

sidetone said:


> How about with packages only, without make, building ports or port options?
> 
> Edit: in response to


This was already covered earlier by SirDice. There is no simple solution as for example python packages currently start with "py38-" and those packages will fail once python moves to version 3.9. May be you can use category/package-name and flavored package names and that would work -- except for the case where flavored packaged depend on a python version! For this more general case you need something more complicated, such as mapping from category/package-name to package-name-flavor by using make.


----------



## Sergei_Shablovsky (May 25, 2022)

bakul said:


> What I am trying to tell you is, it doesn't matter. Stop reading the menus pasted outside restaurants; go inside one and eat! Find out from your own experience what works and what doesn't. in Patanjali's Yoga Sutra, the seventh Sutra says "right knowledge comes from direct experience, inference or testimony (of authority -- wise people or the scriptures)". Notice how direct experience is the first recommendation! And do not treat random people as authority! Even test what so called "authoritative" sources tell you!


Hm... Some time ago I study in University and live in India in several different places. (This is as a background for this reply).

And I need to say: if You have a lot of time and have a fun from trying that, than trying that, no matter is this crashing system or not, - Your arguments are ok.

But a lot of technical forums nowadays exist to eliminate time and risks in everyday tech work. So, *if the case are more or less trivial* - better not loose time (both Yours and employer) and asking the peoples who was SPENDING A DECADES to be PRO.


----------



## sidetone (May 25, 2022)

I already got that, and already acknowledged that about using `pkg origin`.

I wanted to know if there was (or wasn't) a way to go with flavors regarding pkg and `pkg origin` without `make`.

@ bakul


----------



## bakul (May 25, 2022)

Sergei_Shablovsky said:


> And I need to say: if You have a lot of time and have a fun from trying that, than trying that, no matter is this crashing system or not, - Your arguments are ok.
> 
> But a lot of technical forums nowadays exist to eliminate time and risks in everyday tech work. So, *if the case are more or less trivial* - better not loose time (both Yours and employer) and asking the peoples who was SPENDING A DECADES to be PRO.


In my view, it is more a question of attitude than time. These forums may help you eliminate wasted time and reduce risks in the short term but won't give you a real insight or an ability to gain insight. It is like always eating junk food. It stops being satisfying after a while -- may be a long while! Now if you want to always remain a student (who has to ask) and never be a master then it doesn't matter. But if you want to master, wouldn't be better to practice on simple things first?!

Anyway, we are off on a tangent so I will stop here!


----------



## Sergei_Shablovsky (May 25, 2022)

SirDice said:


> Practice makes perfect
> 
> sko already mentioned it, if you're looking for ways to automate server installation and configuration you should invest time in learning chef/puppet/ansible.


As I wrote before, in certainly this project:
1. we already have a lot of active network applience, mixed vendors, and only one common they have,- COM & SSH client inside. Only few have orchestrate client inside;
2. we already have some amount of bare metal servers, several well-known vendors, all of them have Integrated Management Module, so, even replacing crashed HDD and remote loading from mounted memstik possible;



SirDice said:


> I learned Puppet a long time ago[*] and have grown to like it (it has a rather steep learning curve in my opinion). Ansible is a bit easier to get started with. It's also quite popular at the moment and many companies are migrating their Puppet infrastructure to Ansible.


I definitely agree that Ansible are easiest orchestrate system from bunch of others.
And another BIG advantage (in addition to *ability work with almost all active network applience, because only need SSH server built in applience*) for SysAdmins are A *LOT OF MODULES, EXAMPLES & DOCS.*

Migrating most experienced SysAdmins, and services companies to Ansible based on several things:
- easy to start with (a lot of docs, examples, solutions);
- not locked to certain network appliances vendors (because needs of clients on appliances), this also impact TOC; 
- easy to migrate from other old systems for experienced SysAdmins;
- for small companies with up to 200 bare metal servers units and up to 100 physical network appliances, with not so big and complexity infrastructure - Ansible would be Ideal;

If I wrong here, Il happy to read Your correction.


SirDice said:


> Learning Puppet or Ansible is also quite beneficial for other operating systems and will definitely increase your position on the unix admin job market.


Totally agree.

Little bit off-topic:


SirDice said:


> The "traditional" admin jobs are dying out, it's all devops and infrastucture-as-code nowadays.


I still thinking, the political and economical changes on the Far East, Europe and USA, increasing numbers of global outage, increasing numbers of hackers attacks on cloud infrastructures would pushing to situation where small/middle companies stepping back to using own facilities. 

I do not remember authors name, in one book about high-loading reading calculation when after numbers of servers that company need to have come over 20, much better using own small infrastructure, from TCO, management, stability, security, enhancement point of view. 



SirDice said:


> [*] In the past 5-6 years I've been writing puppet code for most of my working days.


Could You be so please to describe in short for what exactly type of solutions?


----------



## Sergei_Shablovsky (May 25, 2022)

sko said:


> consider using something like sysutlis/ansible





Sergei_Shablovsky said:


> 3.
> Which ansible2 or ansible I need for FreeBSD 13 ?


?


----------



## shkhln (May 25, 2022)

Any Ansible. (And learn you some grammar, by the  way. Or at least stop capitalizing "you".)


----------



## astyle (May 25, 2022)

Some workplaces maintain sort of a 'template' image that they can quickly deploy.  This may work for user-facing workstations, but servers are a different beast.  You may have one box for email, another box hosting fileshares, yet another for firewall, and yet another for DNS. Yeah, they may have some common base (like all running same version of FreeBSD, like 13-RELEASE), and you want to have a good backup scenario so that you can recover quickly and be back up and running. Trouble is, that 'common base' is actually awfully minimal. FreeBSD already provides a bare-bones install image where you just need to turn on SSH and DHCP, but beyond that, an admin would need to maintain a backup copy of friggin' EVERYTHING. Automating the re-install procedure helps some, but it's still necessary to keep track of what goes where. This is partly why I kind of soured on Puppet and Ansible - Not only you gotta maintain the production stuff, you also gotta maintain a copy of it, which is frankly double the workload, even with those config managers helping out.   Even if you manage to set it up to make it not quite double the workload, where do you store all that extra data?


----------



## sko (May 26, 2022)

astyle said:


> where do you store all that extra data?


With ansible it's only textfiles. I put it (as almost all config-related stuff) in git-repositories hosted on a local server.
Our backups are zfs snpashot-based, so recovery is pretty straightforward, and with configurations in git-repos we always have another path for recovery if snapshots should somehow fail.


----------



## Sergei_Shablovsky (May 26, 2022)

sko said:


> Our backups are zfs snpashot-based, so recovery is pretty straightforward, and with configurations in git-repos we always have another path for recovery if snapshots should somehow fail.


Could You be so please explain how to organize automated backups in case we need “current snapshot” of whole system ?

Where better to read about ZFS snapshotting (with real working examples)?


----------



## Sergei_Shablovsky (May 26, 2022)

astyle said:


> Some workplaces maintain sort of a 'template' image that they can quickly deploy.  This may work for user-facing workstations, but servers are a different beast.  You may have one box for email, another box hosting fileshares, yet another for firewall, and yet another for DNS. Yeah, they may have some common base (like all running same version of FreeBSD, like 13-RELEASE), and you want to have a good backup scenario so that you can recover quickly and be back up and running. Trouble is, that 'common base' is actually awfully minimal. FreeBSD already provides a bare-bones install image where you just need to turn on SSH and DHCP, but beyond that, an admin would need to maintain a backup copy of friggin' EVERYTHING.


So, the making snapshots - is only solution?. (Especially HDD space are quite cheap nowadays...)



astyle said:


> Automating the re-install procedure helps some, but it's still necessary to keep track of what goes where. This is partly why I kind of soured on Puppet and Ansible - Not only you gotta maintain the production stuff, you also gotta maintain a copy of it, which is frankly double the workload, even with those config managers helping out.


I still thinking for really big and geographically spread infrastructure (like 500+ servers, 1000+ net devices) Salt are really better because server-client architecture, when each client tracking changes on server/device and give this info to central mgmt server.


----------



## astyle (May 26, 2022)

sko said:


> With ansible it's only textfiles.


I'm not talking about the config files. You also need to maintain all the data that the config files point to. File backups, specific software versions, ZFS snapshots, etc. Or are you gonna go through the entire process of re-acquiring LibreOffice for FreeBSD 11-RELEASE by taking your chances that the correct versions are still available in public git repos? Even if you can automate things with Ansible or Puppet, there's just no way to avoid maintaining your own archive for quick disaster recovery. And proper disaster recovery strategy really means having 2 or 3 copies in as many different places, to boot.   Gotta think things through, all the way through.


----------



## astyle (May 26, 2022)

Sergei_Shablovsky said:


> I still thinking for really big and geographically spread infrastructure (like 500+ servers, 1000+ net devices) Salt are really better because server-client architecture, when each client tracking changes on server/device and give this info to central mgmt server.


That works a bit different... client devices only track their own changes, and report them back to the management console on the server.  Gotta pay attention to how the data even moves.


----------



## sidetone (May 26, 2022)

Seed files backed up on a flash disk and/or harddisk storage partition. Having it on both is better. On this install, I simplified my understanding of needed files, and which settings I had stored which weren't essential. Instead of having them all in separate subdirectories, I can have them all in one directory with additional prefixes or suffixes, then figure out where they go later, when I need them.

/etc/rc.conf and /boot/loader.conf go a long way. Then, custom config files in home directories. Crontab files, KERNCONF, and make.conf, if there are any, as well. A lot of old arguments that I had on my old configuration options are obsolete. I can figure out which kinds of settings I really needed.

Source files use dots "." as placeholders for additional directories, which automated scripts translate to the needed slashes "/" and subdirectories. This can be seen in /usr/src/ (if you have it) in reference of bin and sbin.



astyle said:


> are you gonna go through the entire process of re-acquiring LibreOffice for FreeBSD 11-RELEASE by taking your chances that the correct versions are still available in public git repos?


A program like that from a dedicated organization will always be there. When that goes away, there will be news about that. Then, some other organization will pick it up, because its license allows it. If a program did become defunct upstream, it would still need security updates from somewhere. If any version is always available, then it should work, or it should work with a production version of FreeBSD anyway. If a repo changes, maintainers and other users of it will address that and try to fix it. I'm not sure if I understood you on this.

edit: in reference to opensource, as LibreOffice is.


----------



## astyle (May 26, 2022)

At my workplace, there are copies  of some old software archived. Licenses were bought at the time that software was new. It was a business decision to just keep an old copy of that software for everybody's use. That old version is no longer on the market, and up-to-date stuff - it would be  incredibly expensive to keep buying the licenses.  This is the kind of stuff that the ansible/puppet config files need to point to. And yeah, it's a lot of work to make sure the server the archive lives on is backed up and maintained on a regular basis.

Even FreeBSD Foundation doesn't exactly go out of its way to announce that, for example, 11-RELEASE is already EOL, and no longer supported. It would not make tech news. I don't expect LibreOffice to make such news, either - MS is about the only organization with a sufficiently large installed base who'd be able to make tech news by declaring that a version of Office is no longer supported (like Office 2007 or 2010). Frankly, even Wolfram Mathematica or SAS, huge companies in their own right, they don't have the clout to make news about dropped support for a specific version.


----------



## sidetone (May 26, 2022)

For closed source software, where re-downloading isn't available for free, or where a company may drop support for a software, or a company can go out of business, it always makes sense to have backups. For opensource products supported by a major backer, a lot about that doesn't apply to it.

For a client company, where it's difficult to update many computers to a newer FreeBSD version, keeping backups of packages of opensource products would be relevant. An OpenOffice version is going to be supported, and it may be for any available supported FreeBSD, even if it becomes limited to the latest one in production.

Users have a general idea of when a FreeBSD version is going EOL.

That case may be limited and relevant to short term, especially relevant to bugs and security vulnerabilities. Perhaps like when there was a problem for majorly used pieces of software like Firefox and Thunderbird that wouldn't build properly. People discussed that on mailing lists, bug reports, and forums. A major organization couldn't afford to have that go down for a few days, and they would have a backup of anything that could do, including what you suggested. Vuxml (vulnerability/security) warnings for ports would be another issue, where a backup or also ignoring the warning to bypass building it would be enough to run it, but not enough to overcome what the security warnings were about. An alternate port would have to be used anyway, or they would have to weigh the risks of running it.


----------



## sko (May 26, 2022)

astyle said:


> I'm not talking about the config files. You also need to maintain all the data that the config files point to. File backups, specific software versions, ZFS snapshots, etc. Or are you gonna go through the entire process of re-acquiring LibreOffice for FreeBSD 11-RELEASE by taking your chances that the correct versions are still available in public git repos? Even if you can automate things with Ansible or Puppet, there's just no way to avoid maintaining your own archive for quick disaster recovery. And proper disaster recovery strategy really means having 2 or 3 copies in as many different places, to boot.   Gotta think things through, all the way through.


Why would I want to keep old cruft around? 
The whole point of using something like ansible is to (re)build hosts with the latest packages and not from images with outdated or even EOL software.


----------



## astyle (May 26, 2022)

sko said:


> Why would I want to keep old cruft around?
> The whole point of using something like ansible is to (re)build hosts with the latest packages and not from images with outdated or even EOL software.


Then just maintain one image with up-to-date software, and dd that image onto other hosts' hard drives (not over the network, but over SATA/PCIe/USB). No need for no stinkin' complexity that ansible introduces, then.


----------



## sko (May 26, 2022)

did you actually use ansible?


----------



## bakul (May 26, 2022)

sko said:


> The whole point of using something like ansible is to (re)build hosts with the latest packages and not from images with outdated or even EOL software.


I would think the point of automation such as ansible would be to build *repeatable* system state, not necessarily the latest. That is, if you run the same playbook (or whatever) 3 times, each time you should get exactly the same thing. Especially in production you don’t want surprises. I haven’t used ansible so this is just speculation.


----------



## Sergei_Shablovsky (May 27, 2022)

astyle said:


> I'm not talking about the config files. You also need to maintain all the data that the config files point to. File backups, specific software versions, ZFS snapshots, etc. Or are you gonna go through the entire process of re-acquiring LibreOffice for FreeBSD 11-RELEASE by taking your chances that the correct versions are still available in public git repos? Even if you can automate things with Ansible or Puppet, there's just no way to avoid maintaining your own archive for quick disaster recovery.


What about next “hw failure backup strategy”:

Inside case of each server there in motherboard (rarely on RAID controller PCIe card) are 1-2 USB free connectors who which You may attach 8/16/32 Gb flash memstick.

In case of main RAID card die, boot drive die or corrupted, system follow boot order and boot from this “internal memstick”.
(Booting from PXE is another things, because infrastructure fw applience may prohibit this option, or not working correctly).

This memstick are GPT (UEFI/BIOS), that:
1. it is custom made bootable FreeBSD RELEASE + post-install scripts memstick;
2. contain snapshot archive of working system of same server;
3. contain playbooks for Ansible for installing/updating/tuning all needed things in FreeBSD;

And after server was completely restored and working normally, there are croned shell script to creating and rewrite this “internal memstick”.

P.S. Is this method great for creating *whole* BSD system (not whole drive) snapshot and store on internal memstick + copy by VPN/SSH to cloud storage?
If not, please suggest the better way.


----------



## astyle (May 27, 2022)

Sergei_Shablovsky said:


> P.S. Is this method great for creating *whole* BSD system (not whole drive) snapshot and store on internal memstick + copy by VPN/SSH to cloud storage?
> If not, please suggest the better way.


Read that thread very carefully.  Even OP asks about checking the integrity of the backups, and is aware of that important ptifall in the method.

With backups, I'd prefer to deal with whole files, rather than streams of bits. First have a usable backup created locally, and then copy that anywhere you like. With streams of bits, you run a much higher risk of corrupt backups. Streaming is OK for Netflix movies, but I'd avoid streaming as a backup mechanism. Files need to be whole every step of the way.


----------



## Sergei_Shablovsky (May 27, 2022)

astyle said:


> Read that thread very carefully.  Even OP asks about checking the integrity of the backups, and is aware of that important ptifall in the method.
> 
> With backups, I'd prefer to deal with whole files, rather than streams of bits. First have a usable backup created locally, and then copy that anywhere you like. With streams of bits, you run a much higher risk of corrupt backups. Streaming is OK for Netflix movies, but I'd avoid streaming as a backup mechanism. Files need to be whole every step of the way.


Totally agree with You.

What the best way You suggest for creating “snapshot” of whole working system?

Because we discussing about servers, a lot of files be created/changed/deleted during making “snapshot” of a working system.
So creating snapshot “on a fly” may be not possible...

Is the only one option when pulling server from work on maintenance time -> booting from removable memstick -> creating snapshot of whole system -> check snapshot consistency -> reboot server and push it back to work environment ?


----------



## astyle (May 27, 2022)

Sergei_Shablovsky said:


> Totally agree with You.
> 
> What the best way You suggest for creating “snapshot” of whole working system?
> 
> ...


Study ZFS in the Handbook. It's perfectly possible to create a snapshot of a working system on the fly if you use ZFS.  ZFS does have a very different design than UFS. 

Study your systems, diagram out your plans, and find ways to practice your ideas.  Try to connect the dots between documentation and 'best practices' to what you actually have on your systems. Buzzwords you see on the Internet are meaningless if you can't mentally make a connection to the systems you actually control.


----------



## Sergei_Shablovsky (May 27, 2022)

astyle said:


> Study ZFS in the Handbook. It's perfectly possible to create a snapshot of a working system on the fly if you use ZFS.  ZFS does have a very different design than UFS.


ZFS on all systems already.

I read a lot on forums about ZFS ideology, how ZFS used in RAIDs and make conclusion this is the best (stability, installbase, support community, hw vendors support, etc...) filesystem for the next 7-10 years. 

I just asking opinion about some toolset, that may be usable.



astyle said:


> Study your systems, diagram out your plans, and find ways to practice your ideas.  Try to connect the dots between documentation and 'best practices' to what you actually have on your systems.


Agree. This is my way. 



astyle said:


> Buzzwords you see on the Internet are meaningless if you can't mentally make a connection to the systems you actually control.


Agree. The same feeling.


----------



## astyle (May 27, 2022)

Sergei_Shablovsky said:


> ZFS on all systems already.


Well, the initial setup method still matters. It starts at fresh system install. 

Elsewhere on these forums, I've seen confusion between a couple methods:

using using ZFS for the whole disk from get-go
partitioning the disk using other 3rd-party tools, and trying to tell the FreeBSD installer to find those partitions and use ZFS instead of UFS at that step.
The first method is frankly simpler, saves a LOT of prep-work, and opens up amazing flexibility down the road - not even Linux can boast that. And yes, that flexibility includes taking a snapshot of the system on the fly.

Once you have those snapshots, then you can organize your backups.


----------



## getopt (May 27, 2022)

Sergei_Shablovsky said:


> ZFS *ideology*


I'd prefer to use the term *technology *instead.


----------



## astyle (May 27, 2022)

getopt said:


> I'd prefer to use the term *technology *instead.


There's a difference between *ideology*, *technology*, and *design*. I see the etymology, but at this point, it's just nitpicking on natural language technicalities. My preferred term is *design logic*. I do see design logic dictating how the filesystem is meant to be used.


----------



## getopt (May 27, 2022)

astyle said:


> it's just nitpicking on natural language technicalities.


With all respects in this case it is not nitpicking.



			
				https://en.wikipedia.org/wiki/Ideology said:
			
		

> An *ideology* is a set of beliefs or philosophies attributed to a person or group of persons, especially as held for reasons that are not purely epistemic, in which "practical elements are as prominent as theoretical ones." Formerly applied primarily to economic, political, or religious theories and policies, in a tradition going back to Karl Marx and Friedrich Engels, more recent use treats the term as mainly condemnatory.


----------



## astyle (May 27, 2022)

getopt said:


> With all respect in this case it is not nitpicking.


Yeah, looks like I was conflating a couple terms: logic and ideas.  When technical ideas have logical alignments, I'd imagine that design is a more appropriate English term. Both Russian and German natural languages have their share of long words that are a fusion of 2 or even 3 separate words that have been shortened or otherwise modified for convenience of pronunciation. That modification is often at the base of puns and mis-understandings.


----------



## Sergei_Shablovsky (May 28, 2022)

Bobi B. said:


> Don't use bash(1), as you'll have to install it 1st. Perhaps you can base it on this
> 
> ```
> #!/bin/sh
> ...



How to add in this script to “pkg_autoinstaller_” the name of host (for example “server.lical.net”) ?

I try to seek in a Global Environment Variables, but unsuccessfully...


----------



## Bobi B. (May 28, 2022)

Sergei_Shablovsky said:


> How to add in this script to “pkg_autoinstaller_” the name of host (for example “server.lical.net”) ?
> 
> I try to seek in a Global Environment Variables, but unsuccessfully...


Use something like

```
command | tee pkg_autoinstaller_$(hostname -f)_$(date +%Y%m%dT%H%M%S).log
```
For csh(1) replace `$(subshell-command)` with ``subshell-command``. For example `echo pkg_autoinstaller_`hostname -f`_`date +%Y%m%dT%H%M%S`.log`.
You can even run pkg(8) remotely: `control# cat pkg_list.txt | ssh root@remote env ASSUME_ALWAYS_YES=yes xargs pkg install | tee pkg_autoinstaller_remote_`date +%Y%m%dT%H%M%S`.log`. This way cat(1), tee(1) and date(1) will run on the `control` (local) host, whereas pkg(8) will run on the `remote` host.


----------



## Sergei_Shablovsky (May 28, 2022)

According “ideilology, technology, design..” discussion: astyle, getopt, gentlemens, may I buy a bottle of whiskey for both of You to make discussion more comfortable because a weekend? 

My mistake, the word “design” would be right in that exactly context.


----------



## Sergei_Shablovsky (May 28, 2022)

Bobi B. said:


> You can even run pkg(8) remotely: `control# cat pkg_list.txt | ssh root@remote env ASSUME_ALWAYS_YES=yes xargs pkg install | tee pkg_autoinstaller_remote_`date +%Y%m%dT%H%M%S`.log`. This way cat(1), tee(1) and date(1) will run on the `control` (local) host, whereas pkg(8) will run on the `remote` host.


Thank You!
For things like this I prefer Ansible rather typing in Terminal app, less possible to make mistyping...


----------



## Bobi B. (May 29, 2022)

For installing and updating many servers with uniform roles take a look at nanobsd. Works with filesystem "images" and enables image testing in advance, plus robust upgrades and rollbacks (version updates are close to atomic and require a single server reboot for the new version to be ran; in fact if kernel version remains the same, `reboot -r` works once, as it appears that kernel does not unlock the old filesystem prohibiting further updates until a real reboot). Not covering all the bases nor working in every case, but is very solid. Couple of years ago I got the impression Netflix uses nanobsd (or something similar) to manage their CDNs. Found it interesting, implemented nanobsd at work and never went back.


----------



## Sergei_Shablovsky (Jun 2, 2022)

Bobi B. said:


> Don't use bash(1), as you'll have to install it 1st. Perhaps you can base it on this
> 
> ```
> #!/bin/sh
> ...


Next small step to improve the automated script:
*how to automatically choose the package list accordingly the name of server*?

Example:
In work directory (on a usbmemstick or internal disk) exist
*pkg_autoinstaller.sh*
and a set of pkg list files
*pkg_list_server1.local.net.txt
pkg_list_server2.local.net.txt
...*

Script look at the server hostname in FreeBSD, than try to automatically finding appropriate pkg list file and installing all listed pkg.
If appropriate pkg list file not finded, script propose to choose pkg list manually OR entering server new hostname, apply changes to FreeBSD and then loading appropriate pkg list file.

Script onscreen menu in this case:
———
[ERROR] Package Autoinstaller not able to find pkg list that linked to this server.
You may choose pkg list manually or correct the server hostname to install appropriate packages.

1. server1.local.net
2. server2.local.net
...
C. Correct hostname (freebsd.local.net) of this server to install appropriate packages.

Enter You choose:
———

So User have to enter “1”, “2”, ... or “server2.local.net” and hit Enter. After that script doing the rest.


----------



## Alain De Vos (Jun 2, 2022)

My script is :

```
export P="/jails/a/pou/mypackages"
pkg update -f
cat $P | grep -v pkg | xargs -I {} sh -c "echo {}; pkg install  -y  {}"
pkg update -f
pkg upgrade
```


----------



## Bobi B. (Jun 2, 2022)

```
#!/bin/sh
host=$(hostname -f)
pkgdir=~/test1 # where .txt files are
pkgfile=${pkgdir}/pkg_list_${host}.txt
if [ ! -s ${pkgfile} ] ; then
    echo "No packages list file (${pkgfile}) for host '${host}'"

    pkgfile=
    while [ -z "${pkgfile}" ] ; do
    echo "Which list file to use?"
    n=0
    for fn in ${pkgdir}/*.txt; do
        n=$((n+1))
        printf " %2d. %s\n" ${n} ${fn#${pkgdir}/}
    done

    sel=
    while :; do
        printf "Enter file number to use: "
        read sel
        [ "${sel}" -eq "${sel}" 2>/dev/null ] || continue
        [ ${sel} -gt 0 -a ${sel} -le ${n} ] || continue
        break
    done

    n=0
    for fn in ${pkgdir}/*.txt; do
        n=$((n+1))
        if [ ${n} -eq ${sel} ] ; then
        pkgfile=${fn}
        break
        fi
    done
    done
fi
echo "Using ${pkgfile}"
```
But have you considered using role-based packages lists? Also using interactive install scripts kind-of defeats the purpose


----------



## Sergei_Shablovsky (Jun 2, 2022)

Bobi B. said:


> ```
> #!/bin/sh
> host=$(hostname -f)
> pkgdir=~/test1 # where .txt files are
> ...


Thank You so much! 

What about last “C” option for changing hostname, apply settings and start script from scratch ?



> C. Correct hostname (freebsd.local.net) of this server to install appropriate packages.





Bobi B. said:


> But have you considered using role-based packages lists? Also using interactive install scripts kind-of defeats the purpose


Normally whole script intended for running manually (or with help of Ansible), it just find appropriate pkg list file, run all installs, save log file (and Ansible pull it back together with sys log). Done.
Useful for admins who have no ZFS snapshots, whole drive backups, Ansible, and just have usb memstick with fresh FreeBSD install image and up to 100 FreeBSD boxes (colleges, campus, etc...)

This additional code needed in case
- server have default FreeBSD hostname (operator mistake during bare metal setup from memstick, for example);
- server have the name that not recognized as part of  infrastructure;
Script waiting 20sec, after that if no entry, write “[ERROR] Appropriate pkg list not found for $(hostname -f)” to log file, write same to syslog, and exit with code “1”.

Of course, for bulk installing may be some pkg list file that named according default FreeBSD hostname. Like template...

So it would have fully automated behavior.

Could You be so please to correct code (adding “C” option + exit after 20s no entry) when having coffee break?

At the end of the week we all receive a nice script for making package list file and automatic installation. To not installing manually from scratch...
Each one of users here care about 3+ FreeBSD boxes (home, work, friends, etc), so script may be usable...

P.S. After You done, I would adding “S” option to selection and “-s” parameter to “making packages list and exit”.



> S. Make list of packages that installed on this server.


----------



## astyle (Jun 2, 2022)

You do need to have a working repo on the other end. If you build it yourself, there are tools available to take care of LOTS of details for you. One such tool is Poudriere. But  from a personal and incomplete experience, I can say that Poudriere is a VERY unwieldy beast.


----------



## Bobi B. (Jun 3, 2022)

Sergei_Shablovsky said:


> Could You be so please to correct code (adding “C” option + exit after 20s no entry) when having coffee break?


Let's not confuse charity with work  
What you're asking will take a bit more time than a coffee break.


----------



## astyle (Jun 3, 2022)

Yeah, OP's gotta be willing to do the work to adjust the available code to fit his own scenario. I do that all the time, I don't expect a runnable line of code, just some helpful ideas of what might work, and then I do the rest myself.


----------



## Sergei_Shablovsky (Jun 3, 2022)

Bobi B. said:


> Let's not confuse charity with work
> What you're asking will take a bit more time than a coffee break.


Sorry, I cannot even have a thought about using You!

Just from the point of view that something on which I spend 3-7h, for You may be 20 min. Sorry again. I go to seeking Stackoverflow and Google...


----------



## Sergei_Shablovsky (Jun 3, 2022)

astyle said:


> Yeah, OP's gotta be willing to do the work to adjust the available code to fit his own scenario. I do that all the time, I don't expect a runnable line of code, just some helpful ideas of what might work, and then I do the rest myself.


Agree with You. 

Because I using FreeBSD a long time ago before now, I forgot all details. And now in time of war in Ukraine, I try to solve the problem because no sufficient time to learn. 

Sorry again one time. And thank You and all here for help!


----------

