# Good reading sources for FreeBSD scripting for dummies??



## PacketMan (Apr 19, 2018)

Good day gentlemen,

Slowly but surely I progress in my knowledge of the OS.  Now the time has come for me to learn how to write my own scripts.  I did some searching on this site, but didn't find any discussion on sources of good material for 'how-to' script writing.

I'll do some internet searching too, and I got a couple older FreeBSD books at the house which I will look through soon too, but I though you folks would be best to provide some comments.

As always, thanks in advance,
P.M.


----------



## Minbari (Apr 19, 2018)

sh - the Bourne Shell


----------



## SirDice (Apr 19, 2018)

Minbari said:


> sh - the Bourne Shell


Probably the best one, I still use it quite regularly for reference.


----------



## ShelLuser (Apr 19, 2018)

PacketMan said:


> Slowly but surely I progress in my knowledge of the OS.  Now the time has come for me to learn how to write my own scripts.  I did some searching on this site, but didn't find any discussion on sources of good material for 'how-to' script writing.


Start at the basis, as hinted at by Minbari: determine what scripting actually is.

In its essence you'll be creating shell scripts, and as its name more or less implies this means you'll be using one of the many available shells (most likely the Bourne shell because that's the most common) and then using a script to instruct that shell to run one or more commands.

So what you need to do is look into whatever shell it is you're going to use. My recommendation would be to start by looking into the Bourne shell (/bin/sh) simply because that's the de-facto standard on FreeBSD. Every FreeBSD installation will have /bin/sh and/or /rescue/sh at its disposal. Which also means that scripts written on server 'A' will also run without too much problems on server 'B' (assuming you don't use software which is available on one but not the other).

My suggestion: manual pages, aka `man sh`, which will point you to sh(1). It can be tricky to get into but once you grasped the syntax and the synopsis then this will automatically become _one heck_ of a reference resource. Always available on whatever FreeBSD box you log into.

And my next suggestion: don't rely on dry theory too much, find things to automate and then also actually start automating things. Create ~/bin (bin subdirectory in your home directory), add it to the search path and start scripting away.


----------



## ondra_knezour (Apr 19, 2018)

I have Serious shell programming by Devin Teske in the queue for some time.


----------



## Oko (Apr 19, 2018)

PacketMan said:


> Good day gentlemen,
> 
> Slowly but surely I progress in my knowledge of the OS.


Have you read and understood The Unix Programming Environment by Brian W. Kernighan  and Rob Pike.

https://www.amazon.com/Unix-Programming-Environment-Prentice-Hall-Software/dp/013937681X

If you have not we have nothing to talk about.




PacketMan said:


> Now the time has come for me to learn how to write my own scripts.  I did some searching on this site, but didn't find any discussion on sources of good material for 'how-to' script writing.
> 
> I'll do some internet searching too, and I got a couple older FreeBSD books at the house which I will look through soon too, but I though you folks would be best to provide some comments.
> 
> ...



My favorite book on shell programming is
Unix Shell Programming (3rd Edition) by Stephen G. Kochan and‎ Patrick Wood

https://www.amazon.com/Unix-Shell-Programming-Stephen-Kochan/dp/0672324903

Here are further resources

http://www.shelldorado.com/

and my favorite

http://www.softpanorama.org/

No I am not NIkolai Bezroukov and I actually have never met him but I share many of his strong opinions.

Minbari gave you are a very good link as well!


----------



## unitrunker (Apr 20, 2018)

Click on the "Read" button.

https://legacy.gitbook.com/book/freebsdfrau/serious-shell-programming/details

quote:

"All currently-maintained UNIX/Linux Operating Systems have a POSIX-compliant /bin/sh. If you want to write shell scripts that run on any UNIX/Linux system, this is the book for you."


----------



## skhal (Apr 20, 2018)

sh(1) -- must read + look at FreeBSD code
"The C Programming Language" by K&R + read FreeBSD basic utilities


----------



## SirDice (Apr 20, 2018)

skhal said:


> "The C Programming Language" by K&R


How is a book about C going to help learn shell scripting? It's a good book but it's the wrong topic.


----------



## PacketMan (Apr 20, 2018)

Thanks everyone, so far so good.



ShelLuser said:


> ....find things to automate and then also actually start automating things.



As some of you may recall I bought some IP security cameras, which simply dump (via FTP) their jpeg images into a folder. Then I use net-p2p/btsync  to distribute to my other servers, and my mobile devices. It's the slickest thing since sliced bread.

However I bought a new HD camera recently that does HD quality images and videos, and still uses FTP. However the cameras use of the network is horrible. It ignores the path I set, simply goes to /home, and in there creates a slew of directories, and then to make matters worse its file naming convention is just awful. So in my crontab I use multiple lines to make things work. But the real dilemma is the file naming convention prevents each camera (of this manufacture and type) from being unique with other cameras of same. So if buy a 2nd or 3rd camera, we will have filename conflicts. So I said to myself, well maybe I won't buy that brand anymore. And I just might not. But then I also said, well here is a perfect opportunity to learn scripting, use that to fix my problem, and then also use that to fix what some would say is a 2nd problem; a messier than it needs to be crontab that is also doing the commands / work.

So again thanks to everyone for your comments, and certainly more are welcome for the benefit of other people too.


----------



## skhal (Apr 20, 2018)

SirDice said:


> How is a book about C going to help learn shell scripting? It's a good book but it's the wrong topic.



Oh, sorry, I didn’t realize it was shell-only.  Obviously, K&R is not going to help much.

However, the book sheds some light on internals and how the applications are written, which completes the global pucture.

I think it useful to read both, but if one is only interested in shell, then sh(1) and FreeBSD source code in GitHub are probably the best sources of information.


----------



## PacketMan (Apr 20, 2018)

skhal said:


> Oh, sorry, I didn’t realize it was shell-only.



Me either. But I did know that scripting via shells is very popular, assuming its not the only way of doing scripting.  For my needs which are basic, and of learning purposes, shells will be more than adequate for the time being.

Thanks again.


----------



## SirDice (Apr 20, 2018)

PacketMan said:


> But I did know that scripting via shells is very popular, assuming its not the only way of doing scripting.


Shells are certainly not the only way for scripting, there's also Perl, Python and Ruby for example. These are also seen as scripting languages.



PacketMan said:


> For my needs which are basic, and of learning purposes, shells will be more than adequate for the time being.


It's definitely a good start. Shell scripting is extremely useful for automating relatively simple, repetitive, tasks.


----------



## PacketMan (Apr 20, 2018)

Always a pleasure SirDice to read your '_confirmations of thought_'.


----------



## Bobi B. (Apr 20, 2018)

Remember, that the POSIX shell and utilities are (almost?) always installed and readily available, whereas Perl and/or Python might not be.


----------



## sko (Apr 20, 2018)

Minbari said:


> sh - the Bourne Shell





SirDice said:


> Probably the best one, I still use it quite regularly for reference.



I fully agree - I also often use it as a quick reference.

For a very comprehensive book on shell scripting I can recommend the "Turtle Book" from O'Reilly: Classic Shell Scripting (Robbins, Beebe)
It's a pretty thorough sweeping blow to anything you need to know for shell scripting, including introductions to all the basic tools usually used with shell scripts and even a whole chapter on awk (aptly named "Enough awk to be dangerous").
The chapter on shell portability is a real gem as it sums up lots of caveats when using non-sh shells on varying systems. Throughout the book differences in bourne shell implementations (especially when they differ from POSIX specs) are also highlighted to save you from falling in one of the many pitfalls with varying (esp. commercial) bourne-shell implementations. So if you aim to write portable scripts I can highly recommend this book.

If some day sh + awk/sed isn't sufficient any more for your needs, have a look at Perl, especially if you also learned (or plan to learn) some C.
The initial transition is pretty straghtforward as you can get surprisingly far with a lot of shell knowledge/habits and some C sprinkled in. I'm pretty sure this is where Perls reputation for ugly code came from, so one should make an effort to really learn proper Pearl - especially because it is a much more powerful language and most tasks that are rather complicated with shell-style code can often be done with just a few lines of "pure" Perl.
For Perl the Camel book still is the definitive reference document, but I can also recommend the series of "smaller Camelids": Learning/Intermediate/Mastering Perl.


----------



## ralphbsz (Apr 20, 2018)

Bobi B. said:


> Remember, that the POSIX shell and utilities are (almost?) always installed and readily available, whereas Perl and/or Python might not be.


In theory, you might be correct.  There could be a machine which has only a Posix-compliant shell, and nothing else.

In practice, in the free software world (Linux, *BSD, ... even Raspberry Pi) that simply will not happen.  Perl and Python are available for all of those; and probably even automatically pre-installed by the default install.  The same is true on the Mac (which is after all nothing but a free installation of a Unix OS that comes with your hardware), perl and python are available and pre-installed.  On the commercial Unixes (AIX, HP-UP, ...) the situation is a little more tricky, as the default distributions that come from the vendors (IBM, HP, ...) sometimes do not put Perl and Python on the machine, but these are always easily available as an add-on. 

Matter-of-fact, you can more rely on Perl and Python than on bash.  Personally, my habit (formed from having worked on projects that often used 3 or 5 Unix flavors simultaneously) is to rely on having perl or python, but then only use the minimal set of Posix standard shell, and not rely on bash.

Why do I say this here?  Because learning shell scripting (meaning sh, not perl/python) is indeed a very good thing, as Sir Dice said.  It's incredibly useful to do off-the-cuff three line scripts right at the command line, or put 20 or 50 line scripts in a little file and use them over and over.  But one has to know the limitations of the technology.  A 1000-line script in shell is a maintenance nightmare, and very hard to get correct and debug.  At that point, one has to think about using appropriate tools, and instead write a 500-line program in perl or python that accomplishes the same thing, but can be written in a much more structured fashion.

So what I'm saying is: definitely learn how to do shell scripting.  Whether you use a book, a web site, or just trial and error is up to you.  But quickly afterwards, learn a scripting language that is more appropriate for larger projects.


----------



## ShelLuser (Apr 20, 2018)

danh said:


> My post was removed for what reason? I guess because it bothered someone that I responded to the tool called Oko.
> 
> What a place. Some bone smoker makes stupid ass comments and has caused people to leave this forum and you all allow the idiot to continue unabated.
> 
> This isn’t the place for me, nor is FreeBSD. You can thank Oko for that. He drives away quite a few.


Goodbye then, and thank you Oko. </sarcasm>

The only reason I'm responding is because you're pretty new and because I know Oko can be direct at times.

If you take offense to his post _in this thread_ then honestly: you're the problem and not Oko. Even your wording here shows as much: if you have so much problems with this particular member then why not just ignore 'm?
Why bring this to us in public? PM a moderator / admin! You talk about a deleted post which I can no longer read for myself, so I'll just have to take your word on it that it didn't violate any rules (hint: posts hardly get deleted around here...). Then I see that you can't even control yourself in calling Oko names here (so childish) and then I can only imagine how abusive your original post had to be (even if it wasn't, Keyword: I can't know for sure!).
See what I'm saying here? This post alone makes you look like the tool, Not Oko. I mean.. he never called someone names here, you did..

Perception and intent. Was Oko direct? Sure. Did he insult anyone? Hardly. Once again: if you did feel insulted: why not ignore him instead of trying to call him out like this?
What was the real reason behind your post? Whining? Trolling? Calling out Oko? Why bother me with this nonsense? PM Oko and share your grievances there. Then again, he might have you on ignore, something you should have done as well.
Don't try to mini mod please... If someone violates the rules the admins will step in.
For the record: after this post I intend to report both yours and mine posts in order to make sure the admins see it.


----------



## Crivens (Apr 20, 2018)

As a moderator, I should see a deleted post. But this thread does not seem to have deleted posts. I checked twice.


----------



## Oko (Apr 20, 2018)

Crivens said:


> As a moderator, I should see a deleted post. But this thread does not seem to have deleted posts. I checked twice.


Yes his post and my answer were removed. I reported his post not because of the flame bite to which I responded in respectful manner with technical facts (it was about use of the "obsolete" editor ed in the book Unix Programming Environment which was used as an argument against my recommendation) but because he had a link to  the copyright infringement material (there was a link to an illegal pdf copy of The Unix Programming Environment by Brian W. Kernighan and Rob Pike). On the top of it he actively encouraged other users to violate copy right laws by downloading illegal copy instead of getting a legal "too expensive" one.


----------



## Bobi B. (Apr 20, 2018)

ralphbsz said:


> In theory, you might be correct.  There could be a machine which has only a Posix-compliant shell, and nothing else.
> 
> In practice, in the free software world (Linux, *BSD, ... even Raspberry Pi) that simply will not happen.  Perl and Python are available for all of those; and probably even automatically pre-installed by the default install...


I see your point and I won't argue. My point is, that for a simple script to automate some basic tasks, IMHO, it is better to use something "guaranteed" to be installed, instead of something that might not be (neither Perl, nor Python were part of FreeBSD base system the last time I've checked). I'm aware that shell is not flexible enough to write something more complex. However in my, fairly limited, Perl experience, apart from the simplest scripts, you'll have to install, therefore depend on, a dozen of 3rd party libraries/modules, as well.


----------



## alexseitsinger (Jul 8, 2018)

Shell scripting is one of my favorites, next to Javascript. The best advice I think you can get is to just start tinkering. Write some simple scripts to automate different tasks, and slowly increase the complexity. You'll get the hang of it quickly. Its helpful to look up references to learn how to form your scripts properly. If you're writing "global" functions, make sure the PATH has your directory included. Otherwise, make sure you source you scripts (. script_name.sh) before trying to run them. Else, execute them using a direct call to the interpreter, ie: "sh script_name.sh". There are some common best practices that I've learned, like double quoting all your variables, or even better, also including them in curly brackets, but that can be confusing at first.

Although BASH isn't the same thing as just SH, and doesn't come built in with FreeBSD, its certainly useful to add/use. BASH supports things like arrays that SH doesn't.

A quick search reveals this gem: Bash Reference

Happy scripting!


----------

