# Knowing WHEN to use higher lv lang



## Seeker (Apr 19, 2011)

When it comes to scripting ... How to know, WHEN to use high level lang (php, ruby, py, perl ...) instead of a low level (/bin/sh)

How to recognize situation?


----------



## SirDice (Apr 19, 2011)

Seeker said:
			
		

> How to know, WHEN to use high level lang (php, ruby, py, perl ...) instead of a low level (/bin/sh)


They're all high level. A low level language would be assembler for example.

But choosing between any of them is tricky and it all depends on the situation and how good you are in using any of the languages.

I prefer to do things in Perl instead of a shell script simply because I'm more fluent in Perl. If it's really something simple I will create a shell script, if only to take out the additional overhead of loading a different interpreter. But sometimes I will use Perl instead of the shell because in some cases Perl is a lot faster.


----------



## Seeker (Apr 19, 2011)

For me this is question of selection between PHP (As I am webdev) and bourne shell for scripting task.
PHP has more abilities (OO, arrays, ...), then sh.
Actually, it is crazy to compare them.

sh is in base, while PHP isn't and has to be installed from ports


----------



## roddierod (Apr 19, 2011)

When you say "scripting" what exactly are you planing to do? For most general system scripting tasks you are not going to need to worry about an OO approach to things.

In my opinion PHP should only be used on the web (if used at all ). If you're going to do large-scale scripts that you want to share with people then you should look into something like Python or Perl if you feel things like sh or tcsh don't offer you enough features.


----------



## Seeker (Apr 19, 2011)

PHP is very dominant:
http://trends.builtwith.com/
But that is not a subject here.

Please guys, refrain from, *"lang A, is better from lang B"* chatter.

Focus on *this*:
HIGH: php, ruby, py, perl ...
LOW: sh

Dev is knowing 1 lang, inside-out, from HIGH group (DON'T compare or comment them!)

Target TASK is being done in 'sh'!
Based on WHAT and when, should dev decide, to drop 'sh' and do that TASK in 1 lang from HIGH group?


----------



## roddierod (Apr 19, 2011)

I'm not trying to say A is better than B, but that PHP was designed as a language for the web not for general purpose. Even your link state, "Web and Internet Technology".


----------



## Seeker (Apr 19, 2011)

PHP besides web code interpreter, also has a php_cli binary for use on a command line and defined constant STDIN STDOUT ... bla, bla
So all his features can be used for OS scripting.
Actually I use this already for postfix.

Now defining complexity limit, for shifting from 'sh' to 1 of a HIGH group?


----------



## jnbek (Apr 19, 2011)

For me, it depends on the mood I'm in. Since I'm learning multiple languages, I'm prone to use whatever happens to be in my thought process at the moment I open vim. I am a Perl geek, so usually things get done in Perl, but I've been known to use Zsh, Ruby, Io or even FORTRAN just to be a wisecrack. I would not, however choose PHP, as it's really just a template language that tries to be a scripting language.


----------



## Seeker (Apr 19, 2011)

I already know PHP inside out.
If I wouldn't know it and just starting to learn lang of HIGH group, I would go for python, as it could also be used for a web development.

Simply ditching 3 years of learning PHP and going from start with python is not an option, for me.


----------



## roddierod (Apr 19, 2011)

I'm not suggesting ditch PHP, but to use the right tool for the job. If you are doing web development use PHP. If you are doing OS scripting use a more apporiate tool. In the long run you will give yourself more possiblities and skills.

I use C#, Python, SQL and even PHP almost daily in my current job. Although I could probably just use C# for everything except the SQL queries. But, I use Python for scripts that don't require any GUI interaction or run in a batch mode, because why deal with all the overhead of what is needed to do console programming with C#. Python has also proven to be faster in processing files, so in my case less is more. And the maintenance cost is lower with Python in terms of man hours.

I use PHP on the website currently because I don't believe you should have to recompile a website to change a word, but I need to interact with serveral MS SQL databases so I can't just stick with plain HTML and I didn't want to deal with building a bunch of abstract classes in Java.

SQL, if you are going to deal with enterprise level databases you have to know SQL, preferably ANSI SQL, because you can then use it in Oracle or Microsoft or any other vendors product. All this No SQL type stuff isn't going to cut it.


----------



## wblock@ (Apr 19, 2011)

Seeker said:
			
		

> When it comes to scripting ... How to know, WHEN to use high level lang (php, ruby, py, perl ...) instead of a low level (/bin/sh)
> 
> How to recognize situation?



Use a language that fits the problem.  If you find yourself doing string operations in sh, that's a sign.  Replacing chunks of sed/awk/grep with a more powerful Perl routine can make it easier to write and maintain.  And don't assume that a language is slower because it's bigger, that's not always true.

Some of the more powerful languages also have well-tested module libraries like CPAN, which solve lots of problems without having to manually code and test something.

The ports versus base issue is usually overstated.  Sometimes it matters, but usually it doesn't.


----------



## ckester (Apr 19, 2011)

I agree with what wblock says about string operations being a good indicator that it's time to move "up" from sh.  (But I also agree with SirDice and have trouble thinking of sh as "low level".  Like him, I'm accustomed to thinking of assembly as low level, with C above that but still well "below" the interpreted languages like sh, perl, python, php, etc.)

Another reason to step "up" from sh is when the problem lends itself to a solution using associative arrays, aka dictionaries.   This is an example of the general point about knowing what features each language provides -- and having an opinion about which one implements a given feature in the most congenial way.


----------



## aragon (Apr 20, 2011)

Can only echo ckester and wblock - heavy string manipulation or the need for associative arrays is not optimal in Bourne shell.  The line at which you cross over to Perl/Python/PHP really depends on your project.  Bourne shell is really appealing for me as it's included in most (all?) Unixes, and is very good at interacting with the Unix system on which it runs.  Linux distros are finally moving from Bash to Dash for their /bin/sh, so cross compatibility should improve too.


----------



## expl (Apr 21, 2011)

Shell scripts should be used mostly for simple tasks as it is extremely slow, its really amazing how much code has to be ran just to load and run executables and then killing them and how much time it takes to complete that. In addition all shells are very primitive as languages. But shells are probably fastest method to write system management  programs as nix* systems comes with many native tools to make it very easy even though languages like python already come with wrapper functions for most of the system program functionality (/bin /sbin).

It all comes to what you are trying to accomplish, in many cases shell script is fine to complete the task (like lets say back up some files) while in other cases it just produces to much overhead (like lets say parsing some big files).


----------



## Seeker (Apr 21, 2011)

Thx, for yours insights.
You were very helpful.

Yes, complex string operations, is what I've thought about, as a limit.
Next thing was, arrays, but someone said "Go learn awk".


----------



## UNIXgod (Apr 22, 2011)

Seeker said:
			
		

> Next thing was, arrays, but someone said "Go learn awk".



You lose associative arrays with /bin/sh but we can still simulate indexed arrays. look at expr() for indexed simulation.

Arrays are simulated by the carriage return in the almquist shell. You may learn awk if you feel you want to but that may be the breaking point into when to use (Ruby|Perl|Python|Lisp|Java|C++|C). 

Perl was designed to displace awk and sed and was the first "does everything for you the shell already does language" breaking Doug McIlroy's take on UNIX philosophy "Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface."

I personally <3 ruby. It's designed to be the better perl. Ruby is a very decent unix tool indeed and sits right on top of our paradigm with the shell.

Also another perspective is to look at our toolchain and history.

ed() to grep() to sed() to awk() <-- to perl to ruby =)

Initially when the general purpose text processing tools where being created the concept was if the tool didn't exist break out k&r and go break out the compiler and program a tool in c. 

This is how they wanted us to use the system. They also didn't want bloat to our utilities and Kernighan complained when cat() became endowed with the ability to number all output lines which goes beyond the concept of what the tool was originally created for.

As to answer your initial question. Here is my policy in which your mileage may vary so please take it as a suggested use case but also as my own effort to understand myself the question you propose:

USE CASES:


I would keep my sh scripts simple and task oriented. (does one thing and does it well). As the time needs to build on those scripts I can call or include the functions into a larger script and now I have begun to build a library.

I want to move mass amount of files either from jails to physical machines one or more times a week and send myself a status email. use posix scripting for portability and if I hit the need for awk reconsider lighter weighted commands (i.e. cut()). the same goes for sed() and tr().

I need to calculate and process input from various utilities ps(), last() or parse my logs or filter input  /etc/passwd and once again piping that input through and email.

Larger programs can be built on your now created script library. making sure there is a help option as well as input from file/pipe or args will save your ass one day if you build a larger library. any third party software or non posix commands should be documented in the header. oh and don't use xargs() unless absolutely necessary.

Need a service running all the time. Use cron if you can. If you need real time results you can build a loop inside you script and launch at boot with nice() but using a high level language will give you threading and concurrency. If you need zero latency write your service in c.

You need a web frontend for your programs on the net. the shell may not be the best tool for that. not saying you can't do it but the right tool for the right job. Use (PHP| Ruby|Perl|Python). You already have a script archive to fall back on for calling from these languages if you ever need it. So now the question becomes reversed when do I rewrite my sh scripts in (PHP| Ruby|Perl|Python)? When you need concurrency or when they stop being fast enough for your load. I would love to advocate the use of ruby on rails to you but it's only fair to be pragmatic as you already hit an apex with php.

Finally when you need a gui you have even less options for the shell than you have in other languages. Since php is a specialized language for the web it probably wouldn't be the right tool. your mileage may vary from picking an interpreted language vs compiled language for this task.

Finally I digress. As much as I enjoy learning new tools there is nothing wrong with using the tools we already know. If one app can be done in a single language in which you are already acclimated with there is nothing wrong with just using that language whatever it may be.

~


----------



## olav (Apr 27, 2011)

I use Python with os.popen()


----------



## DungeonMaster3000 (Apr 28, 2011)

I was just asking myself this question today hence stumbling across this thread. I am interested in what the community mainly do things in as for when reading examples, tutorial, or sharing something I've been working on?

I figure is best to settle on what's in the minimal install?


----------



## Carpetsmoker (Apr 29, 2011)

I have found that for many not-to-complicated task, using /bin/sh is usually easier and shorter then using a language such as Python.
Above a certain level of complexity though, using something like Python becomes shorter and easier ...

To comment a bit on the PHP mentioned earlier:

One of the severe limitations of PHP is the lack of multiprocessing capabilities.
Right now, the only method is using pcntl_fork(), this is basically the "classic" fork() offered by UNIX/C.

AFAIK this copies all of the memory used by the parent process. There is no PHP equivalent of vfork() or rfork().

Also note that fork() doesn't work at all on Windows. (This has nothing to do with PHP, but with the fact that Windows doesn't implement fork() but uses threads)

PHP doesn't implement threads at all. AFAIK there is no plan to support this either.

This limits PHP's use as a general purpose language more than any other single factor.
(In web development, this usually isn't much if an issue, since you rarely use long-running scripts).


----------



## olav (Apr 29, 2011)

My problem with PHP is that it is so inconsistent. For example array["0"] is the same as array[0]
I really prefer Python's strong typing rather than PHP's weak typing.


----------



## Carpetsmoker (Apr 29, 2011)

> I really prefer Python's strong typing rather than PHP's weak typing.



This can actually be done if you set error_reporting to E_ALL. Many of these errors are an E_NOTICE and not shown with the default settings.

I usually go a step further and define a custom error handler with set_error_handler() and just exit() on *every* error. This way you get a behavior more or less similar to Python.


----------

