# Several Instances of Apache Running from a Single Installation (1 Instance per User)



## vask (Feb 5, 2009)

Reasoning:
- each instance of apache would run as a different user so that you could better limit access to files through permissions. ie) user1 would not be able to use PHP/Apache to read code within files that he/she would not otherwise be able to read.
- you could still share a global codebase of files with only certain users by applying permissions correctly.

The following link gives a quick breakdown as how this may be possible:
http://www.unitedknowledge.nl/howtos/multiple_apache_instances

Concerns:
- 50 users on a system would mean 50 Instances of apache running.
- How much DRAM does each instance of apache reserve/use?
- Are there other resource concerns?
- I only have 1 server.
- Is there an easier/better way to do this?  I am not sure if chroot or jails would accomplish my needs.

Please advise...


----------



## gilinko (Feb 5, 2009)

The best know process that I'm aware of is to use virtual hosts to run this with a mod_proxy frontend and mutliple http backends as virtual hosts.

ie.
http://www.example.com is the frontend/mod_proxy server.
user1.example.com as one vhosted backend server.

This particular server has 4GB ram and uses about 3,5G of it with a mod_proxy/virtual hosts setting for 600 users and a mysql server on the same host.

It's not the easiest of setup and still require a lot of manual maintenance, but with only one server it will be difficult.


----------



## vask (Feb 5, 2009)

When we talk about mod_proxy as a [front end] I assume we are talking about a single installation of apache.

However, I am unsure what you mean by [multiple http backends].  Are we talking about multiple instances of apache that are separate from the [front end] instance?

If we are talking about multiple instances of apache still I am not sure I would need mod-proxy. I don't see the benefit for my single server setup.

Also I should emphasize that the main security focus I have is file permissions...  I don't want human users to be able to manipulate the apache user to gain access to other human users files on the system to which they would otherwise not have access...

Perhaps I should explain [my proposed setup] a little better.  And perhaps there are exploits with this setup... but suppose I have:

[Artificial Users]
www1
www2
www3

[Human Users]
user1
user2
user3

So...
- www1 is the apache instance for user1
- www2 is the apache instance for user2
- www3 is the apache instance for user3

Then when I set up up permissions I would just put the instance of apache in the group ( ie) [user1], [user2], [user3], etc ) that I wanted it to have file access to... Suppose I have:

#groups www1
www1 user1
#ls -la
-rwxr-----  1 user1  user1   105 Feb  2 12:50 user1.php

#groups www2
www2 user1 user2
#ls -la
-rwxr-----  1 user2  user2   105 Feb  2 12:50 user2.php

#groups www3
www3 user2 user3
#ls -la
-rwxr-----  1 user3  user3   105 Feb  2 12:50 user3.php

Notice:
- Although ANYONE can see the way the file renders on the web, they can not access the files directly.
- user3 CAN see user2's files.
- user2 can NOT see user3's files. Not even by using this PHP trick:
<?
// random file owned by user2
echo `whoami`; // Output: www2
`cat ~user3/user3.php > ~user2/stolen_code_from_user3.txt`;
?>

Unix file permissions work great in this way...

I should also mention that for my proposed method that I was considering using IP aliases to run each instance of apache on the same port:
http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/configtuning-virtual-hosts.html


----------



## gilinko (Feb 5, 2009)

What you need is the experimental MPM module peruser http://www.telana.com/peruser.php

You will however still need two instances running (ie a frontend and a backend) if you don't want to spend the next years patching and maintaining your installation and nothing else.

The way we use is a request goes to http://www.example.com/~user1 on a external interface(public IP). The frontend translates this using mod_proxy into a backend domain call on the loopback device to user1.http://www.example.com and the backend server responds to this request. Using the peruser MPM makes the backend process behave as if it were run by the actual user(with all file/acl permissions that apply) and not a "global" apache user, and with the benefits of a simplified vhost configuration.

A note is that this will create ony ONE process per user so any server side technologies(like php) should be run in fastcgi mode and not shared mode(mod_php.so). Otherwise calls to your webserver will be unbelievable slow as there will be no forking of the process. 

This is all based on the premise that all your users is under the same domain name(ie http://www.example.com/~user1 etc) and not different domain names. It is possible to host more domain names on the frontend, but it gets increasingly more problematic.


----------



## vask (Feb 7, 2009)

I looked at:
- mpm-peruser
- mpm-itk

It seems that apache will need to be run as root!  And I am not sure how I feel about "experimental" software, but these things are a good option.

On another note, I am still wondering impact of 50 or so instances of apache on a single server with only 1 installation of PHP/Apache/MySQL.  I am very curious to try this method.  If apache can somehow pool resources with all 50 instances I would think this would be ok...  I know it would look ridiculous in the process list... But I still feel like I have unanswered questions:
- Has anyone tried this method?
- Wouldn't it at least work on a single server using IP aliases?
- Possible side effects?

For easy reference her are my links for "Several Instances of Apache Running from a Single Installation (1 Instance per Human User)"
- http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/configtuning-virtual-hosts.html
- http://www.unitedknowledge.nl/howtos/multiple_apache_instances


----------



## gilinko (Feb 7, 2009)

The initial apache server always runs as root(as it needs to bind to a port bellow 1024), but the spawned processes will be run as the user when using the peruser mpm. One very important point is that mpm-peruser development is targeted at linux, and it's still experimental in that environment.

As for the impact of 50 instances, a normal prefork mpm spawns about 10 instances upon start, and with 2-3 users you could easily increase to 30-40 simultaneously spawned processes and that is something I run on a 233 MHz ARM CPU with 32MB of RAM.

For the peruser module it is run using named virtual hosts where you need just one IP, and we are running it on a 4 year old server which serve 600 users of wich 60-70 have active sessions when running a "ps". I would say that almost all mpm's that use any sort of "peruser" is experimental in some way or other.


----------



## MG (Feb 8, 2009)

> - Is there an easier/better way to do this? I am not sure if chroot or jails would accomplish my needs.



I would give it a try. With jails it's possible to give each user his own userland. With mount_nullfs you can connect host directories to any directory of a jail tree, readonly if neccesary.
It might be some work, but it must be quite possible to write a script that makes a jail for any user depending on his permissions.

This way users can build and run their own apache port and still have access to a group-shared collection of files on the host system.
It's even possible to route an aliased ip and resolve a domain name for each jail running httpd to make them all remotely visible.


----------



## vask (Feb 8, 2009)

Using FreeBSD Jails and nullfs seems like a good option as well...

However, I am not sure if FreeBSD Jails will put too much load on the server (PIII 1GHz with 768MB RAM).

Also, do chown and chmod attributes copy over on a nullfs mount into the jail?  And if so would I be able to assign a FreeBSD Jail user to a group outside the jail on the host system?

ie) If I wanted to revoke jail_user1's group permissions on a certain nullfs file...  Would I be able to just remove jail_user1 from the corresponding group?


----------



## vask (Feb 9, 2009)

I was doing some more research today and found mod_suexec for apache. It is even included as an apache22 option when you do:
make config

See:
http://httpd.apache.org/docs/2.0/mod/mod_suexec.html
http://httpd.apache.org/docs/2.0/suexec.html

mod_suexec seems like it is a stable release, more popular than some other options, and is NOT experimental... so I like that over mpm-peruser.

Also, suexec is not a virtualized environment like a FreeBSD Jails and nullfs, which might be overkill for my needs.

Maybe I will settle on this mod_suexec option. Thanks for the help guys!

-John


----------



## vask (Feb 10, 2009)

After more research it seems that suexec is actually more difficult to configure than I first realized... I gave up on mod_suexec and decided to go with mpm-itk for the time being. I also signed up for the mpm-itk mailing list...  See:
http://mpm-itk.sesse.net/

Reasoning for Choosing mpm-itk:
- Installation was simple from ports I just had to specify WITH_MPM=itk when [make]ing apache
- http.conf configuration is so simple just put [AssignUserId myuser mygroup] in the virtual host
- Basically all I really care about on my webserver is PHP files and keeping the code secure/safe/separate. ( If an attacker can get www privileges on my server under the previous setup it would be just as bad as having root privileges as the code would already be compromised... )  So maybe in mpm-itk the following isn't really as bad as it seems:

```
# ps aux | grep http
root    3822  0.0  4.4 19692 11240  ??  Ss   12:54PM   0:02.72 /usr/local/sbin/httpd -k start
root    3929  0.0  4.4 19716 11272  ??  I     1:02PM   0:00.01 /usr/local/sbin/httpd -k start
root    3930  0.0  4.4 19716 11272  ??  I     1:02PM   0:00.01 /usr/local/sbin/httpd -k start
root    3931  0.0  4.4 19716 11272  ??  I     1:02PM   0:00.01 /usr/local/sbin/httpd -k start
```
- Most everything on my server is designed for display on the web. 
- Research from a Comparison of Several Similar User Separation Technologies. See: http://blog.stuartherbert.com/php/category/the-web-platform/

@gilinko
Thank you very much for your information...


----------



## gilinko (Feb 10, 2009)

Glad to be of service. Finally got some use of my 8+ years of battling with apache


----------



## vivek (Feb 11, 2009)

No offense Apace is good server but to run only apache and with various instance you could have tried out nginx or lighttpd. Both have simple chroot() support. For example, I've over 40 website hosted on a single freebsd amd 64 bit with 2GiB and 250GB (RAID 1) hard disk. as follows

```
/chroot/domain1.com/home/lighttpd
/chroot/domain2.com/home/lighttpd
/chroot/domain3.com/home/lighttpd
...
/chroot/domain40.com/home/lighttpd
```

Some of them are on singe public IP (lighttpd itself can proxy out so no big config required) or dedicated IP as per users requirements. 

Each domain is chrooted to /chroot/domain1.com/ using in built chroot configuration. All users , php, perl only see  /home/lighttpd from script. If One of the site called domain2.com cracked and it will only affect  /chroot/domain2.com/.

This kind of setup is not easy as most out of box rc.d script and config do not work. You need to pay more time to get it secured.


----------



## vask (Feb 11, 2009)

@vivek
Would I be able to share a global code base with selected users with your setup?  ie) Have access to files outside chroot?  Because that is what I need.

mpm-itk has made things so simple for me... In fact, to use it I have only had to learn a few configuration commands to put in httpd.conf  mainly:
AssignUserId user1 group1

I may try different webservers in the future... And it is something I hadn't really considered but it is always good to have options...


----------



## Mel_Flynn (Feb 11, 2009)

The complexity of jails is overrated. You can *union*fs mount /usr/local from 1 template jail, so that configuration changes can be made and software on a per-user base can be installed. FreeBSD will recognize the exact same binary and libraries and not re-read image from disk.
If you don't need different software per user, symlink /usr/local/etc to /var/etc in the template jail and just use nullfs. You may need to adjust /etc/mtree/BSD.local.dist in the template jail to reflect the change.

You can strip down jails to the bare bone using /etc/src.conf, example:

```
.ifdef INSTALLING_JAIL
WITHOUT_BLUETOOTH=yes
WITHOUT_CXX=yes
WITHOUT_CPP=yes
WITHOUT_BIND_DNSSEC=yes
WITHOUT_BIND_ETC=yes
WITHOUT_BIND_NAMED=yes
.endif
```

You can tune Min/Max server parameters based on price as well


----------



## vivek (Feb 11, 2009)

vask said:
			
		

> @vivek
> Would I be able to share a global code base with selected users with your setup?  ie) Have access to files outside chroot?  Because that is what I need.



If you give them proper FTP / shell access and proper group access, they can access files outside chroot. Lighttpd chroot only protect you from web users / attackers by locking down them to a specific directory.


----------



## vivek (Feb 11, 2009)

Mel_Flynn said:
			
		

> You can tune Min/Max server parameters based on price as well



Nice, but I do not see any option / parameters for Min/Max .. I also went though src.conf man page.. may be I misunderstood your post...


----------



## Mel_Flynn (Feb 12, 2009)

vivek said:
			
		

> Nice, but I do not see any option / parameters for Min/Max .. I also went though src.conf man page.. may be I misunderstood your post...



In httpd.conf, sorry, had to leave and didn't proofread. Since you have complete httpd.conf to work with, you can tune every user to needs, compensation or whatever policy.

You will need a reverse proxy on the external to translate the incoming requests to the correct IP addresses. Squid can do this, and I'm sure there's more lightweight options.


----------

