====== Nginx-Proxying-to-Apache Install Guide ======

<note warning>
If your Accelerator was built using the new pkgsrc-based template (April 2008 and on), you do not need to built either of these packages (OpenSSL, PCRE, Nginx). You should already have Nginx built and ready at /opt/local/sbin/nginx. In fact, if you do follow this page, you'll break the dependencies of Postfix and other packages on the PCRE library. You **can** build your own, but you'd need to use a different prefix than /opt/local, which is used by the new template layout.
</note>

Apache is a fantastic web server. It has been around a long time and has proven itself to be highly compatible, flexible and reliable. Apache's performance is generally very good as well. However, in a resource-constrained environment such as an "S" or "M" accelerator, Apache does not necessarily provide optimal performance or resource utilization. This becomes especially evident in a high traffic site that makes use of dynamic content such as php, python/django, ruby/rails, etc.

One of the "best practice" approaches to reducing the strain on Apache is to let a leaner lighter web server take over the serving of the site's static files (images, javascript, css, etc.), leaving the heavy lifting of serving the dynamic content to Apache. When Apache is relieved of the burden of serving up all the ancillary static content, its full power can be focused on the dynamic content. Memory consumption is also reduced when there is no need to fork multiple memory-hungry Apache processes just to serve up simple images or JavaScript files.

In this how-to, I will show you how I configured Nginx as a front-end web server to handle static content while proxying all other requests to Apache. Nginx is one of several possible choices for a front-end proxying server. Lighttpd is another popular choice. However, lighttpd has been widely reported to suffer some stability and memory-leakage problems when used in a proxying configuration. I have found nginx to be rock-solid in this scenario.

==== Preliminaries ====

Some Solaris build tools reside in /usr/ccs/bin, which needs to be added to your path if it not there. I just tacked it onto the end:

<code>
$ nano -w ~/.bashrc

export PATH=/usr/sbin:/usr/bin:/usr/sfw/bin:/opt/csw/bin:/opt/csw/sbin:/opt/csw/gnu:/opt/csw/gcc3/bin:/opt/csw/mysql5/bin:/opt/csw/postgresql/bin:/opt/csw/apache2/bin:/opt/csw/apache2/sbin:/usr/ccs/bin
</code>


Now refresh your shell so it sees the modified PATH variable:

<code>
$ source ~/.bashrc
</code>

I like to build local packages in /opt/local/src. This directory was not included in my accelerator so I created it:

<code>
$ sudo mkdir -p /opt/local/src
$ sudo chgrp staff /opt/local/src
$ sudo chmod 775 /opt/local/src
</code>

==== Building/Installing Nginx ====

In order to build Nginx for Solaris, we must first satisfy a couple of dependencies, namely pcre (Perl Regular Expression Library) and OpenSSL. (Although OpenSSL is not strictly needed, I like to include it for completeness.) All builds will be full 64-bit.

=== PCRE ===

<code>
$ cd /opt/local/src/
$ wget http://internap.dl.sourceforge.net/sourceforge/pcre/pcre-7.1.tar.gz
$ tar xzf pcre-7.1.tar.gz
$ cd pcre-7.1
$ CFLAGS="-m64 -O2 -g -Wall" CXXFLAGS="-m64 -O2 -g -Wall" LDFLAGS=" -L/lib/64 -L/usr/sfw/lib/64 -R/usr/sfw/lib/64 -R/lib/64" ./configure  --prefix=/opt/local --enable-utf8 --enable-unicode-properties
$ make
$ make test
$ sudo make install
</code>

**Note**: Putting the libpcre* file in /opt/local/lib will overwrite the existing version used by apache, postfix, and others. I installed this version in /opt/local/lib64. I added this lib to the nginx install instructions below.

=== OpenSSL ===

<code>
$ cd /opt/local/src/
$ wget http://www.openssl.org/source/openssl-0.9.8e.tar.gz
$ tar xzf openssl-0.9.8e.tar.gz
$ cd openssl-0.9.8e
$ ./config threads --prefix=/opt/local --openssldir=/opt/local/openssl
$ make
$ make test
$ sudo make install
</code>

=== Nginx ===

<code>
$ cd /opt/local/src/
$ wget http://sysoev.ru/nginx/nginx-0.5.24.tar.gz
$ tar xzf nginx-0.5.24.tar.gz
$ cd nginx-0.5.24
$ ./configure --prefix=/opt/local/nginx --pid-path=/var/run/nginx.pid --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --http-client-body-temp-path=/var/tmp/nginx/client_body_temp --http-proxy-temp-path=/var/tmp/nginx/proxy_temp --http-fastcgi-temp-path=/var/tmp/nginx/fastcgi_temp --with-http_ssl_module --with-openssl=/opt/local/src/openssl-0.9.8e --with-cc-opt="-m64 -I/usr/local/include -I/usr/local/ssl/include" --with-ld-opt="-L/lib/64 -L/usr/sfw/lib/64 -R/usr/sfw/lib/64 -R/lib/64 -L/usr/local/ssl/lib -m64 -L/usr/local/lib -R/usr/local/lib"
$ make
$ sudo make install
</code>

Add nginx user to webservd group:

<code>
$ sudo /usr/sbin/useradd -g webservd -s /bin/false nginx 
</code>

Create Nginx directories for temp files and logs:

<code>
$ sudo mkdir /var/log/nginx
$ sudo chown -R nginx:webservd /var/log/nginx
$ sudo chmod -R o-rwx /var/log/nginx
$ sudo mkdir /var/tmp/nginx
$ sudo chown -R nginx:webservd /var/tmp/nginx
$ sudo chmod -R o-rwx /var/tmp/nginx
</code>

Create Nginx config file:

<code>
$ sudo nano -w /opt/local/nginx/conf/nginx.conf
</code>

Here is my nginx.conf file. Notice that I enable gzip compression (for javascript and css files). I also disable logging for a performance boost, as I don't much care about monitoring access to static content. I leave Apache logging enabled for dynamic content. Feel free to set up logging in the way that best suits your own needs.

<code>
user nginx webservd;
worker_processes 2;
error_log  /var/log/nginx/nginx_error.log;

events {
	worker_connections 256;
}

http {
	include conf/mime.types;
	default_type  application/octet-stream;
	
	log_format  main  '$remote_addr - $remote_user [$time_local] $request '
							'"$status" $body_bytes_sent "$http_referer" '
							'"$http_user_agent" "$http_x_forwarded_for"';
	
	access_log /var/log/nginx/nginx_access.log main;
	
	sendfile                        on;
	#tcp_nopush                     on;
	keepalive_timeout               5;
	tcp_nodelay                     on;
	gzip                            on;
	
	server {
		listen your.public.ip.here:80;
		server_name your.domain.com;
		root /path/to/your/web/root;
		access_log /var/log/nginx/your-domain.access.log main;
		
		location ~* \.(jpg|jpeg|gif|css|png|js|ico)$ {
			access_log      off;
			expires         30d;
		}
		
		location / {
			access_log      off;
			proxy_pass http://localhost:8080;
			proxy_set_header X-Real-IP  $remote_addr;
			proxy_set_header Host $host;
			proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
		}
	}
}
</code>

Once you have created the config file, run a quick syntax check like so:

<code>
$ sudo /opt/local/nginx/sbin/nginx -c /opt/local/nginx/conf/nginx.conf -t
</code>

Now we create a SMF file so that Solaris can manage the Nginx service for us, restarting it when we reboot the accelerator, etc.

Here is my nginx.smf file:

<code xml>
<?xml version='1.0'?>
<!DOCTYPE service_bundle SYSTEM '/usr/share/lib/xml/dtd/service_bundle.dtd.1'>
<service_bundle type='manifest' name='export'>
  <service name='network/nginx' type='service' version='0'>
    <create_default_instance enabled='true'/>
    <single_instance/>
    <dependency name='fs' grouping='require_all' restart_on='none' type='service'>
      <service_fmri value='svc:/system/filesystem/local'/>
    </dependency>
    <dependency name='net' grouping='require_all' restart_on='none' type='service'>
      <service_fmri value='svc:/network/loopback'/>
    </dependency>
    <dependent name='nginx' restart_on='none' grouping='optional_all'>
      <service_fmri value='svc:/milestone/multi-user'/>
    </dependent>
    <exec_method name='start' type='method' exec='/opt/local/nginx/sbin/nginx -c /opt/local/nginx/conf/nginx.conf' timeout_seconds='60'>
      <method_context working_directory='/var/log'>
        <method_credential user='root' group='root'/>
        <method_environment>
          <envvar name='PATH' value='/usr/bin:/bin:/opt/csw/bin:/opt/local/bin'/>
        </method_environment>
      </method_context>
    </exec_method>
    <exec_method name='stop' type='method' exec=':kill' timeout_seconds='60'>
      <method_context/>
    </exec_method>
  </service>
</service_bundle>
</code>

Create this file anywhere on your Accelerator, and then import with the following command:

<code>
$ sudo svccfg import /path/to/your/nginx.smf
</code>

Before we can start up Nginx, we need to reconfigure Apache, because both web servers cannot listen on the same IP and port. We reconfigure Apache to listen only on localhost (127.0.0.1), which is the address to which we configured Nginx to proxy requests. If your stock Apache config was like mine, there are two files you will need to change:

In /opt/csw/apache2/etc/httpd.conf there is a default virtual host definition. I don't use a default virtual host, so I have this definition commented out. If you want to keep it, change the IP address from your public IP to 127.0.0.1.

<code>
$ sudo nano -w /opt/csw/apache2/etc/httpd.conf
</code>

In /opt/csw/apache2/etc/includes/listenIPs.conf you will need to change the IP address from your public IP to 127.0.0.1.

<code>
$ sudo nano -w /opt/csw/apache2/etc/includes/listenIPs.conf
</code>

Now we can restart Apache and it will be listening on localhost, ready to accept proxied requests from Nginx.

<code>
$ sudo svcadm restart /network/http
</code>

Now that Apache is listening on localhost, we can start up Nginx listening on your public IP.

<code>
$ sudo svcadm enable network/nginx
</code>

Check that both Apache and Nginx are "online":

<code>
$ svcs
</code>

Check that both Apache and Nginx are running:

<code>
$ prstat
</code>

Now, you should be able to browse to your web site and view it just as you did before. The only difference is that Nginx is now serving all your static content.

=== Optional: Install mod-rpaf ===

One side effect of proxying through Apache is that Apache will see all requests as coming from localhost (127.0.0.1). You can confirm this by checking your Apache logs. If you are interested in tracking visitors to your sites this presents a problem. Luckily, there is a solution. Notice in the nginx.conf file the following lines:

<code>
proxy_set_header X-Real-IP  $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
</code>

This tells Nginx to forward the original remote IP address in the header as it proxys ro Apache. But in order to get Apache to utilize this information, we need to install a small Apache module. Normally, this is a very simple matter. Unfortunately, Accelerators are delivered with an Apache built with Sun's compiler tools but without the compiler tools themselves. In order to build an Apache module, we need to use the same tools that were used to build Apache itself. So before you can follow these instructions, you must first follow the [[accelerators:kb:sun_studio|instructions to install Sun Studio onto your Accelerator]]. 

Assuming Sun Studio is installed to /opt/SUNWspro, add the directory with the Sun binaries (/opt/SUNWspro/bin) to your path:

<code>
$ nano -w ~/.bashrc

export PATH=/usr/sbin:/usr/bin:/usr/sfw/bin:/opt/csw/bin:/opt/csw/sbin:/opt/csw/gnu:/opt/csw/gcc3/bin:/opt/SUNWspro/bin:/opt/csw/mysql5/bin:/opt/csw/postgresql/bin:/opt/csw/apache2/bin:/opt/csw/apache2/sbin:/usr/ccs/bin
</code>

Now refresh your shell so it sees the modified PATH variable:

<code>
$ source ~/.bashrc
</code>

Download the mod_rpaf package:

<code>
$ cd /opt/local/src/
$ wget http://stderr.net/apache/rpaf/download/mod_rpaf-0.5.tar.gz
$ tar xzf mod_rpaf-0.5.tar.gz
$ cd mod_rpaf-0.5
</code>

Next, edit the make file to have the correct path to your apxs binary. Find the line near the top of the file and change it as follows:

<code>
$ nano -w Makefile

APXS=/opt/csw/apache2/sbin/apxs
</code>

Now we're ready to build.

<code>
$ make rpaf-2.0
$ sudo make install-2.0
</code>

Finally, locate the config file where your Apache virtual host(s) are defined (probably /opt/csw/apache2/etc/virtualhosts/) and add the following lines to each of your virtual host definitions:

<code>
RPAFenable On
RPAFsethostname On
RPAFproxy_ips 127.0.0.1
</code>

Restart Apache, visit your site, and check your logs. You should see correct visitor IP addresses.

<code>
$ sudo svcadm restart /network/http
</code>

One last thing you should do is reboot your Accelerator and make sure that both Apache and Nginx come up OK.

<code>
$ sudo reboot
</code>

Enjoy!