Tag Archives: apache

Varnish on CentOS - full guide

Varnish – how to make websites fly?

In this episode we will make our websites fly with Varnish on CentOS. What exactly is Varnish? Varnish is an HTTP accelerator designed for websites with high traffic. It can speed up you website significantly, especially when you have more reads than writes. It's designed to handle heavily loaded websites as well as APIs. It's used by many high-traffic websites like The Guardian, New York Times etc. In this episode we will learn how to install Varnish on CentOS.

LAMP on steroids

This is part of our series LAMP on steroids. Check the links below to learn how to setup awesome webserver!

  1. Choosing VPS
  2. Install EPEL
  3. Install and configure Apache HTTPD server
  4. Harden Apache with ModSecurity and OWASP Core Rule Set
  5. Install and configure PHP
  6. Install and configure MySQL server
  7. Configure firewall based on iptables
  8. Create developer user and setup SSH key-pair
  9. Configure SSH
  10. Install and configure Varnish to speed up websites
  11. More to come...

How to install Varnish on CentOS?

Installation is pretty straightforward:

rpm --nosignature -i https://repo.varnish-cache.org/redhat/varnish-4.0.el6.rpm
yum install varnish -y
varnishd -V

Varnish comes in two versions. Widely used version 3 and version 4 which is recommended at the moment. Version 3 is generally deprecated  since the end of April 2015, so we will install latest version of Varnish 4. After installation  varnishd -V  shows the version that is inside our system. In our case it's 4.0.3

Now we can try to run Varnish by executing following command:

sudo service varnish start

...and you should see the error - varnish failed to start. Why's that? Read next section about SElinux policy. If you don't see the error you can move forward to next part of this tutorial.

SElinux policy for Varnish on CentOS

The problem is that Varnish in version 4 requires some additional permissions in order to run under CentOS. We have SElinux enabled by default. We shouldn't not disable that (which is sometimes common practice in such situations). We will add the rule to the policy, so Varnish can start without any problems.

First of all we need the tool called audit2allow. Let's install it:

sudo yum install policycoreutils-python -y

Now, try to start Varnish one more time. It should give you fail error. After that execute following command:

cd ~/sources
mkdir varnish
cd varnish
sudo grep varnishd /var/log/audit/audit.log | audit2allow -M varnishd2
sudo semodule -i varnishd2.pp

We will use our sources directory that we created at the beginning of our series. You can of course use any directory you want.

Next we look through audit.log on CentOS and try to find varnishd occurrences. audit.log file contains blocked activities by SELinux. From these blocked activities we will generate module -M for SELinux that will grant access to required permissions.

Note 2 at the end of varnishd2  name. We need to add new policy. varnishd  policy already exists in SELinux policies.

Enable the module and try to start Varnish service again. In our case it didn't work out, and we still got fail error. So we did it again:

sudo grep varnishd /var/log/audit/audit.log | audit2allow -M varnishd3
sudo semodule -i varnishd3.pp
sudo service varnish start

and Varnish started without any errors!

Setup firewall for Varnish

So now it's the time to play around with Varnish. The installation only is not enough to be able to have Varnish fully working. Let's start by adding some firewall rules in order to be able to test Varnish on CentOS.

By default Varnish is running on port 6081 , but we don't have rules in our firewall, so you won't be able to see the actual result. Open firewall rules file and add these rules:

-A INPUT -p tcp --dport 6081 -m state --state NEW,ESTABLISHED -j ACCEPT
-A OUTPUT -p tcp --sport 6081 -m state --state ESTABLISHED -j ACCEPT
-A INPUT -p tcp --sport 6081 -m state --state ESTABLISHED -j ACCEPT
-A OUTPUT -p tcp --dport 6081 -m state --state NEW,ESTABLISHED -j ACCEPT

It's the same rules like for instance for port 80 for Apache. For testing purposes we will enable to access our website by two different ports - 80 (by Apache) and 6081 (by Varnish). Port 80 will work without caching, port 6081 will serve cached content.  At the end we will change our setup, so only port 80 will be available and it'll be Varnish port, not Apache like for now.

Restart the firewall to apply the rules:

sudo service firewall restart

And now you can try to test it by calling http://example.com:6081

You should see the error, and that is perfectly fine.

Test with microtime()

Now it's time to test Varnish cache. The easiest way is to create PHP script with following content in /var/www/example/com/htdocs :

<?php
#/var/www/example.com/htdocs/varnishtest.php
echo microtime();

then try to run it in your browser: http://example.com/varnishtest.php and refresh couple of time. On each refresh you should see different contents.  Now it's time to make http://example.com:6081/varnishtest.php  working.

First of all we need to configure backend for Varnish. Edit file /etc/varnish/default.vcl

You should see backend default there. By default port is set to 8080, but our Apache is working on port 80, so we need to change that:

vcl 4.0;

# Default backend definition. Set this to point to your content server.
backend default {
    .host = "127.0.0.1";
    .port = "80";
}

Exit editor and reload Varnish:

sudo service varnish reload

Try to reload the page couple of times. At second refresh you should see exact same thing as before. So result from microtime()  will not change. If you achieve such effect, you can be happy! Varnish is working correctly:D

If you will see at response headers you will get additional headers. Most important is Age  header. It should have value > 0. By that you can see that Varnish is caching your website correctly. But if you still can see that it's not working (content is still changing) you should see next section of this article.

Troubleshooting Varnish Cache

There are plenty of reasons why Varnish might not cache your website. You must know that it's not easy to diagnose what's wrong with your cache/website. Here are few tips that might be helfpul

ModSecurity is running

If you have ModSecurity enabled there might be an issue with caching. Or you might see 403 pages sometimes. There's an easy fix for that. You need to disable 960020 rule. In order to do that you must edit the httpd-security.conf  file that we created during Apache setup and add line  so the file will look like that:

# File location: /usr/local/apache2/httpd/conf/extra/httpd-security.conf

#...
<IfModule security2_module>
      Include conf/crs/modsecurity_crs_10_setup.conf
      Include conf/crs/base_rules/*.conf
      # Include conf/crs/experimental_rules/*.conf
      # Include conf/crs/optional_rules/*.conf


      #.... configure ModSecurity here
      
      #Add this line: 
        SecRuleRemoveById 960020
</IfModule>
#... Rest of the file

Remember that you can't do it in .htaccess. It's important to remove the rule after you include rules from OWASP!

After that restart HTTP daemon and check if page is caching or not:

sudo service httpd restart

Missing caching header

Another issue might be connected to missing caching header. You can set how long each page can be cached by setting the cache file. Try to change varnishtest.php file like that:

<?php
header( 'Cache-Control: max-age=60' );
echo microtime();

It means that this file should be keep in cache no longer than 60 seconds. Try to refresh the page and see if it brings any help or not?

Double caching header

If You are using mod_expire for Apache, there might be situation where you have two caching headers. One is set from PHP script, second one by Apache. The easiest way to check it is to see header in Chrome for instances. If You will see two caching headers like that:

Cache-Control:max-age=10
Cache-Control:max-age=0

You need to adjust your Vhost file for file caching. For instance sometimes you can find such settings in your httpd.conf or Virtual host file:

<IfModule mod_expires.c>
    #If you are using caching for static contents this line must be present, don't remove it!
    ExpiresActive on

    #Default cache is set to 0, comment it out
     ExpiresDefault 0
   
     #Disable cache per content type, comment it out
     ExpiresByType application/json                      "access plus 0 seconds"
     ExpiresByType application/xml                       "access plus 0 seconds"     ExpiresByType text/html                             "access plus 0 seconds"
</IfModule>

If you plan to use Varnish cache, you should control your cache inside your application, and don't rely on Apache cache when it comes to website content, API endpoints etc. You can still cache static assets like images, scripts or styles with Apache. Double Cache-Control header will prevent Varnish from caching PHP scripts.

Those cookies...

This is really important. By default Varnish will NOT cache any output if there are any cookies. That's how usually authentication mechanisms works for logged in users. If you are logged in you want to see most recent content instead of cached one. Make sure that you are not sending any Cookies. If you have some problems with Tracking cookies like from Optimizely or Google Analytics  you an always unset them in Varnish. Try this approach. Open default.vcl  file where you set the backend port. Change sub vcl_recv section to something like this:

sub vcl_recv {
	 if(req.http.Cookie){
    set req.http.Cookie = ";" + req.http.Cookie;
    set req.http.Cookie = regsuball(req.http.Cookie, "; +", ";");
    set req.http.Cookie = regsuball(req.http.Cookie, ";(PHPSESSID)=", "; \1=");
    	set req.http.Cookie = regsuball(req.http.Cookie, ";[^ ][^;]*", "");
    set req.http.Cookie = regsuball(req.http.Cookie, "^[; ]+|[; ]+$", "");

    if (req.http.Cookie == "") {
        	unset req.http.Cookie;
    }
  	}
}

This is example taken from Varnish webiste. It will unset all cookies but not the cookie contains PHPSESSID. Save changes in file and restart varnish:

sudo service varnish restart

Check your page now.

Nothing helps 🙁

If nothing from above helped, you need to find the issue. There is also great documentation that comes from Varnish and you can see Varnish troubleshooting guide. If still nothing helps you can post comments here below, use StackOverflow or some forums. We learned that Varnish sometimes can be pain in the a** and it's hard to debug why something doesn't work...

Purging Varnish Cache

Great! Now we have our cache up and running! We change something in our script, refresh the page, but hey! There is still cached content! So now the question is, how to clear Varnish cache?

Method 1 - varnishadm

Let's introduce first way to purge Varnish cache. First method is with usage of Varnish tools. One of them is varnishadm which will help us to clear the cache. Let's try it:

sudo varnishadm ban 'req.http.host == "lamp.com:6081" && req.url ~ "/varnishtest.php"'

This is very basic usage. We will use ban command from varnishadm to add ban in Varnish settings. In ban command we can use any parameters that are available in Varnish settings, but here are two basic things that you will need.

First we need to check host req.http.host == "lamp.com:6081" . This is very useful if we have multiple domains on our server. We need to be sure that we are clearing the cache only for one, particular website, not for all websites. Port is necessary for now, because we don't have Varnish on port 80 now.

Next is the most important thing - URL. req.url ~ "/varnishtest.php"  We can use here any regex expression, which is really helpful. This command will clear only one particular URL. You can use req.url ~ "^/article" to ban all URLs that starts with /article phrase. Sometimes helpful command is req.url ~ ".*"  that will clear cache for every page and file.

Short note about clearing the cache - if you have really high traffic on your website you need to be very careful when it comes to clearing Varnish cache. You don't want to kill your website with really high traffic that will goes straight to Apache for instance. So sometimes it's better to wait for Varnish cache to expire by itself instead of clearing the cache manually. It all depends on your website and architecture.

Method 2 - PURGE request

OK, but how to clear the cache from PHP script for instance? It's not safe to execute shell commands from script directly. Therefore Varnish comes with another way of clearing cache - PURGE request.

You can send PURGE command with cURL, for instance:

curl -X PURGE http://example.com:6081/varnishtest.php

ModSecurity fix

If you have ModSecurity enabled, you will get 403 error. That's because PURGE request method is not allowed. We need to edit OWASP settings file located in /usr/local/apache2/conf/crs/modsecurity_crs_10_setup.conf  and we need to find the line with tx.allowed_methods . Then we need to add PURGE request, so it will look like that:

setvar: 'tx.allowed_methods=GET HEAD POST OPTIONS PURGE', \

Restart Apache to load new policy:

sudo service httpd restart

Then try to execute curl command from above. You should get contents of our test file instead of an error.

 

As you can see, you will get our test file contents, but it won't clear Varnish cache for this URL. We need to prepare Varnish to enable PURGE requests. Edit VCL file located in /etc/varnish/default.vcl  We need to add following section:

acl purge {
        "localhost";
        "192.168.77.1"/24;
}

sub vcl_recv {
        #....

        if (req.method == "PURGE") {
                if (!client.ip ~ purge) {
                        return(synth(405,"Not allowed."));
                }
                return (purge);
        }
        #.....

}

In acl purge  we define addresses that can send PURGE requests. Localhost is obvious, if we want to send requests from PHP scripts. But sometimes we need to clear the cache from external IP address. So it's good to add it here. If you need to clear the cache only from PHP script, and do not send it from outside, leave localhost  only.

In sub vcl_recv  section we check if request method is PURGE, if so, we need to check if client.ip  is within allowed list for PURGE requests.

Try to send request again. It should clear the cache.

VCL

So what is this magic VCL file? In short words it a file designed to define request handling and caching policies for Varnish. We can specify additional rules, like removing request cookies, or disable the cache if user is logged in. Configuration strongly depends on your website or CMS you are using. You can use ready VCL files like this one - mattiasgeniar/varnis-4.0-configration-templates. This is really cool and well commented example VCL file. You can find multiple tricks for Varnish there, and it's really worth checking. It has configuration for WordPress and Drupal, but it should also work well with Symfony2 or Zend Framework. You can also find more details about Varnish and Symfony2 on their website.

It's all really depends on your website, used CMS or framework, installed plugins etc. So it's really hard write one VCL for everyone. You need to do it by trial and error or googling. But really good start is to check VCL file mentioned above.

Varnish settings per domain/Virtual Host

What if we have multiple websites and we want to have separate configuration per website? For instance one website is based on Drupal, second one is just pure PHP. Varnish settings for global is not possible due to some conflicts. How to deal with such issue?

VCL file and language is pretty powerful and we can resolve such conflict It's not that easy, but doable. Edit  default.vcl  file - /etc/varnish . Take a look on example file below:

vcl 4.0;

backend default {
    .host = "127.0.0.1";
    .port = "80";
}

# ...

sub vcl_recv {

    # ... global rules for all websites here

    # Return 404 error error in requests without Host. It's optional but helpful
    if(! req.http.Host){
        return (synth(404, "Host header missing"));
    }

    if(req.http.Host == "example.com"){
        include "/etc/varnish/example.com.vcl";
    }else if (req.http.Host == "secondexample.com"){
        # you can also place normal rules here, no need to include files
    }
}

Varnish has include command. With this command you can include pieces of code inside different files. So you can have kind of per domain settings. It's not perfect, because you need do it inside sub directive. It's not possible to include whole sub vcl_recv sections  etc.

With this there comes second disadvantage. If you have larger pieces of instructions per domain and you have to modify another sub  like vcl_backend_response , you need to make similar if statement inside each section.

You can modify the if as you wish. But remember, that you can also use ~  to mark domains with subdomains + www etc.

In this example there's no port. If you are testing Varnish on different port, so it's not running on port 80, you need to specify the port as well. if(req.http.Host == "example.com:6081")

This is not perfect solution, but it's working. In real life you probably won't have to create statement like that. Most of the rules can be applied globally. But if you need to make an exception for particular domain, now you know how to do that;)

Varnish useful tools

Here are couple of really useful tools that comes with Varnish.

varnishstat

This command will give you an access to Varnish statistics. You can see lot of data there, but really important are cache_hit  and cache_miss . If you have lot of cache_miss , that means, that your content is not served from Varnish, but from backend like Apache. It also means that something is not OK and you should really look through your configuration and caching.

If you have great number of cache_hit  and not that many cache_miss  it's perfectly alright. When Varnish cache expire for particular URL it goes directly to backed and cache_miss  value is incremented.

varnishtop

varnishtop is yet another useful piece of software you can find in varnish toolbox. It shows the URLs that have most hits to your backend. It is also great debugging tool. The cool thing is that you are able to filter the logs. There are so many filters that it's not possible to list all of them here. But there is great documentation for this tool, that you can find here.

Here are two examples, that are really useful

  • varnishtop -i ReqURL  it will show list of URLs that comes from Varnish most frequently 
  • varnishtop -i BareqURL  it will show you list of URLs that goes to your backend the most. This is really great command, because you can easily catch most frequent misses.

varnishlog

This is another great tool. It logs all requests in real-time with tons of data. It's great for debugging why particular URL doesn't work (ie. it's not cached, but it should). It has very similar syntax to varnishtop, so if you master varnishtop, you can use varnishlog without any problems.

By deafult, Varnish doesn't log to file. This is really important. Why's that? Think about such situation. You have an API with some endpoint or one particular page in WordPress. This endpoint/page has heavy load, let's say 1000 requests per second. Apache can't handle it, so you install Varnish and suddenly everything is OK. But you don't know what's going on in logs. In Apache access.log you have only few requests which has cache_miss in Varnish. But Varnish doesn't log anything to file. And to be honest - it's correct behavior. First of all, it will speed up entire service, because you don't need to waste the time for I/O operations on your hard-drive every requests. Second of all, digging in really big access.log is pain in the a** and it's difficult to find anything there.

But if you need logging to file, because let's say, you don't have some really high traffic you can enable logging to file by executing:

sudo service varnishlog start

and if you want to run it on system start

sudo chkconfig varnishlog on

Varnish logs can be found by default in /var/log/varnish . But please, be aware, that it will generate lot of writes to your hard-drive and log file will increase it's size pretty fast on heavy load websites. So this solution is not recommended.

varnishncsa

It's another useful tool for logging. It's different output from varnishlog. varnishlog will log every detail into your log about the query. If you need logs similar to for instance Apache logs, you should use varnishncsa command. By default it's disabled just like varnishlog. If you want to log to file you should start the service varnishncsa and add it to autostart with chkconfig.

varnishncsa

Performance

OK, awesome, Varnish is super cool , my websites goes blazing fast! But hey! Is it possible to speed it up even more? Well, theoretically - yes. There are few tips that can increase your throughput. But you must remember one thing - the MOST important part is to have high hit rate. If you have low hit rate and high miss rate, this tips won't help you that much. So remember, achieve high hit rate first. Then take care of Varnish tuning:)

Few tips before you start tuning up Varnish:

  • There is no one common ctrl+c ctrl+v method that will improve results on all machine
  • Change one thing at the time and then WAIT for results if it helps or not
  • Monitor statistics with Varnish tools. They will tell you whether the change will work or not
  • Remember that Varnish is blazing fast by default. If you don't have enormous traffic, you'll probably don't need these tips.
  • Increasing particular values too much can't make your hardware die. It will eat up all your CPU or RAM resources, so please be careful.
  • Monitor, monitor and monitor your system:)

OK, let's start tuning. First, i suggest to read Varnish book and chapter about tuning. It will help better understanding of what's going on.

Varnish has configuration file that contains important options that will help you increase Varnish power. Edit the file by sudo vi /etc/sysconfig/varnish

Now Varnish supports couple of ways how to configure Varnish daemon. They are all present in this file so figure out which one are you using. By default they are called Alternative 1, Alternative 2 and Alternative 3.

In my opinion Alternative 3 is the best, because you only need to change variable value instead of whole DEAMON_OPTS  string. In Varnish 4 it's enabled by default, rest of alternatives should be commented out. If not, comment out Alternative 1 and Alternative 2, and uncomment Alternative 3. But if you want you can stick to any Alternative you wish. It's just the matter of passing configuration to Varnish daemon.

You can set Varnish port there, secret file etc. But among the settings you will find threads section, and storage type. Let's start with storage type.

Files or RAM for cache? 

# # Cache file size: in bytes, optionally using k / M / G / T suffix,
# # or in percentage of available disk space using the % suffix.
VARNISH_STORAGE_SIZE=256M
#
# # Backend storage specification
VARNISH_STORAGE="malloc,${VARNISH_STORAGE_SIZE}"

Now it's up to You and your server resources how to set these values. By default Varnish will use your RAM as a storage. It also can use the file method, but keeping things in RAM is lot faster than reading them from hard-drive, especially if it's not SSD. But if you have little RAM on your server and lot of URLs to cache, use file , instead of malloc .

So this is one thing that will speed up your Varnish - using RAM. How many? Larger value is definitely better, but keep in mind, that it's not strict value. Varnish will add overhead per each object in cache (around 1kb). So if you have large amount of objects, it will increase storage size. You need to play around with this value a bit and check how much is enough. Keep in mind that you need to have enough free RAM for other processes like httpd or php-fpm.

If you will use RAM check if the storage size is big enough to handle all your requests. Use this command:

varnishstat -1 | grep n_lru_nuked

If the number of LRU nuked objects is greater than 0, then it means that or your storage size is not big enough, or you have something wrong with caching. For instance you try to cache assets or images for very long period of time.

Threads settings

Second thing that will increase Varnish performance are threads. In the same file you will find settings for them:

# # The minimum number of worker threads to start
VARNISH_MIN_THREADS=50
#
# # The Maximum number of worker threads to start
VARNISH_MAX_THREADS=1000

There are multiple options for threading in Varnish, but you should mostly play around with these two values. You can increase both values here, but you should keep withing the Varnish guidelines. Here is explanation how Varnish threads works from Varnish documentation.

When a connection is accepted, the connection is delegated to one of these thread pools. The thread pool will further delegate the connection to available thread if one is available, put the connection on a queue if there are no available threads or drop the connection if the queue is full.

Read here how to calculate how many threads you should run to not to kill your server.

Varnish and security

There are various of methods how to secure more your Varnish. We will not cover them here. But to make things clear - most important thing is to secure your purging requests. Do not allow anyone to purge your cache from outside. He can purge entire domain, and if you have huge traffic - it can kill your website when all requests will come to your backend server.

You can install also firewall for Varnish. It's really similar to ModSecurity for Apache. You have basically two options:

They are just additional rules in VCL that will block unwanted requests.

Our opinion about that? They are OK, but we like to keep Varnish VCL as simple as possible and do not add additional overhead to parsing requests. In addition we have ModSecurity for Apache installed, and it does basically the same job. So is it worth to have two security layers and add exception rules to both of them? We will just stick to Apache ModSecurity:)

Varnish on port 80

Now it's the most important part. For now we use Varnish on "test" port 6081. Before you will switch Varnish to port 80, make sure, that IT WORKS. If requests are cached correctly etc. It's important to put working Varnish as an entry point to your server, instead of just additional layer that will forward all your requests to Apache.

First you need to pick the new port for Apache. Common practice is to use 8080, but this port is often used for different tools like SOLR. We usually set port 81 for Apache. After you will pick a port number, you need to edit few files.

Note that if you are not following our tutorial from start, your files might be located elsewhere. 

httpd.conf

We need to change main port for Apache in httpd.conf file

# File located in /usr/local/apache2/conf/httpd.conf
Listen 81

All VirtualHosts files

All Virtual Hosts files also contain port. You need to change every single one file with new port.

#All files with .conf extension located in /usr/local/apache2/conf/vhosts
<VirtualHost *:81>
        ServerName example.com

Firewall rules for new Apache port (optional step)

This is optional step. We have working firewall on our server. If you will need to access website directly (not through Varnish cache) you can open new port. It's usually useful when it comes to debugging. If this is your app issue or is it Varnish cache fault. If You need to enable new port, add appropriate lines inside firewall rules.

# Firewall rules are in /etc/iptables.firewall.rules
-A INPUT -p tcp --dport 81 -m state --state NEW,ESTABLISHED -j ACCEPT
-A OUTPUT -p tcp --sport 81 -m state --state ESTABLISHED -j ACCEPT
-A INPUT -p tcp --sport 81 -m state --state ESTABLISHED -j ACCEPT
-A OUTPUT -p tcp --dport 81 -m state --state NEW,ESTABLISHED -j ACCEPT

Restart firewall to load new rules

sudo service firewall restart

Varnish daemon settings

Next thing is that we need to change Varnish daemon settings to listen on port 80.

# File is located in /etc/sysconfig/varnish
VARNISH_LISTEN_PORT=80

Varnish VCL file

Last thing is to change VCL file backend port, and update all rules to remove previous port 6081

# File is located in /etc/varnish/default.vcl
backend default {
    .host = "127.0.0.1";
    .port = "81";
}

# Remove :6081 if present in host
if(req.http.Host == "example.com"){

Remove old Varnish port from firewall

Remember when you added port 6081 to firewall in order to test some stuff? You will not need that port to be open.You can remove these lines from your firewall rules, and restart the firewall.

Restart httpd and Varnish

Once you change all files it's time to restart all daemons and check if everything works as expected.

sudo service httpd restart
sudo service varnish restart

Now check your website. It should be powered by Varnish now!

What's next?

Lengthy tutorial, wasn't it? But now your websites should be blazing fast! Remember to check it from time to time, if you have good hit/miss ratio:)

As always you can do everything here with Ansible and our repo, which you can find on GitHub.

Now you should be in place where you have secure CentOS, working LAMP and Varnish on top of everything. What's next? I believe that we should add nginx to our setup to keep everything nice and slim, and after that start installing some real useful things like Redis, RabbitMQ and add some good tools to PHP etc. See you next time!

Install PHP from source on CentOS

PHP – how to install from source on CentOS

In this part we will cover LAMP part of our webserver - PHP. So we will learn how to install PHP from source. - Why from source?  - You might ask. It's the same reason as with Apache. On most of the systems it's not possible to install latest version of PHP with yum or apt-get. Installation from source is relatively easy, but it might take some time. But if You are following with our series of how to setup webserver, you should have some experience with compiling Apache from source.

LAMP on steroids

This is part of our series LAMP on steroids. Check the links below to learn how to setup awesome webserver!

  1. Choosing VPS
  2. Install EPEL
  3. Install and configure Apache HTTPD server
  4. Harden Apache with ModSecurity and OWASP Core Rule Set
  5. Install and configure PHP
  6. Install and configure MySQL server
  7. Configure firewall based on iptables
  8. Create developer user and setup SSH key-pair
  9. Configure SSH
  10. Install and configure Varnish to speed up websites
  11. More to come...

PHP - install from source - how to?

We will use our Vagrant box of course, with installed Apache and ModSecurity. SSH to the server first if You are not there already and we can begin:)

Prerequirements

First we need to install some libraries that are necessary to install PHP:

yum install bzip2-devel curl-devel libjpeg-devel libpng-devel freetype-devel libc-client-devel.i686 libc-client-devel libmcrypt-devel -y

Download and unpack sources

Go to php.net download website and pick latest version of PHP. In our case it's 5.6.6, but this tutorial should work for any higher version.

cd ~/sources
wget -O php-5.6.6.tar.gz http://pl1.php.net/get/php-5.6.6.tar.gz/from/this/mirror
tar -zxvf php-5.6.6.tar.gz
cd php-5.6.6

Compile PHP from source

Now it's probably the hardest part of compiling PHP. You must provide the ./configure options and choose which modules do You want to install. For lots of needs the commands below will be sufficient, but if You need any particular library I suggest to check the PHP extensions list and find out installation options.

Commands below will enable required and basic extensions like curl, ftp, GD, IMAP, MySQL, PDO, etc. Two important things for this tutorial are --enable-opcache and --enable-fpm. We will use PHP OPCache that comes with newer versions of PHP and will use FPM instead of Apache mod_php.

./configure --enable-bcmath --with-bz2 --enable-calendar --with-curl --enable-exif --enable-ftp --with-gd --with-jpeg-dir --with-png-dir --with-freetype-dir --enable-gd-native-ttf --with-imap --with-imap-ssl --with-kerberos --enable-mbstring --with-mcrypt --with-mhash --with-mysql --with-mysqli --with-openssl --with-pcre-regex --with-pdo-mysql --with-zlib-dir --with-regex --enable-sysvsem --enable-sysvshm --enable-sysvmsg --enable-soap --enable-sockets --with-xmlrpc --enable-zip --with-zlib --enable-inline-optimization --enable-mbregex --enable-opcache --enable-fpm --prefix=/usr/local/php
make
make install

Notice, that it might take lot of time. Much longer than Apache compilation. After make install You should find PHP installed in /usr/local/php directory.

PHP Configuration

PHP-FPM setup

Before we will be able to run PHP from Apache we need to setup PHP-FPM worker. After installation there should be PHP-FPM default configuration file in installation directory. We will alter the file and then change it a bit.

cd /usr/local/php/etc
mkdir fpm.d
cp php-fpm.conf.default php-fpm.conf
vi php-fpm.conf

We need to  uncomment/change these lines:

include=etc/fpm.d/*.conf
pid = /var/run/php-fpm.pid
error_log = log/php-fpm.log

COPY EVERYTHING UNDER Pool Definitions TO CLIPBOARD AND REMOVE IT FROM php-fpm.conf FILE
;;;;;;;;;;;;;;;;;;;;
; Pool Definitions ;
;;;;;;;;;;;;;;;;;;;;

include=/etc/fpm.d/*.conf - by default there is one pool defined inside php-fpm.conf file. The best way to solve it is the same way as we solved Apache vhosts. We will include each pool in separate directory. In php-fpm.conf file one pool is already defined. We need to delete it from this file and put it inside fpm.d directory. We will have better control over the pools. The easiest way is just to Cut it from this file and paste it into new one.

Now let's create the file inside fpm.d for our example.com domain:

cd fpm.d
vi example.com.conf

PASTE TEXT FROM CLIPBOARD HERE AND CHANGE THESE LINES:

[www] -> [example_com] //Must be unique per file
user = apache
group = www
listen = 127.0.0.1:9000 //Port must be unique per file
catch_workers_output = yes
slowlog = /var/www/example.com/logs/php-fpm.slow.log
request_slowlog_timeout = 30s
php_flag[display_errors] = off
php_admin_value[error_log] = /var/www/example.com/logs/php-fpm.error.log
php_admin_flag[log_errors] = on
php_admin_value[memory_limit] = 64M
php_admin_value[open_basedir] = /var/www/example.com/htdocs

Each pool must have different name. So we need to change it from [www] to something else, for instance to domain name. It'll be easier to find the issues inside log files.

We set user and group to the same user as apache to have access to files.

Port will be different per pool. Standard way is to start from port 9000. Next will be 9001 etc.

We will catch errors and log them to file. In addition we set logging for  slow requests.

Nice part is that we can overwrite the settings from php.ini here. So we can overwrite error_log or memory_limit for instance. We should also set open_basedir so PHP will have access only to files inside our htdocs directory. Our server will be more secure with this setting.

php.ini and OPCache configuration

Second thing is php.ini file. After installation  php.ini file should located in /usr/local/php/lib. This is only the location. After compiling from source You won't anything there so we need to copy it from uncompressed sources.

cd /usr/local/php/lib
cp ~/sources/php-5.6.6/php.ini-development ./php.ini
vi php.ini

This is pretty large file with lot of configuration settings. Fortunately we only need to change some of the options:

short_open_tag = On
open_basedir = /var/www
disable_functions = exec,passthru,shell_exec,system,proc_open,popen
expose_php = Off
max_execution_time = 30
memory_limit = 64M
date.timezone = Europe/Warsaw
error_reporting = E_ALL & ~E_DEPRECATED & ~E_STRICT
display_errors = Off
display_startup_errors = Off
log_errors = On
post_max_size = 5M
upload_max_filesize = 4M

opcache.enable=1
opcache.memory_consumption=64
opcache.interned_strings_buffer=16
opcache.max_accelerated_files=7000
opcache.validate_timestamps=0 ;set this to 1 on production server
opcache.fast_shutdown=1

So we set few things here, enable <? tag, limit access to files from PHP level, disabled dangerous functions, adjusts timezone, security, max execution times, errors etc. In addition we have enable OPCache for PHP.

Each one of these options are well commented inside php.ini file. If You don't like the settings here or You need something else, feel free to change it for Your purposes.

Useful shell scripts for PHP

/etc/init.d/php-fpm

As You probably remember during Apache setup we create script so we can use service command to start / stop Apache process. Now we will do the same for PHP-FPM

With PHP source code there comes ready script for that purpose.

cd /etc/init.d
cp ~/sources/php-5.6.6/sapi/fpm/init.d.php-fpm php-fpm
vi php-fpm

Now we need to setup configuration for the file:

prefix=/usr/local/php
exec_prefix=${prefix}

php_fpm_BIN=${exec_prefix}/sbin/php-fpm
php_fpm_CONF=${prefix}/etc/php-fpm.conf
php_fpm_PID=/var/run/php-fpm.pid

Save the file and add executable permission.

chmod +x php-fpm
servcie php-fpm status
service php-fpm start
servcie php-fpm status

After that we should have php-fpm process up and running!

Add PHP to $PATH

We can do one more thing to make our life easier:) Add PHP executable to PATH, so we'll be able to call php command from every directory.

echo 'pathmunge /usr/local/php/bin' > /etc/profile.d/php.sh

Execute such command, log out, log in and You'll be able to execute:

php -v

Setup Apache for PHP-FPM

Now is the time to finally setup Apache for .php files. Let's edit one of the Virtual Hosts now.

vi /usr/local/apache2/conf/vhosts/example.com.conf

<VirtualHost *80>
    ServerName example.com

    <LocationMatch "^/(.*\.php(/.*)?)$">
        ProxyPass fcgi://127.0.0.1:9000/var/www/example.com/htdocs/$1
    </LocationMatch>

////Rest of the file below

So basically we need to proxy all files with .php extension to our PHP-FPM process.  Also we need to restart Apache and make sure PHP-FPM is running httpd server:

service php-fpm start
service httpd restart

How to test if PHP is working?

We need to test if our PHP installation works. The easiest way to debug and check what's going on would be to create test.php file inside our /var/www directory.

vi /var/www/example.com/htdocs/test.php

and paste phpinfo() function there:

<?php

phpinfo();

Save the file and open the file in Your browser, assuming that your vagrant setup is correct. For instance http://example.com/test.php or 192.168.99.99/test.php

If everything is OK you should get information about PHP installation. Well done!

What's next

We are one step closer to our LAMP server. The only thing we are missing now is MySQL which we will install in upcoming episodes.

If You are running Ansible for provisioning You can find everything from this series inside my GitHub.

Hardening Apache with Mod Security

Apache hardening with mod_security

In this part of setup complete webserver we will harden Apache with popular mod_security. Mod_security is a small module that works like application firewall. It protect the app before most common attacks and vulnerabilities. It's good to have such thing on the webserver.

I assume that You have Apache already installed, if not check out previous part - How to install Apache on CentOS?

LAMP on steroids

This is part of our series LAMP on steroids. Check the links below to learn how to setup awesome webserver!

  1. Choosing VPS
  2. Install EPEL
  3. Install and configure Apache HTTPD server
  4. Harden Apache with ModSecurity and OWASP Core Rule Set
  5. Install and configure PHP
  6. Install and configure MySQL server
  7. Configure firewall based on iptables
  8. Create developer user and setup SSH key-pair
  9. Configure SSH
  10. Install and configure Varnish to speed up websites
  11. More to come...

How to install mod_security on Apache httpd server

First thing is installation of required tools. We need them to compile mod_security.

yum install automake libtool libxml2-devel

Next thing that we need is to download and decompress mod_security. Download links can be found on official mod_security page.

cd ~/sources
wget https://www.modsecurity.org/tarball/2.9.0/modsecurity-2.9.0.tar.gz
tar -zxvf modsecurity-2.9.0.tar.gz

Now it's time to compile mod_security. While ./confgure we need to pass paths to axps and apr binaries. All binaries should be inside bin directory in apache installation path.

cd modsecurity-2.9.0
./autogen.sh
./configure --with-apxs=/usr/local/apache2/bin/apxs --with-apr=/usr/local/apache2/bin/apr-1-config --with-apu=/usr/local/apache2/bin/apu-1-config
make
make install
cp /usr/local/modsecurity/lib/mod_security2.so /usr/local/apache2/modules

If there was no error mod_security is ready to use. It can be found in /usr/local/modsecurity We need to copy generated .so file to apache extension directory.

ModSecurity and OWASP rules

mod_security is nothing without the rules that tells what attacks should be blocked. Fortunately there is a great package with lot of rules provided by OWASP. We will use such rules package to harden Apache HTTPD. So let's download, unzip and copy the rules to apache configuration directory.

cd ~/sources
wget -O owasp.tar.gz https://github.com/SpiderLabs/owasp-modsecurity-crs/tarball/master
mkdir /usr/local/apache2/conf/crs
tar -zxvf owasp.tar.gz -C /usr/local/apache2/conf/crs --strip 1
cd /usr/local/apache2/conf/crs
cp modsecurity_crs_10_setup.conf.example modsecurity_crs_10_setup.conf

Now we are ready to use ModSecurity!

ModSecurity configuration

In previous article we added httpd-security.conf file with some basic rules that improves Apache security.  We will modify this file to load mod_security with OWASP rules and add some basic configuration. You need to know that mod_security is pretty large module with tons of configuration option. You can find them in ModSecurity reference manual.

vi /usr/local/apache2/conf/extra/httpd-security.conf

Once You open the file add these lines somewhere in the file:

LoadModule security2_module modules/mod_security2.so

<IfModule security2_module>
      Include conf/crs/modsecurity_crs_10_setup.conf
      Include conf/crs/base_rules/*.conf
      # Include conf/crs/experimental_rules/*.conf
      # Include conf/crs/optional_rules/*.conf

      SecRuleEngine On
      SecRequestBodyAccess On
      SecResponseBodyAccess On 
      SecResponseBodyMimeType text/plain text/html text/xml application/octet-stream
      SecDataDir /tmp

      # Debug log
      SecDebugLog /usr/local/apache2/logs/modsec_debug.log
      SecDebugLogLevel 3

      SecAuditEngine RelevantOnly
      SecAuditLogRelevantStatus ^2-5
      SecAuditLogParts ABCIFHZ
      SecAuditLogType Serial
      SecAuditLog /usr/local/apache2/logs/modsec_audit.log
</IfModule>

So from the top:

  • First we need to load mod_security module.
  • Next are rules from OWASP that we will include to ModSecurity. We need to include the setup and base rules. OWASP core rule set comes with lot more features that are marked as optional or experimental. We can enable those rules, but we also need to remember that it might not play well with our website. It's rather testing by trial and error then one rule will work well on every website. But in general including base_rules is OK.
  • SecRuleEngine enables detection and blocking of malicious attacks.
  • SecRequestBodyAccess enable inspection of data transported request  bodies
  • SeResponseBodyAccess buffer response bodies matched by SecResponseBodyMimeType
  • SecDataDir working directory for ModSecurity temporary purposes
  • Next thing is Debug log. By default all error logs goes to apache error log, but we can set different path to debug log. Best practice would be to change it per domain inside particular VirtualHost file. In previous article we setup directory structure and we have logs directory there. It would be wise to used it for debug log as well.
  • Audit Log is complementary log for Debug log. It has detail information about every error. It's disabled by default so we need to enable it and turn on logging relevant (warnings and errors) issues. Next options are for configuration the audit log. In general there are lot more of discussing at this topic.

If You want to learn more about how to setup and read mod_security logs, here is really great article about mod_security logging by Infosec Institue.

Now we just need to save the file restart apache and our httpd server has better security.

service httpd restart

What's next?

If You are following our series, You should have now part of LAMP stack (Linux Apache MySQL PHP). Apache is secured with mod_security.

Small note to those who would like to install mod_evasive as well to increase the security. To be honest, it's really not worth to install it on Apache. Why? Because when You run multiple instances via MPM mod_evasive doesn't share the info between the MPM instances. It means that one instance of apache can block the attacker but others wont. So if You have many MPM workers mod_evasive is just useless.

As always, if You are using Ansible for server provisioning You can use ready playbook, that will cover everything in this series. You can find it on GitHub.

In next episode we will add P to our LAMP server.

How to install apache from source on CentOS

How to install Apache 2.4.23 on CentOS from source

Hi there! Today I'd like to show you how I install and configure Apache httpd on CentOS. I like to have it installed in minimal and secure way. First part is about installation, second about configuration. Last part is about setting up Virtual Hosts.

How to install Apache httpd on CentOS - easy way

There are two ways to install Apache httpd on CentOS. First is with yum and it is the simplest version:

sudo yum install httpd -y

Volia! You have httpd installed. However, if you check the version:

httpd -v

You will most probably get 2.4.6 version or slightly newer. If you check Apache website, you will note, that they have 2.4.23 version available. So, if you want to have access to latest features such as HTTP/2 support or latest bugfixes, you will have to try more difficult method which is installing Apache httpd  from source.

How to install Apache httpd from source on CentOS?

Installing and compiling software from the source code might sound scary. But trust me, it's not. It takes more time to have Apache httpd up and running, than installing it with yum. However it comes with some benefits that I will mention during this tutorial.

Remove old Apache httpd

Before you will even begin, make sure that you don't have Apache httpd installed. In general you don't want to have two version installed on the same machine, unless you are doing some A/B testing or testing versions on different ports etc. But for this tutorial I want to start clean, so I remove installed Apache httpd:

sudo yum remove httpd -y

Compile and install Apache httpd

Required tools for building

You need to install some tools that will help us compile Apache. It's basic stuff like compiler, required libraries etc:

sudo yum install autoconf libtool openssl-devel pcre-devel -y

Download and unpack source code

Next thing that you need are packages with source files. For compiling Apache, you will need 3 different packages - httpd itself, apr and apr-util. Last two are Apache Runtime libraries. They are required for Apache httpd. When you install Apache httpd with yum they come as dependencies.

There are two ways of getting them. Either you can clone them from git repository or you can download them as  tar.gz package. I must say that as I love git, for downloading sources I prefer downloading compressed package. Why? When you clone repository it usually downloads whole history, branches etc. Apache is for instance around 290MB of files. Compressed package is about 10MB. So it is much faster to download just required files instead of cloning whole repo.

I like to download packages from GitHub releases. Here are the links to the packages:

Click on tar.gz icon, copy the link to package and download them with curl or wget. Or simply copy commands below:

curl -O -L https://github.com/apache/httpd/archive/2.4.23.tar.gz
curl -O -L https://github.com/apache/apr/archive/1.5.2.tar.gz
curl -O -L https://github.com/apache/apr-util/archive/1.5.4.tar.gz

Unpack downloaded sources:

tar -zxvf 2.4.23.tar.gz
tar -zxvf 1.5.2.tar.gz
tar -zxvf 1.5.4.tar.gz

APR and APR-Util

Apache requires APR library to be present in the system. You can manually compile and install APR and APR-util first and then do the same with httpd. But I prefer to do it in one shot. First you need to copy the source codes to correct directory:

cp -r apr-1.5.2 httpd-2.4.23/srclib/apr
cp -r apr-util-1.5.4 httpd-2.4.23/srclib/apr-util

It's important to not to include version number in APR directories. If you just copy apr-1.5.2 without changing the name, it will give you a warning about missing apr directory.

Compilation

Now you are ready to compile Apache httpd. It's important that you should not use root user for compilation. It can lead to serious security issues. I described it more on my other tutorial about installing GIT. In short words, imagine that you downloaded package from wrong source with malicious code. If you would compile it as root user, anything can happen to your server. Including cutting of your root access. I'm not saying that it's not possible to compile packages as root, because it is. It's just not safe. If you want to create separate user with sudo powers, you can read this tutorial.

So get inside httpd directory and compile your Apache httpd version:

cd httpd-2.4.23
./buildconf
./configure --enable-ssl --enable-so --with-mpm=event --witn-included-apr --prefix=/usr/local/apache2
make

First command ./buildconf will build ./configure file required for configuration of the build.

./configure command will setup everything for compilation of Apache httpd. Here are the options that I use:

  • --enable-ssl will build Apache with SSL support, so you can enable HTTPS on your websites.
  • --enable-so will enable dynamically loaded modules. So you can enable and disable modules without recompilation (I will describe modules in configuration part)
  • --with-mpm will set multiprocessing modules for Apache. I'm using event, but you can use worker or prefork instead. event works best for me and I think that it is mpm that will give you most performance.
  • --with-included-apr It will use APR library that you copied to srclib directory
  • --prefix is the installation path for Apache httpd compiled package

Whole process might take a while. It depends how fast your server is.

Installation

After it's compiled you can install it. For that you need sudo or root account:

sudo make install

Apache should be installed in the directory you specified with --prefix option.

Cleanup

Last thing you can do now is to remove downloaded files. You won't need them now. It's not mandatory, but it's nice to keep server clean.

rm -rf 1.5.2.tar.gz 1.5.4.tar.gz 2.4.23.tar.gz apr-1.5.2 apr-util-1.5.4 httpd-2.4.23

Set system scripts for Apache httpd

Before I will show you how I configure my Apache httpd server, I want to show you two really helpful scripts.

Add Apache httpd to $PATH

If you try to type httpd -v in your command line, it will result in command not found. That's because httpd is not on your $PATH. I'd like to have all executables from Apache available from everywhere. In order to achieve that, create file

sudo vi /etc/profile.d/httpd.sh

and paste there following contents:

pathmunge /usr/local/apache2/bin

Save the file, log out and log in from your current session to reload your profile. After that you should be able to use httpd -v command:)

SystemD entry

Second really useful script is SystemD entry. It will allow you to start, restart and stop Apache httpd from systemctl. You need to create another file:

sudo vi /etc/systemd/system/httpd.service

and paste there following contents:

[Unit]
Description=The Apache HTTP Server
After=network.target

[Service]
Type=forking
ExecStart=/usr/local/apache2/bin/apachectl -k start
ExecReload=/usr/local/apache2/bin/apachectl -k graceful
ExecStop=/usr/local/apache2/bin/apachectl -k graceful-stop
PIDFile=/usr/local/apache2/logs/httpd.pid
PrivateTmp=true

[Install]
WantedBy=multi-user.target

Save the file and reload the systemctl daemon

sudo systemctl daemon-reload

Now you can try to start your Apache httpd server with following command:

sudo systemctl start httpd

It should start properly. If you will have any warnings, don't bother with them now. I will show you proper configuration in next step.

Once it's up and running you can try to type your server IP address in your browser like http://43.184.89.190/ and check if you see It works! message:) If so, you have Apache httpd running fine!

How to configure Apache httpd properly

Although Apache httpd is working and you can use it's default configuration it's good to tune it up a bit. It's always nice to gain additional milliseconds and security.

Apache user and group

Before I will show you how to change configuration I usually create additional user and group for httpd daemon. It's good practice from security side. Each service should operate as separate user. It limits possible damage during attacks, httpd exploitation etc.

If you want to learn more about creating user and groups I recommend reading this tutorial. Here I'll just simply create group and user without shell.

sudo groupadd www
sudo useradd httpd -g www --no-create-home --shell /sbin/nologin

You can change the names as you wish. I like to use www group instead of httpd group for example. I usually add there other services as well, like nginx or php-fpm.

Configuration of httpd.conf

httpd.conf is main Apache httpd configuration file. You should start by editing this file:

sudo vi /usr/local/apache2/conf/httpd.conf

There are couple of options that we should set. Just scroll the file and edit what you need. Values in code blocks are the values that I use:

ServerRoot

ServerRoot /usr/local/apache2

Make sure that ServerRoots direct to the same path as you set via --prefix during .configure It needs to point to installation directory.

Port

Listen 80

This is Apache port under which it should listen for incoming connections.

Modules

LoadModule .....

List of modules is pretty long. Some of the are disabled (they have # at the beginning of the line). Some of them are enabled. You can leave them as they are. Eventually you can enable additional modules.

You should know, that the more modules are enabled the  "slower" Apache httpd is. I'm not saying that it's super slow, but you can google for some benchmarks showing different configurations.

Here is what I like to do. First of all, I comment out all modules = everything is disabled. I enable only that modules that I really use + the modules that are required for proper functioning of Apache httpd. It has few benefits - Apache is faster, eats less resources (CPU and RAM) and it's more resistant for given attacks. Usually when new security issue pops out, it's rather connected to one of the modules, than whole httpd. So you can have more chances to avoid potential security risk with having some stuff disabled.

So here is what I'm using. First part are essential modules that must be enabled for proper Apache functioning under UNIX systems:

LoadModule authz_core_module modules/mod_authz_core.so
LoadModule mime_module modules/mod_mime.so
LoadModule log_config_module modules/mod_log_config.so
LoadModule unixd_module modules/mod_unixd.so
LoadModule dir_module modules/mod_dir.so

Next thing are optional modules that you might want or might not want to enable.

LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_fcgi_module modules/mod_proxy_fcgi.so

I'm using php-fpm for running PHP on my servers. I'm using proxy for that purpose. More about that topic in next part of the article.

LoadModule alias_module modules/mod_alias.so

It allows Alias command. Super useful when you are running WordPress with composer and wpackagist for setting up alias to wp-content directory.

LoadModule access_compat_module modules/mod_access_compat.so

Also useful for WordPress pages. With Apache 2.4 Order command is disabled by default in favor of Require. However some WordPress plugins are still using Order. If you have any issues with that, just enabled access_compat module

LoadModule rewrite_module modules/mod_rewrite.so

Most popular module for almost all CMSes and Framworks. It allows pretty urls and so on. I still wonder why it's note enabled by default?

And that is the whole list of the modules I'm using. Sometimes when I can see that website is not working properly I check error logs. You can figure out that there is some module that is required. Simply enable it, restart the server and check if it's working fine.

User and group

User httpd
Group www

User and group that you created in previous step. It should be set to corresponding values.

ServerName

ServerName localhost

Simplest to set it to localhost here. It will suppress the warning during Apache start for default value.

DirectoryIndex

DirectoryIndex index.php index.html

This is the file that should be loaded when accessing directory. If you are using PHP, you should add index.php at first place, like in example above. However if you just using node.js or plain HTML, index.html is enough.

Save the changes to file and try to restart httpd.

sudo systemctl restart httpd

Check if everything is working fine. That was just some basic configuration. Here are some additional parameters that will increase security on the server. They are not present in httpd.conf file, so you need to add them manually. I like to add them at the end of the file.

ServerTokens

ServerTokens prod

By default in headers that are full information about Apache version, PHP version etc. I set it to prod, so it will only show, that it's powered by Apache. No versions etc.

ServerSignature

ServerSignature off

Disable server signature in internal Apache document's footer. It will hide Apache version in those files as well

FileETag

FileETag none

Disable tagging files with tags. Usually there are additional headers added with information that should not be visible outside (ie. Inode)

Remember that after the changes you need to restart Apache. Otherwise changes won't be applied!

 

MPM configuration

Last thing from configuration I would like to show you is enabling MPM settings. At the bottom of httpd.conf you need to uncomment the line:

Include conf/extra/httpd-mpm.conf

It will enable advanced Apache httpd MPM configuration and it will override the defaults.

Now edit enabled file:

sudo vi /usr/local/apache2/conf/extra/httpd-mpm.conf

There are configuration for each MPM module, so make sure that you are configuring correct values. I enabled event mode, so this is the section I care about:


StartServers 5
MinSpareThreads 75
MaxSpareThreads 250
ThreadsPerChild 25
MaxRequestWorkers 400
MaxConnectionsPerChild 0

Important thing that you need to know - there is no one config to rule them all. What I want to say that configuration that works on one server, will work on second server. There are dozens of factors like application specific, CPU and RAM, traffic etc. I encourage you to play around with these values to find the optimal settings for your server.

Here is great post that explains in depth MPM configuration. I have two advice's about performance tuning.

First one, that you probably don't even need to change anything if you have regular website without huge amount of traffic. It's just good to know what settings are applied by mpm configuration. When httpd-mpm.conf file was commented out, you didn't have any idea what the settings are. Once it's enabled, you at least know what is configured.

Second advice is - do it slowly. Performance tuning is lengthy process, and as I said, it highly depends on various factors. I like to change one settings, like StartServers for instance and I wait day or two and monitor response times, CPU and RAM usage etc. Sometimes even if you increase something you won't see a difference in response time, but you will get higher CPU usage. Then you just can rollback the changes. If you modify 3 or 4 values at one time, it's hard to say which comes with best (or any) result.

Remember to restart Apache httpd after making changes:)

 

Setting up Virtual Hosts for websites

I want to show you how I setup VirtualHosts for my websites.  There are tons of different ways of storing Vhost files. Some of people like to keep everything in one file, some likes symbolic links etc. It actually doesn't really matter how you do it. But it's nice to keep it consistent and easy to manage.

Root directory

First I make sure that I have /var/www directory present in my system. If you don't have one, create it. Make sure that it has root as an owner and 755 permissions.

sudo mkdir /var/www
sudo chmod 755 /var/www
sudo chown root:root /var/www

Website directories

Second thing are websites directories. I create such directory, even if I have only one website on my server. For directory name I'm using domain name. In each directory I create two subdirectories - htdocs for storing website files and logs for storing logs. So for instance:

sudo mkdir /var/www/blacksaildivision.com
sudo mkdir /var/www/blacksaildivision.com/htdocs
sudo mkdir /var/www/blacksaildivision.com/logs

Directory ownership

When it comes to ownership to main directory I set root:root. When it comes to subdirectories I set owner as developer or any other user account (other than root) that can clone repositories, use files etc. As group, I'm using www group. As you might remember Apache httpd is added to this group as well.

sudo chown root:root /var/www/blacksaildivision.com
sudo chown developer:www /var/www/blacksaildivision.com/htdocs
sudo chown developer:www /var/www/blacksaildivision.com/logs

Directory permissions

Permissions to website directory is the same as for /var/www. For htdocs and logs I'm using 2775. So each user in www group will be able to write to these files. 2 at the beginning means that every new subdirectory created will have the same ownership as parent directory (developer:www)

sudo chmod 755 /var/www/blacksaildivision.com
sudo chmod 2775 /var/www/blacksaildivision.com/htdocs
sudo chmod 2775 /var/www/blacksaildivision.com/logs

 VirtualHost file

Once you have your directory structure ready, you can create VirtualHost files. I create one file per domain. All files are stored in conf/extra directory with the same name scheme as other files there. So let's create file there:

sudo vi /usr/local/apache2/conf/extra/httpd-vhost-blacksaildivision.com.conf

and paste there following contents:

<VirtualHost *:80>
ServerName blacksaildivision.com

# Directory settings
DocumentRoot /var/www/blacksaildivision.com/htdocs
<Directory /var/www/blacksaildivision.com/htdocs>
AllowOverride All
Require all granted
Options +FollowSymLinks -Indexes -Includes
</Directory>

# PHP-FPM settings
ProxyPassMatch "^/(.*\.php(/.*)?)$" "fcgi://127.0.0.1:9000/var/www/blacksaildivision.com/htdocs"

# Logging
ErrorLog "/var/www/blacksaildivision.com/logs/httpd-error.log"
CustomLog "/var/www/blacksaildivision.com/logs/httpd-access.log" common

</VirtualHost>

File has few sections. Everything is wrapped with VirtualHost block with Apache httpd port.

ServerName stands for domain name. Each file has to have different domain name to avoid conflicts.

Next are directory config. DocumentRoot is basically the path to directory you created before. Now you need to specify settings of this directory. AllowOverride All means that I'm allowing .htaccess files to override all settings. Require all granted is a must here if you want to have access to this directory. Last thing are Options. I disable Indexes and Includes for security reasons. FollowSymLinks is usually required by frameworks and CMSes.

I'm using PHP in PHP-FPM mode, so I need to pass all requests to PHP files via Proxy to FPM daemon.

Last parts are logs. Error log will contain information about errors. Custom log is like access log. All requests to your domain will be stored there.

Save the changes to the file. Now you need to include this file in main Apache httpd config file. So edit it:

sudo vi /usr/local/apache2/conf/httpd.conf

and at the end of the file include newly created file:

Include conf/extra/httpd-vhost-blacksaildivision.com.conf

Restart Apache httpd daemon and try to access your website. It should work fine. If not, please check httpd-error.log in logs directory for possible errors.

 

 Run Apache httpd on system start

Last thing is to add Apache httpd daemon to start with system boot. So after server start/restart  httpd will run automatically:

sudo systemctl enable httpd

So that's it. You have fully working Apache httpd in latest version installed on your system 🙂 This process might take some time, but you will have full control over httpd.

As always you can use our LampOnSteroids project (based on Ansible) to speed and automate everything up!