Tag Archives: Ansible

Hardening Apache with Mod Security

Apache hardening with mod_security

In this part of setup complete webserver we will harden Apache with popular mod_security. Mod_security is a small module that works like application firewall. It protect the app before most common attacks and vulnerabilities. It's good to have such thing on the webserver.

I assume that You have Apache already installed, if not check out previous part - How to install Apache on CentOS?

[sc:lamp_series]

How to install mod_security on Apache httpd server

First thing is installation of required tools. We need them to compile mod_security.

yum install automake libtool libxml2-devel

Next thing that we need is to download and decompress mod_security. Download links can be found on official mod_security page.

cd ~/sources
wget https://www.modsecurity.org/tarball/2.9.0/modsecurity-2.9.0.tar.gz
tar -zxvf modsecurity-2.9.0.tar.gz

Now it's time to compile mod_security. While ./confgure we need to pass paths to axps and apr binaries. All binaries should be inside bin directory in apache installation path.

cd modsecurity-2.9.0
./autogen.sh
./configure --with-apxs=/usr/local/apache2/bin/apxs --with-apr=/usr/local/apache2/bin/apr-1-config --with-apu=/usr/local/apache2/bin/apu-1-config
make
make install
cp /usr/local/modsecurity/lib/mod_security2.so /usr/local/apache2/modules

If there was no error mod_security is ready to use. It can be found in /usr/local/modsecurity We need to copy generated .so file to apache extension directory.

ModSecurity and OWASP rules

mod_security is nothing without the rules that tells what attacks should be blocked. Fortunately there is a great package with lot of rules provided by OWASP. We will use such rules package to harden Apache HTTPD. So let's download, unzip and copy the rules to apache configuration directory.

cd ~/sources
wget -O owasp.tar.gz https://github.com/SpiderLabs/owasp-modsecurity-crs/tarball/master
mkdir /usr/local/apache2/conf/crs
tar -zxvf owasp.tar.gz -C /usr/local/apache2/conf/crs --strip 1
cd /usr/local/apache2/conf/crs
cp modsecurity_crs_10_setup.conf.example modsecurity_crs_10_setup.conf

Now we are ready to use ModSecurity!

ModSecurity configuration

In previous article we added httpd-security.conf file with some basic rules that improves Apache security.  We will modify this file to load mod_security with OWASP rules and add some basic configuration. You need to know that mod_security is pretty large module with tons of configuration option. You can find them in ModSecurity reference manual.

vi /usr/local/apache2/conf/extra/httpd-security.conf

Once You open the file add these lines somewhere in the file:

LoadModule security2_module modules/mod_security2.so

<IfModule security2_module>
      Include conf/crs/modsecurity_crs_10_setup.conf
      Include conf/crs/base_rules/*.conf
      # Include conf/crs/experimental_rules/*.conf
      # Include conf/crs/optional_rules/*.conf

      SecRuleEngine On
      SecRequestBodyAccess On
      SecResponseBodyAccess On 
      SecResponseBodyMimeType text/plain text/html text/xml application/octet-stream
      SecDataDir /tmp

      # Debug log
      SecDebugLog /usr/local/apache2/logs/modsec_debug.log
      SecDebugLogLevel 3

      SecAuditEngine RelevantOnly
      SecAuditLogRelevantStatus ^2-5
      SecAuditLogParts ABCIFHZ
      SecAuditLogType Serial
      SecAuditLog /usr/local/apache2/logs/modsec_audit.log
</IfModule>

So from the top:

  • First we need to load mod_security module.
  • Next are rules from OWASP that we will include to ModSecurity. We need to include the setup and base rules. OWASP core rule set comes with lot more features that are marked as optional or experimental. We can enable those rules, but we also need to remember that it might not play well with our website. It's rather testing by trial and error then one rule will work well on every website. But in general including base_rules is OK.
  • SecRuleEngine enables detection and blocking of malicious attacks.
  • SecRequestBodyAccess enable inspection of data transported request  bodies
  • SeResponseBodyAccess buffer response bodies matched by SecResponseBodyMimeType
  • SecDataDir working directory for ModSecurity temporary purposes
  • Next thing is Debug log. By default all error logs goes to apache error log, but we can set different path to debug log. Best practice would be to change it per domain inside particular VirtualHost file. In previous article we setup directory structure and we have logs directory there. It would be wise to used it for debug log as well.
  • Audit Log is complementary log for Debug log. It has detail information about every error. It's disabled by default so we need to enable it and turn on logging relevant (warnings and errors) issues. Next options are for configuration the audit log. In general there are lot more of discussing at this topic.

If You want to learn more about how to setup and read mod_security logs, here is really great article about mod_security logging by Infosec Institue.

Now we just need to save the file restart apache and our httpd server has better security.

service httpd restart

What's next?

If You are following our series, You should have now part of LAMP stack (Linux Apache MySQL PHP). Apache is secured with mod_security.

Small note to those who would like to install mod_evasive as well to increase the security. To be honest, it's really not worth to install it on Apache. Why? Because when You run multiple instances via MPM mod_evasive doesn't share the info between the MPM instances. It means that one instance of apache can block the attacker but others wont. So if You have many MPM workers mod_evasive is just useless.

As always, if You are using Ansible for server provisioning You can use ready playbook, that will cover everything in this series. You can find it on GitHub.

In next episode we will add P to our LAMP server.

How to install apache from source on CentOS

How to install latest Apache HTTPD on CentOS

Hi there! Today I'd like to show you how do I install and configure Apache HTTPD on CentOS 7. I like to have it installed in minimal and secure way. Beware! This is pretty long tutorial covering lot's of aspects from compilation, through configuration, SSL/HTTPS/HTTP2, basic hardening etc.

How to install Apache HTTPD on CentOS using yum - easy way

There are two ways to install Apache HTTPD on CentOS. First is with yum and it is the simplest version:

sudo yum install httpd -y

Volia! You have httpd installed. However, if you check the version:

httpd -v

You will most probably get 2.4.6 version or slightly newer. If you check Apache website, you will note, that they have 2.4.33 version available. So, if you want to have access to latest features such as HTTP/2 support or latest bugfixes, you will have to try more difficult method which is compiling Apache from source. It might seem complicated but it's really not.

OpenSSL - do you have latest version?

If you want to enable HTTP/2 in Apache HTTPD which I strongly recommend for increased performance you need to have latest version of OpenSSL installed in your system. Older version does not support it, so you need to compile new version from source. I have separate tutorial how to install latest OpenSSL - make sure that you follow that first and then get back to this tutorial.

Remove old HTTPD first

Make sure that you don't have httpd installed. On some machines it comes by default, or you might be using older version. In order to avoid complications later I advise you to remove it first. However you must know that if you have some websites online that are using Apache, they will have some downtime before you setup new Apache. Execute following command to remove current Apache httpd from your system:

sudo yum remove httpd -y

Install EPEL

There are lots of different libraries in EPEL, but for compiling Apache HTTPD with HTTP/2 support we need one thing that EPEL provides - libnghttp2

In order to install EPEL repository execute following command:

sudo yum install epel-release -y

Install required tools for compilation

You need to install some tools that will help us compile Apache. It's basic stuff like compiler, required libraries etc:

sudo yum install autoconf expat-devel libtool libnghttp2-devel pcre-devel -y

Download and unpack source code

Next thing that you need are packages with source files. For compiling Apache, you will need 3 different packages - httpd itself, apr and apr-util. Last two are Apache Runtime libraries. They are required for Apache HTTPD.

I like to download packages from GitHub releases. Here are the links to the packages:

Click on tar.gz icon, copy the link to package and download them with curl or wget. Or simply copy commands below:

curl -O -L https://github.com/apache/httpd/archive/2.4.33.tar.gz
curl -O -L https://github.com/apache/apr/archive/1.6.3.tar.gz
curl -O -L https://github.com/apache/apr-util/archive/1.6.1.tar.gz

Unpack downloaded sources:

tar -zxvf 2.4.33.tar.gz
tar -zxvf 1.6.3.tar.gz
tar -zxvf 1.6.1.tar.gz

APR and APR-Util

Apache requires APR library for compilation. You need to copy the source codes to correct directory:

cp -r apr-1.6.3 httpd-2.4.33/srclib/apr
cp -r apr-util-1.6.1 httpd-2.4.33/srclib/apr-util

It's important to not to include version number in APR directories. If you just copy apr-1.6.3 without changing the name, it will give you a warning about missing apr directory.

Compile source code

Now you are ready to compile Apache httpd. It's important that you should not use root user for compilation. It can lead to serious security issues. I described it more on my other tutorial about installing GIT. In short words, imagine that you downloaded package from wrong source with malicious code. If you would compile it as root user, anything can happen to your server. Including cutting of your root access. I'm not saying that it's not possible to compile packages as root, because it is. It's just not safe. If you want to create separate user with sudo powers, you can read this tutorial.

So get inside httpd directory and compile your Apache version:

cd httpd-2.4.33
./buildconf
./configure --enable-ssl --enable-so --enable-http2 --with-mpm=event --with-included-apr --with-ssl=/usr/local/openssl --prefix=/usr/local/apache2
make

First command ./buildconf will build ./configure file required for configuration of the build.

./configure command will setup everything for compilation of Apache HTTPD. Here are the options that I use:

  • --enable-ssl will build Apache with SSL support, so you can enable HTTPS on your websites.
  • --enable-so will enable dynamically loaded modules. So you can enable and disable modules without recompilation (I will describe modules in configuration part)
  • --enable-http2 will enable HTTP/2 support.
  • --with-mpm will set multiprocessing modules for Apache. I'm using event, but you can use worker or prefork instead. event works best for me and I think that it is mpm that will give you most performance.
  • --with-included-apr It will use APR library that you copied to srclib directory
  • --with-ssl will point compiler to newer version of OpenSSL. Make sure that you compiled it first!
  • --prefix is the installation path for Apache httpd compiled package

Whole process might take a while. It depends how fast your server is.

Install HTTPD

After it's compiled you can install it. For that you need sudo or root account:

sudo make install

Apache should be installed in the directory you specified with --prefix option.

Cleanup

Last thing you can do now is to remove downloaded files. You won't need them now. It's not mandatory, but it's nice to keep server clean.

cd ..
rm -rf 1.6.3.tar.gz 1.6.1.tar.gz 2.4.33.tar.gz apr-1.6.3 apr-util-1.6.1 httpd-2.4.33

Add Apache executables to PATH

If you try to type httpd -v in your command line, it will result in command not found. That's because httpd is not on your $PATH. I'd like to have all executables from Apache available from everywhere. In order to achieve that, create file

sudo vi /etc/profile.d/httpd.sh

and paste there following contents:

pathmunge /usr/local/apache2/bin

Save the file, log out and log in from your current session to reload your profile. After that you should be able to use httpd -v command:)

Add Systemd entry

Starting, restarting, and enabling Apache on server start via systemctl command is very important thing.  You need to create another file:

sudo vi /etc/systemd/system/httpd.service

and paste there following contents:

[Unit]
Description=The Apache HTTP Server
After=network.target

[Service]
Type=forking
ExecStart=/usr/local/apache2/bin/apachectl -k start
ExecReload=/usr/local/apache2/bin/apachectl -k graceful
ExecStop=/usr/local/apache2/bin/apachectl -k graceful-stop
PIDFile=/usr/local/apache2/logs/httpd.pid
PrivateTmp=true

[Install]
WantedBy=multi-user.target

Save the file and reload the systemctl daemon

sudo systemctl daemon-reload

Now you can try to start your Apache httpd server with following command:

sudo systemctl start httpd

It should start properly. If you will have any warnings, don't bother with them now. I will show you proper configuration in next step.

Once it's up and running you can try to type your server IP address in your browser like http://43.184.89.190/ and check if you see It works! message:) If so, you have Apache httpd running fine!

Create dedicated user and group for Apache

I usually create additional user and group for httpd daemon. It's good practice from security side. Each service should operate as separate user. It limits possible damage during attacks, httpd exploitation etc.

If you want to learn more about creating user and groups I recommend reading this tutorial. Here I'll just simply create group and user without shell.

sudo groupadd www
sudo useradd httpd -g www --no-create-home --shell /sbin/nologin

You can change the names as you wish. I like to use www group instead of httpd group for example. I usually add there other services as well, like nginx or php-fpm.

Adjust main config file httpd.conf

httpd.conf is main Apache httpd configuration file. You should start by editing this file with editor of your choice. I like to use vi, but you can use nano etc...:

sudo vi /usr/local/apache2/conf/httpd.conf

I usually don't remove any of the configuration directives from that file. If something needs to be deleted I comment it out with # at the beginning of new line. It's easier to revert changes.

Now I scroll through the file from top to bottom looking for particular lines and check if it has correct value. If I don't find given line I just add it usually at the bottom of the file.

Here are the important values:

# Make sure that ServerRoot is set to the same value as --prefix during ./configure
ServerRoot /usr/local/apache2

# Set ServerName to prevent warning on Apache start
ServerName localhost

# Default port set to 80 - HTTP protocol
Listen 80

# Set user and group
User httpd
Group www

# Configure entry file for your application. If you plan to use PHP make sure that it's as first possible file
DirectoryIndex index.php index.html

# Hide Apache version from header and from error files
ServerTokens prod
ServerSignature off

# Disable ETag to prevent disposing sensitive values like iNode
FileETag none

Save the file. After each change to configuration file you must restart HTTPD in order to apply changes. In order to do so execute following command:

sudo systemctl restart httpd

After restarting, make sure that Apache is working fine!

Configure loaded modules

During compilation we set --enable-so modules which means that we can disable and enable modules in configuration files.

List of modules is pretty long. Some of the are disabled (they have # at the beginning of the line). You should know, that the more modules are enabled the  "slower" Apache httpd is. I'm not saying that it's super slow, but you can google for some benchmarks showing different configurations.

Here is what I like to do. First of all, I comment out all modules = everything is disabled. I enable only that modules that I really use + the modules that are required for proper functioning of Apache httpd. It has few benefits - Apache is faster, eats less resources (CPU and RAM) and it's more resistant for given attacks. Usually when new security issue pops out, it's rather connected to one of the modules, than whole httpd. So you can have more chances to avoid potential security risk with having some stuff disabled.

So edit main configuration file one more time:

sudo vi /usr/local/apache2/conf/httpd.conf

And set modules that you use. Here is my go-to list of enabled modules:

# These modules must be enabled if you want Apache to start
LoadModule authz_core_module modules/mod_authz_core.so
LoadModule mime_module modules/mod_mime.so
LoadModule log_config_module modules/mod_log_config.so
LoadModule unixd_module modules/mod_unixd.so
LoadModule dir_module modules/mod_dir.so

# If you are using PHP with PHP-FPM which I highly suggest enable proxy modules 
LoadModule proxy_module modules/mod_proxy.so 
LoadModule proxy_fcgi_module modules/mod_proxy_fcgi.so 

# Enable pretty links and mod_rewrite that is highly used in all frameworks and CMSes 
LoadModule rewrite_module modules/mod_rewrite.so 

# Useful for WordPress sites - enables Require for setting up access to given resources. 
LoadModule access_compat_module modules/mod_access_compat.so 

# One more useful thing for WordPress and Let's Encrypt - enables Alias. If you are using composer and wpackagist it's a must, otherwise if you don't plan to use aliases, leave that disabled 
LoadModule alias_module modules/mod_alias.so 

# Enable gzip extension for compressing static files 
LoadModule deflate_module modules/mod_deflate.so 
LoadModule filter_module modules/mod_filter.so 

# Enable expires header for caching assets on browser side 
LoadModule expires_module modules/mod_expires.so 

# Enable SSL 
LoadModule http2_module modules/mod_http2.so 
LoadModule socache_shmcb_module modules/mod_socache_shmcb.so 
LoadModule ssl_module modules/mod_ssl.so 

# Enable status module for monitoring Apache. If you don't plan to use that, leave that commented out 
LoadModule authz_host_module modules/mod_authz_host.so 
LoadModule status_module modules/mod_status.so

Save the changes and restart Apache. Make sure that it works after changes!

Configure MPM

Configure Multi-Processing modules contains two parts. First I override default Apache configuration for event mode that I picked during compilation. In that case you will know what is actually configured. So let's start with that first.

At the bottom of httpd.conf you need to uncomment the line:

Include conf/extra/httpd-mpm.conf

It will enable advanced Apache httpd MPM configuration and it will override the defaults.

Now edit enabled file:

sudo vi /usr/local/apache2/conf/extra/httpd-mpm.conf

This file contains configuration of all MPM There are configuration for each MPM module, so make sure that you are setting correct values. I enabled event mode, so this is the section I care about:

<IfModule mpm_event_module>
    StartServers 5
    MinSpareThreads 75
    MaxSpareThreads 250
    ThreadsPerChild 25
    MaxRequestWorkers 400
    MaxConnectionsPerChild 0
</IfModule>

Save the file and restart Apache. This was the easiest part of MPM configuration.

The real part of MPM configuration is later on when you start running your production website. When you feel that your website/application start running low on higher traffic you can start playing around with that values.

Important thing that you need to know - there is no one config to rule them all. What I want to say is that one config that works fine on one server might not work as fine as on second server. There are dozens of factors like application specific, CPU and RAM, traffic etc. I encourage you to play around with these values to find the optimal settings for your server.

Here is great post that explains in depth MPM configuration. I have two advice's about performance tuning.

First one, that you probably don't even need to change anything if you have regular website without huge amount of traffic. It's just good to know what settings are applied by mpm configuration. When httpd-mpm.conf file was commented out, you didn't have any idea what the settings are. Once it's enabled, you at least know what is configured.

Second advice is - do it slowly. Performance tuning is lengthy process, and as I said, it highly depends on various factors. I like to change one settings, like StartServers for instance and I wait day or two and monitor response times, CPU and RAM usage etc. Sometimes even if you increase something you won't see a difference in response time, but you will get higher CPU usage. Then you just can rollback the changes. If you modify 3 or 4 values at one time, it's hard to say which comes with best (or any) result.

Setup GZIP compression

Using GZIP compression has serious impact on performance and your website loading time. Let's turn it on in few simple steps. In order to use it make sure that mod_deflate and mod_filter are enabled.

First create new file that will contain GZIP settings for Apache:

sudo vi /usr/local/apache2/conf/extra/httpd-deflate.conf

Paste there following content and save the file:

<IfModule mod_deflate.c>
    <IfModule mod_filter.c>
        AddOutputFilterByType DEFLATE application/ecmascript
        AddOutputFilterByType DEFLATE application/javascript
        AddOutputFilterByType DEFLATE application/rss+xml
        AddOutputFilterByType DEFLATE application/xml
        AddOutputFilterByType DEFLATE application/x-javascript
        AddOutputFilterByType DEFLATE text/css
        AddOutputFilterByType DEFLATE text/html
        AddOutputFilterByType DEFLATE text/plain
        AddOutputFilterByType DEFLATE text/xml
    </IfModule>
</IfModule>

It will add GZIP compression to most popular file types such as HTML, CSS, JS etc. If you need anything else that requires compressing, simply add more MIME types.

Once you save the file you need to include it to main Apache configuration. So open httpd.conf one more time and on the bottom of the page there is section where you include different things, such as MPM. In this section add following line:

Include conf/extra/httpd-deflate.conf

Save changes and restart HTTPD. Gzip compression will be enabled!

Setup assets caching with expires headers

Another important thing that has serious impact on performance is caching. If you want to use cache headers make sure that you enabled mod_expires for that purpose. Rest of procedure is similar to enabling GZIP compression.

Start with creating file that will contain configuration for caching:

sudo vi /usr/local/apache2/conf/extra/httpd-expires.conf

Paste following configuration and add/remove mime types or adjust time of expiration it if you need:

<IfModule mod_expires.c>
    # Enable expirations
    ExpiresActive On

    # Expirations for given mime type
    ExpiresByType image/gif "access plus 1 month"
    ExpiresByType image/ico "access plus 1 month"
    ExpiresByType image/jpg "access plus 1 month"
    ExpiresByType image/jpeg "access plus 1 month"
    ExpiresByType image/png "access plus 1 month"
    ExpiresByType text/css "access plus 1 month"
    ExpiresByType text/javascript "access plus 1 month"
</IfModule>

Same as before, you need to add entry to httpd.conf file and after that, restart the server:

Include conf/extra/httpd-expires.conf

Enable SSL configuration

If you plan to use HTTPS which I highly recommend you need to include SSL configuration. By default it is commented out from main configuration file. Before including this file, let's edit it first:

sudo vi /usr/local/apache2/conf/extra/httpd-ssl.conf

File contains secure configuration and sensible defaults that you don't need to change. It will provide secure connection that targets most browsers. However this file also contains default VirtualHost for HTTPS which I always remove. The reason is - I like to know what entry points to the server are available. Since this page doesn't display anything useful we can simply remove it.

So scroll to the point where VirtualHost section starts and remove whole section. It's pretty long so make sure that you remove it completely including <VirutalHost...></VirtualHost> lines :

## Remove this part:
##
## SSL Virtual Host Context
##
<VirtualHost _default_:443>
#...
</VirtualHost>

Save changes to the file. Now it's time to include this file into httpd.conf file. Edit it and scroll to the bottom. SSL is commented out by default so all you need to do is to uncomment it like so:

# Secure (SSL/TLS) connections
Include conf/extra/httpd-ssl.conf

Save the changes and restart HTTPD service. This is only initial part of adding HTTPS to your website. To be able to successfully connect to your server with SSL there are few more steps.

Add Let's Encrypt integration

If you are not using Let's Encrypt free SSL certificates you can skip that part.

When you install certbot for obtaining Let's Encrypt certificates you must somehow validate the domain. I use webroot as authenticator. It means that inside your website directory certbot will create .well-known directory and will try to put some files there. It works in most cases unless you start playing around with root path to your website, like placing it in subdirectory. Sometimes there is also permission issue.

There is neat trick to solve all this issues. When you create new certificate, point webroot to /var/lib/letsencrypt
directory instead to your domain directory like /var/www/blacksaildivision.com/htdocs

Next, create new file for Apache configuration:

sudo vi /usr/local/apache2/conf/extra/httpd-acme.conf

and paste there following contents:

Alias /.well-known/acme-challenge/ "/var/lib/letsencrypt/.well-known/acme-challenge/"
<Directory "/var/lib/letsencrypt/">
    AllowOverride None
    Options MultiViews Indexes SymLinksIfOwnerMatch IncludesNoExec
    Require method GET POST OPTIONS
</Directory>

Please note that mod_alias must be enabled. What it will do on each request to .well-known/acme-challenge that certbot uses it will point it to /var/lib/letsencrypt directory. It solves the problem for all domains globally.

Last thing is to enable this feature in main configuration of Apache. Edit the file and add following section near the bottom of the file where all other Includes are placed:

# Let's Encrypt integration
Include conf/extra/httpd-acme.conf

Restart the Apache to apply new changes.

Enable extended status

If you want to get more insights about Apache or you plan to use monitoring tools like Datadog it's wise to enable extended version of status. It will give you basic info about workers and httpd itself. In order to enable, edit main configuration file and somewhere and following code (I like to place it just before all Include sections):

# Enable extended status
<IfModule status_module>
    ExtendedStatus On
</IfModule>

Remember that you need to enable mod_status and mod_authz_host in order to get status working. Mod_authz_host will be required later on when I show you how to configure localhost. Mod_status response should be restricted to avoid exposing sensible information.

In order to apply changes, please restart httpd.

Setting up root directory

Let's start with final part - setting up Virtual Hosts. There are couple of approaches. It depends how many websites you plan to have on single server, does all your apps have domains or are they just microservices etc. I'll show you two of my approaches. Regardless of any approach you need a directory where you will store you app/website files and logs.

On CentOS there should be directory /var/www which is recommended place to store website files. If you don't have such directory create one first:

sudo mkdir /var/www

Make sure that this directory has valid permission and valid owner which should be root:

sudo chmod 755 /var/www
sudo chown root:root /var/www

Setting up website directories for files and logs

Once root directory is ready it's time to setup directories for websites. Let's assume that I want to create folder structure for blacksaildivision.com website. So first I create directory inside /var/www and name it as domain:

sudo mkdir /var/www/blacksaildivision.com

Inside this directory I need two separate directories. One for my website files like index.html etc and one for logs. There are many ways of keeping logs on the server. Some people uses native directories like /usr/local/apache2/logs or /var/log but I prefer to keep everything in one place. So all nginx/httpd/php logs will go to logs directory inside my domain directory.

sudo mkdir /var/www/blacksaildivision.com/htdocs
sudo mkdir /var/www/blacksaildivision.com/logs

Once directories are created it's time to set valid owner and permissions. Let's start with ownership:

sudo chown root:root /var/www/blacksaildivision.com
sudo chown developer:www /var/www/blacksaildivision.com/htdocs
sudo chown developer:www /var/www/blacksaildivision.com/logs

So main directory for domain should be owned by root. However two directories inside should be owned by custom user. In that case main user for the server is called developer. He has sudo powers and has access to website files and can perform commands like git pull to download latest code.

Why not httpd user that we created for Apache? The reasons are permissions. When you download files with git pull as developer user, but directory is owned by httpd user it most often leads to unexpected errors and mismatch in owners.

If you need ( and you should ) create separate account for developer/server owner read this tutorial.

Group is set to www which I usually use as generic group for all website related applications such as httpd, php-fpm etc. Please note that developer user also belongs to that group.

Now it's time to set directory permissions:

sudo chmod 755 /var/www/blacksaildivision.com
sudo chmod 2775 /var/www/blacksaildivision.com/htdocs
sudo chmod 2775 /var/www/blacksaildivision.com/logs

2 at the beginning means that every new subdirectory created will have the same ownership as parent directory (developer:www)

So what if I need to store more than one domain? Simply create new directory inside /var/www like mywebsite.com and repeat the steps with htdocs, logs directory, set ownership and permissions.

Do I need to create domain directory? It depends. I usually create such directory even if I host single website only. There are some cases like microservices that does not have a domain and are run behind load-balancer. In that case you can skip creating domain directory and use /var/www/htdocs and /var/www/logs. Just remember to change paths accordingly in next steps.

Create basic Virtual Host file for HTTP website

Directories were created so now it's the most important part - setting up VirtualHost. Again, there are many different approaches how to store the virtual hosts etc. I'll show you mine. Let's create basic Virtual Host file that will only load static files from given domain. As example I use blacksaildivision.com domain.

First I create a file that will contain configuration for HTTP version of the website. I always add domain to the filename so it's easier to find given configuration if you have multiple domains:

sudo vi /usr/local/apache2/conf/extra/httpd-vhost-blacksaildivision.com.conf

Paste Virtual Host config there and adjust your domain and directory structure:

<VirtualHost *:80>

ServerName blacksaildivision.com

# Directory settings
DocumentRoot /var/www/blacksaildivision.com/htdocs
<Directory /var/www/blacksaildivision.com/htdocs>
    AllowOverride All
    Require all granted
    Options +FollowSymLinks -Indexes -Includes
</Directory>

# Logging
ErrorLog "/var/www/blacksaildivision.com/logs/httpd-error.log"
CustomLog "/var/www/blacksaildivision.com/logs/httpd-access.log" common

</VirtualHost>

File starts and ends with block defining VirtualHost. Inside this blocks you need to place your domain configuration. First one is ServerName which is your domain.

Next part is directory settings. You need to point httpd to read files from correct directory which is defined by DocumentRoot. Just below that line there are settings for root directory. AllowOverride will allow .htaccess files to override all default instructions in this block. Require all granted adds general access to this directory. It's a must if you want to read files from given location. Lastly I allow following symlinks which are often used by various frameworks and CMSes. I disable Indexes and Includes because of security reasons.

Last parts are logs. I define two logs. One is error log which will contain all invalid requests details etc. CustomLog is simply access log that will contain all requests.

So in general if you want to add different or another domain, simply create file with configuration like above and update ServerName, DocumentRoot and paths to logs.

Last part is to include this configuration file in main httpd configuration file:

sudo vi /usr/local/apache2/conf/httpd.conf

and at the very bottom of the file Include configuration:

Include conf/extra/httpd-vhost-blacksaildivision.conf

Save changes and restart Apache. Now try to visit your website! In case of any issues like 4xx/5xx errors simply check error log and see what the problem is. Usually there are typos or incorrect configuration of your domain causing the issues.

Adding PHP support

Most of you will probably use Apache with PHP. I use PHP in PHP-FPM mode. In my opinion it gives greater control over mod_php when it comes to performance tuning, configuration etc. It also has one more benefit - if you will ever need to switch from Apache to Nginx it's much easier as preferred way of using nginx + php is PHP-FPM.

If you want to read more about how to install and configure PHP - please follow this tutorial.

Edit the VirtualHost file where you want to add PHP support and start adding there following lines ( I prefer to add it just before logs section):

# PHP-FPM settings
<Proxy "fcgi://127.0.0.1:9001" timeout=30>
</Proxy>

It will define Proxy for PHP-FPM daemon. In my case FPM process is running on localhost (127.0.0.1) and on port 9001. Optional parameter is timeout. It's not required but sometimes it's helpful. Imagine situation where one of your scripts has bug like infinite loop or it's execution time takes way more than 30s. Apache will have to wait until proxy will return something. Now imagine that someone called this script 10000 times. It will kill your httpd daemon. It's safe to add timeout as additional guard. You can experiment with timeoutvalue for your application.

Proxy is defined, not it's time to use it. Just after section add  following lines:

<FilesMatch "\.php$">
    <If "-f %{SCRIPT_FILENAME}">
        SetHandler "proxy:fcgi://127.0.0.1:9001"
    </If>
</FilesMatch>

First I use FilesMatch to detect if requested file has .php extension. If it has I'm checking if the file actually exists. If you won't check it, request will be passed to FPM process anyway which can result in unexpected results and unnecessary lines in log files.

If everything is OK I'm setting Handler to defined proxy and PHP will handle rest of the request.

You can save the file and restart Apache and check if everything works fine:)

If you are using status or ping from PHP-FPM for performance debugging or tools such as Datadog and you don't want to expose additional port in your firewall, you can achieve it with HTTPD as well. Simply add following lines before <FilesMatch> section:

<LocationMatch "/(status|ping)">
    RewriteEngine Off
    Require all denied
    Require ip 127.0.0.1 60.120.72.81
    SetHandler "proxy:fcgi://127.0.0.1:9001"
</LocationMatch>

I'm checking if requested path matches /status or /ping. If match is found I'm turning of RewriteEngine in case of redirect in .htaccess etc. Than I'm turning off access for everyone but I allow access from list of IPs. I allow access from localhost, but I also allow access from my server IP address. So for instance If my server has IP 60.120.72.81 I add it as well.

Last thing is to set handler to use to proxy for PHP-FPM daemon.

Enable HTTPS inside VirtualHost

It's time to enable HTTPS for our website! I assume that you already have SSL certificates purchased and downloaded or generated via Let's Encrypt.

Edit the file first:

sudo vi /usr/local/apache2/conf/extra/httpd-vhost-blacksaildivision.com.conf

Just for now you can leave the regular HTTP version and add following contents below VirtualHost section. In such way you will have both HTTP and HTTPS enabled for single website. Later on I'll show you how to redirect HTTP to HTTPS. So just paste the following contents at the end of edited file:

<IfModule mod_ssl.c>
<VirtualHost *:443>

ServerName blacksaildivision.com
ServerAlias www.blacksaildivision.com

# Redirect www to non-www
RedirectMatch permanent "^www\.(.*)$" "https://blacksaildivision.com/"

# Directory settings
DocumentRoot /var/www/blacksaildivision.com/htdocs
<Directory /var/www/blacksaildivision.com/htdocs>
    AllowOverride All
    Require all granted
    Options +FollowSymLinks -Indexes -Includes
</Directory>

# PHP-FPM settings
<Proxy "fcgi://127.0.0.1:9001" timeout=30>
</Proxy>

<LocationMatch "/(status|ping)">
    RewriteEngine Off
    Require all denied
    Require ip 127.0.0.1
    SetHandler "proxy:fcgi://127.0.0.1:9001"
</LocationMatch>

<FilesMatch "\.php$">
    <If "-f %{SCRIPT_FILENAME}">
        SetHandler "proxy:fcgi://127.0.0.1:9001"
    </If>
</FilesMatch>

# Logging
ErrorLog "/var/www/blacksaildivision.com/logs/httpd-error.log"
CustomLog "/var/www/blacksaildivision.com/logs/httpd-access.log" common

# SSL configuration
SSLEngine on
SSLCertificateFile "/etc/letsencrypt/live/blacksaildivision.com/cert.pem"
SSLCertificateKeyFile "/etc/letsencrypt/live/blacksaildivision.com/privkey.pem"
SSLCertificateChainFile "/etc/letsencrypt/live/blacksaildivision.com/chain.pem"

</VirtualHost>
</IfModule>

Everything is wrapped with IfModule section. I usually check if mod_ssl is enabled. In case someone by accident would remove that module from list of enabled modules, section with VirutalHost simply won't be loaded and will not throw an error.

This is new VirtualHost so it must be wrapped in  <VirtualHost *:443>  section. Please note that port is now set to 443 instead of 80. It's default port for HTTPS. You might see some similarities between HTTP and HTTPS version. Most of things are just copied from HTTP version, so I won't be covering this parts here. Instead I will focus on new things.

You already know ServerName directive. But below there is another one - ServerAlias. You can basically access your website in two ways - with www. (https://www.blacksaildivision.com) and without www. (https://blacksaildivision.com). I prefer to use version without www. Less typing and URL looks more friendly IMHO:) So I want to redirect all www traffic to non-www. We don't have such redirect in HTTP version, but we will add it later on when I'll show you how to redirect from HTTP to HTTPS.

ServerAlias create an alias to given VirtualHost identified by ServerName. So no matter which URL you will use - the one from ServerName or from ServerAlias(es), Apache will use configuration from given VirtualHost. For instance we can add another alias like - ServerAlias blacksaildivisioncopy.com and it can point to same configuration. So your website can be accessible from two different URLs. It's very rare case but this is just an example.

What is important that your website should have redirect, so there always will be only single copy of your website. For instance if you have both www and non-www versions available without any redirect, your content will be indexed twice and will be seen as two different websites. It's bad for the SEO and your ranking in search engines. In order to redirect from non-www to www you should use RedirectMatch directive. This is permanent redirect, so it won't (and should not) change in the future. Also please make sure that you have your SSL cert valid for www. domain as well. If you won't have it, you will get an error when visiting version with www. prefix.

Last new part is SSL configuration at the bottom of the file. First we need to tell Apache that we will use SSL with SSLEngine on. Next are the paths to Certificate, Key and Chain. Chain is sometimes not mandatory in some SSL providers. So if you don't have Chain file, skip this directive. I'm using Let's Encrypt so paths to SSL are set to /etc/letsencrypt/live/blacksaildivision.com/ If you just have certificates bought from SSL provider, simply create new directory inside your domain like so:

sudo mkdir /var/www/blacksaildivision.com/ssl

and place all cert files inside this directory. Remember to update paths inside your VirtualHosts!

Once it's done simply save the changes and restart Apache. Try to visit your website with HTTPS and HTTP. Both version should work just fine. You can also check your server with https://www.ssllabs.com/ssltest/ If you configured everything correctly, you should get A score. You can aim for A+, but in my opinion there is too much disadvantages like limited access to your website from older devices etc.

Redirect HTTP to HTTPS

As we want to serve traffic over HTTPS only, we need to redirect all HTTP requests to HTTPS. Edit the file with your domain configuration and replace <VirtualHost *:80> to following code:

<VirtualHost *:80>

ServerName blacksaildivision.com
ServerAlias www.blacksaildivision.com

# Redirect everything permanently to https://blacksaildivision.com/
Redirect permanent "/" "https://blacksaildivision.com/"

# Turn off logging
ErrorLog /dev/null
CustomLog /dev/null common

</VirtualHost>

It's much shorter than previous version. I added ServerAlias with www. prefix. Using Redirect directive I'm redirecting all traffic to https:// version. And last thing is turning off logging. It's not necessary to log redirects from HTTP to HTTPS. If you have logging in HTTPS VirutalHost, logs will be there anyway. There's no need to duplicate code.

Save changes, restart Apache and check if redirect is working fine:)

Run Apache httpd on system start

Last thing is to add Apache httpd daemon to start with system boot. So after server start/restart  httpd will run automatically:

sudo systemctl enable httpd

So that's it. You have fully working Apache httpd in latest version installed on your system 🙂 This process might take some time, but you will have full control over httpd.

As always you can use our LampOnSteroids project (based on Ansible) to speed and automate everything up!

How to install EPEL on CentOS

How to install EPEL repository on CentOS

We need to install EPEL repository for our CentOS. This will be first step to setup our working Web Server. EPEL is additional repository for yum, with tons of useful packages which we won't find in default base repository. We will use it in the future so let's start using it now;)

In this post we will also setup local vagrant box. We will be using it for future operations like installing Apache or Varnish.

[sc:lamp_series]

Vagrant setup

The best idea without messing everything out is to use Vagrant.

If You don't have Vagrant installed please download and install it first. After installation You should be ready to use Vagrant. Create directory in place where You want to keep Your vagrant file.

mkdir vms
cd vms

Next thing what we need to do is to download CentOS box. We need the box to replace default one that comes with Vagrant. We will use chef/centos-6.6 from Vagrant boxes directory. You can search more boxes here if You want, for instance You can use 6.5 instead of 6.6. Decision is Yours:)

vagrant box add chef/centos-6.6

Vagrant will download the image of OS You selected. It might take a while, depending on Your network performance.

Next we need to init Vagrant instance.

vagrant init chef/centos-6.6

It will create Vagrantfile in Your directory. You can check out this file in any editor. This file has basic Vagrant configuration, you can adjust the options like forwarded ports,  synced folders etc. For now, let's leave this configuration file and let's bring our machine to life!

vagrant up

It will init the machine. After that You can login via SSH and start playing around:)

vagrant ssh

tl;dr

mkdir vms
cd vms
vagrant init chef/centos-6.6
vagrant up
vagrant ssh

 

How to install EPEL

Installing EPEL on CentOS is really easy.  First of all You need to get the URL for EPEL repo. If You are just following the instructions in this tutorial You can execute the command. But for instance if You have 32bit version instead of 64bit You need to replace x86_64 to i386

yum install http://download.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm

Depending on Your vagrant / server setup You might need to execute this command as root or at least as sudo user. If You are just following the steps in this tutorial You are probably Vagrant user. If You execute this command You will see permission denied error. Start this command with sudo (sudo yum install http://....) and everything should be OK:)

Next thing that we need to do is to import GPG key for this repo:

rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-6

And basically that's it! You are now able to use EPEL packages with yum!

To validate the installation You can execute yum repolist command  and You should see epel Extra Packages for Enterprise Linux 6 - x86_64

yum repolist

It means that everything is OK. Now, if You want to install any packages with the EPEL repository You need to use

yum --enablerepo=epel install PACKAGE_NAME

tl;dr

yum install http://download.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rp -y
rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-6

 

Using Ansible to install EPEL

If You are using Ansible for server provisioning as I do, You can find ansible-webserver project on GitHub

https://github.com/astaz3l/ansible-webserver

It has EPEL role, that will help You install EPEL via Ansible

Our services: