Author Archives: astaz3l

How to clean up WordPress head tag?

Default <head> tag on clean installation (before)

For the purpose of this article I installed a clean WordPress 5.2 with default theme Twenty Nineteen 1.4.

Here is how the <head> tag looks like for fresh version of WordPress:

<head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1" />
    <link rel="profile" href="https://gmpg.org/xfn/11" />
    <title>Clean wp-head</title>
    <link rel='dns-prefetch' href='//s.w.org' />
    <link rel="alternate" type="application/rss+xml" title="Clean wp-head » Feed" href="https://clean-wp-head.dev/feed" />
    <link rel="alternate" type="application/rss+xml" title="Clean wp-head » Comments Feed" href="https://clean-wp-head.dev/comments/feed" />
    <script type="text/javascript">black
        window._wpemojiSettings = {"baseUrl":"https:\/\/s.w.org\/images\/core\/emoji\/12.0.0-1\/72x72\/","ext":".png","svgUrl":"https:\/\/s.w.org\/images\/core\/emoji\/12.0.0-1\/svg\/","svgExt":".svg","source":{"concatemoji":"https:\/\/clean-wp-head.dev\/wp-includes\/js\/wp-emoji-release.min.js?ver=5.2"}};
        !function(a,b,c){function d(a,b){var c=String.fromCharCode;l.clearRect(0,0,k.width,k.height),l.fillText(c.apply(this,a),0,0);var d=k.toDataURL();l.clearRect(0,0,k.width,k.height),l.fillText(c.apply(this,b),0,0);var e=k.toDataURL();return d===e}function e(a){var b;if(!l||!l.fillText)return!1;switch(l.textBaseline="top",l.font="600 32px Arial",a){case"flag":return!(b=d([55356,56826,55356,56819],[55356,56826,8203,55356,56819]))&&(b=d([55356,57332,56128,56423,56128,56418,56128,56421,56128,56430,56128,56423,56128,56447],[55356,57332,8203,56128,56423,8203,56128,56418,8203,56128,56421,8203,56128,56430,8203,56128,56423,8203,56128,56447]),!b);case"emoji":return b=d([55357,56424,55356,57342,8205,55358,56605,8205,55357,56424,55356,57340],[55357,56424,55356,57342,8203,55358,56605,8203,55357,56424,55356,57340]),!b}return!1}function f(a){var c=b.createElement("script");c.src=a,c.defer=c.type="text/javascript",b.getElementsByTagName("head")[0].appendChild(c)}var g,h,i,j,k=b.createElement("canvas"),l=k.getContext&&k.getContext("2d");for(j=Array("flag","emoji"),c.supports={everything:!0,everythingExceptFlag:!0},i=0;i<j.length;i++)c.supports[j[i]]=e(j[i]),c.supports.everything=c.supports.everything&&c.supports[j[i]],"flag"!==j[i]&&(c.supports.everythingExceptFlag=c.supports.everythingExceptFlag&&c.supports[j[i]]);c.supports.everythingExceptFlag=c.supports.everythingExceptFlag&&!c.supports.flag,c.DOMReady=!1,c.readyCallback=function(){c.DOMReady=!0},c.supports.everything||(h=function(){c.readyCallback()},b.addEventListener?(b.addEventListener("DOMContentLoaded",h,!1),a.addEventListener("load",h,!1)):(a.attachEvent("onload",h),b.attachEvent("onreadystatechange",function(){"complete"===b.readyState&&c.readyCallback()})),g=c.source||{},g.concatemoji?f(g.concatemoji):g.wpemoji&&g.twemoji&&(f(g.twemoji),f(g.wpemoji)))}(window,document,window._wpemojiSettings);
    </script>
    <style type="text/css">
        img.wp-smiley,
        img.emoji {
            display: inline !important;
            border: none !important;
            box-shadow: none !important;
            height: 1em !important;
            width: 1em !important;
            margin: 0 .07em !important;
            vertical-align: -0.1em !important;
            background: none !important;
            padding: 0 !important;
        }
    </style>
    <link rel='stylesheet' id='wp-block-library-css'  href='https://clean-wp-head.dev/wp-includes/css/dist/block-library/style.min.css?ver=5.2' type='text/css' media='all' />
    <link rel='stylesheet' id='wp-block-library-theme-css'  href='https://clean-wp-head.dev/wp-includes/css/dist/block-library/theme.min.css?ver=5.2' type='text/css' media='all' />
    <link rel='stylesheet' id='twentynineteen-style-css'  href='https://clean-wp-head.dev/wp-content/themes/twentynineteen/style.css?ver=1.4' type='text/css' media='all' />
    <link rel='stylesheet' id='twentynineteen-print-style-css'  href='https://clean-wp-head.dev/wp-content/themes/twentynineteen/print.css?ver=1.4' type='text/css' media='print' />
    <link rel='https://api.w.org/' href='https://clean-wp-head.dev/wp-json/' />
    <link rel="EditURI" type="application/rsd+xml" title="RSD" href="https://clean-wp-head.dev/xmlrpc.php?rsd" />
    <link rel="wlwmanifest" type="application/wlwmanifest+xml" href="https://clean-wp-head.dev/wp-includes/wlwmanifest.xml" />
    <meta name="generator" content="WordPress 5.2" />
</head>

Lot of stuff, right? I usually remove like 80% of this code! Things that I leave unchanged are:

meta tags for setting charset and viewport
title tag
links to theme stylesheets

Remove or set up DNS prefetch links

By default WordPress is prefetching s.w.org domain. This is only used for emojis script in WordPress. Since emojis are working fine in most browsers I remove the DNS prefetch attribute or I add the things that I really use.

There is a way to remove all the resource hints from the <head>:

remove_action( 'wp_head', 'wp_resource_hints', 2 );

However I prefer slightly different and safer approach:

/**
 * Alter dns-prefetch links in <head>
 */
add_filter('wp_resource_hints', function (array $urls, string $relation): array {
    // If the relation is different than dns-prefetch, leave the URLs intact
    if ($relation !== 'dns-prefetch') {
        return $urls;
    }

    // Remove s.w.org entry
    $urls = array_filter($urls, function (string $url): bool {
        return strpos($url, 's.w.org') === false;
    });

    // List of domains to prefetch:
    $dnsPrefetchUrls = [
        'fonts.googleapis.com', // Google fonts,
        'any.other.website.url.you.need...'
    ];
    return array_merge($urls, $dnsPrefetchUrls);
}, 10, 2);

The first method (removing everything) might interfere with some plugins that could actually add something helpful. So I leave everything as is, except of removing the s.w.org entry and adding the things I want:)

Remove XML feeds

There are two entries to for the XML feed:

<link rel="alternate" type="application/rss+xml" title="Clean wp-head » Feed" href="https://clean-wp-head.dev/feed" />
<link rel="alternate" type="application/rss+xml" title="Clean wp-head » Comments Feed" href="https://clean-wp-head.dev/comments/feed" />

In most cases I don't want to have it in my<head> tag. I also like to have it disabled entirely. Even if you remove the entries from <head> the URLs in href attribute will still be accessible.

First of all, let's disable the feeds by redirecting the URLs to homepage:

/**
 * Disable RSS feeds by redirecting their URLs to homepage
 */
foreach (['do_feed_rss2', 'do_feed_rss2_comments'] as $feedAction) {
    add_action($feedAction, function (): void {
        // Redirect permanently to homepage
        wp_redirect(home_url(), 301);
        exit;
    }, 1);
}

Now it's safe to remove the tags from the <head> section:

/**
 * Remove the feed links from <head>
 */
remove_action('wp_head', 'feed_links', 2);

Remove emojis

DNS prefetch entry for emojis is already removed. Now it's time to get rid of the entire code for emojis that remains in head section:

<script type="text/javascript">black
        window._wpemojiSettings = {"baseUrl":"https:\/\/s.w.org\/images\/core\/emoji\/12.0.0-1\/72x72\/","ext":".png","svgUrl":"https:\/\/s.w.org\/images\/core\/emoji\/12.0.0-1\/svg\/","svgExt":".svg","source":{"concatemoji":"https:\/\/clean-wp-head.dev\/wp-includes\/js\/wp-emoji-release.min.js?ver=5.2"}};
        !function(a,b,c){function d(a,b){var c=String.fromCharCode;l.clearRect(0,0,k.width,k.height),l.fillText(c.apply(this,a),0,0);var d=k.toDataURL();l.clearRect(0,0,k.width,k.height),l.fillText(c.apply(this,b),0,0);var e=k.toDataURL();return d===e}function e(a){var b;if(!l||!l.fillText)return!1;switch(l.textBaseline="top",l.font="600 32px Arial",a){case"flag":return!(b=d([55356,56826,55356,56819],[55356,56826,8203,55356,56819]))&&(b=d([55356,57332,56128,56423,56128,56418,56128,56421,56128,56430,56128,56423,56128,56447],[55356,57332,8203,56128,56423,8203,56128,56418,8203,56128,56421,8203,56128,56430,8203,56128,56423,8203,56128,56447]),!b);case"emoji":return b=d([55357,56424,55356,57342,8205,55358,56605,8205,55357,56424,55356,57340],[55357,56424,55356,57342,8203,55358,56605,8203,55357,56424,55356,57340]),!b}return!1}function f(a){var c=b.createElement("script");c.src=a,c.defer=c.type="text/javascript",b.getElementsByTagName("head")[0].appendChild(c)}var g,h,i,j,k=b.createElement("canvas"),l=k.getContext&&k.getContext("2d");for(j=Array("flag","emoji"),c.supports={everything:!0,everythingExceptFlag:!0},i=0;i<j.length;i++)c.supports[j[i]]=e(j[i]),c.supports.everything=c.supports.everything&&c.supports[j[i]],"flag"!==j[i]&&(c.supports.everythingExceptFlag=c.supports.everythingExceptFlag&&c.supports[j[i]]);c.supports.everythingExceptFlag=c.supports.everythingExceptFlag&&!c.supports.flag,c.DOMReady=!1,c.readyCallback=function(){c.DOMReady=!0},c.supports.everything||(h=function(){c.readyCallback()},b.addEventListener?(b.addEventListener("DOMContentLoaded",h,!1),a.addEventListener("load",h,!1)):(a.attachEvent("onload",h),b.attachEvent("onreadystatechange",function(){"complete"===b.readyState&&c.readyCallback()})),g=c.source||{},g.concatemoji?f(g.concatemoji):g.wpemoji&&g.twemoji&&(f(g.twemoji),f(g.wpemoji)))}(window,document,window._wpemojiSettings);
    </script>
    <style type="text/css">
        img.wp-smiley,
        img.emoji {
            display: inline !important;
            border: none !important;
            box-shadow: none !important;
            height: 1em !important;
            width: 1em !important;
            margin: 0 .07em !important;
            vertical-align: -0.1em !important;
            background: none !important;
            padding: 0 !important;
        }
    </style>

There is both a JS code and some inline styles. Here is the code to remove both of them:

/**
 * Remove emoji script and styles from <head>
 */
remove_action('wp_head', 'print_emoji_detection_script', 7);
remove_action('wp_print_styles', 'print_emoji_styles');

First line removes the JS code, second one removes the inline CSS styles.

Disable WordPress REST API

If you don't plan to build SPA or make Ajax calls to WordPress REST API you can safely disable it. I only leave it enable for admin user. Some of the plugins uses the new REST API for their functionalities in wp-admin so it's good to have it working there:

/**
 * Disable REST-API for all users except of admin
 */
add_filter('rest_authentication_errors', function ($access) {
    if (!current_user_can('administrator')) {
        return new WP_Error('rest_cannot_access', 'Only authenticated users can access the REST API.', ['status' => rest_authorization_required_code()]);
    }
    return $access;
});

This code will disable the endpoint for all users that are not administrators of your page. If you try to visit it for instance as not logged in user, you will get 401 error.

In order to remove the entry from <head> section:

<link rel='https://api.w.org/' href='https://clean-wp-head.dev/wp-json/' />

use following code:

/**
 * Remove REST-AI link from <head>
 */
remove_action('wp_head', 'rest_output_link_wp_head');

Disable XML-RPC

New WordPress REST-API most probably will replace the old XML-RPC protocol. I like to keep it disabled by using following code:

/**
 * Disable XML-RPC
 */
add_filter('xmlrpc_enabled', function (): bool {
    return false;
});

With this code when you try to send request to xmlrpc.php file you will get following message: XML-RPC services are disabled on this site.

But still, in <head> tag there is an entry to the XML-RPC:

<link rel="EditURI" type="application/rsd+xml" title="RSD" href="https://clean-wp-head.dev/xmlrpc.php?rsd" />

Remove it with following code:

/**
 * Remove XML-RPC link from <head>
 */
remove_action('wp_head', 'rsd_link');

Remove Windows Live Writer manifest

If you don't use Windows Live Writer you can easily remove the manifest link from <head> section:

<link rel="wlwmanifest" type="application/wlwmanifest+xml" href="https://clean-wp-head.dev/wp-includes/wlwmanifest.xml" />

In order to remove it, use following code:

/**
 * Remove Windows Live Writer manifest from <head>
 */
remove_action('wp_head', 'wlwmanifest_link');

Remove info about generator

WordPress adds the information about the version of currently installed WP:

<meta name="generator" content="WordPress 5.2" />

You can easily remove it with this code:

/**
 * Remove info about WordPress version from <head>
 */
remove_action('wp_head', 'wp_generator');

Bonus 1 - remove Gutenberg default styles

If you don't use Gutenberg or you plan to override the default styles that comes with this editor, you can easily remove the styles from <head> tag:

<link rel='stylesheet' id='wp-block-library-css'  href='https://clean-wp-head.dev/wp-includes/css/dist/block-library/style.min.css?ver=5.2' type='text/css' media='all' />
<link rel='stylesheet' id='wp-block-library-theme-css'  href='https://clean-wp-head.dev/wp-includes/css/dist/block-library/theme.min.css?ver=5.2' type='text/css' media='all' />

In order to remove these links, use following code:

/**
 * Remove Gutenberg default styles
 */
add_action('wp_print_styles', function (): void {
    wp_dequeue_style('wp-block-library');
    wp_dequeue_style('wp-block-library-theme');
});

Bonus 2 - remove unnecessary attributes from link stylesheets

That might be crazy, but if id and type attributes in link styles are annoying, you can do something about it like I do:

/**
 * Remove unnecessary attributes from style tags
 */
add_filter('style_loader_tag', function (string $tag, string $handle): string {
    // Remove ID attribute
    $tag = str_replace("id='${handle}-css'", '', $tag);

    // Remove type attribute
    $tag = str_replace(" type='text/css'", '', $tag);

    // Change ' to " in attributes:
    $tag = str_replace('\'', '"', $tag);

    // Remove trailing slash
    $tag = str_replace(' />', '>', $tag);

    // Remove double spaces
    return str_replace('  ', '', $tag);
}, 10, 2);

Moreover I don't really like that the attributes are wrapped in ' instead of " so this code changes it too:)

Bonus 3 - Remove XFN meta data profile

Just right below meta tags there is a particular line:

<link rel="profile" href="https://gmpg.org/xfn/11" />

It is a link for XFN which is XHTML Friends Network. This is the way of describing relationships between users in machine-readable way. You can read more about how to use it on WordPress website here.

This tag is added directly in header.php file and it needs to be removed manually. There is no hook for removing it. Simply edit the file and remove this tag from <head> section.

<head> tag after the changes

The contents of <head> tag after some additional clean-ups in header.php file looks like this:

<head>
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
    <title>Clean wp-head</title>
    <link rel="stylesheet" href="https://clean-wp-head.dev/wp-content/themes/twentynineteen/style.css?ver=1.4" media="all" >
    <link rel="stylesheet" href="https://clean-wp-head.dev/wp-content/themes/twentynineteen/print.css?ver=1.4" media="print" >
</head>

Quite a difference, isn't it? From 35 lines just down to 7 ?

Can I get full code, please?

I putted everything together in single file on GitHub Gist.

You can include this code in your plugin or functions.php. You can also copy the entire thing (without the <?php tag of course!) into the functions.php. However the best place to keep it is inside of a plugin:)

How to install latest version of OpenSSL on CentOS?

20 Replies

Hi there, today I would like to show you how to install latest version of OpenSSL (1.1.1c) on CentOS 7

Do I need latest version of OpenSSL?

In general - you don't. Default version is doing great job and it's secure. I needed it for compiling Apache HTTP with HTTP/2 support back then and now I'm using new version every time it's released. If you need it for any other reason, this tutorial is for you:)

How to check current version of OpenSSL?

In order to check current version of installed package you need to execute following command:

openssl version

It will print out version of installed package like OpenSSL 1.0.2k-fips 26 Jan 2017

How to install latest version of OpenSSL?

I compile OpenSSL from source code. In order to compile it successfully you need to install some tools that will help you compile it:

sudo yum install libtool perl-core zlib-devel -y

It will install compiler and few other libraries that are required to compile OpenSSL.

Next download latest version of OpenSSL source code. I like to use releases page on GitHub. I choose the version without FIPS simply because I don't need compatibility with it. And I think that it's a bit more secure to have OpenSSL without FIPS, as fixes are usually included much faster in regular version than in FIPS version. If you want to read more about it, use this link.

In order to download source code, use following command:

curl -O -L https://github.com/openssl/openssl/archive/OpenSSL_1_1_1c.tar.gz

Source code comes in compressed package. In order to decompress it use following command:

tar -zxvf OpenSSL_1_1_1c.tar.gz
cd openssl-OpenSSL_1_1_1c

Now it's time to configure and compile OpenSSL:

./config --prefix=/usr/local/openssl --openssldir=/usr/local/openssl shared zlib
make
make test

prefix and openssldir sets the output paths for OpenSSL. shared will force crating shared libraries and zlib means that compression will be performed by using zlib library

It is worth to run the tests to see if there are any unexpected errors. If there are any, you need to fix them before installing library.

In order to install library you need to execute:

sudo make install

Once the OpenSSL is installed, you can remove the sources and tar.gz package.

Add new version to PATH

After the installation you will probably want to check the version of OpenSSL but it will print out old version. Why? Because it's also installed on your server. I rarely override packages installed via yum. The reason is that when there is new version of OpenSSL and you will install it via yum, it will simply override compiled version, and you will have to recompile it again.

Instead of overriding files I personally like to create new profile entry and force the system to use compiled version of OpenSSL.

In order to do that, create following file:

sudo vi /etc/profile.d/openssl.sh

and paste there following content:

# /etc/profile.d/openssl.sh
pathmunge /usr/local/openssl/bin

Save the file and reload your shell, for instance log out and log in again. Then you can check the version of your OpenSSL client. Or maybe...

Link libraries

Or maybe you will get an error with loading shared libraries? In order to fix that problem we need to create an entry in ldconfig.

Create following file:

sudo vi /etc/ld.so.conf.d/openssl-1.1.1c.conf

And paste there following contents:

# /etc/ld.so/conf.d/openssl-1.1.1c.conf
/usr/local/openssl/lib

We simply told the dynamic linker to include new libraries. After creating the file you need to reload linker by using following command:

sudo ldconfig -v

And volia! Check the version of your OpenSSL now. It should print out OpenSSL 1.1.1c 28 May 2019

How to disable Transparent Huge Pages on CentOS

5 Replies

Hi there! I'd like to show you how to disable Transparent Huge Pages on CentOS 7. First question you might ask - why do you even want to disable Transparent Huge Pages (THP)? Well, when you are using your server for database purposes and you have tools like MySQL, Oracle or MongoDB, the recommendation is to disable THP for performance purposes.

How to check if THP is enabled or not?

In order to test THP you need to check two files:

cat /sys/kernel/mm/transparent_hugepage/enabled
cat /sys/kernel/mm/transparent_hugepage/defrag

Value in square brackets [] is currently active value. If you have [never] it means that transparent huge pages are disabled. If you have [always] - THP are enabled.

How to disable THP manually?

If you want to disable THP you can do it manually by typing these two commands:

echo 'never' > /sys/kernel/mm/transparent_hugepage/enabled
echo 'never' > /sys/kernel/mm/transparent_hugepage/defrag

You can cat the files one more time to see if value is changed to [never]

How to disable THP automatically?

Method above works fine until you restart the server. It will turn on on system restart. In order to disable them on system startup, you need to add Unit file with script that will disable THP.

Create following file:

sudo vi /etc/systemd/system/disable-thp.service

and paste there following content:

[Unit]
Description=Disable Transparent Huge Pages (THP)

[Service]
Type=simple
ExecStart=/bin/sh -c "echo 'never' > /sys/kernel/mm/transparent_hugepage/enabled && echo 'never' > /sys/kernel/mm/transparent_hugepage/defrag"

[Install]
WantedBy=multi-user.target

Save the file and reload SystemD daemon:

sudo systemctl daemon-reload

Than you can start the script and enable it on boot level:

sudo systemctl start disable-thp
sudo systemctl enable disable-thp

How to install GIT on CentOS

11 Replies

Today I would like to show you how I install GIT on CentOS servers. It might be super easy or slightly harder. It depends if you want to have the latest version (2.16.3) or you don't care:)

Easy way to install git on CentOS with yum

The easiest way is to use yum of course. Simply type

sudo yum install git -y

and it will install git on your server. That's it 😀 However if you will check your git version:

git --version

you will note that it will install 1.8.3.1 version (or higher). But wait! Isn't there git v2.0? Yes, there is. And I will show you how to install it.

How to compile git from source on CentOS?

I show you two approaches. The only difference is with using OpenSSL version. By default CentOS is shipped with older version. I like to have most recent version so I compile OpenSSL from source. It is not mandatory, since you can use default version. If you prefer to upgrade it follow this tutorial first and get back here once you will have latest version. If not just follow along.

There are 5 mandatory and 2 optional steps to have latest Git version on CentOS.

First you need to install some tools that are required to compile git. You will need some libraries and compiler. Install them by executing following command:

sudo yum install autoconf libcurl-devel expat-devel gcc gettext-devel kernel-headers openssl-devel perl-devel zlib-devel -y

Here is the fun part. Even if you don't have git installed, after executing command above you will actually have git on your server. It is because one of the packages has git as a dependency. But you shouldn't worry about that too much now.

If you didn't compile latest OpenSSL like I did you must install one more package:

sudo yum install openssl-devel -y

Step two is to download git source code. The easiest option is to download it from GitHub. Visit git release page - https://github.com/git/git/releases and pick desired version. I'm usually using latest, non-rc version. In my case it's 2.16.3. Right click on tar.gz icon and copy the link.

Now execute following command on your server (link you see is copied from GitHub):

curl -O -L https://github.com/git/git/archive/v2.16.3.tar.gz

It will download tarball with git source files to your server. -O flag will save the file instead of redirecting curl output to stdout. -L will follow redirects if any (usually there is one when using GitHub releases).

Step three is obviously unpacking downloaded source code:

tar -zxvf v2.16.3.tar.gz

Now is the fun part - compiling. Very important is to know, that you should never ever use root account for compiling. This might lead to serious security issues. Imagine that you will compile code that contains malicious parts as root. This code can modify your server and cut off your root access. Google about it, there were such cases in the past. So if you use root account, think about creating separate account to manage your server. Read this tutorial to learn how to setup additional account with root privileges. I'm not saying that you can't compile it as root, because obviously you can. But you just shouldn't do that.

Execute following command as non-root account:

cd git-v2.16.3
make clean
make configure

Depending on which OpenSSL you are using...

If you compiled latest OpenSSL use following command:

./configure CFLAGS='-I/usr/local/openssl/include' LDFLAGS='-L/usr/local/openssl/lib' --prefix=/usr/local/git --with-openssl=/usr/local/openssl

If you are using regular version:

./configure --prefix=/usr/local/git

Note parameter --prefix in configure command above. It will prevent overriding Git installed via yum. It will also help with overriding compiled version with packages installed from yum. Once package is configure run:

make

For installing you need to have root account or be in sudoers group. So last step is installation:

sudo make install

Now you have latest version of git in your system!

Optionally you can remove downloaded package and uncompressed directory. To cleanup execute:

cd ..
rm -rf git-v2.16.3 v2.16.3.tar.gz

Adding GIT to $PATH

Here is the important part of this tutorial. To force using GIT that was just installed we need to add it to $PATH. It will prevent running git installed with yum.

Create file git.sh in this location:

sudo vi /etc/profile.d/git.sh

and paste there following contents:

pathmunge /usr/local/git/bin

After exiting and logging again to your SSH session you will always call GIT compiled from source:)

How to update GIT to newer version?

When new version of GIT will be released, simply repeat the steps in this tutorial, but with newer version. You don't need to remove GIT that is currently installed. New build will override the one that you already have.

Drawbacks of compiling git from source

You must know that there are certain drawbacks of compiling git from source. There are not really painful to be honest.

You must manually check if there is new version of git. When you try to do yum update, you won't see it on the list
Theoretically you will have two GITs installed - one via yum (old version), and latest version compiled from source. In practice, you will only use the one that was compiled.

If you don't want to install git manually you can use our Lamp On Steroids project based on Ansible. We have git role that automates the whole process.

How to configure NTP on CentOS 7

1 Reply

This will be pretty short tutorial about NTP. I will show you how to install and configure NTP on your server.

What is NTP?

NTP stands for Network Time Protocol and it keeps the server synchronised with correct time. Moreover you can synchronise all servers to one of your server to make sure that all have the same time and there are no time-shifts between them. It's nice to have such tool installed on your server.

How to set system timezone on CentOS?

First thing before that you need to do before installing and configuring NTP server is to set correct timezone. By using timezonectl command you can get information about your current timezone:

timedatectl | grep "Time zone"

Now, you need to decide which timezone you want to use. I'm strongly recommend using UTC as a timezone. Some servers are configured to particular timezone like Pacific/Honolulu for instance. But you should use UTC. If you are curious why, I highly recommend reading this article about UTC.

It will have some downsides, especially when you will have to translate output of date command to your local timezone. Most probably, there will be a time-shift.

In order to set timezone as UTC execute following command:

sudo timedatectl set-timezone UTC

However if you want to stick to your local timezone you can use replace UTC in command above to one of the options from this command:

timedatectl set-timezone UTC

For instance:

sudo timedatectl set-timezone Europe/Kiev

Now once you have your correct timezone set, you can proceed to setting up NTP server

How to install and configure Chrony as NTP server?

There are couple of options when you want to choose tool for NTP server. Two most popular are:

NTP
Chrony

In this tutorial I will use Chrony. It comes out of the box with CentOS and it's a bit faster than NTP. There is nice comparison of these two tools here.

Before you will install chrony make sure that you don't have ntp installed. Having two servers on one machine can cause some issues.

Execute following command to remove ntp and ntpdate packages:

sudo yum remove ntp ntpdate -y

Once you are sure that you don't have ntp installed, make sure that chrony is installed. It should be installed by default, but it is nice to check it:

sudo yum install chrony -y

Now you need to configure pooling from external NTP server. I highly recommend commonly used pool.ntp.org for that purpose.

In order to use the servers for pooling edit chrony configuration file:

sudo vi /etc/chrony.conf

At very top of the file there should be lines that starts with server. Remove them all and use the one from pool.ntp.org. So the beginning of your file after changes should look like that:

# Use public servers from the pool.ntp.org project.
# Please consider joining the pool (http://www.pool.ntp.org/join.html).
server 0.pool.ntp.org iburst
server 1.pool.ntp.org iburst
server 2.pool.ntp.org iburst
server 3.pool.ntp.org iburst

# Ignore stratum in source selection.
stratumweight 0
...

Save the changes. Now it's time to start NTP service and enable it on boot, so it will start automatically after system reboot:

sudo systemctl start chronyd.service
sudo systemctl enable chronyd.service

That's it! You configured NTP server!

You can use following commands here to check synchronisation status:

chronyc tracking
chronyc sources -v

You can use our Ansible LAMP on Steroids project to make configuration of your server easier!

It is based on Ansible. If you don't know what Ansible is, check our tutorial first.

Clone our repository and setup your server faster with LAMP on steroids.

Yum package manager – tips and tricks

Create list of installed packages

One of the first thing that I do on new server is making a list of installed packages. When you have fresh system everything works out of the box. But after some time you installed so many packages etc. and you might wonder - was this package installed at the beginning? Can I safely remove it? Such list comes handy then:)

sudo yum list installed > /etc/yum/initial-packages-list

Enable GPG check before you will install anything

GPG checks are really important. They tell yum to verify signature of downloaded package. It lower the chance of installing suspicious packages from weird sources. Usually they are enabled, but it's worth to check that.

You need to look for following line:

gpgcheck=1

in few files. The most important file is /etc/yum.conf Make sure that line above is present in [main] section. If it's set to 0 or line is missing, add it. If there are any other sections than [main], check them too.

In addition to that you need to check all .repo files in /etc/yum.repos.d directory. Most of them will have gpgcheck enabled. If they have gpgcheck=0 you can either remove the line, or change it to gpgcheck=1. If you will remove the line, default from /etc/yum.conf will be used.

Install packages with yum and speed it up with delta RPM

If you want to install additional package you need to execute following command:

sudo yum install PACKAGE_NAME -y

You can install multiple packages at once by separating them with space. -y modifier will skip prompt with confirmation.

You can speed up packages installation with Delta RPMs. Delta RPM contains the difference between old and new package. It means that you don't have to download entire package, just the difference between the versions. It results in smaller size of downloaded file and generally makes installing packages faster. It doesn't work for all packages, but it's nice to have it installed anyway:

sudo yum install deltarpm -y

That's it, it will be enabled from now on, you don't need to do anything else!

Remove unnecessary packages with yum

There is an unwritten rule - the less packages the better. It's important from security and performance side. If you have less stuff installed you will be able to keep everything updated. You won't have to fix new security issue in package you don't need etc.

There are couple of packages that can be safely removed from servers with CentOS.

To remove the package use command:

sudo yum remove PACKAGE_NAME_HERE -y

Use -y argument with caution, it will immediately remove package from your system without the need for confirmation from you side:)

Command here will remove unnecessary packages from CentOS. If you plan to use MariaDB instead of MySQL, don't remove mariadb-libs!

It's also good to use yum info command to check information about given package if you are in doubt that you need to delete it.

Here is the list of packages that I usually delete and system is still working fine (CentOS 7.4)

sudo yum remove btrfs-progs gssproxy hyperv-daemons-license libgudev1 mariadb-libs NetworkManager NetworkManager-libnm NetworkManager-tui NetworkManager-wifi postfix -y

Get information about given package

If you are not sure about given package, you can always get information about it by executing following command:

yum info PACKAGE_NAME_HERE

Update core packages with yum

There are some packages that can't be (easily) removed from CentOS. They are usually safe to update and it's worth to keep the up to date.

In order to update given package you need to execute following command:

sudo yum update PACKAGE_NAME -y

You can also update all packages by executing following command:

sudo yum update -y

But I prefer to update packages in few steps, so I can control everything.

Update all core packages with following command. These are the packages that comes with CentOS and must be installed for proper system functioning. You can safely update them:

sudo yum update bash binutils ca-certificates centos-release chkconfig coreutils cpio cryptsetup-libs cyrus-sasl-lib curl dbus dbus-libs dracut elfutils-libelf expat filesystem gawk glib2 glibc glibc-common gmp gnupg2 grep gzip kmod kmod-libs kpartx krb5-libs libblkid libcap libdb libffi libgcc libgcrypt libmount libuuid libsemanage libstdc++ libssh2 libtasn1 libxml2 lua ncurses nspr nss nss-softokn nss-softokn-freebl nss-sysinit nss-tools nss-util openldap openssl-libs pam pcre pinentry procps-ng python python-libs python-pycurl readline rpm setup shadow-utils shared-mime-info systemd systemd-libs systemd-sysv util-linux tar tzdata xz yum yum-plugin-fastestmirror zlib -y

Update other packages with yum

Apart from core packages there are some tools added to CentOS minimal installation. They can be removed but it's good to keep them installed as they add neat features to OS. Let's update them too:

sudo yum update audit authconfig bind-libs-lite bind-license biosdevname cronie cronie-anacron crontabs device-mapper-persistent-data dhclient dmidecode dnsmasq dracut-network dracut-config-rescue e2fsprogs epel-release ethtool file fipscheck freetype gettext gnutls gobject-introspection grub2 grubby hwdata iproute iprutils iputils initscripts irqbalance kbd kernel kernel-tools kernel-tools-libs kexec-tools libcroco libdrm libgomp libnetfilter_conntrack libpciaccess lsscsi lvm2 make microcode_ctl mozjs17 nettle openssh openssh-clients openssh-server openssl os-prober parted pciutils-libs plymouth plymouth-scripts policycoreutils polkit python-gobject-base python-perf python-pyudev rdma-core rsync rsyslog selinux-policy selinux-policy-targeted sudo tuned vim-minimal virt-what xfsprogs yum-utils -y

Remove not used packages with yum

Yum comes with nice feature - it can removes packages that are not used on the system. You can remove them by executing

sudo yum autoremove -y

Block certain package updates

Sometimes there is an need to block certain packages from being updated or installed. You can block them in two ways.

First is to --exclude option that you can use on any yum command. For instance:

sudo yum update --exclude="git*"

* means that anything that will be after git will also be excluded. Command above will update all packages, but git won't be updated or installed.

Second option is to add same rule to /etc/yum.conf file under [main] section. It will affect all yum commands and you will not have to use --exclude.

Take a look at the example below:

[main]
cachedir=/var/cache/yum/$basearch/$releasever
keepcache=0
exclude=git*
...

One thing that you should remember is that it might also affect other packages. For instance if you try to install/update package that has git as a dependency, you won't be able to do that.

Check if there are any other packages to update?

After I remove unwanted stuff, updates core and other packages and making autoremove I check if there are any other packages marked for update.

You can check it with

yum list updates

If there are any other packages left for update, you should get info about them and update them or remove them from your system.

Power yum with cron

If you have slower connection on your server, sometimes making yum update takes a lot of time. You can speed it up with yum-cron. It adds a cronjob that periodically checks for packages to update. It can also download them and install them automatically.

First, lets install yum-cron:

sudo yum install yum-cron -y

Before running it I strongly suggest to remove automatic updates. It's OK to check for updates and download them, but you should NEVER install them automatically. They can really screw up your server (been there, done that (unfortunately...)).

If packages are broken or have configuration changes they will break your server anyway, but it's way better to fix it right away, than break it with auto update at 2 AM.

So make sure that auto installing is disabled. Edit /etc/yum/yum-cron.conf and make sure that apply-updates option is set to no. If it is, you can safely start yum-cron with

sudo systemctl start yum-cron.service

You should also add it to system startup, so when you reboot your machine, it will start automatically:

sudo systemctl enable yum-cron.service

Searching for packages

Sometimes you need to install package name, but you are not sure about the name. You can easily search available packages with yum search command:

yum search PACKAGE_NAME

Additional tools

There are some nice packages that are not installed by default on minimal installation, like net-tools (provides netstat command). You can install it on your system, it is pretty handy:)

sudo yum install net-tools -y

Easier way?

You can use our Ansible LAMP on Steroids project to make configuration of your server easier!

If you don't know what Ansible is, check our tutorial first.

Clone our repository and setup your server faster with LAMP on steroids.

Ansible tutorial – part 4

1 Reply

In this part we will focus on Ansible variables. How to use them in playbooks, tasks and templates.

A short word about Ansible tutorial

This is multipart series about Ansible, great tool for provisioning and configuration management. You will learn how to create useful roles and use Ansible in efficient way. You will also pick up some good practices:)

If you like tl;dr take a look at shorter veresion of this tutorial on GitHub: https://github.com/blacksaildivision/ansible-tutorial
There are also some examples!

Part 1: Intro to Ansible and Inventory files
Part 2: Playbooks and tasks
Part 3: Templates and handlers
Part 4: Variables

What is the purpose of variables in Ansible?

Imagine the situation where you have two servers. On each of them you need to install nginx. But each server requires a bit different configuration. Let's take simplest case - on server no. 1 we want nginx to be running on port 80, which is default port. But on the second server it should work on port 8080.

We could create two separate roles and set different tasks, but this is an overkill. Lot of copy pasting and much harder maintenance of two roles that does exactly the same. Let's use variables for that!

Defaults

We will expand the role we created in previous part - nginx. First, we need to create a directory inside nginx directory and call it defaults (it should be on the same level as tasks directory). Inside this directory create file and name it main.yml Ansible will automatically include that file.

Take a look at example variable:

nginx_port: 80

Variable should have a name, if you are using any other programming language, there are the same rules as everywhere, it can't start with number etc.

We always prefix our variables with role name. It's good practice and it helps avoid conflicts between roles. We could name it simply with port, but if we would create additional role that will have also port variable, prefixes becomes really handy:)

After that there is a colon following by variable value. You can use different values like numbers, strings, lists etc.

Let's add following variables to our default/main.yml file:

nginx_service_name: nginx
nginx_yum_packages:
 - epel-release
 - nginx
nginx_port: 80

These are only example variables. Normally we would use only nginx_port variable. First two are not necessary in real playbooks. There is no need to convert everything to variables. It can lead to mess in playbook. In general we only use variables for values that can be different among hosts. For instance the port. Sometimes we run nginx on different port than 80. But service name and required yum packages are always the same. So we shouldn't use it as variable. Here we added it only for example purposes.

How to use variables in tasks?

We can use variables in tasks and in templates. We will start with simples usage in task. Take a look at the task we have in our tasks/main.yml file:

- name: enable nginx
  service:
    name: nginx
    state: started
  tags: [nginx, status]

Let's say that we want to replace service name with our variable nginx_service_name . So change this task like so:

- name: enable nginx
  service:
    name: "{{ nginx_service_name }}"
    state: started
  tags: [nginx, status]

In order to use variables in Ansible, we need to wrap them into double curly brackets {{}} . Spaces before and after variables are not necessary, but they add a bit of readability. However, if we want to use variable in task, we need to wrap everything in quotes "" . So here is the general rule for variables: "{{ variable_name_here }}" .

You can re-run the playbook, but you should see no difference there, as we just replace nginx name to the same value, but hidden under the variable.

You can use single variable in multiple places in your playbook.

Lists and loops in Ansible

Let's take another look at our tasks, in particular, first two tasks:

- name: install epel
  yum:
    name: epel-release
    state: present
  tags: [nginx]

- name: install nginx
  yum:
    name: nginx
    state: present
  tags: [nginx]

As you can see, they are doing the same thing - installing packages to our server. We can combine them into one task to avoid code duplication. Take a look at combined tasks (and remove these two tasks from above):

- name: install required packages
  yum:
    name: "{{ item }}"
    state: present
  with_items:
    - epel-release
    - nginx
  tags: [nginx]

Let start with new statement with_items . In short words it means - execute given module like yum , template , user etc for each value on the list. So yum module will be executed twice, because we have two items on our list - epel-release and nginx. You can specify any number of items you need. We just need two, but you can add any additional software you want to install.

Next, you need to use special variable item . The value will contains item from with_items list. So in first iteration under item we will have epel-release value, and in second iteration we will have nginx value there.

Lists are really useful and this is only basic usage of them. We will get into lists deeper in next parts of this tutorial.

You can also pass variable from defaults to with_items statement. We defined variable called nginx_yum_packages in defaults/main.yml . It is a list, so we can use it with with_items like so:

- name: install required packages
  yum:
    name: "{{ item }}"
    state: present
  with_items: "{{ nginx_yum_packages }}"
  tags: [nginx]

Variables in templates

We can also use variables inside template files. In our nginx role we have template named nginx.conf.j2 . Open it in editor and look for port in server block:

....
server {
        listen       80 default_server;
        listen       [::]:80 default_server;
        server_name  _;
....

As you can see we have port 80 there. We can replace it to our variable that we defined previously - nginx_port .

In templates we also need to use double curly brackets, but we need to omit quotes. They are not necessary in template files. If you will use "" in template it will also appear in end result file, which in some cases is not a good thing.

Let's replace 80 with our variable:

....
server {
        listen       {{ nginx_port }} default_server;
        listen       [::]:{{ nginx_port }} default_server;
        server_name  _;
....

So now it's easy and we can just replace the value in our variable files to change the port. No need to mess with template 🙂

You can re-run the playbook, it should give you the same result as the end template after processing is not changed.

Overriding variables

You may wonder, why the directory with variables is called defaults ? It's because they are default variables if you don't specify host related value.

In most cases default value is enough to execute the playbook, but sometimes you need to change something in particular host. Imagine the situation we mentioned at the beginning - we want to change the port from 80 to 8080, but without changing anything in the role.

Open playbook.yml file and simply set the variable value like so:

- hosts: ansible_tutorial
  become: yes
  become_user: root
  roles:
   - nginx
  vars:
    nginx_port: 8080

There is special block called vars . You can specify host(s) related values there. So when you execute the playbook, variable nginx_port will have value 8080, not 80 like we defined in defaults. Host related variables has priority over role defaults variables.

You can override as many variables as you need. Keep in mind that you should override existing values in defaults. You shouldn't add custom variable in playbook file and use it inside role without having the default value specified. It will work fine, but if you try to use the role in another host and you won't specify missing variable, you will get an error.

Ansible tutorial – part 3

3 Replies

In this part we will use templates and handlers in Ansible.

A short word about Ansible tutorial

If you like tl;dr take a look at shorter veresion of this tutorial on GitHub: https://github.com/blacksaildivision/ansible-tutorial
There are also some examples!

Part 1: Intro to Ansible and Inventory files
Part 2: Playbooks and tasks
Part 3: Templates and handlers
Part 4: Variables

Ansible templates

Templates are simple text files that we can use in Ansible. Most of the time you will use them to replace configuration files or place some documents on the server.

Start from creating new directory inside our nginx role and name it templates . It should be on the same level as tasks directory. This it the place where we will store our templates.

Let's say that we want to change the configuration of nginx. We will use simplest way and we will just replace whole nginx.conf file. Inside template directory create a file and name it for instance nginx.conf.j2 J2 is extension of Jinja2 templating language that Ansible is using.

Copy+paste following content into that file:

# For more information on configuration, see:
#   * Official English Documentation: http://nginx.org/en/docs/
#   * Official Russian Documentation: http://nginx.org/ru/docs/

user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;

events {
    worker_connections 1024;
}

http {
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile            on;
    tcp_nopush          on;
    tcp_nodelay         on;
    keepalive_timeout   65;
    types_hash_max_size 2048;

    include             /etc/nginx/mime.types;
    default_type        application/octet-stream;

    # Load modular configuration files from the /etc/nginx/conf.d directory.
    # See http://nginx.org/en/docs/ngx_core_module.html#include
    # for more information.
    include /etc/nginx/conf.d/*.conf;

    server {
        listen       80 default_server;
        listen       [::]:80 default_server;
        server_name  _;
        root         /usr/share/nginx/html;

        # Load configuration files for the default server block.
        include /etc/nginx/default.d/*.conf;

        location / {
        }

        error_page 404 /404.html;
            location = /40x.html {
        }

        error_page 500 502 503 504 /50x.html;
            location = /50x.html {
        }
    }
}

We just logged in into the server and copied the contents of /etc/nginx/nginx.conf file. This is most common way to obtain configs for template files. Copy their contents from server after installation of the given tool. We can be sure that this is valid config and it should work out of the box.

Task for template

We need the task that will replace current file on the server with the template.

Just before the task that checks if nginx is working add another task:

- name: copy nginx configuration file
  template:
    src: nginx.conf.j2
    dest: /etc/nginx/nginx.conf
    owner: root
    group: root
    mode: u=rw,g=r,o=r
  tags: [nginx, config]

We need to name the task first. After that we are using template module from Ansible. It takes couple of parameters. Two mandatory and couple of optional parameters.

src is required parameter. You need to specify name of the template file from templates directory. In our case it's the file we just created - nginx.conf.j2
dest is another required parameter. It's the path where the file should be placed on the server. Nginx in default installation keeps it config files in /etc/nginx directory. It's important to use filename as well, not only directory!

Rest of the parameters are optional, but when you are using template module, good practice is to specify the ownership and file permissions.

owner is username that should own the file, like in chown command
group is the group that should own the file, like in chown command
mode is set of permissions for the file, like from chmod command. There are couple of ways to set the permissions. We prefer the syntax like we used in the task above. But you can also specify numeric version like that:

mode: 0644

Handlers

As you might now, changing something in configuration file won't give any results without restarting nginx. One way to solve that problem, would be to edit service task in our role and set the state to restarted instead of started . It would result in restarting nginx. But if you would run this playbook over and over again, on each run it would restart nginx, which should not happen.

It would be the best if we could restart nginx automatically, but only when there are changes in configuration file.

Handlers to the rescue! Ansible comes with awesome feature called handlers. You can execute handler, only when given task will have status changed.

Edit the template task above and add notifier:

- name: copy nginx configuration file
  template:
    src: nginx.conf.j2
    dest: /etc/nginx/nginx.conf
    owner: root
    group: root
    mode: u=rw,g=r,o=r
  notify: restart nginx
  tags: [nginx, config]

You can add notifier to any task. You must specify the name of the notifier. There are almost no restrictions, it's just a string.

Next, we need to add yet another directory to our role and call it handlers (directory should be on the same level as tasks and templates).

Inside that directory create text file and name it main.yml (All main.yml files are automatically included by Ansible).

Place following content in handlers/main.yml file, you just created:

- name: restart nginx
  service:
    name: nginx
    state: restarted

Each handler should have a name. You need to pass the name to notify in task like we did in example above. Next you can pick any module you want from Ansible and use it. Most of the time it'll be service module. Here we can use restarted state.

When something will change in the template, task will result in changed status. Handler will be notified, will execute and it will restart nginx.

Run your playbook couple of times, and make sure that you will have everything to OK in recap. Change the template and add epoll support for nginx in templates/nginx.conf.j2 file:

events {
    worker_connections 1024;
    use epoll;
}

Run playbook. You will notice that the handler will execute and will restart nginx.

Ansible tutorial – part 2

A short word about Ansible tutorial

If you like tl;dr take a look at shorter veresion of this tutorial on GitHub: https://github.com/blacksaildivision/ansible-tutorial
There are also some examples!

Part 1: Intro to Ansible and Inventory files
Part 2: Playbooks and tasks
Part 3: Templates and handlers
Part 4: Variables

Playbooks

Playbooks are essential feature of Ansible. In Playbooks we specify what and how our server will be configured and provisioned.

Go to our directory that we created in first part - vagrant_ansible_tutorial . Create a file inside and name it playbook.yml . Name is irrelevant here, you can use anything you want. Open this file in editor and paste following content:

- hosts: ansible_tutorial
  tasks:
   - debug:
      msg: Test of Ansible Playbook!

First line contains host(s) that we want to provision. You must specify group or single host from Inventory file hosts that we created in part 1. Good practice is to specify group instead of single host. In our case it is ansible_tutorial , so we will provision our Vagrant box.

You can also specify multiple hosts here and set different tasks or roles:

- hosts: webservers
  tasks:
   - debug:
      msg: Installing nginx
   - debug:
      msg: Installing PHP

- hosts: databases
  tasks:
   - debug:
      msg: Installing MongoDb

Next, we need to specify tasks . Task in Ansible is like single command in bash on steroids. With tasks you can install software, add users, remove files etc. Our task will just print out the message in command line, so no harm will be done if it won't work as expected.

We will get more into the tasks later on. For now just use the playbook above.

If you are familiar with YAML or Ansible you might notice that there are three dashes --- missing. If you will browser through the example playbooks on GitHub, you might note that all files starts with --- . This is coming from YAML and it is optional in Ansible. You might or might now use it, it doesn't make any difference. We never use it on our side and we won't use it in this tutorial as well. But if you prefer, you can add --- to start of each file like this:

---
- hosts: ansible_tutorial

Go to your terminal and execute following command:

ansible-playbook -i hosts playbook.yml

We need to pass two arguments. First one is an inventory file. Second argument is playbook file that we want to run.

Your output should look similar to:

PLAY [ansible_tutorial] ********************************************************

TASK [setup] *******************************************************************
ok: [192.168.60.70]

TASK [debug] *******************************************************************
ok: [192.168.60.70] => {
    "msg": "Test of Ansible Playbook!"
}

PLAY RECAP *********************************************************************
192.168.60.70              : ok=2    changed=0    unreachable=0    failed=0

You might note that there is also task named setup . This is Ansible internal task for gathering information about the server. We will use them later on. For now, just know, that such additional task exists.

Next lines contains our debug task. It got status ok and it was executed correctly.

Probably the most important part of this output is recap . It contains information about current run of playbook. There are 4 metrics that are worth checking:

ok - tasks that were executed correctly or didn't change anything on provisioned server.
changed - things that were modified, like you just add new user. If you run the same playbook twice it will have ok status.
unreachable - usually should not happen, you will get it when host becomes unreachable, connection was dropped or you are out of internet etc
failed - when tasks were unable to execute correctly

In the end, perfect playbook should not contain any failed and unreachable errors. On first run you should get only ok and changed tasks. Second execution of the same playbook should result only in ok statuses.

Roles in Ansible

Role, apart from Playbook, is another essential feature of Ansible. In previous example we created tasks directly in playbook file. You must know that this is wrong approach. 99% of the time you will use roles instead of using tasks directly in playbooks. We used it only to show you how to execute working playbook.

Think of how do you use a server. For sure you need to install some kind of web server like apache or nginx. There will be couple of things to do (tasks) in order to install nginx. We can group them into single role. Roles will help you organise your structure.

Role should have single responsibility like installing and configuring nginx. It should also be reusable. So you can for instance use it in different group of servers or even use it in separate project.

Let's create our first test role that will install and configure nginx. In the project directory create directory and name it roles . In this directory you should keep all roles you create.

Inside there you should create another directory and name it nginx . Name of the directory is the same as name of your role. Each role directory, like nginx, usually contains couple of directories inside like tasks, defaults, handlers etc. For now we will just create directory named tasks . Inside tasks directory, create file and name it main.yml In the end your directory structure should look like this:

At the very beginning it might look messy, but after a while you will see that it's quite clever way of organisation.

Just for the start, we will use our debug message as simple start. Your main.yml file should look like this:

- debug:
    msg: Test of Ansible Playbook!

This is just copy+paste from playbook.yml Now we need to include our role into the playbook itself. Edit playbook.yml so it will look like that:

- hosts: ansible_tutorial
  roles:
   - nginx

As you can see, we just removed tasks from there and added roles instead. Each hosts group can contain multiple roles. You can also reuse given role in different hosts.

Now let's run our playbook again:

ansible-playbook -i hosts playbook.yml

Output will be very similar as before and the end result should be the same.

Tasks in Ansible

Let's write some some real tasks that will help us install nginx. Our OS is CentOS 7 as we specified it in Vagrantfile . In order to get nginx up we need to install epel repostitory and nginx itself. Start by removing everything from main.yml in tasks directory. We don't need debug message there, it was only for test purposes. Instead create following task:

- name: install epel
  yum:
    name: epel-release
    state: present
  tags: [nginx]

Each task should start with name. It's not necessary but it's very informative and easier to read in ansible command output.

Next line contains module name that we will use. In CentOS we are using yum for installing files. If the base images was Ubuntu for instance, we would use apt module. Yum/Apt module needs to know two things. First is the name of the package we would like to install. In our case it's epel-release . Second thing is state - whether application should be installed or removed. We want to install it, so we should use present state.

Last thing are tags. They are optional in each task but usually it's good to use them. When you use ansible-playbook command, you can run tasks that contains only given tag. We basically add tag(s) for each task we create. Sometimes it's not necessary, but it's just convenient to have them there.

Run your playbook with ansible-playbook command like before. If you are using the same image as we in our tutorial, you might wonder that task resulted in ok status, instead of changed . It should be changed when we install new packages, right? Yes, it should, but the vagrant image has epel-release already installed. So Ansible is checking if it's installed, and if so, it won't do anything. Clever, isn't it?

Once we have epel repository we can install nginx. Let's add another task to our role:

- name: install epel
  yum:
    name: epel-release
    state: present
  tags: [nginx]
  
- name: install nginx
  yum:
    name: nginx
    state: present
  tags: [nginx]

As you can see, we are just installing nginx. Let's run the playbook. Whooops! You should get a failure - You need to be root to perform this command.

Privileges

In hosts file we specified that the user that should connect to the server is vagrant. But it does not have enough permissions to install things. We need to use privileges escalation.

Let's edit our playbook file and add become lines:

- hosts: ansible_tutorial
  become: yes
  become_user: root
  roles:
   - nginx

What we just did is we enabled become so now we can allow Ansible to execute command as different user. As we need to install some packages we need root user. Ansible will login as vagrant user, but each command will be executed as it would be root account.

You might ask - why not to edit hosts file and use root as a user. Answer is simple - the secure way is to disable ssh for root account to prevent harm to your server. root should not have the option to direct login via ssh.

You can also specify these values per task. But usually most of the tasks we do require root power, so we are defining it for entire connection to given host.

Try to run the command again. It should succeed this time. You should get changed=1 in output. It means that nginx has been installed. Try to run the playbook again. It will get ok status, so nothing was changed.

Finish installing nginx with Ansible

Last missing task is to start nginx. Add following task inside main.yml

- name: enable nginx
  service:
    name: nginx
    state: started
  tags: [nginx, status]

We are using Ansible module called service. It is responsible for managing services. In general you can find all possible modules for Ansible on this page. It's super useful and it contains lot of examples for each command. It's also well written. If you don't know something, just take a look at documentation of given module and you should get an answer to your question:)

Service module requires two things - name of the service, which in our case is nginx, and state of the service. We want to start it, so state should be set to started .

Run your playbook. After thatm open your web browser and type the IP of your server (you can find it in Vagrant file). If you are following the tutorial, simply visit this URL: http://192.168.60.70/

You should see nginx welcome page!

Ansible tutorial – part 1

1 Reply

What you will learn in part 1 of this tutorial? What is Ansible? How to install Ansible on Windows/Linux/MacOS? How to prepare test environment for Ansible? Intro to Inventory files. First connection to the server.

A short word about Ansible tutorial

If you like tl;dr take a look at shorter veresion of this tutorial on GitHub: https://github.com/blacksaildivision/ansible-tutorial
There are also some examples!

Part 1: Intro to Ansible and Inventory files
Part 2: Playbooks and tasks
Part 3: Templates and handlers
Part 4: Variables

First thing first - what is Ansible?

Ansible is a simple yet powerful tool for configuration management and orchestration of your infrastructure. It speeds up installing software, configuring servers and most important - it reduces manual steps you need to make, to setup your server(s). It is also great alternative to Puppet and Chef. Both are similar tools to Ansible, but in my opinion Ansible is much easier to learn and master.

How do WE use Ansible (possible use cases)?

We have lot of servers that we need to take care of. At the beginning we were just logging into our servers via SSH and fire all commands in terminal. It was painful job, so we decided that we need to automate things. We checked Puppet, Chef and Ansible and we decided to stick with the last one. It was really good choice. Now we use Ansible for:

Installing our entire LAMP stack on new server
Configuration of most of the software on our servers
Testing if servers are configured properly
Deploying our applications
Keeping Infrastructure As Code (IAC) - all infrastructure is in single Git repository

Most of these tasks are done with single command. For instance we can update nginx on all servers with bumping the version in our variable file and execute one command in terminal. It will be updated on all servers at once. No more logging it via SSH and executing commands one by one:)

What do I need to know about Ansible?

Ansible is using files for configuration and provisioning. By using Ansible you can go with IAC - Infrastructure As Code. It means that you can keep you entire infrastructure in GIT repository. You can track changes and use all goodies that comes with source version control.

Ansible is based on YAML files. If you don't know it, you will learn it in minutes. Really easy and simple.

For templates it uses Jinja2 engine. Powerful and easy to learn.

It does not require additional software to be installed on the server. It uses old-school ssh protocol for connecting and configuring the server. One requirements is to have Python installed on the server, but these days, all common Linux distributions have it out of the box.

Let's get started - how to install Ansible?

No matter what is you operating system, when you will be installing Ansible, make sure that you will install version 2.0 or higher. It has lot more features than 1.x versions. It is stable and commonly used these days.

If you are using OS X or Linux, it is not a big deal for you. Here are some tutorials from Ansible documentation - how to install Ansible on OS X or Linux? The easiest way is to use APT, Yum for Linux and Pip or Homebrew. Please make sure that you are installing Ansible in version 2.0 or higher:)

If you are using Windows, it's a bit more hassle, unless you already have the Windows with Bash terminal! If you do, simply use APT for installing Ansible. However if you don't have it yet, you need to install more things. On our Windows machines we use Babun (much better version of Cygwin, if you didn't hear about it, check it out right now!). To install Ansible with Babun, please use this tutorial.

Prepare test environment

You will need a server or virtual machine where you can test Ansible. If you are just starting with I highly recommend to use empty and fresh server. You don't want to mess up production server while learning it!

If you have some clean server on DigitalOcean for example, you can use it. However easier (and free) way is to install Vagrant and use dummy box. If something will go wrong, you can always remove everything and start fresh. For the purpose of this tutorial, we will use Vagrant.

Start with a new directory for example: vagrant_ansible_tutorial . In this directory create file named Vagrantfile and paste following content into that file:

# -*- mode: ruby -*-
# vi: set ft=ruby :

Vagrant.configure(2) do |config|
  config.vm.box = "geerlingguy/centos7"
  config.vm.network "forwarded_port", guest: 80, host: 8080
  config.vm.network "private_network", ip: "192.168.60.70"
  config.vm.provider "virtualbox" do |vb|
     vb.memory = "2048"
  end
end

You can adjust some parameters if you like. Save the changes in file and execute vagrant up in your terminal. You must be in vagrant_ansible_tutorial directory. Wait until your box will be up and running.

Inventory file

First thing that we need to do before we can actually start using Ansible is to create an Inventory file. Inventory files are the way to specify connection to the servers for Ansible. You can treat is as an alternative to SSH command. So instead of executing:

ssh -i my_private_key_file_path username@server_ip_or_domain.com

you need to add following line to inventory file:

server_ip_or_domain.com ansible_user=username ansible_private_key_file=my_private_key_file_path

Let's create a file in our directory project and name it hosts . Paste following code into that file:

[ansible_tutorial]
192.168.60.70 ansible_user=vagrant ansible_private_key_file="./.vagrant/machines/default/virtualbox/private_key"

Let's start from 2nd line. We specified IP address to our server. This is the same IP as in Vagrantfile . Instead of using the IP you can use domain address like blacksaildivision.com.

Next is the user that we will use to connect to the server. Vagrant default user is vagrant . Last thing is private key that we are using for accessing the server. After starting the box, Vagrant will create new key in the .vagrant directory.

Alternatively you can specify the password instead of SSH key. But if you are using passwords instead of keys for accessing your server, you should change it ASAP, because you are doing it wrong. Passwords are easy to hack. Trust me, at the beginning our server was hacked twice, even though we had pretty strong password. We didn't know about such thing as SSH back then:) If you will use password, Ansible will also give you a warning.

If you need to alter the port or add additional arguments for your connection, check Ansible documentation here.

Groups in Inventory files

In first line we have [ansible_tutorial] and it's name of the group of the servers. You can omit it, but the good practice is to specify the group for server(s). Take a look on following file:

[web]
147.120.24.3 ansible_user=developer ansible_private_key_file="~/.ssh/web"
147.120.24.4 ansible_user=developer ansible_private_key_file="~/.ssh/web"
147.120.24.5 ansible_user=developer ansible_private_key_file="~/.ssh/web"
 	 	
[database]
11.60.215.157 ansible_user=db ansible_private_key_file="~/.ssh/db"
11.60.215.158 ansible_user=db ansible_private_key_file="~/.ssh/db"

There are two groups of the servers - web and database. Now we can execute different commands on each group. For instance on database group we can install MongoDB related software and on servers that belongs to web group we can install nginx+php.

It's good practice to specify a group, even if you have single server. It's more descriptive way than using IP or host name.

Testing connection to server

Last thing in this part is to test connection between Ansible and the server. Execute following command:

ansible ansible_tutorial -m ping -i hosts

If you didn't connect to this server via ssh before, you will get message about authentication for new host. Just type yes.

You should get following input:

192.168.60.70 | SUCCESS => {
    "changed": false, 
    "ping": "pong"
}

What we just did is we checked if Ansible is able to connect to the server. If you get SUCCESS message, you are ready to start provisioning with Ansible!

Explaining the command you just executed. There are 3 parameters. First one is the group from inventory file. You can also specify particular host or IP from inventory file or you can use all to test all servers within given inventory file:

ansible all -m ping -i hosts

Second thing is module name. We need to use ping module to check the connection. Last thing is inventory file, which in our case is created hosts file.

If you have the problems with getting SUCCESS message, you need to debug the connection. Use -vvvv to get debug output. Usually you are able to figure out what is wrong with the connection from debug output:

ansible all -m ping -i hosts -vvvv

In most cases it is wrong private key path or some typos.

Default <head> tag on clean installation (before)

Remove or set up DNS prefetch links

Remove XML feeds

Remove emojis

Disable WordPress REST API

Disable XML-RPC

Remove Windows Live Writer manifest

Remove info about generator

Bonus 1 - remove Gutenberg default styles

Bonus 2 - remove unnecessary attributes from link stylesheets

Bonus 3 - Remove XFN meta data profile

<head> tag after the changes

Can I get full code, please?

Do I need latest version of OpenSSL?

How to check current version of OpenSSL?

How to install latest version of OpenSSL?

Add new version to PATH

Link libraries

How to check if THP is enabled or not?

How to disable THP manually?

How to disable THP automatically?

Easy way to install git on CentOS with yum

How to compile git from source on CentOS?

Adding GIT to $PATH

How to update GIT to newer version?

Drawbacks of compiling git from source

What is NTP?

How to set system timezone on CentOS?

How to install and configure Chrony as NTP server?

Create list of installed packages

Enable GPG check before you will install anything

Install packages with yum and speed it up with delta RPM

Remove unnecessary packages with yum

Get information about given package

Update core packages with yum

Update other packages with yum

Remove not used packages with yum

Block certain package updates

Check if there are any other packages to update?

Power yum with cron

Searching for packages

Additional tools

Easier way?

A short word about Ansible tutorial

What is the purpose of variables in Ansible?

Defaults

How to use variables in tasks?

Lists and loops in Ansible

Variables in templates

Overriding variables

A short word about Ansible tutorial

Ansible templates

Task for template

Handlers

A short word about Ansible tutorial

Playbooks

Roles in Ansible

Tasks in Ansible

Privileges

Finish installing nginx with Ansible

Tags

A short word about Ansible tutorial

First thing first - what is Ansible?

How do WE use Ansible (possible use cases)?

What do I need to know about Ansible?

Let's get started - how to install Ansible?

Prepare test environment

Inventory file

Groups in Inventory files

Testing connection to server