Tag Archives: nginx

Ansible variables

Ansible tutorial – part 4

In this part we will focus on Ansible variables. How to use them in playbooks, tasks and templates.

A short word about Ansible tutorial

This is multipart series about Ansible, great tool for provisioning and configuration management. You will learn how to create useful roles and use Ansible in efficient way. You will also pick up some good practices:)

If you like tl;dr take a look at shorter veresion of this tutorial on GitHub: https://github.com/blacksaildivision/ansible-tutorial
There are also some examples!

Part 1: Intro to Ansible and Inventory files
Part 2: Playbooks and tasks
Part 3: Templates and handlers 
Part 4: Variables

 

What is the purpose of variables in Ansible?

Imagine the situation where you have two servers. On each of them you need to install nginx. But each server requires a bit different configuration. Let's take simplest case - on server no. 1 we want nginx to be running on port 80, which is default port. But on the second server it should work on port 8080.

We could create two separate roles and set different tasks, but this is an overkill. Lot of copy pasting and much harder maintenance of two roles that does exactly the same. Let's use variables for that!

Defaults

We will expand the role we created in previous part - nginx. First, we need to create a directory inside nginx directory and call it defaults  (it should be on the same level as tasks directory). Inside this directory create file and name it main.yml  Ansible will automatically include that file.

Take a look at example variable:

nginx_port: 80

Variable should have a name, if you are using any other programming language, there are the same rules as everywhere, it can't start with number etc.

We always prefix our variables with role name. It's good practice and it helps avoid conflicts between roles. We could name it simply with port, but if we would create additional role that will have also port variable, prefixes becomes really handy:)

After that there is a colon following by variable value. You can use different values like numbers, strings, lists etc.

Let's add following variables to our default/main.yml  file:

nginx_service_name: nginx
nginx_yum_packages:
 - epel-release
 - nginx
nginx_port: 80

These are only example variables. Normally we would use only nginx_port variable. First two are not necessary in real playbooks. There is no need to convert everything to variables. It can lead to mess in playbook. In general we only use variables for values that can be different among hosts. For instance the port. Sometimes we run nginx on different port than 80. But service name and required yum packages are always the same. So we shouldn't use it as variable. Here we added it only for example purposes.

How to use variables in tasks?

We can use variables in tasks and in templates. We will start with simples usage in task. Take a look at the task we have in our tasks/main.yml  file:

- name: enable nginx
  service:
    name: nginx
    state: started
  tags: [nginx, status]

Let's say that we want to replace service name with our variable nginx_service_name . So change this task like so:

- name: enable nginx
  service:
    name: "{{ nginx_service_name }}"
    state: started
  tags: [nginx, status]

In order to use variables in Ansible, we need to wrap them into double curly brackets {{}} . Spaces before and after variables are not necessary, but they add a bit of readability. However, if we want to use variable in task, we need to wrap everything in quotes "" . So here is the general rule for variables: "{{ variable_name_here }}" .

You can re-run the playbook, but you should see no difference there, as we just replace nginx name to the same value, but hidden under the variable.

You can use single variable in multiple places in your playbook.

Lists and loops in Ansible

Let's take another look at our tasks, in particular, first two tasks:

- name: install epel
  yum:
    name: epel-release
    state: present
  tags: [nginx]

- name: install nginx
  yum:
    name: nginx
    state: present
  tags: [nginx]

As you can see, they are doing the same thing - installing packages to our server. We can combine them into one task to avoid code duplication. Take a look at combined tasks (and remove these two tasks from above):

- name: install required packages
  yum:
    name: "{{ item }}"
    state: present
  with_items:
    - epel-release
    - nginx
  tags: [nginx]

Let start with new statement with_items . In short words it means - execute given module like yum , template , user  etc for each value on the list. So yum module will be executed twice, because we have two items on our list - epel-release and nginx. You can specify any number of items you need. We just need two, but you can add any additional software you want to install.

Next, you need to use special variable item . The value will contains item from with_items  list. So in first iteration under item  we will have epel-release value, and in second iteration we will have nginx value there.

Lists are really useful and this is only basic usage of them. We will get into lists deeper in next parts of this tutorial.

You can also pass variable from defaults to with_items statement. We defined variable called nginx_yum_packages in defaults/main.yml . It is a list, so we can use it with with_items like so:

- name: install required packages
  yum:
    name: "{{ item }}"
    state: present
  with_items: "{{ nginx_yum_packages }}"
  tags: [nginx]

Variables in templates

We can also use variables inside template files. In our nginx role we have template named nginx.conf.j2 . Open it in editor and look for port in server block:

....
server {
        listen       80 default_server;
        listen       [::]:80 default_server;
        server_name  _;
....

As you can see we have port 80 there. We can replace it to our variable that we defined previously - nginx_port .

In templates we also need to use double curly brackets, but we need to omit quotes. They are not necessary in template files. If you will use ""  in template it will also appear in end result file, which in some cases is not a good thing.

Let's replace 80 with our variable:

....
server {
        listen       {{ nginx_port }} default_server;
        listen       [::]:{{ nginx_port }} default_server;
        server_name  _;
....

So now it's easy and we can just replace the value in our variable files to change the port. No need to mess with template 🙂

You can re-run the playbook, it should give you the same result as the end template after processing is not changed.

Overriding variables

You may wonder, why the directory with variables is called defaults ? It's because they are default variables if you don't specify host related value.

In most cases default value is enough to execute the playbook, but sometimes you need to change something in particular host. Imagine the situation we mentioned at the beginning - we want to change the port from 80 to 8080, but without changing anything in the role.

Open playbook.yml  file and simply set the variable value like so:

- hosts: ansible_tutorial
  become: yes
  become_user: root
  roles:
   - nginx
  vars:
    nginx_port: 8080

There is special block called vars . You can specify host(s) related values there. So when you execute the playbook, variable nginx_port  will have value 8080, not 80 like we defined in defaults. Host related variables has priority over role defaults variables.

You can override as many variables as you need. Keep in mind that you should override existing values in defaults. You shouldn't add custom variable in playbook file and use it inside role without having the default value specified. It will work fine, but if you try to use the role in another host and you won't specify missing variable, you will get an error.

Ansible templates and handlers

Ansible tutorial – part 3

In this part we will use templates and handlers in Ansible.

A short word about Ansible tutorial

This is multipart series about Ansible, great tool for provisioning and configuration management. You will learn how to create useful roles and use Ansible in efficient way. You will also pick up some good practices:)

If you like tl;dr take a look at shorter veresion of this tutorial on GitHub: https://github.com/blacksaildivision/ansible-tutorial
There are also some examples!

Part 1: Intro to Ansible and Inventory files
Part 2: Playbooks and tasks
Part 3: Templates and handlers 
Part 4: Variables

 

Ansible templates

Templates are simple text files that we can use in Ansible. Most of the time you will use them to replace configuration files or place some documents on the server.

Start from creating new directory inside our nginx role and name it templates . It should be on the same level as tasks directory. This it the place where we will store our templates.

Let's say that we want to change the configuration of nginx. We will use simplest way and we will just replace whole nginx.conf file. Inside template directory create a file and name it for instance nginx.conf.j2  J2 is extension of Jinja2 templating language that Ansible is using.

Copy+paste following content into that file:

# For more information on configuration, see:
#   * Official English Documentation: http://nginx.org/en/docs/
#   * Official Russian Documentation: http://nginx.org/ru/docs/

user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;

events {
    worker_connections 1024;
}

http {
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile            on;
    tcp_nopush          on;
    tcp_nodelay         on;
    keepalive_timeout   65;
    types_hash_max_size 2048;

    include             /etc/nginx/mime.types;
    default_type        application/octet-stream;

    # Load modular configuration files from the /etc/nginx/conf.d directory.
    # See http://nginx.org/en/docs/ngx_core_module.html#include
    # for more information.
    include /etc/nginx/conf.d/*.conf;

    server {
        listen       80 default_server;
        listen       [::]:80 default_server;
        server_name  _;
        root         /usr/share/nginx/html;

        # Load configuration files for the default server block.
        include /etc/nginx/default.d/*.conf;

        location / {
        }

        error_page 404 /404.html;
            location = /40x.html {
        }

        error_page 500 502 503 504 /50x.html;
            location = /50x.html {
        }
    }
}

We just logged in into the server and copied the contents of /etc/nginx/nginx.conf file. This is most common way to obtain configs for template files. Copy their contents from server after installation of the given tool. We can be sure that this is valid config and it should work out of the box.

Task for template

We need the task that will replace current file on the server with the template.

Just before the task that checks if nginx is working add another task:

- name: copy nginx configuration file
  template:
    src: nginx.conf.j2
    dest: /etc/nginx/nginx.conf
    owner: root
    group: root
    mode: u=rw,g=r,o=r
  tags: [nginx, config]

We need to name the task first. After that we are using template module from Ansible. It takes couple of parameters. Two mandatory and couple of optional parameters.

  • src  is required parameter. You need to specify name of the template file from templates directory. In our case it's the file we just created - nginx.conf.j2
  • dest  is another required parameter. It's the path where the file should be placed on the server. Nginx in default installation keeps it config files in /etc/nginx directory. It's important to use filename as well, not only directory!

Rest of the parameters are optional, but when you are using template module, good practice is to specify the ownership and file permissions.

  • owner  is username that should own the file, like in chown command
  • group  is the group that should own the file, like in chown command
  • mode  is set of permissions for the file, like from chmod command. There are couple of ways to set the permissions. We prefer the syntax like we used in the task above. But you can also specify numeric version like that:
mode: 0644

Handlers

As you might now, changing something in configuration file won't give any results without restarting nginx. One way to solve that problem, would be to edit service task in our role and set the state to restarted  instead of started . It would result in restarting nginx. But if you would run this playbook over and over again, on each run it would restart nginx, which should not happen.

It would be the best if we could restart nginx automatically, but only when there are changes in configuration file.

Handlers to the rescue! Ansible comes with awesome feature called handlers. You can execute handler, only when given task will have status changed.

Edit the template task above and add notifier:

- name: copy nginx configuration file
  template:
    src: nginx.conf.j2
    dest: /etc/nginx/nginx.conf
    owner: root
    group: root
    mode: u=rw,g=r,o=r
  notify: restart nginx
  tags: [nginx, config]

You can add notifier to any task. You must specify the name of the notifier. There are almost no restrictions, it's just a string.

Next, we need to add yet another directory to our role and call it handlers  (directory should be on the same level as tasks and templates).

Inside that directory create text file and name it main.yml  (All main.yml files are automatically included by Ansible).

Place following content in handlers/main.yml  file, you just created:

- name: restart nginx
  service:
    name: nginx
    state: restarted

Each handler should have a name. You need to pass the name to notify  in task like we did in example above. Next you can pick any module you want from Ansible and use it. Most of the time it'll be service module. Here we can use restarted state.

When something will change in the template, task will result in changed status. Handler will be notified, will execute and it will restart nginx.

Run your playbook couple of times, and make sure that you will have everything to OK in recap. Change the template and add epoll support for nginx in templates/nginx.conf.j2  file:

events {
    worker_connections 1024;
    use epoll;
}

Run playbook. You will notice that the handler will execute and will restart nginx.

Ansible tutorial – part 2

This is part 2 of Ansible tutorial. In this part you will learn how to create playbooks, what are roles and tasks and how to use privileges escalation in Ansible.

A short word about Ansible tutorial

This is multipart series about Ansible, great tool for provisioning and configuration management. You will learn how to create useful roles and use Ansible in efficient way. You will also pick up some good practices:)

If you like tl;dr take a look at shorter veresion of this tutorial on GitHub: https://github.com/blacksaildivision/ansible-tutorial
There are also some examples!

Part 1: Intro to Ansible and Inventory files
Part 2: Playbooks and tasks
Part 3: Templates and handlers 
Part 4: Variables

 

Playbooks

Playbooks are essential feature of Ansible. In Playbooks we specify what and how our server will be configured and provisioned.

Go to our directory that we created in first part -  vagrant_ansible_tutorial . Create a file inside and name it playbook.yml . Name is irrelevant here, you can use anything you want. Open this file in editor and paste following content:

- hosts: ansible_tutorial
  tasks:
   - debug:
      msg: Test of Ansible Playbook!

First line contains host(s) that we want to provision. You must specify group or single host from Inventory file  hosts  that we created in part 1. Good practice is to specify group instead of single host. In our case it is  ansible_tutorial , so we will provision our Vagrant box.

You can also specify multiple hosts here and set different tasks or roles:

- hosts: webservers
  tasks:
   - debug:
      msg: Installing nginx
   - debug:
      msg: Installing PHP

- hosts: databases
  tasks:
   - debug:
      msg: Installing MongoDb

 

Next, we need to specify tasks . Task in Ansible is like single command in bash on steroids. With tasks you can install software, add users, remove files etc. Our task will just print out the message in command line, so no harm will be done if it won't work as expected.

We will get more into the tasks later on. For now just use the playbook above.

If you are familiar with YAML or Ansible you might notice that there are three dashes ---  missing. If you will browser through the example playbooks on GitHub, you might note that all files starts with --- . This is coming from YAML and it is optional in Ansible. You might or might now use it, it doesn't make any difference. We never use it on our side and we won't use it in this tutorial as well. But if you prefer, you can add ---  to start of each file like this:

---
- hosts: ansible_tutorial

Go to your terminal and execute following command:

ansible-playbook -i hosts playbook.yml

 We need to pass two arguments. First one is an inventory file. Second argument is playbook file that we want to run.

Your output should look similar to:

PLAY [ansible_tutorial] ********************************************************

TASK [setup] *******************************************************************
ok: [192.168.60.70]

TASK [debug] *******************************************************************
ok: [192.168.60.70] => {
    "msg": "Test of Ansible Playbook!"
}

PLAY RECAP *********************************************************************
192.168.60.70              : ok=2    changed=0    unreachable=0    failed=0

You might note that there is also task named setup . This is Ansible internal task for gathering information about the server. We will use them later on. For now, just know, that such additional task exists.

Next lines contains our debug task. It got status ok  and it was executed correctly.

Probably the most important part of this output is recap . It contains information about current run of playbook. There are 4 metrics that are worth checking:

  • ok  - tasks that were executed correctly or didn't change anything on provisioned server.
  • changed  - things that were modified, like you just add new user. If you run the same playbook twice it will have ok status.
  • unreachable - usually should not happen, you will get it when host becomes unreachable, connection was dropped or you are out of internet etc
  • failed  - when tasks were unable to execute correctly

In the end, perfect playbook should not contain any failed and unreachable errors. On first run you should get only ok and changed tasks. Second execution of the same playbook should result only in ok statuses.

Roles in Ansible

Role, apart from Playbook, is another essential feature of Ansible. In previous example we created tasks directly in playbook file. You must know that this is wrong approach. 99% of the time you will use roles instead of using tasks directly in playbooks. We used it only to show you how to execute working playbook.

Think of how do you use a server. For sure you need to install some kind of web server like apache or nginx. There will be couple of things to do (tasks) in order to install nginx. We can group them into single role. Roles will help you organise your structure.

Role should have single responsibility like installing and configuring nginx. It should also be reusable. So you can for instance use it in different group of servers or even use it in separate project.

Let's create our first test role that will install and configure nginx. In the project directory create directory and name it roles . In this directory you should keep all roles you create.

Inside there you should create another directory and name it nginx . Name of the directory is the same as name of your role. Each role directory, like nginx, usually contains couple of directories inside like tasks, defaults, handlers etc. For now we will just create directory named tasks . Inside tasks directory, create file and name it main.yml  In the end your directory structure should look like this:

Screen Shot 2016-06-22 at 13.02.08

At the very beginning it might look messy, but after a while you will see that it's quite clever way of organisation.

Just for the start, we will use our debug message as simple start. Your main.yml file should look like this:

- debug:
    msg: Test of Ansible Playbook!

This is just copy+paste from playbook.yml  Now we need to include our role into the playbook itself. Edit playbook.yml  so it will look like that:

- hosts: ansible_tutorial
  roles:
   - nginx

As you can see, we just removed tasks  from there and added roles  instead. Each hosts group can contain multiple roles. You can also reuse given role in different hosts.

Now let's run our playbook again:

ansible-playbook -i hosts playbook.yml

Output will be very similar as before and the end result should be the same.

Tasks in Ansible

Let's write some some real tasks that will help us install nginx. Our OS is CentOS 7 as we specified it in Vagrantfile . In order to get nginx up we need to install epel repostitory and nginx itself. Start by removing everything from main.yml  in tasks directory. We don't need debug message there, it was only for test purposes. Instead create following task:

- name: install epel
  yum:
    name: epel-release
    state: present
  tags: [nginx]

Each task should start with name. It's not necessary but it's very informative and easier to read in ansible command output.

Next line contains module name that we will use. In CentOS we are using yum for installing files. If the base images was Ubuntu for instance, we would use apt module. Yum/Apt module needs to know two things. First is the name of the package we would like to install. In our case it's epel-release . Second thing is state - whether application should be installed or removed. We want to install it, so we should use present  state.

Last thing are tags. They are optional in each task but usually it's good to use them. When you use ansible-playbook command, you can run tasks that contains only given tag. We basically add tag(s) for each task we create. Sometimes it's not necessary, but it's just convenient to have them there.

Run your playbook with ansible-playbook command like before. If you are using the same image as we in our tutorial, you might wonder that task resulted in ok  status, instead of changed . It should be changed when we install new packages, right? Yes, it should, but the vagrant image has epel-release already installed. So Ansible is checking if it's installed, and if so, it won't do anything. Clever, isn't it?

Once we have epel repository we can install nginx. Let's add another task to our role:

- name: install epel
  yum:
    name: epel-release
    state: present
  tags: [nginx]
  
- name: install nginx
  yum:
    name: nginx
    state: present
  tags: [nginx]

As you can see, we are just installing nginx. Let's run the playbook. Whooops! You should get a failure - You need to be root to perform this command.

Privileges

In hosts file we specified that the user that should connect to the server is vagrant. But it does not have enough permissions to install things. We need to use privileges escalation.

Let's edit our playbook file and add become  lines:

- hosts: ansible_tutorial
  become: yes
  become_user: root
  roles:
   - nginx

What we just did is we enabled become so now we can allow Ansible to execute command as different user. As we need to install some packages we need root user. Ansible will login as vagrant user, but each command will be executed as it would be root account.

You might ask - why not to edit hosts file and use root as a user. Answer is simple - the secure way is to disable ssh for root account to prevent harm to your server. root should not have the option to direct login via ssh.

You can also specify these values per task. But usually most of the tasks we do require root power, so we are defining it for entire connection to given host.

Try to run the command again. It should succeed this time. You should get changed=1 in output. It means that nginx has been installed. Try to run the playbook again. It will get ok  status, so nothing was changed.

Finish installing nginx with Ansible

Last missing task is to start nginx. Add following task inside main.yml

- name: enable nginx
  service:
    name: nginx
    state: started
  tags: [nginx, status]

We are using Ansible module called service. It is responsible for managing services. In general you can find all possible modules for Ansible on this page. It's super useful and it contains lot of examples for each command. It's also well written. If you don't know something, just take a look at documentation of given module and you should get an answer to your question:)

Service module requires two things - name of the service, which in our case is nginx, and state of the service. We want to start it, so state should be set to started .

Run your playbook. After thatm open your web browser and type the IP of your server (you can find it in Vagrant file). If you are following the tutorial, simply visit this URL: http://192.168.60.70/

You should see nginx welcome page!

Tags

Now it's the time to use some tags power. Note that we added another tag named status  to our task. Imagine the situation when you just want to check if nginx is working. You don't want to execute other tasks that are installing things. We should only execute tasks that have given tag:

ansible-playbook -i hosts playbook.yml --tags="status"

It should execute only two tasks. One is default setup task, and second one is the task with status tag. If you have multiple roles, like nginx, php, mysql and you will add status tag to all tasks connected to service module, command above will look through all roles. In one command you are able to check if all services are working fine:)

You can also specify multiple tags using comma separated list:

ansible-playbook -i hosts playbook.yml --tags="status,nginx"
Our services: