Monday, January 6, 2020

Ansible - Roles

Roles provide a framework for fully independent, or interdependent collections of variables, tasks, files, templates, and modules.
In Ansible, the role is the primary mechanism for breaking a playbook into multiple files. This simplifies writing complex playbooks, and it makes them easier to reuse. The breaking of playbook allows you to logically break the playbook into reusable components.
Each role is basically limited to a particular functionality or desired output, with all the necessary steps to provide that result either within that role itself or in other roles listed as dependencies.
Roles are not playbooks. Roles are small functionality which can be independently used but have to be used within playbooks. There is no way to directly execute a role. Roles have no explicit setting for which host the role will apply to.
Top-level playbooks are the bridge holding the hosts from your inventory file to roles that should be applied to those hosts.

Creating a New Role

The directory structure for roles is essential to create a new role.

Role Structure

Roles have a structured layout on the file system. The default structure can be changed but for now let us stick to defaults.
Each role is a directory tree in itself. The role name is the directory name within the /roles directory.
$ ansible-galaxy -h 

Usage

ansible-galaxy [delete|import|info|init|install|list|login|remove|search|setup] [--help] [options] ... 

Options

  • -h, --help − Show this help message and exit.
  • -v, --verbose − Verbose mode (-vvv for more, -vvvv to enable connection debugging)
  • --version − Show program's version number and exit.

Creating a Role Directory

The above command has created the role directories.
$ ansible-galaxy init vivekrole 
ERROR! The API server (https://galaxy.ansible.com/api/) is not responding, please try again later. 

$ ansible-galaxy init --force --offline vivekrole 
- vivekrole was created successfully 

$ tree vivekrole/ 
vivekrole/ 
├── defaults 
   └── main.yml 
├── files ├── handlers 
   └── main.yml 
├── meta 
   └── main.yml 
├── README.md ├── tasks 
   └── main.yml 
├── templates ├── tests    ├── inventory 
   └── test.yml 
└── vars 
    └── main.yml 
 
8 directories, 8 files
Not all the directories will be used in the example and we will show the use of some of them in the example.

Utilizing Roles in Playbook

This is the code of the playbook we have written for demo purpose. This code is of the playbook vivek_orchestrate.yml. We have defined the hosts: tomcat-node and called the two roles – install-tomcat and start-tomcat.
The problem statement is that we have a war which we need to deploy on a machine via Ansible.
--- 
- hosts: tomcat-node 
roles: 
   - {role: install-tomcat} 
   - {role: start-tomcat} 
Contents of our directory structure from where we are running the playbook.
Directory
$ ls 
ansible.cfg  hosts  roles  vivek_orchestrate.retry vivek_orchestrate.yml 
Roles
There is a tasks directory under each directory and it contains a main.yml. The main.yml contents of install-tomcat are −
--- 
#Install vivek artifacts 
-  
   block: 
      - name: Install Tomcat artifacts
         action: > 
            yum name = "demo-tomcat-1" state = present 
         register: Output 
          
   always: 
      - debug: 
         msg: 
            - "Install Tomcat artifacts task ended with message: {{Output}}" 
            - "Installed Tomcat artifacts - {{Output.changed}}" 
The contents of main.yml of the start tomcat are −
#Start Tomcat          
-  
   block: 
      - name: Start Tomcat 
      command: <path of tomcat>/bin/startup.sh" 
      register: output 
      become: true 
   
   always: 
      - debug: 
         msg: 
            - "Start Tomcat task ended with message: {{output}}" 
            - "Tomcat started - {{output.changed}}" 
The advantage of breaking the playbook into roles is that anyone who wants to use the Install tomcat feature can call the Install Tomcat role.

Breaking a Playbook into a Role

If not for the roles, the content of the main.yml of the respective role can be copied in the playbook yml file. But to have modularity, roles were created.
Any logical entity which can be reused as a reusable function, that entity can be moved to role. The example for this is shown above
Ran the command to run the playbook.
-vvv option for verbose output – verbose output 
$ cd vivek-playbook/
This is the command to run the playbook
$ sudo ansible-playbook -i hosts vivek_orchestrate.yml –vvv 
-----------------------------------------------------------------
----------------------------------------------------------------------- 

Output

The generated output is as seen on the screen −
Using /users/demo/vivek-playbook/ansible.cfg as config file.
PLAYBOOK: vivek_orchestrate.yml *********************************************************
*********************************************************** 
1 plays in vivek_orchestrate.yml 

PLAY [tomcat-node] **********************************************************************
******** ************************************************* 
 
TASK [Gathering Facts] *************************************************
****************************** ********************************************* 
Tuesday 21 November 2017  13:02:05 +0530 (0:00:00.056) 0:00:00.056 ****** 
Using module file /usr/lib/python2.7/sitepackages/ansible/modules/system/setup.py 
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: root 
<localhost> EXEC /bin/sh -c 'echo ~ && sleep 0' 
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo 
   /root/.ansible/tmp/ansible-tmp-1511249525.88-259535494116870 `" && 
   echo ansible-tmp-1511249525.88-259535494116870="` 
   echo /root/.ansible/tmp/ansibletmp-1511249525.88-259535494116870 `" ) && sleep 0' 
<localhost> PUT /tmp/tmpPEPrkd TO 
   /root/.ansible/tmp/ansible-tmp-1511249525.88259535494116870/setup.py 
<localhost> EXEC /bin/sh -c 'chmod u+x 
   /root/.ansible/tmp/ansible-tmp1511249525.88-259535494116870/ 
   /root/.ansible/tmp/ansible-tmp-1511249525.88259535494116870/setup.py && sleep 0' 
<localhost> EXEC /bin/sh -c '/usr/bin/python 
   /root/.ansible/tmp/ansible-tmp1511249525.88-259535494116870/setup.py; rm -rf 
   "/root/.ansible/tmp/ansible-tmp1511249525.88-259535494116870/" > /dev/null 2>&1 && sleep 0' 
ok: [server1] 
META: ran handlers 
 
TASK [install-tomcat : Install Tomcat artifacts] ***********************************
*************************************************************** 
task path: /users/demo/vivek-playbook/roles/install-tomcat/tasks/main.yml:5 
Tuesday 21 November 2017  13:02:07 +0530 (0:00:01.515)       0:00:01.572 ****** 
Using module file /usr/lib/python2.7/sitepackages/ansible/modules/packaging/os/yum.py 
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: root 
<localhost> EXEC /bin/sh -c 'echo ~ && sleep 0' 
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo 
   /root/.ansible/tmp/ansible-tmp-1511249527.34-40247177825302 `" && echo 
   ansibletmp-1511249527.34-40247177825302="` echo 
   /root/.ansible/tmp/ansible-tmp1511249527.34-40247177825302 `" ) && sleep 0' 
<localhost> PUT /tmp/tmpu83chg TO 
   /root/.ansible/tmp/ansible-tmp-1511249527.3440247177825302/yum.py 
<localhost> EXEC /bin/sh -c 'chmod u+x 
   /root/.ansible/tmp/ansible-tmp1511249527.34-40247177825302/ 
   /root/.ansible/tmp/ansible-tmp-1511249527.3440247177825302/yum.py && sleep 0' 
<localhost> EXEC /bin/sh -c '/usr/bin/python 
   /root/.ansible/tmp/ansible-tmp1511249527.34-40247177825302/yum.py; rm -rf 
   "/root/.ansible/tmp/ansible-tmp1511249527.34-40247177825302/" > /dev/null 2>
   &1 && sleep 0' 
changed: [server1] => { 
   "changed": true, 
   "invocation": { 
      "module_args": { 
         "conf_file": null, 
         "disable_gpg_check": false, 
         "disablerepo": null, 
         "enablerepo": null, 
         "exclude": null, 
         "install_repoquery": true, 
         "installroot": "/", 
         "list": null, 
         "name": ["demo-tomcat-1"], 
         "skip_broken": false, 
         "state": "present", 
         "update_cache": false, 
         "validate_certs": true 
      } 
   }, 
   "msg": "", 
   "rc": 0, 
   "results": [ 
      "Loaded plugins: product-id, 
      search-disabled-repos, 
      subscriptionmanager\nThis system is not registered to Red Hat Subscription Management. 
      You can use subscription-manager to register.\nResolving Dependencies\n--> 
      Running transaction check\n---> 
      Package demo-tomcat-1.noarch 0:SNAPSHOT-1 will be installed\n--> Finished Dependency 
      Resolution\n\nDependencies Resolved\n
      \n================================================================================\n 
      Package Arch Version Repository         
      Size\n==================================================================\nInstalling:\n 
      demo-tomcat-1 noarch SNAPSHOT-1 demo-repo1 7.1 M\n\nTransaction 
      Summary\n==================================================================\nInstall  1 
      Package\n\nTotal download size: 7.1 M\nInstalled size: 7.9 M\nDownloading 
         packages:\nRunning transaction 
      check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n  Installing : 
      demotomcat-1-SNAPSHOT-1.noarch 1/1 \n  Verifying  : 
      demo-tomcat-1-SNAPSHOT-1.noarch 1/1 \n\nInstalled:\n  
      demo-tomcat-1.noarch 0:SNAPSHOT-1 \n\nComplete!\n" 
   ] 
} 
 
TASK [install-tomcat : debug] **********************************************************
*************************************************************************** 
task path: /users/demo/vivek-playbook/roles/install-tomcat/tasks/main.yml:11 
Tuesday 21 November 2017  13:02:13 +0530 (0:00:06.757) 0:00:08.329 ****** 
ok: [server1] => { 
   "changed": false, 
   "msg": [ 
      "Install Tomcat artifacts task ended with message: {
         u'msg': u'', u'changed': True, u'results': 
         [u'Loaded plugins: product-id, 
         search-disabledrepos, 
         subscription-manager\\nThis system is not registered to Red Hat Subscription Management. 
         You can use subscription-manager to register.\\nResolving Dependencies\\n--> 
         Running transaction check\\n---> 
         Package demo-tomcat-1.noarch 0:SNAPSHOT-1 will be installed\\n--> 
         Finished Dependency Resolution\\n
         \\nDependencies 
         Resolved\\n\\n==================================================================\\n 
         Package Arch Version Repository         
         Size\\n======================================================================== 
         =====\\nInstalling:\\n demo-tomcat-1 noarch SNAPSHOT-1 demo-repo1 7.1 M\\n\\nTransaction 
         Summary\\n=========================================================\\nInstall  1 
         Package\\n\\nTotal download size: 7.1 M\\nInstalled size: 7.9 M\\nDownloading 
            packages:\\nRunning 
         transaction check\\nRunning transaction test\\nTransaction test succeeded\\nRunning 
            transaction\\n  
         Installing : demo-tomcat-1-SNAPSHOT-1.noarch 1/1 \\n  Verifying  : 
         demo-tomcat-1-SNAPSHOT-1.noarch
         1/1 \\n\\nInstalled:\\n  demo-tomcat-1.noarch 0:SNAPSHOT-1  \\n\\nComplete!\\n'], u'rc': 0
      }", 
      "Installed Tomcat artifacts - True" 
   ] 
} 
 
TASK [install-tomcat : Clean DEMO environment] ****************************************
************************************************************ 
task path: /users/demo/vivek-playbook/roles/install-tomcat/tasks/main.yml:19 
Tuesday 21 November 2017  13:02:13 +0530 (0:00:00.057) 0:00:08.387 ****** 
[WARNING]: when statements should not include jinja2 templating delimiters such as {{ }} or 
   {% %}. Found: {{installationOutput.changed}} 
 
Using module file /usr/lib/python2.7/sitepackages/ansible/modules/files/file.py 
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: root 
<localhost> EXEC /bin/sh -c 'echo ~ && sleep 0' 
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo 
   /root/.ansible/tmp/ansible-tmp-1511249534.13-128345805983963 `" && echo 
   ansible-tmp-1511249534.13-128345805983963="` echo 
   /root/.ansible/tmp/ansibletmp-1511249534.13-128345805983963 `" ) && sleep 0' 
<localhost> PUT /tmp/tmp0aXel7 TO 
   /root/.ansible/tmp/ansible-tmp-1511249534.13128345805983963/file.py 
<localhost> EXEC /bin/sh -c 'chmod u+x 
   /root/.ansible/tmp/ansible-tmp1511249534.13-128345805983963/ 
   /root/.ansible/tmp/ansible-tmp-1511249534.13128345805983963/file.py && sleep 0' 
<localhost> EXEC /bin/sh -c '/usr/bin/python 
   /root/.ansible/tmp/ansible-tmp1511249534.13-128345805983963/file.py; rm -rf 
   "/root/.ansible/tmp/ansible-tmp1511249534.13-128345805983963/" > /dev/null 2>&1 
   && sleep 0' 
changed: [server1] => { 
   "changed": true, 
      "diff": { 
         "after": { 
            "path": "/users/demo/DEMO", 
            "state": "absent" 
      }, 
      "before": { 
         "path": "/users/demo/DEMO", 
         "state": "directory" 
      } 
   },

   "invocation": { 
      "module_args": { 
         "attributes": null, 
         "backup": null, 
         "content": null, 
         "delimiter": null, 
         "diff_peek": null, 
         "directory_mode": null, 
         "follow": false, 
         "force": false, 
         "group": null, 
         "mode": null, 
         "original_basename": null, 
         "owner": null, 
         "path": "/users/demo/DEMO", 
         "recurse": false, 
         "regexp": null, 
         "remote_src": null, 
         "selevel": null, 
         "serole": null, 
         "setype": null, 
         "seuser": null, 
         "src": null, 
         "state": "absent", 
         "unsafe_writes": null, 
         "validate": null 
      } 
   }, 
   "path": "/users/demo/DEMO", 
   "state": "absent" 
} 
 
TASK [install-tomcat : debug] ********************************************************
************************************************************* 
task path: /users/demo/vivek-playbook/roles/install-tomcat/tasks/main.yml:29 
Tuesday 21 November 2017  13:02:14 +0530 (0:00:00.257)       0:00:08.645 ****** 
ok: [server1] => {
   "changed": false, 
   "msg": [ 
      "Clean DEMO environment task ended with message:{u'diff': {u'after': {u'path': 
         u'/users/demo/DEMO', u'state': u'absent'}, 
      u'before': {u'path': u'/users/demo/DEMO', u'state': u'directory'}}, u'state': u'absent', 
         u'changed': True, u'path': u'/users/demo/DEMO'}", 
      "check value  :True" 
   ] 
} 
 
TASK [install-tomcat : Copy Tomcat to user home] *************************************
******************************************************** 
task path: /users/demo/vivek-playbook/roles/install-tomcat/tasks/main.yml:37 
Tuesday 21 November 2017  13:02:14 +0530 (0:00:00.055)       0:00:08.701 ****** 
[WARNING]: when statements should not include jinja2 templating delimiters such as {{ }} or 
   {% %}. Found: {{installationOutput.changed}} 
 
Using module file /usr/lib/python2.7/sitepackages/ansible/modules/commands/command.py 
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: root 
<localhost> EXEC /bin/sh -c 'echo ~ && sleep 0' 
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo 
   /root/.ansible/tmp/ansible-tmp-1511249534.43-41077200718443 `" && echo 
   ansibletmp-1511249534.43-41077200718443="` echo 
   /root/.ansible/tmp/ansible-tmp1511249534.43-41077200718443 `" ) && sleep 0' 
<localhost> PUT /tmp/tmp25deWs TO 
   /root/.ansible/tmp/ansible-tmp-1511249534.4341077200718443/command.py 
<localhost> EXEC /bin/sh -c 'chmod u+x 
   /root/.ansible/tmp/ansible-tmp1511249534.43-41077200718443/ 
   /root/.ansible/tmp/ansible-tmp-1511249534.4341077200718443/command.py && sleep 0' 
<localhost> EXEC /bin/sh -c '/usr/bin/python 
   /root/.ansible/tmp/ansible-tmp1511249534.43-41077200718443/command.py; rm -rf 
   "/root/.ansible/tmp/ansibletmp-1511249534.43-41077200718443/" > /dev/null 2>&1 
   && sleep 0' 
changed: [server1] => { 
   "changed": true, 
   "cmd": [ 
      "cp", 
      "-r", 
      "/opt/ansible/tomcat/demo", 
      "/users/demo/DEMO/" 
   ],
   "delta": "0:00:00.017923", 
   "end": "2017-11-21 13:02:14.547633", 
   "invocation": { 
      "module_args": { 
         "_raw_params": "cp -r /opt/ansible/tomcat/demo /users/demo/DEMO/", 
         "_uses_shell": false, 
         "chdir": null, 
         "creates": null, 
         "executable": null, 
         "removes": null, 
         "warn": true 
      } 
   }, 
   "rc": 0, 
   "start": "2017-11-21 13:02:14.529710", 
   "stderr": "", 
   "stderr_lines": [], 
   "stdout": "", 
   "stdout_lines": [] 
} 
 
TASK [install-tomcat : debug] ********************************************************
********************************************************** 
task path: /users/demo/vivek-playbook/roles/install-tomcat/tasks/main.yml:47 
Tuesday 21 November 2017  13:02:14 +0530 (0:00:00.260)       0:00:08.961 ****** 
ok: [server1] => { 
   "changed": false, 
   "msg": "Copy Tomcat to user home task ended with message {
      'stderr_lines': [], u'changed': True, u'end': u'2017-11-21 13:02:14.547633', u'stdout': 
      u'', u'cmd': [u'cp', u'-r', u'/opt/ansible/tomcat/demo', u'/users/demo/DEMO/'], u'rc': 0, 
      u'start': u'2017-11-21 13:02:14.529710', u'stderr': u'', u'delta': u'0:00:00.017923', 
      'stdout_lines': []}" 
} 
 
TASK [start-tomcat : Start Tomcat] **************************************************
********************************************************** 
task path: /users/demo/vivek-playbook/roles/start-tomcat/tasks/main.yml:5 
Tuesday 21 November 2017  13:02:14 +0530 (0:00:00.044)       0:00:09.006 ****** 
Using module file /usr/lib/python2.7/sitepackages/ansible/modules/commands/command.py 
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: root 
<localhost> EXEC /bin/sh -c 'echo ~ && sleep 0' 
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo 
   /root/.ansible/tmp/ansible-tmp-1511249534.63-46501211251197 `" && echo 
   ansibletmp-1511249534.63-46501211251197="` echo 
   /root/.ansible/tmp/ansible-tmp1511249534.63-46501211251197 `" ) && sleep 0' 
<localhost> PUT /tmp/tmp9f06MQ TO 
   /root/.ansible/tmp/ansible-tmp-1511249534.6346501211251197/command.py 
<localhost> EXEC /bin/sh -c 'chmod u+x 
   /root/.ansible/tmp/ansible-tmp1511249534.63-46501211251197/ 
   /root/.ansible/tmp/ansible-tmp-1511249534.6346501211251197/command.py && sleep 0' 
<localhost> EXEC /bin/sh -c '/usr/bin/python 
   /root/.ansible/tmp/ansible-tmp1511249534.63-46501211251197/command.py; rm -rf 
   "/root/.ansible/tmp/ansibletmp-1511249534.63-46501211251197/" > /dev/null 2>&1 
   && sleep 0' 
changed: [server1] => { 
   "changed": true, 
   "cmd": [ "/users/demo/DEMO/bin/startup.sh" ], 
   "delta": "0:00:00.020024", 
   "end": "2017-11-21 13:02:14.741649", 
   "invocation": { 
      "module_args": { 
         "_raw_params": "/users/demo/DEMO/bin/startup.sh", 
         "_uses_shell": false, 
         "chdir": null, 
         "creates": null, 
         "executable": null, 
         "removes": null, 
         "warn": true 
      } 
   }, 
   "rc": 0, 
   "start": "2017-11-21 13:02:14.721625", 
   "stderr": "", 
   "stderr_lines": [], 
   "stdout": "Tomcat started.", 
   "stdout_lines": [ "Tomcat started." ] 
} 
 
TASK [start-tomcat : debug] *************************************************
********************************************************************** 
task path: /users/demo/vivek-playbook/roles/start-tomcat/tasks/main.yml:10 
Tuesday 21 November 2017  13:02:14 +0530 (0:00:00.150)       0:00:09.156 ****** 
ok: [server1] => { 
   "changed": false, 
   "msg": [ 
      "Start Tomcat task ended with message: {'
         stderr_lines': [], u'changed': True, u'end': u'2017-11-21 13:02:14.741649', u'stdout': 
         u'Tomcat started.', u'cmd': [u'/users/demo/DEMO/bin/startup.sh'], u'rc': 0, u'start': 
         u'2017-11-21 13:02:14.721625', u'stderr': u'', u'delta': u'0:00:00.020024', 
         'stdout_lines': [u'Tomcat started.']}", 
      "Tomcat started - True" 
   ] 
} 
META: ran handlers 
META: ran handlers 
 
PLAY RECAP ******************************************************************************* 
********************************************************* 
server1  : ok = 9    changed = 4    unreachable = 0    failed = 0 
 
Tuesday 21 November 2017  13:02:14 +0530 (0:00:00.042)       0:00:09.198 ****** 
=============================================================================== 
install-tomcat : Install Tomcat artifacts ------------------------------- 6.76s 
/users/demo/vivek-playbook/roles/install-tomcat/tasks/main.yml:5 -------------- 
Gathering Facts --------------------------------------------------------- 1.52s 
 ------------------------------------------------------------------------------ 
install-tomcat : Copy Tomcat to user home ------------------------------- 0.26s 
/users/demo/vivek-playbook/roles/install-tomcat/tasks/main.yml:37 ------------- 

install-tomcat : Clean DEMO environment --------------------------------- 0.26s 
/users/demo/vivek-playbook/roles/install-tomcat/tasks/main.yml:19 ------------- 

start-tomcat : Start Tomcat --------------------------------------------- 0.15s 
/users/demo/vivek-playbook/roles/start-tomcat/tasks/main.yml:5 ----------------

install-tomcat : debug -------------------------------------------------- 0.06s 
/users/demo/vivek-playbook/roles/install-tomcat/tasks/main.yml:11 ------------- 

install-tomcat : debug -------------------------------------------------- 0.06s 
/users/demo/vivek-playbook/roles/install-tomcat/tasks/main.yml:29 ------------- 

install-tomcat : debug -------------------------------------------------- 0.04s 
/users/demo/vivek-playbook/roles/install-tomcat/tasks/main.yml:47 ------------- 

start-tomcat : debug ---------------------------------------------------- 0.04s 
/users/demo/vivek-playbook/roles/start-tomcat/tasks/main.yml:10 --------------- 
Hit the following URL and you will be directed to a page as shown below − http://10.76.0.134:11677/HelloWorld/HelloWorld
Hello World
The deployed war just has a servlet which displays “Hello World”. The detailed output shows the time taken by each and every task because of the entry added in ansible.cfg file −
[defaults] 
callback_whitelist = profile_tasks 

Friday, January 3, 2020

Getting started writing your first playbook

What is a Playbook?

Playbooks are essentially sets of instructions (plays) that you send to run on a single target or groups of targets (hosts). Think about the instructions you get for assembling an appliance or furniture. The manufacturer includes instructions so you can put the parts together in the correct order. When followed in order, the furniture looks like what was purchased.
That's basically how a playbook works.
Modules
The Playbook we're building will install a web server on a target RHEL/CentOS 7 host, then write an index.html file based on a template file that will reside with the final Playbook. You'll be able to take the example Playbook and additional files from this blog and test it out for yourself. While going over the example Playbook, we'll explain the modules that are used.
Authors
The author adds instructions for the modules to run, often with additional values (arguments, locations, etc.). The target host has modules run against it in the order the Playbook lays out (with includes or other additional files). The host's state is changed (or not) based on the results of the module running, which Ansible and Tower displays in output.

Running Playbooks

Keeping that in mind, you're still going to need to understand a few things about running Playbooks. With the furniture analogy, a Playbook is shorthand to tell the modules to perform a task. You must understand the following to run your Playbook successfully:
1. The target
Because the Playbooks are providing direction and interactivity with the modules, Ansible assumes you know how to do what you're trying to do and automates it. That's why Playbooks are like instructions or directions - you're telling the automated parts how you want the task configured. You'll still need to understand the target you're running the Playbook against.
2. The tasks
If part of the Playbook needs to start the web server, you're going to need to know how that's done so you know to use the service module and start the web server by name. If the Playbook is installing software, then you have to know how installation is done on the target. You're also still going to need to understand the basics of the tasks being performed. Does the software you're installing have a configuration setup to it? Are there multiple steps that require conditions or argument values? If there are variables that are being passed, these will all need to be understood by those constructing a Playbook as well.

Example Playbook

We'll share an example Playbook with a simple play to demonstrate what I've explained. The target host will be a RHEL/CentOS 7 base install. There will be a web server installed (NGINX) and then an index.html file will be created in the default webroot. After the install and file tasks are completed the service will be started.
*Note to run this example Playbook with Ansible Tower your inventory and credentials must be configured. 
Playbooks start with the YAML three dashes (---) followed by:
Name: good for keeping the Playbooks readable

Hosts: identifies the target for Ansible to run against

Become statements: true statement is included here to ensure nginx installs without a problem (it's not always required)
---
- name: Install nginx
  hosts: host.name.ip
  become: true
On the same indent level as the three prior statements will go the tasks: statement, after which any plays are listed another indent deeper (per YAML nesting). There are two tasks listed but both are using the Yum module. The first Yum task is adding the epel-release repo so that nginx can be installed. Once epel is present Yum is used to install the nginx package.
The state: present statement lets Ansible check the state on the target first before performing any further action. In both cases if the repo or package is already present, Ansible knows it doesn't have to do any more for this task and continues.
tasks:
  - name: Add epel-release repo
    yum:
      name: epel-release
      state: present

  - name: Install nginx
    yum:
      name: nginx
      state: present
The default install page for nginx is fine if you want to test that nginx installed properly, but you might have a basic html file that you'd like to have as your confirmation. For simplicity, the template index file is in the same directory I'll run the Playbook from for the example. The destination is simply the default for nginix with no configured sites.
  - name: Insert Index Page
    template:
      src: index.html
      dest: /usr/share/nginx/html/index.html
The last thing the Playbook will do is make sure that the nginx service has been started (and if not, start it).
- name: Start NGiNX
    service:
      name: nginx
      state: started
The entire Playbook is about the same length as the introduction paragraph:
---
- name: Install nginx
  hosts: host.name.ip
  become: true

  tasks:
  - name: Add epel-release repo
    yum:
      name: epel-release
      state: present

  - name: Install nginx
    yum:
      name: nginx
      state: present

  - name: Insert Index Page
    template:
      src: index.html
      dest: /usr/share/nginx/html/index.html

  - name: Start NGiNX
    service:
      name: nginx
      state: started

Thursday, January 2, 2020

Ansible Installation, Playbook, Roles, Commands

What is Ansible?

Ansible is an automation and orchestration tool popular for its simplicity of installation, ease of use in what concerns the connectivity to clients, its lack of agent for ansible clients and the multitude of skills.
Ansible functions by connecting via SSH to the clients, so it doesn't need a special agent on the client-side, and by pushing modules to the clients. The modules are then executed locally, on the client-side, and the output is pushed back to the Ansible server.
Since it uses SSH, it can very easily connect to clients using SSH-Keys, simplifying though the whole process. Client details, like hostnames or IP addresses and SSH ports, are stored in files called inventory files. Once you have created an inventory file and populated it, ansible can use it.

Why use Ansible?

Here are some important pros/benefits of using Ansible
  • One of the most significant advantages of Ansible is that it is free to use by everyone.
  • It does not need any special system administrator skills to install and use Ansible, and the official documentation is very comprehensive.
  • Its modularity regarding plugins, modules, inventories, and playbooks make Ansible the perfect companion to orchestrate large environments.
  • Ansible is very lightweight and consistent, and no constraints regarding the operating system or underlying hardware are present.
  • It is also very secure due to its agentless capabilities and due to the use of OpenSSH security features.
  • Another advantage that encourages the adoption of Ansible is its smooth learning curve determined by the comprehensive documentation and easy to learn structure and configuration.

Important terms used in Ansible

  • Ansible server:

    The machine where Ansible is installed and from which all tasks and playbooks will be ran
  • Module:

    Basically, a module is a command or set of similar commands meant to be executed on the client-side
  • Task:

    A task is a section that consists of a single procedure to be completed
  • Role:

    A way of organizing tasks and related files to be later called in a playbook
  • Fact:

    Information fetched from the client system from the global variables with the gather-facts operation
  • Inventory:

    File containing data about the ansible client servers. Defined in later examples as hosts file
  • Play:

    Execution of a playbook
  • Handler:

    Task which is called only if a notifier is present
  • Notifier:

    Section attributed to a task which calls a handler if the output is changed
  • Tag:

    Name set to a task which can be used later on to issue just that specific task or group of tasks.

Ansible Installation in Linux

Once you have compared and weighed your options and decided to go for Ansible, the next step is to have it installed on your system. We will go through the steps of installation in different Linux distributions, the most popular ones, in the next small tutorial.

Install Ansible on Centos/RedHat systems

Step 1) Install EPEL repo
[root@ansible-server ~]# sudo yum install epel-release
Step 2) Install ansible package
[root@ansible-server ~]# sudo  yum install -y ansible

Install ansible on Ubuntu/Debian systems

Step 1) Perform an update to the packages
$ sudo apt update
Step 2) Install the software-properties-common package
$ sudo apt install software-properties-common
Step 3) Install ansible personal package archive
$ sudo apt-add-repository ppa:ansible/ansible
Step 4) Install ansible
$ sudo apt update
$ sudo apt install ansible

Ansible ad-hoc commands

One of the simplest ways Ansible can be used is by using ad-hoc commands. These can be used when you want to issue some commands on a server or a bunch of servers. Ad-hoc commands are not stored for future uses but represent a fast way to interact with the desired servers.
For this tutorial, a simple two servers hosts file will be configured, containing host1 and host2.
You can make sure that the hosts are accessible from the ansible server by issuing a ping command on all hosts.
[root@ansible-server test_ansible]# ansible -i hosts all -m ping
host1 | SUCCESS => {
    "changed": false,
    "ping": "pong"
}
host2 | SUCCESS => {
    "changed": false,
    "ping": "pong"
}

Wednesday, January 1, 2020

Container security

Secure your container environment on GCP, GKE, or Anthos.



Overview

Containerization allows development teams to move fast, deploy software efficiently, and operate at an unprecedented scale. As enterprises create more containerized workloads, security must be integrated at each stage of the build-and-deploy life cycle. Learn how to secure any container environment you run on GCP—whether that's Google Kubernetes Engine or Anthos—in three critical areas.

Infrastructure security

Infrastructure security means that your container management platform provides the right security features. Kubernetes includes security features to protect your identities, secrets, and network, and Google Kubernetes Engine uses native GCP functionality—like Cloud IAM, Cloud Audit Logging, and Virtual Private Clouds—and GKE-specific features like application layer secrets encryption and workload identity to bring the best of Google security to your workloads.

Software supply chain

Securing the software supply chain means that container images are safe to deploy. This is how you make sure your container images are vulnerability free and that the images you build aren't modified before they're deployed.

Runtime security

Runtime security allows you to identify a container acting maliciously in production and take action to protect your workload.

Running containers allows you to adopt a fundamentally different security model



Simpler patch management and immutability

Simpler patch management and immutability

Containers are meant to be immutable, so you deploy a new image in order to make changes. You can simplify patch management by rebuilding your images regularly, so the patch is picked up the next time a container is deployed. Get the full picture of your environment with regular image security reviews.
Smaller surface of attack

Smaller surface of attack

Containers are meant to run on a much smaller host OS than for a VM, as more is packaged into the application directly. This minimal host OS reduces the potential surface of attack for your workload.
Resource and workload isolation

Resource and workload isolation

Containers provide an easy way to isolate resources, such as storage volumes, to certain processes using cgroups and namespaces. With technologies like GKE Sandbox, you can logically isolate workloads in a sub-VM sandbox, separate from other applications.


Infrastructure security

Container infrastructure security is about ensuring that your developers have the tools they need to securely build containerized services. These capabilities are typically built into the container orchestrator, like Kubernetes. If you use Google Kubernetes Engine, this functionality is surfaced natively, in addition to other features of Google Cloud.


Identity and authorization

On Google Kubernetes Engine, use Cloud IAM to manage access to your projects and role-based access control (RBAC) to manage access to your clusters and namespaces.

Audit logging

In Kubernetes, API audit logs are automatically captured. On Google Kubernetes Engine, Cloud Audit Logging records API audit logs automatically for you.

Networking

On Google Kubernetes Engine, create a network policy to manage pod-to-pod communications in your cluster. Use private clusters for private IPs and include Google Kubernetes Engine resources in a shared VPC.

Compliance

Google Kubernetes Engine features many compliance certifications, including ISO 27001, ISO 27017, ISO 27108, HIPAA, and PCI-DSS.

Minimal host OS

Google Kubernetes Engine uses Container-Optimized OS (COS) by default, an OS purpose-built and optimized for running containers. COS is maintained by Google in open source.

Automatically upgraded components

On GKE, masters are automatically patched to the latest Kubernetes version, and you can use node auto-upgrade to keep your security up to date by automatically applying the latest patches for your nodes.

Customer-managed encryption keys

Users in regulated industries may need to be in control of the keys used to encrypt data stored in GKE. With customer-managed encryption keys, you can pick a key from Cloud KMS to protect your GKE persistent disk.

Application layer secrets encryption

By default, Kubernetes secrets are stored in plaintext. GKE encrypts these secrets on disk and monitors this data for insider access. But that alone might not be enough to protect those secrets from a malicious application in your environment. Application layer secrets encryption protects secrets with envelope encryption, with a key you manage in Cloud KMS.

Workload Identity

Your containerized application probably needs to connect to other services, like a database, to fulfill its duties. To do so, your application first needs to authenticate to them. Workload Identity uses a Google-managed service account to share credentials for authentication, following the principles of least privilege for application authentication.

Managed SSL certs

In GKE, HTTPS load balancers need to be associated with an SSL certificate. You can obtain, manage, and renew these certificates yourself, or have Google automatically obtain, manage and renew those certificates for you—saving you the burden of renewing (or forgetting to renew) them yourself.

Networking terms in basic level

Basics of Computer Networking Open system: A system which is connected to the network and is ready for communication. Closed system...