Adam Delyea

What is Ansible?

Ansible is a configuration management, deployment, and general command execution framework. This framework is similar to Chef, or Puppet, except that it is extremely lightweight, using only python and ssh to get the job done. It also requires very little learning before you can start managing your servers.

Getting Things Installed

Okay, so you’re excited and want to start managing your servers, so lets get some stuff installed first.

$ git clone git://github.com/ansible/ansible.git
$ cd ./ansible
$ source ./hacking/env-setup

$ git clone git://github.com/ansible/ansible.git

$ cd ./ansible

$ source ./hacking/env-setup

Believe it or not, you’re more than 50% done already. Time to install some python modules:

sudo easy_install jinja2 pyyaml paramiko

1	sudo easy_install jinja2 pyyaml paramiko

The only other step you need to do now is to specify a server to manage.

$ echo "127.0.0.1" > ~/ansible_hosts
$ export ANSIBLE_HOSTS=~/ansible_hosts

1 2	$ echo "127.0.0.1" > ~/ansible_hosts $ export ANSIBLE_HOSTS=~/ansible_hosts

Okay, so that’s it, you’re ready to rock. Just to test things out:

$ ansible all -m ping --ask-pass

1	$ ansible all -m ping --ask-pass

You should get something that looks a little bit like this:

SSH password: 
127.0.0.1 | success >> {
    "changed": false, 
    "ping": "pong"
}

SSH password:

127.0.0.1 | success >> {

"changed": false,

"ping": "pong"

}

Some Other Neat Stuff

What is the external IP address of all of my servers?

$ ansible all -a "wget -qO - http://ifconfig.me/ip" --ask-pass

1	$ ansible all -a "wget -qO - http://ifconfig.me/ip" --ask-pass

SSH password: 
127.0.0.1 | success | rc=0 >>
1.2.3.4

SSH password:

127.0.0.1 | success | rc=0 >>

1.2.3.4

How about restarting all of your webservers?

$ ansible webservers -m service -a "name=httpd state=restarted"

1	$ ansible webservers -m service -a "name=httpd state=restarted"

There is plenty more that you can do with Ansible, this is just a quick getting started intro.

Some next steps would be to read through Ansible’s documentation, its pretty good.

Intro

This article is a very brief introduction to using Fluentd, an open source log collector.

Recently, I’ve been given the task of centralizing all of my companies log files, and decided that this was going to be a perfect opportunity to use Fluentd in a production environment.

Getting Everything Installed

Ok so first things first, you need to get Fluentd installed. There are many already provided ways to get this installed, but I’m using CentOS, so I’m going to be using the rpm:

curl -L http://toolbelt.treasure-data.com/sh/install-redhat.sh | sh

1	curl -L http://toolbelt.treasure-data.com/sh/install-redhat.sh \| sh

You can check out Fluentd’s own documentation if you are using something else.

Setting Up Configs

Fluentd puts all of the config files in /etc/td-agent/, and the one we’re looking at editing is /etc/td-agent/td-agent.conf

So, we’re going to need to specify a source for Fluentd to read from, which in this case is going to be your apache access log:

<source>
  type tail
  format apache
  path /var/log/httpd/access_log
  tag local.apache.access
</source>

type tail

format apache

path /var/log/httpd/access_log

tag local.apache.access

</source>

Breaking this down line by line:

type tail : tail is a source plugin that essentially tails whatever file you tell it to.
format apache : This tells Fluentd what format the log file is written in, so it can easily parse it. There are many built in parsers you can check out here.
path /var/log/httpd/access_log : The path to the log file you want to read.
tag local.apache.access : A tag is used for routing the messages.

Next, we need to specify a place to put all these messages:

<match local.apache.*>
    type file
    path /var/log/td-agent/td.apache
</match>

type file

path /var/log/td-agent/td.apache

</match>

Breaking it down:

match local.apache.* : This will grab all messages that match this tag. The tag can be written as a regular expression.
type file : This is telling Fluentd to use the file plugin to write messages.
path /var/log/td-agent/td.apache : The path to write to.

Test Away

So at this point, everything should be set up and ready to test out. For this test, I’m going to be using ab

ab -n 50 -c 10 http://localhost/

1	ab -n 50 -c 10 http://localhost/

Tada! If everything worked out, you should have some log data sitting in /var/log/httpd/access_log.*.log, where * is the current date.

2012-12-21T11:27:59-05:00	local.apache.access	{"host":"127.0.0.1","user":"-","method":"GET","path":"/","code":"403","size":"5039","referer":"-","agent":"ApacheBench/2.3"}
2012-12-21T11:27:59-05:00	local.apache.access	{"host":"127.0.0.1","user":"-","method":"GET","path":"/","code":"403","size":"5039","referer":"-","agent":"ApacheBench/2.3"}
2012-12-21T11:27:59-05:00	local.apache.access	{"host":"127.0.0.1","user":"-","method":"GET","path":"/","code":"403","size":"5039","referer":"-","agent":"ApacheBench/2.3"}

2012-12-21T11:27:59-05:00 local.apache.access {"host":"127.0.0.1","user":"-","method":"GET","path":"/","code":"403","size":"5039","referer":"-","agent":"ApacheBench/2.3"}

Final Thoughts

Fluentd was really easy to get everything set up and running with a very minimal footprint.

This particular example is not all that helpful in actually solving the centralized logging problem, but there are much more advanced configurations and plugins to use with Fluentd. I may cover this in another post later on, so stay tuned.

Adam Delyea Move along, nothing to see here...

Getting Started With Ansible