Linux os monitoring using Telegraf

It has been almost year since I’ve run any kind of performance tests on Unix. Since recently i’ve been tasked on running one, I’ve decided to look up for new tools which does just that. After some looking around i’ve stumbled upon Telegraf. As they mention it’s The plugin-driven server agent for collecting & reporting metrics. So I would be using Telegraf to collect logs & send them to graphite to store and visualize.

Let’s go over the steps for installing and configuring the tool:

1 Telegraf Installation:

Fastest way is to use prepared packets (or if you have GO installed - install from source)

wget https://dl.influxdata.com/telegraf/nightlies/telegraf-nightly.i386.rpm &&\
rpm -i telegraf-nightly.i386.rpm

2 Grafana Installation:

Metrics can be stored in log file, or sent to a log server. I’ve ended up using Grafana log server.

I was using Docker version:

docker run -d \
--name graphite \
--restart=always \
-p 80:80 \
-p 2003-2004:2003-2004 \
-p 2023-2024:2023-2024 \
-p 8128:8125/udp \
-p 8126:8126 \
-v /opt/graphite/storage:/opt/graphite/storage \
graphiteapp/graphite-statsd

3 Generating config file for telegraf:

There are plenty of plugins that can be plugged into telegraf, but I was using only this guys:

Plugin name	Description	Configuration
cpu	Info about cpu Usage	percpu=false
filestat	Collect file size on App server log files	list of files
http_response	Get response time & code	url, method, body
mem	memory Info	none
net	network bytes sent/received	interface
procstat	Collects usage metrics per-process	process id, name

So we need to generate config file with this plugins on the collector machine, that would be used later to run agent:

telegraf --input-filter cpu:mem:filestat:http_response:net:procstat -outputfilter graphite config > telegraf.conf

Open generated file with your favorite editor and edit parameters.

See table above for example of list of things to change.

4 Run agents:

telegraf --config telegraf.conf &

5 Visualize

Login to Graphite and start creating graphs. You can copy-paste path’s from metrics page or use cli to get metrics from dashboard page.

Some notes on visualization:

It is possible to export & import dashboards from/to JSON. So you can add just few of dashboards and then export them to file and add others manually. (And have a backup)