Monitoring data.fao.org

A session at Devopsdays Europe 2012, Rome

Saturday 6th October, 2012

10:15am to 10:45am (CET)

The UN Food and Agriculture Organization (FAO) is a sprawling organization that monitors all aspects of food, agriculture, forests, and fisheries worldwide. Its main product its diverse and deep datasets. Unfortunately, those datasets have too often been inaccessible outside of the originating department, let alone to the rest of the international community. Data.fao.org is an ambitious effort to provide the rest of the world with an API for FAO's datasets. Supporting this effort is an array of complicated web services and presentation layers.

However, all this work will work will be nought if this infrastructure doesn't remain "up" and accessible. I will present how we are using collectd, JMX, statsd, logstash, and Graphite to monitor the data.fao.org infrastructure. This has been done entirely with open-source tools, which while powerful, but are woefully lacking in documentation. This presentation will go into detail how we use these to monitor our infrastructure.

Some Snippets

  • How to make meaningful comparisons using the wildcards and the averageSeries, movingAverage, and timeShift functions in graphite
  • What the values from statsd actually mean
  • How to filter out distorting values
  • Why the 90th percentile matters
  • Using JMX everywhere w/out application slow down
  • How to fight bureaucracy and CYA w/ graphs

About the speaker

This person is speaking at this event.

Coverage of this session

Sign in to add slides, notes or videos to this session

Tell your friends!


Time 10:15am10:45am CET

Date Sat 6th October 2012

Short URL


View the schedule


See something wrong?

Report an issue with this session