Facter Part 3: Caching and TTL

In facter part 2, we explored how to write custom facts in ruby along with testing and distribution. Since facter is executed every single time puppet runs, there are benefits to cache the information on the client and refresh based on some TTL. Looking ahead, both features are on our roadmap for facter 2.0, but it's not complicated to implement them for custom facts. To get started, here's a practical custom fact that retrieves geo information using api.hostip.info:

require 'facter'
require 'rubygems'
require 'rest_client'

geo_raw=(RestClient.post "http://api.hostip.info/get_html.php", :position=>"true")
geo_data=Hash[*geo_raw.scan(/(.*): (.*)n/).to_a.flatten]

geo_data.each do |key, val|
    Facter.add("geo_#{key}") { setcode { val } }
end

This fact generates some neat information about where I'm located at the moment:

geo_city => FAIRVIEW, OR
geo_country => UNITED STATES (US)
geo_ip => 66.239.37.190
geo_latitude => 45.5464
geo_longitude => -122.435

To implement caching and TTL, the values are serialized into yaml and the fact "freshness" is determined by the yaml file modify time:

require 'facter'
require 'rubygems'
require 'rest_client'
require 'time'
require 'yaml'

# facts cache directory/file on puppet agent:
facts_cache_dir='/etc/puppet/facts.d'
geo_cache_file=facts_cache_dir+'/geo.yaml'
# facts cache time to live in seconds (1 hour)
facts_cache_ttl=3600
# debug flag, for now not concerned about failed fact retrieval
debug=true

if File::exist?(geo_cache_file) then
  geo_cache=YAML.load_file(geo_cache_file)
  geo_data=geo_cache
  cache_time=File.mtime(geo_cache_file)
else   
  puts "Err: failed to load geo.yaml local cache" if debug
  geo_cache=nil
  cache_time=Time.at(0)
end

# retrieve new geo data if no local cache, or fact exceeds TTL
if ! geo_cache || (Time.now - cache_time) > facts_cache_ttl
 begin
    geo_raw=(RestClient.post "http://api.hostip.info/get_html.php", :position=>"true")
    geo_data=Hash[*geo_raw.scan(/(.*): (.*)n/).to_a.flatten]

    begin
      Dir.mkdir(facts_cache_dir) if !File::exists?(facts_cache_dir)
      File.open(geo_cache_file, 'w') do |out|
        YAML.dump( geo_data, out)
      end
    rescue
      puts "Err: failed to write geo.yaml cache file." if debug
    end
  rescue
    puts "Err: failed to obtain geo data." if debug
    geo_data=geo_cache
  end
end

if geo_data then
  geo_data.each do |key, val|
    Facter.add("geo_#{key}") { setcode { val } }
  end
end

The local geo.yaml file cache will provide the geo data in the event the remote system is unavailable, and the same technique can be used to mitigate performance impact for complex facts that require significant resource to parse and process. If we wish to set the TTL to run once on every system reboot, we can take advantage of uptime value:

facts_cache_ttl=Facter["uptime_seconds"].value()

If you expect a custom fact that might hang, wrapping the call with Ruby timeout and taking sensible actions will avoid facter hanging during information retrieval:

begin
  Timeout::timeout(10) { ... } # 10 second execution timeout
rescue Timeout::Error
  puts "Err: ... custom fact not loaded."
end

One of the benefits of this simple implementation is the custom facts can be forced to refresh by purging all files in /etc/puppet/facts.d directory, and it's fairly straightforward to extend this example to set a global custom fact TTL value using an environment variable. Hopefully this gives you some ideas and inspiration on writing your own facts. If you have any comments or suggestions, please feel free to leave them below.