Testing storage changes

In order to keeps PuppetDB’s storage code nice and fast, we have developed some tooling to help test its performance.

The load testing tool

The load testing (aka benchmark) tool is useful for generating sample data and for loading the storage engine. See the docs for information on its use.

Fact storage benchmark

If you’re making changes to fact storage specifically, there is a special-purpose fact storage benchmark tool to check its performance in different scenarios.

Running

Run the tool with lein run fact-storage-benchmark localhost. Pass it the hostname of a machine where PuppetDB is running, without ssl, on port 8080. If you’re testing a significant amount of load, you should run the benchmark on a different machine from puppetdb.

This will automatically run test with different fact profiles and report the time it took to store all facts, in milliseconds. For focused testing or optimization purposes, you can select just one to run: lein run fact-storage-benchmark localhost medium.

The data generated by this tool is intentionally deterministic, so that results can be compared between subsequents runs. But it does not clear the database; if you want to evaluate performance against an empty database, you should clear it yourself first.

Adding more profiles

Fact storage benchmark profiles are configured in test-configs var in src/puppetlabs/puppetdb/cli/fact_storage_benchmark.clj. The profiles are run in order if none is specified at the command line.