Agent-server HTTPS communications

The Puppet agent and primary server communicate via mutually authenticated HTTPS using client certificates.

Access to each endpoint is controlled by auth.conf settings. For more information, see Puppet Server configuration files: auth.conf.

Persistent HTTP and HTTPS connections and Keep-Alive

When acting as an HTTPS client, Puppet reuses connections by sending Connection: Keep-Alive in HTTP requests. This reduces transport layer security (TLS) overhead, improving performance for runs with dozens of HTTPS requests.

You can configure the Keep-Alive duration using the http_keepalive_timeout setting, but it must be shorter than the maximum keepalive allowed by the primary server's web server.

Puppet caches HTTP connections and verified HTTPS connections. If you specify a custom HTTP connection class, Puppet does not cache the connection.

Puppet always requests that a connection is kept open, but the server can choose to close the connection by sending Connection: close in the HTTP response. If that occurs, Puppet does not cache the connection and starts a new connection for its next request.

For more information about the http_keepalive_timeout setting, see the Configuration reference.

For an example of a server disabling persistent connections, see the Apache documentation on KeepAlive.

The process of Agent-side checks and HTTPS requests during a single Puppet run.

  1. Check for keys and certificates:
    1. The agent downloads the CA (Certification Authority) bundle.
    2. If certificate revocation is enabled, the agent loads or downloads the Certificate Revocation List (CRL) bundle using the previous CA bundle to verify the connection.
    3. The agent loads or generates a private key. If the agent needs a certificate, it generates a Certificate Signing Request (CSR), including any dns_alt_names and csr_attributes, and submits the request using PUT /puppet-ca/v1/certificate_request/:certname.
    4. The agent attempts to download the signed certificate using GET /puppet-ca/v1/certificate/:certname.
      • If there is a conflict that must be resolved on the Puppet server, such as cleaning the old CSR or certificate, the agent sleeps for waitforcert seconds, or exits with 1 if waiting is not allowed, such as when running puppet agent -t.
        Tip: This can happen if the agent's SSL directory is deleted, as the Puppet server still has the valid, unrevoked certificate.
      • If the downloaded certificate fails verification, such as it does not match its private key, then Puppet discards the certificate. The agent sleeps for waitforcert seconds, or exits with 1 if waiting is not allowed, such as when running puppet agent -t.
  2. Request a node object and switch environments:

    • Do a GET request to /puppet/v3/node/<NAME> .

      • If the request is successful, read the environment from the node object. If the node object has an environment, use that environment instead of the one in the agent’s config file in all subsequent requests during this run.

      • If the request is unsuccessful, or if the node object had no environment set, use the environment from the agent’s config file.

  3. If pluginsync is enabled on the agent, fetch plugins from a file server mountpoint that scans the lib directory of every module:

    • Do a GET request to /puppet/v3/file_metadatas/plugins with recurse=true and links=manage.

    • Check whether any of the discovered plugins need to be downloaded. If so, do a GET request to /puppet/v3/file_content/plugins/<FILE> for each one.

  4. Request catalog while submitting facts:
    • Do a POST request to /puppet/v3/catalog/<NAME>, where the post data is all of the node’s facts encoded as JSON. Receive a compiled catalog in return.
      Note: Submitting facts isn't logically bound to requesting a catalog. For more information about facts, see Language: Facts and built-in variables.

  5. Make file source requests while applying the catalog:

    File resources can specify file contents as either a content or source attribute. Content attributes go into the catalog, and the agent needs no additional data. Source attributes put only references into the catalog and might require additional HTTPS requests.

    • If you are using the normal compiler, then for each file source, the agent makes a GET request to /puppet/v3/file_metadata/<SOMETHING> and compares the metadata to the state of the file on disk.

      • If it is in sync, it continues on to the next file source.

      • If it is out of sync, it does a GET request to /puppet/v3/file_content/<SOMETHING> for the content.

    • If you are using the static compiler, all file metadata is embedded in the catalog. For each file source, the agent compares the embedded metadata to the state of the file on disk.

      • If it is in sync, it continues on to the next file source.

      • If it is out of sync, it does a GET request to /puppet/v3/file_bucket_file/md5/<CHECKSUM> for the content.
        Note: Using a static compiler is more efficient with network traffic than using the normal (dynamic) compiler. Using the dynamic compiler is less efficient during catalog compilation. Large amounts of files, especially recursive directories, amplifies either issue.
  6. If report is enabled on the agent, submit the report:

    • Do a PUT request to /puppet/v3/report/<NAME>. The content of the PUT should be a Puppet report object in YAML format.