Troubleshooting

Sections

Use this guide to troubleshoot issues with your Continuous Delivery for Puppet Enterprise (PE) installation.

Restarting Continuous Delivery for PE

Continuous Delivery for PE is run in a managed Kubernetes cluster, so restarting the pod is an appropriate first step when troubleshooting.

To restart the pod, run:
kubectl rollout restart deployment cd4pe

For more on the kubectl rollout command, see the Kubernetes documentation.

Accessing logs

Because Continuous Delivery for PE is run in a managed Kubernetes cluster, the software's logs are accessed with the kubectl logs command.

To access the Continuous Delivery for PE logs, run:
kubectl logs deployment/cd4pe
To access your installation's PostgreSQL logs, run:
kubectl logs statefulset/postgres

Trace-level logging

Enable or disable trace-level logging by setting Enable trace logging in the Runtime configuration section of the Config page on the platform admin console.

PE component errors in logs

The logs include errors for both Continuous Delivery for PE and for the numerous PE components used by the software. It’s important to realize that an error in the Continuous Delivery for PE log may actually indicate an issue with Code Manager, r10k, or another component.

For example, this log output indicates a Code Manager issue:
Module Deployment failed for PEModuleDeploymentEnvironment[nodeGroupBranch=cd4pe_lab, 
nodeGroupId=a923c759-3aa3-43ce-968a-f1352691ca02, nodeGroupName=Lab environment, 
peCredentialsId=PuppetEnterpriseCredentialsId[domain=d3, name=lab-MoM], 
pipelineDestinationId=null, targetControlRepo=null, targetControlRepoBranchName=null, 
targetControlRepoHost=null, targetControlRepoId=null]. 
Puppet Code Deploy failure: Errors while deploying environment 'cd4pe_lab' (exit code: 1): 
ERROR -> Unable to determine current branches for Git source 'puppet' (/etc/puppetlabs/code-staging/environments) 
Original exception: malformed URL 'ssh://git@bitbucket.org:mycompany/control_lab.git' 
at /opt/puppetlabs/server/data/code-manager/worker-caches/deploy-pool-3/ssh---git@bitbucket.org-mycompany-control_lab.git

For help resolving issues with PE components, see the Puppet Enterprise troubleshooting documentation.

Error IDs in web UI error messages

Occasionally, error messages shown in the Continuous Delivery for PE web UI include an error ID and instructions to contact the site administrator. For example:
Screenshot of the web UI showing an error message instructing the user to "Contact the site administrator for support along with errorid"

Errors of this kind are shown without additional detail for security purposes. Users with root access to the Continuous Delivery for PE host system can search the log for the error ID to learn more.

Duplicate job logs after reinstall

A job's log is housed in object storage after the job is complete. If you reinstall Continuous Delivery for PE but reuse the same object storage without clearing it, you might notice logs for multiple jobs with the same job number, or job logs already present when a new job has just started.

To remove duplicate job logs and prevent their creation, make sure you clear both the object storage and the database when reinstalling Continuous Delivery for PE.

Name resolution

Continuous Delivery for PE uses CoreDNS for name resolution. Many can't reach and timeout connecting errors reported in the logs are actually DNS lookup failures.

To add DNS entries to CoreDNS, run:
kubectl -n kube-system edit configmaps coredns
The system will open the configmap file in an editor. Add a hosts stanza directly after the kubernetes stanza using the following format:
hosts /etc/hosts <DOMAIN> {
  <IP ADDRESS> <HOSTNAME> [ALIASES]
  fallthrough
}
Here's an example edited configmap with a hosts stanza added:
apiVersion: v1
data:
  Corefile: |
    .:53 {
        errors
        health {
           lameduck 5s
        }
        ready
        kubernetes cluster.local in-addr.arpa ip6.arpa {
           pods insecure
           fallthrough in-addr.arpa ip6.arpa
           ttl 30
        }
        hosts /etc/hosts puppetdebug.vlan {
          10.234.4.29 pe-201922-master.puppetdebug.vlan pe-201922-master
          fallthrough
        }
        prometheus :9153
        forward . /etc/resolv.conf
        cache 30
        loop
        reload
        loadbalance
    }
kind: ConfigMap
metadata:
  creationTimestamp: "2020-08-25T17:34:17Z"
  name: coredns
  namespace: kube-system
  resourceVersion: "10464"
  selfLink: /api/v1/namespaces/kube-system/configmaps/coredns
  uid: ba2907be-0067-4382-b103-fc248974719a

Looking up information about Continuous Delivery for PE

Use kubectl commands to access information about your Continuous Delivery for PE installation.

Look up the environment variables in use

To list the environment variables in use on your installation, run:

kubectl describe deployments.apps cd4pe
Note: For information on tuning your Continuous Delivery for PE instance by setting environment variables, including adjusting HTTP and job timeout periods, changing the size of LDAP server searches, or enabling Git repository caching, see Advanced Continuous Delivery for PE configuration.

Look up Continuous Delivery for PE version

To print the version of Continuous Delivery for PE running in your installation, run:
kubectl get deployment cd4pe -o jsonpath='{.spec.template.spec.containers[0].image}' ;  printf "\n"

Resetting the platform admin console password

As part of the installation process, the platform admin console generates a password for you. You can update this password to one of your choosing after installation.

To reset the platform admin console password, run:
kubectl -n default kots reset-pasword

The system will prompt you to enter a new password of your choosing.

If the command fails with an unknown command "kots" for "kubectl" error it's because /usr/local/bin is not in the path. To address this error, either update the path to include /usr/local/bin, or run:
/usr/local/bin/kubectl-kots reset-password default
How helpful was this page?

If you leave us your email, we may contact you regarding your feedback. For more information on how Puppet uses your personal information, see our privacy policy.

Puppet sites use proprietary and third-party cookies. By using our sites, you agree to our cookie policy.