A walkthrough of pdksync
What is pdksync?
The pdksync gem was built as an extension of Puppet Development Kit (PDK) to provide a simple and straightforward way to manage and update multiple different modules to match the latest templates in a single command.
Why was it created?
Having to manually update multiple modules is a chore that no one should have to endure. We were using modulesync to tackle the automation of this process, but it was not ideal for us. With the advent of PDK, we could build on a tool that would give us better template handling and allow us to add more functionality going forward. Paula Muir, Helen Campbell and I agreed that further implementation of modulesync was not the way forward, so we decided to come up with a new solution.
How it works
Running the pdksync gem creates a local copy of the target repository and runs the PDK update command against it, changing it so that it matches the updated PDK templates. Once this is complete, the updated files are moved to a separate branch. They are then staged and commited — with the retrieved template reference serving as a unique identifier for the branch and the commit.
At this point we had a “happy path” where we had automated the update of a single module. We then consolidated what we had by producing automated testing and adding a ‘constants.rb’ file that contained every hard coded variable that could be brought into the main ‘pdksync.rb’ file. We then added new functionality that pushes our local branch to the remote GitHub master and creates a pull request.
Automating the update of a single module is fine, but we wanted to automate the process of updating multiple modules at one time. We decided to utilise a simple yaml file, containing the name of each module we wanted to update.
We now have a gem that automatically runs a PDK update against any number of modules that have previously been converted. Back when we had to manually update a module it could take up to 30 minutes — depending on the complexity of the updates. Now, PDK update function reduces this to no more than five minutes. This time is reduced even further by the pdksync gem — with a module update and a pull request created in less than a minute. It is configurable via the managed_modules.yml file.
Despite accomplishing our goals, we still have numerous improvements planned — for example, the ability to convert, rather than update and passing in a PDK version.
Open source is close to our heart. If there are features you want, or if you would like to get involved, please reach out to us. The gem has been published as open source and is available on both GitHub and rubygems. We eagerly await your feedback.
David Swan is an associate software engineer at Puppet.