Combining PowerShell, Bolt and Puppet Tasks – Part 2
In Part 1 of this blog series we created and ran a PowerShell script on our local computer, and then turned it into a Bolt Task. We then packaged the module so that others could use the task.
In Part 2, we are going to look at how you test the PowerShell script.
Why test tasks?
So the most obvious question is, why would I want to test my Puppet Tasks? When we first start writing tasks, testing isn't really at the front our minds. However, if you stop and look at the actions we took in Part 1, you can start to see that we were actually testing our code— it was just a manual process. For example, to add Task metadata, we did the following:
- Added the metadata information to the
update_history.json
file - Ran the
bolt task show
command - Verified that the information output from the command (Step 2) was the same as information we added in the metadata file (Step 1)
In software testing this is known as Arrange, Act, Assert. So we can think of what we did as;
- (Arrange) Added the metadata information
- (Act) Ran the bolt command
- (Assert) The output from the command is the same as the metadata information
So this means we're already doing some kind of testing, but it still doesn't answer the question of "Why should I test?". Testing our tasks means that as we add functionality or change things we can be sure that it still behaves the same way. And by using an automated testing tool (because let's be honest who likes manual testing anyway!) we can run the tests in our module CI pipeline.
What testing tools are out there?
While we could use a testing tool to create a Windows Virtual Machine, in the case of the WSUS Client Module this would be difficult and time consuming to do. So instead we can use Pester which is a testing and mocking framework for PowerShell. In fact it's one of only a few Open Source projects which is shipped in Windows itself!.
Note: this blog post won't go into detail how to write Pester tests. A quick search will help you, as well as some great talks by Jakub Jares and myself, Glenn Sarti.
Writing testable PowerShell
Great, so we can use Pester to test our PowerShell task file, but ... there is a problem. In order to test the script we need to import it. We do this by dot sourcing the test script. However this actually runs the script and outputs information.
Here’s the Source Code Link.
Also, because the script is written with the logic in the root, instead of in a function, we have no easy way to execute the script in our tests.
In short, the code I wrote may work, but it was not easily testable!
Wrapping the main function
Firstly we need to be able to separate loading the script and running the script. To do this we needed to move all of the logic into its own function. For example, if the script used to look like;
We would wrap all of this in a PowerShell function and then call it;
Notice how the function Invoke-ExecuteTask
just wraps around the old logic. It still does the same thing, just in a function.
Note: For those more advanced in PowerShell you may ask why I didn't use Cmdlet Binding in the function header. I could have easily defined this is an advanced function however I did not think it was necessary. The input validation already happens at the top of the script and as this is private function, and no user would be explicitly calling it.
Stopping execution
So now we could call the logic of the script in Pester, but we still had the problem of it actually running the script when we imported it. What we needed was a flag of some kind which could tell the script to execute or not when imported. There are a number of different types of flags; Setting environment variables or registry keys of files on disk. However in PowerShell the simplest method is to just have a script parameter.
Note: Using a script parameter was appropriate for the WSUS Client module but you may prefer to use something else
At the top of the script we added the NoOperation
parameter;
We also added a simple if statement at the bottom of the script which conditionally execute the script
This created a switch parameter called NoOperation
which would default to false
, that is, it would execute the script. By using . .\tasks\update_history.ps1 -NoOperation
we could tell the script to not execute and just import the functions for testing.
Note: For those more advanced in PowerShell you may ask why I didn't use the WhatIf parameter instead. The WhatIf
parameter is more geared towards a user interaction. While yes it could've been used, all we needed was just simple switch parameter. Also, noop
or NoOperation
, are common terms for Puppet users.
Writing Pester tests
Now that we could successfully import the PowerShell Task file, it was time to write the tests.
Writing simple tests
The first tests simply tested the enumeration functions. These functions converted the number style codes into their text version; for example an OperationResultCode
of 1 means the update is "In Progress"
So first we add the Pester standard PowerShell commands;
These commands:
- Calculate the name of the script being tested (also known as the System Under Test or
$sut
) based on the test file name - Import any shared helper functions (
spec_helper.ps1
). This blog post didn't add any, but in the future they may be used - Imports the script under test. Note the use of the new
-NoOperation
parameter
When then test each of the enumeration functions to ensure the the conversions of number to text are what we expect. For example the tests for the Convert-ToServerSelectionString function check the output for the numbers 0 to 3
Writing more tests
So now we had some simple tests written, and passing, we could finish off writing the rest of the tests. Fortunately with testing, we should be describing each of our tests in simple english. I decided that the following tests would be sufficient:
- should return empty JSON if no history
- should return a JSON array for a single element
- should not return detailed information when Detailed specified as false
- should return detailed information when Detailed specified as true
- should return only the maximum number of updates when specified
- should return a single update when UpdateID is specified
- should return matching updates when Title is specified
More testing issues
While writing the test it became apparent that the Invoke-ExecuteTask
function still wasn't easily testable. The function created aMicrosoft.Update.Session
COM object. This object was then used to query the system for update history. However this meant the testing could only query the existing system, and that we wouldn't be able to see the behaviour if there no updates available, or 1000 updates. What we needed to do was mock the response of the COM object so we could test the function properly.
Fortunately Pester provides a mocking feature, however the function needed to be modified so we could mock the response. So again we wrapped the logic in another function:
Previously we had
and after wrapping the object creation
Now we could mock the response from Get-UpdateSessionObject
to simulate any number or kind of updates with the testing helpers New-MockUpdateSession
and New-MockUpdate
.
For example, the should return empty JSON if no history
test mocks an update session with no updates, using the Pester Mock
function;
Failing tests
Running the Pester tests showed a failure. The should return a JSON array for a single element
was failing;
This failure turned out to be a valid. When the bolt task runs it should return a JSON Array, even for a single update. This turned out to be a peculiarity with PowerShell and piping objects. With a single object in the pipe, the JSON conversion just returns the object, whereas with two or more objects the JSON conversion returns an array.
In this case the fix was fairly simple. I manually added the opening and closing brackets to the string if there was only one object in the pipe! Running Pester again showed all tests passed!
Running tests automatically
Having a suite of tests to run was nice, but we really needed them to be run in a Continuous Integration (CI) pipeline. Fortunately the WSUS_Client module was already setup with an AppVeyor CI pipeline;
-
I added a small helper script to install Pester if it didn't exist and then actually run Pester
-
I modified the Rakefile which is used by the PDK and Ruby, to run testing tasks. This change called the helper script I created previously
-
I modified the AppVeyor configuration file to call the new
spec_pester
Rake task
Now whenever anyone raised a Pull Request, the pester test suite would be run!
Note: Why did I create a Rake task instead of calling the helper script directly? All the of PuppetLabs modules execute Rake tasks in the AppVeyor configuration file. While I could have hacked the configuration to run the script directly it would cause this module to become a unique configuration which is hard to manage over time.
Wrapping up
We modified the Bolt Task PowerShell script to be easily testable and then wrote a test suite. We then configured our CI tool, AppVeyor, to run the tests for new Pull Requests. Now we can more easily make changes to the task and be confident we don't break the existing behaviour
In Part 3, we'll look at how tasks, Puppet Enterprise and PowerShell can integrate together.
Learn more
- Read more about Bolt
- Using Bolt to detect Spectre and Meltdown
- For more examples on using Puppet tasks, check out the Puppet Tasks Hands-on-Lab GitHub repo