How Hiera Hashes Can Help Reduce Code Complexity

Code complexity is a problem everywhere. In this blog, we break down an example of using Hiera hashes to reduce Puppet code complexity. 

Table of Contents:

• What Is Code Complexity?
• Why Use Hiera to Reduce Code Complexity?
• The Problem: Increasing Code Complexity
• How Hiera Reduced Code Complexity
• Using a Hash Within a Hash
• Reduce Code Complexity With Hiera Today

What Is Code Complexity?

Code complexity is exactly as it sounds: how complex code is. Code complexity tends to increase over time, as more changes are made to code. Reducing code complexity is important for the (re)usability of code. 

Why Use Hiera to Reduce Code Complexity?

As part of our effort to standardize our Puppet code design, we have moved as much of the data layer as possible to Hiera. This allows our code to be utilized by different parts of business rather than just our team.

The product that I manage the infrastructure for is used not only by us for testing purposes but also by our users to provide services to our clients. That means when we create new Puppet classes, we have to make them as flexible as possible.

Moving the data layer to Hiera means that we only have to understand a simple yaml file when building a new environment. This allows for great code reuse and helps explain to others not familiar with our Puppet code where the settings are.

The Problem: Increasing Code Complexity

As part of our migration to cloud-based services, we were tasked with migrating and merging our file servers to a single central point. We were keen to utilize Puppet to configure shares and their folders correctly up front, thus preventing unstructured top-level folders from being applied with hard-to-manage security configurations.

🛑 Don't stop there: learn how to simplify AWS Automation with our free guide. Click here to read more👈

Basically, if someone requests a new top-level folder, we ensure the group access controls, including which groups have read, write, or full access and how those controls are inherited.

To reduce the number of code changes that we needed to make in the lifetime of the code, we also wanted to separate the data from the code so that the implementation could be configurable from the data layer. We didn’t want to have to add another bit of Puppet code for each new folder required on our file server.

How Hiera Reduced Code Complexity

Hiera was the clear solution for our problem because it provides you with an entry point to store data that you can then read with Puppet code. The values are also written in yaml — a standard which other non-Puppet developers understand.

Our idea was to store data in a format that we could then loop through, allowing us to write simple code in Puppet. One of the problems we came up against was lack of evidence of using Hiera in this way.

How could we structure the data in a way that wasn’t a mess and was easily readable for a non-Puppet developer? So, after a bit of playing about, we settled on something like this:

our_module::folders_hash:
  sales_hash:
    folder_name:          Sales
    share_name:           CentralStore$
    full_control_group:   domain\R-File-Sales-Full
    modify_group:         domain\R-File-Sales-RW
    read_group:           domain\R-File-Sales-RO
    inherit_permissions:   no
  hr_hash:
    folder_name:          HR
    share_name:           CentralStore$
    full_control_group:   domain\R-File-HR-Full
    modify_group:         domain\R-File-HR-RW
    read_group:           domain\R-File-HR-RO
    inherit_permissions:   no

Notice we have a hash within a hash. Adding more folders is a simple case of copying the inner hash and renaming the variable values. Nice and simple to maintain and very readable. The bonus of this approach is that we have clear documentation telling us which Security Group protects which folder. If we need to provide access to a folder, we can review the member of the group by looking up the name in this file — no need to access the folder directly. The next step was to look up the top-level value in Puppet:

$folders_hash = lookup('our_module::folders_hash') #looks up hashtable in {node}.yaml

Then collect the values from within the hash:

#iterates through each folder in hash and gets permissions
$folders_hash.each | $folder | {
    # $folder[0] is "folder_hash" as a string
    $folder_name = $folder[1][folder_name]
    $share_name = $folder[1][share_name]
    $full_control_group = $folder[1][full_control_group]
    $modify_group = $folder[1][modify_group]
    $read_group = $folder[1][read_group]
    $inherit_permissions = $folder[1][inherit_permissions]
 
    $folder_path = "${drive}:\\${share_name}\\${folder_name}"
 
    file { $folder_path:
      ensure  => directory, # ensures share folder exists
    }
    -> acl {     $folder_path:
      purge                      => true,
      permissions                => [
                                      { identity => $full_control_group, rights => ['full'] },
                                      { identity => $modify_group, rights => ['modify'] },
                                      { identity => $read_group, rights => ['read'] },
                                    ],
      inherit_parent_permissions => $inherit_permissions,
    }
 
  }
 
} 

As you can see from the code above, this approach allows for the code to directly read the settings defined in the hash by name, preventing the need for the settings to be in a particular order. For completeness, as an example lookup, the first iteration through the loop would provide the following values to these settings:

$folder_name = 'Sales'
$share_name = 'CentralStore$'
$full_control_group = 'domain\R-File-Sales-Full'
$modify_group = 'domain\R-File-Sales-RW'
$read_group = ''"domain\R-File-Sales-RO
$inherit_permissions = 'no'

Another Example: SQL Server

We were pleased with the resulting work from migrating our file servers and started to use the same concept in other areas of our code. We applied this same approach to managing the SQL Server instances on a production SQL Server. When a team requests another instance, we can simply extend our SQL hash in the Hiera yaml file to include the new instance.

our_module::sql_instance_hash:
  instance_001_hash:
    instance_name:        SQL_001
    sql_admins_group:     domain\R-SQL-Sales-SA
    data_drive_letter:    G
    log_drive_letter:     H
  instance_002_hash:
    instance_name:        SQL_002
    sql_admins_group:     internal\R-SQL-Sales-SA
    data_drive_letter:    I
    log_drive_letter:     J

This led us to our next revelation: Could we use this same approach for additionally requested features like SQL Replication?

We really didn’t want to have a setting for each possible feature so we tried using an array entry within the hash. This worked a treat and meant that we could be flexible on the features included with each instance:

our_module::sql_instance_hash:
  instance_001_hash:
    instance_name:        SQL_001
    sql_admins_group:     domain\R-SQL-Sales-SA
    data_drive_letter:    G
    log_drive_letter:     H
    features:
      - "SQLEngine"
      - "Replication"
  instance_002_hash:
    instance_name:        SQL_002
    sql_admins_group:     domain\R-SQL-Sales-SA
    data_drive_letter:    I
    log_drive_letter:     J
    features: 
      - "SQLEngine"

Accessing the array is like any other field:

 $features = $instance[1][features],

This will give you an array variable that you can then use as you normally would:

sqlserver_instance{ $sql_instance_name:
    features              => $features, #Expects an []
    security_mode         => 'SQL', #Allows SQL and Windows authentication.
    source                => '\\centralshare\Sql',
    install_switches      => {
    'TCPENABLED'          => 1,
    'SQLBACKUPDIR'        => $sql_backup_dir,
    'SQLTEMPDBDIR'        => $sql_tempdb_dir,
    'SQLTEMPDBLOGDIR'     => $sql_tempdb_logdir,
    'INSTALLSQLDATADIR'   => $sql_install_datadir,
    'INSTANCEDIR'         => $sql_instance_dir,
    'SQLUSERDBLOGDIR'     => $sql_userdb_logdir,
    'INSTALLSHAREDDIR'    => 'C:\\Program Files\\Microsoft SQL Server',
    'INSTALLSHAREDWOWDIR' => 'C:\\Program Files (x86)\\Microsoft SQL Server',
    }

Using a Hash within a Hash

What if you wanted to host multiple versions of the same application on the same server and needed settings for each. Then you could do something like this:

 our_module::my_app_hash_map:
  app1:
    name: "test"
    install_location: c:\test-apps\
    version: "5.1.193-Beta"
    use_prerelease: true
  app2:
    name: "uat"
    install_location: c:\uat-apps\
    version: "5.1.100"
    use_prerelease: false

These settings define how the application looks on the installation, but they do not allow you to configure internal settings, such as: Error retry attempts, Error Backoff time, Log Level, etc. These app settings could be vast and you would not want to drive these upfront like you would with the installation location for example. 

The great thing about hiera, is that it allows you use any of the standard Data Types with a hash, like so: 

our_module::my_app_hash_map:
  app1:
    name: "test"
    install_location: c:\test-apps\
    version: "5.1.193-Beta"
    use_prerelease: true
    app_settings:
      ErrorRetries: 5
      ErrorRetryIntervalSeconds: 10
      LogLevel: "VERBOSE"
  app2:
    name: "uat"
    install_location: c:\uat-apps\
    version: "5.1.100"
    use_prerelease: false
    app_settings:
      LogLevel: "DEBUG"

Then I could wrap this in Puppet code like this:

#configure settings in web.config
  $app_settings.each |$app_setting| {
    file_line {"web.config-${app_setting}":
      path  => "${install_dir}/web.config",
      line  => "<add key=\"App.Setting.${app_setting[0]}\" value=\"${app_setting[1]}\" />",
      match => "<add key=\"App.Setting.${app_setting[0]}\"*",
    }
  }

This keeps my code flexible to new settings in Hiera, without needing to change any of the logic. 

Reduce Code Complexity With Hiera Today

Using hashes in Hiera this way allows your Puppet code to be very data-driven, preventing the need to write the same code twice and providing flexibility for different configurations across environments. We have used this functionality to our benefit, allowing us to have single-service servers or multi-service servers based on the same Puppet code logic.

Not Using Puppet Yet?

Get started with Puppet Enterprise today.

🎉START MY TRIAL🎉

Editor’s note: This is a guest post from Darren Gipson, a DevOps engineer at Willis Towers Watson.