Tag Archives: CloudFormation

AWS UserData Multiple Run Framework Part IV (b)


Notice: The following post was originally published on another website. As the post is no longer accessible, it is being republished here on tommymaynard.com. The post was originally published on September 13, 2019.


This article began with Part a. Read that, if you haven’t already.

First off, the code-producing function has been renamed from New-AWSMultiRunTemplate to New-AWSUserDataMultipleRunTemplate. Sure, it’s longer in name, but it’s more clear (to me at least) when I see its name. Other small changes were the removal of unnecessary code comments. While these may have been in the New-AWSUserDataMultipleRunTemplate function itself, they were definitely changed in the produced code. Additionally, I’ve added a ProjectName parameter. The value, supplied to this parameter, is used throughout the generated code for naming purposes within the Registry. There are code changes to the produced code to run against Server 2016 and greater, as well. Therefore, it runs against Windows Server 2012 and 2012 R2 (and probably older), as well as Server 2016 and 2019. Hopefully, it will be extended further, but only AWS knows that for sure. Server 2012 R2 and earlier used EC2Config to configure a Windows instance, while 2016 and 2019 used EC2Launch.

If (test1) {
    {<statement list 1>}
} # End If.

As seen above, I tend to comment on my closing language construct brackets (see # End If.). These are now included in the produced code, both statically and dynamically. It’s a personal preference (that you’ll have to deal with if you use this free, code offering).

This is the new, New-AWSUserDataMultipleRunTemplate code producing function. This isn’t in the PowerShell Gallery, or anywhere else. It probably will never be, either. In fact, this is likely the last time I’ll prep it for public consumption. So again, this is the code you run to create the code you enter in UserData. We won’t discuss what’s in this code; however, we’ll run it further below and discuss that at a minimum.

Function New-AWSUserDataMultipleRunTemplate {
    [CmdletBinding()]
    Param (
        [Parameter(Mandatory)]
        [string]$ProjectName,

        [Parameter()]
        [ValidateRange(1,10)]
        [int]$CodeSectionCount = 2,

        [Parameter()]
        [ValidateSet('All','AllButLast')]
        [string]$EnableUserData = 'AllButLast',

        [Parameter()]
        [ValidateSet('All','AllButLast')]
        [string]$EnableRestart = 'AllButLast'
    )

    Begin { 
        #region Set Write-Verbose block location.
        $BlockLocation = '[BEGIN  ]'
        Write-Verbose -Message "$BlockLocation Entering the Begin block [Function: $($MyInvocation.MyCommand.Name)]."
        #endregion

        Write-Verbose -Message "$BlockLocation Storing the $ProjectName template's function to memory."
        Write-Verbose -Message "$BlockLocation Ensure the server where the code will reside does not already have the ""HKLM:\SOFTWARE\$ProjectName"" path."
        $TemplateFunction = @"
# >> Add function to memory.
Function Set-SystemForNextRun {
    Param (
        [string]`$CodeSectionComplete,
        [switch]`$ResetUserData,
        [switch]`$RestartInstance
    )
    If (`$CodeSectionComplete) {
        [System.Void](New-ItemProperty -Path 'HKLM:\SOFTWARE\$ProjectName' -Name "CodeSection`$CodeSectionComplete" -Value 'Complete')
    } # End If.
    If (`$ResetUserData) {
        try {
            `$Path = 'C:\Program Files\Amazon\Ec2ConfigService\Settings\config.xml'
            [xml]`$ConfigXml = Get-Content -Path `$Path -ErrorAction Stop
            (`$ConfigXml.Ec2ConfigurationSettings.Plugins.Plugin |
                Where-Object -Property Name -eq 'Ec2HandleUserData').State = 'Enabled'
            `$ConfigXml.Save(`$Path)
        } catch {
            C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -ExecutionPolicy Bypass -File C:\ProgramData\Amazon\EC2-Windows\Launch\Scripts\InitializeInstance.ps1 -Schedule
        } # End try-catch.
    } # End If.
    If (`$RestartInstance) {
        Restart-Computer -Force
    } # End If.
} # End Function: Set-SystemForNextRun.

# >> Create/Check for Registry Subkey.
If (-Not(Get-Item -Path 'HKLM:\SOFTWARE\$ProjectName' -ErrorAction SilentlyContinue)) {
    [System.Void](New-Item -Path 'HKLM:\SOFTWARE\' -Name '$ProjectName')
} # End If.

# >> Run user code/invoke Set-SystemForNextRun function.

"@
    } # End Begin.

    Process {
        #region Set Write-Verbose block location.
        $BlockLocation = '[PROCESS]'
        Write-Verbose -Message "$BlockLocation Entering the Process block [Function: $($MyInvocation.MyCommand.Name)]."
        #endregion

        Write-Verbose -Message "$BlockLocation Beginning to create the If-ElseIf code for the template."
        1..$CodeSectionCount | ForEach-Object {
            If ($_ -eq 1) {
                $Start = 'If'
                If ($CodeSectionCount -eq 1) {
                    $End = '# End If.'
                } # End If.
            } ElseIf ($_ -eq $CodeSectionCount -and $CodeSectionCount -eq 2) {
                $Start = 'ElseIf'; $End = '# End If-ElseIf.'
            } ElseIf ($_ -eq $CodeSectionCount -and $CodeSectionCount -ne 2) {
                $End = "# End If-ElseIf x$($CodeSectionCount - 1)."
            } Else {
                $Start = 'ElseIf'
            } # End If.

            If ($EnableUserData -eq 'All') {
                $UserData = '-ResetUserData '
            } ElseIf ($_ -eq $CodeSectionCount) {
                $UserData = $null
            } Else {
                $UserData = '-ResetUserData '
            } # End If.

            If ($EnableRestart -eq 'All') {
                $Restart = '-RestartInstance'
            } ElseIf ($_ -eq $CodeSectionCount) {
                $Restart = $null
            } Else {
                $Restart = '-RestartInstance'
            } # End If.

            $TemplateIfElseIf += @"
$Start (-Not((Get-ItemProperty -Path 'HKLM:\SOFTWARE\$ProjectName').CodeSection$_ -eq 'Complete')) {

    # CodeSection $_.

    Set-SystemForNextRun -CodeSectionComplete $_ $UserData$Restart
} $End
"@
        } # End ForEach-Object.
    } # End Process.

    End {
        #region Set Write-Verbose block location.
        $BlockLocation = '[END    ]'
        Write-Verbose -Message "$BlockLocation Entering the End block [Function: $($MyInvocation.MyCommand.Name)]."
        #endregion

        Write-Verbose -Message "$BlockLocation Creating the AWS UserData Mulitple Run Framework code with $CodeSectionCount code section$(If ($CodeSectionCount -gt 1) {'s'})."
        "$TemplateFunction$TemplateIfElseIf"
    } # End End.
} # End Function: New-AWSUserDataMultipleRunTemplate.

Copy and paste the above, New-AWSUserDataMultipleRunTemplate function into VS Code, or another preferred PowerShell development environment, if there is such a thing. You’re not still using the ISE, are you? Then, add the function to memory for use (in VS Code, that’s Ctrl + A to select all, then F8 to run the selection). Once the function is sitting in memory, we can use it to create our UserData code, as seen below. In this version—it doesn’t even really have a version number, which is weird—there’s now a mandatory ProjectName parameter. Keep this short and simple, and if it were me, I keep spaces and odd characters out of it. This is the value that will be used in the Windows Registry, and within the code that’s produced for UserData.

PS > New-AWSUserDataMultipleRunTemplate -ProjectName MistFit
# >> Add function to memory.
Function Set-SystemForNextRun {
    Param (
       [string]$CodeSectionComplete,
       [switch]$ResetUserData,
        [switch]$RestartInstance
    )
    If ($CodeSectionComplete) {
        [System.Void](New-ItemProperty -Path 'HKLM:\SOFTWARE\MistFit' -Name "CodeSection$CodeSectionComplete" -Value 'Complete')
    } # End If.
    If ($ResetUserData) {
        try {
            $Path = 'C:\Program Files\Amazon\Ec2ConfigService\Settings\config.xml'
            [xml]$ConfigXml = Get-Content -Path $Path -ErrorAction Stop
            ($ConfigXml.Ec2ConfigurationSettings.Plugins.Plugin |
                Where-Object -Property Name -eq 'Ec2HandleUserData').State = 'Enabled'
            $ConfigXml.Save($Path)
        } catch {
            C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -ExecutionPolicy Bypass -File C:\ProgramData\Amazon\EC2-Windows\Launch\Scripts\InitializeInstance.ps1 -Schedule
        } # End try-catch.
    } # End If.
    If ($RestartInstance) {
        Restart-Computer -Force
    } # End If.
} # End Function: Set-SystemForNextRun.

# >> Create/Check for Registry Subkey.
If (-Not(Get-Item -Path 'HKLM:\SOFTWARE\MistFit' -ErrorAction SilentlyContinue)) {
    [System.Void](New-Item -Path 'HKLM:\SOFTWARE\' -Name 'MistFit')
} # End If.

# >> Run user code/invoke Set-SystemForNextRun function.
If (-Not((Get-ItemProperty -Path 'HKLM:\SOFTWARE\MistFit').CodeSection1 -eq 'Complete')) {

    # CodeSection 1.

Set-SystemForNextRun -CodeSectionComplete 1 -ResetUserData -RestartInstance
} ElseIf (-Not((Get-ItemProperty -Path 'HKLM:\SOFTWARE\MistFit').CodeSection2 -eq 'Complete')) {

    # CodeSection 2.

Set-SystemForNextRun -CodeSectionComplete 2
} # End If-ElseIf.
PS C:>

By default, it still creates two code sections, as can be seen just above in the If-ElseIf statement. It can create just one, although that makes less sense for something that can provide multiple opportunities for configuration between restarts. Even if you only need one, this may still be the framework for you. Maybe you need a restart after a single configuration pass. It will do up to 10 code sections if, for some reason, you need that many opportunities to configure a single instance. Not me. I’ve only ever needed three or four total. The New-AWSUserDataMultipleRunTemplate function still includes the EnableUserData and EnableRestart switch parameters. Both have the default parameter value AllButLast, however, both parameters can accept All as the value, too. If this is used for the EnableRestart switch parameter, the EC2 instance will restart after the last time it’s configured (the last code section). In my experience, it’s not always necessary to restart an instance after its final configuration, but this would allow for the time it’s needed. Once you get to know the produced code well, you can manually edit it if necessary.

The produced code has three definitive sections. We’ll start with the below, third section. By default, the New-AWSUserDataMultipleRunTemplate function creates two code sections within the third section. Read that a few times; it’s confusing. It’s like this: The produced code consists of three sections (they each start with # >>). Inside the third one, is where you can have multiple code sections, where user code, that someone else provides, maybe you, is executed against an instance. Notice that there are two comments inside our If-ElseIf statement. On the first pass, it’ll run whatever the code that the user enters to replace “# CodeSection 1.” After the UserData is enabled and the instance is restarted, it’ll run whatever code is entered to replace “# CodeSection 2.” We’ll see more about how it does this shortly.

# >> Run user code/invoke Set-SystemForNextRun function.
If (-Not((Get-ItemProperty -Path 'HKLM:\SOFTWARE\MistFit').CodeSection1 -eq 'Complete')) {

    # CodeSection 1.

    Set-SystemForNextRun -CodeSectionComplete 1 -ResetUserData -RestartInstance
} ElseIf (-Not((Get-ItemProperty -Path 'HKLM:\SOFTWARE\MistFit').CodeSection2 -eq 'Complete')) {

    # CodeSection 2.

    Set-SystemForNextRun -CodeSectionComplete 2
} # End If-ElseIf.

Now that we’ve spent some time with the third section of our produced code, let’s move upward and focus on the middle, or second section, of the code that’s been produced. Here’s that second section, now. It’s real simple. If a specific Windows Registry Subkey doesn’t exist, it’s created. The need to create this will only happen once (the first run). Every other time this If statement fires, it’ll be false and therefore, it won’t attempt to create (something that’s already been created).

# >> Create/Check for Registry Subkey.
If (-Not(Get-Item -Path 'HKLM:\SOFTWARE\MistFit' -ErrorAction SilentlyContinue)) {
    [System.Void](New-Item -Path 'HKLM:\SOFTWARE\' -Name 'MistFit')
} # End If.

And, here’s the first section, last. This function, Set-SystemForNextRun, is placed into memory after this portion of the UserData is executed. It’s invoked by the third section. If you go back up to where we discussed the third section, you’ll see where.

# >> Add function to memory.
Function Set-SystemForNextRun {
    Param (
        [string]$CodeSectionComplete,
        [switch]$ResetUserData,
        [switch]$RestartInstance
    )
    If ($CodeSectionComplete) {
        [System.Void](New-ItemProperty -Path 'HKLM:\SOFTWARE\MistFit' -Name "CodeSection$CodeSectionComplete" -Value 'Complete')
    } # End If.
    If ($ResetUserData) {
        try {
            $Path = 'C:\Program Files\Amazon\Ec2ConfigService\Settings\config.xml'
            [xml]$ConfigXml = Get-Content -Path $Path -ErrorAction Stop
            ($ConfigXml.Ec2ConfigurationSettings.Plugins.Plugin |
                Where-Object -Property Name -eq 'Ec2HandleUserData').State = 'Enabled'
            $ConfigXml.Save($Path)
        } catch {
            C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -ExecutionPolicy Bypass -File C:\ProgramData\Amazon\EC2-Windows\Launch\Scripts\InitializeInstance.ps1 -Schedule
        } # End try-catch.
    } # End If.
    If ($RestartInstance) {
        Restart-Computer -Force
    } # End If.
} # End Function: Set-SystemForNextRun.

In the first code section of the third overall section, Set-SystemForNextRun was first invoked with all three possible parameters. The first parameter is sent in a numeric 1 to the CodeSectionComplete parameter. This created a new Registry value in the “HKLM:\SOFTWARE\MistFit” path. It coerced the number into a string, creating a Registry value named CodeSection1 that had a string value of Complete. These values are how the code knows what’s been done before, and what still needs to be completed. The second parameter was the ResetUserData switch parameter, indicating to enable UserData to run again, the next time the computer is restarted. The third parameter, RestartInstance, as seen below, restarts the computer, right then and there.

Set-SystemForNextRun -CodeSectionComplete 1 -ResetUserData -RestartInstance

In the second code section, it was invoked differently. This time it only updated the Registry. As it was the last code section to execute against the instance, we didn’t opt to enable UserData again or restart the instance. This won’t always be the case. It’s Windows; we may want a final, nerve-calming restart to take place.

Set-SystemForNextRun -CodeSectionComplete 2

This is a good amount of information in which to wrap your head around. The whole PowerShell function to create PowerShell can make it difficult. I did a bunch of renaming of parameters in this version and tripped myself up a few times. Luckily, it was only briefly. Had it not been, I wouldn’t be able to finish writing now and go to bed. All that said, if you have any questions, I shouldn’t be too difficult to track down.

AWS UserData Multiple Run Framework Part IV (a)


Notice: The following post was originally published on another website. As the post is no longer accessible, it is being republished here on tommymaynard.com. The post was originally published on September 12, 2019.


This is the fourth installment of this series. I never thought I’d write another article about it and yet, here I am. One thing before we recap, I’m breaking this fourth installment into two parts: a and b. It’s too much to read at once. Well, it might be for me at least, if I wasn’t the one writing it. The link for Part b will be at the bottom of this post when it’s published, too.

This series of articles began in 2017. First, I’ll explain what each article brought to this series. Then, I’ll discuss the newest changes—the purpose of this fourth article. You know how it goes, though. You get a chance to see your old code, and you almost immediately find things you would’ve done differently. That’s what this is partially about. That and the fact that I may use this framework again here soon.

When you provision a new Windows virtual server (an EC2 Instance) in AWS or Amazon Web Services, you get an option to run batch and/or PowerShell against the instance at the first launch. This allows you, by default, a one-time shot to configure your server as you see fit. In my AWS UserData Multiple Run Framework, you’ve long been able to configure an instance multiple times between multiple restarts. Follow along, as I quickly catch you up and introduce the changes. This is partially due to how AWS has changed things from Server 2012 R2 and earlier, and Server 2016 and later. There’s also some of that traditional code clean-up, to which I eluded, and the addition of new features, as well.

In the first article, I introduced the AWS UserData Multiple Run Framework. It used text files, stored at the root of the C:\ drive, to determine where in the UserData code it should proceed after each restart.

In the second article, I picked up where the first one left off. This article was the first one to include code to create the code that would be used in the EC2 instance’s UserData. It was, and still is, PowerShell creating PowerShell. It still used text files, as it ran each code section against the server.

In the third article, things changed. Here I introduced using the Windows Registry to maintain what had and hadn’t yet been run in UserData. Additionally, I included an updated New-AWSMultiRunTemplate (as you’ll see, the name has changed) function to create the UserData code. As stated in that article and above, this isn’t the code you place inside the UserData section—it’s the code that creates the code you place inside the UserData section. It’s an important distinction.

In this fourth article—as you’ll see in part b—I’ve added and corrected a good deal. I’ve made additions, removals, and changes in both the function that produces the UserData code and within the UserData produced code, as well. In part b we’ll do a quick rundown on the modifications I remember making. Then, we’ll include and invoke our code-producing code. Following that, we’ll cover the produced code for those unfamiliar with this project. This is backward from the previous articles; however, it’s more of a logical flow for moving forward.

Part b

AWS Service Acronyms


Notice: The following post was originally published on another website. As the post is no longer accessible, it is being republished here on tommymaynard.com. The post was originally published on August 30, 2019.


I work with some of the most intelligent people I may have ever met.

It seems that a good deal of these brilliant minds focus on AWS or Amazon Web Services. It’s also much of what’s important to our enterprise right now, so it’s logical. I also focus on AWS (partly). A portion of the people with whom I work use the acronym CF for AWS CloudFormation. But, that acronym is reserved for Amazon CloudFront. How do they not know this? How do I know this!? It’s of no surprise, but I see this acronym used incorrectly outside work, as well. Here’s one: https://stackoverflow.com/questions/54752989/aws-cloudformation-how-to-use-if-else-conditions. I won’t deny that CF does make sense. I’ll agree to that, but not alongside the consideration of Amazon CloudFront.

I’ve yet to find that single source of AWS acronyms. Or have I?

AWS has authored two PowerShell modules (one for Windows PowerShell and one for the cross-platform version: PowerShell), with the term AWSPowerShell included in the name. They are both at version 3.3.563.1 and have over 5,900 Cmdlets (pronounced, but not spelled as, “commandlets” [for those that don’t work closely with PowerShell in any form but may end up here]). There are over 6,200 total items in each of these two modules if you include the aliases. The PowerShell command naming convention best practice is to use an approved verb (the Get-Verb Cmdlet returns a list of those), a dash, and a singular noun. If more than one noun is used, developers should use PascalCase—this happens quite often. And again, this is for the non-PowerShell people in the room—if you’re out there.

Some cmdlet examples from these modules are Get-CFDistribution, Write-S3ObjectTagSet, Set-SQSQueueAttribute, and New-WAFRule. These commands work just as the AWS CLI does, in that they call an API at Amazon. As far as best practice goes, another thing both individuals and vendors often do is include a prefix after the dash, but before the noun(s) in their command names. Each of the AWSPowerShell Cmdlets I included above makes use of a prefix. There were CF, S3, SQS, and WAF. Another one of the cmdlets in these modules is called Get-AWSPowerShellVersion (AWS prefix); I’ve brought it up here before. It spits out some version text, which is all it should do. However, if you use the ListServiceVersionInfo parameter provided by this cmdlet, you can return all the services and their matching noun prefixes, as well. You can return the API Versions too, which are dates. I didn’t include those below, however.

PS> Get-AWSPowerShellVersion -ListServiceVersionInfo | Select-Object -Property Service,'Noun Prefix' | Sort-Object -Property 'Noun Prefix'
Service                                               Noun Prefix
-------                                               -----------
Application Auto Scaling                              AAS
AWS Certificate Manager                               ACM
Application Discovery Service                         ADS
Amazon API Gateway                                    AG
Amazon API Gateway V2                                 AG2
Amazon API Gateway Management API                     AGM
Alexa For Business                                    ALXB
AWS Amplify                                           AMP
AWS App Mesh                                          AMSH
AWS AppStream                                         APS
Auto Scaling                                          AS
AWS Support API                                       ASA
AWS Auto Scaling Plans                                ASP
AWS AppSync                                           ASYN
Amazon Athena                                         ATH
Amazon Backup                                         BAK
AWS Batch                                             BAT
AWS Budgets                                           BGT
AWS Cloud9                                            C9
AWS CodeBuild                                         CB
AWS CodeCommit                                        CC
AWS CodeDeploy                                        CD
AWS Cloud Directory                                   CDIR
AWS Cost Explorer                                     CE
Amazon CloudFront                                     CF
AWS Config                                            CFG
AWS CloudFormation                                    CFN
Amazon Cognito Identity                               CGI
Amazon Cognito Identity Provider                      CGIP
Amazon Cognito Sync                                   CGIS
Amazon Chime                                          CHM
AWS Comprehend Medical                                CMPM
Amazon Comprehend                                     COMP
Amazon Connect Service                                CONN
AWS CodePipeline                                      CP
Amazon CloudSearch                                    CS
Amazon CloudSearchDomain                              CSD
AWS CodeStar                                          CST
AWS CloudTrail                                        CT
AWS Cost and Usage Report                             CUR
Amazon CloudWatch                                     CW
Amazon CloudWatch Application Insights                CWAI
Amazon CloudWatch Events                              CWE
Amazon CloudWatch Logs                                CWL
Amazon DynamoDB Accelerator (DAX)                     DAX
AWS Direct Connect                                    DC
Amazon DynamoDB                                       DDB
AWS Device Farm                                       DF
Amazon Data Lifecycle Manager                         DLM
AWS Database Migration Service                        DMS
Amazon DocumentDB                                     DOC
AWS Data Pipeline                                     DP
AWS Directory Service                                 DS
AWS DataSync                                          DSYN
AWS Elastic Beanstalk                                 EB
Amazon ElastiCache                                    EC
Amazon Elastic Compute Cloud                          EC2
Amazon EC2 Container Registry                         ECR
Amazon EC2 Container Service                          ECS
Amazon Elastic File System                            EFS
Amazon Elastic Container Service for Kubernetes       EKS
Elastic Load Balancing                                ELB
Elastic Load Balancing V2                             ELB2
AWS Elemental MediaConvert                            EMC
AWS Elemental MediaConnect                            EMCN
AWS Elemental MediaLive                               EML
AWS Elemental MediaPackage                            EMP
AWS Elemental MediaPackage VOD                        EMPV
Amazon Elastic MapReduce                              EMR
AWS Elemental MediaStore                              EMS
AWS Elemental MediaStore Data Plane                   EMSD
AWS Elemental MediaTailor                             EMT
Amazon Elasticsearch                                  ES
Amazon Elastic Transcoder                             ETS
Amazon EventBridge                                    EVB
Firewall Management Service                           FMS
Amazon FSx                                            FSX
AWS Global Accelerator                                GACL
Amazon GuardDuty                                      GD
AWS Greengrass                                        GG
Amazon Glacier                                        GLC
AWS Glue                                              GLUE
Amazon GameLift Service                               GML
AWS Ground Station                                    GS
AWS Health                                            HLTH
AWS Cloud HSM                                         HSM
AWS Cloud HSM V2                                      HSM2
AWS Identity and Access Management                    IAM
AWS Import/Export                                     IE
Amazon Inspector                                      INS
AWS IoT                                               IOT
AWS IoT Events                                        IOTE
AWS IoT Events Data                                   IOTED
AWS IoT Jobs Data Plane                               IOTJ
AWS IoT Things Graph                                  IOTTG
Amazon Kinesis                                        KIN
Amazon Kinesis Analytics                              KINA
Amazon Kinesis Analytics (v2)                         KINA2
Amazon Kinesis Firehose                               KINF
AWS Key Management Service                            KMS
Amazon Kinesis Video Streams                          KV
Amazon Kinesis Video Streams Media                    KVM
Amazon Lex                                            LEX
AWS License Manager                                   LICM
AWS Lambda                                            LM
Amazon Lex Model Building Service                     LMB
Amazon Lightsail                                      LS
Amazon Macie                                          MAC
Amazon Managed Blockchain                             MBC
AWS Marketplace Commerce Analytics                    MCA
AWS Marketplace Entitlement Service                   MES
AWS Migration Hub                                     MH
Amazon Machine Learning                               ML
AWS Marketplace Metering                              MM
AWS Mobile                                            MOBL
Amazon MQ                                             MQ
Managed Streaming for Kafka                           MSK
Amazon MTurk Service                                  MTR
Amazon Neptune                                        NPT
AWS OpsWorks                                          OPS
AWS Organizations                                     ORG
AWS OpsWorksCM                                        OWCM
AWS Certificate Manager Private Certificate Authority PCA
AWS Personalize                                       PERS
Amazon Personalize Events                             PERSE
AWS Personalize Runtime                               PERSR
AWS Performance Insights                              PI
Amazon Pinpoint                                       PIN
Amazon Pinpoint Email                                 PINE
AWS Price List Service                                PLS
Amazon Polly                                          POL
Amazon QuickSight                                     QS
Amazon Route 53                                       R53
Amazon Route 53 Domains                               R53D
Amazon Route 53 Resolver                              R53R
AWS Resource Access Manager                           RAM
Amazon Relational Database Service                    RDS
AWS RDS DataService                                   RDSD
Amazon Rekognition                                    REK
AWS Resource Groups                                   RG
AWS Resource Groups Tagging API                       RGT
AWS RoboMaker                                         ROBO
Amazon Redshift                                       RS
Amazon Simple Storage Service                         S3
Amazon S3 Control                                     S3C
AWS Serverless Application Repository                 SAR
AWS Service Catalog                                   SC
Amazon Route 53 Auto Naming                           SD
AWS Secrets Manager                                   SEC
Amazon Simple Email Service                           SES
AWS Step Functions                                    SFN
AWS Storage Gateway                                   SG
AWS Shield                                            SHLD
AWS Security Hub                                      SHUB
Amazon SageMaker Service                              SM
Amazon SageMaker Runtime                              SMR
Amazon Server Migration Service                       SMS
AWS Import/Export Snowball                            SNOW
Amazon Simple Notification Service                    SNS
AWS Service Quotas                                    SQ
Amazon Simple Queue Service                           SQS
AWS Systems Manager                                   SSM
AWS Security Token Service                            STS
AWS Simple Workflow Service                           SWF
AWS Transfer for SFTP                                 TFR
Amazon Translate                                      TRN
Amazon Transcribe Service                             TRS
Amazon Textract                                       TXT
AWS WAF                                               WAF
AWS WAF Regional                                      WAFR
Amazon WorkDocs                                       WD
Amazon WorkSpaces                                     WKS
Amazon WorkLink                                       WL
Amazon WorkMail                                       WM
AWS X-Ray                                             XR

This is the closest we have to an official, AWS acronym list that I’ve yet to find. And yes, I’ve looked.

As can be seen, the CF acronym is used for CloudFront and CFN for CloudFormation. Use whatever you like, I suppose. With my teammates, I can’t think of a time when I didn’t know it was CloudFormation due to the context. I may have lost a few valuable seconds in life rereading a few sentences a few times, but I’ve so far survived. Still, I felt like it should be mentioned, so I can say I did my part to sleep better at night. Just kidding. It doesn’t keep me awake at night.

I’ll continue to use CFN. Perhaps they’ll think it’s me.

AWS UserData Multiple Run Framework

In AWS, we can utilize the UserData section in EC2 to run PowerShell against our EC2 instances at launch. I’ve said it before; I love this option. As someone that speaks PowerShell with what likely amounts to first language fluency, there’s so much I do to automate my machine builds with CloudFormation, UserData, and PowerShell.

I’ve begun to have a need to do various pieces of automation at different times. This is to say I need to have multiple instance restarts, as an instance is coming online, in order to separate different pieces of configuration and installation. You’ll figure out when you need that, too. And, when you do, you can use what I’ve dubbed the “multiple run framework for AWS.” But really, you call it what you want. That hardly matters.

We have to remember, that by default, UserData only runs once. It’s when the EC2 instance launches for the first time. In the below example, we’re going to do three restarts and four separate code runs.

Our UserData section first needs to add a function to memory. I’ve called it Set-SystemForNextRun and its purpose is to (1) create what I call a “passfile” to help indicate where we are in the automation process, (2) enable UserData to run the next time the service is restarted (this happens at instance restart, obviously), and (3) restart the EC2 instance. Let’s have a look. Its three parameters and three If statements; simple stuff.

Function Set-SystemForNextRun {
    Param (
        [string]$PassFile,
        [switch]$UserData,
        [switch]$Restart
    )
    If ($PassFile) {
        [System.Void](New-Item -Path "$env:SystemDrive\passfile$PassFile.txt" -ItemType File)
    }
    If ($UserData) {
        $Path = "$env:ProgramFiles\Amazon\Ec2ConfigService\Settings\config.xml"
        [xml]$ConfigXml = Get-Content -Path $Path
        ($ConfigXml.Ec2ConfigurationSettings.Plugins.Plugin |
            Where-Object -Property Name -eq 'Ec2HandleUserData').State = 'Enabled'
        $ConfigXml.Save($Path)
    }
    If ($Restart) {
        Restart-Computer -Force
    }
}

The above function accepts three parameters: PassFile, UserData, and Restart. PassFile accepts a string value. You’ll see how this works in the upcoming If-ElseIf example. UserData and Restart are switch parameters. If they’re included when the function is invoked, they’re True ($true), and if they’re not included, they’re False ($false).

Each of the three parameters has its own If statement within the Set-SystemforNextRun function. If PassFile is included, it creates a text file called C:\passfile<ValuePassedIn>.txt. If UserData is included, it resets UserData to enabled (it effectively, checks the check box in the Ec2Config GUI). If Restart is included, it restarts the instance, right then and there.

Now let’s take a look at the If-ElseIf statement that completes four code runs and three restarts. We’ll discuss it further below, but before we do, a little reminder. Our CloudFormation UserData PowerShell is going to contain the above Set-SystemForNextRun function, and something like you’ll see below after you’ve edited it for your needs.

If (-Not(Test-Path -Path "$env:SystemDrive\passfile1.txt")) {

    # Place code here (1).

    # Invoke Set-SystemForNextRun function.
    Set-SystemForNextRun -PassFile '1' -UserData -Restart

} ElseIf (-Not(Test-Path -Path "$env:SystemDrive\passfile2.txt")) {

    # Place code here (2).

    # Invoke Set-SystemForNextRun function.
    Set-SystemForNextRun -PassFile '2' -UserData -Restart

} ElseIf (-Not(Test-Path -Path "$env:SystemDrive\passfile3.txt")) {

    # Place code here (3).

    # Invoke Set-SystemForNextRun function.
    Set-SystemForNextRun -PassFile '3' -UserData -Restart

} ElseIf (-Not(Test-Path -Path "$env:SystemDrive\passfile4.txt")) {

    # Place code here (4).

    # Invoke Set-SystemForNextRun function.
    Set-SystemForNextRun -PassFile '4'

}

In line 1, we test whether or not the file C:\passfile1.txt exists. If it doesn’t exist, we run the code in the If portion. This will run whatever PowerShell we add to that section. Then it’ll pass 1 to the Set-SystemForNextRun function to have C:\passfile01.txt created. Additionally, because the UserData and Restart parameters are included, it’ll reset UserData to enabled, and restart the EC2 instance. Because the C:\passfile1.txt file now exists, the next time the UserData runs, it’ll skip the If portion and evaluate the first ElseIf statement.

This ElseIf statement determines whether or not the C:\passfile2.txt file exists, or not. If it doesn’t, and it won’t after the first restart, then the code in this ElseIf will run. When it’s done, it’ll create the passfile2.txt file, reset UserData, and restart the instance. It’ll do this for the second ElseIf (third code run), and the final ElseIf (fourth code run), as well. Notice that the final invocation of the Set-SystemForNextRun function doesn’t enable UserData or Restart the instance. Be sure to add those if you need either completed after the final ElseIf completes.

And that’s it. At this point in time, I always use my Set-SystemForNextRun function and a properly written If-ElseIf statement to separate the configuration and installation around the necessary amount of instance restarts. In closing, keep in mind that deleting those pass files from the root of the C:\ drive is not something you’ll likely want to do. In time, I may do a rewrite that stores entries in the Registry perhaps, so there’s less probability that one of these files might be removed by someone.

Either way, I hope this is helpful for someone! If you’re in this space—AWS, CloudFormation, UserData, and PowerShell—then chances are good that at some point you’re going to want to restart an instance, and then continue to configure it.

Read II

PowerShell Code and AWS CloudFormation UserData

Note: This post was written well over a month ago, but was never posted, due to some issues I was seeing in AWS GovCloud. It works 100% of the time now, in both GovCloud and non-GovCloud AWS. That said, if you’re using Read-S3Object in GovCloud, you’re going to need to include the Region parameter name and value.

As I spend more and more time with AWS, I end up back at PowerShell. If I haven’t said it yet, thank you Amazon Web Services, for writing us a PowerShell module.

In the last month, or two, I’ve been getting into the CloudFormation template business. I love the whole UserData option we have—injecting PowerShell code into an EC2 instance during its initialization, and love, that while we can do it in the AWS Management Console, we can do it with CloudFormation (CFN) too. In the last few months, I’ve decided to do things a bit differently. Instead of dropping large amounts of PowerShell code inside my UserData property in the CFN template, I decided to use Read-S3Object to copy PowerShell modules to EC2 instances, and then just issue calls to the functions in the remainder of the CFN UserData. In one instance, I went from 200+ lines of PowerShell in the CFN template to just a few.

To test, I needed to verify if I could get a module folder and file into the proper place on the instance and be able to use the module’s function(s) immediately, without any need to end one PowerShell session and start a new one. I suspected this would work just fine, but it needed to be seen.

Here’s how the testing went: On my Desktop, I have a folder called MyModule. Inside the folder, I have a file called MyModule.psm1. If you haven’t seen it before, this file extension indicates the file is a PowerShell module file. The contents of the file, are as follows:

Function Get-A {
    'A'
}

Function Get-B {
    'B'
}

Function Get-C {
    'C'
}

The file contents indicate that the module contains three functions: Get-A, Get-B, and Get-C. In the next example, we can see that the Desktop folder isn’t a place where a module file and folder can exist, where we can expect that the modules will be automatically loaded into the PowerShell session. PowerShell isn’t aware of this module on its own, as can be seen below.

Get-A
Get-A : The term 'Get-A' is not recognized as the name of a cmdlet, function, script file, or operable program. Check
the spelling of the name, or if a path was included, verify that the path is correct and try again.
At line:1 char:1
+ Get-A
+ ~~~~~
    + CategoryInfo          : ObjectNotFound: (Get-A:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException
Get-B
Get-B : The term 'Get-B' is not recognized as the name of a cmdlet, function, script file, or operable program. Check
the spelling of the name, or if a path was included, verify that the path is correct and try again.
At line:1 char:1
+ Get-B
+ ~~~~~
    + CategoryInfo          : ObjectNotFound: (Get-B:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException
Get-C
Get-C : The term 'Get-C' is not recognized as the name of a cmdlet, function, script file, or operable program. Check
the spelling of the name, or if a path was included, verify that the path is correct and try again.
At line:1 char:1
+ Get-C
+ ~~~~~
    + CategoryInfo          : ObjectNotFound: (Get-C:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

While I could tell PowerShell to look on my desktop, what I wanted to do is have my CFN template copy the module folder out of S3 and place it on the instances, in a preferred and proper location: C:\Program Files\WindowsPowerShell\Modules. This is a location that PowerShell checks for modules automatically, and loads them the moment a contained function, or cmdlet, from the module, is requested. My example uses a different path, but PowerShell will check here automatically, as well. As a part of this testing, we’re pretending that the movement from my desktop is close enough to the movement from S3 to an EC2 instance. I’ll obviously test this more with AWS.

Move-Item -Path .\Desktop\MyModule\ -Destination C:\Users\tommymaynard\Documents\WindowsPowerShell\Modules\
Get-A
A
PS > Get-B
B
Get-C
C

Without the need to open a new PowerShell session, I absolutely could use the functions in my module, the moment the module was moved from the Desktop into a folder PowerShell looks at by default. Speaking of those locations, you can view them by returning the value in the $env:PSModulePath environmental variable. Use $env:PSModulePath -split ';' to make it easier to read.

Well, it looks like I was right. I can simply drop those module folders on the EC2 instance, into C:\Program Files\WindowsPowerShell\Modules, just before they’re used with no need for anything more than the current PowerShell session that’s moving them into place.

Update: After this on-my-own-computer test, I took it to AWS. It works, and now it’s the only way I use my CFN template UserData. I write my function(s), house them in a PowerShell module(s), copy them to S3, and finally use my CFN UserData to copy them to the EC2 instance. When that’s complete, I can call the contained function(s) without any hesitation, or additional work. It wasn’t necessary, but I added sleep commands between the function invocations. Here’s a quick, modified example you might find in the UserData of one of my CloudFormation templates.

      UserData:
        Fn::Base64:
          !Sub |
          <powershell>
            # Download PowerShell Modules from S3.
            $Params = @{
              BucketName = 'windows'
              Keyprefix = 'WindowsPowerShell/Modules/ProjectVII/'
              Folder = "$env:ProgramFiles\WindowsPowerShell\Modules"
            }
            Read-S3Object @Params | Out-Null

            # Invoke function(s).
            Set-TimeZone -Verbose -Log
            Start-Sleep -Seconds 15

            Add-EncryptionType -Verbose -Log
            Start-Sleep -Seconds 15

            Install-ProjectVII -Verbose -Log
            Start-Sleep -Seconds 15
            </powershell>