Tag Archives: AWS

AWS Vendor-Written Generated Code


Notice: The following post was originally published on another website. As the post is no longer accessible, it is being republished here on tommymaynard.com. The post was originally published on September 13, 2019.


Sometimes you read an error message, or in this case, come across some vendor-written code that you can’t find anywhere else on the Internet. It’s been years, but once PowerShell generated an error I had never seen. I couldn’t find a hit for it online anywhere either. I felt that once I had figured out the problem behind that error message, it was my duty to write about it—to help get that error message picked up by search engines, as well as my experience. I feel nearly the same way about the below code to which I was recently introduced, written by AWS, or Amazon Web Services. I’ll share it now. Just a note. This is exactly how it was found. There were no indentations. I’m not too concerned about the lack of indentations—I don’t have to stare at it each day. Perhaps it’s generated by something that may not be preserving the spacing/tabs. That’s just a guess.

try { 
Get-Service Ec2Config 
$EC2SettingsFile='C:\Program Files\Amazon\Ec2ConfigService\Settings\Config.xml' 
$xml = [xml](get-content $EC2SettingsFile) 
$xmlElement = $xml.get_DocumentElement() 
$xmlElementToModify = $xmlElement.Plugins 
foreach ($element in $xmlElementToModify.Plugin){ 
if ($element.name -eq 'Ec2SetPassword') {$element.State='Enabled'} 
elseif ($element.name -eq 'Ec2HandleUserData') {$element.State='Enabled'} 
elseif ($element.name -eq 'Ec2DynamicBootVolumeSize') {$element.State='Enabled'} 
} 
$xml.Save($EC2SettingsFile) 
} 
catch 
{ 
C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -ExecutionPolicy Bypass -File 'C:\ProgramData\Amazon\EC2-Windows\Launch\Scripts\InitializeInstance.ps1' -Schedule 
} 
finally 
{ 
New-Item -Path HKLM:\Software\Amazon -Name WarmBoot 
Invoke-Expression -Command:'shutdown.exe /s /t 0' 
} 
while ($true){ 
}

It’s rare that I ever manually launch an AWS EC2 instance (a virtual server). Well, I was doing that recently for some quick testing and my UserData PowerShell script was not landing in the C:\Program Files\Amazon\Ec2ConfigService\Scripts\UserScript.ps1 file on my Windows Server 2012 R2 instance, as it should have been. I was doing something wrong and it wasn’t clear to me what, soon enough. Before we get to that, let’s discuss what we have here. We have a try-catch language construct. I know from my AWS experience that most of what’s going on in the try block was done for Windows Server 2012 R2 and newer. I also know that what’s in the catch block is how we ensure UserData is enabled in Windows Server 2016 and later. AWS couldn’t take my UserData and drop it on this instance. Instead, I got this code in its place. Ugh.

This code also includes the finally block. That code is run regardless of whether the try or catch block fired. The code creates a value in the Windows Registry and then restarts the computer using Invoke-Expression—interesting choice. It’s always fun to see vendor code. It closes with an empty While language construct. While $true is $true, this While loop will run—and do absolutely nothing. It will do that successfully, however.

Again, my UserData PowerShell wasn’t getting into this UserScript file. It was, however, available by “navigating” to 169.254.169.254 on the EC2 instance.

Invoke-RestMethod -Uri http://169.254.169.254/latest/user-data
# >> Add function to memory.
Function Set-SystemForNextRun {
...
    Set-SystemForNextRun -CodeSectionComplete 2
} # End If-ElseIf.

The problem was that my code didn’t have the begin and end PowerShell tags. In order for the UserScript.ps1 file to be populated with my code and not this code from Amazon, I needed to ensure I was including everything required by me. It seems like something in the AWS Management Console (the web front end) could’ve notified me after looking at my code, but before moving to the next step in manually building my instance. Or, be even less helpful, and additionally write something else in the UserScript.ps1 file. They could’ve just started their code with a comment to tell me I didn’t follow the directions. I’ve used UserData in CloudFormation; I know these tags are required.

<#
This doesn't look right, does it?
Did you remember to use script or
powershell start and end tags?
#>

Anyway, once I enclosed my PowerShell in the proper tags, it worked, and I moved on. And by moved on I mean, I found another problem to consume me—as is typical in this industry. It should’ve look liked this from the start. Ugh.

Invoke-RestMethod -Uri http://169.254.169.254/latest/user-data
<powershell>
# >> Add function to memory.
Function Set-SystemForNextRun {
...
    Set-SystemForNextRun -CodeSectionComplete 2
} # End If-ElseIf.
</powershell>

Things changed between Windows Server 2012 R2 and 2016, 2019. I’m not exactly sure where the UserData code ends up in the newer OS if it even does end up in a file on disk as it has previously.

AWS UserData Multiple Run Framework Part IV (b)


Notice: The following post was originally published on another website. As the post is no longer accessible, it is being republished here on tommymaynard.com. The post was originally published on September 13, 2019.


This article began with Part a. Read that, if you haven’t already.

First off, the code-producing function has been renamed from New-AWSMultiRunTemplate to New-AWSUserDataMultipleRunTemplate. Sure, it’s longer in name, but it’s more clear (to me at least) when I see its name. Other small changes were the removal of unnecessary code comments. While these may have been in the New-AWSUserDataMultipleRunTemplate function itself, they were definitely changed in the produced code. Additionally, I’ve added a ProjectName parameter. The value, supplied to this parameter, is used throughout the generated code for naming purposes within the Registry. There are code changes to the produced code to run against Server 2016 and greater, as well. Therefore, it runs against Windows Server 2012 and 2012 R2 (and probably older), as well as Server 2016 and 2019. Hopefully, it will be extended further, but only AWS knows that for sure. Server 2012 R2 and earlier used EC2Config to configure a Windows instance, while 2016 and 2019 used EC2Launch.

If (test1) {
    {<statement list 1>}
} # End If.

As seen above, I tend to comment on my closing language construct brackets (see # End If.). These are now included in the produced code, both statically and dynamically. It’s a personal preference (that you’ll have to deal with if you use this free, code offering).

This is the new, New-AWSUserDataMultipleRunTemplate code producing function. This isn’t in the PowerShell Gallery, or anywhere else. It probably will never be, either. In fact, this is likely the last time I’ll prep it for public consumption. So again, this is the code you run to create the code you enter in UserData. We won’t discuss what’s in this code; however, we’ll run it further below and discuss that at a minimum.

Function New-AWSUserDataMultipleRunTemplate {
    [CmdletBinding()]
    Param (
        [Parameter(Mandatory)]
        [string]$ProjectName,

        [Parameter()]
        [ValidateRange(1,10)]
        [int]$CodeSectionCount = 2,

        [Parameter()]
        [ValidateSet('All','AllButLast')]
        [string]$EnableUserData = 'AllButLast',

        [Parameter()]
        [ValidateSet('All','AllButLast')]
        [string]$EnableRestart = 'AllButLast'
    )

    Begin { 
        #region Set Write-Verbose block location.
        $BlockLocation = '[BEGIN  ]'
        Write-Verbose -Message "$BlockLocation Entering the Begin block [Function: $($MyInvocation.MyCommand.Name)]."
        #endregion

        Write-Verbose -Message "$BlockLocation Storing the $ProjectName template's function to memory."
        Write-Verbose -Message "$BlockLocation Ensure the server where the code will reside does not already have the ""HKLM:\SOFTWARE\$ProjectName"" path."
        $TemplateFunction = @"
# >> Add function to memory.
Function Set-SystemForNextRun {
    Param (
        [string]`$CodeSectionComplete,
        [switch]`$ResetUserData,
        [switch]`$RestartInstance
    )
    If (`$CodeSectionComplete) {
        [System.Void](New-ItemProperty -Path 'HKLM:\SOFTWARE\$ProjectName' -Name "CodeSection`$CodeSectionComplete" -Value 'Complete')
    } # End If.
    If (`$ResetUserData) {
        try {
            `$Path = 'C:\Program Files\Amazon\Ec2ConfigService\Settings\config.xml'
            [xml]`$ConfigXml = Get-Content -Path `$Path -ErrorAction Stop
            (`$ConfigXml.Ec2ConfigurationSettings.Plugins.Plugin |
                Where-Object -Property Name -eq 'Ec2HandleUserData').State = 'Enabled'
            `$ConfigXml.Save(`$Path)
        } catch {
            C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -ExecutionPolicy Bypass -File C:\ProgramData\Amazon\EC2-Windows\Launch\Scripts\InitializeInstance.ps1 -Schedule
        } # End try-catch.
    } # End If.
    If (`$RestartInstance) {
        Restart-Computer -Force
    } # End If.
} # End Function: Set-SystemForNextRun.

# >> Create/Check for Registry Subkey.
If (-Not(Get-Item -Path 'HKLM:\SOFTWARE\$ProjectName' -ErrorAction SilentlyContinue)) {
    [System.Void](New-Item -Path 'HKLM:\SOFTWARE\' -Name '$ProjectName')
} # End If.

# >> Run user code/invoke Set-SystemForNextRun function.

"@
    } # End Begin.

    Process {
        #region Set Write-Verbose block location.
        $BlockLocation = '[PROCESS]'
        Write-Verbose -Message "$BlockLocation Entering the Process block [Function: $($MyInvocation.MyCommand.Name)]."
        #endregion

        Write-Verbose -Message "$BlockLocation Beginning to create the If-ElseIf code for the template."
        1..$CodeSectionCount | ForEach-Object {
            If ($_ -eq 1) {
                $Start = 'If'
                If ($CodeSectionCount -eq 1) {
                    $End = '# End If.'
                } # End If.
            } ElseIf ($_ -eq $CodeSectionCount -and $CodeSectionCount -eq 2) {
                $Start = 'ElseIf'; $End = '# End If-ElseIf.'
            } ElseIf ($_ -eq $CodeSectionCount -and $CodeSectionCount -ne 2) {
                $End = "# End If-ElseIf x$($CodeSectionCount - 1)."
            } Else {
                $Start = 'ElseIf'
            } # End If.

            If ($EnableUserData -eq 'All') {
                $UserData = '-ResetUserData '
            } ElseIf ($_ -eq $CodeSectionCount) {
                $UserData = $null
            } Else {
                $UserData = '-ResetUserData '
            } # End If.

            If ($EnableRestart -eq 'All') {
                $Restart = '-RestartInstance'
            } ElseIf ($_ -eq $CodeSectionCount) {
                $Restart = $null
            } Else {
                $Restart = '-RestartInstance'
            } # End If.

            $TemplateIfElseIf += @"
$Start (-Not((Get-ItemProperty -Path 'HKLM:\SOFTWARE\$ProjectName').CodeSection$_ -eq 'Complete')) {

    # CodeSection $_.

    Set-SystemForNextRun -CodeSectionComplete $_ $UserData$Restart
} $End
"@
        } # End ForEach-Object.
    } # End Process.

    End {
        #region Set Write-Verbose block location.
        $BlockLocation = '[END    ]'
        Write-Verbose -Message "$BlockLocation Entering the End block [Function: $($MyInvocation.MyCommand.Name)]."
        #endregion

        Write-Verbose -Message "$BlockLocation Creating the AWS UserData Mulitple Run Framework code with $CodeSectionCount code section$(If ($CodeSectionCount -gt 1) {'s'})."
        "$TemplateFunction$TemplateIfElseIf"
    } # End End.
} # End Function: New-AWSUserDataMultipleRunTemplate.

Copy and paste the above, New-AWSUserDataMultipleRunTemplate function into VS Code, or another preferred PowerShell development environment, if there is such a thing. You’re not still using the ISE, are you? Then, add the function to memory for use (in VS Code, that’s Ctrl + A to select all, then F8 to run the selection). Once the function is sitting in memory, we can use it to create our UserData code, as seen below. In this version—it doesn’t even really have a version number, which is weird—there’s now a mandatory ProjectName parameter. Keep this short and simple, and if it were me, I keep spaces and odd characters out of it. This is the value that will be used in the Windows Registry, and within the code that’s produced for UserData.

PS > New-AWSUserDataMultipleRunTemplate -ProjectName MistFit
# >> Add function to memory.
Function Set-SystemForNextRun {
    Param (
       [string]$CodeSectionComplete,
       [switch]$ResetUserData,
        [switch]$RestartInstance
    )
    If ($CodeSectionComplete) {
        [System.Void](New-ItemProperty -Path 'HKLM:\SOFTWARE\MistFit' -Name "CodeSection$CodeSectionComplete" -Value 'Complete')
    } # End If.
    If ($ResetUserData) {
        try {
            $Path = 'C:\Program Files\Amazon\Ec2ConfigService\Settings\config.xml'
            [xml]$ConfigXml = Get-Content -Path $Path -ErrorAction Stop
            ($ConfigXml.Ec2ConfigurationSettings.Plugins.Plugin |
                Where-Object -Property Name -eq 'Ec2HandleUserData').State = 'Enabled'
            $ConfigXml.Save($Path)
        } catch {
            C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -ExecutionPolicy Bypass -File C:\ProgramData\Amazon\EC2-Windows\Launch\Scripts\InitializeInstance.ps1 -Schedule
        } # End try-catch.
    } # End If.
    If ($RestartInstance) {
        Restart-Computer -Force
    } # End If.
} # End Function: Set-SystemForNextRun.

# >> Create/Check for Registry Subkey.
If (-Not(Get-Item -Path 'HKLM:\SOFTWARE\MistFit' -ErrorAction SilentlyContinue)) {
    [System.Void](New-Item -Path 'HKLM:\SOFTWARE\' -Name 'MistFit')
} # End If.

# >> Run user code/invoke Set-SystemForNextRun function.
If (-Not((Get-ItemProperty -Path 'HKLM:\SOFTWARE\MistFit').CodeSection1 -eq 'Complete')) {

    # CodeSection 1.

Set-SystemForNextRun -CodeSectionComplete 1 -ResetUserData -RestartInstance
} ElseIf (-Not((Get-ItemProperty -Path 'HKLM:\SOFTWARE\MistFit').CodeSection2 -eq 'Complete')) {

    # CodeSection 2.

Set-SystemForNextRun -CodeSectionComplete 2
} # End If-ElseIf.
PS C:>

By default, it still creates two code sections, as can be seen just above in the If-ElseIf statement. It can create just one, although that makes less sense for something that can provide multiple opportunities for configuration between restarts. Even if you only need one, this may still be the framework for you. Maybe you need a restart after a single configuration pass. It will do up to 10 code sections if, for some reason, you need that many opportunities to configure a single instance. Not me. I’ve only ever needed three or four total. The New-AWSUserDataMultipleRunTemplate function still includes the EnableUserData and EnableRestart switch parameters. Both have the default parameter value AllButLast, however, both parameters can accept All as the value, too. If this is used for the EnableRestart switch parameter, the EC2 instance will restart after the last time it’s configured (the last code section). In my experience, it’s not always necessary to restart an instance after its final configuration, but this would allow for the time it’s needed. Once you get to know the produced code well, you can manually edit it if necessary.

The produced code has three definitive sections. We’ll start with the below, third section. By default, the New-AWSUserDataMultipleRunTemplate function creates two code sections within the third section. Read that a few times; it’s confusing. It’s like this: The produced code consists of three sections (they each start with # >>). Inside the third one, is where you can have multiple code sections, where user code, that someone else provides, maybe you, is executed against an instance. Notice that there are two comments inside our If-ElseIf statement. On the first pass, it’ll run whatever the code that the user enters to replace “# CodeSection 1.” After the UserData is enabled and the instance is restarted, it’ll run whatever code is entered to replace “# CodeSection 2.” We’ll see more about how it does this shortly.

# >> Run user code/invoke Set-SystemForNextRun function.
If (-Not((Get-ItemProperty -Path 'HKLM:\SOFTWARE\MistFit').CodeSection1 -eq 'Complete')) {

    # CodeSection 1.

    Set-SystemForNextRun -CodeSectionComplete 1 -ResetUserData -RestartInstance
} ElseIf (-Not((Get-ItemProperty -Path 'HKLM:\SOFTWARE\MistFit').CodeSection2 -eq 'Complete')) {

    # CodeSection 2.

    Set-SystemForNextRun -CodeSectionComplete 2
} # End If-ElseIf.

Now that we’ve spent some time with the third section of our produced code, let’s move upward and focus on the middle, or second section, of the code that’s been produced. Here’s that second section, now. It’s real simple. If a specific Windows Registry Subkey doesn’t exist, it’s created. The need to create this will only happen once (the first run). Every other time this If statement fires, it’ll be false and therefore, it won’t attempt to create (something that’s already been created).

# >> Create/Check for Registry Subkey.
If (-Not(Get-Item -Path 'HKLM:\SOFTWARE\MistFit' -ErrorAction SilentlyContinue)) {
    [System.Void](New-Item -Path 'HKLM:\SOFTWARE\' -Name 'MistFit')
} # End If.

And, here’s the first section, last. This function, Set-SystemForNextRun, is placed into memory after this portion of the UserData is executed. It’s invoked by the third section. If you go back up to where we discussed the third section, you’ll see where.

# >> Add function to memory.
Function Set-SystemForNextRun {
    Param (
        [string]$CodeSectionComplete,
        [switch]$ResetUserData,
        [switch]$RestartInstance
    )
    If ($CodeSectionComplete) {
        [System.Void](New-ItemProperty -Path 'HKLM:\SOFTWARE\MistFit' -Name "CodeSection$CodeSectionComplete" -Value 'Complete')
    } # End If.
    If ($ResetUserData) {
        try {
            $Path = 'C:\Program Files\Amazon\Ec2ConfigService\Settings\config.xml'
            [xml]$ConfigXml = Get-Content -Path $Path -ErrorAction Stop
            ($ConfigXml.Ec2ConfigurationSettings.Plugins.Plugin |
                Where-Object -Property Name -eq 'Ec2HandleUserData').State = 'Enabled'
            $ConfigXml.Save($Path)
        } catch {
            C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -ExecutionPolicy Bypass -File C:\ProgramData\Amazon\EC2-Windows\Launch\Scripts\InitializeInstance.ps1 -Schedule
        } # End try-catch.
    } # End If.
    If ($RestartInstance) {
        Restart-Computer -Force
    } # End If.
} # End Function: Set-SystemForNextRun.

In the first code section of the third overall section, Set-SystemForNextRun was first invoked with all three possible parameters. The first parameter is sent in a numeric 1 to the CodeSectionComplete parameter. This created a new Registry value in the “HKLM:\SOFTWARE\MistFit” path. It coerced the number into a string, creating a Registry value named CodeSection1 that had a string value of Complete. These values are how the code knows what’s been done before, and what still needs to be completed. The second parameter was the ResetUserData switch parameter, indicating to enable UserData to run again, the next time the computer is restarted. The third parameter, RestartInstance, as seen below, restarts the computer, right then and there.

Set-SystemForNextRun -CodeSectionComplete 1 -ResetUserData -RestartInstance

In the second code section, it was invoked differently. This time it only updated the Registry. As it was the last code section to execute against the instance, we didn’t opt to enable UserData again or restart the instance. This won’t always be the case. It’s Windows; we may want a final, nerve-calming restart to take place.

Set-SystemForNextRun -CodeSectionComplete 2

This is a good amount of information in which to wrap your head around. The whole PowerShell function to create PowerShell can make it difficult. I did a bunch of renaming of parameters in this version and tripped myself up a few times. Luckily, it was only briefly. Had it not been, I wouldn’t be able to finish writing now and go to bed. All that said, if you have any questions, I shouldn’t be too difficult to track down.

AWS UserData Multiple Run Framework Part IV (a)


Notice: The following post was originally published on another website. As the post is no longer accessible, it is being republished here on tommymaynard.com. The post was originally published on September 12, 2019.


This is the fourth installment of this series. I never thought I’d write another article about it and yet, here I am. One thing before we recap, I’m breaking this fourth installment into two parts: a and b. It’s too much to read at once. Well, it might be for me at least, if I wasn’t the one writing it. The link for Part b will be at the bottom of this post when it’s published, too.

This series of articles began in 2017. First, I’ll explain what each article brought to this series. Then, I’ll discuss the newest changes—the purpose of this fourth article. You know how it goes, though. You get a chance to see your old code, and you almost immediately find things you would’ve done differently. That’s what this is partially about. That and the fact that I may use this framework again here soon.

When you provision a new Windows virtual server (an EC2 Instance) in AWS or Amazon Web Services, you get an option to run batch and/or PowerShell against the instance at the first launch. This allows you, by default, a one-time shot to configure your server as you see fit. In my AWS UserData Multiple Run Framework, you’ve long been able to configure an instance multiple times between multiple restarts. Follow along, as I quickly catch you up and introduce the changes. This is partially due to how AWS has changed things from Server 2012 R2 and earlier, and Server 2016 and later. There’s also some of that traditional code clean-up, to which I eluded, and the addition of new features, as well.

In the first article, I introduced the AWS UserData Multiple Run Framework. It used text files, stored at the root of the C:\ drive, to determine where in the UserData code it should proceed after each restart.

In the second article, I picked up where the first one left off. This article was the first one to include code to create the code that would be used in the EC2 instance’s UserData. It was, and still is, PowerShell creating PowerShell. It still used text files, as it ran each code section against the server.

In the third article, things changed. Here I introduced using the Windows Registry to maintain what had and hadn’t yet been run in UserData. Additionally, I included an updated New-AWSMultiRunTemplate (as you’ll see, the name has changed) function to create the UserData code. As stated in that article and above, this isn’t the code you place inside the UserData section—it’s the code that creates the code you place inside the UserData section. It’s an important distinction.

In this fourth article—as you’ll see in part b—I’ve added and corrected a good deal. I’ve made additions, removals, and changes in both the function that produces the UserData code and within the UserData produced code, as well. In part b we’ll do a quick rundown on the modifications I remember making. Then, we’ll include and invoke our code-producing code. Following that, we’ll cover the produced code for those unfamiliar with this project. This is backward from the previous articles; however, it’s more of a logical flow for moving forward.

Part b

AWS Service Acronyms


Notice: The following post was originally published on another website. As the post is no longer accessible, it is being republished here on tommymaynard.com. The post was originally published on August 30, 2019.


I work with some of the most intelligent people I may have ever met.

It seems that a good deal of these brilliant minds focus on AWS or Amazon Web Services. It’s also much of what’s important to our enterprise right now, so it’s logical. I also focus on AWS (partly). A portion of the people with whom I work use the acronym CF for AWS CloudFormation. But, that acronym is reserved for Amazon CloudFront. How do they not know this? How do I know this!? It’s of no surprise, but I see this acronym used incorrectly outside work, as well. Here’s one: https://stackoverflow.com/questions/54752989/aws-cloudformation-how-to-use-if-else-conditions. I won’t deny that CF does make sense. I’ll agree to that, but not alongside the consideration of Amazon CloudFront.

I’ve yet to find that single source of AWS acronyms. Or have I?

AWS has authored two PowerShell modules (one for Windows PowerShell and one for the cross-platform version: PowerShell), with the term AWSPowerShell included in the name. They are both at version 3.3.563.1 and have over 5,900 Cmdlets (pronounced, but not spelled as, “commandlets” [for those that don’t work closely with PowerShell in any form but may end up here]). There are over 6,200 total items in each of these two modules if you include the aliases. The PowerShell command naming convention best practice is to use an approved verb (the Get-Verb Cmdlet returns a list of those), a dash, and a singular noun. If more than one noun is used, developers should use PascalCase—this happens quite often. And again, this is for the non-PowerShell people in the room—if you’re out there.

Some cmdlet examples from these modules are Get-CFDistribution, Write-S3ObjectTagSet, Set-SQSQueueAttribute, and New-WAFRule. These commands work just as the AWS CLI does, in that they call an API at Amazon. As far as best practice goes, another thing both individuals and vendors often do is include a prefix after the dash, but before the noun(s) in their command names. Each of the AWSPowerShell Cmdlets I included above makes use of a prefix. There were CF, S3, SQS, and WAF. Another one of the cmdlets in these modules is called Get-AWSPowerShellVersion (AWS prefix); I’ve brought it up here before. It spits out some version text, which is all it should do. However, if you use the ListServiceVersionInfo parameter provided by this cmdlet, you can return all the services and their matching noun prefixes, as well. You can return the API Versions too, which are dates. I didn’t include those below, however.

PS> Get-AWSPowerShellVersion -ListServiceVersionInfo | Select-Object -Property Service,'Noun Prefix' | Sort-Object -Property 'Noun Prefix'
Service                                               Noun Prefix
-------                                               -----------
Application Auto Scaling                              AAS
AWS Certificate Manager                               ACM
Application Discovery Service                         ADS
Amazon API Gateway                                    AG
Amazon API Gateway V2                                 AG2
Amazon API Gateway Management API                     AGM
Alexa For Business                                    ALXB
AWS Amplify                                           AMP
AWS App Mesh                                          AMSH
AWS AppStream                                         APS
Auto Scaling                                          AS
AWS Support API                                       ASA
AWS Auto Scaling Plans                                ASP
AWS AppSync                                           ASYN
Amazon Athena                                         ATH
Amazon Backup                                         BAK
AWS Batch                                             BAT
AWS Budgets                                           BGT
AWS Cloud9                                            C9
AWS CodeBuild                                         CB
AWS CodeCommit                                        CC
AWS CodeDeploy                                        CD
AWS Cloud Directory                                   CDIR
AWS Cost Explorer                                     CE
Amazon CloudFront                                     CF
AWS Config                                            CFG
AWS CloudFormation                                    CFN
Amazon Cognito Identity                               CGI
Amazon Cognito Identity Provider                      CGIP
Amazon Cognito Sync                                   CGIS
Amazon Chime                                          CHM
AWS Comprehend Medical                                CMPM
Amazon Comprehend                                     COMP
Amazon Connect Service                                CONN
AWS CodePipeline                                      CP
Amazon CloudSearch                                    CS
Amazon CloudSearchDomain                              CSD
AWS CodeStar                                          CST
AWS CloudTrail                                        CT
AWS Cost and Usage Report                             CUR
Amazon CloudWatch                                     CW
Amazon CloudWatch Application Insights                CWAI
Amazon CloudWatch Events                              CWE
Amazon CloudWatch Logs                                CWL
Amazon DynamoDB Accelerator (DAX)                     DAX
AWS Direct Connect                                    DC
Amazon DynamoDB                                       DDB
AWS Device Farm                                       DF
Amazon Data Lifecycle Manager                         DLM
AWS Database Migration Service                        DMS
Amazon DocumentDB                                     DOC
AWS Data Pipeline                                     DP
AWS Directory Service                                 DS
AWS DataSync                                          DSYN
AWS Elastic Beanstalk                                 EB
Amazon ElastiCache                                    EC
Amazon Elastic Compute Cloud                          EC2
Amazon EC2 Container Registry                         ECR
Amazon EC2 Container Service                          ECS
Amazon Elastic File System                            EFS
Amazon Elastic Container Service for Kubernetes       EKS
Elastic Load Balancing                                ELB
Elastic Load Balancing V2                             ELB2
AWS Elemental MediaConvert                            EMC
AWS Elemental MediaConnect                            EMCN
AWS Elemental MediaLive                               EML
AWS Elemental MediaPackage                            EMP
AWS Elemental MediaPackage VOD                        EMPV
Amazon Elastic MapReduce                              EMR
AWS Elemental MediaStore                              EMS
AWS Elemental MediaStore Data Plane                   EMSD
AWS Elemental MediaTailor                             EMT
Amazon Elasticsearch                                  ES
Amazon Elastic Transcoder                             ETS
Amazon EventBridge                                    EVB
Firewall Management Service                           FMS
Amazon FSx                                            FSX
AWS Global Accelerator                                GACL
Amazon GuardDuty                                      GD
AWS Greengrass                                        GG
Amazon Glacier                                        GLC
AWS Glue                                              GLUE
Amazon GameLift Service                               GML
AWS Ground Station                                    GS
AWS Health                                            HLTH
AWS Cloud HSM                                         HSM
AWS Cloud HSM V2                                      HSM2
AWS Identity and Access Management                    IAM
AWS Import/Export                                     IE
Amazon Inspector                                      INS
AWS IoT                                               IOT
AWS IoT Events                                        IOTE
AWS IoT Events Data                                   IOTED
AWS IoT Jobs Data Plane                               IOTJ
AWS IoT Things Graph                                  IOTTG
Amazon Kinesis                                        KIN
Amazon Kinesis Analytics                              KINA
Amazon Kinesis Analytics (v2)                         KINA2
Amazon Kinesis Firehose                               KINF
AWS Key Management Service                            KMS
Amazon Kinesis Video Streams                          KV
Amazon Kinesis Video Streams Media                    KVM
Amazon Lex                                            LEX
AWS License Manager                                   LICM
AWS Lambda                                            LM
Amazon Lex Model Building Service                     LMB
Amazon Lightsail                                      LS
Amazon Macie                                          MAC
Amazon Managed Blockchain                             MBC
AWS Marketplace Commerce Analytics                    MCA
AWS Marketplace Entitlement Service                   MES
AWS Migration Hub                                     MH
Amazon Machine Learning                               ML
AWS Marketplace Metering                              MM
AWS Mobile                                            MOBL
Amazon MQ                                             MQ
Managed Streaming for Kafka                           MSK
Amazon MTurk Service                                  MTR
Amazon Neptune                                        NPT
AWS OpsWorks                                          OPS
AWS Organizations                                     ORG
AWS OpsWorksCM                                        OWCM
AWS Certificate Manager Private Certificate Authority PCA
AWS Personalize                                       PERS
Amazon Personalize Events                             PERSE
AWS Personalize Runtime                               PERSR
AWS Performance Insights                              PI
Amazon Pinpoint                                       PIN
Amazon Pinpoint Email                                 PINE
AWS Price List Service                                PLS
Amazon Polly                                          POL
Amazon QuickSight                                     QS
Amazon Route 53                                       R53
Amazon Route 53 Domains                               R53D
Amazon Route 53 Resolver                              R53R
AWS Resource Access Manager                           RAM
Amazon Relational Database Service                    RDS
AWS RDS DataService                                   RDSD
Amazon Rekognition                                    REK
AWS Resource Groups                                   RG
AWS Resource Groups Tagging API                       RGT
AWS RoboMaker                                         ROBO
Amazon Redshift                                       RS
Amazon Simple Storage Service                         S3
Amazon S3 Control                                     S3C
AWS Serverless Application Repository                 SAR
AWS Service Catalog                                   SC
Amazon Route 53 Auto Naming                           SD
AWS Secrets Manager                                   SEC
Amazon Simple Email Service                           SES
AWS Step Functions                                    SFN
AWS Storage Gateway                                   SG
AWS Shield                                            SHLD
AWS Security Hub                                      SHUB
Amazon SageMaker Service                              SM
Amazon SageMaker Runtime                              SMR
Amazon Server Migration Service                       SMS
AWS Import/Export Snowball                            SNOW
Amazon Simple Notification Service                    SNS
AWS Service Quotas                                    SQ
Amazon Simple Queue Service                           SQS
AWS Systems Manager                                   SSM
AWS Security Token Service                            STS
AWS Simple Workflow Service                           SWF
AWS Transfer for SFTP                                 TFR
Amazon Translate                                      TRN
Amazon Transcribe Service                             TRS
Amazon Textract                                       TXT
AWS WAF                                               WAF
AWS WAF Regional                                      WAFR
Amazon WorkDocs                                       WD
Amazon WorkSpaces                                     WKS
Amazon WorkLink                                       WL
Amazon WorkMail                                       WM
AWS X-Ray                                             XR

This is the closest we have to an official, AWS acronym list that I’ve yet to find. And yes, I’ve looked.

As can be seen, the CF acronym is used for CloudFront and CFN for CloudFormation. Use whatever you like, I suppose. With my teammates, I can’t think of a time when I didn’t know it was CloudFormation due to the context. I may have lost a few valuable seconds in life rereading a few sentences a few times, but I’ve so far survived. Still, I felt like it should be mentioned, so I can say I did my part to sleep better at night. Just kidding. It doesn’t keep me awake at night.

I’ll continue to use CFN. Perhaps they’ll think it’s me.

CIDR Notation Host Count


Notice: The following post was originally published on another website. As the post is no longer accessible, it is being republished here on tommymaynard.com. The post was originally published on January 15, 2019.


Late last year, I spent some time studying for the Amazon Web Services (AWS) Solutions Architect — Associate exam. In doing so, I briefly ended up covering CIDR again and the two-step math problem required to determine the number of available hosts in a CIDR IP address range.

As a recap, it works this way. Let’s consider the largest VPC (Virtual Private Cloud), or a virtual network, one can define in AWS. That’s 10.0.0.0/16. The way to determine the number of available host IPs is to subtract 16 (as indicated by the /16) from 32 — a constant value. In this example, that total is also 16. We then raise 2 (the base — another constant) to the power of 16 (the exponent/our difference), which results in 65,536 possible hosts. Because PowerShell can often distract me from AWS, let’s take a look at a small function I quickly wrote out — yes, during my AWS study time — to do the conversion for me.

Function Get-CidrHostCount {
    [CmdletBinding()]
    Param (
        [Parameter(Mandatory)]
        [ValidateRange(1,32)]
        $Cidr
    )

    Begin {
    } # End Begin.

    Process {
        "Number of hosts for /$Cidr`: $([System.Math]::Pow(2,32-$Cidr))"
    } # End Process.

    End {
    } # End End.
} # End Function: Get-CidrHostCount.

When invoked, the above Get-CidrHostCount function will accept any numeric value from 1 through 32 and determine how many hosts the CIDR range would allow. This 1 to 32 value is equivalent to the number after the forward-slash in the CIDR notation. Do notice our two-step math problem. The System namespace’s Math class includes a method called Pow. This method accepts two values. The first value is our base again — 2 — and the second number is the exponent. In the function, we use the constant value of 32 and subtract the CIDR value that’s passed in when the function is invoked. These two values are then used to complete the calculation.

The below examples display a few results but eventually return all the possible results.

PS> Get-CidrHostCount -Cidr 16

Number of hosts for /16: 65536

PS> Get-CidrHostCount -Cidr 20

Number of hosts for /20: 4096

PS> Get-CidrHostCount -Cidr 24

Number of hosts for /24: 256

PS> Get-CidrHostCount -Cidr 28

Number of hosts for /28: 16

PS> 1..32 | ForEach-Object {
>> Get-CidrHostCount -Cidr $_
>> }

Number of hosts for /1: 2147483648
Number of hosts for /2: 1073741824
Number of hosts for /3: 536870912
Number of hosts for /4: 268435456
Number of hosts for /5: 134217728
Number of hosts for /6: 67108864
Number of hosts for /7: 33554432
Number of hosts for /8: 16777216
Number of hosts for /9: 8388608
Number of hosts for /10: 4194304
Number of hosts for /11: 2097152
Number of hosts for /12: 1048576
Number of hosts for /13: 524288
Number of hosts for /14: 262144
Number of hosts for /15: 131072
Number of hosts for /16: 65536
Number of hosts for /17: 32768
Number of hosts for /18: 16384
Number of hosts for /19: 8192
Number of hosts for /20: 4096
Number of hosts for /21: 2048
Number of hosts for /22: 1024
Number of hosts for /23: 512
Number of hosts for /24: 256
Number of hosts for /25: 128
Number of hosts for /26: 64
Number of hosts for /27: 32
Number of hosts for /28: 16
Number of hosts for /29: 8
Number of hosts for /30: 4
Number of hosts for /31: 2
Number of hosts for /32: 1

Before we really wrap it up here, let’s change the results as many of us would prefer to see them. Here’s a mildly modified version of the function and the last above command run again.

Function Get-CidrHostCount {
    [CmdletBinding()]
    Param (
        [Parameter(Mandatory)]
        [ValidateRange(1,32)]
        $Cidr
    )

    Begin {
    } # End Begin.

    Process {
        "Number of hosts for /$Cidr`: $('{0:N0}' -f [System.Math]::Pow(2,32-$Cidr))"
    } # End Process.

    End {
    } # End End.
} # End Function: Get-CidrHostCount.
PS> 1..32 | ForEach-Object {
>> Get-CidrHostCount -Cidr $_
>> }

Number of hosts for /1: 2,147,483,648
Number of hosts for /2: 1,073,741,824
Number of hosts for /3: 536,870,912
Number of hosts for /4: 268,435,456
Number of hosts for /5: 134,217,728
Number of hosts for /6: 67,108,864
Number of hosts for /7: 33,554,432
Number of hosts for /8: 16,777,216
Number of hosts for /9: 8,388,608
Number of hosts for /10: 4,194,304
Number of hosts for /11: 2,097,152
Number of hosts for /12: 1,048,576
Number of hosts for /13: 524,288
Number of hosts for /14: 262,144
Number of hosts for /15: 131,072
Number of hosts for /16: 65,536
Number of hosts for /17: 32,768
Number of hosts for /18: 16,384
Number of hosts for /19: 8,192
Number of hosts for /20: 4,096
Number of hosts for /21: 2,048
Number of hosts for /22: 1,024
Number of hosts for /23: 512
Number of hosts for /24: 256
Number of hosts for /25: 128
Number of hosts for /26: 64
Number of hosts for /27: 32
Number of hosts for /28: 16
Number of hosts for /29: 8
Number of hosts for /30: 4
Number of hosts for /31: 2
Number of hosts for /32: 1

And that’s it — numbers I can actually read. And yes, I finally did get back to studying for my exam.

Add Tags to Existing AWS Parameter Store Entries

While I work with AWS, it’s unfortunately not a regular, I’m-going-to-be-in-there-a-few-times-a-week kind of a thing. Perhaps that’ll change in time, but for now, I take myself in there occasionally when there’s something I want to try, use, fix, or experiment with. Today started when I ran up against a handful of Systems Manager Parameter Store entries I had created where I didn’t include any Tags. I didn’t want to be that guy, but I also didn’t want to be the guy that manually updated each of the 12 entries by hand. Yeah, that’s not me. There were two tags per entry. It’s not that many, but still.

The documentation to do this is available in the AWS PowerShell documentation. Even so, I’m going to share what I wrote and briefly discuss it. Perhaps someone will find it instead of, or in addition to, what’s already been written about it. Anyway, let’s get started. The below code is mostly one large, continuous code block, however, I’m breaking it up to discuss it.

The first code block creates and assigns a $CommonParams variable. This holds values that each of the AWS commands needs to have included. This is just an easier way to include these parameters and parameter values in each command. This variable, when included with an AWS command (as @CommonParams), will include the Region in which I’m working and the local AWS profile (on my computer) that I’m using to run these commands. Behind this profile is an Access Key Id and corresponding Secret Access Key.

$CommonParams = @{
	Region = 'us-west-2'
	ProfileName = 'tommymaynard_api@ecs-nonprod'
}

Using the below PowerShell we can isolate the Systems Manager Parameter Store entries that we want to modify. Each entry begins with a forward slash (/), but doesn’t begin with a forward-slash followed by fdn.

$Parameters = Get-SSMParameterList @CommonParams | Where-Object -FilterScript {
	$_.Name -like '/*' -and $_.Name -notlike '/fdn*'} | Select-Object -Property Name

This next section isn’t a part of the code. It’s being included, however, so that you’re able to view the values stored in the $Parameters variable. The entire code is included at the bottom of this evening’s post so that it’s easy to capture for those that are interested.

$Parameters

Name
----
/PowerShell/FunctionTemplate/Splunk/Logging/HEC_Token  
/PowerShell/FunctionTemplate/Splunk/Logging/HEC_URL    
/PowerShell/FunctionTemplate/Splunk/Telemetry/HEC_Token
/PowerShell/FunctionTemplate/Splunk/Telemetry/HEC_URL  
/ad_join/domain  
/ad_join/password
/ad_join/user    
/agents/duo/host 
/agents/duo/ikey 
/agents/duo/skey
/agents/omsagent/primarykey 
/agents/omsagent/workspaceid
/agents/sophos/linux_url        
/agents/tenable/nessus/host     
/agents/tenable/nessus/key      
/agents/tenable/nessus/linux_url

In order to apply Tags to each of the above Parameter Store entries, we need to first create them. In $Tag01 we’ll store the createdby key and its corresponding value. In $Tag02 we’ll store the contactid and its corresponding value.

$Tag01 = New-Object Amazon.SimpleSystemsManagement.Model.Tag
$Tag02 = New-Object Amazon.SimpleSystemsManagement.Model.Tag
$Tag01.Key = 'createdby'; $Tag01.Value = 'tommymaynard'
$Tag02.Key = 'contactid'; $Tag02.Value = $Tag01.Value

This section also isn’t a part of the code. It’s been included as verification that our Tags have been properly assigned, prior to being applied to each of the Parameter Store entries.

$Tag01; $Tag02

Key       Value
---       -----
createdby tommymaynard
contactid tommymaynard

This next code block returns the current Tags for each of the Parameter Store entries. Running this code here allows us to view the current Tags, if there are any, prior to adding the Tags we know we want on each of the entries.

$Parameters.Name | ForEach-Object {
	$_; Get-SSMResourceTag @CommonParams -ResourceType 'Parameter' -ResourceId $_; '---'
} # End ForEach-Object.

This is the code block that adds our Tags to each of the Parameter Store entries. We ensured we were working with the correct entries, we created our Tags—both the keys and the corresponding values—and now we’re applying them to each entry.

$Parameters.Name | ForEach-Object {
	Add-SSMResourceTag @CommonParams -ResourceType 'Parameter' -ResourceId $_ -Tag $Tag01,$Tag02
} # End ForEach-Object.

This is the same code block we saw two blocks earlier. All it does is return the Tags for each of the Parameter Store entries. It’s included again in order to review the changes since running the Add-SSMResourceTag command. While we might normally output this as PowerShell objects, I didn’t find that to be necessary since it was simple output that only I would see and then disregard.

$Parameters.Name | ForEach-Object {
	$_; Get-SSMResourceTag @CommonParams -ResourceType 'Parameter' -ResourceId $_; '---'
} # End ForEach-Object.

As mentioned, and since I stuck some not to be included code within the code, here’s the full code for adding new Parameter Store Tags entries.

$CommonParams = @{
	Region = 'us-west-2'
	ProfileName = 'tommymaynard_api@ecs-nonprod'
}

$Parameters = Get-SSMParameterList @CommonParams | Where-Object -FilterScript {
	$_.Name -like '/*' -and $_.Name -notlike '/fdn*'} | Select-Object -Property Name

$Tag01 = New-Object Amazon.SimpleSystemsManagement.Model.Tag
$Tag02 = New-Object Amazon.SimpleSystemsManagement.Model.Tag
$Tag01.Key = 'createdby'; $Tag01.Value = 'tommymaynard'
$Tag02.Key = 'contactid'; $Tag02.Value = $Tag01.Value

$Parameters.Name | ForEach-Object {
	$_; Get-SSMResourceTag @CommonParams -ResourceType 'Parameter' -ResourceId $_; '---'
} # End ForEach-Object.

$Parameters.Name | ForEach-Object {
	Add-SSMResourceTag @CommonParams -ResourceType 'Parameter' -ResourceId $_ -Tag $Tag01,$Tag02
} # End ForEach-Object.

$Parameters.Name | ForEach-Object {
	$_; Get-SSMResourceTag @CommonParams -ResourceType 'Parameter' -ResourceId $_; '---'
} # End ForEach-Object.

I didn’t specifically test whether this overwrote Tags of the same name that existed on each entry. That said, I believe it did in fact appear to overwrite them without a care, concern, or prompt. If this is important, then it would probably be wise to test this and/or write the conditional code to do something different, if that’s what you want.

AWS PowerShell Command Count

Back in the day, I was astonished by the number of PowerShell commands that AWS (Amazon Web Services) had written. It was a huge number—I believe it was around 5,000. There’s probably a post or two on this site where it’s mentioned. Based on the number, it was clear that AWS had made a commitment to (wrapping their APIs with) PowerShell. Back then, it was easy to calculate the number of commands because of that single, PowerShell module. Install the module, count the commands (programmatically of course), and done. Over the last two or three years—I’m not 100% sure—-they moved to a model where modules are separated out by service. I greatly suspect that the single-module model was no longer suitable. Who wants to import a few thousand commands by default, when they only needed maybe one or two? Over time, I suspect that it probably wasn’t the most efficient way in which to handle the high number of commands.

Now, AWS has modules with names that begin with AWS.Tools. In the below example, I have assigned the $AWSModules variable with all the AWS modules located in the PowerShell Gallery that begin with the name AWS.Tools.*.

$AWSModules = Find-Module -Name AWS.Tools.*
$AWSModules.Count
241
$AWSModules | Select-Object -First 10 -Property Name,Version
Name                              Version
----                              -------
AWS.Tools.Common                  4.1.10.0
AWS.Tools.EC2                     4.1.10.0
AWS.Tools.S3                      4.1.10.0
AWS.Tools.Installer               1.0.2.0
AWS.Tools.SimpleSystemsManagement 4.1.10.0
AWS.Tools.SecretsManager          4.1.10.0
AWS.Tools.SecurityToken           4.1.10.0
AWS.Tools.IdentityManagement      4.1.10.0
AWS.Tools.Organizations           4.1.10.0
AWS.Tools.CloudFormation          4.1.10.0

Once I determined that there were over 240 individual modules, I piped the $AWSModule variable to the Get-Member command to view its methods and properties. I didn’t know what I was after, but out of the returned properties, I was intrigued by the Includes property. I used that property as a part of the below command. Each of the 241 entries includes a Command, DscResource, Cmdlet, Workflow, RoleCapability, and Function nested property inside the Includes property.

$AWSModules.Includes | Select-Object -First 3
Name Value
---- -----
Command {Clear-AWSHistory, Set-AWSHistoryConfiguration, Initialize-AWSDefaultConfiguration, Clear-AWSDefaultConfiguration…}
DscResource {}
Cmdlet {Clear-AWSHistory, Set-AWSHistoryConfiguration, Initialize-AWSDefaultConfiguration, Clear-AWSDefaultConfiguration…}
Workflow {}
RoleCapability {}
Function {}
Command {Add-EC2CapacityReservation, Add-EC2ClassicLinkVpc, Add-EC2InternetGateway, Add-EC2NetworkInterface…}
DscResource {}
Cmdlet {Add-EC2CapacityReservation, Add-EC2ClassicLinkVpc, Add-EC2InternetGateway, Add-EC2NetworkInterface…}
Workflow {}
RoleCapability {}
Function {}
Command {Add-S3PublicAccessBlock, Copy-S3Object, Get-S3ACL, Get-S3Bucket…}
DscResource {}
Cmdlet {Add-S3PublicAccessBlock, Copy-S3Object, Get-S3ACL, Get-S3Bucket…}
Workflow {}
RoleCapability {}
Function {}

After some more inspection, I decided that each command populated the Command property and the Cmdlet or the Function property, as well, depending on what type of commands were included in the module. Next, I just returned the commands using dot notation. This isn’t all of them, but you get the idea.

$AWSModules.Includes.Command
Clear-AWSHistory
Set-AWSHistoryConfiguration
Initialize-AWSDefaultConfiguration
Clear-AWSDefaultConfiguration
Get-AWSPowerShellVersion
...
Remove-FISExperimentTemplate
Remove-FISResourceTag
Start-FISExperiment
Stop-FISExperiment
Update-FISExperimentTemplate

If you’re curious, as I was, Update-FISExperimentTemplate, “Calls the AWS Fault Injection Simulator UpdateExperimentTemplate API operation.” I have no idea.

With a little more dot notation, I was able to get the count.

$AWSModules.Includes.Command.Count
8942

And now, I know what I used to know. I also know that AWS has been busy. And, that they’ve continued to make a huge effort in the PowerShell space. If you don’t go straight to the PowerShell Gallery, and I would recommend you do, you can always start at the AWS PowerShell webpage.

AWS re:Invent – AWS Tools for PowerShell

Earlier this week at the AWS re:Invent conference, I was able to sit alongside another PowerShell enthusiast in a workshop-based PowerShell session. That’s a laptop-open-work-through-a-scenario-with-your-table type of session. Today’s a little different.

Because of a session, I took yesterday (on Lambda and Step Functions), I know a little about today’s format. If all goes well for me getting in there, the session I’ll sit through in the next hour is called, “AWS Tools for PowerShell.”

In this event, a small group of maybe five or six people will each sit with an instructor and work through an example on whatever topic that table covers. While yesterday I was looking for a lecture (after another hands-on event), I simply didn’t get that. I choose incorrectly and did the whole small table hands-on thing. There was no traditional lecture.

Wish me luck, as again there’s no guarantee I’m getting into this session. That’s really starting to potentially come clear for me… ugh. I did pass the AWS Certified Solutions Architect – Associate exam, so at least I’ve got that, I suppose.

Update: Well, I got a chair at the PowerShell table. We created a VPC, etc., etc. all using the PowerShell Core AWS PowerShell module. It has a rather interesting name and it’s escaping me at the moment. It was a good walk though and it did provide me with something I haven’t actually done in AWS with PowerShell.

AWS re:Invent (and PowerShell, of Course)

It’s 11:30 p.m. where I’m from, so just perhaps that’s why I’m showered, in my hotel room, and ready to study for an AWS certification I’ll be taking in a couple of days. It’s 10:30 p.m. here in Las Vegas. Or maybe, while there’s a bunch of people set to do some sort of chicken wing eating contest a hotel or two over, I’m in my room and preparing to go to sleep, just before I quickly write out a new blog post, because I’m getting old. That’s probably it. There are a couple of things, I wanted to quickly mention.

One, it’s been a few years since I’ve been to one of these gigantic conferences. I’ve forgotten what it’s like, and honestly, I’m not even sure if those Microsoft TechEd conferences of the past, match up to the 45,000-person event this is set to be.

While I won’t be at the PowerShell and DevOps Global Summit 2019, I’ve been to a couple and after returning to something this big, I have something to say. The smaller ones have something these don’t. Or at least, don’t seem to have at the moment. I get it; it’s early still; chicken wings or not, the thing hasn’t really begun. But anyway, for those on the fence about going to the PowerShell conference, if that’s even possible for you, do it! Well, unless you want to ruin the ones with a billion people.

Two, a PowerShell community member contacted me on Twitter today and we’re meeting up for lunch tomorrow. How great is that!? And after that, we’re off to a PowerShell session. You read that right. Just because I’m at an AWS conference doesn’t mean I haven’t found some PowerShell content to devour. It’s called “Hands-On: Automating AWS Infrastructure with PowerShell.” To me, there’s no question that AWS has long accepted PowerShell. There are 4,499 PowerShell cmdlets in their AWSPowerShell module, and PowerShell was recently added to Lambda. There’s some commitment there. Oh! And one time, they were highly responsive and even created the Remove-EC2Instance cmdlet on my suggestion: https://tommymaynard.com/more-aws-powershell-changes-due-to-twitter-and-me-2016.

Looking forward to tomorrow and that sweet combination of AWS, PowerShell, and PowerShell community member meetups. Now to bed, and maybe I’ll still study.

Those AWS Region Commands

More and more, Amazon Web Services has become a significant part of my day. Luckily for me, PowerShell and AWS, work well together. There’s long been the AWSPowerShell module, which much like the AWS CLI, allows us to issue API calls to AWS from a command line.

As a part of continuing my journey into AWS, and maintaining my mild obsession with PowerShell, I’ve decided to better learn a few PowerShell cmdlets, from the AWSPowerShell module throughout at least a couple of posts. As of version 3.3.365, the module only contains a few thousand cmdlets. It seems like AWS has gone ahead and made an investment here in PowerShell. Especially, when it wasn’t even terribly long ago that there were only 2,000.

(Get-Command -Module AWSPowershell | Measure-Object).Count
4499

Oh yeah, that reminds me, Lambda supports PowerShell (Core) now, too. As I read somewhere recently, “It’s not Bash, it’s not Ruby. It’s PowerShell.”

In a few previous, AWS-specific posts I was able to point out some things I thought should be changed. And somehow, AWS paid close enough attention, that some changes were actually made. It’s still hard to believe; it’s a bit surreal.

Hashtag AWS Tweet Prompts Fix to AWSPowerShell Module
AWS Stop-EC2Instance Needs Complimentary Cmdlet
More AWS PowerShell Changes Due to Twitter (and Me)

I’m mostly writing this evening to help solidify a few commands for my own education, and anyone else who is reading along. But… as I experimented with some of these AWS PowerShell cmdlets, I couldn’t help but feel that some changes were in order. So, with that knowledge, let’s review a few Region-specific cmdlets and touch on some potential changes, as they make the most sense to me.

Get-AWSRegion: “Returns the set of available AWS regions.”

When the Get-AWSRegion cmdlet is invoked, it returns the Region Name (the full name), the Region (abbreviated Region Name), and whether or not the Region is the default in the shell. In this first example, you can see that the default output returns all the Regions.

Get-AWSRegion

Region         Name                      IsShellDefault
------         ----                      --------------
ap-northeast-1 Asia Pacific (Tokyo)      False
ap-northeast-2 Asia Pacific (Seoul)      False
ap-south-1     Asia Pacific (Mumbai)     False
ap-southeast-1 Asia Pacific (Singapore)  False
ap-southeast-2 Asia Pacific (Sydney)     False
ca-central-1   Canada (Central)          False
eu-central-1   EU Central (Frankfurt)    False
eu-west-1      EU West (Ireland)         False
eu-west-2      EU West (London)          False
eu-west-3      EU West (Paris)           False
sa-east-1      South America (Sao Paulo) False
us-east-1      US East (Virginia)        False
us-east-2      US East (Ohio)            False
us-west-1      US West (N. California)   False
us-west-2      US West (Oregon)          False

Wrong. Be sure to take a look at the examples further below. As you’ll see there are a couple of parameters—IncludeChina and IncludeGovCloud—that add some Regions that aren’t there by default. I’m not suggesting a change here, mostly, but Get-AWSRegion should return all the Regions, right?

Based on the cmdlet name alone, I suspected that all Regions were going to be listed in the output that’s returned. Good thing I looked into this cmdlet’s parameters, instead of assuming that the Regions were actually all included. And why China? I get why we might make an exception for GovCloud—we shouldn’t—but what was the thought in regard to China? You’ll see what I mean in the following examples.

(Get-AWSRegion | Measure-Object).Count
15
(Get-AWSRegion -IncludeChina | Measure-Object).Count
17
(Get-AWSRegion -IncludeChina -IncludeGovCloud | Measure-Object).Count
18

Now, let’s take a look at the SystemName parameter included in Get-AWSRegion. This is where it becomes quickly evident to me that we can definitely do better. Why does the Region property use a parameter called SystemName? I think maybe that parameter needs a Region parameter alias, at minimum.

Get-AWSRegion -SystemName us-west-1

Region    Name                    IsShellDefault
------    ----                    --------------
us-west-1 US West (N. California) False

Get-AWSRegion -SystemName us-west-2

Region    Name             IsShellDefault
------    ----             --------------
us-west-2 US West (Oregon) False

Get-AWSRegion -SystemName us-west-*

Region    Name    IsShellDefault
------    ----    --------------
us-west-* Unknown False

I didn’t spend any time reading the help for Get-AWSRegion, but as you can see directly above, the wildcard character isn’t supported with the SystemName parameter. That’s too bad. That would’ve been a great addition to this cmdlet (and one that can still be added). To use a wildcard character against this value, you’re required to pipe your output to the Where-Object cmdlet and therefore, filter it further down the pipeline. Yes, this does mean that all results are piped to Where-Object, whereas a wildcard character built in to Get-AWSRegion, would filter immediately and avoid the need for the pipeline. The pipeline is crucial to the language, but when it’s not needed, we’re better off.

Get-AWSRegion | Where-Object Region -like 'us-west-*'

Region    Name                    IsShellDefault
------    ----                    --------------
us-west-1 US West (N. California) False
us-west-2 US West (Oregon)        False

And if you’re wondering, Get-AWSRegion doesn’t include a Name parameter, so there’s no checking there for the ability to use wildcards.

Get-DefaultAWSRegion: “Returns the current default AWS region for this shell, if any, as held in the shell variable $StoredAWSRegion.”

This command’s purpose, as indicated, returns the Region the AWSPowerShell module’s commands should use by default. If the command returns nothing, then a default hasn’t been set. If it does, then someone has likely already used the command we’ll discuss after Get-DefaultAWSRegion: Set-DefaultAWSRegion.

Get-DefaultAWSRegion
# Nothing here yet...

Before we move on, the commands that include the string “Default” before “AWS,” should have instead maintained the same noun prefix as Get-AWSRegion. That’s right, each of these three cmdlets should’ve included the verb, the dash, and then the string AWS, before the remaining portion of the included nouns. Why in the world would we stuff “AWS” into the middle of some command names and at the beginning of others? Amazon should have maintained a consistent prefix. Every Microsoft Active Directory command uses an AD prefix, right after the dash. You’ll never find it anywhere else. Even in the office, the prefix we use on our self-written functions is right where we’ve been trained to expect it:

<ApprovedVerb>-<Dept><SingularNoun(s)InCamelCase>

In my experience, AWS isn’t afraid of using command aliases—so one command can resolve to another—discontinuing the use of parameter names at times, and changing cmdlet names altogether. Therefore, I suspect someone needs to revisit these three. It’s not like Get-AWSRegionDefault, Set-AWSRegionDefault, and Clear-AWSRegionDefault don’t make sense and are already being used. The current commands should be aliases to these three, keeping the prefixes in the proper place.

Get-DefaultAWSRegion -> Get-AWSRegionDefault
Set-DefaultAWSRegion -> Set-AWSRegionDefault
Clear-DefaultAWSRegion -> Clear-AWSRegionDefault

While we’re here, we need to stop using the plural form of any nouns. That said, I do recognize this move is happening, such that Get-AWSCredentials is an alias that resolves to Get-AWSCredential. Oh look at that, the AWS prefix is in the right place on those two!

Set-DefaultAWSRegion: “Sets a default AWS region system name (e.g. us-west-2, eu-west-1 etc) into the shell variable $StoredAWSRegion. AWS cmdlets will use the value of this variable to satisfy their -Region parameter if the parameter is not specified.”

The first thing to notice here is that we have a Region parameter. That’s what Get-AWSRegion should’ve been included (in addition to SystemName [as not to create a breaking change]). Maybe make Region the actual parameter, and SystemName an alias to the Region parameter. That sounds like the best way to phase out that parameter.

Set-DefaultAWSRegion -Region ca-central-1
Get-DefaultAWSRegion

Region       Name             IsShellDefault
------       ----             --------------
ca-central-1 Canada (Central) True 

Clear-DefaultAWSRegion: “Clears any default AWS region set in the shell variable $StoredAWSRegion.”

Get-DefaultAWSRegion
Region       Name             IsShellDefault
------       ----             --------------
ca-central-1 Canada (Central) True
Clear-DefaultAWSRegion
Get-DefaultAWSRegion
# Nothing here again.

This evening we covered some Region cmdlets from the AWSPowerShell module. They all do what they should in the end, but in my mind, there’s some room for some changes for consistency’s sake and overall improvement. Perhaps we’ll do this again… there are some credential-related AWS cmdlets I’m going to need to learn once. and. for. all.