Tag Archives: AWS

PowerShell Code and AWS CloudFormation UserData

Note: This post was written well over a month ago, but was never posted, due to some issues I was seeing in AWS GovCloud. It works 100% of the time now, in both GovCloud and non-GovCloud AWS. That said, if you’re using Read-S3Object in GovCloud, you’re going to need to include the Region parameter name and value.

As I spend more and more time with AWS, I end up back at PowerShell. If I haven’t said it yet, thank you Amazon Web Services, for writing us a PowerShell module.

In the last month, or two, I’ve been getting into the CloudFormation template business. I love the whole UserData option we have — injecting PowerShell code into an EC2 instance during its initialization, and love, that while we can do it in the AWS Management Console, we can do it with CloudFormation (CFN) too. In the last few months, I’ve decided to do things a bit differently. Instead of dropping large amounts of PowerShell code inside my UserData property in the CFN template, I decided to use Read-S3Object to copy PowerShell modules to EC2 instances, and then just issue calls to the functions in the remainder of the CFN UserData. In one instance, I went from 200+ lines of PowerShell in the CFN, to just a few.

To test I needed to verify if I could get a module folder and file into the proper place on the instance and be able to use the module’s function(s) immediately, without any need to end one PowerShell session, and start a new one. I suspected this would work just fine, but it needed to be seen.

Here’s how the testing went: On my Desktop, I have a folder called MyModule. Inside the folder I have a file called MyModule.psm1. If you haven’t seen it before, this file extension indicates the file is a PowerShell module file. The contents of the file, are as follows:

Function Get-A {
    'A'
}

Function Get-B {
    'B'
}

Function Get-C {
    'C'
}

The file contents indicate that the module contains three functions: Get-A, Get-B, and Get-C. In the next example, we can see that the Desktop folder isn’t a place where a module file and folder can exist, where we can expect that the modules will be automatically loaded into the PowerShell session. PowerShell isn’t aware of this module on its own, as can be seen below.

PS > Get-A
Get-A : The term 'Get-A' is not recognized as the name of a cmdlet, function, script file, or operable program. Check
the spelling of the name, or if a path was included, verify that the path is correct and try again.
At line:1 char:1
+ Get-A
+ ~~~~~
    + CategoryInfo          : ObjectNotFound: (Get-A:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

PS > Get-B
Get-B : The term 'Get-B' is not recognized as the name of a cmdlet, function, script file, or operable program. Check
the spelling of the name, or if a path was included, verify that the path is correct and try again.
At line:1 char:1
+ Get-B
+ ~~~~~
    + CategoryInfo          : ObjectNotFound: (Get-B:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

PS > Get-C
Get-C : The term 'Get-C' is not recognized as the name of a cmdlet, function, script file, or operable program. Check
the spelling of the name, or if a path was included, verify that the path is correct and try again.
At line:1 char:1
+ Get-C
+ ~~~~~
    + CategoryInfo          : ObjectNotFound: (Get-C:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

While I could tell PowerShell to look on my desktop, what I wanted to do is have my CFN template copy the module folder out of S3 and place it on the instances, in a preferred and proper location: “C:\Program Files\WindowsPowerShell\Modules.” This is a location that PowerShell checks for modules automatically, and loads them the moment a contained function, or cmdlet, from the module is requested. My example uses a different path, but PowerShell will check here automatically, as well. As a part of this testing, we’re pretending that the movement from my desktop is close enough to the movement from S3 to EC2 instance. I’ll obviously test this more with AWS.

PS > Move-Item -Path .\Desktop\MyModule\ -Destination C:\Users\tommymaynard\Documents\WindowsPowerShell\Modules\
PS > Get-A
A
PS > Get-B
B
PS > Get-C
C

Without the need to open a new PowerShell session, I absolutely could use the functions in my module, the moment the module was moved from the Desktop into a folder PowerShell looks at by default. Speaking of those locations, you can view them by returning the value in the $env:PSModulePath environmental variable. Use $env:PSModulePath -split ‘;’ to make it easier to read.

Well, it looks like I was right. I can simply drop those modules folders on the EC2 instance, into “C:\Program Files\WindowsPowerShell\Modules,” just before they’re used with no need for anything more but the current PowerShell session that’s moving them into place.

Update: After this on-my-own-computer test, I took it to AWS. It works, and now it’s the only way I use my CFN template UserData. I write my my function(s), house them in a PowerShell module(s), copy them to S3, and finally use my CFN UserData to copy them to the EC2 instance. When that’s complete, I can call the contained function(s) without any hesitation, or additional work. It wasn’t necessary, but I added sleep commands between the function invocations. Here’s a quick, modified example you might find in the UserData of one of my CloudFormation templates.

      UserData:
        Fn::Base64:
          !Sub |
          <powershell>
            # Download PowerShell Modules from S3.
            $Params = @{
              BucketName = 'windows'
              Keyprefix = 'WindowsPowerShell/Modules/ProjectVII/'
              Folder = "$env:ProgramFiles\WindowsPowerShell\Modules"
            }
            Read-S3Object @Params | Out-Null

            # Invoke function(s).
            Set-TimeZone -Verbose -Log
            Start-Sleep -Seconds 15

            Add-EncryptionType -Verbose -Log
            Start-Sleep -Seconds 15

            Install-ProjectVII -Verbose -Log
            Start-Sleep -Seconds 15
            </powershell>

Silent Install from an ISO

In the last several weeks, I’ve been having a great time writing PowerShell functions and modules for new projects moving to Amazon Web Services (AWS). I’m thrilled with the inclusion of UserData as a part of provisioning an EC2 instance. Having developed my PowerShell skills, I’ve been able to leverage them in conjunction with UserData to do all sorts of things to my instances. I’m reaching into S3 for installers, expanding archive files, creating folders, bringing down custom written modules in UserData and invoking the contained functions from them there, too. I’m even setting the timezone. It’s seems so straight forward sure, but getting automation and logging wrapped around that need, is rewarding.

As a part of an automated SQL installation — yes, the vendor told me they don’t support AWS RDS — I had a new challenge. It wasn’t overly involved by any means, but it’s worthy of sharing, especially if someone hits this post in a time of need, and gets a problem solved. I’ve said it a millions times: I often write, so I have a place to put things I may forget, but truly, it’s about anyone else I can help, as well. I’ve been at that almost three years now.

Back to Microsoft SQL: It’s on an ISO. I’ve been pulling down Zip files for weeks, in various projects, with CloudFormation, and expanding them, but this was a new one. I needed to get at the files in that ISO to silently run an installation. Enter the Mount-DiskImage function from Microsoft’s Storage module. Its help synopsis says this: “Mounts a previously created disk image (virtual hard disk or ISO), making it appear as a normal disk.” The command to pull just that help information is listed below.

PS > (Get-Help -Name Mount-DiskImage).Synopsis

As I typically do, I started working with the function in order to learn how to use it. It works as described. Here’s the command I used to mount my ISO.

PS > Mount-DiskImage -ImagePath 'C:\Users\tommymaynard\Desktop\SQL2014.ISO'

The above example doesn’t produce any output by default, and I rather like it that way. After a dismount — it’s the same above command with Dismount-DiskImage instead of Mount-DiskImage — I tried it with the -PassThru parameter. This parameter returns an object with some relevant information.

PS > Mount-DiskImage -ImagePath 'C:\Users\tommymaynard\Desktop\SQL2014.ISO' -PassThru

Attached          : False
BlockSize         : 0
DevicePath        :
FileSize          : 2606895104
ImagePath         : C:\Users\tommymaynard\Desktop\SQL2014.ISO
LogicalSectorSize : 2048
Number            :
Size              : 2606895104
StorageType       : 1
PSComputerName    :

The first thing I noticed about this output is that it didn’t provide the drive letter used to mount the ISO. I was going to need that drive letter in PowerShell, in order to move to that location and run the installer. Even if I didn’t move to that location, I needed the drive letter to create a full path. The drive letter was vital, and this, is why we’re here today.

Update: See the below post replies where Get-Volume is used to discover the drive letter.

Although the warmup here seemed to take a bit, we’re almost done here for today. I’ll drop the code below, and we’ll do a quick, line-by-line walk through.

# Mount SQL ISO and run setup.exe.
PS > $DrivesBeforeMount = (Get-PSDrive).Name
PS >
PS > Mount-DiskImage -ImagePath 'C:\Users\tommymaynard\Desktop\SQL2014.ISO'
PS >
PS > $DrivesAfterMount = (Get-PSDrive).Name
PS >
PS > $DriveLetterUsed = (Compare-Object -ReferenceObject $DrivesBeforeMount -DifferenceObject $DrivesAfterMount).InputObject
PS >
PS > Set-Location -Path "$DriveLetterUsed`:\"

Line 2: The first command in this series, stores the name property of all the drives in our current PowerShell session in a variable named $DrivesBeforeMount. That name should offer some clues.

Line 4: This line should look familiar; it mounts our SQL 2014 ISO (to a mystery drive letter).

Line 6: Here, we run the same command as in Line 2, however, we store the results in $DrivesAfterMount. Do you see what we’re up to yet?

Line 8:  This command compares our two recently created Drive* variables. We want to know which drive is there now, that wasn’t when the first Get-PSDrive command was run.

Line 10: And finally, now that we know the drive letter used for our newly mounted ISO, we can move there in order to access the setup.exe file.

Okay, that’s it for tonight. Now back to working on a silent SQL install on my EC2 instance.

Get the AWS Noun Prefixes

One of the nice things about the Get-AWSPowerShellVersion cmdlet is the ListServiceVersionInfo switch parameter. It returns properties for the Service (as in the AWS Service offering name), the “Noun Prefix,” and “API Version.” Yes, there really are spaces in those last two property names; however, they’ve been fixed in my function. I had been hoping for an easier way to determine the prefixes used in the AWS cmdlets, and here we have it. I actually considered parsing cmdlet names myself, so a huge thanks to AWS, for making sure that wasn’t necessary.

It’s almost as though this should have been its own cmdlet though — a get the version cmdlet and a get the noun prefixes cmdlet. Therefore, I’ve wrapped this command and its parameter in a quick and easy to use function for my user base. Copy, paste, try it out; it’s all yours. Super simple.

Function Get-AWSNounPrefix {
<#
.SYNOPSIS
    The function returns the AWS PowerShell cmdlet noun prefixes, along with the corresponding AWS Service.

.DESCRIPTION
    The function returns the AWS PowerShell cmdlet noun prefixes, along with the corresponding AWS Service name. This function utilizes the AWSPowerShell module's Get-AWSPowerShellVersion function.

.EXAMPLE
    -------------------------- EXAMPLE 1 --------------------------
    PS > Get-AWSNounPrefix
    This examples returns all the noun prefixes and their corresponding AWS Service name.

.EXAMPLE
    -------------------------- EXAMPLE 2 --------------------------
    PS > Get-AWSNounPrefix | Where-Object NounPrefix -match 'cf'
    NounPrefix Service            APIVersion
    ---------- -------            ----------
    CF         Amazon CloudFront  2016-09-29
    CFG        AWS Config         2014-11-12
    CFN        AWS CloudFormation 2010-05-15

    This example uses the Where-Object cmdlet and the Match operator to filter the results by the NounPrefix.

.EXAMPLE
    -------------------------- EXAMPLE 3 --------------------------
    PS > Get-AWSNounPrefix | Where-Object Service -like '*formation*'
    NounPrefix Service            APIVersion
    ---------- -------            ----------
    CFN        AWS CloudFormation 2010-05-15

    This example uses the Where-Object cmdlet and the Like operator to filter the results by the service name.

.EXAMPLE
    -------------------------- EXAMPLE 4 --------------------------
    PS > Get-AWSNounPrefix | Where-Object -Property APIVersion -like '2016*' | Sort-Object -Property APIVersion -Descending
    NounPrefix Service                          APIVersion
    ---------- -------                          ----------
    SMS        Amazon Server Migration Service  2016-10-24
    BGT        AWS Budgets                      2016-10-20
    CF         Amazon CloudFront                2016-09-29
    EC2        Amazon Elastic Compute Cloud     2016-09-15
    SNOW       AWS Import/Export Snowball       2016-06-30
    CGIP       Amazon Cognito Identity Provider 2016-04-18
    INS        Amazon Inspector                 2016-02-16
    AAS        Application Auto Scaling         2016-02-06
    MM         AWS Marketplace Metering         2016-01-14
    DMS        AWS Database Migration Service   2016-01-01

    This example uses the Where-Object, and Sort-Object cmdlet, to find services updated in 2016 and sorts by the most recently added and updated.

.NOTES
    Name: Get-AWSNounPrefix
    Author: Tommy Maynard
    Comments: Current version of AWSPowerShell module at the first, last edit: 3.3.20.0.
    Last Edit: 11/18/2016
    Version 1.0
#>
    [CmdletBinding()]
    Param (
    )

    Begin {
        $AWSServices = Get-AWSPowerShellVersion -ListServiceVersionInfo | Sort-Object -Property 'Noun Prefix'
    } # End Begin.

    Process {
        Foreach ($AWSService in $AWSServices) {
            [PSCustomObject]@{
                NounPrefix = $AWSService.'Noun Prefix'
                Service = $AWSService.Service
                APIVersion = $AWSService.'API Version'
            }
        } # End Foreach.
    } # End Process.

    End {
    } # End End.
} # End Function: Get-AWSNounPrefix.

 

Compare AWSPowerShell Versions II

Series links: Part I, Part II, Part III (Coming)

I wrote a post a day or two ago that indicated how to compare your currently installed AWSPowerShell module with the offering on the PowerShell Gallery (see Part I above). Well, as I suspected, I’m back with an update.

Before we get there, however, I want to mention that Steve Roberts, at AWS, got in touch with me via Twitter about the post. I suspect we’ll be seeing some updates to Get-AWSPowerShellVersion. What’s interesting, is that I found the ListServiceVersionInfo parameter after my post. It seems that the cmdlet does create objects for some of the results it can return, just not everything. As always, I’m looking forward to Steve’s ideas and implementation.

Today, I’m going to share a quickly written advanced function. This will do the comparison for you and return the results in a custom object. Don’t be surprised if I make some more additions and/or changes to the function and post that as well. It really should have an option to download and install the new module if you want it, right?

Function Compare-AWSPowerShellVersion {
<#
.SYNOPSIS
    This advanced function compares the currently installed version of the AWSPowerShell module and the version on the PowerShell Gallery (http://powershellgallery.com).

.DESCRIPTION
    This advanced function compares the currently installed version of the AWSPowerShell module and the version on the PowerShell Gallery (http://powershellgallery.com).

.EXAMPLE
    PS > Compare-AWSPowerShellVersion
    This example compares the currently installed version of the AWSPowerShell module and the version on the PowerShell Gallery.

.NOTES 
    NAME: Compare-AWSPowerShellVersion
    AUTHOR: Tommy Maynard
    COMMENTS: --
    LASTEDIT: 10/19/2016
    VERSION 1.1:
        - Edited notes.
        - Changed Switch to -regex and made first option an OR for both side indicators.
    VERSION 1.2:
        - Modified results to use a custom object.
    VERSION 1.3:
        - Removed Switch statement: added version determination logic inside custom object creation.
        - Only returning highest version number from currently installed if more than one version. This is a leftover due to AWS Toolkit install vs. PowerShell Gallery install.
#>
    [CmdletBinding()]
    Param (
    )

    Begin {
        # Set continuation variable for Process block.
        $Continue = $true

        'PowerShellGet','AWSPowerShell' | ForEach-Object {
            Write-Verbose -Message "Determining if the $_ module is available."
            If (-Not(Get-Module -Name $_ -ListAvailable)) {
                Write-Warning -Message "Unable to locate the required $_ module."
                $Continue = $false
            }
        } # End Foreach.
    } # End Begin.

    Process {
        If ($Continue) {
            Write-Verbose -Message 'Collecting the current and newest AWSPowerShell module versions.'
            $CurrentAWSPSModule = ((Get-Module -Name AWSPowerShell -ListAvailable).Version |
                Sort-Object -Descending | Select-Object -First 1).ToString()
            $NewestAWSPSModule = (Find-Module -Name AWSPowerShell).Version.ToString()

            Write-Verbose -Message 'Comparing the AWSPowerShell version and creating custom results object.'
            [PSCustomObject]@{
                Match = If ($CurrentAWSPSModule -eq $NewestAWSPSModule) {$true} Else {$false}
                Current = $CurrentAWSPSModule
                Newest = $NewestAWSPSModule
            }
        } # End If.
    } # End Process.

    End {
    } # End End.
} # End Function: Compare-AWSPowerShellVersion.

Update: After updating to the newest version, I was returning a System.Object[] error for my current version, due to having multiple versions and using the ToString method. I’ve modified the code above to only select the first (and highest) module version from those on my system. I think it’s important to point out that I’ve always downloaded and installed using the .msi until this version, and that when I used Install-Module, I included the Scope parameter name with the CurrentUser parameter value. I suspect this is where the problem is originating. Well, this and the fact I haven’t uninstalled the .msi version from Programs and Features.

I shouldn’t have to wait too long to be on an old build to test some more! It seems as though AWS does at least one, daily build. Here’s the full command as I ran it to install the newest version: Find-Module -Name AWSPowerShell | Install-Module -Scope CurrentUser.

Here’s the usage and results since updating to 3.3.11.0.

PS > Compare-AWSPowerShellVersion

Match Current  Newest  
----- -------  ------  
 True 3.3.11.0 3.3.11.0

Update: As suspected, there’s a new build this evening, and it appears the comparison function is still working, as it’s correctly indicated there isn’t a match between what’s installed and what’s available to install.

PS > Compare-AWSPowerShellVersion

Match Current  Newest
----- -------  ------
False 3.3.11.0 3.3.12.0

I updated to the newest version of the module and ran it again. It’s looking good.

PS > Compare-AWSPowerShellVersion

Match Current  Newest
----- -------  ------
 True 3.3.12.0 3.3.12.0

Compare AWSPowerShell Versions

Series links: Part I, Part II, Part III (Coming)

Well, I wrote enough example code inside my PowerShell ConsoleHost recently, to write another post. That’s typically how these works. Find something great… write. Find something I hate… write. Most of all, just write. Put it on Twitter and help people learn — someone did it for me once.

Today’s goal was to compare my currently installed version of the AWSPowerShell module with the newest version of the AWSPowerShell module. We’ll do this a bit backwards, and start with obtaining the newest version of the AWSPowerShell Module first. Because AWS has elected to put their module on the PowerShell Gallery — thanks, guys — we don’t have to parse the AWS PowerShell home page. I’m not sure if it’s even there, but luckily for us, it doesn’t need to be, and so I don’t need to try.

PS &gt; Find-Module -Name AWSPowerShell

Version    Name                                Repository           Description
-------    ----                                ----------           -----------
3.3.9.0    AWSPowerShell                       PSGallery            The AWS Tools for Windows PowerShell lets develo...

PS &gt; (Find-Module -Name AWSPowerShell).Version

Major  Minor  Build  Revision
-----  -----  -----  --------
3      3      9      0

PS &gt; (Find-Module -Name AWSPowerShell).Version.ToString()
3.3.9.0

Now that we can obtain the version number of the newest release, let’s work on getting the currently installed version number. My first thought was to use Get-AWSPowerShellVersion. It was a mess, but here’s what I did. This cmdlet does not return an object, unfortunately (and should be replaced and/or corrected).

Seriously, we expect that cmdlets and functions will return a usable object, or objects. If we really need additional information stuffed in the results, let’s only include it with the Verbose parameter. Maybe just drop all that text in a file and have Verbose indicate the file to open. Maybe make a secondary cmdlet. Anything really, that doesn’t require that we parse text… which is exactly what I did (at first).

PS &gt; ((Get-AWSPowerShellVersion).Split('`n')[2])[1..7] -join ''
3.3.0.0
PS &gt; [version](((Get-AWSPowerShellVersion).Split('`n')[2])[1..7] -join '')

Major  Minor  Build  Revision
-----  -----  -----  --------
3      3      0      0

The problem with this approach is that it’s much too exact. It’s grabbing certain characters. What happens when the version is longer than seven characters? We would’ve lost our final zero, if I was still on a previous version, such as 3.1.66.0, 3.1.71.0, or 3.1.73.0. Not good enough.

I knew the AWSPowerShell module put something somewhere; every module does. I browsed over to “C:\Program Files (x86)\AWS Tools\PowerShell\AWSPowerShell” to take a look around. Maybe there was something there that would be more reliable. I quickly spotted the module manifest file: AWSPowerShell.psd1. I opened it up in a text editor, and as suspected, there was the version… right inside that beautiful hash table. I closed up the file, and used Test-ModuleManifest inside my ConsoleHost.

PS &gt; $Path = 'C:\Program Files (x86)\AWS Tools\PowerShell\AWSPowerShell\AWSPowerShell.psd1'
PS &gt; Test-ModuleManifest -Path $Path

ModuleType Version    Name                                ExportedCommands
---------- -------    ----                                ----------------
Binary     3.3.0.0    AWSPowerShell                       {Clear-AWSHistory, Set-AWSHistoryConfiguration, Initialize...

PS &gt; (Test-ModuleManifest -Path $Path).Version

Major  Minor  Build  Revision
-----  -----  -----  --------
3      3      0      0

PS &gt; (Test-ModuleManifest -Path $Path).Version.ToString()
3.3.0.0

That’s way more reliable. I’d much rather blame AWS if it doesn’t return the correct version, than my parsing against the results of Get-AWSPowerShellVersion. I mean, seriously.

If you’ve been following my writings, then it’ll come as no surprise that I started with the hardest way and overlooked the obvious. I could’ve simply just used Get-Module to return the version; it’s getting its information from the .psd1 file. As obnoxious as this is, it’s keeping me sharp. I get to repeatedly practice what I know, and give my mind time to sort out different resolutions to the same problem. Here’s how I returned the version of the currently installed AWSPowerShell module.

PS &gt; Get-Module -Name AWSPowerShell

ModuleType Version    Name                                ExportedCommands
---------- -------    ----                                ----------------
Binary     3.3.0.0    AWSPowerShell                       {Add-AASScalableTarget, Add-ACMCertificateTag, Add-ASAAtta...

PS &gt; (Get-Module -Name AWSPowerShell -ListAvailable).Version

Major  Minor  Build  Revision
-----  -----  -----  --------
3      3      0      0

PS &gt; (Get-Module -Name AWSPowerShell -ListAvailable).Version.ToString()
3.3.0.0

Now I can return both the currently installed version of the AWSPowerShell module, and the newest version of the AWSPowerShell module from the PowerShell Gallery. Before we compare them, notice that in the above examples that I did and didn’t use the -ListAvailable parameter. If you don’t use it, you better be absolutely certain the module has already been imported.

So, let’s get these version numbers into a couple variables and compare them.

PS &gt; $CurrentAWSPSModule = (Get-Module -Name AWSPowerShell -ListAvailable).Version.ToString()
PS &gt; $CurrentAWSPSModule
3.3.0.0
PS &gt; $NewestAWSPSModule = (Find-Module -Name AWSPowerShell).Version.ToString()
PS &gt; $NewestAWSPSModule
3.3.9.0
PS &gt; Compare-Object -ReferenceObject $CurrentAWSPSModule -DifferenceObject $NewestAWSPSModule -IncludeEqual

InputObject SideIndicator
----------- -------------
3.3.9.0     =&gt;
3.3.0.0     &lt;=

As we can tell, there is a difference between my version and the one in the PowerShell Gallery. I should probably download the newest version. Watch for a follow up to this article, as I may go ahead and write a second part, as I have some other ideas.

AWS EC2 Instance Type Count

There’s a project that I’m on that allows me to work in AWS. It’s pretty important stuff at work, and since there’s an AWS PowerShell module, I couldn’t be more interested. As of today, I’m going to start to include worthy AWS PowerShell work, right here. As I become more involved, so will my blog.

I’ve said it a bunch of times now, but this site isn’t just about helping others — although that’s a big part — but it’s about giving myself a place to store the things, I think I’ll want one day. With over 2,000 cmdlets in version 3.3.0.0 of the AWSPowerShell module, I’m going to need a place to store this, and any help I can get in retaining this information, I’ll take. Writing helps with that; you know, the whole writing to remember concept.

So, here’s the error I was up against today:

“Your quota allows for 0 more running instance(s). You requested at least 1 – You can request an increase of the instance limit here: http://aws.amazon.com/contact-us/ec2-request/.”

As relayed by our upcoming, but practically already there, AWS expert, “Each instance size (t2.small, c3.large, etc) has a separate quota in each region in AWS.” The decision to do this was preventative in that we wouldn’t accidentally spin up too many instances and cost ourselves dearly. I get that.

I’m a curious type, so I couldn’t help but wonder, how many of each instance type do we have in the current AWS account? I switched over to my PowerShell console and started writing. This is straightforward and basic PowerShell, but it still seemed like something worth keeping around, and sharing, as well.

$Instances = Get-EC2Instance

Foreach ($Instance in $Instances) {
    [array]$Types += $Instance.Instances.InstanceType
}

$Types | Group-Object -NoElement

The example works this way: Line 1: Capture all the EC2 instances, Line 3: Begin to iterate over the returned collection. With each pass, in line 8, append the current instance’s instance type to the $Types variable — an array. Finally, in line 7, we group the $Types variable in that it’ll automatically provide the Count property for each Instance Type.

I’ve included a slight modification in the example below. In this instance, I didn’t cast the $Types variable as an array, and instead created the variable as an empty array prior to the Foreach. Same end goal, however, I wanted to highlight how others might be inclined to write something similar.

$Instances = Get-EC2Instance

$Types = @()
Foreach ($Instance in $Instances) {
    $Types += $Instance.Instances.InstanceType
}

$Types | Group-Object -NoElement

If you run this repeatedly, you’ll quickly realize that it’ll continue to add to the $Types variable, thus making the results incorrect as soon as the second run. You can add this as the last line: Remove-Variable -Name Types, or better, make it a function.

Function Get-EC2InstanceTypeCount {
<# #>
    [CmdletBinding()]
    Param (
    )

    Begin {
        $Instances = Get-EC2Instance
    } # End Begin.

    Process {
        Foreach ($Instance in $Instances) {
            [array]$Types += $Instance.Instances.InstanceType
        }

        $Types | Group-Object -NoElement
    } # End Process.

    End {
    } # End End.
}

Remember, you’re going to need to have already used Set-AWSCredentials and stored a set of credentials to a persistent store. This function is dependent, just like any AWS cmdlet, on there being a set of stored credentials. I did not write the function to accept an AccessKey and SecretKey parameter, as this isn’t a recommended scenario in much of the documentation I’ve read from AWS. Here’s the function — with comment-based help — if might be useful for your team.

Function Get-EC2InstanceTypeCount {
<# .SYNOPSIS This advanced function will return the AWS Instance Types used in an AWS account. .DESCRIPTION This advanced function will return the AWS Instance Types used in an AWS account. It will return a name property, such as t2.medium, m4.large, etc., and a Count property. The results are sorted on the count, as they are produced using the Sort-Object cmdlet. .EXAMPLE PS > Get-EC2InstanceTypeCount
    This example returns the EC2 Instance Types and how many of each are being used.

    Count Name
    ----- ----
    32    t2.medium
    18    t2.micro
     6    c3.large
     6    m4.large
     7    t2.small
     2    r3.large
     4    r3.xlarge
     5    g2.2xlarge
     5    t2.large
     1    t2.nano

.EXAMPLE
    PS > Get-EC2InstanceTypeCount | Sort-Object -Property Name
    This example returns the EC2 Instance Types and how many of each are being used, sorted by Name.

.EXAMPLE
    PS > Get-EC2InstanceTypeCount | Where-Object -Property Name -like *large*
    This example returns the EC2 Instance Types that include the term "large" in the Name property.

    Count Name
    ----- ----
        6 c3.large                 
        6 m4.large                 
        2 r3.large                 
        4 r3.xlarge                
        5 g2.2xlarge               
        5 t2.large

.NOTES
    NAME: Get-EC2InstanceTypeCount
    AUTHOR: Tommy Maynard
    COMMENTS: --
    LASTEDIT: 09/27/2016
    VERSION 1.0
#>
    [CmdletBinding()]
    Param (
    )

    Begin {
        $Instances = Get-EC2Instance
    } # End Begin.

    Process {
        Foreach ($Instance in $Instances) {
            [array]$Types += $Instance.Instances.InstanceType
        }

        $Types | Group-Object -NoElement
    } # End Process.

    End {
    } # End End.
}

Twitter Reply – More AWS PowerShell Changes Due to Twitter (and Me)

It happened again. My Tweet, that was linked to a post here, became the catalyst for a change in the AWS PowerShell module. I’m 2 for 2. This time, AWS didn’t simply correct the spelling of a parameter name; they wrote a new cmdlet. Remove-EC2Instance works as advertised. It’ll prompt for confirmation before terminating an EC2 instance, unless the -Force parameter is also included with the command. It is flawless. You can always get the newest AWSPowerShell module at http://aws.amazon.com/powershell.

What this didn’t do, however, is fix Stop-EC2Instance. Steve, the individual that spotted this and my last AWS Tweet, and likely made these corrections and additions, said they couldn’t remove the malfunctioning -Terminate parameter from Stop-EC2Intance. For those that didn’t read my last AWS post, the -Terminate parameter never prompted for confirmation before terminating an instance (unless a change has been made since Remove-EC2Instance was released).

I read an interesting article recently that indicated to use termination protection:

Use termination protection for non-auto-scaling instances. Thank me later.
If you have any instances which are one-off things that aren’t under auto-scaling, then you should probably enable termination protection, to stop anyone from accidentally deleting the instance. I’ve had it happen, it sucks, learn from my mistake!”

While it was still quite amazing to get a second change made to the AWSPowerShell module, I’d recommend that everyone continue to be careful with Stop-EC2Instance. Perhaps you can write your own wrapper function, with the same name as the cmdlet (and therefore, it’ll be the default when Stop-EC2Instance is called), that forces confirmation. Then again, you can just use Remove-EC2Instance now. We’ll hope that when people look to terminate an EC2 instance, that they find Remove-EC2Instance before they find the -Terminate parameter in Stop-EC2Instance, such as I would have, had it previously existed. 🙂

Twitter Reply – AWS Stop-EC2Instance Needs Complimentary Cmdlet

Well, it’s time to try this again: Can a Tweet get another change to the AWSPowerShell module? Maybe you’ve read a recent post about an inconsistent parameter name between the same parameter, in two corresponding cmdlets. One tweet later and that was on its way to being fixed. You can read about that here: http://tommymaynard.com/twitter-reply-hashtag-aws-tweet-prompts-fix-to-awspowershell-module-2016.

I get that a tweet probably isn’t the preferred method of reporting problems, but it worked before, so why shouldn’t it again? That said, if there is a preferred method, then I’d be happy to use that, although it doesn’t make for as good a story.

So, here’s where we need a fix. The Stop-EC2Instance cmdlet includes a -Terminate switch parameter that indicates it will terminate the instance (think, remove) in addition to stopping it. It works; it really does terminate the instance. The problem here is that the help file indicates it will prompt the user for confirmation unless the -Force parameter is also included. Nope; it doesn’t prompt. Not for me at least. It does terminate the instance, however, and it does so with no questions asked. It didn’t get me, but it might ruin someone’s day someday.

I didn’t try it, but it seems that there’s some built-in protection, although it’s not the default.

aws-stop-ec2instance-needs-complimentary-cmdlet01

So you can see it yourself, here’s two PowerShell commands I ran against two different EC2 instances. In the first command I simply stop an EC2 instance. In the second, I added the -Terminate switch. As I mentioned, it works, but there wasn’t a confirmation prompt. This was tested on AWS Tools for Windows PowerShell 3.1.66.0 and the newest version as of this writing, 3.1.71.0.

aws-stop-ec2instance-needs-complimentary-cmdlet02

aws-stop-ec2instance-needs-complimentary-cmdlet03

My suggestion to resolve this problem is to remove the -Terminate switch from the Stop-EC2Instance cmdlet, and make a Remove-EC2Instance cmdlet. Maybe alias it with Terminate-EC2Instance, as Terminate isn’t an approved verb. In fact, when I first wanted to remove an EC2 instance, I started by hunting for a Remove-EC2Instance cmdlet. Tracking down the -Terminate switch of Stop-EC2Instance was the last place I would’ve looked. It seems to go against all PowerShell conventions. Does Stop-Transcript have an option to remove a transcript as part of its usage? Does Stop-VM in Hyper-V delete the VM? Does Stop-Cluster include a way to delete the cluster. No. Terminating an instance shouldn’t be a part of stopping an instance unless Stop-EC2Instance can be piped to Remove-EC2Instance. Think about the safety that a separate cmdlet would provide (and it’s not like the AWS Tools team is shy about adding new cmdlets).

There’s a couple cmdlets out there that can be used to get the metadata from a compiled PowerShell cmdlet. We use it as a part of writing proxy functions. By using this, I was able to determine the reason why it doesn’t prompt. The problem, as best I can tell, is that Stop-EC2Instance’s ConfirmImpact CmdletBinding argument is set to Medium and the default value of the $ConfirmPreference variable is set to High.

aws-stop-ec2instance-needs-complimentary-cmdlet04

This means that no matter how the Stop-EC2Instance cmdlet is run, it’s never going to prompt for confirmation to terminate, unless someone has modified their $ConfirmPreference value to Medium, and that’s not likely at all. Since a cmdlet, or advanced function, can’t have more than one ConfirmImpact argument, we need the terminate functionality removed from Stop-EC2Instance and the development of a Remove-EC2Instance cmdlet. For comparison, here’s the CmdletBinding attribute for the Remove-ADGroupMembership Active Directory cmdlet. This one always prompts unless we include -Confirm:$false.

aws-stop-ec2instance-needs-complimentary-cmdlet05

I get this fix is much more involved than the last one I brought up (changing the name of a parameter), and would require more work. Additionally, it’s probably one of those things that have to be discussed among the AWS Tools team members and management. Before I close, I should make sure everyone knows that AWS is still new to me, and as such, there may be things that weren’t considered, or known, when this post was written.

Thanks for reading, and enjoy the upcoming weekend.

Twitter Reply – Hashtag AWS Tweet Prompts Fix to AWSPowerShell Module

I started the Twitter Reply category so I would be able to reply to things I saw on Twitter that needed to be done outside of the 140 character limit. I’ve used it this way a few times, however, today usage is a little different. Although it incorporates Twitter and it’s going to take longer than 140 characters, I’m not actually replying to someone so much. Instead, I am bringing up an event that transpired on Twitter.

First, let me begin by saying how impressed I am with at least one of the developers — I’m guessing he’s a developer — working on the AWSPowerShell module. Last Wednesday, I tweeted that I had started and then stopped an EC2 instance using the Start-EC2Instance and Stop-EC2Instance cmdlets. I noticed a naming difference between identical parameters used by these two, complimentary cmdlets, as I mentioned in my Tweet.

Steve Roberts — a complete stranger to me — replies to my tweet as he must follow the #AWS hashtag. The part the sticks out is that he wrote, “…Fixing…,” as if he was going to correct this difference. Hilarious, right?

Cut to just over 24 hours later and it’s fixed. I downloaded the newest version of the module, installed it, and tested it. It was fixed. The complimentary cmdlets now use the same, -InstanceId parameter.

Did that really just happen? For all I knew, this Steve guy was messing with me, but no, my Tweet really did initiate a (very minor) fix included in the newest version of the AWSPowerShell module. That’s a first. Well, I’ve found a new problem. Maybe I can get that one fixed, too. I’ll be writing about that in an upcoming post and will link it from here, as soon as it’s complete.