Category Archives: AWS

Documents and resources involving AWS Tools for PowerShell.

Get the AWS Noun Prefixes

One of the nice things about theĀ Get-AWSPowerShellVersion cmdlet is the ListServiceVersionInfo switch parameter. It returns properties for the Service (as in the AWS Service offering name), the Noun Prefix, and API Version. Yes, there really are spaces in those last two property names; however, they’ve been fixed in my function. I had been hoping for an easier way to determine the prefixes used in the AWS cmdlets, and here we have it. I actually considered parsing cmdlet names myself, so a huge thanks to AWS, for making sure that wasn’t necessary.

It’s almost as though this should have been its own cmdlet—a get the version cmdlet and a get the noun prefixes cmdlet. Therefore, I’ve wrapped this command and its parameter in a quick and easy-to-use function for my user base. Copy, paste, and try it out; it’s all yours. Super simple.

Function Get-AWSNounPrefix {
<#
.SYNOPSIS
    The function returns the AWS PowerShell cmdlet noun prefixes, along with the corresponding AWS Service.

.DESCRIPTION
    The function returns the AWS PowerShell cmdlet noun prefixes, along with the corresponding AWS Service name. This function utilizes the AWSPowerShell module's Get-AWSPowerShellVersion function.

.EXAMPLE
    -------------------------- EXAMPLE 1 --------------------------
    PS > Get-AWSNounPrefix
    This examples returns all the noun prefixes and their corresponding AWS Service name.

.EXAMPLE
    -------------------------- EXAMPLE 2 --------------------------
    PS > Get-AWSNounPrefix | Where-Object NounPrefix -match 'cf'
    NounPrefix Service            APIVersion
    ---------- -------            ----------
    CF         Amazon CloudFront  2016-09-29
    CFG        AWS Config         2014-11-12
    CFN        AWS CloudFormation 2010-05-15

    This example uses the Where-Object cmdlet and the Match operator to filter the results by the NounPrefix.

.EXAMPLE
    -------------------------- EXAMPLE 3 --------------------------
    PS > Get-AWSNounPrefix | Where-Object Service -like '*formation*'
    NounPrefix Service            APIVersion
    ---------- -------            ----------
    CFN        AWS CloudFormation 2010-05-15

    This example uses the Where-Object cmdlet and the Like operator to filter the results by the service name.

.EXAMPLE
    -------------------------- EXAMPLE 4 --------------------------
    PS > Get-AWSNounPrefix | Where-Object -Property APIVersion -like '2016*' | Sort-Object -Property APIVersion -Descending
    NounPrefix Service                          APIVersion
    ---------- -------                          ----------
    SMS        Amazon Server Migration Service  2016-10-24
    BGT        AWS Budgets                      2016-10-20
    CF         Amazon CloudFront                2016-09-29
    EC2        Amazon Elastic Compute Cloud     2016-09-15
    SNOW       AWS Import/Export Snowball       2016-06-30
    CGIP       Amazon Cognito Identity Provider 2016-04-18
    INS        Amazon Inspector                 2016-02-16
    AAS        Application Auto Scaling         2016-02-06
    MM         AWS Marketplace Metering         2016-01-14
    DMS        AWS Database Migration Service   2016-01-01

    This example uses the Where-Object, and Sort-Object cmdlet, to find services updated in 2016 and sorts by the most recently added and updated.

.NOTES
    Name: Get-AWSNounPrefix
    Author: Tommy Maynard
    Comments: Current version of AWSPowerShell module at the first, last edit: 3.3.20.0.
    Last Edit: 11/18/2016
    Version 1.0
#>
    [CmdletBinding()]
    Param (
    )

    Begin {
        $AWSServices = Get-AWSPowerShellVersion -ListServiceVersionInfo | Sort-Object -Property 'Noun Prefix'
    } # End Begin.

    Process {
        Foreach ($AWSService in $AWSServices) {
            [PSCustomObject]@{
                NounPrefix = $AWSService.'Noun Prefix'
                Service = $AWSService.Service
                APIVersion = $AWSService.'API Version'
            }
        } # End Foreach.
    } # End Process.

    End {
    } # End End.
} # End Function: Get-AWSNounPrefix.

 

The Unzip Time Difference

As part of an upcoming deployment, I’ve been getting intimate with AWS OpsWorks and Chef. What I mean by Chef is, using the PowerShell resource in Chef. I was recently looking for a way to save some time on a deployment, when I considered removing the unzipping of files downloaded from S3. I wanted to know if there would be a time savings in getting the files to the EC2 Instance in their decompressed format.

This brought me over to my console to compare the newer Expand-Archive cmdlet and .NET. I’ve always considered that dropping down to .NET is a time savings.

In my Chef recipe I’m using .NET for decompression, as I’m deploying to Windows Server 2012 R2 and it includes PowerShell 4.0 by default. The Compress-Archive and Expand-Archive cmdlets were introduced in PowerShell 5.0. This isn’t to say I couldn’t get PowerShell 5.0 in place, but I needed to know if it would even be necessary.

I had a little testing to do. The below command measured the time it took to expand a 133MB zip file using .NET.

Measure-Command -Expression {
    Add-Type -AssemblyName System.IO.Compression.FileSystem
    [System.IO.Compression.ZipFile]::ExtractToDirectory('C:\Users\tommymaynard\Desktop\HCM-920-UPD-018-WIN_1of10.zip','C:\Users\tommymaynard\Desktop\unzip\')
}

When the command was run five separate times, it resulted in following times: 9 seconds 393 milliseconds, 9 seconds 117 milliseconds, 9 seconds 455 milliseconds, 8 seconds 489 milliseconds, and 10 seconds 338 milliseconds. I wasn’t loosing any time by unzipping this file.

The below command does the same thing as the one above; however, it uses the Expand-Archive cmdlet introduced in PowerShell 5.0.

Measure-Command -Expression {
    Expand-Archive -Path 'C:\Users\tommymaynard\Desktop\HCM-920-UPD-018-WIN_1of10.zip' -OutputPath 'C:\Users\tommymaynard\Desktop\unzip\'
}

I executed the above command five times, too. The results were 2 minutes 11 seconds, 2 minutes 9 seconds, 2 minutes 10 seconds, 2 minutes 13 seconds, and 2 minutes 11 seconds.

This is a huge difference in time. Now, I do want to mention that I tested this on Windows 8.1 with PowerShell 5.1 (it’s in preview). The results may be better on different versions of Windows and with different versions of PowerShell. Let me know if you see different results with different configurations, and maybe I’ll do the same. The point is this, however: If you have a reason to speed up your project, you might consider .NET over a PowerShell cmdlet, or function. It seems I’m glad I did. Be sure you test different ways, to do the same thing.

Compare AWSPowerShell Versions

Series links: Part I, Part II, Part III (Coming)

Well, I wrote enough example code inside my PowerShell ConsoleHost recently, to write another post. That’s typically how these work. Find something great… write. Find something I hate… write. Most of all, just write. Put it on Twitter and help people learn—someone did it for me once.

Today’s goal was to compare my currently installed version of the AWSPowerShell module with the newest version of the AWSPowerShell module. We’ll do this a bit backward, and start with obtaining the newest version of the AWSPowerShell Module first. Because AWS has elected to put their module on the PowerShell Gallery—thanks, guys—we don’t have to parse the AWS PowerShell home page. I’m not sure if it’s even there, but luckily for us, it doesn’t need to be, and so I don’t need to try.

Find-Module -Name AWSPowerShell

Version    Name                                Repository           Description
-------    ----                                ----------           -----------
3.3.9.0    AWSPowerShell                       PSGallery            The AWS Tools for Windows PowerShell lets develo...

(Find-Module -Name AWSPowerShell).Version

Major  Minor  Build  Revision
-----  -----  -----  --------
3      3      9      0

(Find-Module -Name AWSPowerShell).Version.ToString()
3.3.9.0

Now that we can obtain the version number of the newest release, let’s work on getting the currently installed version number. My first thought was to use Get-AWSPowerShellVersion. It was a mess, but here’s what I did. This cmdlet does not return an object, unfortunately (and should be replaced and/or corrected).

Seriously, we expect that cmdlets and functions will return a usable object or objects. If we really need additional information stuffed in the results, let’s only include it with the Verbose parameter. Maybe just drop all that text in a file and have Verbose indicate the file to open. Maybe make a secondary cmdlet. Anything really, that doesn’t require that we parse text… which is exactly what I did (at first).

((Get-AWSPowerShellVersion).Split('`n')[2])[1..7] -join ''
3.3.0.0
[version](((Get-AWSPowerShellVersion).Split('`n')[2])[1..7] -join '')

Major  Minor  Build  Revision
-----  -----  -----  --------
3      3      0      0

The problem with this approach is that it’s much too exact. It’s grabbing certain characters. What happens when the version is longer than seven characters? We would’ve lost our final zero, if I was still on a previous version, such as 3.1.66.0, 3.1.71.0, or 3.1.73.0. Not good enough.

I knew the AWSPowerShell module put something somewhere; every module does. I browsed over to C:\Program Files (x86)\AWS Tools\PowerShell\AWSPowerShell to take a look around. Maybe there was something there that would be more reliable. I quickly spotted the module manifest file: AWSPowerShell.psd1. I opened it up in a text editor, and as suspected, there was the version… right inside that beautiful hash table. I closed up the file and used Test-ModuleManifest inside my ConsoleHost.

$Path = 'C:\Program Files (x86)\AWS Tools\PowerShell\AWSPowerShell\AWSPowerShell.psd1'
Test-ModuleManifest -Path $Path
ModuleType Version    Name                                ExportedCommands
---------- -------    ----                                ----------------
Binary     3.3.0.0    AWSPowerShell                       {Clear-AWSHistory, Set-AWSHistoryConfiguration, Initialize...
(Test-ModuleManifest -Path $Path).Version

Major  Minor  Build  Revision
-----  -----  -----  --------
3      3      0      0

(Test-ModuleManifest -Path $Path).Version.ToString()
3.3.0.0

That’s way more reliable. I’d much rather blame AWS if it doesn’t return the correct version, than my parsing against the results of Get-AWSPowerShellVersion. I mean, seriously.

If you’ve been following my writings, then it’ll come as no surprise that I started with the hardest way and overlooked the obvious. I could’ve simply just used Get-Module to return the version; it’s getting its information from the .psd1 file. As obnoxious as this is, it's keeping me sharp. I get to repeatedly practice what I know and give my mind time to sort out different resolutions to the same problem. Here's how I returned the version of the currently installed AWSPowerShell module.

Get-Module -Name AWSPowerShell
ModuleType Version    Name                                ExportedCommands
---------- -------    ----                                ----------------
Binary     3.3.0.0    AWSPowerShell                       {Add-AASScalableTarget, Add-ACMCertificateTag, Add-ASAAtta...
(Get-Module -Name AWSPowerShell -ListAvailable).Version

Major  Minor  Build  Revision
-----  -----  -----  --------
3      3      0      0

(Get-Module -Name AWSPowerShell -ListAvailable).Version.ToString()
3.3.0.0

Now I can return both the currently installed version of the AWSPowerShell module and the newest version of the AWSPowerShell module from the PowerShell Gallery. Before we compare them, notice that in the above examples that I did and didn’t use the ListAvailable parameter. If you don’t use it, you better be absolutely certain the module has already been imported.

So, let’s get these version numbers into a couple of variables and compare them.

$CurrentAWSPSModule = (Get-Module -Name AWSPowerShell -ListAvailable).Version.ToString()
$CurrentAWSPSModule
3.3.0.0
$NewestAWSPSModule = (Find-Module -Name AWSPowerShell).Version.ToString()
$NewestAWSPSModule
3.3.9.0
Compare-Object -ReferenceObject $CurrentAWSPSModule -DifferenceObject $NewestAWSPSModule -IncludeEqual

InputObject SideIndicator
----------- -------------
3.3.9.0     =>
3.3.0.0     <=

As we can tell, there is a difference between my version and the one in the PowerShell Gallery. I should probably download the newest version. Watch for a follow-up to this article, as I may go ahead and write a second part, as I have some other ideas.

AWS EC2 Instance Type Count

There’s a project that I’m on that allows me to work in AWS. It’s pretty important stuff at work, and since there’s an AWS PowerShell module, I couldn’t be more interested. As of today, I’m going to start to include worthy AWS PowerShell work, right here. As I become more involved, so will my blog.

I’ve said it a bunch of times now, but this site isn’t just about helping others—although that’s a big part— it’s about giving myself a place to store the things, I think I’ll want one day. With over 2,000 cmdlets in version 3.3.0.0 of the AWSPowerShell module, I’m going to need a place to store this, and I’ll take any help I can get in retaining this information. Writing helps with that; you know, the whole writing-to-remember concept.

So, here’s the error I was up against today:

“Your quota allows for 0 more running instance(s). You requested at least 1 – You can request an increase of the instance limit here: http://aws.amazon.com/contact-us/ec2-request/.”

As relayed by our upcoming, but practically already there, AWS expert, “Each instance size (t2.small, c3.large, etc.) has a separate quota in each region in AWS.” The decision to do this was preventative in that we wouldn’t accidentally spin up too many instances and cost ourselves dearly. I get that.

I’m a curious type, so I couldn’t help but wonder, how many of each instance type do we have in the current AWS account? I switched over to my PowerShell console and started writing. This is straightforward and basic PowerShell, but it still seemed like something worth keeping around and sharing, as well.

$Instances = Get-EC2Instance

Foreach ($Instance in $Instances) {
    [array]$Types += $Instance.Instances.InstanceType
}

$Types | Group-Object -NoElement

The example works this way: Line 1: Capture all the EC2 instances, Line 3: Begin to iterate over the returned collection. With each pass, in line 8, append the current instance’s instance type to the $Types variable—an array. Finally, in line 7, we group the $Types variable in that it’ll automatically provide the Count property for each Instance Type.

I’ve included a slight modification in the example below. In this instance, I didn’t cast the $Types variable as an array and instead created the variable as an empty array prior to the Foreach. It’s the same end goal, however, I wanted to highlight how others might be inclined to write something similar.

$Instances = Get-EC2Instance

$Types = @()
Foreach ($Instance in $Instances) {
    $Types += $Instance.Instances.InstanceType
}

$Types | Group-Object -NoElement

If you run this repeatedly, you’ll quickly realize that it’ll continue to add to the $Types variable, thus making the results incorrect as soon as the second run. You can add this as the last line: Remove-Variable -Name Types, or better, make it a function.

Function Get-EC2InstanceTypeCount {
<# #>
    [CmdletBinding()]
    Param (
    )

    Begin {
        $Instances = Get-EC2Instance
    } # End Begin.

    Process {
        Foreach ($Instance in $Instances) {
            [array]$Types += $Instance.Instances.InstanceType
        }

        $Types | Group-Object -NoElement
    } # End Process.

    End {
    } # End End.
}

Remember, you’re going to need to have already used Set-AWSCredentials and stored a set of credentials in a persistent store. This function is dependent, just like any AWS cmdlet, on there being a set of stored credentials. I did not write the function to accept an AccessKey and SecretKey parameter, as this isn’t a recommended scenario in much of the documentation I’ve read from AWS. Here’s the function—with comment-based help—if it might be useful for your team.

Function Get-EC2InstanceTypeCount {
<# .SYNOPSIS This advanced function will return the AWS Instance Types used in an AWS account. .DESCRIPTION This advanced function will return the AWS Instance Types used in an AWS account. It will return a name property, such as t2.medium, m4.large, etc., and a Count property. The results are sorted on the count, as they are produced using the Sort-Object cmdlet. .EXAMPLE PS > Get-EC2InstanceTypeCount
    This example returns the EC2 Instance Types and how many of each are being used.

    Count Name
    ----- ----
    32    t2.medium
    18    t2.micro
     6    c3.large
     6    m4.large
     7    t2.small
     2    r3.large
     4    r3.xlarge
     5    g2.2xlarge
     5    t2.large
     1    t2.nano

.EXAMPLE
    PS > Get-EC2InstanceTypeCount | Sort-Object -Property Name
    This example returns the EC2 Instance Types and how many of each are being used, sorted by Name.

.EXAMPLE
    PS > Get-EC2InstanceTypeCount | Where-Object -Property Name -like *large*
    This example returns the EC2 Instance Types that include the term "large" in the Name property.

    Count Name
    ----- ----
        6 c3.large                 
        6 m4.large                 
        2 r3.large                 
        4 r3.xlarge                
        5 g2.2xlarge               
        5 t2.large

.NOTES
    NAME: Get-EC2InstanceTypeCount
    AUTHOR: Tommy Maynard
    COMMENTS: --
    LASTEDIT: 09/27/2016
    VERSION 1.0
#>
    [CmdletBinding()]
    Param (
    )

    Begin {
        $Instances = Get-EC2Instance
    } # End Begin.

    Process {
        Foreach ($Instance in $Instances) {
            [array]$Types += $Instance.Instances.InstanceType
        }

        $Types | Group-Object -NoElement
    } # End Process.

    End {
    } # End End.
}