Tag Archives: array

AWS EC2 Instance Type Count

There’s a project that I’m on that allows me to work in AWS. It’s pretty important stuff at work, and since there’s an AWS PowerShell module, I couldn’t be more interested. As of today, I’m going to start to include worthy AWS PowerShell work, right here. As I become more involved, so will my blog.

I’ve said it a bunch of times now, but this site isn’t just about helping others — although that’s a big part — but it’s about giving myself a place to store the things, I think I’ll want one day. With over 2,000 cmdlets in version 3.3.0.0 of the AWSPowerShell module, I’m going to need a place to store this, and any help I can get in retaining this information, I’ll take. Writing helps with that; you know, the whole writing to remember concept.

So, here’s the error I was up against today:

“Your quota allows for 0 more running instance(s). You requested at least 1 – You can request an increase of the instance limit here: http://aws.amazon.com/contact-us/ec2-request/.”

As relayed by our upcoming, but practically already there, AWS expert, “Each instance size (t2.small, c3.large, etc) has a separate quota in each region in AWS.” The decision to do this was preventative in that we wouldn’t accidentally spin up too many instances and cost ourselves dearly. I get that.

I’m a curious type, so I couldn’t help but wonder, how many of each instance type do we have in the current AWS account? I switched over to my PowerShell console and started writing. This is straightforward and basic PowerShell, but it still seemed like something worth keeping around, and sharing, as well.

$Instances = Get-EC2Instance

Foreach ($Instance in $Instances) {
    [array]$Types += $Instance.Instances.InstanceType
}

$Types | Group-Object -NoElement

The example works this way: Line 1: Capture all the EC2 instances, Line 3: Begin to iterate over the returned collection. With each pass, in line 8, append the current instance’s instance type to the $Types variable — an array. Finally, in line 7, we group the $Types variable in that it’ll automatically provide the Count property for each Instance Type.

I’ve included a slight modification in the example below. In this instance, I didn’t cast the $Types variable as an array, and instead created the variable as an empty array prior to the Foreach. Same end goal, however, I wanted to highlight how others might be inclined to write something similar.

$Instances = Get-EC2Instance

$Types = @()
Foreach ($Instance in $Instances) {
    $Types += $Instance.Instances.InstanceType
}

$Types | Group-Object -NoElement

If you run this repeatedly, you’ll quickly realize that it’ll continue to add to the $Types variable, thus making the results incorrect as soon as the second run. You can add this as the last line: Remove-Variable -Name Types, or better, make it a function.

Function Get-EC2InstanceTypeCount {
<# #>
    [CmdletBinding()]
    Param (
    )

    Begin {
        $Instances = Get-EC2Instance
    } # End Begin.

    Process {
        Foreach ($Instance in $Instances) {
            [array]$Types += $Instance.Instances.InstanceType
        }

        $Types | Group-Object -NoElement
    } # End Process.

    End {
    } # End End.
}

Remember, you’re going to need to have already used Set-AWSCredentials and stored a set of credentials to a persistent store. This function is dependent, just like any AWS cmdlet, on there being a set of stored credentials. I did not write the function to accept an AccessKey and SecretKey parameter, as this isn’t a recommended scenario in much of the documentation I’ve read from AWS. Here’s the function — with comment-based help — if might be useful for your team.

Function Get-EC2InstanceTypeCount {
<# .SYNOPSIS This advanced function will return the AWS Instance Types used in an AWS account. .DESCRIPTION This advanced function will return the AWS Instance Types used in an AWS account. It will return a name property, such as t2.medium, m4.large, etc., and a Count property. The results are sorted on the count, as they are produced using the Sort-Object cmdlet. .EXAMPLE PS > Get-EC2InstanceTypeCount
    This example returns the EC2 Instance Types and how many of each are being used.

    Count Name
    ----- ----
    32    t2.medium
    18    t2.micro
     6    c3.large
     6    m4.large
     7    t2.small
     2    r3.large
     4    r3.xlarge
     5    g2.2xlarge
     5    t2.large
     1    t2.nano

.EXAMPLE
    PS > Get-EC2InstanceTypeCount | Sort-Object -Property Name
    This example returns the EC2 Instance Types and how many of each are being used, sorted by Name.

.EXAMPLE
    PS > Get-EC2InstanceTypeCount | Where-Object -Property Name -like *large*
    This example returns the EC2 Instance Types that include the term "large" in the Name property.

    Count Name
    ----- ----
        6 c3.large                 
        6 m4.large                 
        2 r3.large                 
        4 r3.xlarge                
        5 g2.2xlarge               
        5 t2.large

.NOTES
    NAME: Get-EC2InstanceTypeCount
    AUTHOR: Tommy Maynard
    COMMENTS: --
    LASTEDIT: 09/27/2016
    VERSION 1.0
#>
    [CmdletBinding()]
    Param (
    )

    Begin {
        $Instances = Get-EC2Instance
    } # End Begin.

    Process {
        Foreach ($Instance in $Instances) {
            [array]$Types += $Instance.Instances.InstanceType
        }

        $Types | Group-Object -NoElement
    } # End Process.

    End {
    } # End End.
}

Quick Learn – Protect your Variables with ReadOnly and Constant Options

I wrote a post a day to two back about creating an array variable that contained other arrays. I then went on to create additional, easier-to-remember variables, to use as the indexes. Here’s the post if you’d like to read it: http://tommymaynard.com/ql-working-with-an-array-of-arrays-2015/.

I started thinking, what if you create a variable for this easier-to-remember purpose, and then acidentally overwrite it? Well, as assumed, it is no longer going to work as first intended. Here’s a quick example starting back at the array of arrays concept.

PS C:\> Set-Variable -Name TeamWareServers -Value @(('serv01','serv02'),('serv03','serv04','serv05'))
PS C:\> $TeamWareServers
serv01
serv02
serv03
serv04
serv05
PS C:\> Set-Variable -Name f -Value 0 # Front end servers
PS C:\> $f
0
PS C:\> Set-Variable -Name b -Value 1 # Back end servers
PS C:\> $b
1
PS C:\> $TeamWareServers[$f]
serv01
serv02
PS C:\> $TeamWareServers[$b]
serv03
serv04
serv05

Although it’s easier to remember which servers are which, we have the possibility that our variables, $f and $b, could easily be overwritten. Here’s an example of overwriting the variables’ values and then not being able to use them as we did in the last example. I added an extra space after the results, of which there were none, so it’s easy to tell that these variable no longer work.

PS C:\> $f = 20
PS C:\> $b = 'newvalue'
PS C:\> $TeamWareServers[$f]
PS C:\>
PS C:\> $TeamWareServers[$b]
PS C:\>

So, how can we better protect our variables? There’s two ways we’ll discuss: ReadOnly and Constant. These two options will protect our variables from being assigned any new value(s). Take a look at this example where we’ll reset our $f and $b variables back to their original values.

PS C:\> Set-Variable -Name f -Value 0 -Option ReadOnly
PS C:\> $f
0
PS C:\> Set-Variable -Name b -Value 1 -Option Constant
Set-Variable : Existing variable b cannot be made constant. Variables can be made constant only at creation time.
At line:1 char:1
+ Set-Variable -Name b -Value 1 -Option Constant
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : WriteError: (b:String) [Set-Variable], SessionStateUnauthorizedAccessException
    + FullyQualifiedErrorId : VariableCannotBeMadeConstant,Microsoft.PowerShell.Commands.SetVariableCommand

In the example above, we assigned a new value to our $f variable and made it ReadOnly using the -Option parameter. When we tried to modify the $b variable to make it a Constant, we received an error. This is because we don’t have the ability to make an existing variable a Constant. In the next example, below, we’ll remove the $b variable and then recreate it with the Constant option. Keep in mind that Set-Variable will work like New-Variable, if the variable doesn’t already exist.

PS C:\> Remove-Variable -Name b
PS C:\> Set-Variable -Name b -Value 1 -Option Constant
PS C:\> $b
1

Now let’s try what we did earlier and assign new values to these variables. You’ll soon see that when the variable’s option is set to Read-Only or Constant, that we’re not able to change their values.

PS C:\> $f = 20
Cannot overwrite variable f because it is read-only or constant.
At line:1 char:1
+ $f = 20
+ ~~~~~~~
    + CategoryInfo          : WriteError: (f:String) [], SessionStateUnauthorizedAccessException
    + FullyQualifiedErrorId : VariableNotWritable

PS C:\> $b = 'newvalue'
Cannot overwrite variable b because it is read-only or constant.
At line:1 char:1
+ $b = 'newvalue'
+ ~~~~~~~~~~~~~~~
    + CategoryInfo          : WriteError: (b:String) [], SessionStateUnauthorizedAccessException
    + FullyQualifiedErrorId : VariableNotWritable

If you’re anything like me, then you might be wondering what the difference is between ReadOnly and Constant. We’ll let the next example help explain.

PS C:\> Remove-Variable -Name f
Remove-Variable : Cannot remove variable f because it is constant or read-only. If the variable is read-only, try the
operation again specifying the Force option.
At line:1 char:1
+ Remove-Variable -Name f
+ ~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : WriteError: (f:String) [Remove-Variable], SessionStateUnauthorizedAccessException
    + FullyQualifiedErrorId : VariableNotRemovable,Microsoft.PowerShell.Commands.RemoveVariableCommand

PS C:\> Remove-Variable -Name f -Force # This works.
PS C:\>
PS C:\> Remove-Variable -Name b
Remove-Variable : Cannot remove variable b because it is constant or read-only. If the variable is read-only, try the
operation again specifying the Force option.
At line:1 char:1
+ Remove-Variable -Name b
+ ~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : WriteError: (b:String) [Remove-Variable], SessionStateUnauthorizedAccessException
    + FullyQualifiedErrorId : VariableNotRemovable,Microsoft.PowerShell.Commands.RemoveVariableCommand

PS C:\> Remove-Variable -Name b -Force # This doesn't work.
Remove-Variable : Cannot remove variable b because it is constant or read-only. If the variable is read-only, try the
operation again specifying the Force option.
At line:1 char:1
+ Remove-Variable -Name b -Force
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : WriteError: (b:String) [Remove-Variable], SessionStateUnauthorizedAccessException
    + FullyQualifiedErrorId : VariableNotRemovable,Microsoft.PowerShell.Commands.RemoveVariableCommand

When a variable is ReadOnly, the variable can be removed. This does, however, require the use of the -Force parameter. Consider that a safety net. The difference here, is that when a variable is a Constant, it cannot be removed, even with the use of the -Force parameter. If you want to remove a user-defined Constant variable, you’re going to have to end your PowerShell console and start another one.

Keep these options in mind if you ever want to protect the value(s) in your variables. While we’re at it, we probably should have made the array (of arrays) variable, $TeamWareServers, ReadOnly or Constant, too.

Quick Learn – Working with an Array of Arrays

There’s a group of servers that I use for a specific project. I can never remember their names or how those correspond to their roles (web front end vs. data back end). Although I’ve updated their Active Directory (AD) descriptions, and created two, specifically named AD groups for them, I wanted an even quicker way to remind myself whose who. With that in mind, I updated my profile with a custom variable that is an array, of arrays.

In this Quick Learn, we’ll work with an example that actually uses three groups of roles — our DCs, our web servers, and our SQL servers. As you’ll soon see, our servers are named after Santa Claus’ reindeer. These names have nothing to do with the role of these servers, and since their names are all closely related, it’s difficult to remember who does what.

This first example, below, demonstrates how we create a new variable, or modify an already existing variable. When we echo the contents of our variable, we get all the computer names, regardless of what array they are in, within the base array. The term ‘base array’ is probably not something you’ll hear or read about outside this post. It’s being used here to help distinguish the array that holds all the other arrays — the container array.

PS C:\> Set-Variable -Name Computers -Value @(('dasher','vixen','cupid'),('comet','dancer','donner'),('blitzen','rudolph','prancer'))
PS C:\> $Computers
dasher
vixen
cupid
comet
dancer
donner
blitzen
rudolph
prancer

We can use an index to return one of the arrays within the base array. In the examples below, you can see how each can be accessed. This is probably a good time to review indexes: The first item in an array is index zero, the second item is index one, the third item is index two, and so on.

PS C:\> $Computers[0]
dasher
vixen
cupid
PS C:\> $Computers[1]
comet
dancer
donner
PS C:\> $Computers[2]
blitzen
rudolph
prancer

In the following example, we can use two indexes to access a specific server. The first index represents which array (within the base array) I want to return, like it did above, and the second index indicates which server I want to return.

PS C:\> $Computers[0][2]
cupid
PS C:\> $Computers[1][0]
comet
PS C:\> $Computers[2][1]
rudolph

The difficult part is going to be able to remember which index is for the DCs, the web servers, or the SQL servers. In that case, I’ll create three more variables to use in place of those index integers.

PS C:\> Set-Variable -Name DCs -Value 0
PS C:\> Set-Variable -Name Web -Value 1
PS C:\> Set-Variable -Name SQL -Value 2
PS C:\> $DCs,$Web,$SQL
0
1
2

With the combination of my $Computers variable and the three, role-specific variables, I am able to easily return the set of servers I want.

PS C:\> $Computers[$DCs]
dasher
vixen
cupid
PS C:\> $Computers[$Web]
comet
dancer
donner
PS C:\> $Computers[$SQL]
blitzen
rudolph
prancer

Now that we have this all figured out, I can use them in different commands. Here’s a couple examples:

PS C:\> $Computers[$SQL] | ForEach-Object {Test-Connection $_ -Count 1}

Source        Destination     IPV4Address      IPV6Address                              Bytes    Time(ms)
------        -----------     -----------      -----------                              -----    --------
TOMMYMS PC... blitzen         10.10.10.80                                               32       1
TOMMYMS PC... rudolph         10.10.10.81                                               32       1
TOMMYMS PC... prancer         10.10.10.82                                               32       1

PS C:\> Get-Service -ComputerName ($Computers[$DCs]) -Name *bit* | Select-Object MachineName,Name,Status | Format-Table -AutoSize

MachineName Name  Status
----------- ----  ------
dasher      BITS Stopped
vixen       BITS Running
cupid       BITS Stopped

The unfortunate thing about hard coding computer names in our profile, is that we’ll run into problems when new servers are added and old ones are decommissioned. Therefore, we’re going to use AD groups — something I mentioned earlier — to populate our array variable. We’ll pull our DCs from the Domain Controllers, and our web servers and SQL servers from two fictions AD groups: WebServers and SQLBoxes. Here’s the command we’ll add to our profile to ensure we always have the correct server names. While this can all be on a single line command I’ve added line breaks to give it an easier-to-read appearance.

Set-Variable -Name Computers -Value @(
    ((Get-ADDomainController -Filter *).Name),
    ((Get-ADGroupMember -Identity WebServers).Name),
    ((Get-ADGroupMember -Identity SQLBoxes).Name)
)

And, that’s it. If for some reason you’re using PowerShell 2.0 or lower (and I really hope you’re not), you’ll need to include Import-Module -Name ActiveDirectory in your profile. As well — and it should go without saying — you’ll need to be working from a computer that actually has the ActiveDirectory module installed. Although I’m using PowerShell 4.0, I still include Import-Module -Name ActiveDirectory in my profile, so I don’t have to wait for it to auto load, when I run my first AD cmdlet.