Category Archives: Quick Learn

Practical examples of PowerShell concepts gathered from specific projects, forum replies, or general PowerShell use.

UX Headache – Joining Lines in a Text File

A part of me seriously wants to be involved in UX. I constantly find problems with just about every UI in which I interface. This one is beautiful, but it is lacking. This one is ugly, but works. Maybe it’s why I love PowerShell; it’s always the same no matter what I’m working with. It’s probably also why I wish every website on the planet was written with APIs first. What an amazing world, if I could do everything using PowerShell: check the bank, register children for school, order Chipotle, and make appointments at the doctor, the dentist, the eye doctor, the auto shop, etc. The list is endless.

Anyway, back to the topic here. I’ve often considered buying a new domain and pointing out awful, real-world experiences of my own until someone comes along, realizes I get it, and employs me to stop all the awful interfaces… at least for that company. Us humans, living in this digital world, are constantly subjected to awful-looking, unhelpful, and inconsistent interfaces that have become a requirement in our lives. If I request that your site show me 100 rows at a time, then it’s not likely I’m going to change my mind when I click into one record and then go back out to the row view again. And if I do want to change it, guess what, that’s on me. I could go on for days.

The company responsible for those, two paragraphs is the reason I’m writing today. I was sent a list of 100 or 200 CIDR ranges. No problem, I’ll just copy and paste them into that one box on the website set to accept both single IPs and CIDR ranges. Nope. That caused an error. It was unable to parse it, and so now it was my job to enter them one at a time!? Well, it would’ve been had I not known PowerShell. Seriously, someone somewhere might be doing that. Copy and paste, or select and drag, or whatever other option was left. Whichever method, it would be much slower than what I did. So, today’s post is both me venting a little of my pent-up UX frustration and providing a quick resolution for anyone in this same situation, that didn’t automatically think PowerShell themselves. You see, the interface would take a comma-separated list, it just couldn’t handle a line-delimited list–if that’s even what that’s called. Maybe new-line-delimited; I don’t know for sure.

Here’s what I had (after removing the public IP addresses):

10.138.80.0/22
10.138.87.160/27
10.138.91.160/27
10.138.129.0/29
10.139.33.96/28
10.139.38.0/27
10.224.1.32/27
10.224.41.128/25
10.224.43.0/24
10.224.73.0/25
10.228.21.192/27
10.120.1.0/27
10.128.1.32/27
10.128.11.64/26
10.128.29.0/24
10.128.205.128/26
10.130.66.0/25
10.152.7.160/27
10.152.12.0/24
10.152.13.0/24
10.152.14.0/24
10.152.15.0/27
10.156.20.0/28
10.156.20.16/28
10.156.20.32/27
10.156.20.96/27
10.156.24.0/22
10.156.28.128/26
10.156.29.0/24
10.156.30.0/24
10.156.31.0/24
10.156.32.0/24
10.156.33.0/24
10.156.34.0/24
10.156.35.0/24
10.156.36.0/24
10.156.42.0/24
10.156.43.0/24
10.160.20.0/25
10.161.43.0/25
10.161.43.128/25
10.161.44.0/22
10.161.48.0/22
10.162.2.0/27
10.166.9.0/24
10.192.237.0/26
10.192.238.0/24
10.192.255.0/25
10.193.120.0/25
10.193.120.128/25
10.193.121.0/25
10.193.121.128/25
10.193.122.0/25
10.193.122.128/25
10.193.123.0/25
10.193.123.128/25
10.193.124.0/25
10.193.124.128/25
10.193.125.0/25
10.208.17.0/24
10.208.21.0/24
10.224.21.0/25
10.224.40.0/24
10.224.61.192/26
10.224.71.32/27
10.224.71.160/27
10.224.72.192/27
10.224.74.0/23
10.224.78.0/24
10.224.81.0/25
10.224.81.128/25
10.224.82.64/26
10.224.83.0/24
10.224.100.0/22
10.224.104.0/22
10.224.108.0/22
10.224.112.0/22
10.224.116.0/22
10.224.120.0/22
10.224.124.0/22
10.224.128.0/23
10.224.130.0/23
10.224.132.0/23
10.224.134.0/23
10.224.136.0/23
10.224.138.0/23
10.224.140.0/23
10.224.142.0/23
10.224.148.0/22
10.226.3.0/26
10.229.16.0/23
10.230.12.128/25
10.140.76.0/24
10.140.78.0/28
10.140.102.0/24
10.140.103.0/24
10.140.113.0/24
10.140.138.0/24
10.140.139.0/26
10.120.1.32/28
10.128.167.32/27
10.192.178.64/26
10.194.3.128/25
10.224.5.128/26
10.224.42.0/25
10.224.76.0/24
10.224.77.0/24
10.224.79.0/24
10.224.96.0/22
10.140.100.0/24
10.140.101.0/24
10.140.104.0/24
10.140.105.0/24
10.140.106.0/24
10.193.120.0/21
10.130.169.0/24
10.224.9.0/24

Let’s save this file to my computer as C:\Users\tommymaynard\Documents\CIDR.txt. Now, let’s see how many entries we’re working with. What kind of time might I save?

$Path = 'C:\Users\tommymaynard\Documents\CIDR.txt'
(Get-Content -Path $Path).Count
117

We’re working with 117 entries, or rather, 116 commas. Yeah, I’m not moving those over one by one; I don’t have the kind of time during my day. Enter PowerShell. To begin testing, I chose a smaller subset of the CIDR ranges. When I was happy with that, which was practically immediately, I added the -join operator.

Get-Content -Path $Path | Select-Object -First 5
10.138.80.0/22
10.138.87.160/27
10.138.91.160/27
10.138.129.0/29
10.139.33.96/28
(Get-Content -Path $Path | Select-Object -First 5) -join ','
10.138.80.0/22,10.138.87.160/27,10.138.91.160/27,10.138.129.0/29,10.139.33.96/28

Once I had this, I only had two things left to do. One, test to see if the company’s UI accepted comma-separated entries like this, and two, if it did, then join all 117 addresses with a comma in between each, and carry on with my day. That’s three things. Well, four if you count writing up this post after work. The UI did accept things that way, and so I ran the below command, pasted it in the box, saved everything, and reported back to my customer that it was set and done, as requested. Next.

(Get-Content -Path $Path) -join ',' | Set-Clipboard

It Begins – Azure with PowerShell I

Posts in this Series

I had a conversation with someone recently, which reminded me of an event in my life. I was sitting in the yard, maybe up to 15 years ago, pulling weeds. In southern Arizona, getting rid of the weeds is what you do, one way or another. Either you do it, or you pay someone else to do it. While I’ve yet to pay for such a service, I remember thinking, why am I doing this? Why am I sitting in the yard during the weekend when I could be sitting in the house and learning more about this IT career of mine? My competition is in there … figuratively. They’re learning more than me now, sitting here, learning nothing, except which weeds are going to leave a sticky residue on my fingertips, or poke me, or never come out of the earth with the roots intact, ever. Every minute I’m not learning more about what I do, someone else is gaining an advantage over me. More dollars, more vacations, more iRobot vacuums, which I hear are great–that’s what this person was telling me–but how would I know?

Don’t get me wrong, everyone has to have non-work-related hobbies and things to do, but pulling weeds probably isn’t it.

So, I am beginning my journey into Azure, alongside PowerShell. Or just maybe, I have that backward: My journey into PowerShell, alongside Azure. No. I have over 10 years’ worth of PowerShell experience and we’ll say a couple of years of Azure? I did receive an Azure (and M365) Fundamentals certificate, so maybe I know something. It’s been a while though, as I work closely with AWS five days a week. Regardless of how I think about this though, I’m going to learn one with the help of the other. I’ve done this before; I started with the cmdlets first: “I’m just starting to get my hands wet with Microsoft Lync. As I often do, I use the Windows PowerShell cmdlets to help learn more about a product; I did this same thing with Hyper-V.” Then, once I’m comfortable with those, maybe I head into the UI and see if I can duplicate what the commands do.

I recommend you do as I did, and begin with the two, below posts to get started with Azure PowerShell. These are quick, easy reads, one of which will assist with ensuring you have the Azure PowerShell module installed.

While I’ve been writing about PowerShell for a while, I think with this series, I’m going to approach things as though my visitors aren’t as experienced with PowerShell as my normal audience. So, if you’re my normal audience, some of the PowerShell concepts I mention may seem a touch basic, but with good reason. Knowing PowerShell and learning how to make things happen, is much different than knowing a product or service, and then learning PowerShell.

And bonus, the 8.0.0 version of the Az PowerShell module was released, today. That’s right, Tuesday, May 24, 2022–the first day of this series. The below, Find-Modulecommand, searches the PowerShell Gallery for the module and populates the $AzModuleInfo variable full of information–not just the Name, Version, and PublishedDate, although Select-Object does filter the output after the variable collects it all. Be sure to inspect the variable closely to view all the things that it contains. While Find-Module won’t install the module, Install-Module will. Before you run away with that command, though, read the two above links. More soon!

Find-Module -Name Az -OutVariable AzModuleInfo | Select-Object -Property Name,Version,PublishedDate

Name Version PublishedDate
---- ------- -------------
Az   8.0.0   5/24/2022 1:05:02 AM
$AzModuleInfo | Select-Object -Property *
Name                       : Az
Version                    : 8.0.0
Type                       : Module
Description                : Microsoft Azure PowerShell - Cmdlets to manage resources in Azure. This module is compatible with PowerShell and Windows PowerShell.
                             For more information about the Az module, please visit the following: https://docs.microsoft.com/powershell/azure/
Author                     : Microsoft Corporation
CompanyName                : azure-sdk...

Let’s Learn the Get-FileHash Command

Someone, somewhere, sent me down a path. At the end of it, while it is not where I needed to be, I learned — or relearned rather — about the Get-FileHash cmdlet. Whether you know about it or not, we will quickly cover it and walk through some examples, as well. Get-FileHash, and I quote, “Computes the hash value for a file by using a specified hash algorithm.” This is what it does, but is not the why. Here is its reference page, however: https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/get-filehash, and the why, is definitely in there; you should read it.

The idea, for those that are not going to read it, is that we can obtain a file’s hash and then check the hash to ensure the file has not been changed. At this point in your career, you have likely seen file hashes near, or alongside, a file download. Checking the file hash against the file, after it is downloaded, allows you to be certain that the file is the right one and that it was not altered by the download process, or anything else. It is what you were expecting.

If you are going to run any of my below commands, first be certain you know your working directory and that it is a location where you have permissions to write. We will start by creating a new file, adding a sentence to it, and then returning that content to ensure it is properly in place.

New-Item -Name hashfile.txt -ItemType File

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
-a---           4/19/2022  7:06 PM              0 hashfile.txt
Add-Content -Path .\hashfile.txt -Value 'This is our file at the beginning.'
Get-Content -Path .\hashfile.txt
This is our file at the beginning.

That all works and so now we have a file with which can experiment. If you are wondering how you learn PowerShell, this is how you do it. Follow along, as there is a goodie further down below. In this example we invoke Get-FileHash against our file, only returning the algorithm used and the hash. We are using Format-List in order to better display this content.

Get-FileHash -Path .\hashfile.txt | Tee-Object -Variable SaveMeForLater | Format-List -Property Algorithm,Hash
Algorithm : SHA256
Hash      : 3C55E3C7D4C2EEF6910CB70FC425549981528CBBC0400A705104DC09A9391356

In this example, we do the same as we did above, however, now we are going to try out the other parameter values that the Algorithm parameter will accept. By default it uses SHA256, but it will accept SHA1, SHA384, SHA512, and MD5, too.

Get-FileHash -Algorithm SHA1 -Path .\hashfile.txt | Format-List -Property Algorithm,Hash

Algorithm : SHA1
Hash      : BD002AAE71BEEBB69503871F2AD3793BA5764097


Get-FileHash -Algorithm SHA256 -Path .\hashfile.txt | Format-List -Property Algorithm,Hash

Algorithm : SHA256
Hash      : 3C55E3C7D4C2EEF6910CB70FC425549981528CBBC0400A705104DC09A9391356


Get-FileHash -Algorithm SHA384 -Path .\hashfile.txt | Format-List -Property Algorithm,Hash

Algorithm : SHA384
Hash      : E6BC50D6465FE3ECD7C7870D8A510DC8071C7D1E1C0BB069132ED712857082E34801B20F462E4386A6108192C076168A


Get-FileHash -Algorithm SHA512 -Path .\hashfile.txt | Format-List -Property Algorithm,Hash

Algorithm : SHA512
Hash      : C0124A846506B57CE858529968B04D2562F724672D8B9E2286494DB3BBB098978D3DA0A9A1F9F7FF0D3B862F6BD1EB86D301D025B80C0FC97D5B9619A1BD7D86


Get-FileHash -Algorithm MD5 -Path .\hashfile.txt | Format-List -Property Algorithm,Hash

Algorithm : MD5
Hash      : 30091603F57FE5C35A12CB43BB32B5F5

For fun, let’s loop through these values and pump out all the hashes, one right after another. Notice that we are using hard-coded values for the Algorithm parameter. Obnoxious. We will get to another way, which is/was the goodie I mentioned above. The more I think about it though — as I have been in the last 10 minutes — the more I think it might need its own post. Anyway, more on that soon.

'SHA1','SHA256','SHA384','SHA512','MD5'
    | ForEach-Object {Get-FileHash -Algorithm $_ -Path '.\hashfile.txt'
    | Select-Object -Property Algorithm,Hash}

Algorithm Hash
--------- ----
SHA1      BD002AAE71BEEBB69503871F2AD3793BA5764097
SHA256    3C55E3C7D4C2EEF6910CB70FC425549981528CBBC0400A705104DC09A9391356
SHA384    E6BC50D6465FE3ECD7C7870D8A510DC8071C7D1E1C0BB069132ED712857082E34801B20F462E4386A6108192C076168A
SHA512    C0124A846506B57CE858529968B04D2562F724672D8B9E2286494DB3BBB098978D3DA0A9A1F9F7FF0D3B862F6BD1EB86D301D025B80C0FC97D5B9619A1BD7D86
MD5       30091603F57FE5C35A12CB43BB32B5F5

And, there they are again. The various hashes for our file. Now, let’s add some new files. All we are going to do is copy and paste our hashfile.txt to the same directory two times. Rename them so that in addition to hashfile.txt, you have hashfilecopy.txt and hashfile.copy. Watch those names and file extensions, although really, how important do you have to be? Think about it…

When checking the hash of a file, we verify the file contents have not changed. And they have not been changed! Only the file name and file extension have. You are starting to see how this can be a useful tool and guess what? It is built-in.

Get-ChildItem | Get-FileHash | Format-List

Algorithm : SHA256
Hash      : 3C55E3C7D4C2EEF6910CB70FC425549981528CBBC0400A705104DC09A9391356
Path      : C:\Users\tommymaynard\Documents\PowerShell_Get-FileHash\hashfile.copy

Algorithm : SHA256
Hash      : 3C55E3C7D4C2EEF6910CB70FC425549981528CBBC0400A705104DC09A9391356
Path      : C:\Users\tommymaynard\Documents\PowerShell_Get-FileHash\hashfile.txt

Algorithm : SHA256
Hash      : 3C55E3C7D4C2EEF6910CB70FC425549981528CBBC0400A705104DC09A9391356
Path      : C:\Users\tommymaynard\Documents\PowerShell_Get-FileHash\hashfilecopy.txt

Now real quick, let’s make another change. I am going to copy and paste hashfile.txt one last time. This copy I have renamed to hashfilechanged.txt. I opened it up and added a second sentence to it. Beneath the first line, I wrote, “This is our file at the end.”

Get-Content -Path .\hashfilechanged.txt
This is our file at the beginning.
This is our file at the end.
Get-FileHash -Path .\hashfilechanged.txt | Tee-Object -Variable SaveMeForNow | Format-List -Property Algorithm,Hash
Algorithm : SHA256
Hash      : 6998575555A0B7086E43376597BBB52582A4B9352AD4D3D642F38C6E612FDA76

I used Tee-Object a couple of times in this post to capture the original hash and this one, after adding a second sentence. As you can see, the file contents are indeed different now, even though the files could have had the same name, were they in different directories.

$SaveMeForLater.Hash
$SaveMeForNow.Hash

3C55E3C7D4C2EEF6910CB70FC425549981528CBBC0400A705104DC09A9391356
6998575555A0B7086E43376597BBB52582A4B9352AD4D3D642F38C6E612FDA76

And, the goodie I mentioned. It is official; it will get its own post. Why not? I make the rules. I’ll link it from here once it is up and published!

GitHub Rate Limit REST API, JSON, and Epoch Time

Note: If you read this post and grab some examples from it, make sure you read to the bottom. I wrote my functions using a GitHub-depreciated JSON object up until the final function. You have been warned.

I am working with the GitHub REST API currently and I learned something new. Why not share it and create a resource for myself for the moment in time when I need a refresh.

Unauthenticated API requests to GitHub are limited to 60 per hour. I did some of my own checks, but there is documentation around this. At home, which is where I work these days, the private/NAT’ed IPs on two different computers, behind my public IP address, were two different devices to GitHub. This was not a huge surprise, but good to know, nonetheless.

Let’s take the GitHub rate limit API for a spin. This API is not rate-limited, which means you can call it as many times as you would like in an hour in an unauthenticated manner. GitHub writes, “Accessing this endpoint does not count against your REST API rate limit.” In the below example, we assign a $Uri and $Header variable and use those as a part of our Invoke-RestMethod command, which stores the results in the $Json variable.

$Uri = 'https://api.github.com/rate_limit'
$Headers = @{Headers = 'application/vnd.github.v3+json'}
$Json = Invoke-RestMethod -Uri $Uri -Headers $Headers | ConvertTo-Json
$Json

Once that is done, we output the contents stored in the $Json variable. Let’s chat about some of the data returned in this JSON object. Beneath resources and then core, we have limit, remaining, and used. Limit is how many unauthenticated requests we have (per hour), remaining is how many of 60 we have left in the hour, and used, is how many API calls we have already used this hour. Therefore if we had 59 remaining, we would have 1 used. Toward the bottom, we have rate. The content stored in this object is identical to what is stored in resources.core.

{
  "resources": {
    "core": {
      "limit": 60,        
      "remaining": 60,    
      "reset": 1646979987,
      "used": 0,
      "resource": "core"  
    },
    "graphql": {
      "limit": 0,
      "remaining": 0,     
      "reset": 1646979987,
      "used": 0,
      "resource": "graphql"
    },
    "integration_manifest": {
      "limit": 5000,
      "remaining": 5000,
      "reset": 1646979987,
      "used": 0,
      "resource": "integration_manifest"
    },
    "search": {
      "limit": 10,
      "remaining": 10,
      "reset": 1646976447,
      "used": 0,
      "resource": "search"
    }
  },
  "rate": {
    "limit": 60,
    "remaining": 60,
    "reset": 1646979987,
    "used": 0,
    "resource": "core"
  }
}

In the above, Invoke-RestMethod command, we piped the output to ConvertTo-Json in order to see the JSON object. Below, we will not do that, in order that we can see the PowerShell object and work with its properties. Remember, this command is quite powerful due to its built-in ability to deserialize returned content into something more useable by individuals working with PowerShell.

Without viewing the nested properties we are presented the resources and rate properties. This should seem familiar from the above JSON. We can return the rate property or the resources.core property and get to limit, remaining, and the used property, as well.

$NotJson = Invoke-RestMethod -Uri $Uri -Method Get -Headers $Headers
$NotJson

resources                                          rate
---------                                          ----
@{core=; graphql=; integration_manifest=; search=} @{limit=60; remaining=60; reset=1646692433; used=0; resource=core}

$NotJson.rate

limit     : 60
remaining : 60
reset     : 1646980018
used      : 0
resource  : core

There is another property that we have yet to discuss, and that is the reset property. This value is set on the first API call. It is set 60 minutes into the future, so we know when our remaining value will return to 60, and our used value will return to 0.

This property is recorded and stored as Epoch time (or Unix time, or Unix epoch). It is the number of seconds since January 1, 1970. The below PowerShell will covert the Epoch time to a human-readable form. We will use this further below inside a function we will use to see all of this working together.

((Get-Date 01.01.1970)+([System.TimeSpan]::fromseconds(1646980018))).ToLocalTime()
Thursday, March 10, 2022 11:26:58 PM

Like I often do, I created a function, which I will include below. I will not do a line-by-line explanation. I will discuss it, however. It accepts three parameters. One is the GitHub rate limit URI, one is the Headers we send in with each API call, and the final parameter is a URI of one of my projects on GitHub. Following the parameters, when the function is invoked, it will execute a rate limit GitHub API call and return some data, execute a GitHub API call against my project (with Out-Null [so no output]), and then execute a rate limit GitHub API call and return some data again. This allows us to see the before rate limit information and the after rate limit information. Take a look at the example beneath the function, after you have looked it over.

function Watch-GitHubRateLimit {
	[CmdletBinding()]
	Param (
		[Parameter()]
		$GHRateLimitUri = 'https://api.github.com/rate_limit',
		[Parameter()]
		$Headers = @{Headers = 'application/vnd.github.v3+json'},
		[Parameter()]
		$GHGeneralUri = 'https://api.github.com/repos/tommymaynard/TMOutput'
	)

	$Json = Invoke-RestMethod -Uri $GHRateLimitUri -Headers $Headers
	[PSCustomObject]@{
		Invocation = 'Before'
		Limit = $Json.Rate.Limit
		Remaining = $Json.Rate.Remaining
		ResetEpoch = $Json.Rate.Reset
		ResetReadable = ((Get-Date -Date 01-01-1970)+
			([System.TimeSpan]::fromseconds($Json.rate.reset ))).ToLocalTime()
		Used = $Json.Rate.Used
	}
	
	Invoke-RestMethod -Uri $GHGeneralUri -Headers $Headers | Out-Null

	$Json = Invoke-RestMethod -Uri $GHRateLimitUri -Headers $Headers
	[PSCustomObject]@{
		Invocation = 'After'
		Limit = $Json.Rate.Limit
		Remaining = $Json.Rate.Remaining
		ResetEpoch = $Json.Rate.Reset
		ResetReadable = ((Get-Date -Date 01-01-1970)+
			([System.TimeSpan]::fromseconds($Json.Rate.Reset ))).ToLocalTime()
		Used = $Json.Rate.Used
	}
}
Watch-GitHubRateLimit
Invocation    : Before
Limit         : 60
Remaining     : 60
ResetEpoch    : 1646980115
ResetReadable : 3/10/2022 11:28:35 PM
Used          : 0

Invocation    : After
Limit         : 60
Remaining     : 59
ResetEpoch    : 1646980115
ResetReadable : 3/10/2022 11:28:35 PM
Used          : 1

Now stop and go back and look at the above function. What is wrong with that? Forget GitHub and API calls (kind of). How could that function be improved?

Figure it out? Do you see all that duplicated code? Yuck. Looking at something like this should trigger a feeling that something is wrong. What we need, instead of all that duplicated code, is a nested function. I have a rewrite below. Take a look and then view the next two examples.

function Watch-GitHubRateLimit {
	[CmdletBinding()]
	Param (
		[Parameter()]
		$GHRateLimitUri = 'https://api.github.com/rate_limit',
		[Parameter()]
		$Headers = @{Headers = 'application/vnd.github.v3+json'},
		[Parameter()]
		$GHGeneralUri = 'https://api.github.com/repos/tommymaynard/TMOutput'
	)

	function Get-GitHubRateLimitStats {
		param ($Invocation)
		$Json = Invoke-RestMethod -Uri $GHRateLimitUri -Headers $Headers
		[PSCustomObject]@{
			Invocation = $Invocation
			Limit = $Json.Rate.Limit
			Remaining = $Json.Rate.Remaining
			ResetEpoch = $Json.Rate.Reset
			ResetReadable = ((Get-Date -Date 01-01-1970)+
				([System.TimeSpan]::fromseconds($Json.Rate.Reset ))).ToLocalTime()
			Used = $Json.Rate.Used
		}
	}

	Get-GitHubRateLimitStats -Invocation 'Before'
	Invoke-RestMethod -Uri $GHGeneralUri -Headers $Headers | Out-Null
	Get-GitHubRateLimitStats -Invocation 'After'
}
Watch-GitHubRateLimit
Invocation    : Before
Limit         : 60
Remaining     : 59
ResetEpoch    : 1646980116
ResetReadable : 3/10/2022 11:28:36 PM
Used          : 1

Invocation    : After
Limit         : 60
Remaining     : 58
ResetEpoch    : 1646980116
ResetReadable : 3/10/2022 11:28:36 PM
Used          : 2

Much better, but then I read this: “Note: The rate object is deprecated. If you’re writing new API client code or updating existing code, you should use the core object instead of the rate object. The core object contains the same information that is present in the rate object.” This is taken from this Git Hub Rate limit page. Now that we know this, let’s update the final function so it uses the correct object.

function Watch-GitHubRateLimit {
	[CmdletBinding()]
	Param (
		[Parameter()]
		$GHRateLimitUri = 'https://api.github.com/rate_limit',
		[Parameter()]
		$Headers = @{Headers = 'application/vnd.github.v3+json'},
		[Parameter()]
		$GHGeneralUri = 'https://api.github.com/repos/tommymaynard/TMOutput'
	)

	function Get-GitHubRateLimitStats {
		param ($Invocation)
		$Json = Invoke-RestMethod -Uri $GHRateLimitUri -Headers $Headers
		[PSCustomObject]@{
			Invocation = $Invocation
			Limit = $Json.Resources.Core.Limit
			Remaining = $Json.Resources.Core.Remaining
			ResetEpoch = $Json.Resources.Core.Reset
			ResetReadable = ((Get-Date -Date 01-01-1970)+
				([System.TimeSpan]::fromseconds($Json.Resources.Core.Reset ))).ToLocalTime()
			Used = $Json.Resources.Core.Used
		}
	}

	Get-GitHubRateLimitStats -Invocation 'Before'
	Invoke-RestMethod -Uri $GHGeneralUri -Headers $Headers | Out-Null
	Get-GitHubRateLimitStats -Invocation 'After'
}
Watch-GitHubRateLimit
Invocation    : Before
Limit         : 60
Remaining     : 58
ResetEpoch    : 1646980116
ResetReadable : 3/10/2022 11:28:36 PM
Used          : 2

Invocation    : After
Limit         : 60
Remaining     : 57
ResetEpoch    : 1646980116
ResetReadable : 3/10/2022 11:28:36 PM
Used          : 3

I know how I feel about a post as I am writing it and those feelings stay consistent until the end. I like what I have written here. There is a great opportunity to learn about working with APIs and even some function writing. I hope is it helpful for others, obviously (as it is kind of what I do). The takeaway though is that if there is an API for something, work with it. It is fascinating to gather information from websites and web services via PowerShell. One day you will be asked to do it, so you might as well get some experience now.

Put the Alias Description Property to Work II

In part I of Put the Alias Description Property to Work, I showed and explained how to add a description property to each of my personal, profile script-created aliases. This allowed me to use a basic function, that filtered the Get-Alias results to return only my aliases. In this post, we will take this one step further and we will add descriptions to the built-in aliases in PowerShell. If you remember, we determined that most of the built-in aliases do not include a source property. Therefore, sorting or filtering on that property is of little value.

The source property is not a settable property, meaning that no matter how hard I try, I cannot edit it. The Description property, as we learned, is open for me to make changes. This is also true with built-in aliases and not just the aliases I create. Here is what we are going to do. All the aliases resolve to a command, and all but four of those commands have a source property. We are going to acquire a command’s source and set it as the description of its alias. This might make more sense with some examples, so let’s start there.

But before we do, let’s look at the default properties returned by Get-Alias.

Get-Alias | Get-Random
CommandType     Name                Version    Source
-----------     ----                -------    ------
Alias           pwd -> Get-Location 

Get-Alias returns CommandType, Name, Version, and Source. Kind of. Name is really a property called DisplayName. It may be better to know that now. In the first, real example, we are also going to use Get-Random to randomly select an alias for us. Then, we will return its ResolvedCommandName, DisplayName, Description, and Options properties.

$Alias = Get-Alias | Get-Random
Get-Alias -Name $Alias |
    Select-Object ResolvedCommandName,DisplayName,Description,Options
ResolvedCommandName DisplayName               Description  Options
------------------- -----------               -----------  -------
Connect-PSSession   cnsn -> Connect-PSSession             ReadOnly

Notice that currently, its Description property is blank. In the next example, we will use the value stored in $Alias and run it through a ForEach-Object loop. Take a look at the PowerShell now, and I will do that recently experimented with, line-by-line explanation further below.

Get-Alias -Name $Alias | ForEach-Object {
    if ($_.Description -eq '') {
        $Params = @{
            Name = $_.Name
            Value = $_.ResolvedCommandName
            Description = if ((Get-Command -Name $_.ResolvedCommandName).Source) {
                ((Get-Command -Name $_.ResolvedCommandName).Source)
            } else {'None'}
            Option = $_.Options
            Force = $true
        }
        Set-Alias @Params -Verbose
    }
}

VERBOSE: Performing the operation "Set Alias" on target "Name: cnsn Value: Connect-PSSession".

Line 1: Pipe our single alias into a ForEach-Object loop.
Line 2: Continue into an if construct if the alias’ description is empty. If it is not, we would move on to the next alias if there were more than just one.
Line 3: Create a hash table and store it in the $Params variable.
Line 4, 5: Add the name and the resolved command name into the Name and Value keys, respectively.
Line 6 – 8: Add a description to the Description key based on whether or not there is a source. If there is a source, add it, if there is not a source add the string None.
Line 9, 10: Add the options and $true value into the Option and Force keys, respectively.
Line 12: Invoke the Set-Alias command, splatting the $Params hash table as parameters and associated parameter values.

Let’s rerun the command we did just a minute ago.

Get-Alias -Name $Alias |
    Select-Object ResolvedCommandName,DisplayName,Description,Options
ResolvedCommandName DisplayName               Description                Options
------------------- -----------               -----------                -------
Connect-PSSession   cnsn -> Connect-PSSession Microsoft.PowerShell.Core ReadOnly

We have taken the source of the command Connect-PSSession and placed its value into the description of an alias that resolves to Connect-PSSession. It is not genius, but it is something!

Moving on, the next command pipes out all the known aliases and the values in each of the included properties. I will only include the first ten, as there are almost 150.

Get-Alias |
    Select-Object ResolvedCommandName,DisplayName,Description,Options
ResolvedCommandName DisplayName           Description              Options
------------------- -----------           -----------              -------
Where-Object        ? -> Where-Object                   ReadOnly, AllScope
ForEach-Object      % -> ForEach-Object                 ReadOnly, AllScope
Add-Content         ac -> Add-Content                             ReadOnly
Clear-Host          c -> Clear-Host        tommymaynard               None
Get-Content         cat -> Get-Content                                None
Set-Location        cd -> Set-Location                            AllScope
Set-Location        chdir -> Set-Location                             None
Clear-Content       clc -> Clear-Content                          ReadOnly
Clear-Host          clear -> Clear-Host                               None
Clear-History       clhy -> Clear-History                         ReadOnly

This should look familiar. There are a couple of differences in this code block compared to the one above. One, we are piping in all of the aliases (not just one), and two, there is no Verbose parameter included when the Set-Alias command is invoked. We do not need to output the change made to each alias. Yeah, no thanks.

Get-Alias | ForEach-Object {
    if ($_.Description -eq '') {
        $Params = @{
            Name = $_.Name
            Value = $_.ResolvedCommandName
            Description = if ((Get-Command -Name $_.ResolvedCommandName).Source) {
                ((Get-Command -Name $_.ResolvedCommandName).Source)
            } else {'None'}
            Option = $_.Options
            Force = $true
        }
        Set-Alias @Params
    }
}

This is the same command we saw earlier. I have only included the first ten here, as well. Notice the changes made to the Description property for each alias. Now I can easily see which alias goes with which source.

Get-Alias |
    Select-Object ResolvedCommandName,DisplayName,Description,Options
ResolvedCommandName DisplayName           Description                                Options
------------------- -----------           -----------                                -------
Where-Object        ? -> Where-Object     Microsoft.PowerShell.Core       ReadOnly, AllScope
ForEach-Object      % -> ForEach-Object   Microsoft.PowerShell.Core       ReadOnly, AllScope
Add-Content         ac -> Add-Content     Microsoft.PowerShell.Management           ReadOnly
Clear-Host          c -> Clear-Host       tommymaynard                                  None
Get-Content         cat -> Get-Content    Microsoft.PowerShell.Management               None
Set-Location        cd -> Set-Location    Microsoft.PowerShell.Management           AllScope
Set-Location        chdir -> Set-Location Microsoft.PowerShell.Management               None
Clear-Content       clc -> Clear-Content  Microsoft.PowerShell.Management           ReadOnly
Clear-Host          clear -> Clear-Host   None                                          None
Clear-History       clhy -> Clear-History Microsoft.PowerShell.Core                 ReadOnly

Like all worthy PowerShell code, I am going to wrap this in a function, and for the foreseeable future, copy this into my profile script. There is one change in this version that should be mentioned. The Scope parameter is now being included inside the $Params hash table with the Global value. Like the other parameters and parameters values in this hash table, it will be splatted onto the Set-Alias cmdlet when it is invoked. While the Scope parameter is not required with Get-Alias, it absolutely is with Set-Alias. We want the aliases that exist outside the function to be the ones we modify.

function Set-AliasDescription
Get-Alias | ForEach-Object {
    if ($_.Description -eq '') {
        $Params = @{
            Name = $_.Name
            Value = $_.ResolvedCommandName
            Description = if ((Get-Command -Name $_.ResolvedCommandName).Source) {
                ((Get-Command -Name $_.ResolvedCommandName).Source)
            } else {'None'}
            Option = $_.Options
            Force = $true
            Scope = 'Global'
        }
        Set-Alias @Params
    }
}
Set-AliasDescription

With the descriptions set, we can filter such as in the next two examples. Remember, the Get-Alias Description property is not displayed by default.

Get-Alias | Where-Object -Property Description -eq 'None'
CommandType     Name                                               Version    Source
-----------     ----                                               -------    ------
Alias           clear -> Clear-Host
Alias           cls -> Clear-Host
Alias           man -> help
Alias           md -> mkdir
Get-Alias | Where-Object -Property Description -like '*core'
CommandType     Name                                               Version    Source
-----------     ----                                               -------    ------
Alias           ? -> Where-Object
Alias           % -> ForEach-Object
Alias           clhy -> Clear-History
Alias           cnsn -> Connect-PSSession
Alias           dnsn -> Disconnect-PSSession
Alias           etsn -> Enter-PSSession
Alias           exsn -> Exit-PSSession
Alias           foreach -> ForEach-Object
Alias           gcm -> Get-Command
Alias           ghy -> Get-History
Alias           gjb -> Get-Job
Alias           gmo -> Get-Module
Alias           gsn -> Get-PSSession
Alias           h -> Get-History
Alias           history -> Get-History
Alias           icm -> Invoke-Command
Alias           ihy -> Invoke-History
Alias           ipmo -> Import-Module
Alias           nmo -> New-Module
Alias           nsn -> New-PSSession
Alias           oh -> Out-Host
Alias           r -> Invoke-History
Alias           rcjb -> Receive-Job
Alias           rcsn -> Receive-PSSession
Alias           rjb -> Remove-Job
Alias           rmo -> Remove-Module
Alias           rsn -> Remove-PSSession
Alias           sajb -> Start-Job
Alias           spjb -> Stop-Job
Alias           where -> Where-Object
Alias           wjb -> Wait-Job

Put the Alias Description Property to Work

I use a small handful of my own PowerShell aliases and of course some of those included with PowerShell natively. There is a best practice around aliases in PowerShell, and that is to not use them in anything that is going to live longer than a one-time use. Therefore, do not use them in scripts, functions, and modules (except in the case where your module exports aliases). I would recommend not even using them in forum posts or emails or work-only documentation. Keep them away from your own personal documentation, too. You might know what spsv, ndr, rcjb, and shcm mean today, at this moment, but you may not a few months from now. They have a place, however. Use them in your console, or shell, to speed up what would otherwise be a longer, manually entered command. Let’s check my $PROFILE script and see how many of my own PowerShell aliases I have.

Get-Content -Path $PROFILE | Select-String -Pattern '-alias'
Set-Alias -Name c -Value Clear-Host
Set-Alias -Name psrh -Value Open-PSReadLineHistoryFile
Set-Alias -Name sel -Value Select-Object
Set-Alias -Name wa -Value Watch-Apartment
Set-Alias -Name psgal -Value Show-PSGalleryProjectJob
'l','link' | ForEach-Object {Set-Alias -Name $_ -Value Find-Link}
Set-Alias -Name cts -Value Convert-TextToSpeech

Based on the above command and its results, I have seven. Even that is too many to remember when they are not being used often. Well, I forgot one recently and I was annoyed I had to open my profile script file and search for -alias. Sure, I could have used the above Get-Content command I wrote and tried out a minute ago, but why? Why not avoid ever having to search for my aliases from inside a source file again. They exist in the PowerShell session; why not search there? I am already in the session.

My first thought was, why does Microsoft not indicate which aliases are built-in? There is a source property. It is just too bad that not many aliases include that information. It would be much simple to filter aliases that way, removing those that are not a part of a Microsoft.<something>.<something> module. But, if you return this property there is often nothing. To begin with, there are currently 147 aliases on my machine.

(Get-Alias).Count
147

Only seven include a value in their source property.

Get-Alias | Where-Object Source -ne ''
CommandType     Name                    Version    Source
-----------     ----                    -------    ------
Alias           fhx -> Format-Hex       7.0.0.0    Microsoft.PowerShell.Utility
Alias           gcb -> Get-Clipboard    7.0.0.0    Microsoft.PowerShell.Management
Alias           gin -> Get-ComputerInfo 7.0.0.0    Microsoft.PowerShell.Management
Alias           gtz -> Get-TimeZone     7.0.0.0    Microsoft.PowerShell.Management
Alias           scb -> Set-Clipboard    7.0.0.0    Microsoft.PowerShell.Management
Alias           stz -> Set-TimeZone     7.0.0.0    Microsoft.PowerShell.Management

Weird right? One command alias is from the Microsoft.PowerShell.Utility module and six are from the Microsoft.PowerShell.Management module. It is not like there are only one and six aliases, respectively, from each entire module. There are plenty more commands and aliases, as well. Take a look; I have included both modules. We will begin with the Microsoft.PowerShell.Utility module.

((Get-Command -Module Microsoft.PowerShell.Utility).Name |
    ForEach-Object {Get-Alias -Definition $_ -ErrorAction SilentlyContinue}).Count
43
(Get-Command -Module Microsoft.PowerShell.Utility).Name |
    ForEach-Object {Get-Alias -Definition $_ -ErrorAction SilentlyContinue}
CommandType     Name                        Version    Source
-----------     ----                        -------    ------
Alias           clv -> Clear-Variable
Alias           compare -> Compare-Object
Alias           diff -> Compare-Object
Alias           dbp -> Disable-PSBreakpoint
Alias           ebp -> Enable-PSBreakpoint
Alias           epal -> Export-Alias
Alias           epcsv -> Export-Csv
Alias           fc -> Format-Custom
Alias           fhx -> Format-Hex           7.0.0.0    Microsoft.PowerShell.Utility
Alias           fl -> Format-List
Alias           ft -> Format-Table
Alias           fw -> Format-Wide
Alias           gal -> Get-Alias
Alias           gerr -> Get-Error
Alias           gm -> Get-Member
Alias           gbp -> Get-PSBreakpoint
Alias           gcs -> Get-PSCallStack
Alias           gu -> Get-Unique
Alias           gv -> Get-Variable
Alias           group -> Group-Object
Alias           ipal -> Import-Alias
Alias           ipcsv -> Import-Csv
Alias           iex -> Invoke-Expression
Alias           irm -> Invoke-RestMethod
Alias           iwr -> Invoke-WebRequest
Alias           measure -> Measure-Object
Alias           nal -> New-Alias
Alias           nv -> New-Variable
Alias           ogv -> Out-GridView
Alias           rbp -> Remove-PSBreakpoint
Alias           rv -> Remove-Variable
Alias           select -> Select-Object
Alias           sls -> Select-String
Alias           sal -> Set-Alias
Alias           sbp -> Set-PSBreakpoint
Alias           set -> Set-Variable
Alias           sv -> Set-Variable
Alias           shcm -> Show-Command
Alias           sort -> Sort-Object
Alias           sleep -> Start-Sleep
Alias           tee -> Tee-Object
Alias           echo -> Write-Output
Alias           write -> Write-Output

And continue with the Microsoft.PowerShell.Management module.

((Get-Command -Module Microsoft.PowerShell.Management).Name |
    ForEach-Object {Get-Alias -Definition $_ -ErrorAction SilentlyContinue}).Count
62
(Get-Command -Module Microsoft.PowerShell.Management).Name |
    ForEach-Object {Get-Alias -Definition $_ -ErrorAction SilentlyContinue}
CommandType     Name                         Version    Source
-----------     ----                         -------    ------
Alias           ac -> Add-Content
Alias           clc -> Clear-Content
Alias           cli -> Clear-Item
Alias           clp -> Clear-ItemProperty
Alias           cvpa -> Convert-Path
Alias           copy -> Copy-Item
Alias           cp -> Copy-Item
Alias           cpi -> Copy-Item
Alias           cpp -> Copy-ItemProperty
Alias           dir -> Get-ChildItem
Alias           gci -> Get-ChildItem
Alias           ls -> Get-ChildItem
Alias           gcb -> Get-Clipboard         7.0.0.0    Microsoft.PowerShell.Management
Alias           gin -> Get-ComputerInfo      7.0.0.0    Microsoft.PowerShell.Management
Alias           cat -> Get-Content
Alias           gc -> Get-Content
Alias           type -> Get-Content
Alias           gi -> Get-Item
Alias           gp -> Get-ItemProperty
Alias           gpv -> Get-ItemPropertyValue
Alias           gl -> Get-Location
Alias           pwd -> Get-Location
Alias           gps -> Get-Process
Alias           ps -> Get-Process
Alias           gdr -> Get-PSDrive
Alias           gsv -> Get-Service
Alias           gtz -> Get-TimeZone          7.0.0.0    Microsoft.PowerShell.Management
Alias           ii -> Invoke-Item
Alias           mi -> Move-Item
Alias           move -> Move-Item
Alias           mv -> Move-Item
Alias           mp -> Move-ItemProperty
Alias           ni -> New-Item
Alias           mount -> New-PSDrive
Alias           ndr -> New-PSDrive
Alias           popd -> Pop-Location
Alias           pushd -> Push-Location
Alias           del -> Remove-Item
Alias           erase -> Remove-Item
Alias           rd -> Remove-Item
Alias           ri -> Remove-Item
Alias           rm -> Remove-Item
Alias           rmdir -> Remove-Item
Alias           rp -> Remove-ItemProperty
Alias           rdr -> Remove-PSDrive
Alias           ren -> Rename-Item
Alias           rni -> Rename-Item
Alias           rnp -> Rename-ItemProperty
Alias           rvpa -> Resolve-Path
Alias           scb -> Set-Clipboard         7.0.0.0    Microsoft.PowerShell.Management
Alias           si -> Set-Item
Alias           sp -> Set-ItemProperty
Alias           cd -> Set-Location
Alias           chdir -> Set-Location
Alias           sl -> Set-Location
Alias           stz -> Set-TimeZone          7.0.0.0    Microsoft.PowerShell.Management
Alias           saps -> Start-Process
Alias           start -> Start-Process
Alias           sasv -> Start-Service
Alias           kill -> Stop-Process
Alias           spps -> Stop-Process
Alias           spsv -> Stop-Service

In the end, the Microsoft.PowerShell.Utilityhas 43 aliases and the Microsoft.PowerShell.Management module has 63 aliases. Maybe there is a good reason for the source, not being included most of the time. But, we are here to make my aliases easier to find. The source property is not a settable property, therefore, we are going to use the Description property to indicate when an alias is one of mine. Here are my aliases from earlier, each with a new addition to their Description property. It is my name.

Set-Alias -Name c -Value Clear-Host -Description 'tommymaynard'
Set-Alias -Name psrh -Value Open-PSReadLineHistoryFile -Description 'tommymaynard'
Set-Alias -Name sel -Value Select-Object -Description 'tommymaynard'
Set-Alias -Name wa -Value Watch-Apartment -Description 'tommymaynard'
Set-Alias -Name psgal -Value Show-PSGalleryProjectJob -Description 'tommymaynard'
'l','link' | ForEach-Object {Set-Alias -Name $_ -Value Find-Link -Description 'tommymaynard'}
Set-Alias -Name cts -Value Convert-TextToSpeech -Description 'tommymaynard'

Now, after my profile script runs, I can invoke a modified Get-Alias command to return just my aliases from the current PowerShell session.

Get-Alias | Where-Object -Property Description -eq 'tommymaynard'
CommandType     Name                               Version    Source
-----------     ----                               -------    ------
Alias           c -> Clear-Host
Alias           cts -> Convert-TextToSpeech
Alias           l -> Find-Link
Alias           link -> Find-Link
Alias           psgal -> Show-PSGalleryProjectJob
Alias           psrh -> Open-PSReadLineHistoryFile
Alias           sel -> Select-Object
Alias           wa -> Watch-Apartment

This post would not be complete if there was no new function to add to my $PROFILE script. So, with that, here is that new function and alias. Notice I added the description to the new alias, too!

Set-Alias -Name gmal -Value Get-MyAlias -Description 'tommymaynard'
function Get-MyAlias {
    Get-Alias | Where-Object -Property Description -eq 'tommymaynard'
}
gmal
CommandType     Name                              Version    Source
-----------     ----                               -------    ------
Alias           c -> Clear-Host
Alias           cts -> Convert-TextToSpeech
Alias           gmal -> Get-MyAlias
Alias           l -> Find-Link
Alias           link -> Find-Link
Alias           psgal -> Show-PSGalleryProjectJob
Alias           psrh -> Open-PSReadLineHistoryFile
Alias           sel -> Select-Object
Alias           wa -> Watch-Apartment

Part II has been published!

Republished Work Table of Contents II

Over the last few weeks in February 2022, I have been ensuring that content I wrote on another site, that was lost, has been republished here on tommymaynard.com. Well, it. is. done! In arriving at this point, I wanted to create a table of contents for these old posts with a corresponding link and explanation about the post in case it may be of interest. We will start with the oldest and arrive at the newest. This was some work to put back together!

Top of the Next Hour – February 7, 2022
Determine the “top” of the next hour and create a ScriptMethod for this purpose.

CIDR Notation Host Count – February 8, 2022
Determine the number of available hosts in a CIDR IP address range.

Return Only the Fixed Disks – February 9, 2022
Filter out optical drives and network mapped drives regardless of the count of either.

Clear Host Deconstructed – February 10, 2022
Learn how the Clear-Host function does what it does.

Looking Busy with PowerShell – February 11, 2022
Make PowerShell appear that you are busy.

Really, Remove the Module – February 12, 2022
Make a self-deleting PowerShell module.

Linux Prompt X – February 13, 2022
Create a Unix/Linux prompt on Windows.

Build-in Measure-Command – February 14, 2022
Make built-in commands measure their own execution time.

Get-History Modified – February 15, 2022
Get Get-History to return the time a command took to run.

Switch-Prompt – February 16, 2022
Create a Unix/Linux prompt on Windows, but better.

Filled-In AD Notes Field – February 17, 2022
Append strings to the AD Notes without going over the character limit.

PowerShellGet Find-Both – February 18, 2022
Determine if something from the PowerShell Gallery is a module or a script.

Keeping a Continuous Total – February 19, 2022
Repeatedly update and display a total during a Foreach loop.

AWS Service Acronyms – February 20, 2022
Gather the proper AWS Service acronyms using PowerShell.

Read-Only and Constant Variables – February 21, 2022
Protect variables using the Read-Only and Constant options.

AWS UserData Multiple Run Framework Part IV (a) – February 22, 2022
AWS UserData Multiple Run Framework Part IV (b) – February 22, 2022
AWS UserData framework to configure a Windows EC2 instance between multiple restarts.

AWS Vendor Written Generated Code – February 23, 2022
AWS-generated code did not include any helpful hints that I could have used.

AWS Vendor-Written Generated Code


Notice: The following post was originally published on another website. As the post is no longer accessible, it is being republished here on tommymaynard.com. The post was originally published on September 13, 2019.


Sometimes you read an error message, or in this case, come across some vendor-written code that you can’t find anywhere else on the Internet. It’s been years, but once PowerShell generated an error I had never seen. I couldn’t find a hit for it online anywhere either. I felt that once I had figured out the problem behind that error message, it was my duty to write about it—to help get that error message picked up by search engines, as well as my experience. I feel nearly the same way about the below code to which I was recently introduced, written by AWS, or Amazon Web Services. I’ll share it now. Just a note. This is exactly how it was found. There were no indentations. I’m not too concerned about the lack of indentations—I don’t have to stare at it each day. Perhaps it’s generated by something that may not be preserving the spacing/tabs. That’s just a guess.

try { 
Get-Service Ec2Config 
$EC2SettingsFile='C:\Program Files\Amazon\Ec2ConfigService\Settings\Config.xml' 
$xml = [xml](get-content $EC2SettingsFile) 
$xmlElement = $xml.get_DocumentElement() 
$xmlElementToModify = $xmlElement.Plugins 
foreach ($element in $xmlElementToModify.Plugin){ 
if ($element.name -eq 'Ec2SetPassword') {$element.State='Enabled'} 
elseif ($element.name -eq 'Ec2HandleUserData') {$element.State='Enabled'} 
elseif ($element.name -eq 'Ec2DynamicBootVolumeSize') {$element.State='Enabled'} 
} 
$xml.Save($EC2SettingsFile) 
} 
catch 
{ 
C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -ExecutionPolicy Bypass -File 'C:\ProgramData\Amazon\EC2-Windows\Launch\Scripts\InitializeInstance.ps1' -Schedule 
} 
finally 
{ 
New-Item -Path HKLM:\Software\Amazon -Name WarmBoot 
Invoke-Expression -Command:'shutdown.exe /s /t 0' 
} 
while ($true){ 
}

It’s rare that I ever manually launch an AWS EC2 instance (a virtual server). Well, I was doing that recently for some quick testing and my UserData PowerShell script was not landing in the C:\Program Files\Amazon\Ec2ConfigService\Scripts\UserScript.ps1 file on my Windows Server 2012 R2 instance, as it should have been. I was doing something wrong and it wasn’t clear to me what, soon enough. Before we get to that, let’s discuss what we have here. We have a try-catch language construct. I know from my AWS experience that most of what’s going on in the try block was done for Windows Server 2012 R2 and newer. I also know that what’s in the catch block is how we ensure UserData is enabled in Windows Server 2016 and later. AWS couldn’t take my UserData and drop it on this instance. Instead, I got this code in its place. Ugh.

This code also includes the finally block. That code is run regardless of whether the try or catch block fired. The code creates a value in the Windows Registry and then restarts the computer using Invoke-Expression—interesting choice. It’s always fun to see vendor code. It closes with an empty While language construct. While $true is $true, this While loop will run—and do absolutely nothing. It will do that successfully, however.

Again, my UserData PowerShell wasn’t getting into this UserScript file. It was, however, available by “navigating” to 169.254.169.254 on the EC2 instance.

Invoke-RestMethod -Uri http://169.254.169.254/latest/user-data
# >> Add function to memory.
Function Set-SystemForNextRun {
...
    Set-SystemForNextRun -CodeSectionComplete 2
} # End If-ElseIf.

The problem was that my code didn’t have the begin and end PowerShell tags. In order for the UserScript.ps1 file to be populated with my code and not this code from Amazon, I needed to ensure I was including everything required by me. It seems like something in the AWS Management Console (the web front end) could’ve notified me after looking at my code, but before moving to the next step in manually building my instance. Or, be even less helpful, and additionally write something else in the UserScript.ps1 file. They could’ve just started their code with a comment to tell me I didn’t follow the directions. I’ve used UserData in CloudFormation; I know these tags are required.

<#
This doesn't look right, does it?
Did you remember to use script or
powershell start and end tags?
#>

Anyway, once I enclosed my PowerShell in the proper tags, it worked, and I moved on. And by moved on I mean, I found another problem to consume me—as is typical in this industry. It should’ve look liked this from the start. Ugh.

Invoke-RestMethod -Uri http://169.254.169.254/latest/user-data
<powershell>
# >> Add function to memory.
Function Set-SystemForNextRun {
...
    Set-SystemForNextRun -CodeSectionComplete 2
} # End If-ElseIf.
</powershell>

Things changed between Windows Server 2012 R2 and 2016, 2019. I’m not exactly sure where the UserData code ends up in the newer OS if it even does end up in a file on disk as it has previously.

AWS UserData Multiple Run Framework Part IV (b)


Notice: The following post was originally published on another website. As the post is no longer accessible, it is being republished here on tommymaynard.com. The post was originally published on September 13, 2019.


This article began with Part a. Read that, if you haven’t already.

First off, the code-producing function has been renamed from New-AWSMultiRunTemplate to New-AWSUserDataMultipleRunTemplate. Sure, it’s longer in name, but it’s more clear (to me at least) when I see its name. Other small changes were the removal of unnecessary code comments. While these may have been in the New-AWSUserDataMultipleRunTemplate function itself, they were definitely changed in the produced code. Additionally, I’ve added a ProjectName parameter. The value, supplied to this parameter, is used throughout the generated code for naming purposes within the Registry. There are code changes to the produced code to run against Server 2016 and greater, as well. Therefore, it runs against Windows Server 2012 and 2012 R2 (and probably older), as well as Server 2016 and 2019. Hopefully, it will be extended further, but only AWS knows that for sure. Server 2012 R2 and earlier used EC2Config to configure a Windows instance, while 2016 and 2019 used EC2Launch.

If (test1) {
    {<statement list 1>}
} # End If.

As seen above, I tend to comment on my closing language construct brackets (see # End If.). These are now included in the produced code, both statically and dynamically. It’s a personal preference (that you’ll have to deal with if you use this free, code offering).

This is the new, New-AWSUserDataMultipleRunTemplate code producing function. This isn’t in the PowerShell Gallery, or anywhere else. It probably will never be, either. In fact, this is likely the last time I’ll prep it for public consumption. So again, this is the code you run to create the code you enter in UserData. We won’t discuss what’s in this code; however, we’ll run it further below and discuss that at a minimum.

Function New-AWSUserDataMultipleRunTemplate {
    [CmdletBinding()]
    Param (
        [Parameter(Mandatory)]
        [string]$ProjectName,

        [Parameter()]
        [ValidateRange(1,10)]
        [int]$CodeSectionCount = 2,

        [Parameter()]
        [ValidateSet('All','AllButLast')]
        [string]$EnableUserData = 'AllButLast',

        [Parameter()]
        [ValidateSet('All','AllButLast')]
        [string]$EnableRestart = 'AllButLast'
    )

    Begin { 
        #region Set Write-Verbose block location.
        $BlockLocation = '[BEGIN  ]'
        Write-Verbose -Message "$BlockLocation Entering the Begin block [Function: $($MyInvocation.MyCommand.Name)]."
        #endregion

        Write-Verbose -Message "$BlockLocation Storing the $ProjectName template's function to memory."
        Write-Verbose -Message "$BlockLocation Ensure the server where the code will reside does not already have the ""HKLM:\SOFTWARE\$ProjectName"" path."
        $TemplateFunction = @"
# >> Add function to memory.
Function Set-SystemForNextRun {
    Param (
        [string]`$CodeSectionComplete,
        [switch]`$ResetUserData,
        [switch]`$RestartInstance
    )
    If (`$CodeSectionComplete) {
        [System.Void](New-ItemProperty -Path 'HKLM:\SOFTWARE\$ProjectName' -Name "CodeSection`$CodeSectionComplete" -Value 'Complete')
    } # End If.
    If (`$ResetUserData) {
        try {
            `$Path = 'C:\Program Files\Amazon\Ec2ConfigService\Settings\config.xml'
            [xml]`$ConfigXml = Get-Content -Path `$Path -ErrorAction Stop
            (`$ConfigXml.Ec2ConfigurationSettings.Plugins.Plugin |
                Where-Object -Property Name -eq 'Ec2HandleUserData').State = 'Enabled'
            `$ConfigXml.Save(`$Path)
        } catch {
            C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -ExecutionPolicy Bypass -File C:\ProgramData\Amazon\EC2-Windows\Launch\Scripts\InitializeInstance.ps1 -Schedule
        } # End try-catch.
    } # End If.
    If (`$RestartInstance) {
        Restart-Computer -Force
    } # End If.
} # End Function: Set-SystemForNextRun.

# >> Create/Check for Registry Subkey.
If (-Not(Get-Item -Path 'HKLM:\SOFTWARE\$ProjectName' -ErrorAction SilentlyContinue)) {
    [System.Void](New-Item -Path 'HKLM:\SOFTWARE\' -Name '$ProjectName')
} # End If.

# >> Run user code/invoke Set-SystemForNextRun function.

"@
    } # End Begin.

    Process {
        #region Set Write-Verbose block location.
        $BlockLocation = '[PROCESS]'
        Write-Verbose -Message "$BlockLocation Entering the Process block [Function: $($MyInvocation.MyCommand.Name)]."
        #endregion

        Write-Verbose -Message "$BlockLocation Beginning to create the If-ElseIf code for the template."
        1..$CodeSectionCount | ForEach-Object {
            If ($_ -eq 1) {
                $Start = 'If'
                If ($CodeSectionCount -eq 1) {
                    $End = '# End If.'
                } # End If.
            } ElseIf ($_ -eq $CodeSectionCount -and $CodeSectionCount -eq 2) {
                $Start = 'ElseIf'; $End = '# End If-ElseIf.'
            } ElseIf ($_ -eq $CodeSectionCount -and $CodeSectionCount -ne 2) {
                $End = "# End If-ElseIf x$($CodeSectionCount - 1)."
            } Else {
                $Start = 'ElseIf'
            } # End If.

            If ($EnableUserData -eq 'All') {
                $UserData = '-ResetUserData '
            } ElseIf ($_ -eq $CodeSectionCount) {
                $UserData = $null
            } Else {
                $UserData = '-ResetUserData '
            } # End If.

            If ($EnableRestart -eq 'All') {
                $Restart = '-RestartInstance'
            } ElseIf ($_ -eq $CodeSectionCount) {
                $Restart = $null
            } Else {
                $Restart = '-RestartInstance'
            } # End If.

            $TemplateIfElseIf += @"
$Start (-Not((Get-ItemProperty -Path 'HKLM:\SOFTWARE\$ProjectName').CodeSection$_ -eq 'Complete')) {

    # CodeSection $_.

    Set-SystemForNextRun -CodeSectionComplete $_ $UserData$Restart
} $End
"@
        } # End ForEach-Object.
    } # End Process.

    End {
        #region Set Write-Verbose block location.
        $BlockLocation = '[END    ]'
        Write-Verbose -Message "$BlockLocation Entering the End block [Function: $($MyInvocation.MyCommand.Name)]."
        #endregion

        Write-Verbose -Message "$BlockLocation Creating the AWS UserData Mulitple Run Framework code with $CodeSectionCount code section$(If ($CodeSectionCount -gt 1) {'s'})."
        "$TemplateFunction$TemplateIfElseIf"
    } # End End.
} # End Function: New-AWSUserDataMultipleRunTemplate.

Copy and paste the above, New-AWSUserDataMultipleRunTemplate function into VS Code, or another preferred PowerShell development environment, if there is such a thing. You’re not still using the ISE, are you? Then, add the function to memory for use (in VS Code, that’s Ctrl + A to select all, then F8 to run the selection). Once the function is sitting in memory, we can use it to create our UserData code, as seen below. In this version—it doesn’t even really have a version number, which is weird—there’s now a mandatory ProjectName parameter. Keep this short and simple, and if it were me, I keep spaces and odd characters out of it. This is the value that will be used in the Windows Registry, and within the code that’s produced for UserData.

PS > New-AWSUserDataMultipleRunTemplate -ProjectName MistFit
# >> Add function to memory.
Function Set-SystemForNextRun {
    Param (
       [string]$CodeSectionComplete,
       [switch]$ResetUserData,
        [switch]$RestartInstance
    )
    If ($CodeSectionComplete) {
        [System.Void](New-ItemProperty -Path 'HKLM:\SOFTWARE\MistFit' -Name "CodeSection$CodeSectionComplete" -Value 'Complete')
    } # End If.
    If ($ResetUserData) {
        try {
            $Path = 'C:\Program Files\Amazon\Ec2ConfigService\Settings\config.xml'
            [xml]$ConfigXml = Get-Content -Path $Path -ErrorAction Stop
            ($ConfigXml.Ec2ConfigurationSettings.Plugins.Plugin |
                Where-Object -Property Name -eq 'Ec2HandleUserData').State = 'Enabled'
            $ConfigXml.Save($Path)
        } catch {
            C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -ExecutionPolicy Bypass -File C:\ProgramData\Amazon\EC2-Windows\Launch\Scripts\InitializeInstance.ps1 -Schedule
        } # End try-catch.
    } # End If.
    If ($RestartInstance) {
        Restart-Computer -Force
    } # End If.
} # End Function: Set-SystemForNextRun.

# >> Create/Check for Registry Subkey.
If (-Not(Get-Item -Path 'HKLM:\SOFTWARE\MistFit' -ErrorAction SilentlyContinue)) {
    [System.Void](New-Item -Path 'HKLM:\SOFTWARE\' -Name 'MistFit')
} # End If.

# >> Run user code/invoke Set-SystemForNextRun function.
If (-Not((Get-ItemProperty -Path 'HKLM:\SOFTWARE\MistFit').CodeSection1 -eq 'Complete')) {

    # CodeSection 1.

Set-SystemForNextRun -CodeSectionComplete 1 -ResetUserData -RestartInstance
} ElseIf (-Not((Get-ItemProperty -Path 'HKLM:\SOFTWARE\MistFit').CodeSection2 -eq 'Complete')) {

    # CodeSection 2.

Set-SystemForNextRun -CodeSectionComplete 2
} # End If-ElseIf.
PS C:>

By default, it still creates two code sections, as can be seen just above in the If-ElseIf statement. It can create just one, although that makes less sense for something that can provide multiple opportunities for configuration between restarts. Even if you only need one, this may still be the framework for you. Maybe you need a restart after a single configuration pass. It will do up to 10 code sections if, for some reason, you need that many opportunities to configure a single instance. Not me. I’ve only ever needed three or four total. The New-AWSUserDataMultipleRunTemplate function still includes the EnableUserData and EnableRestart switch parameters. Both have the default parameter value AllButLast, however, both parameters can accept All as the value, too. If this is used for the EnableRestart switch parameter, the EC2 instance will restart after the last time it’s configured (the last code section). In my experience, it’s not always necessary to restart an instance after its final configuration, but this would allow for the time it’s needed. Once you get to know the produced code well, you can manually edit it if necessary.

The produced code has three definitive sections. We’ll start with the below, third section. By default, the New-AWSUserDataMultipleRunTemplate function creates two code sections within the third section. Read that a few times; it’s confusing. It’s like this: The produced code consists of three sections (they each start with # >>). Inside the third one, is where you can have multiple code sections, where user code, that someone else provides, maybe you, is executed against an instance. Notice that there are two comments inside our If-ElseIf statement. On the first pass, it’ll run whatever the code that the user enters to replace “# CodeSection 1.” After the UserData is enabled and the instance is restarted, it’ll run whatever code is entered to replace “# CodeSection 2.” We’ll see more about how it does this shortly.

# >> Run user code/invoke Set-SystemForNextRun function.
If (-Not((Get-ItemProperty -Path 'HKLM:\SOFTWARE\MistFit').CodeSection1 -eq 'Complete')) {

    # CodeSection 1.

    Set-SystemForNextRun -CodeSectionComplete 1 -ResetUserData -RestartInstance
} ElseIf (-Not((Get-ItemProperty -Path 'HKLM:\SOFTWARE\MistFit').CodeSection2 -eq 'Complete')) {

    # CodeSection 2.

    Set-SystemForNextRun -CodeSectionComplete 2
} # End If-ElseIf.

Now that we’ve spent some time with the third section of our produced code, let’s move upward and focus on the middle, or second section, of the code that’s been produced. Here’s that second section, now. It’s real simple. If a specific Windows Registry Subkey doesn’t exist, it’s created. The need to create this will only happen once (the first run). Every other time this If statement fires, it’ll be false and therefore, it won’t attempt to create (something that’s already been created).

# >> Create/Check for Registry Subkey.
If (-Not(Get-Item -Path 'HKLM:\SOFTWARE\MistFit' -ErrorAction SilentlyContinue)) {
    [System.Void](New-Item -Path 'HKLM:\SOFTWARE\' -Name 'MistFit')
} # End If.

And, here’s the first section, last. This function, Set-SystemForNextRun, is placed into memory after this portion of the UserData is executed. It’s invoked by the third section. If you go back up to where we discussed the third section, you’ll see where.

# >> Add function to memory.
Function Set-SystemForNextRun {
    Param (
        [string]$CodeSectionComplete,
        [switch]$ResetUserData,
        [switch]$RestartInstance
    )
    If ($CodeSectionComplete) {
        [System.Void](New-ItemProperty -Path 'HKLM:\SOFTWARE\MistFit' -Name "CodeSection$CodeSectionComplete" -Value 'Complete')
    } # End If.
    If ($ResetUserData) {
        try {
            $Path = 'C:\Program Files\Amazon\Ec2ConfigService\Settings\config.xml'
            [xml]$ConfigXml = Get-Content -Path $Path -ErrorAction Stop
            ($ConfigXml.Ec2ConfigurationSettings.Plugins.Plugin |
                Where-Object -Property Name -eq 'Ec2HandleUserData').State = 'Enabled'
            $ConfigXml.Save($Path)
        } catch {
            C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -ExecutionPolicy Bypass -File C:\ProgramData\Amazon\EC2-Windows\Launch\Scripts\InitializeInstance.ps1 -Schedule
        } # End try-catch.
    } # End If.
    If ($RestartInstance) {
        Restart-Computer -Force
    } # End If.
} # End Function: Set-SystemForNextRun.

In the first code section of the third overall section, Set-SystemForNextRun was first invoked with all three possible parameters. The first parameter is sent in a numeric 1 to the CodeSectionComplete parameter. This created a new Registry value in the “HKLM:\SOFTWARE\MistFit” path. It coerced the number into a string, creating a Registry value named CodeSection1 that had a string value of Complete. These values are how the code knows what’s been done before, and what still needs to be completed. The second parameter was the ResetUserData switch parameter, indicating to enable UserData to run again, the next time the computer is restarted. The third parameter, RestartInstance, as seen below, restarts the computer, right then and there.

Set-SystemForNextRun -CodeSectionComplete 1 -ResetUserData -RestartInstance

In the second code section, it was invoked differently. This time it only updated the Registry. As it was the last code section to execute against the instance, we didn’t opt to enable UserData again or restart the instance. This won’t always be the case. It’s Windows; we may want a final, nerve-calming restart to take place.

Set-SystemForNextRun -CodeSectionComplete 2

This is a good amount of information in which to wrap your head around. The whole PowerShell function to create PowerShell can make it difficult. I did a bunch of renaming of parameters in this version and tripped myself up a few times. Luckily, it was only briefly. Had it not been, I wouldn’t be able to finish writing now and go to bed. All that said, if you have any questions, I shouldn’t be too difficult to track down.

AWS UserData Multiple Run Framework Part IV (a)


Notice: The following post was originally published on another website. As the post is no longer accessible, it is being republished here on tommymaynard.com. The post was originally published on September 12, 2019.


This is the fourth installment of this series. I never thought I’d write another article about it and yet, here I am. One thing before we recap, I’m breaking this fourth installment into two parts: a and b. It’s too much to read at once. Well, it might be for me at least, if I wasn’t the one writing it. The link for Part b will be at the bottom of this post when it’s published, too.

This series of articles began in 2017. First, I’ll explain what each article brought to this series. Then, I’ll discuss the newest changes—the purpose of this fourth article. You know how it goes, though. You get a chance to see your old code, and you almost immediately find things you would’ve done differently. That’s what this is partially about. That and the fact that I may use this framework again here soon.

When you provision a new Windows virtual server (an EC2 Instance) in AWS or Amazon Web Services, you get an option to run batch and/or PowerShell against the instance at the first launch. This allows you, by default, a one-time shot to configure your server as you see fit. In my AWS UserData Multiple Run Framework, you’ve long been able to configure an instance multiple times between multiple restarts. Follow along, as I quickly catch you up and introduce the changes. This is partially due to how AWS has changed things from Server 2012 R2 and earlier, and Server 2016 and later. There’s also some of that traditional code clean-up, to which I eluded, and the addition of new features, as well.

In the first article, I introduced the AWS UserData Multiple Run Framework. It used text files, stored at the root of the C:\ drive, to determine where in the UserData code it should proceed after each restart.

In the second article, I picked up where the first one left off. This article was the first one to include code to create the code that would be used in the EC2 instance’s UserData. It was, and still is, PowerShell creating PowerShell. It still used text files, as it ran each code section against the server.

In the third article, things changed. Here I introduced using the Windows Registry to maintain what had and hadn’t yet been run in UserData. Additionally, I included an updated New-AWSMultiRunTemplate (as you’ll see, the name has changed) function to create the UserData code. As stated in that article and above, this isn’t the code you place inside the UserData section—it’s the code that creates the code you place inside the UserData section. It’s an important distinction.

In this fourth article—as you’ll see in part b—I’ve added and corrected a good deal. I’ve made additions, removals, and changes in both the function that produces the UserData code and within the UserData produced code, as well. In part b we’ll do a quick rundown on the modifications I remember making. Then, we’ll include and invoke our code-producing code. Following that, we’ll cover the produced code for those unfamiliar with this project. This is backward from the previous articles; however, it’s more of a logical flow for moving forward.

Part b