Philip Colmer
I should start by explaining that I'm quite the Javascript novice, as might become apparent :)
I'm trying to build some automation within xMatters that will call various AWS APIs. To that end, I'm trying to incorporate the AWS SDK as a library in xMatters.
Since I can't use npm to install the package, I'm left trying to download the browser version of the SDK and save that as a library. So I've gone to aws/aws-sdk-js: AWS SDK for JavaScript in the browser and Node.js (github.com) and then downloaded https://sdk.amazonaws.com/js/aws-sdk-2.794.0.min.js and saved that as a library.
My integration script has this snippet in it:
var AWS = require('aws-sdk');
var ec2 = new AWS.EC2({region: 'us-east-1'});
but the code breaks with this error:
TypeError: AWS.EC2 is not a constructor
So, then I try to use Build your own AWS SDK for JavaScript (amazonaws.com) to build a smaller SDK with just the EC2 portion in it.
The AWS documentation says that it is possible to "require" a subset of the entire SDK, which looks like this:
// Import the Amazon S3 service client
var S3 = require('aws-sdk/clients/s3');
// Set credentials and Region
var s3 = new S3({
apiVersion: '2006-03-01',
region: 'us-west-1',
credentials: {YOUR_CREDENTIALS}
});
var S3 = require('aws-sdk/clients/s3');
// Set credentials and Region
var s3 = new S3({
apiVersion: '2006-03-01',
region: 'us-west-1',
credentials: {YOUR_CREDENTIALS}
});
That isn't going to *directly* work with xMatters because it looks like you can only reference the associated library name. So, in my case, I store the smaller library as "aws-sdk-ec2".
Then, with the code looking like this:
var AWS = require('aws-sdk-ec2');
var ec2 = new AWS.EC2({region: 'us-east-1'});
the error is:
Error importing library 'aws-sdk-ec2', line undefined: AWS is not defined
So I'm stuck. Am I trying to achieve something that isn't actually possible in xMatters?
Or is there a solution ... ? :)
Thanks.
0
Comments
Please sign in to leave a comment.
You are definitely on the right track, except the track leads to a dead end :(
We've used browserify to take some of the simpler javascript libraries such as moment.js and lodash and include them as libraries in the flow designer, but the AWS SDK is a different animal entirely. The biggest concern is around the actual HTTP requests. We've built a lot of framework around making these HTTP requests to help protect our infrastructure against abuse and to help secure the credentials involved.
However, I did go down the path of trying to make a call to the API directly, which is essentially building a client library from scratch. I'll add my notes here in case anyone actually wants to continue, but it isn't for the faint of heart.
I started with the startInstance in the EC2 server, here. It makes reference to this AUTHPARAMS, which, unhelpfully is not documented. I managed to dig out this SO post that was more helpful. The links there use the v2, so I found the v4 doc for generating the auth details here.
So far, not bad. However, things get really messy when you get to the actual signing of the request. This is where we get to use all those HMAC libraries to encrypt the signature.
After that it should be pretty easy! :D
But it is more than I'm comfortable walking through on a forum post.
However, not all is lost. You can use the xMatters Agent to execute command line calls, doc here, and from there use the AWS CLI to do whatever you wanted.
If you're looking for something completely serverless, a lambda function could be used to facilitate the call without using an agent. I've done this with a Google Cloud Function, but I'm sure there is similar functionality in AWS.
Thanks, Travis. I really appreciate your replies.
We did start from the perspective of using Lambda but there were a few reasons why I wanted to try and get it working within the xMatters workflow:
1. Security - if SNS triggers the xMatters workflow, that's fine. If we then have to trigger Lambda, it has to be done in a way that ensures that nobody else can trigger it.
2. The AWS operations may "solve" the incident triggered by SNS, or they may not. If they don't, we need to continue with creating a xMatters event. That is much simpler if it is all being handled within the workflow.
I'm not averse to going down the Lambda route if that turns out to be my only option but, having read some of the links you helpfully provided, I'm quite keen to see how far I can take this.
"This is where we get to use all those HMAC libraries to encrypt the signature." - are any of these available directly or do I now need to figure out how to load them as libraries for use within my workflow?
Thanks again.
So, I've found this:
AWS v4 Signature with Crypto Browserify | RunKit
and this:
crypto-browserify/crypto-browserify: partial implementation of node's `crypto` for the browser (github.com)
I've used browserify to build a bundle file that I've then added as a library in xMatters but, when I try to import it with "require", xMatters reports:
Error importing library 'crypto-browserify', line undefined: setTimeout is not defined
I'll carry on digging ...
Travis, is there anything special that has to be done to use the output from browserify as a library in xMatters? I'm partly asking because node.js - Browserify - How to call function bundled in a file generated through browserify in browser - Stack Overflow suggests that the output is intended to be self-contained and not referenced externally.
Hi Travis
"Error importing library 'crypto-browserify', line undefined: setTimeout is not defined" ... from the research I've done, "setTimeout" should be a global function provided by the underlying Javascript environment.
For the xMatters environment, does this function not exist at all, or does it exist under a different namespace?
Thanks, Travis!
Unfortunately, the code you provided was giving me an error when I tried to use the AWS signing example code. However, your suggestion put me on the path to try and find another source of Javascript code that did HMAC SHA-256 ... and I found this:
jsSHA - SHA Hashes and HMAC in JavaScript (coursesweb.net)
I've modified the function "hmacSha256" from the example code, slightly, to get it to work with this library but the good news is that (a) it is generating valid hashes and (b) works in xMatters!
Now I can start crafting functions to call AWS endpoints ...
Winner, winner, chicken dinner! (Sorry - just needed an excuse!)
OK - I've (finally) manage to craft a working AWS EC2 query against the HTTPS endpoint with correct signing. Phew!
Travis, I have a follow-up question if I'm going to be able to get this working "properly" ... the API endpoints *vary by region*. In other words, if I want to do something with an EC2 instance in eu-west-1, I have to use https://ec2.eu-west-1.awsamazon.com.
But xMatters requires the URL to be specified in the endpoint configuration and (as I understand it) that cannot be changed by the script.
Is there any way I can get my script to either modify the endpoint URL or have some way of selecting which pre-configured endpoint I use?
Thanks and have a great weekend.
"Seems kind of silly that the actual API hostname they use varies based on region and service." - their documentation says it is to reduce latency. I had hoped that this meant that you could still go via us-east-1 and just deal with some latency but testing it with the AWS-provided Python script shows that attempting to (say) query an EC2 system in eu-west-2 via the us-east-1 endpoint results in an authentication failure.
"Would you consider writing this up as an xM Labs entry? It would be great to not have to re-create this considering all the work you've put in! If you'd rather just send it over to me, I'd be happy to write it up too." - more than happy to. This work is part of a larger workflow and I'd like to get the complete piece finished before sharing it/writing it up, but definitely happy to share the code with others.
Thanks very much, Travis, for your support and encouragement. I might have gone down the Lambda route if you hadn't given me some of the pointers that have helped me get to this point.
Cheers!
Some blog posts in case anyone else needs to achieve something similar:
Automated AWS EBS expansion with xMatters
Automated AWS EBS expansion with xMatters – part 2
Calling AWS APIs from xMatters
and a couple of git repos:
https://github.com/linaro-its/xmatters-ebs-automation (the individual scripts)
https://github.com/linaro-its/xm-labs-aws-ebs-resize (the complete workflow)