Emergency Change Playbook

Start Here

The Emergency Change Playbook uses the xMatters communications platform to simulate toolchain integrations. This allows you to quickly get a feel for the power of xMatters without spending the time to deploy 3 or more integrations.

Playbook Scenario

Steelbrick Inc., has xMatters integrated into their system. Their IT service management application (check out our full list of ITSM integrations) sends an alert to apply a critical patch to resolve a major incident. The resulting notification include the usual Approve and Reject, but what if you need more information before making a decision? Let's see how this plays out in xMatters.

ECR-example.png

Setting up the playbook

In the playbook, we use constants in xMatters to simulate the loop back to the ITSM, so there's a bit of setup you have to do.

You need access to an xMatters instance, of course, and a device on which you can receive notifications. (If you want to try the conference call scenario, make sure you have a voice device handy...)

Download the sample integration:

To use this playbook, first you need to download the sample integration. (Don't worry, clicking the link just takes you to the bottom of the page.) Don't extract the zip file! You'll import it straight into xMatters.

Now that you've got the file downloaded, let's import the playbook and get it set up.

Get the playbook ready to go:

  1. Navigate to the Developer tab and click Import Plan.
    • If you get warnings about languages, it just means you have languages enabled in your instance that we don't have translations for in the sample plan.

ImportPlan.png

  1. After the import is done, click Edit beside the plan and select Integration Builder, then expand the list of inbound integrations:

IntegrationBuilderEC.png

  1. Click "Inbound for Conference Bridge", scroll down, and copy the URL.

CopyURL.png

  1. Click the breadcrumbs to go back to the Integration Builder and click Edit Constants.
  2. Select the "Conference Bridge Endpoint" constant and paste in the path (or the full url, either works) in the Value field and click Save Changes.

EditConstantEC.png

  1. Repeat for "Inbound for Inform Stakeholders" > "Inform Stakeholders Endpoint".

That's it – you're ready to play.

Following the playbook

Think of the playbook as an ever-changing adventure, where you start at point A, but then get a choice to go to point B, C, or D. Or even all of the above! So let's start adventuring.

Navigate to the Messaging tab and, in the Emergency Change Playbook, click Emergency Change Request. Default information has been entered, except for the recipient – that's you! So, in the recipient field, start typing your username to set yourself as the recipient.

Once you're satisfied, click Send Message.

ECRMessageForm.png

All enabled devices are targeted with the notification content appropriate for that device.

Mobile/Email:

MobileECRAlert.png

 

SMS:

mceclip3.png

 The next sections go over what happens with each response option.

Start Conference Bridge

Does voice-to-voice collaboration fit the style of your team? Or do upper management types like to drop in to get updates, not bothering to read email? This is a good one for you then. This actually generates an xMatters hosted conference bridge and invite the responder – guess who? you! – to the bridge with a phone call. For this particular playbook, we aren't inviting anyone else so you'll just hear music.

The form that launches this bridge is enabled for the Web and Mobile UI, so if you'd like to try it out with your teammates, check out the section below.

Inform Stakeholders

This is the go-to response option if your upper management types or other interested parties would rather just get passive updates.

When selecting this from a mobile device or email, make sure to enter some comments since entering comments is what kicks off the Inform Stakeholders event with the updates you enter. Here the response has been sent, but the comments still need to be entered and submitted:

mceclip10.png

Here's another look on mobile:

MobileECRComment.png

These comments are sent as updates to the stakeholders (in this case, you with your VIP hat on but in a real-life scenario, it can target whatever groups or users you want). Another was to keep stakeholders informed is using subscriptions, which allow users to decide what criteria they'd like to be updated on.

Approve & Reject

These are the "normal" change request response options and they simulate approving or rejecting a change from your device. After selecting one of these, in a real-life scenario, xMatters reaches out to the integrated Change ticketing system, updating it with the approve or reject request. For this playbook, no further action is taken, and no further responses are allow after that. The idea is you've completed any collaboration or informing, and now you're ready to take action.

Escalate

Are you swamped dealing with another issue? Or are you on vacation and forgot to set a replacement? The Escalate response option lets you tell xMatters that you aren't available to deal with the issue, so xMatters can immediately push the alert to the next on-call person. In this playbook scenario, since we only targeted one person (yup, you), the response is simply noted and no further action is taken.

End

Already fixed the problem because you're just that good? This option terminates the event in xMatters, and stops any further notifications about it. In a real-life scenario, this option could reach into the Change ticketing system and resolve the issue. For this playbook, the response is just noted.

Try out the Conference Bridge option - for real

The Conference Bridge form allows you to invite other teammates to a conference bridge. This is also the form triggered when you select the Start Conference Bridge response on the original notifitcation.

The "Bridge" dropdown has entries for any in-progress xMatters conference bridges (like the one you started if you selected the Start Conference Bridge response option above) so you can quickly invite other members to collaborate.

ConferenceBridgeFormECR.png

Ready for your next adventure?

Try out one of our other playbooks:

If you want to dive right in, set up one of our ITSM integrations, then maybe link it to one of our continuous integration / delivery or chat integrations.

Troubleshooting tips

I didn't receive any notifications

Uh oh! That's not good. First, check the Reports tab to make sure the event was created. If the event was created, then you'll see a nice happy entry in there, like so:

EventsReportTabECR.png

You can click on the title to display a dashboard breaking down the delivery:

ECRAlertTrackingReport.png

... and clicking on each tile shows more information. Make sure there are no errors and that Delivered shows at least 1.

I didn't receive an email/phone call/SMS/push notification

Ok, so you got the notification on some devices, but not all the ones you expected. First, check out your devices by clicking the profile icon in the upper right and select Devices.

ProfileDevices.png

Well, here's a problem, I have a 10004 minute delay after my email! Yea, that's not going to work:

DeviceSetup.png

If the delays are in good order, click on the Options drop down for each and make sure the device is enabled and the schedule is as expected:

DeviceOptions.png

You can also see pertinent information in the Tracking Report on the Reports tab. This shows any system errors encountered delivering to the device. You can also check out the Log tab in the event report.

EventLog.png

Choosing a response didn't send me another notification

Well, so not ALL of the responses will send a notification. The Approve, Reject, Escalate and End response options won't trigger an additional notification. Further, the Inform Stakeholders only fires after you enter comments in the dialog box or the text field in the mobile app.

Outside of those, you should have received a new notification. So we might need to do a little more digging. First, review the Setup section and make sure the constants are set properly. If those look ok, we need to dig even deeper.

Head over to the Developer tab and click Edit > Integration Builder next to the Emergency Change Playbook Communication Plan.

EditIB.png

Then, expand the Outbound integrations and click the gear icon next to the response option you chose. Click Activity Stream.

IntegrationActivityStream.png

If it's blank, logging might be turned off. Flip the slider and try your response option again; otherwise inspect for any errors.

If you're still not sure, reply in the comments below or open up a ticket with our cheerful support folks.

Download the playbook

 

Have more questions? Submit a request

0 Comments

Please sign in to leave a comment.
Powered by Zendesk