Major Incident Management Playbook

 

Start Here

The Major Incident Management Playbook uses the xMatters communications platform to simulate toolchain integrations. This allows administrators and developers to quickly get a feel for the power of xMatters without spending the time to deploy 3 or more integrations. 

This is one of a series. The others are:

This playbook will simulate an alert from a monitoring tool (see here for a full list!) indicating a slow user response rate in the ACME App. The notification will contain several response options, of special note is the "Create Major Incident". This will simulate reaching out to the ITSM application to create a Major Incident and will include the alert from the monitoring tool. The Major Incident will then notify on the active devices and have a host of useful response options, such as Inform stakeholders for keeping executives in the loop and Create a conference bridge, and Create chat room to quickly get started collaborating and resolving the issue. 

Setup

Navigate to the Developer tab and click the Import Plan button. 

 

Navigate to the Integration Builder and expand the Inbound Integrations:

 

... and click on "Inbound for Conference Bridge", scroll down and copy the URL. 

 

 

Click the breadcrumbs to go back to the Integration Builder and click Edit Constants. Select the "Conference Bridge Endpoint" and paste in the path (or the full url, either will work) and click Save. Repeat for "Inbound for Create Chat" and "Create Chat Endpoint", "Inbound for Inform Stakeholders" and "Inform Stakeholders Endpoint", "Inbound for Major Incident" and "Major Incident Endpoint". 

 

 

Following the Playbook

Think of the playbook as a Choose Your Own Adventure, where you start at point A, but then get a choice to go to point B, C, or D. Or even all of the above! So let's start adventuring. 

Navigate to the Messaging tab and in the MIM Playbook section, click Monitoring Alert. Default information has been entered for you, except for the recipient. Enter your username into the recipient box and click on the user entry. Once you are satisfied, click the Send Message button. 

 

All enabled devices will be targeted with the notification content appropriate for that device. The response options are all the same so on your favorite device, click the "Create Major Incident" option. 

Here is what the notifications will look like on the various devices:

Mobile/Email:

 

SMS:

 

Choosing this response will generate a new event targeting the responder, you in this case, simulating the creation of a new Major Incident in your favorite ITSM application.

 

This new notification now has a different set of response options, each of which we'll discuss individually. 

Assign to Me

This will simulate reaching back into the ITSM application to set the responder, you again, as the assignee. In a real world environment, this is considered a "closed loop" integration because the event is generated in the ITSM application, fired to xMatters, people are notified, they respond and that information is delivered back to the application. This helps keep your teams informed and confident that someone has owned the incident. 

Create Chat Room

This response option simulates creating a chat room in your favorite chat application with the room name matching the major incident number. Of course any chat room could be used, but matching the chat room with the incident number can help in any root cause analysis that needs to be done. From here, you could use the integrations we have to these chat applications to further invite others to assist in quickly getting the major incident resolved. 

Start Conference Bridge

Does a voice to voice collaboration fit the style of your team? Or do upper management types like to drop in to get updates and don't bother to read email? This is a good one for you then. This will actually generate an xMatters hosted conference bridge and invite the responder, guess who? you!, to the bridge with a follow up phone call. For this particular playbook we aren't inviting anyone else so you will just hear music.

The form that launches this bridge is enabled for the Web and Mobile UI, so if you'd like to try it out with your teammates, check out the section below

Inform Stakeholders

Would your upper management types or other interested parties rather just get passive updates, this is the go-to response option. When selecting this from a mobile device or email, make sure to enter some comments as entering the comments is what kicks off the event with the updates you enter.

The response has been sent, but the comments still need to be entered and submitted:

 

Here is another look on mobile:

 

These comments, turned update, will be included in the outgoing notification, in this case, to the responder, but in a real life scenario can be targeted to a group or users. Alternatively, a subscription panel could be created to allow for users to decide what criteria they'd like to be updated on. 

Resolve

Similar to the Assign to Me response option, in a real-life scenario, this will reach into the ITSM application and resolve the issue. For this playbook, the response is just noted. 

Escalate

The Escalate response option is used to tell xMatters that you aren't available to deal with this, but instead of waiting for you to reach out to someone, this will immediately escalate to the next on-call person. In this playbook scenario, since we only targeted one person, the response is simply noted and no further action is taken.  

Alternative Adventures

In the Messaging tab, under the "MIM Playbook" section, there are two other message panels. Details on these are provided below.

Major Incident

This message panel is great for simulating a major incident from your favorite ITSM application. The data is pre-populated, but you are free to edit to your heart's content. Enter some recipients and hit Send Message!

 

 

 

 Conference Bridge

The Conference Bridge form allows you to invite other teammates to a conference bridge. The "Bridge" drop down will have entries for any in-progress xMatters conference bridges so you can quickly invite other members to collaborate. 

 

Troubleshooting tips

I didn't receive any notifications

Uh oh! That's not good. First, check the Reports tab to make sure the event was created. If the event was created, then you'll see a nice happy entry in there, like so:

Clicking on the link will display a dashboard breaking down the recipients:

.. and clicking on each one will show more information. Make sure there are no errors and that the Delivered shows at least 1. 

 

I didn't receive an email/phone call/SMS/push notification

Ok, so you got the notification on some devices, but not all the ones you expected. First, check out your devices by clicking the profile icon in the upper right and select Devices. 

Well, here's a problem, I have a 10004 minute delay after my email! Yea, that's not going to work:

 

If the delays are in good order, click on the Options drop down for each and make sure the device is enabled and the schedule is as expected:

 

You can also see pertinent information in the Event Tracking Report on the Reports tab. This will show any system errors encountered delivering to the device. 

 

 

Choosing a response didn't send me another notification

Well, so not ALL of the responses will send a notification. The Accept, Escalate and End in the Monitoring Alert and the Assign to Me, Resolve and Escalate response options won't trigger an additional notification. Further, the Inform Stakeholders only fires after you enter comments in the dialog box or the text field in the mobile app. 

Outside of those, you should have received a new notification. So we might need to do a little more digging. First, review the Setup section and make sure the constants are set properly. If those look ok, we'll have to dig even deeper. 

Head over to the Developer tab and click Edit > Integration Builder next to the MIM Playbook Comm Plan.

 

Then, expand the Outbound integrations and click the gear icon next to the response option you chose and click the Activity Stream.

If it is blank and logging is turned off, then flip the slider and try your response option again, otherwise inspect for any errors. 

If you're still not sure, reply in the comments below or open up a ticket with our cheerful support peeps. 

 

Have more questions? Submit a request

0 Comments

Please sign in to leave a comment.
Powered by Zendesk