The information in this article is the intellectual property of xMatters and is intended only for use with xMatters products by xMatters customers and their employees. Further, this intellectual property is proprietary and must not be reused or resold.
By coupling the operational intelligence power of Splunk with the alert intelligence of xMatters, you can:
- Get time critical information to the right person
- Get information to a user via push message, SMS, voice or email
This article details the steps to install the xMatters Alert Action for Splunk.
- Splunk 6.3
- On premise or Splunk Cloud*
- xMatters On Demand 5.5.86
* Note: Contact Splunk support to install the integration in Splunk Cloud.
Create a REST API user
This integration requires a REST API user to authenticate REST web service calls when injecting events.
This user needs to be able to work with events, but not update administrative settings. The best way to create a user for this integration is to have a dedicated “REST Web Service User” role that includes the permissions and capabilities. If this role does not exist in your deployment, you will need to create it, or ask your xMatters Client Success Manager to create it for you. (For detailed procedures about creating the role, see Authentication and Permissions.)
In the following example, this role is named “REST Web Service User”.
To create a REST API user:
- Log in to the target xMatters system.
- On the Users tab, click the Add New User icon.
- Enter the appropriate information for your new user.
- Assign the user to the REST Web Service User role.
- Click Save.
- On the next page, set the web login ID and password.
Create users and groups to target
Import the communication plan
Download the communication plan (.zip file) attached to this article, but do not extract the contents; you can import the archive directly into xMatters.
To import the communication plan:
- In the target xMatters system, on the Developer tab, click Import Plan.
- Click Browse, and then locate the downloaded .zip file.
- Click Import Plan.
- Once the communication plan has been imported, click Plan Disabled to enable the plan.
- In the Edit drop-down list, select Forms.
- For the Splunk Alert Actions form, in the Not Deployed drop-down list, click Create Event Web Service.
- After you create the web service, the drop-down list label will change to Web Service Only.
Accessing web service URLs
To get the web service URL for a form, in the Web Service Only drop-down list, click Access Web Service URL. Copy the highlighted URL at the top of the dialog box.
Note: The Access Web Service URL option may appear twice in the drop-down menu. Ensure that you click the option just below Create Event Web Service.
You’ll need these URLs when you configure the rest of the integration.
Install and configure Splunk
Install the xMatters Alert Actions
In the Splunk UI, click the + icon to head over to the App Store:
In the App Store, search for xMatters and click the install button:
Configure the integration
Click the gear icon to navigate to the list of apps. If necessary, search for the xMatters app, and then click Setup:
Paste in the web service URL for your form, enter the username and password for your REST web services user, and then click Save:
Create an Alert
At this point, everything is configured, and now we just need to tell Splunk to use the new Alert Action. This will vary depending on your use, so this is an example. We are searching for "MY ERROR message here" in all the indexes. After entering the search, click the Save As > Alert option:
Enter a Name and Description and set the Alert Type to
Real-time. Finally, click Add Actions and select xMatters. In the xMatters recipients, enter a semicolon-separated list of recipients (users or groups):
Test the integration
Triggering the alert will depend on the search criteria set up. In the example above, the text
MY ERROR message here will trigger the alert:
$SPLUNK_HOME/var/log/splunkd.log file contains information about the xMatters python script that creates the event. Either inspect this log visually, or use the Splunk web interface to search. The xMatters Alert Action uses the sendmodalert logging channel for debugging. To enable debug level:
- Click Settings > Server settings > Server Logging
- In the search box on the right side, enter sendmodalert to display this alert channel
- Click sendmodalert and change the logging level to DEBUG.
The xMatters.py script will then dump debug level information to the splunkd.log file.
Use the following search to show the latest entries:
sendmodalert to see information printed for the xMatters Alert Action. For example, this shows a successful event creation and shows the event ID returned from xMatters: