Runscope Integration



Configure xMatters

Configure Runscope

Test the integration


Download resources

The information in this article is the intellectual property of xMatters and is intended only for use with xMatters products by xMatters customers and their employees. Further, this intellectual property is proprietary and must not be reused or resold.


This article provides installation, configuration, and implementation details when integrating xMatters with Runscope.

How it works

Runscope provides cloud-based and hybrid on-premises solutions that allow businesses to monitor, test, and debug web service APIs. Runscope API tests can be used to test against services available in the public cloud, running on a private network behind a firewall, or running on a local development environment.

Coupled with the power of xMatters alerts, the integration:

  • Quickly identifies and notifies the on-call resource on a variety of devices.
  • Allows for voice, SMS, and push messages to users.
  • Allows users to reply with "Rerun" from their device to run the test again.

Event injection is initiated when a Runscope runs a test. Runscope builds a payload and sends it to xMatters.

When the test passes, nothing else happens. When the test fails, the integration notifies a recipient (an individual or a group) defined in xMatters. Recipients can respond with "Acknowledge", which causes xMatters to stop notifying other users but keeps the event alive, or "Rerun", which causes the integration to trigger another test run in Runscope, ending the current event in xMatters.

If a Runscope test fails while there is a related active event in xMatters, a new event is not created. If a Runscope test has run and passed, the integration terminates all active events related to the test.

Download the integration package

To begin, download the workflow attached to this article; it contains everything you need for this integration. Don't unzip it – you'll import the zip file into xMatters.

Configure xMatters

The first step in setting up your integration is to configure xMatters.

Set up an integration user

This integration requires a user who can authenticate REST web service calls when working with events – these permissions are provided by the "REST Web Service User" role in xMatters. See Create an integration user for more information.

Note: Make sure you keep the user ID and password of this user handy. You'll need them when configuring other parts of this integration.

Create users and groups to receive notifications

The integration notifies the groups or users defined as recipients in the Test Results form. If the recipients are not defined, the xMatters event is not be created.

Import the workflow

The next step is to import the Runscope workflow.

To import the workflow:

  1. In the target xMatters system, go to the Workflows tab and click Import.
  2. Browse to the .zip file you downloaded, or drag it onto the Import Workflow dialog box.
  3. Click Import Workflow.
  4. Click the gear icon beside the workflow, and select Editor Permissions.
  5. Add the integration user, and then click Save.
  6. Click the workflow name to open the Forms tab.
  7. For the Test Result form, click Web Service and select Sender Permissions.
  8. Enter the integration user, and then click Save Changes.
  9. Click Edit beside the form and select Layout.
  10. In the Recipients area, specify a recipient (a user or a group) for events, and then click Save Changes.

Configure Runscope endpoint

After you have imported the workflow, you need to set the authentication for the Runscope endpoint.

  1. Click the Integration Builder tab, and then click the Endpoints button to display the endpoints for the integration.
  2. Enter a valid username and password for Runscope.
  3. Select the Pre-emptive checkbox and then click Save Changes.


Configure inbound integrations

You need to retrieve the URLs of the inbound integrations to configure Runscope.

To configure an inbound integration and retrieve its URL:

  1. Go to the Integration Builder tab and expand the list of inbound integrations.
  2. Click the name of the Test Results integration to view its details.
  3. Scroll down to How to trigger the integration at the bottom of the page, set the authentication method to URL Authentication, and then select the integration user as the authenticating user. The URL trigger is updated to reflect the new user.
    • To be able to select the integration user, you need to be a supervisor of that user and the user needs to be assigned to the REST Web Services role.
  4. Click Copy beside the Trigger field – you'll need this to configure Runscope:


Configure Runscope

Now that you've configured xMatters, you can configure Runscope to integrate with xMatters. The following sections require you to log into Runscope and access the Tests settings page.

Add the webhook

Runscope uses webhooks to initiate the web service calls when Runscope tests run.

Test Results

Purpose: Injects an event into xMatters when a test fails.

  1. To configure a Test Results webhook, click the Tests tab.
  2. Hover over the test and click the Edit button that appears.
  3. In the Editor, expand Test Settings, and then click Webhooks.
  4. Paste the URL from the "Test Results" inbound integration into the URL field, and then click Save:

Test the integration

After you set up the webhook, the best way to check that the integration is working is to configure the Runscope test so it will fail. Before you do, make sure you have access to the device configured for a targeted recipient then run the test.

You should receive a notification from xMatters that the test failed. You can select Acknowledge, which keeps the xMatters event active but stops notifying other recipients, or Rerun, which terminates the event and reruns the test in Runscope. If you select Rerun, you should see a new run of the test in Runscope. If that test also fails, another event is initiated.


There are several places you can inspect when troubleshooting why an event doesn't seem to make it over to xMatters.

Inbound to xMatters

The first place to look is the Integration Builder Activity Stream. While on the Integration Builder tab, expand the Inbound integrations section, click the gear icon beside the inbound integration, and then click Activity Stream.

The Activity Stream contains the incoming (and for outbound integrations, the outgoing) request, any logging statements as well as the final event creation messages.


Outbound from xMatters

For the two-way integration from xMatters to Runscope, the primary source of logging is the Integration Builder Activity Stream for that particular integration.

For example, if you are troubleshooting why a test was not run again after a user replied with "Rerun", check the Activity Steam for the Test Result Callbacks outbound integration.

Download resources


Have more questions? Submit a request


  • 0
    Christine Astle

    Updated the default endpoint setting in the attached communication plan and the endpoint instructions to streamline setup.

Please sign in to leave a comment.
Powered by Zendesk