Configuring the Integration Agent to deduplicate events

Deduplication allows an integration service to prevent "event storms," which can occur when a management system sends multiple notification requests pertaining to a single incident. By detecting and blocking duplicate notification requests, the Integration Agent can prevent event storms from affecting xMatters users.

The Integration Agent's deduplicator has three components:

  • A "filter" which specifies the conditions that will cause incident requests to be discarded.
  • Script instructions that match incident requests to the filter conditions.
  • Deduplication capability in the Integration Agent, which is invoked by the script instructions.

The filter and script instructions must be understood and used properly for the deduplicator to perform correctly.

Deduplication Filters

Deduplication filters are specified in <IAHome>\conf\deduplicator-filter.xml. In order for an integration service to perform deduplication, that integration service must specify a filter that exists in deduplicator-filter.xml. The filter will typically have the same name as the integration service. For example, the Generic integration might have a filter that is specified as follows:

<filter name="generic">
	<predicates>
		<predicate>incident_id</predicate>
		<predicate>person_or_group_id</predicate>
		<predicate>message</predicate>
	</predicates>
	<suppression_period>60</suppression_period>
	<window_size>10</window_size>
</filter>

In addition to its name, a filter has three components:

  • Predicates: Each predicate must correspond to an APXML token* that is present in the data received by the Integration Agent. Based on the values of these predicates, each message will be categorized as "unique" or "duplicate" by the Integration Agent. See below for examples and explanation.
  • Suppression Period: After each unique notification request is received by the Integration Agent, a record of its predicate values is stored in memory for this number of seconds, and any subsequent notification request is discarded if all its predicates match the stored values. When the suppression period expires for those predicate values, the Integration Agent "forgets" that combination of predicates.
  • Window Size: If this number of unique notification requests is received within the suppression period, then the oldest one is "forgotten", meaning that a second such request will be allowed (even though the suppression period has not yet expired).

*Deduplication works with newer "relevance engine" (REST-based) integrations as well as the older APXML based ones, but behind the scenes the event properties are translated to APXML for the purposes of deduplication.

Here are a couple of helpful examples using the above "generic" filter:

Example 1: Requests are deduplicated until the suppression period expires

  • At 10:10:10, the generic integration service receives a notification request with tokens:
    <incident_id>inc1234</incident_id><person_or_group_id>bsmith</person_or_group_id><message>Test message for Bob</message>

    This request is processed and the deduplicator stores "10:10:10, inc1234, bsmith, 'Test message for Bob'" in memory.

  • At 10:10:11, a second request is received that has the same three tokens. The second request is suppressed.
  • At 10:11:10, the 60-second suppression period expires and the "10:10:10, inc1234, bsmith, 'Test message for Bob'" record is purged from the deduplicator's memory.
  • At 10:11:11, another request is received with tokens:
    <incident_id>inc1234</incident_id><person_or_group_id>bsmith</person_or_group_id><message>Test message for Bob</message>

    This request is processed, and the deduplicator stores "10:11:11, inc1234, bsmith, 'Test message for Bob'" in memory.

Example 2: Requests are deduplicated until the window size is exceeded

  • At 10:10:10, the generic integration service receives a notification request with tokens:
    <incident_id>inc1234</incident_id><person_or_group_id>bsmith</person_or_group_id><message>Test message for Bob</message>

    This request is processed, and the deduplicator stores "10:10:10, inc1234, bsmith, 'Test message for Bob'" in memory.

  • At 10:10:11, nine more requests are received, all of which are different from each other as well as from the original request. All the requests are processed and a time-stamped record of each is stored in memory.
  • At 10:10:12, one more unique request is received. A time-stamped record of it is stored in memory. Since eleven unique records have now been received, the window size is exceeded, causing the "10:10:10, inc1234, bsmith, 'Test message for Bob'" record to be purged from the deduplicator's memory.
  • At 10:10:13, the generic integration service receives another notification request with tokens:
    <incident_id>inc1234</incident_id><person_or_group_id>bsmith</person_or_group_id><message>Test message for Bob</message>

    This request is processed, and the deduplicator stores "10:10:13, inc1234, bsmith, 'Test message for Bob'" in memory.

Script Instructions

The following script instructions are required for the deduplicator to perform correctly:

DEDUPLICATION_FILTER_NAME = "generic";
  • This instruction creates a constant whose value matches the name of the deduplication filter.
  • It is typically found in the integration's configuration.js script file (if there is one); otherwise it should be at the top of the main integration script (e.g., generic.js) immediately after the "importPackage", "importClass", and "load" instructions.
  • The constant is used by the utility functions in the XMUtil script package that ships with the Integration Agent.
load("lib/integrationservices/javascript/event.js");
  • This instruction is typically found at the top of an integration service's main script, e.g., generic.js
  • event.js contains instructions to load the other utility script files that ship with the Integration Agent, including xmutil.js
  • If you don't want your integration to load event.js, you can load xmutil.js directly (i.e., by replacing "event.js" with "xmutil.js" in the above instruction). However, due to dependencies, if you load xmutil.js directly, you will also have to load log.js. See the code in the "Example" section below for details.
if (XMUtil.deduplicator.isDuplicate(apxml)) {
	XMUtil.deduplicate(apxml);
	return;
}
  • After the integration script has prepared a message to be sent to xMatters, the above code determines whether the message should be deduplicated or delivered.
  • The argument can be either an APXML message (typically called "apxml" as shown above), or the properties component of an event object (e.g., "event.properties"). If it receives an event object, the xmutil deduplicator will convert it internally to APXML.
XMUtil.deduplicator.incrementCount(apxml);
  • If the notification data is an APXML message, then this code should immediately precede the instruction that sends the notification request to xMatters.
  • If the notification data is an event object, then this code should immediately follow the instruction that sends the notification request to xMatters.
  • The reason for this distinction is:
    • Event objects are sent from the Integration Agent to xMatters via an HTTP POST (typically "XMIO.post"), which may fail and throw an exception. In this case, the Integration Agent will retry the HTTP POST, but if the XMUtil.deduplicator.incrementCount instruction has already been executed, then the retry will be suppressed by the deduplicator.
    • APXML messages are sent to xMatters via the "return" statement at the end of an apia_input() or apia_http() function. The XMUtil.deduplicator.incrementCount instruction will never be executed if it follows the "return" statement.

Example: Adding deduplication to the generic integration

The generic integration script provides useful examples of various Integration Agent scripting practices, but does not include deduplication. To add deduplication to the generic integration, make the following changes in generic.js:

  • Near the top of the file, replace:
    importClass(Packages.com.thoughtworks.xstream.converters.reflection.PureJavaReflectionProvider);
    

    with:

    importClass(Packages.com.thoughtworks.xstream.converters.reflection.PureJavaReflectionProvider);
    
    // Declare the name of the deduplicator filter
    DEDUPLICATION_FILTER_NAME = "generic";
    
    // Load the Integration Agent script packages that include the deduplicator code
    load("lib/integrationservices/javascript/log.js");
    load("lib/integrationservices/javascript/xmutil.js");
    load("lib/integrationservices/javascript/apxml.js");
    load("lib/integrationservices/javascript/xmio.js");
    
  • Replace the apia_input function with the following code:
    function apia_input(apxml)
    {
    	IALOG.info("Apia_input received the following APXML message: \n" + apxml);
    
    	// Event processing code goes here, ie, before the deduplication code
    
    	// Check whether the processed event message is a duplicate, and discard it if so
    	if (XMUtil.deduplicator.isDuplicate(apxml)) {
    	XMUtil.deduplicate(apxml);
    	return;
    	}
    
    	// Increment the deduplicator.  Note that this instruction goes AFTER the call to XMIO.post in REST-based integrations.
    	XMUtil.deduplicator.incrementCount(apxml);
    
    	// Send the message to xMatters and terminate apia_input
    	return apxml;
    }
    
  • You will also need to add a filter to <IAHome>\conf\deduplicator-filter.xml. You can use the example filter from this KBA, exactly as it appears above.
  • Save the files and restart the Integration Agent.

Deduplicator example

For an example of a configured deduplicator filter, you can compare the settings in the default deduplicator.xml file with the sample communication plan (<IAHOME>\applications\sample-relevance-engine\SampleRelevanceEngine.zip) included with the Integration Agent installation.

A couple of things to note:

  • If the filter includes a predicate that does not match any token in the notification request, then that notification request will be ignored by the deduplicator and will never be suppressed.
  • Some early integration scripts used the deduplicator instructions incorrectly, so if the above information contradicts the released code, then the released code is probably at fault.

 

 xMatters reference: DTN-6286 Originally by Jeremy Brown

 

 

 

Have more questions? Submit a request

0 Comments

Please sign in to leave a comment.
Powered by Zendesk