We want to get the follow data into Splunk so that we can create our own dashboards and I thought maybe creating an integration myself that streams the data via HEC would be ideal. Or even route back to xMatters for alerts if errors in integration.
- Integration logs
- Event logs
- On-call schedule
- tbd maybe more later on
I won't know what's ideal (push or pull) until I do some testing. Either create an app on Splunk's end to pull or a communication plan to push. My main question would be
How would you trigger a communication plan for ANY new event without getting all events?
I know there's a from/to for /events but is there a "latest" or would I maybe need to create on "outbound" integration for all existing communication plans to trigger this complan?
Just spitballing ideas
I saw this:
Which gave me some good ideas, but wanted to check in with the community to see if anyone already did this (less work for me :D). Or if anyone has played around with doing this in a communication plan.
Any suggestions would be much appreciated.
Comments
Please sign in to leave a comment.
Hey Ian!
I'm not too familiar with the Splunk side to talk about Pull, but in theory this might be the best option as it would give you a holistic view of the xMatters instance from the outside. I put together a Splunk HEC step a while back that pushes event data into the HEC, but it has the draw back of needing to be on each canvas you want reports on. Which doesn't scale well.
I was about to talk about how the integration logs aren't available, but I am mistaken! They are here, and pushing these from a canvas would be cumbersome as the flow step properties have a max size of 20,000 characters. So if your logs are longer than that you would need to break them up into several properties.
Same goes for the on-call schedules because we don't have a triggering mechanism for when they change you'd have to query every so often to push the latest data.
If you do something like this I would be happy to collaborate and ultimately get it up on our xM Labs site here, so let me know if you need any help.
I hope that helps.
Happy Friday!
--- Travis
Ah, thanks for this. That's unfortunate though, forgot about the 20k limit. I ran into it a while ago when writing automation for user creation. I'd have to probably involve something 3rd party to get them joined together anyways.
Then I'll probably just go with a Splunk add-on as i'll probably have more freedom with it. While pulling the data will also solve the event from/to question, as i can just specific it to pull every few minutes and grab the events/logs from that time. Just thought i'd try going with the more "dynamic" DevOpsie way instead of just a static cronjob, but oh well. The path of least resistance prevails again.
Any idea how long a session lasts for connection via api, before i have to re-auth? Is it linked with what we set for the GUI session? (60 minutes in our case) Asking for OAuth.
The API requests do not keep the session open, so you will need to pass the creds, either basic or OAuth token, each time.