How to Identify a repeated set up messages [for looping process]?

[Mick_OBrien]

We have a job that occasionally loops around the same code spewing out same set of messages [2 different messages from same job] - is it possible to identify processes where the last 2 messages match the previous 2 messages...

.

.

.

message1

message2

message1 <-- starts repeating/looping here

message2

message1

message2

message1

message2

.

.

Any help appreciated.

Mick

[ITWhisperer]

It is possible but it depends on your messages.

Are the messages from the process unique apart from when they are repeated?

Can you correlate messages from the same instance of the process without confusing them with messages from another instance of the process?

Are the loops any bigger or smaller than two messages?

What do you need to be kept in the report, e.g. all messages, just the process id?, just the time of the first duplicated message, just the fact that a process has looped?

[Mick_OBrien]

The messages are valid but once starting to loop indicates issue with process - messages can be from different processes but I am only interested in messages repeating on same process.

[ITWhisperer]

| eventstats count as repeats by process message
| where repeats > 1

[Mick_OBrien]

Sorry - but how does this pick up a set of messages on the same process repeating?

[ITWhisperer]

| makeresults format=csv data="process,message
A,message 0
B,message 0
A,message 1
B,message 1
A,message 2
B,message 2
A,message 1
B,message 3
A,message 2
A,message 1
A,message 2"
| eventstats count as repeats by process message
| where repeats > 1

As you can see, only messages that are repeated are shown

[Mick_OBrien]

I ran from search prompt bar but nothing was returned for result set - is there a specific way to use 'makeresults' syntax?

[ITWhisperer]

Which version of Splunk are you using (the makeresults command changed in version 9).

The makeresults is only to create some example data to show you that the commands work.

[PickleRick]

Ha! Good to know about the makeresults. I didn't know that.

[Mick_OBrien]

Splunk Enterprise

Version:8.2.7.1

[Mick_OBrien]

See below - no output from search string...

[ITWhisperer]

Try this pre-9 syntax

| makeresults
| eval _raw="process,message
A,message 0
B,message 0
A,message 1
B,message 1
A,message 2
B,message 2
A,message 1
B,message 3
A,message 2
A,message 1
A,message 2"
| multikv forceheader=1
| table process,message
| eventstats count as repeats by process message
| where repeats > 1

[Mick_OBrien]

Thanks - the pre-9 syntax works but multiple instances of the same repeated log are displayed....

Is there a way to limit to one set of logs?