Subsearch within a bucket

Question

I am looking to correlate events from two different sources whereby a rare event in source A, (in a 1 hour window) searches for events in source B (in the same 1 hour window)

These are mobile phone records, and source A lists the Phone Numbers I want to seach for in the CDR logs (source B).

This is what I have so far:

eventtype=phone_logs flagged=true |bucket  _time span=1h|stats dc(phone_log_cli) by _time phone_log_cli |join max=0 phone_log_cli[search sourcetype=cdr SCRIPT_RES="d*" BDA="*" BDUR>5]|dedup ID|eval AOA2=substr(AOA,1, 4)+"xxxxxx"| eval BDA2=substr(BDA,1,6)+"xxxxxxxx"| eval CallTime=ACONDATE|rename SCRIPT_RES as "CallType" |table _time CallTime AOA2 BDA2 BDUR BREASON customer country Network CallType

This returns bucket windows for source A, but as the search runs it populates the first bucket where a cli appears with ALL the CDR records of the same device.

That is to say (over a 4 hour search window 08:00-12:00), if a phone made 1 call an hour in each of the 4 hours, then all 4 calls show up in the 08:00 result set.

I want to run this query over 30 days, and at the moment, every call gets returned in the first window in which its CLI was flagged in source A.

How can I restrict the 'join search' to only search within the same time window as the bucket from first search? I have looked at localize - map, but this seems problematic also.

Answer 1

Not trying to go into your question in depth, so I concentrating on your first paragraph.

Could you pick out date_hour and date_mday for your rare events and use it to limit the search among the more common events.

UPDATE:

I was thinking along the lines of a subsearch, but as I said I haven't really looked at your question, so I'm not sure it would apply. My line of thinking, based on your first paragraph was;

source=B [search source=A phoneno=XXX blaha=YYY| dedup phoneno date_mday date_hour| fields + date_time date_mday phoneno]| the_rest_of_your_search_here

The inner search would look at your rare events and return unique combinations of phoneno date_mday and date_hour, which would be used as part of the outer search in source B.

Hope this helps at least a little bit.

/K

Answer 2

Ok, so... The Subsearch works as per your example, but i want to pull some fields out of source A which are used in a table.

using your example:

source=B [search source=A phoneno=XXX blaha=YYY| dedup phoneno date_mday date_hour| fields + date_time date_mday phone no username]| table srcA_phoneno srcA_username srcB_AOA srcB_BDA

obviously, adding 'username' to the fieldset for the subsearch results in the main search looking for matches on the username also (which will not exist in source B)

how can i get additional fields from the sub search to be available for post processing along with results from the main search?

Subsearch within a bucket

Follow this question

Splunk Resources

Related questions