Splunk Search

Using lookup table matches to limit Web datamodel results

adamblock2
Path Finder

I have created a lookup table which contains iocs, a subset of which are IPv4 addresses. I am trying to use events from the Web datamodel to alert on matches in the lookup table. The following is part of a search which I created that does return results. Unfortunately, it loads full number of datamodel events before attempting to match records in the lookup.

| tstats  summariesonly=true   earliest(_time) AS firstEvent latest(_time) AS lastEvent values(Web.action) AS action  values(Web.http_method) AS http_method values(Web.http_user_agent) AS http_user_agent count from datamodel=Web where nodename=Web.Proxy by Web.app Web.src Web.dest
| rename Web.* AS *
| lookup ioc_entries_lookup ioc_string AS src
| table firstEvent lastEvent action src app http_method http_user_agent dest url
| convert ctime(*Event) timeformat="%m/%d/%Y %H:%M:%S"

I am looking for a way to include the lookup as one of the "where" clauses in the tstats command. I tried using inputlookup as a kind of subsearch. Being that there are close to 200,000 records in the lookup, it failed.

| tstats  summariesonly=true   earliest(_time) AS firstEvent latest(_time) AS lastEvent values(Web.action) AS action  values(Web.http_method) AS http_method values(Web.http_user_agent) AS http_user_agent count from datamodel=Web where [|inputlookup ioc_entries_lookup 
    | fields ioc_string 
    | rename ioc_string AS Web.src
    | table Web.src]

Is there another way that this can be done? Is there a way that this can be done using the "lookup" command?

Thank you.

Tags (1)
0 Karma

starcher
Influencer

As you said too big. just use the lookup as a lookup after your stats and use a ... | where isnotnull()

| tstats  summariesonly=true   earliest(_time) AS firstEvent latest(_time) AS lastEvent values(Web.action) AS action  values(Web.http_method) AS http_method values(Web.http_user_agent) AS http_user_agent count from datamodel=Web where nodename=Web.Proxy by Web.app Web.src Web.dest
 | rename Web.* AS *
 | lookup ioc_entries_lookup ioc_string AS src OUTPUTNEW ioc_string as isFound
 | where isnotnull(isFound)
0 Karma
Get Updates on the Splunk Community!

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...

Wondering How to Build Resiliency in the Cloud?

IT leaders are choosing Splunk Cloud as an ideal cloud transformation platform to drive business resilience,  ...

Updated Data Management and AWS GDI Inventory in Splunk Observability

We’re making some changes to Data Management and Infrastructure Inventory for AWS. The Data Management page, ...