Using CIDR in a lookup table

Question

Fellow Splunkers

I am building a query where I want to report on location based on source IP address. For example within our internal network the subnet 10.50.6.0/24 corresponds to Sydney, whereas 10.50.7.0/24 corresponds to Melbourne. Thus far I have been able to prove my approach using the cidrmatch function as follows:

eval location=case(cidrmatch("10.50.6.0/24",src_ip),"Sydney", cidrmatch("10.50.7.0/24",src_ip),"Melbourne")

Using a case statement doesn't scale when I have hundreds subnets spread across Australia and New Zealand. I was thinking of having a lookup table of the following format:

Subnet,Location
10.50.6.0/24,Sydney
10.50.7.0/24,Melbourne

My problem is that I have not been able to find a way to perform the appropriate lookup.

I have the following questions:

Can any one suggest a way to execute CIDR-based query against a lookup table?
Is there an alternative/better solution to the above that will allow me to match an internal source IP to location of my choosing?

Many thanks

Answer 1

As a matter of personal preference, I don't like using event types the way Stephen suggests, especially if there are lots of subnets.

Try this...

Download this this script (hackish but functional) and put it in apps/search/bin:
http://www.southerington.com/redir.php?id=7
Populate a CSV file with 3 columns, like this:
```
    base,length,subnet_name
    127.0.0.0,8,"Loopback"
    10.50.6.0,24,"Name of Subnet Six"
    10.50.7.0,24,"Name of Subnet Seven"
    ...
```
Sort the CSV by base, and then in descending order by length so that the most precise match will come first. If you put the file in search/lookups, you can still use other searches to generate it, but be aware there are possible race conditions.

Put the following in transforms.conf:

[subnetlookup]
    external_cmd  = subnetlookup.py
    external_type = python
    fields_list   = ip, subnet_name

Run your search, e.g:

src_ip=* | head 20 | lookup subnetlookup ip as src_ip OUTPUT subnet_name

Answer 2

As gkanapathy said, this is not possible with CSV lookup tables, but is feasible with external lookups. That said, I think that eventtypes are probably the right knowledge primitive to use with version 4.1 (and 4.2, when it comes out) to solve this problem. For example, in our internal web analytics app, we use eventtypes to "classify" events as being from bots, browsers, internal IPs and in turn pageviews are driven by eventtypes.

You can configure eventtypes.conf to enumerate the regions like:

[location-australia-sydney]
search = Subnet=10.50.6.0/24

[location-australia-melbourne]
search = Subnet=10.50.7.0/24

To retrieve, you can search for eventtype=location-australia-sydney. If you want all of Australia, you can search for eventtype=location-australia-*. Since these are eventtypes, events are tagged with these for later filtering or reporting.

If you just want to report on these locations, you can add to your search:

... | eval location = mvfilter(eventtype LIKE "location-%") | ...

Answer 3

I realize that my answer may come a bit late, but you could do it through lookups. Perhaps these options were not available at the time of your posting.

Create a CSV lookup table with your subnets (in my example I've used a different type of classification for identifying potentially bad site users). We'll call this file knownips.csv and put it in /opt/splunk/etc/system/lookups.

clientip, honesty, comment
170.192.178.10/32, BAD, open_proxy
63.236.6.247/32, BAD, open_proxy
177.23.21.223/32, BAD, TOR_exit_node
27.35.0.0/16, OK, friendly_partner
27.96.0.0/16, BAD, hostile_competitor
195.69.252.0/24, BAD, suspicious_ISP

in transforms.conf

[checkip]
filename = knownips.csv
max_matches = 1
min_matches = 1
default_match = OK
match_type = CIDR(clientip)

in props.conf

[access_combined]
LOOKUP-check = checkip clientip OUTPUT honesty comment

This lets you run searches like;

sourcetype="access_combined" honesty=BAD | stats count by clientip, comment

with results like;

clientip            comment     count
170.192.178.10  open_proxy   78
177.23.21.223   TOR_exit_node    12
189.222.1.22    TOR_exit_node    26
195.69.252.22   suspicious_ISP  138

Hope this helps,

Kristian

Answer 4

1

No. CSV lookup tables in the current version (4.1.4) must have an exact string match (optionally case-insensitive) to the field.

But you could do this using a scripted lookup instead that executed the logic above. You can take a look at $SPLUNK_HOME/etc/system/bin/external_lookup.py or here.

link

answered 23 Aug '10, 01:39

gkanapathy ♦
26.5k●1●6●22
accept rate: 42%

edited 23 Aug '10, 02:56

Using CIDR in a lookup table

Follow this question

Splunk Resources

Related questions