Hi: I am trying to do looping search using lookup tables and map command, however, I cannot get the correct result. If possible, please help me get correct search command.
It is my logs.
First, I would like to get the value of dnsinfo_hostname field. Then I do lookup from the following csv file
dnsinfo_hostname, resolved_IP etsiunjour.fr, 220.127.116.11 etsiunjour.fr, 18.104.22.168 etsiunjour.fr, 22.214.171.124 etsiunjour.fr, 126.96.36.199 aaa.com, 188.8.131.52 bbb.com, 184.108.40.206 ccc.com, 220.127.116.11
When I did the search to get dnsinfo_hostname=etsiunjour.fr with its resolved_Ip=[18.104.22.168, 22.214.171.124 ,126.96.36.199, 188.8.131.52] . For each resolve_IP, do lookups csv fil again to get:
184.108.40.206 ->[aaa.com, ccc.com] 220.127.116.11 ->[bbb.com]
Finally. I would like to show : hostname=etsiunjour.fr, resolved_IP=[18.104.22.168, 22.214.171.124 ,126.96.36.199(aaa.com, ccc.com), 188.8.131.52]，
Is it possible Splunk can help me do this ? Or I have to do it using external python code. Thanks!
asked 17 Aug '12, 01:43
It's fairly straightforward to get something that KIND OF works. To get a list of all the IP's and domains associated with the input domain, you could do:
This will first of all run a subsearch that gets all IP's associated with the "etsiunjour.fr" domain in the lookup, then once again check the lookup which domains are associated with those IP's. The output will be the resolved IP's and a list of associated domains for each.
Moving on from that, there are a number of tricky things that need to be solved in order to get the exact format you're looking for. You need to find the domains that are NOT the input domain, and add them to a comma separated list that is then put within parantheses after the IP number. Finally, all these IP's including any extra info should be put into a multivalued field. After messing around a bit, I came up with the following search that should do what you want:
While I can't say it looks pretty, it does work and shows some of the flexibility you get with Splunk's search language. :)
Note that the input domain ("etsiunjour.fr") is used in three places in the search. This is because once the initial resolving and reverse resolving is done, it's hard to know what the original input domain was. The easiest would be to implement this search as a macro, or in a form that can just refer to the input domain as a variable name.
answered 17 Aug '12, 06:25
I wouldn't use the map command, it is a very expensive command to use in terms of search processing. Try this one:
sourcetype=YOURSOURCETYPE index=*| lookup YOURFILE.csv dnsinfo_hostname OUTPUT resolved_IP|stats values(dnsinfo_hostname) as original_hostname by resolved_IP|lookup YOURFILE.csv resolved_IP OUTPUT dnsinfo_hostname
answered 17 Aug '12, 06:35