Hi, I've got the latest Splunk Stream app installed & configured to accept Netflow v9 events from my router. This part works fine actually. However, when starting to dig deeper in "useful" fields it seems I'm missing a few...I would expect the Stream App to be able to cope with everything "standard" within v9/IPFIX packets/templates. When going to the STM App -> Configuration -> Configure Stream -> Netflow -> "Edit" you get this nice list of about 156 fields which I all enabled. Now I've taken Wireshark capture from the v9 data arriving at my Splunk server and the "template" contains these fields below. Field (20/23): postNATSourceIPv4Address Type: postNATSourceIPv4Address (225) Length: 4 Field (21/23): postNATDestinationIPv4Address Type: postNATDestinationIPv4Address (226) Length: 4 Field (22/23): postNAPTSourceTransportPort Type: postNAPTSourceTransportPort (227) Length: 2 Field (23/23): postNAPTDestinationTransportPort Type: postNAPTDestinationTransportPort (228) Length: 2 And a typical populated capture would look like this : Cisco NetFlow/IPFIX Version: 9 Count: 7 SysUptime: 873590.040000000 seconds Timestamp: Jun 22, 2020 10:03:08.000000000 CEST CurrentSecs: 1592812988 FlowSequence: 22 SourceId: 0 FlowSet 1 [id=256] (7 flows) FlowSet Id: (Data) (256) FlowSet Length: 532 [Template Frame: 1] Flow 1 [Duration: 0.000000000 seconds (switched)] StartTime: 873528.130000000 seconds EndTime: 873528.130000000 seconds Packets: 1 Octets: 86 InputInt: 15 OutputInt: 14 SrcAddr: IP.OF.INTERNAL.PC DstAddr: SOME.PUBLIC.ISP.DNSADDRESS Protocol: UDP (17) IP ToS: 0x00 SrcPort: 51020 (51020) DstPort: 53 (53) NextHop: SOME.PUBLIC.ISP.DNSADDRESS DstMask: 0 SrcMask: 0 TCP Flags: 0x00 Destination Mac Address: Router12_12:12:c6 (61:3c:61:31:11:b1) Source Mac Address: ASRockIn_84:01:36 (d0:50:99:84:01:36) Post Source Mac Address: 00:00:00_00:00:00 (00:00:00:00:00:00) Post NAT Source IPv4 Address: MY.PUBLIC.ISP.ADDRESS Post NAT Destination IPv4 Address: SOME.PUBLIC.ISP.DNSADDRESS Post NAPT Source Transport Port: 51020 Post NAPT Destination Transport Port: 53 Looking again at my Splunk Enterprise intstallation, there is this "vocabulaire" file under opt/splunk/etc/apps/Splunk_TA_stream/default/vocabularies It contains the "netflow.xml" file with (what I think) all field that can be "interpreted" / "decoded" as the Netflow packets arrive. <Term id="netflow.postNATSourceIPAddress"> <Term id="netflow.postNATDestinationIPAddress"> <Term id="netflow.postNAPTSourceTransportPort"> <Term id="netflow.postNAPTDestinationTransportPort"> So ... these 4 fields seem to already be part of the default vocabulary ... yet they never show up as any accessible "field" in Splunk ? In a moment of madness, I've edited the file below and make some additions. (router = Mikrotik = IANA Vendor ID 14988) user@splunky:/opt/splunk/etc/apps/Splunk_TA_stream/default# more streamfwd.conf [streamfwd] port = 8889 ipAddr = 127.0.0.1 netflowReceiver.0.ip = IP.OF.MY.SPLUNK netflowReceiver.0.port = 9995 netflowReceiver.0.decoder = netflow netflowElement.0.enterpriseid = 14988 netflowElement.0.id = 225 netflowElement.0.termid = netflow.postNATSourceIPAddress netflowElement.1.enterpriseid = 14988 netflowElement.1.id = 226 netflowElement.1.termid = netflow.postNATDestinationIPAddress netflowElement.2.enterpriseid = 14988 netflowElement.2.id = 227 netflowElement.2.termid = netflow.postNAPTSourceTransportPort netflowElement.3.enterpriseid = 14988 netflowElement.3.id = 228 netflowElement.3.termid = netflow.postNAPTDestinationTransportPort ...then stop/start my Splunk but these fields don't show up with the 156 possible fields under the "stream config" tab (see earlier) To cut a long story short : Where are these fields ? Why are they not showing up since they are hitting the Splunk Stream App and it seems they are "known" Thanks!
... View more
