Cisco Firepower eStreamer eNcore Add-on for Splunk version 4 Install

[_joe]

I know this version is still in beta, I personally had a lot of issues getting it to install properly. In case it would help someone else, I wanted to provide some notes I wrote up. 

[_joe]

You do not need to create a bookmark file. One reason you might want to would be to avoid ingesting old data.

[_joe]

# Where "10.0.0.1" is the IP of your defense center

# Install the app through the UI and restart

# Missing steps:
# Edit the '/opt/splunk/etc/apps/TA-eStreamer/bin/splencore.sh' and '/opt/splunk/etc/apps/TA-eStreamer/bin/configure.sh' adding the environment variable.
# For most, that will just be uncommenting out "#SPLUNK_HOME=/opt/splunk"

# Make sure your cert is located here: /opt/splunk/etc/apps/TA-eStreamer/bin/encore/client.pkcs12

# Instructions:
# TLS communication with the FMC, running the start up script, $SPLUNK_HOME/bin/bash splencore.sh test will detail those commands

# I had to change the commands a little bit to get them working in my environment:
/opt/splunk/bin/splunk cmd /opt/splunk/etc/apps/TA-eStreamer/bin/splencore.sh test
# Please Enter the FMC IP:10.0.0.1

/opt/splunk/bin/splunk cmd openssl pkcs12 -in /opt/splunk/etc/apps/TA-eStreamer/bin/encore/"client.pkcs12" -clcerts -nokeys -out "/opt/splunk/etc/apps/TA-eStreamer/bin/encore/10.0.0.1-8302_pkcs.cert"
#Enter Import Password:
#MAC verified OK

/opt/splunk/bin/splunk cmd openssl pkcs12 -in /opt/splunk/etc/apps/TA-eStreamer/bin/encore/"client.pkcs12" -nocerts -nodes -out "/opt/splunk/etc/apps/TA-eStreamer/bin/encore/10.0.0.1-8302_pkcs.key"
# Enter Import Password:
# MAC verified OK

Create and update your local folder:

# app.conf
[install]
state = enabled

# encore.conf
[main]
client_enabled = 0
port = 8302
write_packets = 1
host = 10.0.0.1

# inputs.conf
# Where data is written to
[monitor://$SPLUNK_HOME/etc/apps/TA-eStreamer/data]
index = org_ids
disabled = 0

# The current status
[script://./bin/splencore.sh status]
index = org_admin_events
disabled = 0


# Maintain cleanliness every 15 mins - this should never have any output
[script://./bin/splencore.sh clean]
disabled = 0


# The main data source
[script://./bin/splencore.sh start]
disabled = 0

# props.conf
# Sourcetype
[cisco:estreamer:data]
TIME_FORMAT = %s
MAX_TIMESTAMP_LOOKAHEAD = 20
TIME_PREFIX = \sevent_sec=
# the original 'TIME_PREFIX = event_sec' was being damaged by the 'orig_event_sec' field
# TIME_FORMAT = %s was commented out in the default

# NOTE: (Optional)the following transforms replaces the "source" field with the "rec_type_desc" 
# value. I find this to be helpful as it creates an efficient way to search your events after indexing
TRANSFORMS-extract_rec_type = estreamer_replace_source



# transforms.conf
[estreamer_replace_source]
DEST_KEY = MetaData:Source
REGEX = \srec_type_desc\=\"([^\"\=]+)"
FORMAT = source::$1

/opt/splunk/bin/splunk cmd /opt/splunk/etc/apps/TA-eStreamer/bin/splencore.sh test
Please Enter the FMC IP:10.0.0.1
#2020-09-14T14:04:46.643939 Diagnostics INFO Checking that configFilepath (estreamer.conf) exists
#2020-09-14 14:04:46,655 Diagnostics INFO Check certificate
#2020-09-14 14:04:46,655 Diagnostics INFO Creating connection
#2020-09-14 14:04:46,655 Connection INFO Connecting to 10.0.0.1:8302
#2020-09-14 14:04:46,655 Connection INFO Using TLS v1.2
#2020-09-14 14:04:46,892 Diagnostics INFO Creating request message
#2020-09-14 14:04:46,893 Diagnostics INFO Request message=b'0001000200000008ffffffff48900061'
#2020-09-14 14:04:46,893 Diagnostics INFO Sending request message
#2020-09-14 14:04:46,893 Diagnostics INFO Receiving response message
#2020-09-14 14:04:46,902 Diagnostics INFO Response
#2020-09-14 14:04:46,902 Diagnostics INFO Streaming info response
#2020-09-14 14:04:46,902 Diagnostics INFO Connection successful

# control commands:
# /opt/splunk/bin/splunk cmd /opt/splunk/etc/apps/TA-eStreamer/bin/splencore.sh start
# /opt/splunk/bin/splunk cmd /opt/splunk/etc/apps/TA-eStreamer/bin/splencore.sh status
# /opt/splunk/bin/splunk cmd /opt/splunk/etc/apps/TA-eStreamer/bin/splencore.sh stop

If you run into additional issues here, I would suggest enabling/disabling the app and the scripted inputs through the UI.

Also, this app requires the pid and bookmark files in "/opt/splunk/etc/apps/TA-eStreamer/bin/encore"

Also,

[nickleif]

Hey joe, last line of your post about the bookmarks. Did you have to create that bookmark file manually? I get a "Bookmark file <path> not found" error in my estreamer logs.