I am trying to control ingest rate into Splunk Cloud. I have some firewalls that are very chatty. The firewalls themselves can only point to a single Syslog destination. For security and compliance issues, I need to retain and store ALL logs for one year. We have an appliance that forwards to our SOC and it basically has unlimited storage. For reporting and alerting, I need to send most messages into Splunk Cloud. Logging is controlled by ACL and in the syslog messages, you see ACLs. Based on how my firewall is configured, there are a few ACLs that are chattier than others, for example, the implicit deny ACL is CONSTANTLY chatting. The only time I really need to see this ACL in Splunk logs, is when I am troubleshooting however the SOC wants to see this ACL all the time. The implicit deny rule accounts for about 30% of all syslog data generated. Ideally I when I write to disk on the Syslog-NG server, I would like to drop the implicit deny logs so that when the Universal Forwarder reads the log, it won't be sending that unneeded 30% overhead (the implicit deny rule accounts for about 20-50 gigs of ingest a day alone). My initial log_path statement looks like the following: log {
source(s_udp514);
filter(f_device);
destination(d_socappliance);
destination(d_disk);
flags(final);
}; I then tried 2 different log path statements to try and separate the traffic so that I can apply the message drop filter: filter f_device {
(
host("192.168.1.1") or
host("fqdn.device.com")
)
};
filter f_device_msgdrop {
(
not match("aclID=0" value(MESSAGE));
)
};
log {
source(s_udp514);
filter(f_device);
destination(d_socappliance);
flags(final);
};
log {
source(s_udp514);
filter(f_device);filter(f_device_msgdrop);
destination(d_disk);
flags(final);
}; aclID=0 is the ACL ID of the implicit deny rule. The concept here is that if the string "aclID=0" exists in the syslog message, I don't want to write it to disk and therefore the Universal Forwarder never sees in in the log file and it doesn't get sent to the cloud. When I use the method above, I end up disabling logging to disk. I haven't verified if logging to the SOC appliance stops as well. Any thoughts on how to tackle this?
... View more