Getting Data In

Can you set a certain time forwarding occurs?

aywong
Path Finder

How often does a forwarder check its logs and forward data?
Can I set some sort of configuration where forwarders only forward data at lets say 12:00 AM at night?

Tags (1)
0 Karma
1 Solution

Ayn
Legend

bmacias84
Champion

The easiest way to accomplish this would be a scripted input with Splunk's CLI "add one shotmore" command or with the type/cat command. My example to does not have any logic to handle datetime appends to file names.

Additional info:

  • 35http://docs.splunk.com/Documentation/Splunk/4.3.3/Data/MonitorfilesanddirectoriesusingtheCLI35

  • 35http://docs.splunk.com/Documentation/Splunk/4.3.1/Developer/ScriptedInputsIntro35

  • On your forwarer edit you inputs.conf

    #windows
    [script://.daily_file.bat c:\somedir\somelog.log]
    disabled = true
    index = main
    source = c:\somedir\somelog.log
    sourcetype = ras
    interval = 0 0 * * *

    #*nix
    [script://./bin/daily_file.sh /somedir/somelog.log]
    disabled = true
    index = main
    source = /somedir/somelog.log
    sourcetype = ras
    interval = 0 0 * * *

    Option 1


    #*nix
    #!/bin/bash
    $SPLUNK_HOME/bin/splunk add oneshot $1 -auth admin:changeme
    done


    #windows
    echo off
    %splunk%\splunk add oneshot %1 -auth admin:changeme

    Option 2
    This will read the output from cat or type which would normally be displayed on your CMD prompt.


    #*nix
    #!/bin/bash
    cat $1
    done


    #windows
    echo off
    type %1

    Hope this helps you.

    aywong
    Path Finder

    Thanks this helps!

    0 Karma

    bmacias84
    Champion

    All scripted inputs are ran on the cron notation.

    jguarini
    Path Finder

    for clarification Option1 is a script ran by cron or task manager, while Option 2 is a scripted input in Splunk.

    also note that there should be two additional * for the cron entry

    ie. 0 0 * * *

    bmacias84
    Champion

    @aywong, Why would you want to collect at midnight, can you give your use case?

    Ayn is correct there is not a way directly built into splunk for exact time based forwarding of data, but there are ways to accomplish this. Most monitoring done on a poll interval.

    File monitoring is based on file checksum. I belive it isendpoint_md5 by default.

    0 Karma

    Ayn
    Legend

    No, you cannot.

    Ayn
    Legend

    The forwarder sends data as soon as it has any to send. How often it CHECKS data depends entirely on which type of input you're talking about. Some inputs, like scripted inputs and WMI inputs, will run at certain configurable intervals. Other inputs, like a regular file monitor, will check on such a regular basis that you can consider it to be more or less real-time.

    aywong
    Path Finder

    okay thanks, do you know how often a forwarder checks and forwards data though?

    0 Karma
    Get Updates on the Splunk Community!

    Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

    March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

    What’s New in Splunk App for PCI Compliance 5.3.1?

    The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

    Extending Observability Content to Splunk Cloud

    Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...