How to feed bunch of files from the internet itself into SPLUNK

[abhayneilam]

Hi,

I have some files uploaded to the internet ( one folder is there in which the files have been uploaded by some other team ). Each time I have to download those data from the internet folder and keep it in my desktop and have to import into SPLUNK.
Is there any way to import those data directly from the internet to SPLUNK without downloading and keeping it in the desktop.

For example, I have a link , say, http://abc/internet/folder/no=234561 which gets me the data . I want to directly feed this link to SPLUNK so that I dont have to download the data in my desktop, automatically whatever the data is present in that link will be imported to SPLUNK..

Kindly help me regarding this as this is needed in urgent basis

Thanks in advance

[DaveSavage]

I'd agree with Chris G 😉
If you have access to the absolute address you can do it the long way as Splunk will continually index a folder content..
Manager » Data inputs » Files & directories » \\192.168.0.5\users\Public\document...works...trial it on your system, changing everything after '\' to your specifics...Set to 'Continuously index data from a file or directory this Splunk instance can reach...etc

[DaveSavage]

...there were a few more slashes in there when I left it...

[Ayn]

Write a scripted that fetches the data and echoes it to standard output. Then setup a scripted input in Splunk that uses this script.

http://docs.splunk.com/Documentation/Splunk/5.0/Data/Setupcustominputs

[alacercogitatus]

If this data can be pulled programatically via a script language (python, perl, bash, curl, etc), then you can use one of two options

1. Pull the data into Splunk via Scripted Input
   http://docs.splunk.com/Documentation/Splunk/5.0/AdvancedDev/ScriptSetup

2. Pull the data down via crontab to a file. Splunk the file using monitor.
   http://docs.splunk.com/Documentation/Splunk/5.0/Data/Monitorfilesanddirectories

[ChrisG]

Sounds like a good use case for Splunk Storm.