Getting Data In

How to recreate partial index data with metadata on different Splunk installation?

deepdive100
Loves-to-Learn Everything

I have a Splunk container for development (Dev).  I want to import a slice of data from one index of my production Splunk (Prod) to this container so I can write searches against that data exactly as it appears in Prod. 

Using Export on Prod and Import on Dev is not producing my desired outcome.  Doing this as a single file with a single indexing is creating logs that are indexing the container hostname as the host not the host of the data itself.  The data in the Prod index is of varying sourcetypes so the import is also only creating the sourcetype of the import file, not tha sourcetype from the data itself. 

I'm looking at possibly using the  EventGen app but not sure if this will do what I'm trying to do.

Is what I'm doing possible?  I do not want the entire prod index. I do not want to rsync or otherwise go to the backend to move data.  

EDIT: I modified the title, it seems I want the raw data and metadata to all come over in one package?

Labels (2)
Tags (2)
0 Karma

deepdive100
Loves-to-Learn Everything

So it seems the way forward for me is to write some scripts to pull down `index=app host=each_host sourcetype=each_sourcetype` for a specific time block, export them with the hostname in the title and import each with the hostname widget set to the filename.  One script of API calls with the variables on the hosts and sourcetype should do it.  Will try it out and update here

0 Karma

isoutamo
SplunkTrust
SplunkTrust

Hi

basically you could try to copy those from prod node. Here is an old post about it https://community.splunk.com/t5/Installation/How-to-migrate-indexes-to-new-indexer-instance/m-p/5280...

You should change needed configurations after copy as you want this to be a different host  also you should copy only needed indexes or remove those after rsync.

r. Ismo

 

0 Karma
Get Updates on the Splunk Community!

Enter the Dashboard Challenge and Watch the .conf24 Global Broadcast!

The Splunk Community Dashboard Challenge is still happening, and it's not too late to enter for the week of ...

Join Us at the Builder Bar at .conf24 – Empowering Innovation and Collaboration

What is the Builder Bar? The Builder Bar is more than just a place; it's a hub of creativity, collaboration, ...

Combine Multiline Logs into a Single Event with SOCK - a Guide for Advanced Users

This article is the continuation of the “Combine multiline logs into a single event with SOCK - a step-by-step ...