Use of Summary Indexing for Long Term Data from Rolling Index

[philh]

Hi all,

I have the following problem set:

I have an index that rolls out data every 30 days (ie data older than 30 days is removed). There is a subset of data from this index that I would like to query for a longer period of time, say 12 or 24 months. 

I'm fairly new to the idea of summary indexes, but it sounds like the logical solution. However, I'm concerned about losing previous data (that's been removed from the original index) each time the summary index is scheduled to run. Is there a way for a summary index to store the data from old runs so I can build a dataset that encompasses multiple months from the original index? 

Thanks in advance!

[richgalloway]

The idea of a summary index is to retain a subset of data from another index.  The summary index should have different retention settings so it holds data longer than the original index.  Summarized data is independent of the data in the original index so the original data can be removed without affecting the summary.

[ITWhisperer]

If you still need all the "old" data, extend the retention period for the index. If you only need partial data, extract that to the summary index and keep it for longer.

[philh]

Thanks for the reply. I can't extend the retention period for the index since it is the established company retention length. But you're saying I can extract that partial data and hold it in the summary index without it being overwritten, correct? 

For example, I'd have the summary index run on the original index once every 30 days so it can grab all the partial data before it is removed. So after 2 runs, my summary index would have 60 days of partial data. 

Is this logic correct?

[ITWhisperer]

So long as the retention period for your summary index is long enough then yes. Summary indexes will still have their own retention periods but are usually longer than the initial raw data indexes. You would have to find out from your administrators how long your summary indexes can be retained for.