Thanks For Downloading!
Review the documentation below and follow any custom installation steps. If no install steps are listed, most Splunk Apps and Add-ons can be installed as follows:
Unix/Linux: Decompress the downloaded file using a tool like
Big Data archiving and bulk data movement for Splunk. Supports HDFS, NFS, Amazon S3, and Amazon Glacier.
Shuttl provides data archive management for Splunk. It supports backend storage solutions such as: ApacheHDFS, Amazon S3, or NFS attached storage. Shuttl works on the bucket level, and leverages the standard Splunk mechanism for archiving data based on total data size or time expiration. Use of Shuttl eliminates the need for Splunk users to implement their own homegrown solution for bulk-moving data to storage backends.
In addition to Archiving, Shuttl is useful for both compliance needs of data retention, as well as improving performance of Splunk. Shuttl also supports archiving the data in CSV format, and therefore, when data is moved to HDFS, it opens up the data to other tools such as Apache Hive and Hadoop Map Reduce to do further data processing and analysis.
For more information see the following blog articles:
Source code is available here: <https://github.com/splunk/splunk-shuttl>
Quickstart Guide is available here: <https://github.com/splunk/splunk-shuttl/wiki/Quickstart-Guide>
Setup video is available here: <http://www.youtube.com/watch?v=OP7IYNVR5ms>
For feedback, please email shuttl-dev at splunk.com.
Versions and Release Notes
Version 0.8.4 (current version - updated May 08, 2013)
Now comes with both coldToFrozen AND warmToCold transfer retries!
User happiness is expected to go up by 137%!
Version 0.8.3.1 (updated May 02, 2013)
Automatically retries to transfer coldToFrozen failures every 60 secs (configurable)
Version 0.8.2 (updated Apr 17, 2013)
- Fixes Amazon S3 bug, where the aws.secret couldn't contain some characters. Now it can!
Version 0.6.1.1 (updated Apr 17, 2013)
Version 0.8.1 (updated Mar 11, 2013)
* Fixed bug where if an indexer (which is a search peer) does not have Shuttl installed, the resulting error would result in missing values in the UI.
Version 0.8.0 (updated Feb 28, 2013)
Splunk Shuttl 0.8.0 release is another major milestone release.
The main new feature is distributed Shuttl operations from the Splunk Search Head. This means that all Shuttl operations will operate on the entire cluster, and not on a per-indexer basis.
There is no special configuration that needs to be done on the Search Head. Shuttl when installed on a Search Head will query the Search Head for all Search Peers (the Search Peers should all have Shuttl installed of course), and all Shuttl operations will be distributed operations.
Extensive testing has been done for distributed failure scenarios. However, if problems are encountered, please report them ASAP for them to be addressed.
Version 0.7.2 (updated Jan 01, 2013)
Shuttl version 0.7.2 represents a significant leap in functionality. Thanks to our robust user base for the feature request ideas. Now, Shuttl is easier to setup, supports a new backend (Glacier), reduced archive latency, and full support for the latest Splunk 5.0.
In summary, new features include:
- Amazon Glacier storage! Amazon Glacier is an extremely low-cost storage service that provides secure and durable storage for data archiving and backup.
Version 0.6.3 (updated Sep 11, 2012)
* Shuttl tested on CDH3. There is a wirelevel incompatibility with CDH and Apache, so there's documents on how to configure Shuttl to work with CDH3 on the github wiki. https://github.com/splunk/splunk-shuttl/wiki/System-Requirements
* Fix spurious error message when a thaw request is issued while thaw is in progress. No duplicates happen, and all requests are serviced.
* Simplified the failed bucket transfer dashboard.
* When doing thaw operation, the UI can "return" and list a table, before the thaw action completes - Fix is to do thaw operation asynchronously, so the UI returns immediately, while job happens in background.
Version 0.6.2 (updated Sep 07, 2012)
Known Issues being worked on:
The current download (0.8.2) is a download to the sample test app.
The previous version (by date) is labeled as 0.6.1.1 and has no description.
The version before that is 0.8.1.
I have no clue which one is the latest version.
reviewed 02 May, 11:28
accept rate: 100%
Great and useful app!
reviewed 07 Nov '12, 03:14
accept rate: 0%