Correct me if im wrong here, So basically the following happens
1. When data is received for the first time, it creates a hot bucket. and no replication happens.
2. When the bucket moves from hot to warm, cluster master is notified and replication happens and all the other indexers gets a copy of the bucket.
3. When the bucket moves to cold no changes happen across the cluster.
4. For movement of data from cold to frozen, the following happens? i got this from splunk docs..
when a primary copy freezes, the cluster reassigns the primary to another searchable copy, if one exists. Searching then continues on that bucket with the new primary copy. When that primary also freezes, the cluster attempts to reassign the primary yet again to another searchable copy. Once all searchable copies of the bucket have been frozen, searching ceases on that bucket. when a peer freezes a copy of a bucket, it notifies the master. The master then stops doing fix-ups on that bucket. It operates under the assumption that the other peers will eventually freeze their copies of that bucket as well. If the freezing behavior is determined by the maxTotalDataSizeMB attribute, which limits the maximum size of an index, it can take some time for all copies of the bucket to freeze, as an index will typically be a different size on each peer. Therefore, the index can reach its maximum size on one peer, causing the oldest bucket to freeze, even though the index is still under the limit on the other peers.
Also i have another doubt, when the replication is happening in splunk, and indexer Y gets a bucket because indexer X rolled it from hot to warm, does indexer Y treat it as warm bucket or there is no concept of hot/warm/cold during replication?
... View more