|
Say that you have a huge volume of events, and they come in big batches. Each batch is a discrete unit, and mixing information from the most recent batch with the previous batch is unacceptable. more givens:
Given all this, Is there a good clean way to construct a custom search or a custom view that will be sure to operate only on the events of the most recent batch? |
|
Your best bet is to use the "head" command which can take a predicate instead of an absolute count. For example, the following search only takes (all of) the events from the most recent second from index=_internal: The events here are not in a single second, but this offers a tool that seems to open up a number of other ideas. Is that ability of the head command new in 4.2? It seems like I could use eval and streamstats to keep track of when I see the 'start' event and 'end' event, and then use head to terminate once I get back to the correct head event. Is that what you would do?
(14 Jul '11, 10:19)
nick ♦
Wouldn't this be a good use of a transaction command? especially if you've got a well defined start and stop?
(14 Jul '11, 10:27)
Brian Osburn
transaction wont work here because the set of events needs to be sliced and diced up a number of ways by a lot of different 'stats foo(bar) by baz, bat' searches, and transaction is going to put me in multivalue hell.
(14 Jul '11, 12:15)
nick ♦
This ability of head has been around since 4.1, I believe.
(15 Jul '11, 18:45)
Stephen Sorkin ♦
|
