How do I keep a sum total of events before a dedup?

[nfieglein]

Not sure if I am 100% clear in the question, but here is what I am looking to do. I have a stream of incoming messages which have an identifier which can be duplicated (updated statuses). I have a flag which indicates whether the message is a new message and a flag which indicates whether the message is a close of the event. I want to run a dedup to have the current status of the message, but I want to count all of the new messages and compare them to all of the open and closed messages. I cannot seem to find a mechanism for counting a set of events and then counting a subset of those events created through the dedup command. I cannot use transactions as the number of events can range into the millions and the number of messages per event can range in the 20-30s.

Here are the two searches that I want to combine:

index=dccmtdit
| `GetDatesAndTimes`
| stats sum(IsNewDispatch) as DispatchesOpened by QuarterCreated

And:

search index=dccmtdit
| `GetDatesAndTimes`
| dedup  DpsNum sortby -IsClosed -CurrentStatusDateEpoch
| stats sum(IsOpen) as OpenDispatches, sum(IsClosed) as ClosedDispatches, by QuarterCreated

I want to take those results and work further like this:

| search OpenDispatches!=0 OR DispatchesOpened!=0
| eval PercentOpen=exact(OpenDispatches/DispatchesOpened*100)
| eval PercentClosed=exact(ClosedDispatches/DispatchesOpened*100)

[somesoni2 · Accepted solution]

GIve this a try

 index=dccmtdit
| `GetDatesAndTimes`
| eventstats sum(IsNewDispatch) as DispatchesOpened by QuarterCreated
 | dedup DpsNum sortby -IsClosed -CurrentStatusDateEpoch
| stats sum(IsOpen) as OpenDispatches, sum(IsClosed) as ClosedDispatches first(DispatchesOpened) as DispatchesOpened by QuarterCreated
| search OpenDispatches!=0 OR DispatchesOpened!=0
| eval PercentOpen=exact(OpenDispatches/DispatchesOpened*100)
| eval PercentClosed=exact(ClosedDispatches/DispatchesOpened*100)