We have configured parallelIngestionPipelines as 2 in Splunk HF as we were facing congestion in the TypingQueue while our CPU was underutilized (~2 Cores used/12). However, the load in the pipelines are not balanced. Pipeline 0 is still congested while Pipeline 1 is barely utilized.   Digging around, this seems to be because 80% of our input is on a single UDP port. Will splitting the UDP ports on the source itself solve this issue? i.e. having multiple UDP Inputs on the HF instead of one? ... View more

Was just going through the ‘Masa diagrams’ link: https://community.splunk.com/t5/Getting-Data-In/Diagrams-of-how-indexing-works-in-the-Splunk-platform-the-Masa/m-p/590774 If you look at the "Detail Diagram - Standalone Splunk" , the queues are laid out like this (one example): (persistentQueue) + udp_queue --> parsingQueue --> aggQueue --> typingQueue --> indexQueue  So lets say we have UDP input configured and some congestion occurs in typingQueue, the persistentQueue should still be able to hold the data until the congestion is cleared up. This should be able to prevent the data loss.. right?  Sorry for this loaded assumption based question.. I am trying to figure out what we can do to stop UDP input data from getting dropped due to the typingQueue being filled. (P.S. adding an extra HF is not an option right now).   Thanks in advance! ... View more