About ClubMed

ClubMed · ‎03-29-2024

Thank you! The first appendpipe achieved the desired objective! The size constraint should not be a problem because I had all the unixtime snapped to the month with @mon so there's only 300 rows in this table. The way to explain this odd situation is that each day, we get the data dump of the population but their field values may change by the day. The issue is that Splunk has a 90 day data retention policy for our events. So basing events purely on _time only goes back 90 days. BUT, in our events, there are additional unixtime fields (two to be exact) that go back much further than 90 days and we needed to use these to provide a historical month by month view (hence snapping unixtime with @mon). Total_A was the total sum of the population over time based on Unixtime_A, and Total_B is a conditional sum of the population where only a field met a condition, and Unixtime_B contained the time this condition was first met. That's why I wanted Total_A and Total_B to be seperate, but Unixtime_A and Unixtime_B could be appended together. To put some context to it, Total_A is total vulnerabilities population regardless of whether it was fixed or active based on Unixtime_A being when it was first detected. Total_B is total fixed vulnerabilities population based on Unixtime_B being when it was fixed.

ClubMed · ‎03-28-2024

From the Subject Title, what I mean is it will increase the row count and decrease the column count - that is my intention. After a series of mathematical computations, I ended up with the following table: Unixtime_A Total_A Unixtime_B Total_B imaginary_unix_1 1 imaginary_unix_3 4 imaginary_unix_2 2 imaginary_unix_1 5 imaginary_unix_3 3 imaginary_unix_4 6 Notes: Unixtime_A may not equal Unixtime_B, but they are formatted the same that is snapped to the month with @mon (unixtime) Total_A and Total_B were the result of various conditional counts, so they need to be seperate fields The desired table is: Unixtime_AB Total_A Total_B imaginary_unix_1 1 imaginary_unix_2 2 imaginary_unix_3 3 imaginary_unix_3 4 imaginary_unix_1 5 imaginary_unix_4 6 Which I can then use | fillnull and use a simple stats to sum both totals by Unixtime_AB. Like so: | stats sum(Total_A), sum(Total_B) by Unixtime_AB I'm not 100% sure if transpose, untable, or xyseries could do this - or if I was misusing them somehow.

ClubMed · ‎03-28-2024

Hi Bowesmana, I want to say I really appreciated the level of detail you have provided and it will prove useful for future people who lands here from google searches. So I actually found the problem: I use the Splunk Cloud product so we use a series of API calls to populate objects in the cloud, since we do not have access to the directories on their EC2 instances. I found the Splunk App for Lookup File Editing API that our CI/CD process uses may have not supported kvStore acceleration options. On github, I had the following collections.conf in our master branch – so I incorrectly assumed this was production: [qualys_kb_kvstore] accelerated_fields.QID_accel = {"QID": 1, "PATCHABLE": 1} replicate = true I tested the Splunk KVstore API against Splunk cloud, and discovered the two lines accelerated_fields and replicate was not there. It was not getting applied. Our CI/CD process was NOT getting the acceleration options applied. I used the Splunk KVstore API instead to get acceleration options to apply, and it worked. The lookup is now accelerated as expected. The lookup time was cut from 165 seconds to just 20 seconds.

ClubMed · ‎03-12-2024

Thanks for the reply! My team ended up having an OnDemandService request to look into this. Will report back. To answer your question by question. The irony is that it was originally CSV that we have since converted to kvStore for performance. The CSV file had became big, like 250 MB if I recall. This is the Qualys Knowledge Base we are talking about that Qualys provide out of box so it is not something we can trim down to size as CSV. The row count after the first stats and before lookup is 1,926,000. The fact stats calculates this in 20 seconds is perfectly fine. The problem becomes when lookup is used, it puts on an extra 140 seconds or so according to the Job Inspector. The dc(HOST_ID) ultimately ends with 7,800 rows. Now for your suggested approach - good catch for the stats. That works great for this particular query that only cares about PATCHABLE. In fact, the last stats can be changed from dc() to just count. Much faster at 50 seconds now! At some point however, we will need to provide full vulnerability data pulled from Qualys Knowledge Base through Splunk as the means of reporting for the engineering teams. We will run into this problem of the lookup hanging up for at least 140 seconds again. Regarding poor performance, when you say 'replicate' - is this what you mean for collections.conf? Because the kvstore is already replicated. [qualys_kb_kvstore] accelerated_fields.QID_accel = {"QID": 1} replicate = true I think ultimately, we were under the impression the kvStore replication to the indexers makes it so there's a local copy handy for them, making a lookup really fast in matter of seconds. Maybe we had the wrong expectation?

ClubMed · ‎03-12-2024

This is an odd acceleration behavior that has us stumped... If some of you worked with Qualys Technology Add-on before, Qualys dump their knowledge base into a CSV file which we converted to kvStore with the following collections.conf accelerations enabled - The knowledge base has approx. 137,000 rows of about 20 columns. [qualys_kb_kvstore] accelerated_fields.QID_accel = {"QID": 1} replicate = true Then if you were to run the following query with lookup local= true and local=false (default). According to Job Inspector there was no real difference between lookup on search head vs. the indexers. Without the lookup command, the query takes 3 seconds to complete over 17 million events. With lookup added, it takes an extra 165 seconds for some reason with the accelerators turned on. index=<removed> (sourcetype="qualys:hostDetection" OR sourcetype="qualys_vm_detection") "HOSTVULN" | fields _time HOST_ID QID | stats count by HOST_ID, QID | lookup qualys_kb_kvstore QID AS QID OUTPUTNEW PATCHABLE | where PATCHABLE="YES" | stats dc(HOST_ID) ```Number of patchable hosts!``` An idea I am going to try is to add PATCHABLE as another accelerated field and see if that changes. This change will require me to wait until tomorrow. accelerated_fields.QID_accel = {"QID": 1, "PATCHABLE": 1} Is there something we're missing to help avoid the lookup taking extra 2-3 minutes?

ClubMed · ‎02-28-2024

5 years later and I came upon this. This should be the actual, concise answer for OP's question.

ClubMed · ‎02-28-2024

Hi, on here, I previously had success working with CSS Selectors for Splunk Dashboards with the help of people here. My previous question solved. So please understand I understand CSS Selectors and I've been bashing myself at this for hours. What I have is a standard bar chart of 2 series over time. I am trying to use CSS selector to move the first series' bar position such that it overlaps with the second bar next to it, to give it an appearance the smaller bar is a sub-set of the larger bar. I have attached a photo using Google Inspect on the bar in Splunk Dashboard. You can see the bar for the first series has the class g.highcharts-series.highcharts-series-0 On the right side, you can see I injected a CSS selector into the webpage and no combination of positioning seem to make the series budge, if at all. As of note, I did find this paragraph on Highcharts website - https://www.highcharts.com/docs/chart-design-and-style/style-by-css#what-can-be-styled However, layout and positioning of elements like the title or legend cannot be controlled by CSS. This is a limitation of CSS for SVG, that does not (yet - SVG 2 Geometric Style Properties) allow geometric attributes like x, y, width or height. And even if those were settable, we would still need to compute the layout flow in JavaScript. Instead, positioning is subject to Highcharts JavaScript options like align, verticalAlign etc. Okay so, the bars probably cannot be moved. That is a very unfortunate limitation. Is it possible to make 2 bars overlap each other on a Splunk dashboard at all? I know I can workaround with using math to subtract a series from another and stack the bars, but it is only a workaround.

ClubMed · ‎01-05-2024

Making this the solution as I have provided step-by-step instructions, but all credit go to @PickleRick for the suggestion. Based on your response, I found a relevant helpful post at https://community.splunk.com/t5/Dashboards-Visualizations/How-do-I-update-panel-color-in-Splunk-using-CSS/td-p/364590 I had to use the Browser Inspector to identify the specific elements. I inspected the data label itself (not the line/bar whatever you're looking at) which in my case, revealed the class called 'highcharts-label highcharts-data-label highcharts-data-label-color-undefined'. However, there was not a way to uniquely select these element by itself, so I had to refer to its parent element, which had a class of 'highcharts-data-labels highcharts-series-0 highcharts-line-series', where the numeral character 0 is the series identifier (0,1,2,3 and so on...) Perfect! The series number I wanted to hide is 0. This also helped me to specify the specific element: https://www.w3schools.com/cssref/css_selectors.php With CSS selector, the selector is then (keep in mind the spaces in class name are actually separators and there were 3 seperate classes in that element. I am only selecting the first 2 classes and chaining them: .highcharts-data-labels.highcharts-series-0 The following block is inserted into panel and works like a charm, hiding only the specified data labels while preserving the other data labels, resulting in a cleaner look: <row> <panel> <title>Blah Blah Blah</title> <html depends="$alwaysHideCSSStyle$"> <style> .highcharts-data-labels.highcharts-series-0 { display:none; } </style> </html> <chart> ... A caveat is that display:none treats this element as air. The chart might auto-adjust itself to fill this space and may impact your desired visual layout. An alternative is to use visibility:hidden, which allows the element to take space on the chart, but be hidden. Thank you!

ClubMed · ‎01-04-2024

FYI as of 2024: This command would reach limit specified in limits.conf. As default, it would return 10,000 events, even if there's more than that. Instead, use: | sort 0 -_time This would return the full result, although can impact performance.

ClubMed · ‎01-04-2024

Thank you. For Splunk, would you say it is currently impossible to be able to show/hide the individual fields?* No alternative workarounds? (*To clarify, mimicking the behavior of the showDataLabels setting for the individual fields.)

ClubMed · ‎01-04-2024

Per documentation: https://docs.splunk.com/Documentation/Splunk/9.1.2/Viz/ChartConfigurationReference The property charting.chart.showDataLabels only allow the type (all | minmax | none). I am attempting to hide data labels for a specific field, but enable data labels for other specified fields. I am attempting to do something similar to charting.fieldColors which uses maps, but the types are obviously not accepted for the showDataLabels property: <option name="charting.chart.showDataLabels"> {"field1":none, "field2":all} </option> Is there a workaround possible for this objective?

Posts	11
Solutions	2
Karma Given	7
Karma Received	3
Member Since	‎01-04-2024

Online Status	Offline
Date Last Visited	3 weeks ago

How to break out a column and append it to the bot...

KVStore acceleration not working as expected?

[XML/CSS Selector] Why can't a Highcharts Series p...

[XML] How to accomplish showDataLabels property as...

Re: How to break out a column and append it to the...

How to break out a column and append it to the bot...

Re: KVStore acceleration not working as expected?

Re: KVStore acceleration not working as expected?

KVStore acceleration not working as expected?

Re: How to get sum of all columns into a new colum...

[XML/CSS Selector] Why can't a Highcharts Series p...

Re: [XML] How to accomplish showDataLabels propert...

Re: Sort column in a report based on _time in desc...

Re: [XML] How to accomplish showDataLabels propert...

[XML] How to accomplish showDataLabels property as...