I'm wondering if the following table structure is possible (without custom JS).
Raw events are from Jenkins plugin. Below one event example. I get one per job/test/build.
{ [-]
build_number: 615
build_url: job1
event_tag: build_report
job_name: jobname
job_result: FAILURE
metadata: { [+]
}
page_num: 1
testsuite: { [-]
duration: 1968
errors: 0
failures: 6
passes: 7
skips: 0
testcase: [ [-]
{ [-]
classname: Testsuites.MyCallCenter
duration: 122
failedsince: 0
groupname: Testsuites
skipped: false
status: PASSED
testname: Login
uniquename: Testsuites.MyCallCenter.Login
}
{ [-]
classname: Testsuites.MyCallCenter
duration: 148
failedsince: 0
groupname: Testsuites
skipped: false
status: PASSED
testname: Edit Calendar
uniquename: Testsuites.MyCallCenter.Edit Calendar
}
{ [+]
}
]
tests: 13
time: 1968
total: 13
}
user: (timer)
}
Can I create a table, which has a dynamic build number in the header somehow like this?
classname | testname | build-613 | build-614 | build-615
Testsuites.MyCallCenter | Login | PASSED | PASSED | PASSED
Testsuites.MyCallCenter | Edit Calendar | PASSED | FAILED | PASSED
| rename COMMENT as "the logic"
| streamstats count as session
| spath build_number
| spath testsuite.testcase{} output=testcase
| stats values(eval("build_number_".build_number)) as build_number by session testcase
| spath input=testcase
| table build_number classname testname status
| eval tmp=classname."_".testname
| xyseries tmp build_number status
| eval classname=mvindex(split(tmp,"_"),0), testname=mvindex(split(tmp,"_"),1)
| fields - tmp
| table classname testname *
Have you tried my logic? the result is wrong. "build_number_" is missing.
If it is changed in a strange way, it cannot be fixed.
One thing: If I have build numbers like 4x, 5x, 3xx, the xyseries seems to sort them as string, not as a number.
Now I get: 3xx, 4x, 5x
I would like to have: 4x, 5x, 3xx, 4xx, 5xx
sample:
| makeresults count=20
| eval name=random() % 200
| eval fieldA="TEST", fieldB="TEST2"
| xyseries fieldA name fieldB
| transpose 0 header_field=fieldA
| sort 0 column
| transpose 0 header_field=column
try transpose
index=_internal |head 1 | fields _raw |eval _raw="{\"build_number\":\"615\",\"build_url\":\"job1\",\"event_tag\":\"build_report\",\"job_name\":\"jobname\",\"job_result\":\"FAILURE\",\"metadata\":{},\"page_num\":\"1\",\"testsuite\":{\"duration\":\"1968\",\"errors\":\"0\",\"failures\":\"6\",\"passes\":\"7\",\"skips\":\"0\",\"testcase\":[{\"classname\":\"Testsuites.MyCallCenter\",\"duration\":\"122\",\"failedsince\":\"0\",\"groupname\":\"Testsuites\",\"skipped\":\"false\",\"status\":\"PASSED\",\"testname\":\"Login\",\"uniquename\":\"Testsuites.MyCallCenter.Login\"},{\"classname\":\"Testsuites.MyCallCenter\",\"duration\":\"148\",\"failedsince\":\"0\",\"groupname\":\"Testsuites\",\"skipped\":\"false\",\"status\":\"PASSED\",\"testname\":\"Edit Calendar\",\"uniquename\":\"Testsuites.MyCallCenter.Edit Calendar\"},{}],\"tests\":\"13\",\"time\":\"1968\",\"total\":\"13\"},\"user\":\"(timer)\"}"
| rename COMMENT as "the logic"
| spath build_number
| spath testsuite.testcase{} output=testcase
| stats values(eval("build_number_".build_number)) as build_number by testcase
| spath input=testcase
| table build_number classname testname status
| eval tmp=classname."_".testname
| xyseries tmp build_number status
| eval classname=mvindex(split(tmp,"_"),0), testname=mvindex(split(tmp,"_"),1)
| fields - tmp
| table classname testname *
I would like you to present _raw. I have to process it.
Close already. I put the build number only to short the output.
{"event_tag":"build_report","metadata":{"scm":"git"},"job_name":"job1","testsuite":{"failures":0,"passes":2,"skips":0,"total":2,"duration":1968.0,"tests":2,"time":1968.0,"errors":0,"testcase":[{"duration":122.0,"classname":"Testsuites.MyCallCenter","testname":"Login","groupname":"Testsuites","skipped":false,"failedsince":0,"uniquename":"Testsuites.MyCallCenter.myCallCenter basic tests.Login","status":"PASSED"},
{"duration":148.0,"classname":"Testsuites.MyCallCenter","testname":"Edit Calendar","groupname":"Testsuites","skipped":false,"failedsince":0,"uniquename":"Testsuites.MyCallCenter.myCallCenter basic tests.Edit Calendar","status":"PASSED"}]
},"build_number":615,"page_num":1,"job_result":"PASSED","user":"(timer)","build_url":"job/build/615/"}
| rename COMMENT as "the logic"
| streamstats count as session
| spath build_number
| spath testsuite.testcase{} output=testcase
| stats values(eval("build_number_".build_number)) as build_number by session testcase
| spath input=testcase
| table build_number classname testname status
| eval tmp=classname."_".testname
| xyseries tmp build_number status
| eval classname=mvindex(split(tmp,"_"),0), testname=mvindex(split(tmp,"_"),1)
| fields - tmp
| table classname testname *
Have you tried my logic? the result is wrong. "build_number_" is missing.
If it is changed in a strange way, it cannot be fixed.
Yes, the first example dit not work, I only removed the prefix "build_number_" to make the output shorter.
But your second example looks better with the split by 'session'. Now we get one row per build there. I'll look into and get back to you.
I think I have some testcases larger than 5000 characters and spath only parses the first 5K?
I have 6K of chars, in case of error happens.
BTW, does it help, if I already have index time extractions defined for sourcetype?
[json:jenkins]
pulldown_type = true
INDEXED_EXTRACTIONS = json
KV_MODE = none
TRUNCATE = 0
category = Structured
LEARN_MODEL = false
description = JavaScript Object Notation format.
So my events have already to following fields:
job_name
job_result
level
linecount
log_source
log_thrown
message
metadata.scm
page_num
testsuite.duration
testsuite.errors
testsuite.failures
testsuite.passes
testsuite.skips
testsuite.testcase{}.classname
testsuite.testcase{}.duration
testsuite.testcase{}.errordetails
testsuite.testcase{}.errorstacktrace
testsuite.testcase{}.failedsince
testsuite.testcase{}.groupname
testsuite.testcase{}.skipped
testsuite.testcase{}.status
testsuite.testcase{}.testname
testsuite.testcase{}.uniquename
testsuite.tests
testsuite.time
testsuite.total
...
| streamstats count as session
| stats list(testsuite.testcase{}.*) as * by session build_number
| eval tmp=mvzip(classname,mvzip(testname,status))
| stats values(build_number) as build_number by session tmp
| eval status=mvindex(split(tmp,","),-1)
| xyseries tmp build_number status
| rex mode=sed field=tmp "s/,\w+$//g"
| rename COMMENT as "check this result"
| selfjoin tmp
| eval classname=mvindex(split(tmp,","),0), testname=mvindex(split(tmp,","),1)
| fields - tmp
| table *name *
It's very hard to create a log when there's only one log on display, isn't it?
This is what I have now and it is looking good. I'm still playing and verifying it.
...
| streamstats count as session
| spath output=test "testsuite.testcase{}"
| fields session test host job_name build_number
| mvexpand test
| eval result=if(isnotnull(errordetails), "Failed", if(skipped=="true", "Skipped", "Passed"))
| eval duration=round(duration, 2)
| eval result=result
| stats values(build_number) as build_number by session test
| spath input=test output=status "status"
| spath input=test output=name "name"
| spath input=test output=testname "testname"
| eval testname=coalesce(name, testname)
| spath input=test output=classname "classname"
| spath input=test output=errordetails "errordetails"
| spath input=test output=skipped "skipped"
| eval result=if(isnotnull(errordetails), "Failed", if(skipped=="true", "Skipped", "Passed"))
| table build_number classname testname result
| eval tmp=classname."_".testname
| xyseries tmp build_number result
| eval classname=mvindex(split(tmp,"_"),0), testname=mvindex(split(tmp,"_"),1)
| fields - tmp
| table classname testname *
And your latest example look good as well :-).
I will accept your answer after I have verified a bit.
Thanks A LOT!