Hi MuS
I have been trying to write the comment and reply but I get a message that it will be moderated and then updated, but even after 24 hours its not appearing
Data lets say:
Files Fields ExpectedOutput. OUTPUT FIELDS
File1.csv” Field1, Field2 Field3 Field3 -Perform analysis on field2 wrt filed1 to FIELD1, AnalysisField1
get results as compliant or non compliant
File2.csv Field1 FA FB FC FD FE FG -Combine Fields from File1 and File2 Field1, Analysis Field1, FA FB
FC FD FE FG
Filre3.csv Field1 fa fb fc fd fe fg fh -Analyse based on fa fb fc to Field1, Analysis Field3, fd fe fg
get results as compliant or non compliant fh
-Combine Fields from Fiel1.csv /File2.csv Field 1, FA FB FC FD FE FG fd
and File 3.csv fe fg fh
-Analyse the data to generated a combined JointAnalysisField1UField3,
compliance /non compliance field
File4.csv Field1 , f1 f2 f3 f4 f5 f6 -Analyse File4.csv based on field f1 f2 and get Field 1, Analysis Field4, f3 f4
compliance / non compliant result field f5 f6
-Combine the Analysed fields “JointAnalysisField1UField3”
& “ Analysis Field4” to generate the final analysis result of
COMPLIANCE/NONCOMPLIANCE
Search Query
index= mine source=“File4.csv”
| table Field1 , f1 f2 f3 f4 f5 f6
| eval “ Analysis Field4”=case(like(f1,”Y”), "Compliant", like(f2,”Y”), "NonCompliant")
|join Field 1, Analysis Field4, f3 f4 f5 f6 type=outer
[
search (source=File1.csv or source=File2.csv) index=mine
|fields+Field1, Field2 Field3 Field3 FA FB FC FD FE FG
| eval AnalysisField1= case(like(field2,”Y”), "Compliant", like(field2,”N”), "NonCompliant")
| join Field 1 type=outer
[
search source=File3.csv index=mine
|eval Analysis Field3= case (like(fa,”Y”) AND (like(fb,”Y”) OR like(fb,”N”) ), "Compliant", like(fa,”N”) AND (like(fb,”Y”) OR like(fb,”N”) ),"NonCompliant”)
]
| fileds Field1, Analysis Field1, FA FB FC FD FE FG Analysis Field3, fd fe fg fh
| eval JointAnalysisField1UField3=case (like(“Analysis Field1”,”Compliant”) AND (like(“Analysis Field3”,”Compliant”) OR like(“Analysis Field3”,”NonCompliant”) ),
"Compliant”, like(“Analysis Field1”,”NonCompliant”) AND (like(“Analysis Field3”,”Compliant”) OR like(“Analysis Field3”,”NonCompliant”) ),"NonCompliant”)
]
| fields Field1, JointAnalysisField1UField3, FA FB FC FD FE FG fd fe fg fh Analysis Field4, f3 f4 f5 f6
| eval finalanalysis= if ( match( JointAnalysisField1UField3, “Compliant”) AND match (“Analysis Field1”,”Compliant”), “COMPLIANT”,
if(match( JointAnalysisField1UField3, “NonCompliant”) AND match (“Analysis Field1”,”NonCompliant”), “NONCOMPLIANT”, “UNDEFINED))
010
Note: Eval represent analysis steps , for brevity I have reduced long analysis statements in very simple manner.
What I observed:
1. Join should happen from log file with higher number of logs to log file with less number of logs.
My Question can I improve the search query to make better dashboards.
Also this is just a POC, I am expected to increase the number of parameters in the report by including more logs from various other sources.
Can I continue to use this approach
... View more