Parsing of events using regex

[mukhan1]

Hello community,

Below is my sample log file I want to extract each individual piece of event(starting from @ID to REMARK) from the log file. I tried to achieve this by using following regex: (^@ID[\s\S]*?REMARK.*$)
This regex is taking the whole log file as single event. Attaching the snapshot below. 
Also tried to alter the props.conf by using the same regex:

props.conf
[t24]
SHOULD_LINEMERGE=False
LINE_BREAKER=(^@ID[\s\S]*?REMARK.*$)
NO_BINARY_CHECK=true
disabled=false
INDEXED_EXTRACTIONS = csv

LIST F.PROTOCOL @ID PROTOCOL.ID PROCESS.DATE TIME.MSECS K.USER APPLICATION LEVEL.FUNCTION ID REMARK                                                                         PAGE    1 11:34:02  23 NOV 2023

@ID............ 202309260081340532.21                                                                                                                                                                           
@ID............ 202309260081340532.21                                                                                                                                                                           
PROTOCOL.ID.... 202309260081340532.21                                                                                                                                                                           
PROCESS.DATE... 20230926                                                                                                                                                                                        
TIME.MSECS..... 11:15:32:934                                                                                                                                                                                    
K.USER......... INPUTTER                                                                                                                                                                                        
APPLICATION.... AC.INWARD.ENTRY                                                                                                                                                                                 
LEVEL.FUNCTION. 1                                                                                                                                                                                               
ID.............                                                                                                                                                                                                 
REMARK......... ENQUIRY - AC.INTERFACE.REPORT                                                                                                                                                                   

@ID............ 202309260081340523.16                                                                                                                                                                           
@ID............ 202309260081340523.16                                                                                                                                                                           
PROTOCOL.ID.... 202309260081340523.16                                                                                                                                                                           
PROCESS.DATE... 20230926                                                                                                                                                                                        
TIME.MSECS..... 11:15:23:649                                                                                                                                                                                    
K.USER......... INPUTTER                                                                                                                                                                                        
APPLICATION.... AC.INWARD.ENTRY                                                                                                                                                                                 
LEVEL.FUNCTION. 1                                                                                                                                                                                               
ID.............                                                                                                                                                                                                 
REMARK......... ENQUIRY - AC.INTERFACE.REPORT 

Attaching the screenshot of the data which I'm getting on Splunk by using the regex mentioned above.

Also attaching the snapshot of regex result which i have checked earlier online.

I want my data to be shown in table form following is the example snapshot of how I want my data to be appear on Splunk.

[mukhan1]

Hello @ITWhisperer,
Thanks for the response the provided solution is working however if I edit this file and add more data will this search work same as it is working now? This file keeps getting updated after some time, 

[mukhan1]

Hello @ITWhisperer,
 
Thanks for your response. I was trying to add the search you have provided but failed to get desired value, can you please elaborate further how to use this solution you provided. 

[ITWhisperer]

Try it one line at a time and see where it fails

[mukhan1]

It is not failing since we haven’t define any sourcetype or source in search, we hardcoded the payload, however it is working properly giving the exact same headings required. The problem is how it will extract this information from given sourcetype or source.

[ITWhisperer]

| makeresults
| eval _raw="LIST F.PROTOCOL @ID PROTOCOL.ID PROCESS.DATE TIME.MSECS K.USER APPLICATION LEVEL.FUNCTION ID REMARK                                                                         PAGE    1 11:34:02  23 NOV 2023

@ID............ 202309260081340532.2
@ID............ 202309260081340532.21
PROTOCOL.ID.... 202309260081340532.21
PROCESS.DATE... 20230926
TIME.MSECS..... 11:15:32:934
K.USER......... INPUTTER
APPLICATION.... AC.INWARD.ENTRY
LEVEL.FUNCTION. 1
ID............. 
REMARK......... ENQUIRY - AC.INTERFACE.REPORT

@ID............ 202309260081340523.16
@ID............ 202309260081340523.16
PROTOCOL.ID.... 202309260081340523.16
PROCESS.DATE... 20230926
TIME.MSECS..... 11:15:23:649
K.USER......... INPUTTER
APPLICATION.... AC.INWARD.ENTRY
LEVEL.FUNCTION. 1
ID............. 
REMARK......... ENQUIRY - AC.INTERFACE.REPORT"
``` The lines above set up sample data in line with your example ```
| rex max_match=0 "(?ms)(?<event>^\@ID.*?REMARK.*?$)"
| mvexpand event
| rex max_match=0 field=event "(?m)(?<namevalue>.+\.+\s.*$)"
| streamstats count as row
| mvexpand namevalue
| rex field=namevalue "(?<name>[^\s]+(?<!\.))\.*?\s(?<value>.*$)"
| eval {name}=value
| fields - name value namevalue event
| stats values(*) as * by row
| fields - row