please help extract adsId,offerName, currentProductDescription, offerAccountToken, offerType, offerIdentifier
message={"name":"com. ","level":"info","message":"Create -->|Request identifier : 09accf30-6cf7-4e4f-a633-c19808eff766|CreateAccountOfferEnrollment.v1|REQUEST ---> {\"correlationId\":\"09accf30-6cf7-4e4f-a633-c19808eff766\",\"ccpId\":\"HA6952B\",\"callId\":\"0109\",\"adsId\":\"camar\",\"customerId\":\"63038\",\"eventType\":\"CVP-INSTANT\",\"channelIdentifier\":\"CVP\",\"lineOfBusiness\":\"CCSG\",\"offerName\":\"Additional\",\"offerIdentifier\":\"A000\",\"sourceProductCode\":\"2X\",\"currentProductIdentifier\":\"2X\",\"currentProductDescription\":\"Pl\",\"destinationProductCode\":\"2X\",\"destinationProductName\":\"Plat\",\"fulfillmentCode\":\"GNAS\",\"requestHasSupps\":true,\"offerType\":\"consumer-stand-alone-supp\",\"offerAccountToken\":\"YAS\",\"marketName\":\"US\",\"numberOfSupps\":1,\"calledInAccountToken\":\"YAS\",\"fullName\":{\"firstName\":\"M\",\"lastName\":\"C\",\"middleName\":\"A\",\"prefix\":\"\",\"suffix\":\"\"},\"communicationInformation\":{\"channel\":\"EMAIL\",\"communicationVariables\":[],\"locale\":\"en_US\",\"physicalAddress\":{\"city\":\"P\",\"state\":\"FL\",\"zipCode\":\"33\",\"lines\":[\"48 \",\"#0114\",\"\"]},\"emailAddress\":\"cru@gmail.com\",\"isoCountryCode\":\"840\"},\"enrollmentInformation\":{\"id\":\"2023\",\"is_customer_offline\":false,\"channel_received_datetime\":\"20230109T171713.842 GMT\",\"dynamic_journey\":\"DYNAMIC_INSTANT\",\"rep_id\":\"HA6952B\",\"country_code\":\"840\",\"journey\":\"INSTANT_DECISION\",\"journey_stage\":\"SUPP",\"applicants\":[{\"number\":0,\"amex_relationship\":{\"relationships\":[{\"number\":\"3726\",\"type\""CARD\"}]},\"type\""PRIMARY\"},{\"number\":1,\"type\""NONPRIMARY\",\"has_spending_limit\":false,\"is_signature_available\":true,\"has_cash_restriction\":false,\"experience_id\""829e34d6-e89f-422b-b355-811b1aa2c79c\",\"names\":[{\"language\""EN\",\"name\":{\"first\""V\",\"last\""C\"}}],\"identifiers\":[{\"system\""DELIVERY_METHOD_IDENTIFIER\",\"id\""510DELVIDP256Cn+ has_same_address_as_primary\":false,\"emboss_name\""V\",\"language\""EN\",\"birth_date\""19\",\"spending_limit\":0,\"experience_choices\":[{\"selected_id\""USA_CONSUMER \",\"feature_name\""CARD_DESIGN\"}],\"product\":{\"offer_arrangement_id\""de7c960c46c7\",\"source_code\""A0000FYC4T\",\"short_product_id\""L81\",\"sub_product_code\""2X\"},\"addresses\":[{\"type\""HOME\",\"address\":{\"line1\""7B\",\"city\""HOUSTON\",\"region\""TX\",\"postal_code\""77028-4570\",\"country\""840\"}},{\"type\""TEMPORARY_ADDRESS\",\"address\":{\"line1\""790\",\"city\""HO\",\"region\""T\",\"postal_code\""77\",\"country\""840\"}}]}]},\"misProcessId\""3016428984\"}"}
@ITWhisperer @VatsalJagani please help
As always, I strongly recommend not to use regex to handle structured data as simple strings because there is no guarantee that the string orders will not change. As much as possible, use SPL's built-in capabilities. You data looks like compliant JSON with another compliant JSON enclosed inside message.message. spath is your friend.
Assuming that for some reason, your raw data did not extract message.message, you can do the following:
| rename message AS data
| spath input=data
| eval REQUEST = replace(message, ".*REQUEST ---> ", "")
| spath input=REQUEST
| fields - REQUEST data message
| table *
Now you have all you need. Your sample data gives. (Another point: When you illustrate complex data, use code box. Lots of JSON markups got transformed into smileys which I had to convert back.)
adsId | callId | calledInAccountToken | ccpId | channelIdentifier | communicationInformation.channel | communicationInformation.emailAddress | communicationInformation.isoCountryCode | communicationInformation.locale | communicationInformation.physicalAddress.city | communicationInformation.physicalAddress.lines{} | communicationInformation.physicalAddress.state | communicationInformation.physicalAddress.zipCode | correlationId | currentProductDescription | currentProductIdentifier | customerId | destinationProductCode | destinationProductName | enrollmentInformation.applicants{}.amex_relationship.relationships{}.number | enrollmentInformation.applicants{}.amex_relationship.relationships{}.type | enrollmentInformation.applicants{}.experience_id | enrollmentInformation.applicants{}.has_cash_restriction | enrollmentInformation.applicants{}.has_spending_limit | enrollmentInformation.applicants{}.identifiers{}.id | enrollmentInformation.applicants{}.identifiers{}.system | enrollmentInformation.applicants{}.is_signature_available | enrollmentInformation.applicants{}.names{}.language | enrollmentInformation.applicants{}.names{}.name.first | enrollmentInformation.applicants{}.names{}.name.last | enrollmentInformation.applicants{}.number | enrollmentInformation.applicants{}.type | enrollmentInformation.channel_received_datetime | enrollmentInformation.country_code | enrollmentInformation.dynamic_journey | enrollmentInformation.id | enrollmentInformation.is_customer_offline | enrollmentInformation.journey | enrollmentInformation.journey_stage | enrollmentInformation.rep_id | eventType | fulfillmentCode | fullName.firstName | fullName.lastName | fullName.middleName | fullName.prefix | fullName.suffix | level | lineOfBusiness | marketName | name | numberOfSupps | offerAccountToken | offerIdentifier | requestHasSupps | offerType | requestHasSupps | sourceProductCode |
camar | 0109 | YAS | HA6952B | CVP | cru@gmail.com | 840 | en_US | P | 48 #0114 | FL | 33 | 09accf30-6cf7-4e4f-a633-c19808eff766 | Pl | 2X | 63038 | 2X | Plat | 3726 | CARD | 829e34d6-e89f-422b-b355-811b1aa2c79c | false | false | 510DELVIDP256Cn+ has_same_address_as_primary | DELIVERY_METHOD_IDENTIFIER | true | EN | V | C | 0 1 | PRIMARY NONPRIMARY | 20230109T171713.842 GMT | 840 | DYNAMIC_INSTANT | 2023 | false | INSTANT_DECISION | SUPP | HA6952B | CVP-INSTANT | GNAS | M | C | A | info | CCSG | US | com. | 1 | YAS | A000 | Additional | consumer-stand-alone-supp | true | 2X |
As always, I strongly recommend not to use regex to handle structured data as simple strings because there is no guarantee that the string orders will not change. As much as possible, use SPL's built-in capabilities. You data looks like compliant JSON with another compliant JSON enclosed inside message.message. spath is your friend.
Assuming that for some reason, your raw data did not extract message.message, you can do the following:
| rename message AS data
| spath input=data
| eval REQUEST = replace(message, ".*REQUEST ---> ", "")
| spath input=REQUEST
| fields - REQUEST data message
| table *
Now you have all you need. Your sample data gives. (Another point: When you illustrate complex data, use code box. Lots of JSON markups got transformed into smileys which I had to convert back.)
adsId | callId | calledInAccountToken | ccpId | channelIdentifier | communicationInformation.channel | communicationInformation.emailAddress | communicationInformation.isoCountryCode | communicationInformation.locale | communicationInformation.physicalAddress.city | communicationInformation.physicalAddress.lines{} | communicationInformation.physicalAddress.state | communicationInformation.physicalAddress.zipCode | correlationId | currentProductDescription | currentProductIdentifier | customerId | destinationProductCode | destinationProductName | enrollmentInformation.applicants{}.amex_relationship.relationships{}.number | enrollmentInformation.applicants{}.amex_relationship.relationships{}.type | enrollmentInformation.applicants{}.experience_id | enrollmentInformation.applicants{}.has_cash_restriction | enrollmentInformation.applicants{}.has_spending_limit | enrollmentInformation.applicants{}.identifiers{}.id | enrollmentInformation.applicants{}.identifiers{}.system | enrollmentInformation.applicants{}.is_signature_available | enrollmentInformation.applicants{}.names{}.language | enrollmentInformation.applicants{}.names{}.name.first | enrollmentInformation.applicants{}.names{}.name.last | enrollmentInformation.applicants{}.number | enrollmentInformation.applicants{}.type | enrollmentInformation.channel_received_datetime | enrollmentInformation.country_code | enrollmentInformation.dynamic_journey | enrollmentInformation.id | enrollmentInformation.is_customer_offline | enrollmentInformation.journey | enrollmentInformation.journey_stage | enrollmentInformation.rep_id | eventType | fulfillmentCode | fullName.firstName | fullName.lastName | fullName.middleName | fullName.prefix | fullName.suffix | level | lineOfBusiness | marketName | name | numberOfSupps | offerAccountToken | offerIdentifier | requestHasSupps | offerType | requestHasSupps | sourceProductCode |
camar | 0109 | YAS | HA6952B | CVP | cru@gmail.com | 840 | en_US | P | 48 #0114 | FL | 33 | 09accf30-6cf7-4e4f-a633-c19808eff766 | Pl | 2X | 63038 | 2X | Plat | 3726 | CARD | 829e34d6-e89f-422b-b355-811b1aa2c79c | false | false | 510DELVIDP256Cn+ has_same_address_as_primary | DELIVERY_METHOD_IDENTIFIER | true | EN | V | C | 0 1 | PRIMARY NONPRIMARY | 20230109T171713.842 GMT | 840 | DYNAMIC_INSTANT | 2023 | false | INSTANT_DECISION | SUPP | HA6952B | CVP-INSTANT | GNAS | M | C | A | info | CCSG | US | com. | 1 | YAS | A000 | Additional | consumer-stand-alone-supp | true | 2X |
What have you tried so far?
Nothing. I am a noob, non-technical PM. I want to learn splunk search, if you can share a beginners link and also please help solve this that would be great. @richgalloway
Generally, the community is more inclined to help those who have put in some effort on their own.
Fields can be extracted using the rex command. I find it easiest, especially when new, to use one rex command for each field to extract.
| rex "adsId\\\\\":\\\\\"(?<adsId>[^\\]+)"
@siksaw33 wrote:@richgalloway This gives me error
Error in 'rex' command: Encountered the following error while compiling the regex 'adsId\\":\\"(?<adsId>[^\]+)': Regex: missing terminating ] for character class.
That error means the escaping is off. The '\\":\\"' part tells me there is one too many backslashes before each quotation mark. Try
| rex "adsId\\\\":\\\\"(?<adsId>[^\\]+)"
do I use the same one for other fields? just replace adsID with offerName etc?
Yes
also how did you build the regex any knowledge resource or link you can share? how do I learn that so I can be helped by the community?
There are many resources on the Internet for learning and experimenting with regular expressions. I like regex101.com for testing expressions. Also, there are a few talks available at conf.splunk.com.
Thank you but I still get the same error with the new solution