Alert for filename with previous date

[vm_molson]

I would like to write an alert that runs Tuesday through Saturday, and looks for files that have been dropped off on our FTP server with the previous day in the filename and fires when four of those files have been found.

Example, on 1/6/2021, I want my alert to tell me when 4 files have been delivered matching a filename of xxx_20210105_yyy (where xxx_ and _yyy can be anything). Those files are typically delivered in the morning hours, but possibly have come in as early as 11:30 PM the night before.

What is the best way to do this?

[scelikok]

Hi @vm_molson,

In fact it should have been work with only underscores but you can try below two options;

Only blahblahblah_DATE_blahblahblah OR blahblahblah.DATE.blahblahblah

| eval yesterday=now()-86400 
| eval yesterday=strftime(yesterday,"%Y%m%d")
| where match(filename, "[_\.]".'yesterday'."[_\.]")

 Any separator before and after DATE

| eval yesterday=now()-86400 
| eval yesterday=strftime(yesterday,"%Y%m%d")
| where match(filename, "\w+.".'yesterday'.".\w+")

If this reply helps you an upvote is appreciated.

[vm_molson]

Appreciate you sticking with me, @scelikok ! I actually am still not able to make it work. Let me specify the exact file names I should be getting... maybe I missed something in my explanation? Maybe it is that there is no character directly after the date on three of them? I had assumed a space would count as any character, but I'm not as up on my RegEx as I should be.

The log entries look like:

...created /Companyname_POS_20210126 - ...

...created /Companyname_TRN_20210126 - ...

...created /Companyname_TRD_20210126 - ...

...created /COMPANYIDTAXLOT.20210126.230053 - ...

[scelikok]

Hi @vm_molson,

For sure the problem is because of space. In regex, all options should be defined specifically in order to lower false positive matches. Based on your sample events below should work;

| eval yesterday=now()-86400 
| eval yesterday=strftime(yesterday,"%Y%m%d")
| where match(filename, "[_\.]".'yesterday'."[_\.\s]")

Or if you sure there is no other date in the file you can use below;

| eval yesterday=now()-86400 
| eval yesterday=strftime(yesterday,"%Y%m%d")
| where match(filename, ".+".'yesterday'.".+")

[scelikok]

Hi @vm_molson,

You can use below query to filter events that matches your criteria, schedule the alert like every 5 minutes and set alert condition as result count more than 3.

| your search that has "filename" field contains ftp filenames
| eval yesterday=now()-86400 
| eval yesterday=strftime(yesterday,"%Y%m%d")
| where LIKE (filename, "%_".'yesterday'."_%")

If this reply helps you an upvote is appreciated.

[vm_molson]

Hey @scelikok

 ... thank you for the tip! It seems to work pretty well, but 3 out of the 4 filenames are like:

blahblahblah_DATE_blahblahblah

.. and only one of the filenames is like

blahblahblah.DATE.blahblahblah

The exact syntax you suggested works to get the file with the periods in the filename, but not with the underscores. I've tried various permutations and I know I am missing something.. but any thoughts about how I might get the others as well?

[vm_molson]

Thank you! Will try this out