|
I need to understand how adding fields to raw data will increase our index size growth. We are in the process of adding many fields to individual records. This could run to as much as hundreds per record / line. Will each record contain a field label and a repeat of the data from the full raw record? This could cause the increase in our index size to be very substantial in extracting fields. |
|
Generally, the vast majority of field extractions are performed at search time, so configuring lots of fields per event will not increase your index size by any amount. It is possible to create index-time extractions, so that extra fields are written to the index but that has to be specifically configured and is only recommended in isolated cases where it's absolutely necessary for search-speed to be as fast as possible. The IFX feature, rex command and regular 'EXTRACT' settings in props.conf all work at search-time. Unless you have purposely configured all of your fields to be created at index-time, they will not increase your index size. |