How to replace the part of value using mutate and gsub in logstash

Question

I've got in my log file something like this:

DATATYPE::SERVICEPERFDATA       TIMET::1519222690       HOSTNAME::localhost     SERVICEDESC::Total Processes    SERVICEPERFDATA::procs=59;250;400;0;    SERVICECHECKCOMMAND::check_local_procs!250!400!RSZDT  HOSTSTATE::UP    HOSTSTATETYPE::HARD     SERVICESTATE::OK        SERVICESTATETYPE::HARD

I want to replace SERVICEPERFDATA::procs=59;250;400;0; this key value pair in my message to something like this.

SERVICEPERFDATA::59

so I can use kv filter to split the data into key and value.

I've tried with Logstash mutate and gsub but couldn't find the right regex to achieve my goal.

Thanks, Charan

Solution 1

You can use a capturing group to grab a part of a regex and use it in the replace part of the mutate/gsub configuration.

mutate {
  gsub => ["message","(?<=SERVICEPERFDATA::)procs=(\d+);\S+", "\1"]
}

The (?<=SERVICEPERFDATA::) make it so the regex only apply to SERVICEPERFDATA::. The procs=(\d+);\S+ regex will put all the numbers between procs= and the next ; in a group, which is then used in the replace part of the configuration ("\1").

See a regex explanation.

Result of the filter: SERVICEPERFDATA::59

Another option would be to use two mutate/gsub filters, which would have each a simpler configuration.

Solution 2

Regex: procs=([^;]+)\S+ or (?<=SERVICEPERFDATA::)procs=([^;]+)\S+ Substutution: \1

Details:

• () Capturing group
• [^] Match a single character not present in the list
• + Matches between one and unlimited times
• \S Matches any non-whitespace character (equal to [^\r\n\t\f\v ])
• \1 Group 1.

Code:

mutate {
    gsub => [
      "fieldname", "procs=([^;]+)\S+", "\1",
    ]
}

Comments

[ctwheels]

You can drop the capture groups for the first and third elements as such: regex101.com/r/1hx9dd/2. Also, if you only want to do this for SERVICEPERFDATA:: you can prepend (?<=SERVICEPERFDATA::) to the regex like this link shows.

[baudsp]

@ctwheels Thank you for your remarks. I've updated my answer with what you've proposed.