Introduction

The Regexp Field Processor uses Regular Expressions (commonly known as "regexps") to extract values from email content. Issue fields are mapped to these values. In addition, the field processor can be used to associate emails to an existing issue by finding a common regexp-extracted field value.

It is primarily designed for processing emails that have predictable content. A common source for such emails is an automated system such as Bugzilla.

It is recommended to have at least a basic understanding of how Regular Expressions work before attempting to configure this field processor. Useful references can be found from Wikipedia, and this online tutorial.

Configuration

In the target JEMH profile, ensure that Directive processing is enabled by going to Profile>Directives.  Then, enable the Regexp Field Processor under Profile>Field Processors:

Open

Once enabled, you should see the Regexp Field Processor section listed at the bottom of the main profile screen.  When the Regexp Field Processor is enabled, an example configuration is added:

Open

Editing Configurations

Editing the processor configuration is done by clicking on the pencil icon .  Once in edit mode, the Regexp edit screen shows more details about each Match Configuration.

From JEMH 2.2.6 (JIRA 7.4+) the configuration is displayed as 3 tabs.

Config

This tab contains a range of fields that define when the particular configuration should be used.

Issue Association

The Regexp Field Processor can be used to associate an email with a JIRA issue:

• On issue creation a uniquely identifiable value is extracted from the creating email through the use of a configured regular expression/script

• This value is set on a configured custom field.

• The next time an email is processed by the processor it will extract a value using the same steps

• If an existing issue is found to already have this value in the custom field, and is not in a status defined under Resolved Statuses, an issue association will be made

• Ensure that the custom field set has the Search Template set to Free Text Searcher

Creating a simple Value via Regexp

When the Value Type is Regexp, a text field is visible, enabling simple expressions with capture group support (only first capture group is usable).

Combined with Search In Subject, the following expression matches on the numeric part in the subject with final value 12345678. (the screenshot above shows the Expected Match Value for the compound key example)

Creating a compound value via Value Script

The standard Regexp value type is fine to use in most situations, but what if the value you want requires extracting information from more than one place in an email?  This is where the Script value type can help.  Through the use of scripting, you can extract and combine as many values as you want to create a "compound value".

Script Type options Since 3.3.46

Installed Script Engines can now be selected when creating a Value Script, in additional to legacy Velocity scripting.

To install Script Engines see:

• https://thepluginpeople.atlassian.net/wiki/spaces/JEMH/pages/241106948/Install+a+Scripting+runtime+Engine+like+Groovy

Example Velocity Script

In the above script, lines starting with ## are Velocity comments that are non-functional.  Removing the the hash symbols so that the Preview feature () can be used to send the script to a REST service to execute over the currently persisted Test Case content and return the rendered output (including comments!).  This output is compared to the Expected Value Match so you know the expressions are right - IF  the inbound content is structured the same AND your HTML processing configuration within the Profile remain constant.

The test case above is the default example that is created automatically along with the first regexp configuration.  We make use of two values within the content:

So the capture group (the first within the expression) is extracted and assigned:

The final part of the puzzle is on the last line of the script which actually renders the values into the template:

When the final value is as expected, the comment can be hidden again (prefix with double #).  Remember that NEWLINES matter in velocity so don't leave any in the template.  JEMH will strip them out later anyway as the final compound Value must be a single line ideally contiguous text.

Script value preview

Example ECMAScript Since 3.3.46

Using the Script Type field an installed Engine using ECMAScript e.g. Oracle Nashorn can be used to create a script which may be preferable to using Velocity.

Using the same Email Test case as the previous example, a similar Script can be made that creates a compound value from multiple sources in the email.

In the script above, the ‘//’ characters mark the line as a comment which will be ignored and are non functional, the ‘print();’ method calls however will output information to the console log, which is visible when using the Preview feature() , and in the JEMH Processing report, helping to debug scripts.

In this script the two values making up the final value are extracted using the following Regexp

These first capture group (the value inside the brackets) is then added to the final value

Finally, the extracted value needs to be set as the ‘valueMatch’ to identify the extracted value as the outcome of the Script.

Script console and preview

Using the preview button will output the console log of the script and show the value of ‘valueMatch’.

Test Case

This isn't a JEMH Test case.  Its a combination of (a) Headers and (b) post-processed email content (i.e. not HTML).

Adding a Test Case

Test Cases in the Regexp Field Processor are not to be confused with JEMH Test Cases. Here, they are a combination of two aspects:

1. email headers

2. decoded text body content

The configuration that is created when the processor is first enabled contains an example (see below).  You can use this as a basis for your own specific testing, updating the To: address to match your Profile's catchEmail address, and the From: address to match your remote system sender address.

It is important to make sure you adding just the TEXT (separated from the headers with an empty line).  Typically, you can obtain this TEXT either from a JIRA issue description of a created issue (when the field Processors are disabled) or from the text/plain part of a real email.

Field Mappings

Editing a Field Mapping is done from the edit screen, by selecting the pen icon for the rule concerned. New mappings can be added via the New Field Mapping link.

Once edited, the following screen is shown, allowing the custom field to be nominated (must exist) and for the regexp to be supplied (must be valid syntax) as well as the Expected Value (must be found). NOTE the Test Case 'content' is shown at the bottom of the screen for reference when building the field mapping regexp.

If the validation does not succeed, the following error will be shown, indicating the regexp used, expected value and actual value, in this case the edit fails and needs to be redone.

Multi-value support

It is possible to use a Regexp pattern that matches more than one value.  If the Regexp configuration has Match Many enabled, all matching values will be used.  This Regexp is currently supported at FIELD level, and cannot be used as the issue selector (even if in the body).

Testing

With an issue already received and processed through the Basic Field Processor, it can be converted into a TestCase through the Auditing screen:

Ensure that the desired target profile is selected for the test case, and save.  Once saved, the TestCase can be processed by hovering the cursor over the test case and clicking the Run:

Assuming the Reporter of the issue is defined in either the Regexp Field Processor or the Global Default Reporter has appropriate permissions in the target project, an issue will be created:

Issue creation

Reviewing the created issue, as shown below, this shows that:

1. The subject field was extracted and stored in the bugzillaId custom field.

2. That custom field 1 and custom field 2 both have values, that match the expected values set earlier (because test case payload data was used).

3. That custom field 3 does not contain a value - if the field mapping expression does not extract a value, the field will not get a value set by this field processor configuration.

Association with existing issues

When repeat emails come through with the same Custom Field Name value, a search is done for issues with that matching value, if found, as in the case just run, it is used to comment on with the payload of the incoming message. Re-running the test case (refresh the results page) to see if the issue gets commented on:

Viewing the issue now shows that the content has been added as a comment:

Setting comment/description to extracted value only

You are able to set the comment/description fields to a value that you have extract from the email body this way you can keep only the relevant information within the issue description/comment. This is done by having a Field Mapping that has a Field Name of “comment”.

This is an example Regex Field Mapping

Here is an example Test Case.

Viewing the issue page shows that the description is set to the extracted value

Related articles