Selectively process office365 filtered mail

Since JEMH 3.0.5

Scenario

Office365 can intercept some mail, still delivering it to the destination but with the source message/rfc822 content fully encapsulated in another email with some indicative wording about 'spam' or similar. The desired outcome is to get the sub Message contained within, get the body, and apply some smarts, to determine whether or not JEMH should silently DROP the message or allow it be processed (actually, by extracting the body content).

Dealing with mail from postmaster@

JEMH by default will blacklist mail from postmaster as its usually noise isn't desireable in Jira. New profiles by default have:

Profile > Whitelist > Blacklist > Blacklist Senders : mailer-daemon@googlemail.com, postmaster@.*

In order let mail from postmaster@ get through, that value needs to be removed:

Profile > Whitelist > Blacklist > Blacklist Senders : mailer-daemon@googlemail.com

Enabling Script Field Processor

Set:

Profile > Directives > Directive Processing Behaviour : On Create or Comment

Enable

Profile > Field Processors > Script (Advanced)

Create a Test Case

The following is a redacted email, based on a real email from this scenario, create a JEMH Test Case for this. NOTE: the to: address is "mailbox@blah.com", ensure your example Profile has this as a catchemail to be processable.

From: Microsoft Outlook <postmaster@blah.onmicrosoft.com>
Message-ID: <ddd56737-4c30-49cb-9a47-755b35c541e0@IncidentReport>
Date: Mon, 25 Mar 2019 17:01:38 +0000
Subject: undeliverable: == wrapper message subject ==
Content-Transfer-Encoding: binary
To: <mailbox@blah.com>
MIME-Version: 1.0
Content-Type: multipart/mixed;
	boundary="_c174e17d-77b3-410e-a29d-282be6dba286_"

--_c174e17d-77b3-410e-a29d-282be6dba286_
Content-Type: multipart/alternative;
	boundary="_92e980ec-bb97-4689-9b17-6b97c258e494_"

--_92e980ec-bb97-4689-9b17-6b97c258e494_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit

THIS IS THE TEXT

--_92e980ec-bb97-4689-9b17-6b97c258e494_
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: 7bit

<html>THIS IS THE HTML</html>

--_92e980ec-bb97-4689-9b17-6b97c258e494_--

--_c174e17d-77b3-410e-a29d-282be6dba286_
Content-Type: message/rfc822
Content-Transfer-Encoding: binary

From: Mr Blue <blue@blah.com>
To: Mr Mailbox <mailbox@blah.com>
Subject: undeliverable: == sub part message/rfc822 ==
X-MS-Exchange-MessageSentRepresentingType: 1
Date: Mon, 25 Mar 2019 17:01:38 +0000
Message-ID:
	<MWHPR12MB1376027904BC0F1632D0DD15865E0@MWHPR12MB1376.xxxx.prod.outlook.com>
Accept-Language: en-US
Content-Language: en-US
Content-Type: multipart/related;
	boundary="_002_MWHPR12MB1376027904BC0F1632D0DD15865E0MWHPR12MB1376namp_";
	type="text/html"
MIME-Version: 1.0

--_002_MWHPR12MB1376027904BC0F1632D0DD15865E0MWHPR12MB1376namp_
Content-Type: text/html; charset="us-ascii"

<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal">Poor reputation <o:p></o:p></p>
<p class="MsoNormal">Blah Blah Blah <o:p></o:p></p>
</div>
</body>
</html>

--_002_MWHPR12MB1376027904BC0F1632D0DD15865E0MWHPR12MB1376namp_--

--_c174e17d-77b3-410e-a29d-282be6dba286_--

Update the Script Field Processor script

In this example, we use a utility method in jemhUtils "getMessageSubParts" to locate mime multiparts of mime type message/rfc822. The test here is to see if the body content of the sub part starts with "Poor reputation". If it does, we log a message for auditing to explain why its allowed (but dont affect the outcome) otherwise we use a further utility method to dropMessage silently. Effectively re-instating the beaviour of the blacklist, with a bit of related auditing churn.

This example will store in the "resultMap" values for the 'description' and 'summary' (where no "relatedIssue" is defined, ie during create):

print('senderAddress: '+fromAddress.getAddress())
if (fromAddress.getAddress().startsWith('postmaster@'))
{
    print('subject is '+subject);
    print('message is '+message);
    
    var subParts = jemhUtils.getMessageSubParts(message);
    
    print ('got '+subParts.size()+' message sub parts x');
    if (subParts.size()>0)
    {
        var subMail = subParts.get(0);
        var subMailSubject=subMail.getSubject()
        var subMailBody=jemhUtils.getMessageBody(subMail);
        if (subMailBody.startsWith('Poor reputation'))
        {
            print('POOR REP, probably OK');
            resultMap.put('description', 'exchange sub mail body:\n'+subMailBody);
    		if (relatedIssue === null)
            {
    			resultMap.put('summary', subMailSubject);
            }        
        }
        else
        {
            print('test failed');
            jemhUtils.dropMessage('reject mail from postmaster with message/rfc822 sub-part that *failed the test*');
        }
    }
    else
    {
        print('postmaster mail');
        jemhUtils.dropMessage('reject mail from postmaster without message/rfc822 sub-parts by default');
    }
}   
else
{
    print('mail not from postmster@, all good');
}

Allowed mail processing Screenshots

Edit time Test outcome	Test Case execution	Created Issue

Edit time Test outcome	Test Case execution	Created Issue

Example of a postmaster@ mail that DIDNT pass the above test:

MIME-Version: 1.0
Received: by 10.223.112.12 with HTTP; Sat, 18 Jun 2011 22:42:26 -0700 (PDT)
Date: Sun, 19 Jun 2011 17:42:26 +1200
Message-ID: <BANLkTinB1mfSh+GwOXGNWoL4SyDvOpdBoQ@mail.gmail.com>
Subject: This is a starting email template, update as required
From: postmaster@blah.onmicrosoft.com
To: mailbox@blah.com
Content-Type: text/plain; charset=UTF-8

drop me

Silently dropped processing Screenshots

Test Case execution

Test Case execution