JEMH Installation for Jira 4.x

Upgrading from Jira 3.13.x

Since JEMH 0.9.3/Jira4, packages have been renamed. All references to them must also change, this means the following:

  1. Update the configuration file as follows:
    alternateIDLookupClassList=com.javahollic.jira.emh.engine.LDAPAlternateUserIDLookup
    emailFieldProcessorPackageList=com.javahollic.jira.emh.processor
    
    • Delete the existing JEMH Service
    • update the POP Service class in 'popservice.xml' (see later)
    • restart jira
    • recreate the handler; the new handler class is:
      com.javahollic.jira.emh.service.CreateOrCommentHandler
      

Before you begin

Some basics

JEMH is not a plugin (yet), it is invoked by a POP or IMAP Service that you have to create, it is not a 'Service' and cannot be installed (will result in not a Jira Service exceptions...). Also, until your mail-server and POP settings are correct, JEMH will do not a lot, JEMH processes mail after its been retrieved by the relevant service.

(info) See below for details on where the config file goes, and how you tell JEMH where it is and what its called.

Configuration Options

You can add handler options through the Jira Service UI, if provided they will override the settings in the config file. (warning) Settings applicable to the standard CreateComment handler have been found wanting and are not applicable to JEMH, dont make assumptions, consult the JEMH Usage Guide for detailed information on Configuration Options. Settings that are used should be comma delimited, no spaces.

The Configuration file

The Example configuration file is a starting point, understand what options are applicable or not to your setup.

If you run into trouble

Simplify your config file, try alternate cases. The issue that seems to be biting at the moment is Outlook 2007 HTML mail not being processed correctly, try workarounds e.g. plain text, to get a better feel for the problem before reporting 'it doesn't work'. I'm happy to help on an ad-hoc basis through http://www.getsatisfaction.com/javahollic, if you need hands on support, contact me by email.

JEMH Installation

Add the JEMH JAR

Get the JAR specific for your version of Jira from the top of the page and copy to the Jira WEB-INF/lib folder

Add the ldaputils jar (if you use ldap)

Get the latest ldaputils jar (currently 1.0.13) from the LDAP Util library page, take the example-ldaputil.properties file from the JAR and rename it ldaputil.properties, editing it as appropriate

Setup Logging (optional)

JEMH

Modify / copy-and-modify the WEB-INF/classes/log4j.properties , this will enable logging of all EMH messages to a separate file. Default is DEBUG, other values are 'INFO', 'WARN' and 'ERROR'.

#
# Create a separate appender
#
log4j.appender.EMHFileLog=org.apache.log4j.RollingFileAppender
log4j.appender.EMHFileLog.File=atlassian-jira-emh.log
log4j.appender.EMHFileLog.MaxFileSize=20480KB
log4j.appender.EMHFileLog.MaxBackupIndex=5
log4j.appender.EMHFileLog.layout=org.apache.log4j.PatternLayout
log4j.appender.EMHFileLog.layout.ConversionPattern=%d %t %p [%c{4}] %m%n
log4j.appender.EMHFileLog.Threshold=DEBUG

#
# add entries for the three EMH packages
#
log4j.logger.com.javahollic.jira.emh.service=DEBUG, EMHFileLog
log4j.additivity.com.javahollic.jira.emh.service=false

log4j.logger.com.javahollic.jira.emh.engine=DEBUG, EMHFileLog
log4j.additivity.com.javahollic.jira.emh.engine=false

log4j.logger.com.javahollic.jira.emh.processor=DEBUG, EMHFileLog
log4j.additivity.com.javahollic.jira.emh.processor=false

JIMI

(warning) JIMI is in development and doesnt do anything useful yet, just updating docs for my own use

#---------------------------------------
#
# Create a separate appender
#
log4j.appender.JIMIFileLog=org.apache.log4j.RollingFileAppender
log4j.appender.JIMIFileLog.File=atlassian-jira-jimi.log
log4j.appender.JIMIFileLog.MaxFileSize=20480KB
log4j.appender.JIMIFileLog.MaxBackupIndex=5
log4j.appender.JIMIFileLog.layout=org.apache.log4j.PatternLayout
log4j.appender.JIMIFileLog.layout.ConversionPattern=%d %t %p [%c{4}] %m%n
log4j.appender.JIMIFileLog.Threshold=DEBUG

log4j.logger.com.javahollic.jira.im=DEBUG, JIMIFileLog
log4j.additivity.com.javahollic.jira.im=false

log4j.logger.com.javahollic.im.bot=DEBUG, JIMIFileLog
log4j.additivity.com.javahollic.im.bot=false

log4j.logger.com.javahollic.im.transport=DEBUG, JIMIFileLog
log4j.additivity.com.javahollic.im.transport=false

(info) Enabling detailed logging of lower level mail operations in Jira is covered in Logging email protocol details.

What no log file?

Whether working on Windows or Linux, a lack of recently updated atlassian-jira.log and atlassian-jira-emh.log usually means file permissions are at fault....

  1. Make Jira aware of the new handler

    POP / Imap?

    This example shows how to setup with POP, Imap is equally possible, just not documented, to use Imap, reconfigure Dovecot, and instead of modifying .../pop/popservice.xml modify .../imap/imapservice.xml.

    Update WEB-INF/classes/services/com/atlassian/jira/service/services/pop/popservice.xml to include:
    <value>
      <key>com.javahollic.jira.emh.service.CreateOrCommentHandler</key>
      <value>Extendable Mail Handler - CreateOrComment</value>
    </value>
    
  2. Ensure a valid POP Sever is configured
    This is under Global Settings/Mail Servers. If the virtual mailbox approach is used, only one mailbox needs to be setup, in the example shared@jira.myco.net:
  3. Configure the handler
    Modifiy the two properties files. when done, they should be copied to WEB-INF/classes:
    (info) If you use ldap, the minimum config required is to identify a valid user which is used to validate ldap connections. If not, you don't need this file.
    (info) The EMH config file is just an extension of the properties that a mail handler is given, anything you could put in the Jira UI, you can equally add here. Look at the settings, understand what they do, make changes as appropriate for your needs.
    (warning) The contents of the configuration file changes as new features are added and refinements are made, check the 'example-emh.properties' available inside every JAR.

Configuration file, where it goes, how to refer to it

The handler requires you to provide the name of the config file. You do this through the Service, and also http://confluence.atlassian.com/display/JIRA/Creating+Issues+and+Comments+from+Email:

Some example Handler parameters are:

projectAutoAssign=true,issuetype=3,configFile=emh.properties
  1. Restart Jira

JEMH Handler Configuration File

The following sections describe the fields that can be used to configure JEMH. Organised by where they are used, in future I will try to expand on the documentation and provide more detailed examples.
(info) An example configuration file is packaged in every JAR, called 'example-emh.properties', contrast this file to your local file for changes.

Option

Example value

Description

notifyUsers

true

Notifies users if their accounts are created, also used to notify users on issue creation, necessary when users are anonymous and would like notification of issue details.

notifyUsersOnIssueCreation

none|nonjira|jira|all

This provides a way for users to be notified of email driven issue creation irrespective of any other system or user configuration options

assignNonJiraUsersOriginalEmailAddressToCustomField

afieldname

As the excessively descriptive field describes, will populate the given custom field with String value of the email address in CSV fashion. This value can also be populated as a directive... (warning) This is a security risk, just knowing the issue key (guessing even) would enable anyone with email access to add themselves into the issue. Default is disabled. To Fix (JEMH-51)

defaultReportUsername

abod

identifies a 'default' user for anonymous email submission

defaultIssueType

3 or bug

default issue type for created issues

defaultPriority

3 or critical

default priority for created issues

defaultIssueLinkType

relates

default link type for linked issues

defaultProject

APROJ|Magic Project|10002

an identifier for a project

defaultComponent

Some Component|10012

an identifier for a component in the given project

whitelistSenders

(.*)@(.*)yourco.net

A regexp CSV list. If set, no other domains/accounts will succeed (implying blacklist of other domains), the blacklist can be used to further restrict within the whitelisted domains. If not set, all address will pass. Supports java5 regular expressions

blacklistSenders

postmaster@xyz.com,(.*)@(.*)otherco.net

A regexp CSV list of addresses that will not be processed by Jira, and will be automatically forwarded (if silently drop not set) to the configured 'forwarder' of the related Mail Service, java5 regexp supported

blacklistRecipients

jira-noreply@yourco.net,noemailprojectkey@yourco.net

as above

silentlyDropBlacklistMatches

true

If set to false, causes blacklisted mails to not be forwarded, dropping all of them. Possibly useful for high volume environments, blacklisted mails will always be deleted

greylistSenders

postmaster@yourco.net,.*@yourco.com

CSV regexp list of address that might get blocked depending on subject

greylistSendersSubjectRegexps

Delivery to the following recipients failed,.*Delivery Status Notification.*Failure.*

CSV regexp list, for greylist subject matches

greylistRelatedUserBodyRegexp

[ ]*to:[ ]((.*)@(.*))$

Enables to: addresses on system generated emails to be extracted, and related users tested for being active, inactive users make the related mail a match

silentlyDropGreylistSubjectMatches

true

If set to false, causes blacklisted mails to not be forwarded, dropping all of them, if true.

emailFieldProcessorPriorityLowerLimit

0

If set will set the lower priority limit that processors are enabled, allowing users to configure just a subset of available field processors to be executed in priority order (procPri>=lowerLimit and procPri<=upperLimit). Eg, MailFormNgProcessor=1000, CSVProcessor=10000, 'default' Atlassian Processor=2147483647

emailFieldProcessorPriorityUpperLimit

2147483647

As above, but the upper limit

CommentHandler

Option

Example value

Description

stripQuotes

false

strips > comment lines from included emails when Jira emails are replied to. Overrides system default settings to forward an entire email (from outside Jira) including comments, into a Jira ticket.

AbstractFieldProcessor

Option

Example value

Description

debugPrintEmail

true

will print out emails as they are received for debugging

alternateIDLookupClassList

x.y.AClass,p.q.BClass

List of classes used to do email -> ID lookups

domainEquivalency

my.co.uk:my.com

related to LDAP provider, indicates given domains 'are the same'. _You will need to set this to your domains 'aliases' for ldap alternate lookup to work

emailFieldProcessorPackageList

com.javahollic.jira.emh.service.processor

List ofpackages to be searched for Field Processors that are used to process emails

projectAutoAssign

true

attempt to get the project Key from the to: address (useful if you have a combined destination mailbox)

forceUserIdCase

true

will force the case of userids for lookups, required for case sensitive DB's

lowercaseUserids

true

if true will force lowercase, false or missing indicates upper case

catchEmail

xyz@jira.myco.net

for use with the Basic field processor, enables a common email account to only have email retrieved for the given 'catch' addressee

createUsers

true

enables on the fly creation of users, whether being the 'sender' or a 'watcher', however so determined.

createUsersIDFrom

name/email/alternate

enables the selection of userid to be derived from the name part of the email, the full email, or via the alternate ID provider framework (there is an AD LDAP provider included in the project

createUsersIDFromEmailDomainWhitelist

.*@myco.net,.*@,myco.com

Enables user creation to be be restricted to given domains

createUsersIDFromEmailBlacklist

sales@atlassian.com

Stops emails from sales@atlassian.com (or any other regexp), from being used as a valid userID

notifyUsers

true

notify users of issue creation

deleteEmail

true

if set to false, will prevent Jira from deleting mails from the mailbox after processing

autoFlushMailQueue

true

will cause the Mail Queue to be flushed whenever the EMH handler is run (which only occurs when there is mail to process)

addWatchersFromCC

true

will make all valid users watchers of the created issue, in addition to anything that may be set as directive

createUsersIDFrom

email|name|alternate

select mechanism for determining the userID, email is user@someplace.com, name is user@somplace.com, alternate means use the alternateID provider classes

autoJoinGroup

email-users

enables auto-created users to be automatically joined to a given group

indicateWhichProcessorProcessedMail

true/false

causes a one liner to be inserted at the top of the jira issue, to assist in debugging, or as an indicator for where the issue came from

ccHandling

toWatcher|toCustomField

will cause users to be derived from email cc: entries as well as from the directive ccusers, to be added as watchers to the issue or to be merged into a given custom field (must be capable of accepting user type, eg MultiUserCF type

ccHandlingTargetCustomFieldName

Some CF Name

If the ccHandling field indicates a custom field should be used, this field idetifies that field by name, whitespace allowed

JEMH Poker Installation

This JEMH Poker is a simple java application that triggers the JEMH to do an immediate mail retrieval. The current 1.0 release of the JEMH Poker contains a simple file monitor, suitable for most Unix or Linux platforms and associated file-based mail systems. Exchange is an unknown quantity, as Microsoft products are not known for their openness - the approach I took in my organization was to use Exchange purely as a vehicle to get mail to where I could do something useful with it (see setting up Postfix smtp server and Dovecot POP server pages here).

Installation of the remote agent

  1. Extract the tgz in /usr/local
    cd /usr/local
    tar xzf filename.tgz
    
  2. Make the launch shell script executable
    cd jemh-mail-notifier-......
    chmod +x *.sh
    
  3. Link a common name to the current version
    ln -s jemh-mail-notifier-...... jemh-mail-notifier
    
  4. Ensure you have Java in your path, the following should give some version information. If not, install Java 1.5+ or modify the script (you should know what to do)
    java -version
    
  5. Edit the configuration file emh-poker.properties, set the server (IP address or DNS name) and the port (default is 7777).
  6. Run the script as the appropriate user for your environemnt. With Postfix, you may need to be running as Postfix. Running as root is to be avoided, create a service user as follows:
    root@server:/usr/local/jemh-mail-notifier-1.1-SNAPSHOT# adduser --disabled-login --no-create-home -ingroup postfix jirapoker
    Adding user `jirapoker' ...
    Adding new user `jirapoker' (1009) with group `postfix' ...
    Not creating home directory `/home/jirapoker'.
    Changing the user information for jirapoker
    Enter the new value, or press ENTER for the default
    	Full Name []: 
    	Room Number []: 
    	Work Phone []: 
    	Home Phone []: 
    	Other []: 
    Is the information correct? [Y/n] Y
    

Create a SysV startup script in /etc/init.d

Create it as 'jemh-mail-notifier'

#! /bin/sh
### BEGIN INIT INFO
# Provides:          jemh-mail-notifier
# Required-Start:    
# Required-Stop:
# Default-Start:     2
# Default-Stop:
# Short-Description: Monitors a mailbox and poke jira when mail arrives
### END INIT INFO

PATH=/sbin:/bin

. /lib/lsb/init-functions

case "$1" in
  start)
	cd  /usr/local/jemh-mail-notifier
	./run.sh &
	;;
  stop)
	# No-op
        echo Cant stop it from here, use 'ps -ef | grep java' at the prompt
	;;
  *)
	echo "Usage: $0 start" >&2
	exit 3
	;;
esac

To make the service start at bootup:

sudo apt-get install sysvconfig
sysvconfig

Now find the jemh-mail-notifer and enable for startup, click finish to save.

Installing the Jira JEMH Poker Service

The main JEMH includes everything necessary, the only thing that's required is a bit of configuration:

  1. Login to Jira as admin
  2. Go to the Services screen
  3. Add a service:

    Name

    JEMH Poker Service

    Class

    com.javahollic.jira.emh.service.EMHPokerService

    Delay

    1 note: this is essentially the delay in initial startup as Jira starts

  4. Select the default port options or change as you see fit (must fix Poker configuration, above)

Installing JIMI

See JIMI Installation for more details. Once installed, as a system admin, after setting up all JEMH components you should see something like:

Login ID's

Different providers require different forms of XMPP_LOGIN. Here is whats known:

Provider

Format

Notes

google talk

user@domain.com

Domain must be appropriate for your user, ie, @gmail.com, or @googlemail.com

jabber

user