Common Problems

JEMH (for Server only) Is not licensed through Atlassian Marketplace

Its historic, and with JSD Agent tiers, now a bit more complex.  Its on the list but is not on the short term roadmp.  JEMH remains a vendor licensed addon at this time, all renewals /quotes/ invoices to go through sales@thepluginpeople.com

JEMH-5982 - Getting issue details... STATUS

Jira Server technical upgrade notes (7.13.x - 8.13.x)

The follow covers key features Atlassian included in these releases:

• https://confluence.atlassian.com/jirasoftware/jira-software-8-13-long-term-support-release-change-log-1018783382.html

JavaMail FAQ

JIRA uses the JavaMail library to process all mail, therefore so does JEMH.  There is a JavaMail FAQ that also contains combined experience of JavaMail users working with disparate mail servers, and the various integration problems these throw up - as well as hopefully solutions, its always worth a look:

• http://www.oracle.com/technetwork/java/javamail/faq/index.html 
  

JEMH Server Specific

Couldn't flush user prefs: java.util.prefs.BackingStoreException: java.lang.NullPointerException

This is caused by using JDK 8 and JEMH licensing, and is harmless. Workaround steps are listed below:

Log Volume reduction workaroud

JEMH doesn't use UserPreferences, its not specifically common to Atlassian apps, but I couldn't judge or assert that they aren't used by any application or add-on, even if there were other users, this problem would likely affect them also. Extending the retry (as per site above) is one way, setting an environment option to 30 min

Extend the retry, to reduce the frequency of the misleading error message, add the following startup variable:

JAVA_OPTS="$JAVA_OPTS -Djava.util.prefs.syncInterval=900 "

(see Atlassian docs for background), eg for linux installs (30 is the default (seconds), 900=15 minutes, 3600 is hourly, 86400 is daily)

JEMH 1.7.x Specific

You do not have permission to assign issues

WHY: After 5years of using the IssueManager to create issues, JEMH 1.7.x finally resolved JEMH-280 (fixed in 1.7.0) by a large internal change to use the JIRA IssueService (simply, to set most issue properties, including custom fields, and to support more custom fields in doing so, during the ISSUE_CREATION workflow step).  This change requires the use of IssueInputParameters to describe the issue fields for creation and update.  That descriptor is also used by JIRA to validate the properties, which should (in theory) reduce the redundant checking that JEMH was previously doing, however, it also means JEMH is entirely dependant on JIRA and its validation.  Despite IssueService being used for 5years there appear to be a number of cracks that are still present, some can be worked around (https://jira.atlassian.com/browse/JRA-42609) some cannot, so easily:

 WHAT: In order to create issues with JIRA default values for custom fields present, JEMH uses a method on the IssueInputParmeters to say 'hey, fill in the blanks' ( setApplyDefaultValuesWhenParameterNotProvided(true)  ) how awesome!  Yes, it does exactly that, but also causes JRA-42827, which is not awesome.  Unfortunately, NOT setting that, stops JIRA custom fields from being initialized with default values, but also introduced the You do not have permission to assign issues errors that are showing up in 1.7.13

NEXT STEPS:

In order to impact the least users, the only option I have is to revert to setApplyDefaultValuesWhenParameterNotProvided(true) which will fix You do not have permission to assign issues errors but re-break any users wanting to make use of the empty Resolution date, which affects two common cases:

• JEMH Thread Matching limits, which uses the Resolution date (empty) to determine the issue is not resolved, outcome, new issues created all the time
• JEMH Regexp field processor also uses the same approach when locating an issue to 'thread' to via an external system KEY, outcome, new issues created all the time.

FIXED IN: JEMH 1.7.14 will use setApplyDefaultValuesWhenParameterNotProvided(true) If you are not dependant on the Thread location, then upgrade at will, otherwise, don't, as it will create issues rather than comment.

Please vote for:

• https://jira.atlassian.com/browse/JRA-42827 ( IssueService IssueInputParameters setApplyDefaultValuesWhenParameterNotProvided )
• https://jira.atlassian.com/browse/JRA-42889 ( ASSIGN_ISSUE required when it should not )
• https://jira.atlassian.com/browse/JRA-42609 (  Assignee permission check during IssueService validateUpdate uses the JIRA authentication context user not the supplied user )

Sorry, you can't create any issues right now, as you need to have access to a JIRA application to be able to create issues

 See also later in the page: A JIRA user without right to use can no longer create issues

Primarily the security context of who is creating issues must be allocated a JIRA seat.  The easy case is to automatically join incoming mail JIRA account holders into a group that has application access.  If that's not what you want to do, there are two options:

1. Treat the unprivileged user as an email only user, resorting to storing email addresses of individuals within custom fields.  A reporter (eg the User > Default Reporter) will be required to be set to allow this
2. Create issues via JEMH Default reporter, in such cases, the security context for issue creation is the default reporter.

For both cases see Create Issues.

JEMH DataCenter edition licensing problems

See Licensing for current information on Atlassian problems that affect JEMH.

I upgraded JEMH and it now complains about an Server / DataCenter edition mismatch

A bug in the UPM (AMKTHELP-12797) incorrectly downloads the Server edition of JEMH which is not compatible in DataCenter environments (as of 1 SEP 2018 JEMH is available in two editions, Server and DC, its required that the correct edition is used in the related environment).  The solution is to manually download the DataCenter edition JAR from Marketplace -  Be careful, check version compatibility!

I can't upgrade to JEMH DataCenter edition

The process to install JEMH DataCenter is to uninstall the Server version and use Find Apps to search for JEMH again, when executed from a DataCenter licensed instance, you will get the DataCenter edition of JEMH.

I can't generate an eval for JEMH DataCenter edition from Manage Addons

Evals are available from:

• https://my.atlassian.com/addon/try/com.javahollic.jira.jemh-ui.data-center

Quoting for JEMH DataCenter

• https://www.atlassian.com/purchase/addon/com.javahollic.jira.jemh-ui.data-center

JEMH DataCenter edition problems

Concurrent mail retrieval resulting in duplicate issue creation

If you have massive email volume (mailbox perpetually contains > 00's of emails, and/or they are large (ie take time to download) then the following could affect you:

Data Center (7.2.9 / 7.3.6 tested, probably earlier releases too) can cause concurrent retrieval and processing of emails from a common mailbox.  Private support reference PS-14639, public issue tracker (not currently public).

• https://jira.atlassian.com/browse/JRASERVER-65197 (fixed in 7.0, 7.2.7+) : JIRA Data Center Cache replication stops due to Heartbeat jobs failure
  
• https://jira.atlassian.com/browse/JRASERVER-65581 (fixed in 7.2.11, 7.5.1+): JIRA creates duplicate issues when the standard mail handler has to process a large number of emails or large amounts of data.
  

Common JEMH Problems

Emails without a body are not processed

JEMH will usually forward these mails, if you do want to support them, you can define a default 'Comment' payload used as issue description during create:

• Profile > Issue > Comment

Repeating log errors

The following is a harmless side effect of a library within JEMH Server, tracked internally  JEMH-3889 - Getting issue details... STATUS , 

ERROR: Bundle com.javahollic.jira.jemh-ui [196] Unable to get module class path. (java.lang.NullPointerException)
java.lang.NullPointerException
at org.apache.felix.framework.BundleRevisionImpl.calculateContentPath(BundleRevisionImpl.java:432)
at org.apache.felix.framework.BundleRevisionImpl.initializeContentPath(BundleRevisionImpl.java:365)
at org.apache.felix.framework.BundleRevisionImpl.getContentPath(BundleRevisionImpl.java:351)
at org.apache.felix.framework.BundleRevisionImpl.getResourcesLocal(BundleRevisionImpl.java:515)
at org.apache.felix.framework.BundleWiringImpl.findResourcesByDelegation(BundleWiringImpl.java:1213)
at org.apache.felix.framework.BundleWiringImpl.getResourcesByDelegation(BundleWiringImpl.java:1123)
at org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.getResources(BundleWiringImpl.java:2676)
at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.nextProviderClass(Unknown Source)
at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNextService(Unknown Source)
at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNext(Unknown Source)
at java.base/java.util.ServiceLoader$2.hasNext(Unknown Source)
at java.base/java.util.ServiceLoader$3.hasNext(Unknown Source)
at java.xml/javax.xml.parsers.FactoryFinder$1.run(Unknown Source)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.xml/javax.xml.parsers.FactoryFinder.findServiceProvider(Unknown Source)
at java.xml/javax.xml.parsers.FactoryFinder.find(Unknown Source)
at java.xml/javax.xml.parsers.DocumentBuilderFactory.newInstance(Unknown Source)
at java.prefs/java.util.prefs.XmlSupport.createPrefsDoc(Unknown Source)
at java.prefs/java.util.prefs.XmlSupport.exportMap(Unknown Source)
at java.prefs/java.util.prefs.FileSystemPreferences$7.run(Unknown Source)
at java.prefs/java.util.prefs.FileSystemPreferences$7.run(Unknown Source)

Workaround:

Log Volume reduction workaround

• Useful background: http://allaboutbalance.com/articles/disableprefs/

JEMH doesn't use UserPreferences, its not specifically common to Atlassian apps, but I couldn't judge or assert that they aren't used by any application or add-on, even if there were other users, this problem would likely affect them also. Extending the retry (as per site above) is one way, setting an environment option to 30 min

Extend the retry, to reduce the frequency of the misleading error message, add the following startup variable (see Atlassian docs for background), e.g. for linux installs (30 is the default (seconds), 900=15 minutes, 3600 is hourly, 86400 is daily)

JAVA_OPTS="$JAVA_OPTS -Djava.util.prefs.syncInterval=900 "

One off errors in the log

From time to time you may see the following, caused by periodic housecleaning of the audit history retained data.  It is caused by the slight difference in timestamp between an audit record, and a hint record, where the purge point happens after the audit record but before the hints, so the hints are not purged (happens first) and so when audit records are purged, the last one cannot be removed because that record has child entities with a foreign key reference.  It is not a long term issue, and most likely, will be resolved in subsequent purge operations,  JEMH-6583 - Getting issue details... STATUS  has been logged to address this.

java.sql.SQLIntegrityConstraintViolationException: ORA-02292: integrity constraint (JIRA743USER.fk_ao_78c957_audith1314010416) violated - child record found
at com.atlassian.activeobjects.internal.EntityManagedActiveObjects.deleteWithSQL(EntityManagedActiveObjects.java:118)
at com.atlassian.activeobjects.osgi.TenantAwareActiveObjects.deleteWithSQL(TenantAwareActiveObjects.java:281)
... 2 filtered
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302)
at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.doInvoke(ServiceInvoker.java:56)
at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.invoke(ServiceInvoker.java:60)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:133)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:121)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.eclipse.gemini.blueprint.service.util.internal.aop.ServiceTCCLInterceptor.invokeUnprivileged(ServiceTCCLInterceptor.java:70)
at org.eclipse.gemini.blueprint.service.util.internal.aop.ServiceTCCLInterceptor.invoke(ServiceTCCLInterceptor.java:53)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.eclipse.gemini.blueprint.service.importer.support.LocalBundleContextAdvice.invoke(LocalBundleContextAdvice.java:57)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:133)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:121)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208)
at com.sun.proxy.$Proxy2240.deleteWithSQL(Unknown Source)
c

Issue fails to create with indexing errors

Issue creation fails for no obvious reason and JEMH processing details do not suggest why. In JEMH logs you see something similar to this:

2016-09-16 13:20:15,599 atlassian-scheduler-quartz1.clustered_Worker-2 DEBUG [jira.emh.engine.IssueCreationHelper] Create resulted in 0 errors 
2016-09-16 13:20:15,687 atlassian-scheduler-quartz1.clustered_Worker-2 INFO [jira.emh.engine.IssueCreationHelper] issue not created!!!
2016-09-16 13:20:15,687 atlassian-scheduler-quartz1.clustered_Worker-2 INFO [jira.emh.engine.IssueCreationHelper] Error creating issue: Indexing completed with 4 errors

A possible cause of this is that the UNIX open file limit has been reached by the JIRA application.  See here for more information on this, including workarounds: https://confluence.atlassian.com/jirakb/loss-of-functionality-due-to-too-many-open-files-errors-156862102.html

Auditing History purge does not work with "Retain Failures" enabled (JIRA 7.x)

The audit history purge operation fails when the "retain failures" setting is enabled.  The AO streaming API that is provided by Atlassian is "buggy", and means that our once-efficient data reading is not possible (https://ecosystem.atlassian.net/browse/AO-694).  As a result, we have made the decision to remove this purge setting in newer versions of JEMH JEMH-4962 - Getting issue details... STATUS  (resolved in 1.9.41, 1.8.50, 2.0.5).

Custom Field values are cached causing recipients to not receive notifications (JIRA 7.3.0 - 7.3.4 , JEMH 2.1.0 - 2.1.11)

When a custom field on an issue is modified post issue creation the previous values for the custom field are cached incorrectly. The new custom field values will appear to be correct on the issue screen but this change will not be reflected when JEMH sends notifications.

Example:

• Issue has custom field for CC users
• Issue is created with CC user aaa@test.com
• Custom Field for CC users is populated with aaa@test.com, aaa@test.com is notified of issue creation, and any comments on the issue
• Additional CC user is added to the custom field after issue creation (aab@test.com)
• aab@test.com will not be notified of issue updates

This issue was resolved in JEMH 2.1.12  JEMH-5599 - Getting issue details... STATUS

Mail fails to be retrieved from mail server

When a network problem prevents JIRA from communicating to its database, scheduled jobs can be halted indefinitely.  If this occurs to the job in charge of pulling mail, no mail will be retrieved.  We believe this affects JIRA 7.0.x and 7.1.x, it is reportedly fixed in JIRA  7.2.0, see https://jira.atlassian.com/browse/JRA-62072 (fixed in 7.2.0+).  Fixing this is therefore not possible in JEMH v1.8.x (JIRA 7.0.x) and JEMH v1.9.x (JIRA 7.1.x)

Oauth / Gmail / Switch to apps that use secure OAuth access

If you use gmail with simple user/pass you may receive email warning of upcoming plans to remove such access:

1. June 15, 2020 - Users who try to connect to an LSA for the first time will no longer be able to do so. This includes third-party apps that allow password-only access to Google calendars, contacts, and email via protocols such as CalDAV, CardDAV and IMAP. Users who have connected to LSAs prior to this date will be able to continue using them until usage of all LSAs is turned off.
2. February 15, 2021 - Access to LSAs will be turned off for all G Suite accounts.

Atlassian/Jira is responsible for retrieving mail that feeds JEMH, we cannot support you with related problems, please contact Atlassian support.

OAuth 2.0 was added to Jira 8.10:

• https://confluence.atlassian.com/jirasoftware/jira-software-8-10-x-release-notes-1004948108.html

Related Jira documentation:

• https://confluence.atlassian.com/adminjiraserver/integrating-with-oauth-2-0-1013845729.html

Problems with IMAP

Folder Closed / Repeat Processing

FolderClosedExceptions are always related to your mailboxserver.

IMAP is touted as a more advanced protocol than POP, which is true in some regards, but in some scenarios it can be problematic (see link below).  If you experience FolderClosedExceptions,

• https://jira.atlassian.com/browse/JRA-60200

Unlike POP which has exactly 1 client per mailbox, imap can discriminate mailbox content through folders, allowing mulitple 'clients' to connect to a common mailbox.  If 'many' connections/folders are used from a common mailbox, it is possible your mailbox will start randomly closing 'long(er) running'  connections in order to satisfy a new one.  For example:

• https://social.technet.microsoft.com/Forums/lync/en-US/57a7b662-926e-44aa-af14-7e57d47f3f18/are-there-concurrent-connection-limits-for-a-user-connecting-to-exchange-using-outlook-using-outlook?forum=exchangesvrclients
• https://community.spiceworks.com/topic/2244455-exchange-imap-connection-limit-still-at-16-after-setting-it-to-512

MessageRemoved

If you have concurrent access to a common mailbox its possible that a message may be 'removed' by another handler process, the error is likely to be harmless, unless the configuration caused the message to be dropped.

Performance Issues

In tests we found huge performance issues processing large messages.   If you are have occasional messages that won't process with IMAP, check your mailbox for large messages, its possible they are not able to be downloaded in time, in order to meet mailserver timeouts, or JIRA timeouts.  POP is a much better performing protocol for JIRA usage, as mail handlers require ALL the mail ALL the time, the IMAP 'features' for extracting bits of the message separately are clever but not required here.

Problems with POP

High 'forward' traffic is the result of using POP for mail retrieval on a mailbox that is also used interactively by a person.  What happens is that for interactive mail (experienced in Gmail) that mail sent TO a customer FROM the mailbox, and added to the Sent folder, also gets retrieved by POP, and is then rejected and forwarded.  Reviewing the addresses will show a catchemail Filter match failure, the TO: address not being valid (the customer address).  The only solution here is to stop using the mailbox interactively, or use IMAP (see above).

Notification History tab does not show on issues

Problem: You have event listener enabled and "store notification history" enabled in event listener project mapping.  However, the notification history tab is not showing on issues.

Explanation: There is currently a known bug where if you have a single project mapping with multiple project keys listed in CSV (e.g. "AA, AB, AC"), the notification tab will not be set to appear.  The bug is logged here: JEMH-3999.

Workaround until bug fixed: Separate the projects into multiple event listener project mappings.

Creating duplicate issues to multiple projects fails

In the scenario where you are sending one email to multiple addresses in order to create issues in multiple projects, JIRA fails to create all of the issues.  If Inbound Mail Logging is enabled, the following message will appear:

This is a known JIRA bug and may be fixed in a future version: https://jira.atlassian.com/browse/JRA-41831 (fixed: 7.0.2+) Duplicate issues creation fails - Creating multiple issues by one Email

JIRA fires Issue Assigned event for newly created Issues

https://confluence.atlassian.com/display/JIRAKB/JIRA+fires+Issue+Assigned+event+for+newly+created+issues

Issues aren't created

If you see the following error message in auditing drill down, it could relate to the choice of custom field for Email > Addressee Processing : Assign non jira-user Email to Text CustomField .  This field really needs to be a Multi-Line Text Custom Field type, because it doesn't take many email address to fill a 255 character single line field.  And this message will be shown in that case.  JEMH 1.7.24+ will contain a Profile advisory check for this scenario.  The reason JEMH doesn't 'only' show Multi-line fields is that other fields could be capable and its not a huge support problem.

The email did not result in a creation/comment/update. ...

Just upgraded your JIRA?

If you have upgraded your JIRA and are seeing messages like the one below, one possible cause is that you changed the JIRA organisation text when you upgraded. This would make your JEMH licence invalid. Please Contact Us if you suspect that is the case and we will be happy to help you out. For more information on JEMH Licensing, please refer to our JEMH Licensing Page.

As <username> the issue could not be created, see the following wiki page for more about Issue Creation scenarios: https://thepluginpeople.atlassian.net/wiki/display/JEMH/Create+Issues - The IssueInputParameters used during creation were:

Provided Value
fieldValues: [], actionParams<email parameters>...

nFeed causes JEMH editors to break

v5.3.0 of nFeed loads the ACE editor in jira.admin and jira.general contexts.  This affects Ad-Hoc notifications (general context) and all JEMH Admin editors (jira.admin) context.  Current workaround is to downgrade nFeed to 5.2.5 where this wasn't the case, or upgrade to 1.7.20+ where an effort to be compatible in this situation results in the adhoc editor being available, but that, and all JEMH editors do not have full screen edit.

• JEMH-3476 - Getting issue details... STATUS
• https://jira.valiantys.com/browse/NFEED-1185

nFeed Custom Field Setup

https://valiantys.atlassian.net/wiki/spaces/NFEED512X/pages/279838721/How+to+set+a+nFeed+field+from+JEMH

Unable to process mails encoded with UTF-7

Java doesn't support UTF-7, there are 3rd party (free) libraries like the Java UTF-7 Charset support project (download).  This ZIP contains ./jutf7-1.0.0/jutf7-1.0.0.jar , the JAR must be extracted, and copied into your JAVA installation folder: JAVA_HOME/jre/lib/ext , a JIRA restart will be required to make UTF-7 support available.

Using Exchange? Got duplicate issues?

If you enable inbound mail logging and see things like:

2014-08-25 00:01:25,891 WARN [PT_Email] atlassian-scheduler-quartz1.clustered_Worker-4 ServiceRunner    Blah Mail Handler IT Helpdesk Mail Handler[10100]: javax.mail.AuthenticationFailedException: AUTHENTICATE failed. while connecting to host 'mail.blah.com' as user 'blah\helpdesk' via protocol 'imaps

The following has been reported to help

-Dmail.imap.auth.plain.disable=true

Upgrades take too long / impact performance

When upgrading JEMH, database tables sometimes need to be updated.  If the upgrade jumps across many versions, potentially many such updates will need to run consecutively, adding delay.  These are one off upgrades and must be allowed to complete.

Minimising delays

mail.mime.multipart.ignoreexistingboundaryparameter=true

The delays are proportional to the data in the database, so customers with high volumes of email will notice this more.  To reduce impact of these delays the following actions can be taken:

1. Disable auditing and Delete All audit history (no impact to user data).  If you have already upgraded and disabled JEMH part way through the upgrade, you may be in a catch 22 situation.  To remove audit history with JEMH disabled, a low level database table clean up is needed, please see Audit History is too large (below).
2. Execute a database table content deletion on AO_78C957_NOTIFICATION_HIST (this will remove all Notification History)

Incorrect Message Association through reply

Empty MessageID's in notificationinstance.messageid cause replies with an empty reply-to: to be associated with issues incorrectly.  JEMH and Atlassian mail handlers suffer this problem historically. 

Solution part 1

JEMH 1.7 / 1.6.x latest maintenance will check messageID's for a value prior to using for message association (pre-emptying the problem)

Solution part 2

JEMH 1.7.x /1.6.x latest maintenance will check for replies with an empty reply-to: fields, and not it for a lookup in that case

Database fix

A resolution is to simply remove rows with empty values, combined with Solution part 1 will be enough to stop recurrence.

delete from notificationinstance where messageid='';

Why Attachments are sometimes missing from notifications driven by interactive comments/attachment uploads

When users add attachments 'with a comment' the attachments are added on the fly, timestamped as they are added.  If a user adds a few attachments, the timestamps are spread.  If a user adds a long issue or takes a coffee break before submitting the comment, the final ISSUE_UPDATED event fired due to the comment could be minutes apart from the related attachment timestamps (comments don't track attachments added at the same time).  JEMH uses an hour time-window to correlate that an added attachment is related to the 'event' that is driving the notification.  It works most of the time, but not in some situations where a user may add an attachment and then wait a while before sending.

The only workarounds are to enable "Require a comment to notify" in Event Listener, or to educate interactive users to type the comment in first so that there is a higher probability of correct correlation.

For now, there is nothing else that can be done, an issue has been logged ( https://jira.atlassian.com/browse/JRA-43448 ).

XML Profile Export not working

Its been noted that the profile export fails to 'produce' the XML output.  This has been traced to the Field Security Plugin (see  JEMH-1212 - Getting issue details... STATUS ) that needs to be upgraded to JFS 1.4.15_52 (available at downloads page), or disabled until the export has completed

Unable to get attachments, got an IO error: Expected ';', got ...

JEMH will not be able process emails that do not conform the the http://www.ietf.org/rfc/rfc2183.txt specification, and will complain with hintOgrams (if enabled):

Hint #1:
--------
Message : Unable to get attachments, got an IO error: Expected ';', got ","
Hint Detail : com.javahollic.jira.emh.engine.FieldProcessorFixer.fixupAttachments():1031
Caused By : class="javax".mail.internet.ParseException : message=Expected ';', got ","

Atlassian documentation: 

• https://forums.oracle.com/forums/thread.jspa?messageID=9994717
• https://confluence.atlassian.com/display/CONFKB/Certain+Mail+Messages+Fail+to+Import

Next Steps: The related attachment was not processed and is not processable.  A manual export of the mail, to text, can allow the formatting issues to be dealt with, and the email re-injected via a TestCase.

Possible workaround:

• http://docs.oracle.com/javaee/6/api/javax/mail/internet/package-summary.html , setting the environment variable may help:  
  • -Dmail.mime.parameters.strict=false

Configuration doesn't 'save'

When saves "don't work" this is a general indicator that a field you have entered (should I say crammed!) has exceeded the schema expectations. Its a little annoying to trace, but do this:

1. Uninstall / reinstall JEMH (this will flush the 'cache' of ActiveObject database records held in memory)
2. Edit a boolean form field somewhere, save.

The change should persist. Now, make (and save) one configuration change at a time. If you are storing 'large' quantities of text, for example in catchmail, it may need increasing. JEMH once thought 255characters was enough, but has since made these fields (and many more) unlimited text now. Schema changes aren't possible, so just need be be triaged through support as/when it arises. In this situation you can safely extend the schema, as described here:

The following Configuration fields are now unlimited text, contrast this fields you are changing:

• CreateUsersEmailWhitelist
• CreateUsersEmailBlacklist
• CatchEmailAddress
• JemhAddresseeRegexps
• SubjectCleanupRegexps
• BlacklistedAttachmentTypes
• WhitelistSenders
• GreylistSubjectRegexps
• GreylistEmailBodyRegexp
• GreylistSenders
• BlacklistSenders
• BlacklistRecipients

Updating your schema through a lossless conversion from char (255) to text will enable unlimited text to be entered.

Example:

alter table AO_78C957_CONFIG alter column CREATE_USERS_EMAIL_BLACKLIST text;

Dropping JEMH tables and removing JIRA_HOME/data/jemh still not enough to reinstall from blank

Oracle also has SEQuence tables that need to be cleared, for example:

DROP SEQUENCE JIRA.AO_78C957_ALIASES_ID_SEQ
GO
DROP SEQUENCE JIRA.AO_78C957_AUDITCONF_ID_SEQ
GO
DROP SEQUENCE JIRA.AO_78C957_AUDITEVENTS_ID_SEQ
GO
DROP SEQUENCE JIRA.AO_78C957_AUDITHINT_ID_SEQ
GO
DROP SEQUENCE JIRA.AO_78C957_AUDITMETA_ID_SEQ
GO
DROP SEQUENCE JIRA.AO_78C957_AUDITMETA1808070293
GO
DROP SEQUENCE JIRA.AO_78C957_AUDITMETA2102932222
GO
DROP SEQUENCE JIRA.AO_78C957_AUTOLABEL480471400
GO
DROP SEQUENCE JIRA.AO_78C957_AUTOLABELCONF_ID_SEQ
GO
DROP SEQUENCE JIRA.AO_78C957_AUTOLABELSUBS_ID_SEQ
GO
DROP SEQUENCE JIRA.AO_78C957_BLACKLISTING_ID_SEQ
GO
DROP SEQUENCE JIRA.AO_78C957_CERTCONF_ID_SEQ
GO
DROP SEQUENCE JIRA.AO_78C957_CFDEFAULT_ID_SEQ
GO
DROP SEQUENCE JIRA.AO_78C957_COLONSUFX1973341390
GO
DROP SEQUENCE JIRA.AO_78C957_CONFIG_ID_SEQ
GO
DROP SEQUENCE JIRA.AO_78C957_FIELDPROC_ID_SEQ
GO
DROP SEQUENCE JIRA.AO_78C957_LDAPCONF_ID_SEQ
GO
DROP SEQUENCE JIRA.AO_78C957_LSNR_CONF_ID_SEQ
GO
DROP SEQUENCE JIRA.AO_78C957_LSNR_JPRO1882553326
GO
DROP SEQUENCE JIRA.AO_78C957_LSNR_PROJ_ID_SEQ
GO
DROP SEQUENCE JIRA.AO_78C957_LSNR_PROJ488737470
GO
DROP SEQUENCE JIRA.AO_78C957_NAGCONFIG_ID_SEQ
GO
DROP SEQUENCE JIRA.AO_78C957_NAGPHRASE_ID_SEQ
GO
DROP SEQUENCE JIRA.AO_78C957_NAGPHRASESET_ID_SEQ
GO
DROP SEQUENCE JIRA.AO_78C957_NOTIFICAT1268048665
GO
DROP SEQUENCE JIRA.AO_78C957_PGPCONFIG_ID_SEQ
GO
DROP SEQUENCE JIRA.AO_78C957_PRJDOMMAP_ID_SEQ
GO
DROP SEQUENCE JIRA.AO_78C957_PRJGRPMAP_ID_SEQ
GO
DROP SEQUENCE JIRA.AO_78C957_PROFILE_ID_SEQ
GO
DROP SEQUENCE JIRA.AO_78C957_PROJCONFIG_ID_SEQ
GO
DROP SEQUENCE JIRA.AO_78C957_REGEXPPROCCNF_ID_SEQ
GO
DROP SEQUENCE JIRA.AO_78C957_REGEXPPROCMAP_ID_SEQ
GO
DROP SEQUENCE JIRA.AO_78C957_SUPPORT_ID_SEQ
GO
DROP SEQUENCE JIRA.AO_78C957_TEMPLATESET_ID_SEQ
GO
DROP SEQUENCE JIRA.AO_78C957_TESTCASE_ID_SEQ
GO
DROP SEQUENCE JIRA.AO_78C957_TRANSPORTS_ID_SEQ
GO
DROP SEQUENCE JIRA.AO_78C957_TWITTERCREDS_ID_SEQ
GO
DROP SEQUENCE JIRA.AO_78C957_WEBFORMCONF_ID_SEQ
GO
DROP SEQUENCE JIRA.AO_78C957_WEBFORMITEM_ID_SEQ
GO
DROP SEQUENCE JIRA.AO_78C957_WUFOOCONF_ID_SEQ
GO
DROP SEQUENCE JIRA.AO_78C957_WUFOOFORM1291342826
GO
DROP SEQUENCE JIRA.AO_78C957_WUFOOFORMS_ID_SEQ
GO

CommentQtyOrSizeFilter: MessagingException, failed to get email body: Missing start boundary

Symptom

JEMH Audit History can show some mails as unprocessed, with a filtration message:

"The message was filtered 
CommentQtyOrSizeFilter: MessagingException, failed to get email body: Missing start boundary" 
Filter Name: Comment size or number limit filter

Explanation

This happens because some emails are not correctly formatted, and contain missing mime/multipart boundaries. The JEMH Audit History stored email is a re-serialisation of the mail, the process of which actually generates a 'valid' email which could be re-processed successfully (convert it to a JEMH Test Case via the Audit History operations, edit the Test Case to pick the right profile, run the Test Case).

For background on the problem see: http://stackoverflow.com/questions/2043792/missing-start-boundary-exception-when-reading-messages-with-an-attachment-file

The Fix

The fix requires an environment flag to tell Javamail (the library JIRA and JEMH rely on for email processing) to allow this condition.  Updating JIRA environment variables is documented at: https://confluence.atlassian.com/display/JIRA/Setting+Properties+and+Options+on+Startup 

-Dmail.mime.multipart.ignoreexistingboundaryparameter=true

Message-ID: <201409221917002963505@55.blah.xx.yy>
Content-Type: multipart/alternative;
	boundary="------=_001_NextPart274015426730_=----"
MIME-Version: 1.0

...
Overall, the body of a multipart entity may be specified as follows:
multipart-body := preamble 1*encapsulation 
               close-delimiter epilogue 

encapsulation := delimiter CRLF body-part 

delimiter := CRLF "--" boundary   ; taken from  Content-Type 
field. 
                               ;   when   content-type    is 
multipart 
                             ; There must be no space 
                             ; between "--" and boundary. 

close-delimiter := delimiter "--" ; Again, no  space  before 
"--" 

--xxxx=_001_NextPart274015426730_=xxxx

Data truncation: Data too long for column 'TO_ADDRESS' at row 1

Symptom

2012-08-10 09:35:23,042 ERROR jira ticket import QuartzWorker-0 ServiceRunner jira ticket import jira ticket import10310: Exception: null
java.lang.reflect.UndeclaredThrowableException
at $Proxy1655.save(Unknown Source)
at com.javahollic.jira.emh.service.EnterpriseMessageHandlerImpl.handleMessage(EnterpriseMessageHandlerImpl.java:316)
at com.javahollic.jira.emh.service.EnterpriseMessageHandlerProxy.handleMessage(EnterpriseMessageHandlerProxy.java:43)
at com.atlassian.jira.service.services.mail.MailFetcherService$1.process(MailFetcherService.java:361)
at com.atlassian.jira.service.services.mail.MailFetcherService$MessageProviderImpl.getAndProcessMail(MailFetcherService.java:264)
at com.atlassian.jira.service.services.mail.MailFetcherService.runImpl(MailFetcherService.java:349)
at com.atlassian.jira.service.services.file.AbstractMessageHandlingService.run(AbstractMessageHandlingService.java:250)
at com.atlassian.jira.service.JiraServiceContainerImpl.run(JiraServiceContainerImpl.java:61)
at com.atlassian.jira.service.ServiceRunner.execute(ServiceRunner.java:47)
at org.quartz.core.JobRunShell.run(JobRunShell.java:195)
at com.atlassian.multitenant.quartz.MultiTenantThreadPool$MultiTenantRunnable.run(MultiTenantThreadPool.java:72)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:520)
Caused by: com.mysql.jdbc.MysqlDataTruncation: Data truncation: Data too long for column 'TO_ADDRESS' at row 1

Cause

This error occurs if the addressee list is greater than 255 characters. This has been fixed in current release (1.2.36+) however, the update will not occur in the background and a manual change is required.

Resolution

Option 1

1. On the Audit History screen, hit the Delete All Events button
2. Uninstall the JEMH plugin (no configuration will be affected)
3. Get a database administrator to drop the following tables
   • AO_78C957_AUDITHINT
   • AO_78C957_AUDITEVENTS
4. Install JEMH 1.2.41+, the tables will now be automatically created with types appropriate for your database.
5. Trigger a loading of the JEMH plugin (and creation of missing tables) through accessing the Audit History screen
6. Verify that the AO_78C957_AUDITEVENTS table CC_ADDRESS column is of type 'TEXT'/unlimited characters.
7. Run another Test Case with a long addressee list should now be processed as expected, eg:

   MIME-Version: 1.0
   Received: by 10.223.112.12 with HTTP; Sat, 18 Jun 2011 22:42:26 -0700 (PDT)
   Date: Sun, 19 Jun 2011 17:42:26 +1200
   Message-ID: <BANLkTinB1mfSh+GwOXGNWoL4SyDvOpdBoQ@mail.gmail.com>
   Subject: This is a starting email template, update as required
   From: "Andy Brook" <andy@localhost>
   To: notest@localhost, aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa@place.com, bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb@place.com, cccccccccccccccccccccccccccccccccccccccccccccc@place.com, dddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd@place.com, eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee@place.com, fffffffffffffffffffffffffffffffffffffffffffffffffffffff@place.com
   Content-Type: text/plain; charset=UTF-8

   some text

Option 2

1. With knowledge of your database, it is also possible to update the datatype of TO_ADDRESS column of the AO_78C957_AUDITEVENTS table, updating from a limited 255character type to an unlimited varchar/text type as appropriate (updating the CC_ADDRESS and EMAIL_SUBJECT as well, to fix them too).

Mysql

alter table AO_78C957_AUDITEVENTS change TO_ADDRESS TO_ADDRESS text;
alter table AO_78C957_AUDITEVENTS change CC_ADDRESS CC_ADDRESS text;
alter table AO_78C957_AUDITEVENTS change EMAIL_SUBJECT EMAIL_SUBJECT text;

Postgres

alter table "AO_78C957_AUDITEVENTS" alter column "TO_ADDRESS" type text;
alter table "AO_78C957_AUDITEVENTS" alter column "CC_ADDRESS" type text;
alter table "AO_78C957_AUDITEVENTS" alter column "EMAIL_SUBJECT" type text;

SQLServer

alter table AO_78C957_AUDITEVENTS alter column TO_ADDRESS text;
alter table AO_78C957_AUDITEVENTS alter column CC_ADDRESS text;
alter table AO_78C957_AUDITEVENTS alter column EMAIL_SUBJECT text;

Option 3

1. Ignore the problem. As of 1.2.41, data in these fields will be shortened to fit, some addressee information will be lost as a result.

Notification history update problem:  ERROR: value too long for type character varying(255)

Similar to the last issue, older instances that had the schema created will have char(255) as a column type which isn't enough for some situations.  columns again need converting to text types;

• AO_78C957_NOTIFICATION_HIST.JIRA_ADDRESSEE_USERS
• AO_78C957_NOTIFICATION_HIST.NON_JIRA_ADDRESSEE_USERS
• AO_78C957_NOTIFICATION_HIST.SUBJECT

Routes to accomplish this are also the same as above, either:

1. Option1: Uninstall JEMH, Drop the AO_78C957_NOTIFICATION_HIST table, reinstall JEMH (will recreate tables)
2. Option 2: Make the direct SQL calls appropriate to change the above column types to text.

MYSQL exception - Caused by: java.sql.SQLException: Incorrect string value: '\xD9\x8A\xD8\xB1\xD8\xAC...' for column 'EXCEPTION' at row 1

Cause

Symptoms

• JEMH fails to process email, possibly preventing other emails from being processed

Caused by: java.sql.SQLException: Incorrect string value: '\xD9\x8A\xD8\xB1\xD8\xAC...' for column 'EXCEPTION' at row 1

Resolutions

1. See Handling UTF-8 multi-byte characters with a MySQL database for some steps that can be taken to work around this
2. Migrate to a database that is known to fully support 4-byte characters out-of-the-box, like PostgreSQL
3. As of MySQL 5.7, it may be possible to use the utf8mb4 charset with JIRA 7.3.  We cannot personally vouch for this and so caution is advised: https://confluence.atlassian.com/jirakb/jira-does-not-work-with-emoji-4-byte-characters-429919955.html
4. Convert the column being complained about to UTF-8. For example, updating the AUDITEVENTS table EXCEPTION column to UTF-8 in the JIRA database would be done with:

ALTER TABLE `jira`.`AO_78C957_AUDITEVENTS` MODIFY COLUMN `EXCEPTION` TEXT  CHARACTER SET utf8 COLLATE utf8_general_ci DEFAULT NULL;

Configuration screen won't save updates

Cause

The use of these fields exceeded expectation, most of the text entry fields had at one point a 255 character limit set, e.g. blacklist senders regular expression. If a save was attempted that exceeded this limit, the save failed. Unfortunately, whilst the update failed, the configuration entity object that contained this value got 'left' with the larger value, meaning that any future changes would not work until that previous field value was fixed.

Current builds now have marked the specific fields commonly causing this problem as unlimited length, but unfortunately, such changes are not yet reflected automatically into a pre-existing installation.

Workaround

Locate the text field previously edited, look for a value that is greater than 255 characters, fix it so its less than that. The save, and future changes should now work.

Resolution

The schema needs to be updated. As the AO layer used to access the database doesn't actually reflect UNLIMITED length if the schema already exists, a simple tweak is to change datatype on a per-column basis, given the change is from a 255 character field to unlimited, no data will be lost.

With no data changes going on, this change should be transparent to JIRA, previously failing saves should now work.

Depending on your revision there are two tables that have caused problems:

• AO_78C957_AUDITEVENTS, containing audit history, large addressee lists, this is listed above (Data truncation: Data too long for column)
• AO_78C957_CONFIG, contains core configuration fields

AO_78C957_CONFIG fields

• BLACKLIST_SENDERS
• BLACKLIST_RECIPIENTS
• BLACKLISTED_ATTACHMENT_TYPES
• BODY_DELIMITER_REGEXPS
• CREATE_USERS_EMAIL_BLACKLIST
• CREATE_USERS_EMAIL_WHITELIST
• CATCH_EMAIL_ADDRESS
• GREYLIST_SUBJECT_REGEXPS
• GREYLIST_SENDERS
• GREYLIST_EMAIL_BODY_REGEXP
• JEMH_ADDRESSEE_REGEXPS
• SUBJECT_CLEANUP_REGEXPS
• WHITELIST_SENDERS

Mysql Script

for a Schema called jira514 (set it to the default in tools like emma), update this to reflect your deployment:

alter table AO_78C957_AUDITEVENTS change TO_ADDRESS TO_ADDRESS text;
alter table AO_78C957_AUDITEVENTS change CC_ADDRESS CC_ADDRESS text;
alter table AO_78C957_AUDITEVENTS change EMAIL_SUBJECT EMAIL_SUBJECT text;

alter table AO_78C957_CONFIG change BLACKLISTED_ATTACHMENT_TYPES BLACKLISTED_ATTACHMENT_TYPES text;
alter table AO_78C957_CONFIG change BLACKLIST_SENDERS BLACKLIST_SENDERS text;
alter table AO_78C957_CONFIG change BLACKLIST_RECIPIENTS BLACKLIST_RECIPIENTS text;
alter table AO_78C957_CONFIG change BODY_DELIMITER_REGEXPS BODY_DELIMITER_REGEXPS text;
alter table AO_78C957_CONFIG change CREATE_USERS_EMAIL_BLACKLIST CREATE_USERS_EMAIL_BLACKLIST text;
alter table AO_78C957_CONFIG change CREATE_USERS_EMAIL_WHITELIST CREATE_USERS_EMAIL_WHITELIST text;
alter table AO_78C957_CONFIG change CATCH_EMAIL_ADDRESS CATCH_EMAIL_ADDRESS text;
alter table AO_78C957_CONFIG change GREYLIST_SUBJECT_REGEXPS GREYLIST_SUBJECT_REGEXPS text;
alter table AO_78C957_CONFIG change GREYLIST_SENDERS GREYLIST_SENDERS text;
alter table AO_78C957_CONFIG change GREYLIST_EMAIL_BODY_REGEXP GREYLIST_EMAIL_BODY_REGEXP text;
alter table AO_78C957_CONFIG change JEMH_ADDRESSEE_REGEXPS JEMH_ADDRESSEE_REGEXPS text;
alter table AO_78C957_CONFIG change SUBJECT_CLEANUP_REGEXPS SUBJECT_CLEANUP_REGEXPS text;
alter table AO_78C957_CONFIG change WHITELIST_SENDERS WHITELIST_SENDERS text;
commit;

Postgresql

Postgres likes to lower case everything, you'll need to quote the table and columns that AO has created.

alter table "AO_78C957_CONFIG" alter column "TO_ADDRESS" type text;
alter table "AO_78C957_CONFIG" alter column "CC_ADDRESS" type text;
alter table "AO_78C957_CONFIG" alter column "EMAIL_SUBJECT" type text;

alter table "AO_78C957_CONFIG" alter column "BLACKLIST_SENDERS" type text;
alter table "AO_78C957_CONFIG" alter column "BLACKLIST_RECIPIENTS" type text;
alter table "AO_78C957_CONFIG" alter column "BLACKLISTED_ATTACHMENT_TYPES" type text;
alter table "AO_78C957_CONFIG" alter column "BODY_DELIMITER_REGEXPS" type text;
alter table "AO_78C957_CONFIG" alter column "CREATE_USERS_EMAIL_BLACKLIST" type text;
alter table "AO_78C957_CONFIG" alter column "CREATE_USERS_EMAIL_WHITELIST" type text;
alter table "AO_78C957_CONFIG" alter column "CATCH_EMAIL_ADDRESS" type text;
alter table "AO_78C957_CONFIG" alter column "GREYLIST_SUBJECT_REGEXPS" type text;
alter table "AO_78C957_CONFIG" alter column "GREYLIST_SENDERS" type text;
alter table "AO_78C957_CONFIG" alter column "GREYLIST_EMAIL_BODY_REGEXP" type text;
alter table "AO_78C957_CONFIG" alter column "JEMH_ADDRESSEE_REGEXPS" type text;
alter table "AO_78C957_CONFIG" alter column "SUBJECT_CLEANUP_REGEXPS" type text;
alter table "AO_78C957_CONFIG" alter column "WHITELIST_SENDERS" type text;
commit;

SqlServer

alter table AO_78C957_AUDITEVENTS alter column TO_ADDRESS text;
alter table AO_78C957_AUDITEVENTS alter column CC_ADDRESS text;
alter table AO_78C957_AUDITEVENTS alter column EMAIL_SUBJECT text;

alter table AO_78C957_CONFIG alter column BLACKLIST_SENDERS text;
alter table AO_78C957_CONFIG alter column BLACKLIST_RECIPIENTS text;
alter table AO_78C957_CONFIG alter column BLACKLISTED_ATTACHMENT_TYPES text;
alter table AO_78C957_CONFIG alter column BODY_DELIMITER_REGEXPS text;
alter table AO_78C957_CONFIG alter column CREATE_USERS_EMAIL_BLACKLIST text;
alter table AO_78C957_CONFIG alter column CREATE_USERS_EMAIL_WHITELIST text;
alter table AO_78C957_CONFIG alter column CATCH_EMAIL_ADDRESS text;
alter table AO_78C957_CONFIG alter column GREYLIST_SUBJECT_REGEXPS text;
alter table AO_78C957_CONFIG alter column GREYLIST_SENDERS text;
alter table AO_78C957_CONFIG alter column GREYLIST_EMAIL_BODY_REGEXP text;
alter table AO_78C957_CONFIG alter column JEMH_ADDRESSEE_REGEXPS" text;
alter table AO_78C957_CONFIG alter column SUBJECT_CLEANUP_REGEXPS text;
alter table AO_78C957_CONFIG alter column WHITELIST_SENDERS text;
commit;

Audit History is too large

If the inbound mail volume is significant, history can build up quickly. The retention period can introduce significant delays..

This is being tracked at [JEMH-1067]

Interim solution is to delete the history and set a lower threshhold for retention.  Drop tables when JIRA is DOWN so that they are automatically recreated when JIRA comes UP.

1. Delete Audit Data

   SQL	File Data
   Example for Postgresql, other databases should be similar. DELETE FROM "AO_78C957_AUDITMETA_DOMAIN"; DELETE FROM "AO_78C957_AUDITMETA_PROJECT"; DELETE FROM "AO_78C957_AUDITMETA"; DELETE FROM "AO_78C957_AUDITHINT"; DELETE FROM "AO_78C957_AUDITEVENTS";	Delete the folder JIRA_HOME/data/jemh/auditmail

2. Reset the retention period to 1 day
   

Cannot install license: name is already used by an existing object

This can happen when a previous installation has been wiped (all AO_78C957_* tables) and the JIRA_HOME/jemh folder removed, but licenses still fail due to old sequences being present. All these should be removed too, then a uninstall/reinstall cycle of the plugin should kick everything back into being created. Eg, see https://studio.plugins.atlassian.com/browse/JEMH-1075

Comment Headers have nulls in

If you see strange template output as a comment header, this is an indication that the export/import of the template, which kept the ID's of the templates used on serverA have matched a template on serverB (but is most likely not to be the same template. Migrate templates (1.3+) and edit configuration to match the correct template. see https://studio.plugins.atlassian.com/browse/JEMH-1128 https://studio.plugins.atlassian.com/browse/JEMH-1128

null created null:
-------------

             Summary: $issue.summary
                 Key: ${issue.getKey()}
                 URL: http://floater:8080/browse/${issue.getKey()}
             Project: $issue.getProject().getString("name")
          Issue Type: $issue.getIssueTypeObject().getNameTranslation($i18n) 

Test Cases / Blacklisting image previews / Auditing files are absent after a JIRA backup/restore to another server

JEMH configuration and audit meta data is all stored in the database using Active Objects so as to be included in the XML backup. JEMH also has many resources on-disk that are not backed up and will not be restored, these resources are:

• Test Cases
• Blacklisting (uploaded) preview files
• Auditing supporting emails.

Migration of Test Cases can only be done one by one currently (@1.3), requiring a new Test Case to be created, and a specific email file picked through the file browser button available during edit.

Blacklisting files can be manipulated to appear in the right place, however care must be taken, as getting the filename ID wrong, will be confusing to the user as the wrong image may then be associated with a description /hash.

Audit history cannot be migrated, is not suitable for long term reference and is meant to provide transitory reference only. Best option is to hit Auditing > Delete ALL to flush all database data to start from blank.

Non-latin character support on Microsoft SqlServer

Unfortunately there is a bug in the Active Objects layer that JEMH uses to store data (https://ecosystem.atlassian.net/browse/AO-386).  Effectively, this means that non-latin character support using SQLServer is not possible.  Symptoms of this will be an inability to select correctly displayed non-latin text in priorities/issue types etc, but on save, the non-latin characters are replaced with (from evidence so far) latin equivalents.  This is slated for being fixed in JIRA 7, however the process for apply this is not automatic.  Atlassian (as per AO-386) have advised that a full XML export and import into a JIRA 7 instance is the way to convert the data.

Watcher Custom Field for JIRA Plugin

The Watcher Custom Field for JIRA plugin is incompatible with JEMH, some customers have reported various issues caused by the behaviour of this plugin due to the impact it has on JIRA issue validation and therefore JEMH issue creation. 

Possible workaround:

When using this plugin, do not name the custom field created by the plugin "watchers" as this will cause incompatibilities with JEMH's built in watcher behaviour. Customers have found that naming the custom field with a name other than "watchers" prevents JEMH incompatibilities but your mileage may vary. 

IOException/Mail stuck after adding and then immediately deleting an attachment

While waiting for a notification that contains an attachment to be sent, if you delete any of the attachments included in the notification before it is sent will result in the notification being lost (not sent) and will result in an IOException message being displayed in the atlassian-jira-outgoing-mail.log. This includes the scenario of having Require Comment to Notify enabled in the Event Listener, if any of the attachments in the notification are deleted before the notification is sent, this will result in the loss of the entire notification.

Error 500 - Filter execution threw an exception after running a test case.

If you receive an error 500 message after running a test case to add a comment on an issue in a JSD project and you have Adaptavist ScriptRunner for JIRA installed, ensure that there are no errors in any of the script fields that you you have configured as this has been found to cause this problem.

Blacklist attachments missing after JEMH upgrade

In the event of blacklist attachment entries being missing after upgrading JEMH, the database upgrade scripts might have not ran correctly and running the following commands against your database will restore the entries.

List all blacklisting entries

The missing entries will have the column BLACKLIST_TYPE set as NULL.

SELECT * FROM "AO_78C957_BLACKLISTING";

Reset subject blacklisting types

UPDATE "AO_78C957_BLACKLISTING" SET "BLACKLIST_TYPE"='subject' WHERE "SUBJECT_REGEXP" IS NOT NULL;

Reset attachment blacklisting types

UPDATE "AO_78C957_BLACKLISTING" SET "BLACKLIST_TYPE"='fingerprint' WHERE "ATTACHMENT_HASH" IS NOT NULL;

Verify no untyped entries remain
All entries will have either subject or fingerprint set in the BLACKLIST_TYPE column.

SELECT * FROM "AO_78C957_BLACKLISTING" WHERE "BLACKLIST_TYPE" IS NULL;

Email client shows encoded attachment name instead of decoded file name

JIRA 7 and newer uses a version of JavaMail (a framework for email reading/writing) which has changed its default handling of encoded email parameters to conform with RFC 2231.  Here is an example showing the same attachment, sent using JavaMail in JIRA 6.4 and 7.0 respectively:

Content-Type: text/plain 
Content-Transfer-Encoding: 7bit 
Content-ID: <55228d63-8105-49eb-a841-57e812d6a25c> 
Content-Disposition: attachment;  
    filename="=?UTF-8?B?4pqQ4pqQ4pqQ4pqQ4pqQ4pqQ4pqQ4pqQ?= 
 =?UTF-8?B?4pqQ4pqQ4pqQ4pqQ4pqQ4pqQ4pqQ4pqQ4pqQ?= 
 =?UTF-8?B?4pqQ4pqQ4pqQ4pqQ4pqQ4pqQ4pqQ4pqQ4pqQ4pqQ4pqQ4pqQ4pqQLnR4dA==?="

Content-Type: text/plain 
Content-Transfer-Encoding: 7bit 
Content-ID: <c8890c7d-e24a-407b-a94a-fa1127eb5cb9> 
Content-Disposition: attachment;  
    filename*0="=?UTF-8?B?4pqQ4pqQ4pqQ4pqQ4pqQ4pqQ4pqQ4pqQ?= 
 =?UTF-8?B?4pqQ4";  
    filename*1="pqQ4pqQ4pqQ4pqQ4pqQ4pqQ4pqQ4pqQ?= 
 =?UTF-8?B?4pqQ4pqQ4pqQ4pqQ";  
    filename*2="4pqQ4pqQ4pqQ4pqQ4pqQ4pqQ4pqQ4pqQ4pqQLnR4dA==?="

Some email clients (including Gmail and Outlook) unfortunately do not conform to RFC 2231.  This means that when JEMH sends attachments in emails that have particularly long names including non-ASCII characters, they are not shown and instead the encoded string is displayed instead.

This problem can be worked around by setting the below Java property on JIRA startup in order to disable JavaMail's incompatible (but technically correct) encoding of parameters.

mail.mime.encodeparameters=false

JSD Organization field

JSD features an Organization field that allows customers who are grouped into an organization to view support tickets raised by other users who are apart of the same organization (as long as the field has been set), more information on the field can be found here: https://confluence.atlassian.com/servicedeskcloud/blog/2016/09/group-customers-in-organizations.

Currently in JEMH it is only possible to set this field with custom field defaults and directives by using the Organizations ID and not the organization name. Using the organization name will result in JEMH not being able to process the email.

The Organization ID can be obtained by navigating to the Customers page within a JSD project and then hovering the mouse over your desired organization. The Organization ID is the number at the end of the URL preview.

Currently notifications can not be sent out to members of the organization if they are not actively involved with the issue.

We currently have an Improvement ticket to better support the Organization field, the ticket can be found here: JEMH-4989 - Getting issue details... STATUS .

One method that could be used to associate an email domain with a JSD Organization is to use a Project Mapping Domain Mapping Rule.

The Domain Mapping Rule can be set up to match on a certain domain of inbound mail with a regexp along the lines of the following: .*@myco.co.uk

Then a custom field default can be set meaning all emails from a certain domain will have the Organization X set on all issues (in this example all issues from @myco.co.uk will have the Organization Myco).

However this method would be limited to having a single service desk per JEMH profile.

Upgrade Problems

In Manage Add-ons the Update button does not work and the add-on details cannot be expanded

Version 2.21 of Atlassians Universal Plugin Manager (UPM) broke the ability to update JEMH, or expand the add-ons details on the "Manage Add-ons" page of JIRA.  Atlassian were made aware of the situation and plan to fix in 2.21.2 (as of 20 July 2016 yet to be released).  If you are using this version of UPM, please see JEMH-4748 for details of the workaround prior to the fixed version of UPM being released.

JIRA installer & ClassNotFoundException for ContextFactory?

If you see [java.lang.ClassNotFoundException: com.sun.xml.internal.bind.v2.ContextFactory not found from bundle com.javahollic.jira.jemh-ui] please read https://jira.atlassian.com/browse/JRA-38946 and https://jira.atlassian.com/browse/BAM-12484 adding the referred jaxb-impl-2.1.12.jar to your webapp/WEB-INF/lib folder, and reboot.

Why isn't the latest version visible in JIRA update manager?

There are two possibilities, firstly, the latest release may not be compatible with your JIRA version, and so an upgrade of JIRA will be required.  The other possibility is that the UPM plugin is outdated, sometimes this can prevent plugins from being listed.

Upgrade from 1.2.79 to 1.3 fails on mysql

This problem came about with a Table being created before release, changes made before release resulted in an incompatibility that cannot be resolved by automated means.  The following manual resolutions can be used:

Error Message:

mysql> ALTER TABLE AO_78C957_CONFIG ADD COLUMN EMAIL_FILE_TYPE VARCHAR(255);
ERROR 1118 (42000): Row size too large. The maximum row size for the used table type, not counting BLOBs, is 65535. You have to change some columns to TEXT or BLOBs

ALTER TABLE AO_78C957_CONFIG MODIFY HTML_BLKQT_TAG_HANDLING VARCHAR(64);
ALTER TABLE AO_78C957_CONFIG MODIFY HTML_BOLD_TAG_HANDLING VARCHAR(64);
ALTER TABLE AO_78C957_CONFIG MODIFY HTML_BREAK_TAG_HANDLING VARCHAR(64);
ALTER TABLE AO_78C957_CONFIG MODIFY HTML_CODE_TAG_HANDLING VARCHAR(64);
ALTER TABLE AO_78C957_CONFIG MODIFY HTML_DIV_TAG_HANDLING VARCHAR(64);
ALTER TABLE AO_78C957_CONFIG MODIFY HTML_HEADING_LEVEL VARCHAR(64);
ALTER TABLE AO_78C957_CONFIG MODIFY HTML_HR_TAG_HANDLING VARCHAR(64);
ALTER TABLE AO_78C957_CONFIG MODIFY HTML_IMAGE_HANDLING VARCHAR(64);
ALTER TABLE AO_78C957_CONFIG MODIFY HTML_IMG_TAG_HANDLING VARCHAR(64);
ALTER TABLE AO_78C957_CONFIG MODIFY HTML_ITALIC_TAG_HANDLING VARCHAR(64);
ALTER TABLE AO_78C957_CONFIG MODIFY HTML_LINK_HANDLING VARCHAR(64);
ALTER TABLE AO_78C957_CONFIG MODIFY HTML_PARA_TAG_HANDLING VARCHAR(64);
ALTER TABLE AO_78C957_CONFIG MODIFY HTML_PRE_TAG_HANDLING VARCHAR(64);
ALTER TABLE AO_78C957_CONFIG MODIFY HTML_SPAN_TAG_HANDLING VARCHAR(64);
ALTER TABLE AO_78C957_CONFIG MODIFY HTML_STRIKE_TAG_HANDLING VARCHAR(64);
ALTER TABLE AO_78C957_CONFIG MODIFY HTML_SUB_TAG_HANDLING VARCHAR(64);
ALTER TABLE AO_78C957_CONFIG MODIFY HTML_SUP_TAG_HANDLING VARCHAR(64);
ALTER TABLE AO_78C957_CONFIG MODIFY HTML_TABLE_RENDERING VARCHAR(64);
ALTER TABLE AO_78C957_CONFIG MODIFY HTML_TABLE_TAG_HANDLING VARCHAR(64);
ALTER TABLE AO_78C957_CONFIG MODIFY HTML_TEXT_COLOUR_HANDLING VARCHAR(64);
ALTER TABLE AO_78C957_CONFIG MODIFY HTML_UL_TAG_HANDLING VARCHAR(64);
ALTER TABLE AO_78C957_CONFIG MODIFY HTML_UNDERLINE_TAG_HANDLING VARCHAR(64);

2014-04-05 17:10:14,175 QuartzScheduler_Worker-1 ERROR ServiceRunner     [org.quartz.core.ErrorLogger] Job (DEFAULT.ServicesJob threw an exception.
org.quartz.SchedulerException: Job threw an unhandled exception. [See nested exception: java.lang.OutOfMemoryError: Java heap space]
	at org.quartz.core.JobRunShell.run(JobRunShell.java:206)
	at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:520)
* Nested Exception (Underlying Cause) ---------------
java.lang.OutOfMemoryError: Java heap space
	at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:133)
	at java.io.OutputStreamWriter.write(OutputStreamWriter.java:220)
	at java.io.Writer.write(Writer.java:157)
	at org.apache.log4j.helpers.CountingQuietWriter.write(CountingQuietWriter.java:45)
	at org.apache.log4j.WriterAppender.subAppend(WriterAppender.java:310)
	at org.apache.log4j.RollingFileAppender.subAppend(RollingFileAppender.java:276)
	at org.apache.log4j.WriterAppender.append(WriterAppender.java:162)
	at org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
	at com.atlassian.jira.logging.JiraHomeAppender.doAppend(JiraHomeAppender.java:243)
	at org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
	at org.apache.log4j.Category.callAppenders(Category.java:206)
	at org.apache.log4j.Category.forcedLog(Category.java:391)
	at org.apache.log4j.Category.log(Category.java:838)
	at com.atlassian.jira.web.action.admin.mail.LogPrintStream.flush(LogPrintStream.java:29)
	at com.atlassian.jira.web.action.admin.mail.LogPrintStream.write(LogPrintStream.java:118)
	at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
	at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
	at sun.nio.cs.StreamEncoder.flushBuffer(StreamEncoder.java:104)
	at java.io.OutputStreamWriter.flushBuffer(OutputStreamWriter.java:185)
	at java.io.PrintStream.write(PrintStream.java:527)
	at java.io.PrintStream.print(PrintStream.java:669)
	at java.io.PrintStream.println(PrintStream.java:806)
	at com.atlassian.jira.web.action.admin.mail.LogPrintStream.println(LogPrintStream.java:111)
	at com.sun.mail.pop3.Protocol.issueCommand(Protocol.java:701)
	at com.sun.mail.pop3.Protocol.retr(Protocol.java:447)
	at com.sun.mail.pop3.POP3Message.getRawStream(POP3Message.java:175)
	at com.sun.mail.pop3.POP3Message.getContentStream(POP3Message.java:271)
	at javax.mail.internet.MimePartDataSource.getInputStream(MimePartDataSource.java:99)
	at javax.mail.internet.MimeMultipart.parsebm(MimeMultipart.java:785)
	at javax.mail.internet.MimeMultipart.parse(MimeMultipart.java:503)
	at javax.mail.internet.MimeMultipart.getCount(MimeMultipart.java:244)
	at com.javahollic.jira.emh.engine.support.EMHMailUtils.getBodyImpl(EMHMailUtils.java:1167)

JIRA Software Problems

A JIRA user without right to use can no longer create issues

The JIRA Software application has made some additional requirements relating to issue created by email:

• https://confluence.atlassian.com/jirasoftware/jira-software-7-0-0-release-notes-776997721.html

Changes to issue creation via email

In previous versions of JIRA, users without the JIRA Users Global Permission were not able to log in to JIRA, but if they belonged to a group that had Create Issue permission for a project, they could create issues via email if a JIRA email handler was set up. 

In JIRA Software, you must have application access to the JIRA instance in order to create issues, regardless of how the issue is created. This essentially means that:

• you won't be able to create issues via a JIRA email handler if you don't have application access, and
• the JIRA email handler has not been set up to create a new user account with application access for you.

We have made this change to introduce more consistency in how licenses are enforced in JIRA applications.

This does directly impact JEMH, if you were using email accounts registered to JIRA accounts that did not have "right to use".  In JIRA 7, those users will be unable to continue creating issues as they did previously.

Options

1. Start automatically allocating the access group involved via JEMH Profile > User > Auto-Join Groups  and  JEMH Profile > User > Auto Join Group Condition to users who interact by email, obviously this will increase license usage for JIRA Software, but will expose all the great features of JIRA Software to those users.
2. JEMH-3960 enabling the default reporter to be the security context for the issue creation.

JIRA Service Desk Problems

Cannot set multiple Organizations through Custom Field Defaults or Directives

Organization support (translation of names to id) was added in 2.3.6

Currently JSD Server API cannot process updates from the Jira IssueService (IssueInputParameters) where multiple Organization ID's are supplied.

• https://jira.atlassian.com/browse/JSDSERVER-5483 

Using JIRA Service Desk 3.3.x customers get multiple notifications for same event

JIRA Service Desk 3.3.x changes the way that it handles customer notifications.  The result is that it is no longer a one-click operation in order to disable the JSD notification system and have JEMH as the only notification source.  Each project now has a customer notifications section, where you will need to disable each of the rules present in order to stop them from being sent.  Please see JEMH-5253 - Getting issue details... STATUS for more details on how to do this.

Cannot approve an issue via email directive

JIRA Service Desk 3.2.0 introduced a new approval feature to workflows.  Currently, the only way that users can approve issues is via the issue view or portal view.  Please watch JEMH-5010 - Getting issue details... STATUS for future updates on this matter. Take a look at Use Directive Sets#JSD-Approvals for more information regarding Directive Set Setup for this feature.

JSD entity properties could not be set: Comment Entity could not be set: [sd.public.comment = {"internal":false}] : You do not have the permission for this comment.

When a customer user comments via email, the comment may fail to be made public and you may receive one of the following hints:

First scenario:

During resetCustomerCommentsPublic, JSD entity properties could not be set: Comment Entity could not be set: [sd.public.comment = {"internal":false}] : You do not have the permission for this comment.

This usually occurs because the Customer User does not have the "Edit Own Comments" project permission.  Granting this to the user should prevent further occurrences.

Second scenario:

Unable to set JEMH Comment Entity Property on [issueKey] comment 10215 by customer : You do not have the permission for this comment.

This also may occur if the issue is configured to have a Security Level, but the customer who comments does not belong to the Security Level directly or via the group. To solve this, the user needs to be added to the Security Level.

When Agents use Share with Customer, customers are not notified

 JSD does not order its operations correctly, the property that identifies whether a comment is Public or Private is not updated prior to the ISSUE_UPDATED event being fired, so JEMH has no option but to consider it exactly as it is presented, a Private comment,which should not be communicated to Request Participants (Customers).  See JSD-3533 on Atlassian's issue tracker. 

Issues created by JEMH are not visible in the Customer Portal

In order for issues to be visible in the JSD customer portal, they must have a value set for the Customer Request Type field.  See below for more information on locating and setting a value.

Locating Customer Request Type

Prior to JSD 3.1 it was possible to locate the value that JEMH needed to set for the Customer Request Type custom field, by enabling the field, and examining the REST response.  In 3.1, that is no longer possible ( https://jira.atlassian.com/browse/JSD-3471), as a result, there are only two ways to locate the required value

The format of a Customer Request Type that JEMH needs to set in a Profile > Custom Field Default or Project Mapping is currently xx/yyyyyyyy where xx is the lower cased project key for the related service desk, and yyyyyyyy is the lower case (currently non hyphenated) key of the RequestType, see example below:

JEMH Log Output

From JEMH 1.9.3, and with the latest JSD 3.1 series installed, JEMH will make an attempt to log out the JSD Customer Request Type keys when accessing the JEMH Profile Configuration screen, see Enabling JEMH logging, it will look like this:

2016-03-04 16:11:03,729 http-nio-8080-exec-1 DEBUG [emh.ui.action.JEMHConfiguration] JSD is available
2016-03-04 ... RequestType found: id=15, portalId=2, issueTypeId=10000, iconId=38, key=systemproblem, name=Report a system problem, desc=Having trouble with a system?, help=
2016-03-04 ... RequestType found: id=16, portalId=2, issueTypeId=10001, iconId=27, key=getithelp, name=Get IT help, desc=Get assistance for general IT problems and questions., help=
2016-03-04 ... RequestType found: id=17, portalId=2, issueTypeId=10001, iconId=20, key=accountproblem, name=Fix an account problem, desc=Having trouble accessing certain web sites or systems? We'll help you out., help=
2016-03-04 ... RequestType found: id=18, portalId=2, issueTypeId=10001, iconId=28, key=guestwifi, name=Get a guest wifi account, desc=Raise a request to ask for temp wifi access for guests., help=
2016-03-04 ... RequestType found: id=19, portalId=2, issueTypeId=10001, iconId=34, key=vpn, name=Set up VPN to the office, desc=Want to access work stuff from outside? Let us know., help=
2016-03-04 ... RequestType found: id=20, portalId=2, issueTypeId=10001, iconId=25, key=adminaccess, name=Request admin access, desc=For example, help=
2016-03-04 ... RequestType found: id=21, portalId=2, issueTypeId=10001, iconId=31, key=newaccount, name=Request a new account, desc=Request a new account for a system., help=
2016-03-04 ... RequestType found: id=22, portalId=2, issueTypeId=10001, iconId=4, key=newhires, name=Onboard new hires, desc=Request access for new employees., help=
2016-03-04 ... RequestType found: id=23, portalId=2, issueTypeId=10001, iconId=10, key=compsupport, name=Desktop/Laptop support, desc=If you are having computer problems, help=
2016-03-04 ... RequestType found: id=24, portalId=2, issueTypeId=10001, iconId=21, key=phoneredirect, name=Set up a phone line redirect, desc=Request a redirect of our phone systems for a specific date and time., help=
2016-03-04 ... RequestType found: id=25, portalId=2, issueTypeId=10001, iconId=37, key=newsoftware, name=Request new software, desc=If you need a software license, help=
2016-03-04 ... RequestType found: id=26, portalId=2, issueTypeId=10001, iconId=18, key=newhardware, name=Request new hardware, desc=For example, help=
2016-03-04 ... RequestType found: id=27, portalId=2, issueTypeId=10002, iconId=16, key=upgradeserver, name=Upgrade or change a server, desc=For example, help=
2016-03-04 ... RequestType found: id=28, portalId=2, issueTypeId=10002, iconId=23, key=upgradesystem, name=Upgrade or change a managed system, desc=For example, help=

Database Query

1. Find the issue ID of an existing issue which has the request type you are looking for set. Here are two ways to find this id:
   
   • View an existing issue and hover the cursor over the "Edit" action button.  Your browser should show the issue ID in the hyperlink URL

   • Go to the REST API endpoint for an existing issue (e.g. http://localhost:8080/rest/api/2/issue/SD-13). The issue ID number can be found on line 2:

     {expand: "renderedFields,names,schema,transitions,operations,editmeta,changelog,versionedRepresentations",
     id: "10501",
     self: "http://localhost:8080/rest/api/2/issue/10501",
     key: "SD-13"

2. Use an SQL query on your JIRA database to list the custom field values for the issue.
   

   SELECT id, issue, customfield, stringvalue FROM customfieldvalue where issue=10501;

   The query result should contain a field value that matches the format of projectKey/requestType:

   

    The following query can provide a unique list of Customer Request Type values in your system.  Use the target projectkey lower cased to scope the query result:

   select stringvalue FROM "customfieldvalue" where stringvalue LIKE 'key/%' group by "stringvalue";

   
   

3. This value can be automatically used in future by JEMH, through the use of a /wiki/spaces/JEMH/pages/143491123 for Customer Request Type

Public Sign Up is Disabled for my project when I set up JEMH for Service Desk use

Adding "Project Role (Customers)" to the notification scheme will disable the ability to allow anyone to sign up for a customer account:

We recommend that when setting up your environment for JEMH and JSD, that issue permissions are granted through a user group rather than the project role. You will still want to add this group to the customer role however in order for recipient participants to work correctly.

Permission Scheme problems / JEMH does not have permission

When JSD is enabled, it replaces the standard JIRA Authorization checking, this results in the Permissions you think have been allocated are not (https://jira.atlassian.com/browse/JSD-1739)

• Apparently JEMH (default user, and any participants) needs to be specifically listed as a collaborator in current JSD releases https://jira.atlassian.com/browse/JSD-931

• Some help in resolving errors in JSD enabled projects: https://confluence.atlassian.com/display/SERVICEDESK/Resolving+permission+scheme+errors

Service Desk enabled projects appear to lose events

Assigning an issue seems to turn into an ISSUE_UPDATE, resolving and re-opening issues seem to turn into a GENERIC_EVENT, there could be others:

• JSD default/generated workflow, "Resolve this issue" transition is firing Generic Event – https://jira.atlassian.com/browse/JSD-1540

Service Desk enabled projects in 1.7.x generate: You do not have permission to edit issues in this project

Logged as https://jira.atlassian.com/browse/JSD-931

Step 1: Creating Issues

Portal Users need to be specifically given CREATE_ISSUE privilege to get past the JIRA permission check result: The field [pid] could not be set: You do not have permission to create issues in this project. This would best be done by a group that JEMH auto-joins all users to a given group that is listed in the Project Role: Service Desk Customers that is granted CREATE_ISSUE.

Prior to 1.7.19, not having BROWSE_ISSUE resulted in a hint (eg:  User (customer) is not able to BROWSE the issue (is the users GROUP in the Projects Roles?) : DESK-98, ignoring as a watcher.) due to the inability to manage watchers, to make JSD integration 'better', this has been downgraded to an INFO.

Service Desk and Profile defined assignees

If you make use of JEMH Project Mappings, it is a normal capability of JEMH to allow a specific assignee to manage a specific sender domain (for example).  With Service Desk in the mix, it gets complicated as JSD overrides the permission (even if you grant it in the permission scheme), according to the permission checker, then 'sender' needs to have 1) BROWSE 2) EDIT 3) Agent Role.  Now, clearly, its not possible for all users to be Agents.

Right now, profile defined assignees are not compatible with Service Desk, the necessary improvement for this has been logged:  JEMH-3470 - Getting issue details... STATUS

Service Desk internal comment being sent to customer users

There are sporadic circumstances that can mean that when JEMH checks a comment's entity properties to determine if a comment is internal or not, the properties are sometimes not found.  This has been brought to the attention of Atlassian: https://jira.atlassian.com/browse/JSD-3533. Prior to JEMH 1.8, no attempt to use alternative determining mechanisms was made.

Please upgrade to version 1.9.17 if possible, as this will now "fail safe".  If a comment is made by someone in the "Service Desk Team" or "Administrator" project roles, and the entity property check fails, the comment will be considered internal.  The JEMH debug log (see here) contains output that will make it clear what check was done during processing.  Investigation is continuing, but it is not yet confirmed what causes the check to fail.  We have had reports that upgrading JEMH can restore the correct operation of the entity property check, however this does not always appear to be the case.

Request Channel Type not defaulting to Email

JEMH 1.7.x, 1.8.x and 1.9.x as of 16th March 2016 should set the JSD request channel type to "Email" to differentiate from those created via JIRA. If this is not occurring, you need to make sure that a default reporter is set in the profile, and that "treat unprivileged users as non-JIRA is enabled under Profile>Security.

Microsoft Exchange Related Problems

Issue icons are being included from replies (again)

Affecting JIRA 7.0.0+ , JEMH maintains an internal list of hashes for the vast majority of icons, including SVG icons and their transcoded variants.  However.  Outlook mail clients have been known to transcode these images into PNG files which are then non-matches for hashing, which is one reason they appear in issues.  Its not therefore possible to automatically hash these values, options are:

• for you to upload each image you don't want into JEMH Blacklisting (and hope that outlook image conversion is repeatable so hashing works repeatably)
• for you to use a feature added in JEMH 1.8.10, to blacklist attachments by size and optionally mime type.  eg, block all images less than 1K with:   1024 (bytes) and image/.* regexp

Unable to load BODYSTRUCTURE

• https://java.net/projects/javamail/pages/Exchange

No content, (text, html or attachments) other than... winmail.dat

Winmail.dat usually contains additional details for presentation within proprietary email clients (Outlook).  In all cases to date, winmail.dat as filtered wherever found as it adds no value in issues. In a rare scenario, there may be no content other than the winmail.dat file, extraction of that content has not yet been added to JEMH as demand has yet to be proven.  The one cause seen is described by the customer with Exchange 2013, with resolution:

We have one mailbox configured for a particular JEMH profile; the mailbox has several email aliases which are mapped to a specific jira project by addressee. All other email aliases are working as expected - when you attach a file to the email it shows up as an attachment in the jira ticket. The problematic alias was the only one that was added to the Exchange Global Address List. (I'm not exactly sure how our exchange admin did this since it's only an alias and not an actual mailbox). Removing the entry from the GAL fixed the problem however.. attachments are now showing up as normal.

More information regarding this can be found on this Microsoft KnowledgeBase article: https://support.microsoft.com/en-gb/kb/278061.

IMAP folder not being read by mail handler

A customer has had problems with an IMAP folder not being read by the JIRA mail handler.  In order to make it work, they moved the folder to the same level in the hierarchy as the inbox (not under it).

With Exchange, in JIRA there is an error AuthenticationFailedException: EOF on socket.

 This is not a JEMH issue, however, resolution relates to Exchange mailserver process not running:

• https://social.technet.microsoft.com/Forums/exchange/en-US/5f1a2cee-19ad-43e6-b281-bb7f094d8c09/pop-works-via-localhost-but-not-from-other-networked-machines?forum=exchangesvrclients

Microsoft Outlook Related Problems

Attachments not visible in outlook

JEMH generates perfectly valid email content (we hope!), including attachments.  Most web and email clients will just work, Outlook however may require a setting change that alters the overall email content type:

There is an option in JEMH Event Listener, that allows you to select multipart/mixed rather than the default multipart/related as a 'wrapper type' for the email content:

Outlook 2010 clients deliver HTML mail as text/plain

JEMH will obey the content type that an email declares, it must be correct for HTML to be processed into text.  Investigate the problem by converting a problem email in the JEMH AuditHistory into a Test Case, so that it can be edited. The content type will likely be:

Content-Type: text/plain; charset=ISO-8859-1

A further clue to determine if mail was sourced from windows and that this content is applicable to you is to look for:

<meta name=ProgId content=Word.Document>
<meta name=Generator content="Microsoft Word 10">
<meta name=Originator content="Microsoft Word 10">

If that is there, then read on:

Reference:

Recently we came across an issue on an Exchange 2010 environment where all mails sent via OWA and Outlook 2011 (MAC) were getting received as Plain Text mails externally even though we specifically had set OWA to send in HTML.

Internally Users could send HTML messages via OWA and they would get received by the users as HTML as we wanted. The first thing we did was to compare what transport rules were getting applied to external mail and not internal mail and visa versa. As a test we got a user to have the same transport rules that were getting applied when sending internally but apply this to any external address also, however the issue still remained so we had ruled that out.

The second thing we looked at was the send connectors, as they were set to send mail via Exchange Hosted Services so we wanted to rule out that anything was going wrong at this end. So we basically created a new send connector which would basically route mail via DNS MX records to a specific address space (i.e hotmail.com , domain.com) therefore bypassing EHS. We then tested again but the issues still remained.

The final thing we checked which revealed what the issue was to run the following exchange powershell "get-remotedomain | fl" . We then spotted that the content type was set to MimeText which "converts all messages to MIME messages that use text formatting" (i.e plain text). We then changed the content type to mimehtmltext (Converts messages to MIME messages that use HTML formatting, unless the original message is a text message. If the original message is a text message, the outbound message is a MIME message that uses text formatting) by running the following command "get-remotedomain | set-remotedomain -ContentType MimeHtmlText". Please Note You will have to alter the previous command if you have multiple remote domains so that you change the content type on the specific remote domain you want.

Gmail Specific problems

Gmail recently (noticed as of 4  NOV 2015) blocks the inclusion of some file types that it considers as dangerous (eg .jar files), mail will appear as follows:

There will be an atlassian-jira-outbound-mail.log entry:

com.atlassian.mail.MailException: com.sun.mail.smtp.SMTPSendFailedException: 552-5.7.0 This message was blocked because its content presents a potential
    552-5.7.0 security issue. Please visit
    552-5.7.0  https://support.google.com/mail/answer/6590 to review our message
    552 5.7.0 content and attachment content guidelines. p10sm3867221wjx.36 - gsmtp

As Gmail is rejecting the mail based on attachments, JEMH can do nothing to resolve this (yet).

Unsupported 3rd party Custom Fields

Valiantys nFeed

JEMH cannot populate nFeed custom fields through API issue creation, see issue below.

• https://jira.valiantys.com/browse/NFEED-641
  

Mail Server problems

When sending mail 'interactively' through your mailhost, JIRA process the sent message and forwards it as a catchemail fail

If you have configured JIRA to talk to your mailhost via POP, then any mail sent from the mailbox itself (e.g. gmail web ui) and not from JIRA will end up being returned via POP to JIRA for processing, which is then detected by JEMH as a catchemail fail because it is from support@xx.com not to support@xx.com

This is not a bug in JEMH or JIRA, but a symptom of this scenario.  Resolutions include:

• not sending mail from the mailbox itself

• using an alternate protocol such as IMAP to talk to the mailserver, as IMAP has the concept of Folders, and can therefore filter 'Sent' mails.

Issue processing fails with "IllegalStateException: Folder is not Open"

This occurs when the mailbox connection to JIRA/JEMH is closed unexpectedly, for a variety of reasons.  This is beyond the control of JEMH.  The solution is to resolve the underlying mailserver problem.

For more information, please see https://confluence.atlassian.com/jirakb/jira-stops-creating-tickets-from-emails-330795303.html

Database

Oracle - SQL Exception thrown by Active Objects library: cannot insert null into (<table>.<column>)

If your database's triggers are disabled, this can lead to a processing error occurring, similar to this:

java.sql.SQLIntegrityConstraintViolationException: ORA-01400: cannot insert NULL into ("jiradb"."AO_78C957_NOTIFICATION_HIST"."ID")
	at com.atlassian.activeobjects.internal.EntityManagedActiveObjects.create(EntityManagedActiveObjects.java:113)
	at com.atlassian.activeobjects.osgi.TenantAwareActiveObjects.create(TenantAwareActiveObjects.java:299)
	at sun.reflect.GeneratedMethodAccessor1889.invoke(Unknown Source)

Re-enable the disabled triggers in order to fix this problem.  Source: https://confluence.atlassian.com/jirakb/error-when-deleting-an-issue-720639157.html

Large Emails

When trying to process a large (20MB+) email, JIRA hangs and does not process any more mail

Processing large emails can use a lot of JVM memory.  You may at some point find that an email of significant enough size requires more memory than is available.  This can result in JIRA hanging.  In the "atlassian-jira.log" file you will most likely find this:

java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOfRange(Unknown Source)
    at java.lang.String.<init>(Unknown Source)
    at java.lang.StringBuilder.toString(Unknown Source)
    at com.javahollic.jira.emh.engine.support.EmailHeaderUtil.getInputStream(EmailHeaderUtil.java:119)
    at com.javahollic.jira.emh.service.PreProcTaskRunner.runTasks(PreProcTaskRunner.java:152)
    at com.javahollic.jira.emh.service.EnterpriseMessageHandlerImpl.handleMessage(EnterpriseMessageHandlerImpl.java:243)
    at com.javahollic.jira.emh.service.EnterpriseMessageHandlerProxy.handleMessage(EnterpriseMessageHandlerProxy.java:46)
    at com.atlassian.jira.service.services.mail.MailFetcherService$1.process(MailFetcherService.java:438)
    at com.atlassian.jira.service.services.mail.MailFetcherService$MessageProviderImpl.getAndProcessMail(MailFetcherService.java:304)
    at com.atlassian.jira.service.services.mail.MailFetcherService.runImpl(MailFetcherService.java:426)
    at com.atlassian.jira.service.services.file.AbstractMessageHandlingService.run(AbstractMessageHandlingService.java:261)
    at com.atlassian.jira.service.JiraServiceContainerImpl.run(JiraServiceContainerImpl.java:66)
    at com.atlassian.jira.service.ServiceRunner.runService(ServiceRunner.java:75)
    at com.atlassian.jira.service.ServiceRunner.runServiceId(ServiceRunner.java:53)
    at com.atlassian.jira.service.ServiceRunner.runJob(ServiceRunner.java:36)
    at com.atlassian.scheduler.core.JobLauncher.runJob(JobLauncher.java:135)
    at com.atlassian.scheduler.core.JobLauncher.launchAndBuildResponse(JobLauncher.java:101)
    at com.atlassian.scheduler.core.JobLauncher.launch(JobLauncher.java:80)
    at com.atlassian.scheduler.quartz1.Quartz1Job.execute(Quartz1Job.java:32)
    at org.quartz.core.JobRunShell.run(JobRunShell.java:223)
    at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:549)

As per Atlassian documentation, a workaround to this problem is to increase the memory available to the JVM: JIRA applications crash due to OutOfMemoryError Java heap space.  Steps on how to do this are found here: Increasing Application Memory.

Workflow Auto-Advance Problems

I have JEMH set to change issue status to "Waiting on Support" when the customer replies via email, however it also changes when an internal user emails to the issue

The current implementation of Workflow Auto-Advance in JEMH is fairly limited in terms of flexibility.  If your customer is a non-JIRA account holder, then it is not currently possible to stop this from happening.  However, if your customers are having JIRA accounts created for them, there is a workaround of sorts.  First, check that your customers have a group membership that only they have (for example "customers").  For the transition that moves the issue back to "Waiting on Support", add a condition that allows only if user is member of their group.  This will allow only customers to transition back to this status.

Related Articles

• Questions:
  Emails were attached as .txt in a zip-file and not as .eml
• Questions:
  Send notifications to watchers in a certain group.
• Questions:
  reject message on closed ticket
• Questions:
  Using Soy Template for Notification and TemplateSet
• Questions:
  Set custom field at the beginning of subject