Migrating from Server/DC to Cloud

Introduction

This article is for users of Enterprise Message Handler for Jira, who are considering migrating from a self-hosted deployment to the cloud-based offering of the app.

Migration assistance

There is currently no automated migration tool, primarily due to differences in the data structure used for storing configuration. You will need to do ground up configuration of JEMHC.

Cloud capacity plans

Your JEMHC subscription drives an allocation (1st of every month) of a Capacity Plan based on the number of subscribed users; as your users increase, so does message volume and data capacity. If you consume the allocated plan, inbound mail and outbound notifications stop. No data is lost, it just remains queued.

New users are initially placed on a Starter Plan that is deliberately limited in capacity. This is to fail fast if a problem occurs (for example, a mail server delivers more mail than you expected).

Getting more message volume and data

Regardless of cause, if you consume your plan then JEMHC will stop processing mail, more capacity can be purchased through Data Packs as needed, for through a 12 month term Plan Upgrade.

Typical cases are where you can be impacted are where you have relatively low-users and relatively high message volume/data. JEMHC has options:

Data Packs & Plan Upgrades

Data Packs provide an on-demand short term Message and Data volume increase. Plan Upgrades provide a 1y term to gain higher capacities normally attributed to a higher number of users on a monthly basis. Both Data Packs and Plan Upgrades are purchased out of band from Atlassian Marketplace as it has not yet matured to support such app-specific metrics (see https://thepluginpeople.atlassian.net/wiki/spaces/JEMHC/pages/163164185 for details).

Limiting message volume usage

Related to message volume, JEMHC delivers its notification by BCC to limit the numbers of mails JEMHC needs to send through your mail server. Switching this to TO will increase message volume.

Related to message data, JEMHC does support inline images in outbound notifications, and attachments, if you enable this, and make regular use, it can impact your data usage.

Mail servers

JEMHC can’t use the built-in mailbox address provided by Atlassian, so you must use an external one. When doing so you may need allow connections from the JEMHC public IP address to to your mail service. For finer grained access control, you could setup dedicated certificates that only your configuration has.

What features are missing for cloud?

Scripting support

Script Field Processor or Script Project Mapping Rules are not yet available. This is a important feature and is something we hope to offer between Q4 '21 - Q1 '22.

What's different?

Data Storage

Inbound mail processing

JEMHC periodically retrieves mail from your configured mail servers over SSL for common protocols (POP/IMAP/EWS). By default, JEMHC stores a rolling window of the last 100 mails in full (for mail under 10MB). This data is stored in the us-east-1 AWS region and is encrypted at-rest and in-transit. Users can opt out of email retention, but requires you to contact us to re-enable.

Part of Auditing > Inbound Messages will retain recipient info, including the Project/Issue Key.

We have no access to your mail held here. You have the option to ‘flag’ mail for support, at which point we are able to see/download that mail and the processing report through our back office support tool, accessed by employees only.

Outbound notifications

Your Jira Cloud instance will send event data to JEMHC when changes are made to issues. JEMHC will only retain webhooks for processing where you have configured JEMHC to generate notifications for specific projects, in this way, we retain only the subset required for JEMHC to do what you have configured.

On receipt, webhooks are stored, encrypted, in a FIFO queue that JEMHC subsequently processes. As JEMHC reads webhook data, we create an Auditing > Events record containing this data, also encrypted at rest. JEMHC then makes calls back to your instance to fill in the blanks about the issue, that data is stored in our DB. JEMHC also stores a ‘Report’ of the webhook Event processing, again, we retain only the most recent 100 for diagnostic purposes. Having the event data is key in you being able to create a custom template and preview what it will really look like.

When JEMHC sends mail, we again retain the most recent 100 messages (and their HTML payload) in Auditing > Outbound Messages , where we track the project/issue source, the sent email subject (typically issue summary) as well as the recipients. A ‘Report’ for the actual sending of the mail is also stored, including recipients, filenames of attachments added etc.

About PII

JEMHC communicates with your Jira instance over SSL transport security.

Whilst JEMHC only retains the most recent 100 webhooks and inbound/outbound mails, we retain the last months worth of ‘audit’ records that something happened, meaning, webhooks, inbound mail and outbound mail. If a user choose to ‘be forgotten’ Atlassian will notify JEMHC and removals of related records will occur.

GDPR

See https://thepluginpeople.atlassian.net/wiki/spaces/JEMHC/pages/233996303

Common Data Questions

What data is extracted from the Atlassian Cloud?

For inbound, Jira project metadata defining fields that we need to populate. For outbound, we retrieve any data within the typical REST response for the issue, e.g. /rest/api/latest/issue/ABC-123. In addition, attachment data referred (e.g. inlined screenshots, but could also be any attached file, if you configure JEMHC to do so!), icons for projects, issue types, priorities, all the things you'd expect in a Jira email.

Is it data in transit or data at rest?

Webhooks are in transit until stored in the JEMHC DB, the time webhooks are ‘in transit’ in the FIFO queue depends on the backlog in the queue. Typically, the queue is empty, and messages are retrieved in a matter of seconds. Once data is retrieved from the queue it is encrypted and stored in the JEMHC DB, considered at rest.

If data at rest, where is it stored.

JEMHC data is currently stored in a highly available dedicated JEMHC database, which is not publicly accessible.

Data Residency

JEMHC currently only has one ‘Realm’, located in the USA, we expect to look into JEMHC data residency later in the year.

Profiles

JEMH Server/DC Profile exports are not currently compatible with JEMH Cloud.

The structure of the JEMH Profile is very different from JEMHC. JEMH Profiles still have a lot of top-level configuration whereas JEMHC was designed with lessons-learned from JEMH whereby Project Mappings in JEMHC are Profile top-level feature, containing the majority of all configuration, Rules also then provide a subset of the Project Mappings.

Messaging

Inbound/Outbound mail server connectivity isn’t within JEMH for Server/DC, in JEMHC, we have re-implemented all inbound/outbound mail server connectivity functionality. For example, JEMHC has OAuth authorization support, which requires specific admin actions to support, that cannot be migrated.

Transports

JEMHC has implemented a wider range of transports than existed in JEMH, which only now has Slack and SMS. Slack for JEMHC uses OAuth, whereas JEMH doesn’t yet.

SSL

Customers with self-managed mail infrastructure likely have their own SSL certificate chains, these would have been imported into the JRE ca-certs file in your Server/DC Jira. As JEMHC is multi-tenant, we re-implemented this in the application, but could not easily automate a migration.

Blacklisting

Whilst technically compatible, we don’t yet have an export/import feature, logged as:

  • JEMH-7545 / JEMHC-2380

Notifications

JEMH Server/DC notifications are not compatible with JEMHC.

In JEMH Server/DC notifications are driven by issue events from within Jira, in Cloud these don’t exist. As JEMHC receives only a subset of webhook notification types, the Jira notification scheme ceases to be useful. JEMHC re-imagines the requirements of notifications through Audiences (sourced from custom fields, and/or related to Jira User issue fields like reporter/assignee etc).

In JEMH Server/DC its possible to use regular expressions to match Project keys that should be handled through this particular Notification Mapping. In JEMHC, we avoid regular expressions and simplify things, enabling users to ‘pick’ all or nominated projects.

Status Notifications

This has not been implemented in cloud at this time.

Workflow Transition Attachments Custom Field

The Workflow Transition Attachments Custom Field used to select attachments to send in JEMH post-functions in JEMH Server/DC is not yet available in JEMH Cloud.

Template Sets

JEMH Server/DC templates are not compatible with JEMHC as the Velocity context driving the template renders are different.

In JEMH Server/DC notification templates relate to events fired by Jira, these are not directly applicable in Cloud (we get a much coarser granularity of create/update/commented events). Server/DC templates inherit effectively from the host Jira application - they use Jira provided macros and CSS.

Template Themes

JEMH Server/DC templates and CSS 'theme' content is not compatible with JEMHC.

In JEMH Server/DC the template/CSS all tie into macros and other templates that are internal to Jira. JEMH templates ‘are’ Jira templates and only with Jira IssueEvent/TemplateIssue objects, in JEMHC we have reimplemented Themes from scratch that work with the cloud specific Webhooks (the equivalent of issue events).

Custom Macros

Its possible for custom templates to be taken from a Jira Server/DC instance and loaded into JEMHC > Notifications > Templates > Custom Macros, doing so will allow those templates to be available to all custom Theme templates you define.

Images and Static Resources

JEMHC Server/DC support for Images uses static resources and specific helper methods in $jemhUtils to render image links. In JEMHC we simply this, providing markup in the UI against all uploaded images. Static resources in JEMH can be exported, as yet we don’t have an export all feature.

  • JEMH-7546 / JEMHC-2383 : Add Export All / Import All actions to JEMH > Static Resources

Directive Sets

The definitions are compatible for migration but as yet we have not implemented this.

  • JEMH-7548 / JEMHC-2384 : Add Directive Set Export All / Import All (zip)

Test Cases

Test Cases can currently be exported and imported individually:

  • JEMH-7550 / JEMHC-2385 : Test Case Export All / Import ZIP

Auditing

No historic audit history will be retained during transition to cloud.

We don’t support migrating audit history from JEMH Server/DC to JEMHC as JEMH Server is able to retain unlimited volumes of data. In JEMHC (with customer opt-out) we can store the most recent 100 inbound and outbound messages for your diagnostic purposes.