Migrating from Server/DC to Cloud

Introduction

This article is for users of Enterprise Message Handler for Jira, who are considering migrating from a self-hosted deployment to the cloud-based offering of the app.

Migration assistance

There is currently no automated migration tool, primarily due to differences in the data structure used for storing configuration. You will need to do ground up configuration of JEMHC.

Regions

JEMHC supports Data residency in the following regions:

• US (US West)

• DE (Frankfurt)

See Atlassian docs on pinning your region.

• Data Residency: Manage Where Your Data is Hosted | Atlassian

Region Migration

We have no support for region migration at this time, its a do over, its on our roadmap.

Cloud capacity plans

Your JEMHC subscription drives an allocation (1st of every month) of a Capacity Plan based on the number of subscribed (active) users; as your users increase, so does message volume and data capacity. If you consume the allocated plan, inbound mail and outbound notifications stop. No data is lost, it just remains queued.

New evaluators are initially placed on a limited capacity Starter Plan. We do this to help you fail fast e.g. where you start consuming all mail within a mailbox from long long ago.

If you consume your limited Starter Plan during eval, we can increase toward the plan level that your active users would give. You can request increases toward the level that your active users would allocate can be requested through support@thepluginpeople.com , should you exceed that, you would typically need to get a DataPack (see next).

Common setup problems

No Profile > Catchemail match with mail recipients

When you setup a Profile, the ‘catchemail’ address needs to match an address of incoming mail, so that JEMHC can know its supposed to process the mail, and so that addressee can be specifically excluded from storage (to avoid a mail loop). If you use a test mailbox, likely it will just not match.

Processing mail from beginning of time

When you setup a new inbound mailbox, do clear out the mailbox and/or set the mailbox to return mail ‘from this point forward’, not doing so may lead to ALL mail ever being processed (again) causing confusion (and Plan consumption leading to exhaustion).

Lack of a default SMTP server configuration

If you do not configure an outbound SMTP server, JEMHC cannot communicate to you when your inbound processing encounters a problem like the Profile > Catchemail list doesn’t match any mail addressees, to avoid data loss, the mail is not READ, to avoid repeat reading (and consumption of your plan) the inbound mail handler is taken OFFLINE. You really need a default SMTP server configured, its not used for issue related notifications without further specific configuration.

Getting more message volume and data

Regardless of cause, if you consume your plan then JEMHC will stop processing mail, more capacity can be purchased through Data Packs as needed, for through a 12 month term Plan Upgrade.

Typical cases are where you can be impacted are where you have relatively low-users and relatively high message volume/data. JEMHC has options:

Data Packs & Plan Upgrades

Data Packs provide an on-demand short term Message and Data volume increase. Plan Upgrades provide a 1y term to gain higher capacities normally attributed to a higher number of users on a monthly basis. Both Data Packs and Plan Upgrades are purchased out of band from Atlassian Marketplace as it has not yet matured to support such app-specific metrics (see App licensing for details).

Limiting message volume usage

Related to message volume, JEMHC delivers its notification by BCC to limit the numbers of mails JEMHC needs to send through your mail server. Switching this to TO will increase message volume.

Related to message data, JEMHC does support inline images in outbound notifications, and attachments, if you enable this, and make regular use, it can impact your data usage.

Mail servers

JEMHC can’t use the built-in mailbox address provided by Atlassian, so you must use an external one. When doing so you may need allow connections from the JEMHC public IP address to to your mail service. For finer grained access control, you could setup dedicated certificates that only your configuration has.

Allowlisting IP addressed used by JEMHC

Your mailhost may require JEMHC public IP addresses to be allowed. Your mailhost may have to be configured to be allowed to relay mail sending (symptoms of this are mail that doesn't deliver outside your domain).

EU and DE app data location IP addresses to allow

What's different?

Data Storage

As of MAR 2024 we support a default region of US as well as secondary DE region that new installation can pin their data to. As yet we do not have migration support from/to US deployment, potentially some areas can be manually exported (Profile JSON, templates etc) for reintroduction manually in a new DE deployment.

Inbound mail processing

JEMHC periodically retrieves mail from your configured mail servers over SSL for common protocols (POP/IMAP/Graph). By default, JEMHC stores a rolling window of the last 100 mails in full (for mail under 10MB). This data is stored in either AWS region us-east-1 (USA) or us-central-1 (DE) and is encrypted at-rest and in-transit. Users can opt out of email retention, but requires you to contact us to re-enable.

Part of Auditing > Inbound Messages will retain recipient info, including the Project/Issue Key.

We have no access to your mail held here. You have the option to ‘flag’ mail for support, at which point we are able to see/download that mail and the processing report through our back office support tool, accessed by employees only.

Outbound notifications

Your Jira Cloud instance will send event data to JEMHC when changes are made to issues. JEMHC will only retain webhooks for processing where you have configured JEMHC to generate notifications for specific projects, in this way, we retain only the subset required for JEMHC to do what you have configured.

On receipt, webhooks are stored, encrypted, in a FIFO queue that JEMHC subsequently processes. As JEMHC reads webhook data, we create an Auditing > Events record containing this data, also encrypted at rest. JEMHC then makes calls back to your instance to fill in the blanks about the issue, that data is stored in our DB. JEMHC also stores a ‘Report’ of the webhook Event processing, again, we retain only the most recent 100 for diagnostic purposes. Having the event data is key in you being able to create a custom template and preview what it will really look like.

When JEMHC sends mail, we again retain the most recent 100 messages (and their HTML payload) in Auditing > Outbound Messages , where we track the project/issue source, the sent email subject (typically issue summary) as well as the recipients. A ‘Report’ for the actual sending of the mail is also stored, including recipients, filenames of attachments added etc.

About PII

JEMHC communicates with your Jira instance over SSL transport security.

Whilst JEMHC only retains the most recent 100 webhooks and inbound/outbound mails, we retain the last months worth of ‘audit’ records that something happened, meaning, webhooks, inbound mail and outbound mail. If a user choose to ‘be forgotten’ Atlassian will notify JEMHC and removals of related records will occur.

GDPR

See General Data Protection Regulation (GDPR)

Common Data Questions

What data is extracted from the Atlassian Cloud?

For inbound, Jira project metadata defining fields that we need to populate. For outbound, we retrieve any data within the typical REST response for the issue, e.g. /rest/api/latest/issue/ABC-123. In addition, attachment data referred (e.g. inlined screenshots, but could also be any attached file, if you configure JEMHC to do so!), icons for projects, issue types, priorities, all the things you'd expect in a Jira email.

Is it data in transit or data at rest?

Webhooks are in transit until stored in the JEMHC DB, the time webhooks are ‘in transit’ in the FIFO queue depends on the backlog in the queue. Typically, the queue is empty, and messages are retrieved in a matter of seconds. Once data is retrieved from the queue it is encrypted and stored in the JEMHC DB, considered at rest.

If data at rest, where is it stored.

JEMHC data is currently stored in a highly available dedicated JEMHC database, which is not publicly accessible.

Data Residency

JEMHC currently only has one ‘Realm’, located in the USA, we expect to look into JEMHC data residency later in the year.

Profiles

JEMH Server/DC Profile exports are not currently compatible with JEMH Cloud.

The structure of the JEMH Profile is very different from JEMHC. JEMH Profiles still have a lot of top-level configuration whereas JEMHC was designed with lessons-learned from JEMH whereby Project Mappings in JEMHC are Profile top-level feature, containing the majority of all configuration, Rules also then provide a subset of the Project Mappings.

Scripting support

Scripting support in JEMHC exists for:

• Javascript PreProc Task (header manipulation)

• Javascript Script Field Processor

• Javascript Script Rule support in Project Mappings

• Javascript Custom Field Default

The multi-tennant nature means that security is always at the forefront, and for that reason, scripts are restricted in what they can do, effectively they have access to the ‘issue' JSON structure, and the mail content where applicable, there are no remote calls allowed and no special custom functionality beyond javascript default scripting.

Messaging

Inbound/Outbound mail server connectivity isn’t within JEMH for Server/DC, in JEMHC, we have re-implemented all inbound/outbound mail server connectivity functionality. For example, JEMHC has OAuth authorization support, which requires specific admin actions to support, that cannot be migrated.

Transports

JEMHC has implemented a wider range of transports than existed in JEMH, which only now has Slack and SMS. Slack for JEMHC uses OAuth, whereas JEMH doesn’t yet.

SSL

Customers with self-managed mail infrastructure likely have their own SSL certificate chains, these would have been imported into the JRE ca-certs file in your Server/DC Jira. As JEMHC is multi-tenant, we re-implemented this in the application, but could not easily automate a migration.

Blacklisting

Whilst technically compatible, we don’t yet have an export/import feature, logged as:

• JEMH-7545 / JEMHC-2380

Notifications

JEMH Server/DC notifications are not compatible with JEMHC.

In JEMH Server/DC notifications are driven by issue events from within Jira, in Cloud these don’t exist. As JEMHC receives only a subset of webhook notification types, the Jira notification scheme ceases to be useful. JEMHC re-imagines the requirements of notifications through Audiences (sourced from custom fields, and/or related to Jira User issue fields like reporter/assignee etc).

In JEMH Server/DC its possible to use regular expressions to match Project keys that should be handled through this particular Notification Mapping. In JEMHC, we avoid regular expressions and simplify things, enabling users to ‘pick’ all or nominated projects.

Transition Attachment Custom Field picker to allow

The Use the JEMH Transition Attachments Custom Field attachment picker for workflow transition screen has not yet been implemented in cloud, logged as:

• JEMHC-3764

Status Notifications

This has not been implemented in cloud at this time.

Workflow Transition Attachments Custom Field

The Workflow Transition Attachments Custom Field used to select attachments to send in JEMH post-functions in JEMH Server/DC is not yet available in JEMH Cloud.

Template Sets

JEMH Server/DC templates are not compatible with JEMHC as the Velocity context driving the template renders are different.

In JEMH Server/DC notification templates relate to events fired by Jira, these are not directly applicable in Cloud (we get a much coarser granularity of create/update/commented events). Server/DC templates inherit effectively from the host Jira application - they use Jira provided macros and CSS.

Template Themes

JEMH Server/DC templates and CSS 'theme' content is not compatible with JEMHC.

In JEMH Server/DC the template/CSS all tie into macros and other templates that are internal to Jira. JEMH templates ‘are’ Jira templates and only with Jira IssueEvent/TemplateIssue objects, in JEMHC we have reimplemented Themes from scratch that work with the cloud specific Webhooks (the equivalent of issue events).

Custom Macros

Its possible for custom templates to be taken from a Jira Server/DC instance and loaded into JEMHC > Notifications > Templates > Custom Macros, doing so will allow those templates to be available to all custom Theme templates you define.

Images and Static Resources

JEMHC Server/DC support for Images uses static resources and specific helper methods in $jemhUtils to render image links. In JEMHC we simply this, providing markup in the UI against all uploaded images. Static resources in JEMH can be exported, as yet we don’t have an export all feature.

• JEMH-7546 / JEMHC-2383 : Add Export All / Import All actions to JEMH > Static Resources

Directive Sets

The definitions are compatible for migration but as yet we have not implemented this.

• JEMH-7548 / JEMHC-2384 : Add Directive Set Export All / Import All (zip)

Test Cases

Test Cases can currently be exported and imported individually:

• JEMH-7550 / JEMHC-2385 : Test Case Export All / Import ZIP

Auditing

No historic audit history will be retained during transition to cloud.

We don’t support migrating audit history from JEMH Server/DC to JEMHC as JEMH Server is able to retain unlimited volumes of data. In JEMHC (with customer opt-out) we can store the most recent 100 inbound and outbound messages for your diagnostic purposes.

Issue Association

Foreign Key Issue Association configuration in JEMHC is different due to the structure of the Profile.

In short, JEMHC restricts Issue Association in a Project Mapping, scoping to the mappings selected Project.

However, the default Project Mapping will not scope to the mappings selected Project, but any non-default mappings added that inherit the Foreign Key Issue Association will scope to the non-default mappings selected Project.

For further information, see Understand how Issue association works | Regexp foreign key association

Public REST API

JEMHC has yet to implement a public rest API as we have for DC (Use JEMH Public Rest API to Export, Import and Update profiles ). Whilst tilted as for Profiles, we also exposed a way to deliver Mail directly over REST, which was utilised by some 3rd party email client app integrations. Those integrations will need rework once we have a public API in JEMHC. We are tracking this feature interally as https://thepluginpeople.atlassian.net/browse/JEMHC-3042.