Migrating from Server/DC to Cloud

Introduction

This article is for customers considering migrating from JEMH for Jira Server/Data Center to JEMH Cloud, also known as JEMHC.

Migration assistance

There is currently no automated migration tool, primarily due to differences in the data structure used for storing configuration. You will need to do ground up configuration of JEMHC.

In progress

We have been adding some configuration section-exports in JEMH, (e.g. image blacklisting) but as yet have not added the import aspect to JEMHC.

Things to consider

JEMH Server/DC is managed by you, the volume of email and the MB data volume that they comprise is down to your infrastructure to handle. In JEMHC, our infrastructure handles everything, in order to avoid becoming swamped, your JEMHC Subscription drives an allocation of a Capacity Plan based on the number of subscribed users, as your users increase, so does message volume and data capacity.

JEMHC capacity plans have gone up frequently, they are sufficient to solve most users capacity needs. If your are in an edge case of low-users, high message volume/high data capacity, JEMHC has options, through the purchase of Data Packs (short term Message and Data increases) as well as 1y term Plan Upgrades to gain higher capacities normally attributed to a higher number of users. Both Data Packs and Plan Upgrades are purchased out of band from Atlassian Marketplace as it has not yet matured to support such app-specific metrics (see https://thepluginpeople.atlassian.net/wiki/spaces/JEMHC/pages/163164185 for details).

Related to message volume, JEMHC delivers its notification by BCC to limit the numbers of mails JEMHC needs to send through your mail server. Switching this to TO will increase message volume.

Related to message data, JEMHC does support inline images in outbound notifications, and attachments, if you enable this, and make regular use, it can impact your data usage.

Mailservers

JEMHC cannot use the mailbox provided by Atlassian, you must use an external one, when doing so you may need to refer JEMHC public IP address to allow contact to your instance - this allows anyone on JEMHC to attempt this, for finer grained security, you would need to setup dedicated certificates that only your configuration has.

  • Microsoft office 365 (in:POP/IMAP/EWS, out: SMTP)

  • Gmail (in: POP/IMAP, out: SMTP)

  • Your own dedicated Exchange (in:POP/IMAP/EWS, out: SMTP)

New users start on a Starter Plan that is deliberately limited in capacity, this is to fail fast when problem scenarios occur (e.g. processing a POP mailbox where the ‘from now’ checkbox wasn’t set…)

What's missing?

Whilst there are many subtle differences between JEMH and JEMHC, given JEMHC is an entirely different implementation, the main functionality missing in JEMHC is Scripting support, which means there there is no Script Field Processor or Script Rules yet. This is a big feature to do for cloud and is something we hope to look at this year.

What's different?

Data Storage

Inbound mail flow

As JEMHC is remote, the JEMHC app retrieves mail from mail servers you configure over SSL transport for common protocols (POP/IMAP/EWS). By default, JEMHC currently retains a very limited rolling window of the 100 latest mails in full (for mail under 10MB), encrypted (at rest) in our database, residing in the AWS Virginia data center. We do this as part of auditing to allow failed mails to be retried. Users can opt out of this (requires support interaction to re-enable), doing this would impact our ability to support you with processing problems.

JEMHC communicates with your Jira instance over SSL transport security.

Part of Auditing > Inbound Messages will retain recipient info, including the Project/Issue Key.

We have no access to your mail held here. You have the option to ‘flag’ mail for support, at which point we are able to see/download that mail and the processing report through our back office support tool, accessed by employees only.

Outbound mail flow

For background, see https://thepluginpeople.atlassian.net/wiki/spaces/JEMHC/pages/2552889351 and https://thepluginpeople.atlassian.net/wiki/spaces/JEMHC/pages/40861735

As JEMHC is remote, your instance will send a webhook (JSON payload) for each and every change you make to issues. JEMHC will only retain webhooks for processing where you have configured JEMHC to generate notifications for specific projects, in this way, we retain only the subset required for JEMHC to do what you have configured.

On receipt, webhooks are stored, encrypted, in a FIFO queue that JEMHC subsequently processes. As JEMHC reads webhook data, we create an Auditing > Events record containing this data, also encrypted at rest. JEMHC then makes calls back to your instance to fill in the blanks about the issue, that data is stored in our DB. JEMHC also stores a ‘Report’ of the webhook Event processing, again, we retain only the most recent 100 for diagnostic purposes. Having the event data is key in you being able to create a custom template and preview what it will really look like.

When JEMHC sends mail, we again retain the most recent 100 messages (and their HTML payload) in Auditing > Outbound Messages , where we track the project/issue source, the sent email subject (typically issue summary) as well as the recipients. A ‘Report’ for the actual sending of the mail is also stored, including recipients, filenames of attachments added etc.

About PII

Whilst JEMHC only retains the most recent 100 webhooks and inbound/outbound mails, we retain the last months worth of ‘audit’ records that something happened, meaning, webhooks, inbound mail and outbound mail. If a user choose to ‘be forgotten’ Atlassian will notify JEMHC and removals of related records will occur.

GDPR

See https://thepluginpeople.atlassian.net/wiki/spaces/JEMHC/pages/233996303

Common Data Questions

What data is extracted from the Atlassian Cloud?

For inbound, Jira project metadata defining fields that we need to populate. For outbound, we retrieve any data within the typical REST response for the issue, eg /rest/api/latest/issue/ABC-123. In addition, attachment data referred (eg inlined screenshots, but could also be any attached file, if you configure JEMHC to do so!), icons for projects, issue types, priorities, all the things you’d expect in a Jira email.

Is it data in transit or data at rest?

Webhooks are in transit until stored in the JEMHC DB, the time webhooks are ‘in transit’ in the FIFO queue depends on the backlog in the queue. Typically, the queue is empty, and messages are retrieved in a matter of seconds. Once data is retrieved from the queue it is encrypted and stored in the JEMHC DB, considered at rest.

If data at rest, where is it stored.

JEMHC data is currently stored in a highly available dedicated JEMHC database, which is not publicly accessible.

Data Residency

background: https://www.atlassian.com/software/data-residency , non US realms: https://www.atlassian.com/roadmap/cloud?category=dataManagement&

JEMHC currently only has one ‘Realm’, located in the USA, we expect to look into JEMHC data residency later in the year..

Profiles

JEMH Server/DC Profile exports are not currently compatible with JEMH Cloud.

The structure of the JEMH Profile is very different from JEMHC. JEMH Profiles still have a lot of top-level configuration whereas JEMHC was designed with lessons-learned from JEMH whereby Project Mappings in JEMHC are Profile top-level feature, containing the majority of all configuration, Rules also then provide a subset of the Project Mappings.

Messaging

Inbound/Outbound mail server connectivity isn’t within JEMH for Server/DC, in JEMHC, we have re-implemented all inbound/outbound mail server connectivity functionality. For example, JEMHC has OAuth authorization support, which requires specific admin actions to support, that cannot be migrated.

Transports

JEMHC has implemented a wider range of transports than existed in JEMH, which only now has Slack and SMS. Slack for JEMHC uses OAuth, whereas JEMH doesn’t yet.

SSL

Customers with self-managed mail infrastructure likely have their own SSL certificate chains, these would have been imported into the JRE ca-certs file in your Server/DC Jira. As JEMHC is multi-tenant, we re-implemented this in the application, but could not easily automate a migration.

Blacklisting

Whilst technically compatible, we don’t yet have an export/import feature, logged as:

  • JEMH-7545 / JEMHC-2380

Notifications

JEMH Server/DC notifications are not compatible with JEMHC.

In JEMH Server/DC notifications are driven by issue events from within Jira, in Cloud these don’t exist. As JEMHC receives only a subset of webhook notification types, the Jira notification scheme ceases to be useful. JEMHC re-imagines the requirements of notifications through Audiences (sourced from custom fields, and/or related to Jira User issue fields like reporter/assignee etc).

In JEMH Server/DC its possible to use regular expressions to match Project keys that should be handled through this particular Notification Mapping. In JEMHC, we avoid regular expressions and simplify things, enabling users to ‘pick’ all or nominated projects.

Status Notifications

This has not been implemented in cloud at this time.

Workflow Transition Attachments Custom Field

The Workflow Transition Attachments Custom Field used to select attachments to send in JEMH post-functions in JEMH Server/DC is not yet available in JEMH Cloud.

Template Sets

JEMH Server/DC templates are not compatible with JEMHC as the Velocity context driving the template renders are different.

In JEMH Server/DC notification templates relate to events fired by Jira, these are not directly applicable in Cloud (we get a much coarser granularity of create/update/commented events). Server/DC templates inherit effectively from the host Jira application - they use Jira provided macros and CSS.

Template Themes

JEMH Server/DC templates and CSS 'theme' content is not compatible with JEMHC.

In JEMH Server/DC the template/CSS all tie into macros and other templates that are internal to Jira. JEMH templates ‘are’ Jira templates and only with Jira IssueEvent/TemplateIssue objects, in JEMHC we have reimplemented Themes from scratch that work with the cloud specific Webhooks (the equivalent of issue events).

Custom Macros

Its possible for custom templates to be taken from a Jira Server/DC instance and loaded into JEMHC > Notifications > Templates > Custom Macros, doing so will allow those templates to be available to all custom Theme templates you define.

Images and Static Resources

JEMHC Server/DC support for Images uses static resources and specific helper methods in $jemhUtils to render image links. In JEMHC we simply this, providing markup in the UI against all uploaded images. Static resources in JEMH can be exported, as yet we don’t have an export all feature.

  • JEMH-7546 / JEMHC-2383 : Add Export All / Import All actions to JEMH > Static Resources

Directive Sets

The definitions are compatible for migration but as yet we have not implemented this.

  • JEMH-7548 / JEMHC-2384 : Add Directive Set Export All / Import All (zip)

Test Cases

Test Cases can currently be exported and imported individually:

  • JEMH-7550 / JEMHC-2385 : Test Case Export All / Import ZIP

Auditing

No historic audit history will be retained during transition to cloud.

We don’t support migrating audit history from JEMH Server/DC to JEMHC as JEMH Server is able to retain unlimited volumes of data. In JEMHC (with customer opt-out) we can store the most recent 100 inbound and outbound messages for your diagnostic purposes.