Shard Backend Migration

This feature helps the administrator copy the content that already exists in the repository from one storage backend to another (filesystem or S3), for a single shard or for all of them. Use it whenever a backend is switched, so the content stored before the switch is also available on the new backend:

• To move the whole installation to S3, changing the “repository.{folder}.type” properties — see Change repository home.
• To move a single shard, changing its Storage in the Shards administration.

It is available at Administration > Utilities > Shards > Shard Backend Migration.

Caution

OpenKM always reads the content from the backend currently configured, with no fallback to the previous one: once the backend is switched, any content not yet copied to the new backend can’t be opened until it is migrated. Follow the steps below to keep that window as short as possible.

Switch a backend step by step

1. Run the migration with Keep source copy checked while users keep working on the current backend. Nothing is deleted, and it can be repeated as many times as needed: content already at the destination is skipped, so each run only copies what has been added since the previous one.
2. Switch the backend: the “repository.{folder}.type” property in “$TOMCAT_HOME/openkm.properties” (restart required), or the shard Storage in the Shards administration.
3. Run the migration again with Keep source copy unchecked. It copies anything still missing and deletes the source copy of every item already verified at the destination.

Tip

Purge the trash before the final run: content marked for deletion (pending purge) is not migrated.

Run a migration

• Select the Purpose:
  • DATASTORE: content of documents (every version) and emails.
  • EXTRACTION: extracted text of documents and emails.
  • OCR_TEMPLATE: images of the OCR templates.
• Select the Shard, or All shards.
• Select the source backend (From) and destination backend (To): Filesystem or S3.
• Check or uncheck Keep source copy.
• Click the Start button.

Every item copied is verified at the destination before it is counted as copied: against the checksum stored in the database for document and email content, or by size otherwise. An item that fails the verification is removed from the destination and counted as an error, and its source copy is never deleted.

Note

• OCR_TEMPLATE is always migrated for All shards and keeping the source copy: OCR templates are not assigned to any particular shard.
• The preview cache can’t be migrated: it always stays on the filesystem and is regenerated on demand.
• When several shards share the same folder (the default “repository.datastore.home” has no “${shard.id}”), each item is checked against the database and only migrated, or deleted, as part of the shard it belongs to.

Caution

Only one migration can run at a time, whatever its purpose, shard or direction, and never at the same time as a Node Shard Migration: two overlapping migrations could delete the only copy of an item. The migration runs in the background and can take minutes or hours, depending on the volume of data. The progress page refreshes automatically every 2 seconds, showing Status, Keep source copy, Found, Copied, Already at destination, Deleted from source, Skipped (.bak), Errors, Bytes copied, the current shard and tenant, and the last error.

Note

The detailed log can also be followed on the progress page, filtered by ContentStoreMigrator.