Valid since:
XL Deploy 8.2.0

Move artifacts from the file system to a database

XL Deploy can be configured to store and retrieve artifacts in two local storage repository formats:

  • file: The artifacts are stored on and retrieved from the file system.
  • db: The artifacts are stored in and retrieved from a relational database management system (RDBMS).

XL Deploy can only use one local artifact repository at any time. In the xl-deploy.conf file, you can set the xl.repository.artifacts.type configuration option for the storage repository to either "file" or "db".

The configuration option xl.repository.artifacts.root should be left untouched if you are moving artifacts from filesystem to database as the configuration value is needed by XL Deploy to resolve the artifacts path. You may remove xl.repository.artifacts.root only after the move is completed.

Moving artifacts

When XL Deploy starts, it checks if any artifacts are stored in a storage format that is not configured. If artifacts are detected, XL Deploy checks the xl.repository.artifacts.allow-move configuration option to see if the detected artifacts should be moved.

If xl.repository.artifacts.allow-move is set to the default false setting, XL Deploy does not start and throws an error. XL Deploy interprets this as an error made in the configuration.

To adjust the configuration: choose another local artifact repository type or set the xl.repository.artifacts.allow-move configuration option to true. After the configuration change, you must restart XL Deploy.

If xl.repository.artifacts.allow-move is set to true, XL Deploy starts up and moves the artifacts to the configured local artifact repository. XL Deploy uses both the configured local artifact repository (example: db) and the not-configured local artifact repository (e.g. “file”) to retrieve artifacts during the move process. This makes the artifacts that are waiting to be moved, available to the system.

The process of migrating artifacts moves the data in small batches with pauses between every two batches. This enables the system to be used for normal tasks during the process.

Handling errors and process restart

If an artifact cannot be moved because an error occurs, a report is written in the log file and the process continues. When XL Deploy is restarted during the process of moving the artifacts, the startup sequence as described above will be re-executed. If the xl.repository.artifacts.allow-move option is set to true, the move process will start again. Any artifacts that failed during the previous run, will be re-processed.

When the move process has completed successfully and all artifacts have been moved, a report is written in the log file and the xl.repository.artifacts.allow-move option can be (re)set to false. When artifacts are moved from the file system, empty folders may remain in the configured xl.repository.artifacts.root. These have no impact and can be deleted manually.

Files can remain on the file system, but are not detected as artifacts. This happens when files are no longer in use by the system, but have not been removed (example: files from application versions that are no longer used). The files can be removed after creating a backup.

If you are upgrading from a version that is earlier than XL Deploy 8.0.0, restart the server again after migration has finished to ensure that the artifacts are moved. Once the server has started you should see the following in your logs:

    2018-08-17 15:19:54.323 [xl-scheduler-system-akka.actor.default-dispatcher-2] {sourceThread=xl-scheduler-system-akka.actor.default-dispatcher-4, akkaSource=akka://xl-scheduler-system/user/ArtifactsMover, sourceActorSystem=xl-scheduler-system, akkaTimestamp=13:19:54.320UTC} INFO  c.x.d.r.s.a.m.ArtifactsMoverSupervisor - Found artifacts to move: 25 artifacts from file to db.

And a series of:

    2018-08-17 15:19:54.588 [xl-scheduler-system-akka.actor.default-dispatcher-2] {} INFO  c.x.d.r.s.a.m.FileToDbArtifactMover - Moved file artifact. [1 + 0 failed /25]
    2018-08-17 15:19:54.716 [xl-scheduler-system-akka.actor.default-dispatcher-2] {} INFO  c.x.d.r.s.a.m.FileToDbArtifactMover - Moved file artifact. [2 + 0 failed /25]

If xl.repository.artifacts.allow-move is enabled but you do not see the above logs, restart the server. If after restarting the server you still do not see the above logs, contact support.