Step-by-Step Guide

ICDP INTERNAL - This document describes how to build a new mDIS instance via a CI/CD pipeline.
The build process will create a persistent Docker image, for “real” usage. It will not create a "transient" Docker image, which is used for testing only, and then deleted.

Continued from Step-By-Step Guide - General.

The new instance will be populated with the standard mDIS forms and a default data model, and a few test records. The new instance will be available as a Docker image, which can be downloaded and run on a server.

The new instance gives you a good starting point for further customization and deployment.

Workflow - Executing the pipeline - Detailed Look

Build Strategy

For now, the strategy is:

1. Let Gitlab CI/CD build the new mDIS instance on GFZ Section 4.2's host wb31.gfz.de.
2. Let CI/CD upload it to our container registry on git.gfz.de/mdis-hub/mdis-blank.
3. Manually download (docker pull) that Docker image or a production server.
4. Adapt it for production use and populate it with data manually.

This process is detailed below.

It is a long process; see scroll-down figure and flowchart diagrams. Taking an mDIS into production still requires some manual steps, which are also described below. Some of these steps are quite hacky; process improvements are needed.

Prerequisites

You must know GitLab and Docker. You need a GFZ-GitLab account and write access to this mDIS GitLab repository. Alternatively, you can create your own copy by git-cloning or git-forking the mDIS GitLab repo at GFZ. However, you must also create your own GitLab runners for executing pipelines.

You must have admin access to a MySQL/MariaDB database server where a raw, empty database will be located.

Create a new empty MySQL database

Create a new empty MySQL (or MariaDB) database on the development database server (icdp-osg uses wb31). The database should be empty, without any mDIS tables. If you use an existing database, note that tables will be dropped and overwritten.
The database must be accessible to the GitLab CI/CD pipeline, specifically to the GitLab Runner process. Create a database user whose credentials can be used by GitLab Runner to access the database.

Set permissions. The database user defined in the .env file in the mDIS server root directory must have access to the DB given the credentials in the .env file. (There can be conflicts between .env files and environment variables passed in from docker-compose.yml files. TODO: write more about this.)

Set some gitlab CI/CD Variables

You must set the following variables in order to create an intermediate docker image. These will be stored in an .env file which you then replace with your own .env file after deploying.

	Variable name	Value	Requirement	Meaning
1	MDIS_BLANK_INSTANCE_NAME	mdis-blank	mandatory	mdis name
2	MDIS_BLANK_PATH	mdis/blank	mandatory	URL fragment
3	MYSQL_BLANK_DATABASE	dis_test_blank	mandatory	database name
4	MYSQL_BLANK_PASSWORD	dis_db_password	optional(*)	database password
5	MYSQL_BLANK_USER	dis_db_user	optional(*)	database user
6*	MDIS_CUSTOM_INSTANCE_NAME	mdis-samplehub	mandatory	mdis
7*	MDIS_CUSTOM_PATH	mdis/samplehub	mandatory	mdis
8*	BRANCH_CUSTOM	mdis-blank	mandatory	mdis
9*	GIT- MYSQL, REGISTRY-...	Passwords	mandatory	various

row 6* - mdis-name as it will appear on the dashoard and on various reports
row 7* - URL Path fragment for your instance
row 8* - from which git branch to build the instance, example: mdis-blank (don't build from master, use a foundation branch)
row 9* - if you use Gitlab access tokens, make sure they are not expired. For other passwords, make sure they were not changed.

mandatory: must be changed to reflect the new instance name
optional(*): can be set if you want to use a different database user and password than the default ones.

More about the variables: see reference document conventions/gitlab-variables.

Run the pipeline to build the new instance

Change into the mDIS source code repository at icdp-osg/mdis and select the blank-images branch. Look at previous pipelines runs to see what the pipeline does.
Re-Run the entire pipeline, or just run the decisive step "build_mdis_blank" in any recent pipeline manually. This step will generate a new downloadable Docker image which will have a name like
mdis-<PROJECTNAME>-<PIPELINE_ID>--<DATE>... .

Components of the image name are:

• <PROJECTNAME>: the instance name, e.g, mdis-bushveld23.
• <PIPELINE_ID>: the pipeline-id that will create the image, e.g. 63079.
• <DATE>, e.g. 2023-07-08 is year and month of the build. (This is a convention, not a requirement).

Docker: pull the new image

See USAGE.md in the mdis-hub/mdis-blank repository for more information on how to get the mdis-blank Docker image.

Here we sketch the process:

Do this on your development server.
The image will become available for download with the docker pull command, like this:

docker pull git.gfz.de:5000/mdis-hub/mdis-blank:latest, or more specifically,
docker pull git.gfz.de:5000/mdis-hub/mdis-blank:mdis_blank--84868--2025-03-12

where

• git.gfz.de:5000/mdis-hub/mdis-blank is the registry name.
• other fragments are a group the image name, the instance name, the Gitlab build-pipeline id, the branch name, and the commit hash; as explained above.

Knowing the build-pipeline id, you should be able to reconstruct the exact build job that created the image. This is important for reproducibility. The build log references a lot of buildjob metadata and contains the exact state of the code repository, enhancing traceability what happened at build time.

See DEPLOYMENT.md in the mdis-hub/mdis-blank repository for more information on how to deploy the image.

---

Continued at notes-todo and notes-icdp-internal.