mDIS For System Administrators (SAs)

Note

Many tasks and responsibilities overlap with the mDIS developer role. A clear separation is not always possible.

Related Pages

The list of "Related Pages" is quite long this time 😄. See bottom of page.

This page describes tasks that are usually performed only once, during the setup and configuration of mDIS.

mDIS Client-Server Architecture

Most likely, an end user encounters mDIS by opening a web page. See the top of the left side of the following diagram, showing the mDIS implementation technologies and the mDIS client-server architecture:

mDIS from a technical perspective: implementation technologies used.

The diagram mentions "various technologies" used by web browsers. These include, for instance, Local Storage. Not mentioned above, but still very important, is the build-system, which compiles the mDIS Frontend and tests the Backend.

Software developers and system administrators often prefer more detailed views, including the mDIS build process:

mDIS High-Level Architecture

The following diagram shows the mDIS Frontend, Backend, and related tools, such as a low-level database query tool:

mDIS implementation, high-level components.

This high-level system design is also known as a "LAMP web service stack".

While the primary access method is via a web browser, other mechanisms exist. The database is accessible via an SQL API, conveniently available through the adminer.php web page. Command-line access via SSH is also available for system administration.

mDIS Build System

The following diagram shows a simplified view of the mDIS build system:

The mDIS Build System, Frontend and Backend

Yii migrations and other CLI commands using the yii tool are also used during installation and configuration (not shown in the figure). yii is a lightweight Symfony 5 Console application and follows conventions used in the wider PHP community.

Frontend

The mDIS frontend, what the user sees in their web browser, is based on Vue.js.

Backend

The server-side is a PHP7-based application using the Yii2 Framework, Apache Webserver, and a MySQL Database. You can use different web servers (Nginx, Microsoft IIS) and databases (SQLite, PostgreSQL, MSSQL, Oracle, etc.) supported by Yii2, potentially with extensions (Informix, IBM DB2, Firebird, MariaDB...). We develop and test on MySQL 5.7+ and Apache 2.4+. VueJS development requires NodeJS, and installing Vue dependencies will set up the Webpack devServer on your development environment.

For VueJS development, NodeJS is required. Installation of the Vue dependencies will put the Webpack devServer on your development environment.

Webserver Configuration

Apache 2.4

For a single mDIS instance in production, no non-standard Apache modules are required besides mod_php (e.g., mod_php7 or mod_php8). For HTTPS, you need an HTTPS certificate installed on your webserver (see extra document and the mDIS Installer scripts for hints regarding required PHP extensions; also see the 'PHP 7' section below).

Apache mod_rewrite Code

Yii2 requires minimal URL rewriting. Place the following standard directives in web/.htaccess or an appropriate Apache configuration file in /etc/apache2 (the Yii scaffolding process creates web/.htaccess during project initialization):

# redirect to index.html by default
DirectoryIndex index.html index.php

# if a directory or a file exists, use it directly
RewriteEngine on
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
# otherwise forward it to index.php
RewriteRule . index.php
AddDefaultCharset UTF-8

Correct URL-rewriting allows running PHP scripts from web/ and placing static HTML pages there. Development mode, especially with Docker, may require further configuration (not covered here). For running Adminer on the same server, create an Apache alias or call adminer.php directly.

If you run the Adminer database management tool on the same server as mDIS, you should create an Alias in one of the config files in /etc/apache, or call adminer.php directly.

Installing mDIS side-by-side on a single host

To install multiple mDIS instances alongside each other, or with a specific URL path (e.g., /mdis/mysite), use Apache with mod_proxy and mod_proxy_http enabled. For example, to have the requested URL as "http://dis-app.example.com/mdis/deadsea/" and the reverse proxy target as "http://dis-app.example.com:8111/", use this snippet in your Apache configuration (e.g., /etc/apache2/sites-available/dis-app-example.com.conf on Linux):

ProxyRequests Off
ProxyPass /mdis/deadsea http://dis-app.example.com:8111/dis-app/web
<Location /mdis/deadsea/>
  ProxyPassReverse /dis-app/web
  ProxyHTMLEnable On
  RequestHeader unset Accept-Encoding
  ProxyHTMLURLMap http://dis-app.example.com:8111/mdis/deadsea/ /
  ProxyHTMLURLMap /dis-app/web /
</Location>

This section is still To be completed (TBC). More details on configuring multiple mDIS instances will be added later.

PHP Configuration

For a working mDIS PHP 7 and PHP 8 configuration, see this phpinfo() output. This static snapshot from a development server indicates the required PHP extensions. (Removed, sensitive data)

Yii UrlManager

The urlManager section in backend/config/web.php defines main mDIS URL paths:

'urlManager' => [
            'enablePrettyUrl' => true,
            'enableStrictParsing' => true,
            'showScriptName' => false,
            'rules' => [
                '/' => 'site/index',
                '/test' => 'site/test',
                '/site/error' => 'site/error',
                '<module:(rbac)>/<controller>/<action>' => 'rbac/<controller>/<action>', // role management
                '<module:(user)>/<controller>/<action>' => 'user/<controller>/<action>', // user management
                '<controller:(report)>/<reportName:[\w-]+>' => 'report/generate',        // reports
                '<controller:(files)>/<id:\d+>' => 'files/view',                         // file upload
                '<controller:(files)>/original/<id:\d+>' => 'files/view-original',       // file assignment
                '<controller:(files)>/<action>' => 'files/<action>',
                // csv file importer
                '<controller:(terminal)>/<action>' => 'terminal/<action>',
            ],
        ],

This overview shows the main routes defined in the Web layer. Also check backend/modules/api/Module.php and backend/modules/cg/Module.php for URLManager rules mapping REST API calls.

Write Permissions

Grant write permissions for the webserver user to: backend/dis_templates/forms/, backend/dis_templates/models/, backend/forms/, backend/models/, src/forms/. See also the "for developers" documentation.

Installation with Virtual Box

See the Installation with Virtual Box page. A Vagrant based installer is also available. See the mDIS-installer README page for details.

Native Installation on Linux, or with Docker

See the Native Installation on Linux page. Docker configuration will be documented in a later release.

Template Manager

The mDIS template manager configures data entry forms and database tables. See the Intro and more pages.

Console Commands

This section describes maintenance commands for an existing mDIS installation, such as npm run ... or yii migrate/fresh.

npm run build

This command creates a production build of the mDIS frontend, copying *.js, *.css files, and images to web/ after deleting older versions. See the Developer page.

Directory `web`

Avoid manually deleting web/ contents. Some files there might be static assets not served by mDIS but are still important.

Internally, the node package manager npm relies on the bundler webpack to bundle code, transform code, and incorporate assets. Webpack handles approximately the following subtasks:

• Minification, concatenation, code optimizations
• Reasonable node shims to reduce package size
• Chunk hashing for cache busting
• vue-loader for compiling single-file components
• css-loader
• CSS Modules support
• extract-text plugin
• post-css plugin
• sass-loader, stylus-loader and less-loader support
• babel-loader for modern JavaScript support
• eslint-loader
• UglifyJS
• optimize-css-assets
• pre-load plugin
• HtmlWebpackPlugin
• CommonsChunkPlugin
• InlineSourcePlugin

All of this happens automatically, after configuring these build-system components once. Then it requires no input from the mDIS Admin.

Node Module @vue/cli has Webpack built-in as a dev-dependency and calls Webpack. Webpack gets installed when mDIS and Vue are installed. However, the webpack-cli module is not installed by default.

For developers

Custom tasks are not included in the list above. Our custom tasks are:

• injecting the git-tag and the build number into the mDIS dashboard for display.
• creating sourcemaps during a dev build.
• proxy configs for the Webpack dev server (for accessing dockerized backend services)

The relevant config files are vue.config.js and babel.config.js. Presumably Webpack also has more (internal) config files deep inside the node_modules/ directory.

See also directories node_modules/webpack* and node_modules/@vue, node_modules/vue*, and (less so) node_modules/@vue/cli-service/webpack.config.js. The Babel preset is @vue/app which is a specialization of @babel/preset-env. Babel is also in directory node_modules.

Run vue ui dashboard from command line with a command similar to this one:

vue ui -H localhost -p 8003 --quiet --headless

Older dev-docs in IG Bremen wiki

TBC

Yii Migrations

About Migrations

Read this section about command line tools first

See the console runners section and the migrations section on the developer page.

yii migrate scripts handle initialization tasks, crucial during development, setup, reconfiguration, or rebuilding. Rebuilding might be necessary after database schema updates, initial setup, or data purging. Migration scripts are in /backend/migrations. Forgetting to apply migrations can lead to errors.

Rebuilding the mDIS database

This script applies Yii migrations and custom seed scripts:

The following code empties all tables, deletes all tables and users. Then it rebuilds everything in the correct order, loads a database dump with example data, and sets some default permissions.

# (Optional: Enter the Docker container if applicable)

./yii migrate/fresh --interactive=0  # Drops existing tables (use cautiously)

# Additional tasks
./yii seed/users
./yii seed/example-dump
./yii seed/list-values
./yii seed/widgets

./yii seed/form-permissions # probably obsolete, unnecessary

# optional: promote preferred users/logins to _administrator_-role.
# - Pure SQL: execute this inside Adminer
#     update auth_assignment
#     set item_name = '_administrator_'
#     where user_id = select max(id) from user where username = 'superstar';
# TODO: write a migration script for this

The last 3 seed/ tasks are defined in backend/commands/, e.g. in file SeedController.php and in file backend/modules/api/common/controllers/FormController.php, method updateAccessRights().

The ./yii seed/form-permissions task tries to analyze existing PHP Model files and Forms found in the backend directory, created during previous specializations, and tries to assign appropriate edit permissions to the new users just created with seed/users.

The yii script runner is stored at the top level of the mDIS app. Change into this directory and call it as ./yii .... On Windows, use ./yii.bat ....

TBC

yii fix-data tasks

yii seed tasks

yii migrate/fresh tasks

Updating third-party code

Why updating third-party code?

Note

Both client-side and server-side mDIS components rely heavily on open-source code requiring periodic updates by an administrator.

Upgrades should be tested in a development environment, then on a staging server, and finally deployed to production.

In practice

• Internet access is recommended for updates.
• mDIS is ~70% third-party code. Updates are frequent. Project managers and developers should establish update procedures.

Checking for updates:

• Client-side: npm outdated
• Server-side: composer outdated
• Linux OS: apt list --upgradable
• Virtual Box: Update check via GUI: Help menu / Check For Updates
• Git repository: git fetch --dry-run

Performing updates:

• Client-side: npm update
• Server-side: composer update
• Linux OS: sudo apt-get upgrade
• Virtual Box GUI: Help menu / Check For Updates, then follow the prompts to install any available updates
• Git repository: git pull

Upgrading Client Side (production environment)

In a production environment, all JavaScript dependencies are packaged together (by Webpack, by npm run build, started by the mDIS developer) into a single file, web/assets/js/app.<cachebuster-id>.js. This file might load several other chunk*.js files. Together, they occupy very little space, about 2 MB in the web/assets/js directory of the mDIS source code tree.

For updating or restoring the front end, i.e., the code that runs inside the browser of the mDIS user, it is therefore necessary to copy the contents of the web/ folder from a development machine (or from any storage location) into the web/ folder of the production machine.

Upgrading Client Side (development environment)

See the "For developers" documentation, as it's more complex.

Upgrading Server Side (composer)

Third-party PHP code is in backend/vendor/. Use composer outdated to check for updates and composer upgrade to perform them. composer list lists available commands. composer help provides more details. Verify functionality after upgrades.

MySQL Administration

The mysql console client

mDIS administrators might need to interact directly with the MySQL/MariaDB server using command-line tools like mysql (interactive SQL) and mysqldump (backups), web-based tools like Adminer, PhpMyAdmin, or MySQL Workbench. This is useful for backups, restores, data exports, and performance tuning. See the mysql-backup-and-restore page. Credentials for VirtualBox instances are in dot-files in the $HOME directory of the mdis or vagrant users and in /root.

How to install Adminer inside the virtual machine

In a nutshell:

Download adminer-latest.php and place it in the web/ subdirectory, or install it persistently using the provided shell script. Update Adminer by replacing the adminer.php file.

For a more detailed explanation, see the following sections.

Adminer is a web-based admin tool for MySQL. Adminer consists of a single big PHP file.

You can simply download Adminer-latest.php and put this in the web/ subdirectory:

curl -sSL http://www.adminer.org/latest.php -o web/adminer.php

Alternatively, run this shell script to install Adminer persistently in the /usr directory:

#!/bin/sh
sudo mkdir /usr/share/adminer/
# download 'adminer.php' 1-page standalone php script
sudo wget "http://www.adminer.org/latest.php" -O /usr/share/adminer/latest.php
sudo ln -s /usr/share/adminer/latest.php /usr/share/adminer/adminer.php
# download 'nette' design = 1 css file
sudo wget -nd https://raw.github.com/vrana/adminer/master/designs/nette/adminer.css -O /usr/share/adminer/adminer.css
# create new apache config file
echo "Alias /adminer.php /usr/share/adminer/adminer.php" | sudo tee /etc/apache2/conf-available/adminer.conf
sudo bash -c 'echo "Alias /adminer.css /usr/share/adminer/adminer.css" >> /etc/apache2/conf-available/adminer.conf'
# load config persistently and restart apache
sudo a2enconf adminer.conf
sudo systemctl reload apache2

Occasionally you might need to update the adminer.php file. You can do so by simply replacing the file in the respective directory:

# use appropriate, more recent version numbers
# (these are from 2018/2019)
mv adminer.php adminer.4.6.2.php
curl -LO https://github.com/vrana/adminer/releases/download/v4.7.3/adminer-4.7.3.php
cp adminer-4.7.3.php adminer.php

The Adminer web interface can do most administrative tasks well. However, it cannot rename MySQL foreign keys, for example. For that, you would need a different tool, perhaps phpMyAdmin or HeidiSQL.

Adminer Plugins

Maybe one of the many free Adminer-Plugins can do the missing task.

We recommend also installing the following Adminer plugins:

With Priority 1: floatThead, suggest-tablefields, restore-menu-scroll.
Optional: hideableColumns, dump-json, fk-disable, json-column, stickyColumns, AdminerDumpArray, AdminerDumpMarkdownDict.

restore-menu-scroll: This plugin restores the scroll position of the left (sidebar) menu after a page reload or submitting an SQL query. The plugin needs to be patched to work with the latest Adminer version v4.8.1.

Adminer GUI Tips and Tricks

Pressing Alt while clicking a column header hides the column (requires the patched "hideableColumns" plugin).

Troubleshooting

Frontend problems

• Watch the texts in yellow notification popups and try to understand their meaning. However, often the yellow balloon is too small to display longer error texts, and it disappears after 10 seconds. Therefore:

• Hit the F12 function key on your keyboard (or CTRL-Shift-I on Windows/Linux, or Command+Option+I on a Mac) to open the console of the browser's developer tools. Hit the "Console" tab in the developer tools window. There the same error should be visible, as in the yellow notification box. Also check the "Network" tab to see if any files are missing after mDIS unsuccessfully tried to load them in the background. Loading in the background is expected behavior. mDIS does this with an internal component called "Axios". This "axios" name might also appear in the error texts. But errors or missing files should not appear.

• If you are an administrator with access to the backend, you might run npm run build. This creates/transpiles a new frontend. Doing so might delete older cruft. This is a last-resort measure.

• The Node.js JavaScript runtime is required to build the mDIS frontend. Node.js compiles Vue code to compact JavaScript code.

Which Node.js version?

Use a Node.js LTS version (e.g., v12, v14, v16, v18, v20).

Backend problems

If you have Shell/Console access to the backend, check log files in backend/runtime/log, especially app.log. Use grep error app.log to filter for errors.

Related

1. Installation with Virtual Box
2. Native Installation on Linux, or (more common) with Docker
3. See also this GitLab Repository in particular this detailed description
4. Using the Template Manager
   • See extra Template Manager page
   • See also "For mDIS Developers" page, and references in there
5. OS-Updates, etc
6. Docker, ICDP-internal with links to even more documents
7. Developers page