Configuration

Settings

Settings are in-app administration interface for changing various behavior and look. An adminitrator can navigate to settings from the main (left) menu. (since 2.2.0)

It allows to configure (among others):

  • toggle optional DSW features including registration

  • external authentication using OpenID standard

  • set up information texts and dashboard shown in the client (before login, after login, etc.)

  • connection to registry

  • feedback and support links and repository

  • organization details (such as name, ID, and pre-configured affiliations)

OpenID

To configure OpenID service for external authentication, you need to have the following

  • Client ID and Client secret for the service

  • Root URL of the service (you can verify it that this returns valid JSON: <URL>/.well-known/openid-configuration

You can test the OpenID service using official OpenID Connect Playground

Configuration files

Files are used for setting the server-side configuration.

Server

Server configuration is done using YAML format. We provide examples directly in config folder of the repository https://github.com/ds-wizard/engine-backend. For tests there exist special versions suffixed by -test.

Build Info

A build configuration (build-info.yml file) contains information for the build of the application. This configuration can be created simply by running a prepared script build-info.sh. Normally this script is run by a CI Tool (Travis CI) during build process.

Important

If you build server app locally, you should also run build-info.sh otherwise your app will show bad build information and version.

Application

The mail configuration file that provide a lot of options and contains necessary settings for server to be running properly. You can see the example application.yml below. Next, all options are described in detail.

application.yml
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
general:
  environment: Production
  serverPort: 3000
  secret: TteM1a0QkGe6ED5OvihPkA82LmwkWU6f
  clientUrl: http://localhost:8080
  serviceToken: NlQSNbGvh7EtcpinGnHE9g91
  integrationConfig: config/integration.yml
  templateFolder: engine-wizard/templates

database:
  host: mongo
  databaseName: dsw-server
  port: 27017
  authEnabled: false
  username:
  password:

messaging:
  host: rabbitmq
  port: 5672
  username: rabbit
  password: password
  vhost: /

mail:
  enabled: false
  name:
  email:
  host:
  port:
  ssl:
  authEnabled:
  username:
  password:

analytics:
  enabled: false
  email:

Section aside General may be omitted and default values will be used for whole sections (typically disabled, see below).

General

Configuration related to general operations of the server application.

environment
Type

Enumeration [Production, Staging, Development, Test]

Default

Production

Environment that your deployment is using. This affects which migrations are used and other minor things can be different in various environments.

serverPort
Type

Integer [0-65535]

Default

3000

Port that will be the web server listening on.

secret
Type

String

Secret string of 32 characters for encrypting configuration in database and JWT tokens.

clientUrl
Type

URI

Default

"" (empty)

Address of client application (e.g. https://localhost:8080).

serviceToken
Type

String

Default

"" (empty)

Randomly generated string that matches configuration of Feedback synchronization component.

integrationConfig
Type

String

Default

"engine-wizard/config/integration.yml"

Filename or whole path of integration configuration file (see Integration). The path is relative to the working directory from where the Wizard runs.

templateFolder
Type

String

Default

"engine-wizard/templates"

Path to the folder where DMP and mail template are stored. The path is relative to the working directory from where the Wizard runs.

Database

Configuration of connection to MongoDB database.

host
Type

String

Default

"mongo"

Hostname or IP address of the server running MongoDB.

port
Type

Integer [0-65535]

Default

27017

Port that is used for MongoDB on the server (usually 27017).

databaseName
Type

String

Default

"dsw-server"

Name of the database for DS Wizard within MongoDB.

authEnabled
Type

Boolean

Default

false

Whether authentication is enabled on MongoDB server and is required to connect to the database.

username
Type

String

Default

"" (empty)

Username for authentication to database connection (if authEnabled is true).

password
Type

String

Default

"" (empty)

Password for authentication to database connection (if authEnabled is true).

RabbitMQ

Configuration of connection to RabbitMQ.

host
Type

String

Default

"rabbitmq"

Hostname or IP address of the server running RabbitMQ.

port
Type

Integer [0-65535]

Default

5672

Port that is used for RabbitMQ on the server (usually 5672).

username
Type

String

Default

"guest"

Username for authentication to RabbitMQ connection.

password
Type

String

Default

"guest"

Password for authentication to RabbitMQ connection.

vhost
Type

String

Default

"/"

Virtual host on RabbitMQ server (see RabbitMQ docs).

Mail

Configuration for sending emails (such as registration activation or for forgotten password mechanism) from the DS Wizard using SMTP connection.

Tip

You should enable emails unless you test the application for yourself only. It is used for email confirmations and mechanism of resetting password.

enabled
Type

Boolean

Default

true

If emails will be sent and SMTP configured.

name
Type

String

Name of the DS Wizard instance that will be used as “senders name” in email headers.

email
Type

String

Default

"" (empty)

Email address from which the emails will be sent.

host
Type

String

Default

"" (empty)

Hostname or IP address of SMTP server.

port
Type

Integer [0-65535]

Default

465 (or 25 based on ssl option)

Port that is used for SMTP on the server (usually 25 for plain or 465 for SSL).

ssl
Type

Boolean

Default

false

If SMTP connection is encrypted via SSL (we highly recommend this).

authEnabled
Type

Boolean

Default

false

If authentication using username and password should be used for SMTP.

username
Type

String

Default

"" (empty)

Username for the SMTP connection.

password
Type

String

Default

"" (empty)

Password for the SMTP connection.

Analytics

Configuration of analytic/informational emails for administrators, e.g., that a new user registered into the DS Wizard.

enabled
Type

Boolean

Default

false

If analytic emails should be sent.

email
Type

String

Default

"" (empty)

Target email address where analytics to which will be sent.

Integration

Integrations in the DS Wizard are using external APIs and you might need some configured variables such as API keys or endpoints. For example, integration with ID dbase might use following configuration.

integration.yml
1
2
3
4
dbase:
  apiKey: topSecretDBaseApiKey
  apiUrl: https://api.dbase.example:10666
  someConfig: someValue4Integration

There can be multiple integrations configured in single file. These can be used then when setting up the integration in the Editor as ${apiKey}, ${apiUrl}, etc. More about integrations can be found in separate Integrations documentation.

Note

Different knowledge models may use different variable naming, please read the information in README to find out what is required. We recommend authors to stick with apiKey and apiUrl variables as our convention.

Client

If you are running the client app using “Via Docker”, the all you need is to specify API_URL environment variable inside docker-compose.yml. In case you want to run the client locally, you need to create a config.js file in the project root:

config.js
1
2
3
window.dsw = {
    apiUrl: 'http://localhost:3000'
}

Client also provides wide variety of style customizations using SASS variables. Then you can mount it as volumes in case Docker as well:

volumes:
  # mount SCSS file
  - /path/to/extras.scss:/src/scss/customizations/_extras.scss
  - /path/to/overrides.scss:/src/scss/customizations/_overrides.scss
  - /path/to/variables.scss:/src/scss/customizations/_variables.scss
  # mount other assets, you can then refere them from scss using '/assets/...'
  - /path/to/assets:/usr/share/nginx/html/assets
  • _extras.scss = This file is loaded before all other styles. You can use it, for example, to define new styles or import fonts.

  • _overrides.scss = This file is loaded after all other styles. You can use it to override existing styles.

  • _variables.scss = A lot of values related to styles are defined as variables. The easiest way to customize the style is to define new values for these variables using this file.

For more information about variables and assets, visit Theming Bootstrap. The color of illustrations can be adjusted using $illustrations-color variable.

Document Worker

Configuration of document worker must match with server configuration.

docworker.cfg
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
[mongo]
host=mongo
port=27017
database=dsw-server
collection=documents
fs_collection=documentFs

[mq]
host=rabbitmq
port=5672
vhost=/
username=rabbit
password=password
queue=document.generation

[logging]
level=DEBUG

[pandoc]
executable=pandoc
args=--standalone
timeout=5

[wkhtmltopdf]
executable=wkhtmltopdf
args=
timeout=5

mongo

host
Type

String

Default

"localhost"

Hostname or IP address of the server running MongoDB.

port
Type

Integer [0-65535]

Default

27017

Port that is used for MongoDB on the server (usually 27017).

username
Type

String

Default

None (optional)

Username for authentication to database connection (will be used if set).

password
Type

String

Default

"" (optional)

Password for authentication to database connection (will be used if set).

database
Type

String

Name of the database for DS Wizard within MongoDB.

collection
Type

String

Name of the collection for documents (typically documents).

fs_collection
Type

String

Name of the collection for files (typically documentFs).

auth_database
Type

String

Default

<database> (optional)

Authentication database used for MongoDB, defaults to the same value provided in database option.

auth_mechanism
Type

String

Default

"SCRAM-SHA-256" (optional)

Authentication mechanism used for MongoDB.

mq

host
Type

String

Default

"localhost"

Hostname or IP address of the server running RabbitMQ.

port
Type

Integer [0-65535]

Default

5672

Port that is used for RabbitMQ on the server (usually 5672).

vhost
Type

String

Default

"/"

Virtual host on RabbitMQ server (see RabbitMQ docs).

username
Type

String

Default

None (optional)

Username for authentication to RabbitMQ connection (if any).

password
Type

String

Default

None (optional)

Password for authentication to RabbitMQ connection (if any).

queue
Type

String

Name of queue used for passing document jobs (typically document.generation).

logging

level
Type

String

Default

"WARNING"

Name of logging level (use names of Python logging levels).

pandoc

executable
Type

String

Default

"pandoc"

Executable command for running Pandoc (can be aliased or defined with absolute path).

args
Type

String

Default

"--standalone"

Default arguments used for all Pandoc calls.

timeout
Type

Float

Default

None (optional)

Timeout for Pandoc subprocess call.

wkhtmltopdf

executable
Type

String

Default

"pandoc"

Executable command for running WkHtmlToPdf (can be aliased or defined with absolute path).

args
Type

String

Default

"" (empty)

Default arguments used for all WkHtmlToPdf calls.

timeout
Type

Float

Default

None (optional)

Timeout for WkHtmlToPdf subprocess call.

Feedback synchronization

Because our feedback functionality is based on GitHub issues, it requires synchronization. If you are using our Docker image, all you have to do is define environment variables for the server image:

API_URL: "..."
SERVICE_TOKEN: "..."
ENABLE_CRON: "1"
ENABLE_CRON_FEEDBACK_SYNC: "1"

Locally, you need to set up cron job on your machine similarly. See our script for setting up the cron job.

DMP templates

You can freely customize and style templates of DMPs (filled questionnaires). HTML and CSS knowledge is required and for doing more complex templates that use some conditions, loops, or macros, knowledge of Jinja templating language (pure Python implementation) is useful. The location of templates root folder is configurable via templateFolder.

Template files

We provide currently basic default template (see here) but it is possible to get inpired and create more or edit it. The basic structure is following:

  • templates/dmp/my-template/template.json = metadata about the template (must be named template.json)

  • templates/dmp/my-template/... = other template files (and sub-directories)

Templates allow you to iterate through questions and answers and find what you need to compose some output. For example, you can generate longer text based on answers of various questions by knowing its texts or UUIDs. To the template, object ctx (document context) is injected and can be used as variable - for information about its structure, browse current default template or visit source code.

You can have multiple DMP templates and users will be able to pick one of them when exporting a filled questionnaire. Each template must have its metadata JSON file that contain random and unique UUID, name to be displayed when picking a template, and relative path to root file of the template:

template.json
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
{
  "uuid": "43a3fdd1-8535-42e0-81a7-5edbff296e65",
  "name": "Common DSW Template",
  "description": "This is a default template",
  "recommendedPackageId": "dsw:root:2.0.1",
  "allowedPackages": [
      {
        "orgId": null,
        "kmId": null,
        "minVersion": null,
        "maxVersion": null
      }
  ],
  "formats": [
    {
      "uuid": "d3e98eb6-344d-481f-8e37-6a67b6cd1ad2",
      "name": "JSON Data",
      "shortName": "json",
      "icon": "far fa-file",
      "color": "#f15a24",
      "steps": [
        {
          "name": "json",
          "options": {}
        }
      ]
    },
    {
      "uuid": "a9293d08-59a4-4e6b-ae62-7a6a570b031c",
      "name": "HTML Document",
      "shortName": "html",
      "icon": "far fa-file-code",
      "color": "#f15a24",
      "steps": [
        {
          "name": "jinja",
          "options": {
            "template": "default.html.j2",
            "content-type": "text/html",
            "extension": "html"
          }
        }
      ]
    },
    {
      "uuid": "68c26e34-5e77-4e15-9bf7-06ff92582257",
      "name": "PDF Document",
      "shortName": "pdf",
      "icon": "far fa-file-pdf",
      "color": "#f15a24",
      "steps": [
        {
          "name": "jinja",
          "options": {
            "template": "default.html.j2",
            "content-type": "text/html",
            "extension": "html"
          }
        },
        {
          "name": "wkhtmltopdf",
          "options": {
            "args": ""
          }
        }
      ]
    },
    {
      "uuid": "f4bd941a-dfbe-4226-a1fc-200fb5269311",
      "name": "MS Word Document",
      "shortName": "docx",
      "icon": "far fa-file-word",
      "color": "#f15a24",
      "steps": [
        {
          "name": "jinja",
          "options": {
            "template": "default.html.j2",
            "content-type": "text/html",
            "extension": "html"
          }
        },
        {
          "name": "pandoc",
          "options": {
            "from": "html",
            "to": "docx"
          }
        }
      ]
    }
  ]
}

For allowedKMs, you can specify a list of knowledge models that the template can be used with (for example, when it is bound to its questions). You can even bound minimal and maximal version or let it unbound using null value. Using wkhtmltopdf and pandoc, you can specify template-related extra arguments for calls of those commands in case of document conversion.

Each template has information what formats and how are provided. A format has its own uuid, name, and icon for UI. Then there is a list of steps how to produce the output. Types of steps:

  • json = only dumps document context as JSON; no options; must be first step; produces a JSON document

  • jinja = uses Jinja2 templates to produce a document starting by root file specified in options.template (full Jinja2 can be used), options.content-type and options.extension must be used to specify type of the output; must be first step

  • wkhtmltopdf - runs wkhtmltopdf; options.args can be used for additional arguments and options to run it; must be after step producing a HTML document; produces a PDF document

  • pandoc - runs Pandoc; options.from and options.to define from which and to which type the transformation is used (use names according to docs), additionally options.args can be used for additional arguments and options to run it; must be used after step producing a document conforming options.from; produces a document according to options.to

Graphics and scripts

If you want to include some graphics or JavaScript, we recommend you to put it directly into the HTML template file. In case of graphics, use base64 encoded content (suitable for smaller images like icons and logos):

<img src="data:image/png;base64, iVBORw0KGgoAAAANSU
 hEUgAAAAUAAAAFCAYAAACNbyblAAAAHElEQVQI12P4//8/w38GI
 AXDIBKE0DHxgljNBAAO9TXL0Y4OHwAAAABJRU5ErkJggg=="
 alt="Red dot" />

Alternatively, you can of course reference picture that is accessible online. For JavaScript, again you can put there directly some script or reference it, for example, from some CDN:

<style type="text/javascript" src="https://code.jquery.com/jquery-3.3.1.min.js"></style>
<style type="text/javascript">
  jQuery(".btn").click(function(){
    jQuery(this).toggleClass(".clicked");
  });
</style>

You can split your template code into multiple files and the use include directive that opens the file and inserts its content where the directive is placed - like we do for including CSS style in HTML template (only one complex HTML file is generated in the end):

<head>
  <title>Data Management Plan</title>
  <meta charset="utf-8">
  <style>{% include "root.css" %}</style>
</head>

Docker deployment

If you deploy the DS Wizard using Docker, you can mount custom files to templates/dmp folder and overwrite default template within docker-compose.yml:

server:
  image: datastewardshipwizard/wizard-server
  restart: always
  ports:
    - 3000:3000
  volumes:
    - /dsw/server/application.yml:/application/engine-wizard/config/application.yml
    - /dsw/templates/dmp:/application/engine-wizard/templates/dmp:ro
  # ... (continued)

docworker:
  # ...
  volumes:
   - /dsw/docworker/config.cfg:/app/config.cfg:ro
   - /dsw/templates/dmp:/app/templates:ro

Email templates

Similarly to DMP templates, you can customize templates for emails sent by the Wizard located in templates/mail folder. It also uses Jinja templating language (for email templates we use its implementation called Ginger). And you can create HTML template, Plain Text template, add attachments, and add inline images (which can be used inside the HTML using Content-ID equal to the filename). The location of templates root folder is configurable via templateFolder.

Templates structure

The structure is following:

  • templates/mail/_common = layout, styles, common files

  • templates/mail/_common/attachments = attachments for all emails

  • templates/mail/_common/images = inline images for all emails

  • templates/mail/<name> = templates specific for this email type, should contain message.html.j2 and message.txt.j2 files (or at least one of them, mail servers prefer both variants)

  • templates/mail/<name>/attachments = attachments specific for email type

  • templates/mail/<name>/images = inline images specific for email type

All attachments are loaded from the template-specific and common folders and included to email with detected MIME type. It similarly works for inline images but those are not displayed as attachments just as related part to HTML part (if present). We highly recommend to use ASCII-only names without whitespaces and with standard extensions. Also, sending minimum amount of data via email is suggested.

Templates variables

All templates are provided also with variables:

  • appTitle = from the configuration appTitle

  • clientAddress = from the configuration clientUrl

  • mailName = from the configuration name

  • user = user (subject of an email), structure with attributes accessible via . (dot, e.g. user.name)

Email types

Currently, there are following types of mail:

  • registrationConfirmation = email sent to user after registration to verify email address, contains activationLink variable

  • registrationCreatedAnalytics = email sent to address specified in the configuration about registration of a new user (see Analytics config)

  • resetPassword = email sent to user when requests resetting a password, contains resetLink variable

Docker deployment

Including own email templates while using dockerized Wizard is practically the same as for DMP templates. You can also bind whole templates/mail folder (or even templates if want to change both):

server:
  image: datastewardshipwizard/wizard-server
  restart: always
  ports:
    - 3000:3000
  volumes:
    - /dsw/application.yml:/application/engine-wizard/config/application.yml
    - /dsw/staging/templates/mail:/application/engine-wizard/templates/mail:ro
# ... (continued)