Configuration¶

Settings¶

Settings are in-app administration interface for changing various behavior and look. An adminitrator can navigate to settings from the main (left) menu. (since 2.2.0)

It allows to configure (among others):

toggle optional DSW features including registration or configurable questionnaire visibility by users
external authentication using OpenID standard
set up information texts and dashboard shown in the client (before login, after login, etc.)
connection to registry
feedback and support links and repository
organization details (such as name, ID, and pre-configured affiliations)

OpenID¶

To configure OpenID service for external authentication, you need to have the following

Client ID and Client secret for the service
Root URL of the service (you can verify it that this returns valid JSON: <URL>/.well-known/openid-configuration

You can test the OpenID service using official OpenID Connect Playground

Configuration files¶

Files are used for setting the server-side configuration.

Server¶

Server configuration is done using YAML format. We provide examples directly in config folder of the repository https://github.com/ds-wizard/engine-backend. For tests there exist special versions suffixed by -test.

Build Info¶

A build configuration (build-info.yml file) contains information for the build of the application. This configuration can be created simply by running a prepared script build-info.sh. Normally this script is run by a CI Tool (Travis CI) during build process.

Important

If you build server app locally, you should also run build-info.sh otherwise your app will show bad build information and version.

Application¶

The mail configuration file that provide a lot of options and contains necessary settings for server to be running properly. You can see the example application.yml below. Next, all options are described in detail.

application.yml¶

general:
  environment: Production
  serverPort: 3000
  secret: TteM1a0QkGe6ED5OvihPkA82LmwkWU6f
  clientUrl: http://localhost:8080
  serviceToken: NlQSNbGvh7EtcpinGnHE9g91
  integrationConfig: config/integration.yml

database:
  host: mongo
  databaseName: dsw-server
  port: 27017
  authEnabled: false
  username:
  password:

messaging:
  host: rabbitmq
  port: 5672
  username: rabbit
  password: password
  vhost: /

mail:
  enabled: false
  name:
  email:
  host:
  port:
  ssl:
  authEnabled:
  username:
  password:

analytics:
  enabled: false
  email:

Section aside General may be omitted and default values will be used for whole sections (typically disabled, see below).

General¶

Configuration related to general operations of the server application.

environment¶

Type: Enumeration [Production, Staging, Development, Test]
Default: Production

Environment that your deployment is using. This affects which migrations are used and other minor things can be different in various environments.

serverPort¶

Type: Integer [0-65535]
Default: 3000

Port that will be the web server listening on.

secret¶

Type: String

Secret string of 32 characters for encrypting configuration in database and JWT tokens.

clientUrl¶

Type: URI
Default: "" (empty)

Address of client application (e.g. https://localhost:8080).

serviceToken¶

Type: String
Default: "" (empty)

Randomly generated string for API authentication of service endpoints.

integrationConfig¶

Type: String
Default: "engine-wizard/config/integration.yml"

Filename or whole path of integration configuration file (see Integration). The path is relative to the working directory from where the Wizard runs.

Database¶

Configuration of connection to MongoDB database.

host¶

Type: String
Default: "mongo"

Hostname or IP address of the server running MongoDB.

port¶

Type: Integer [0-65535]
Default: 27017

Port that is used for MongoDB on the server (usually 27017).

databaseName¶

Type: String
Default: "dsw-server"

Name of the database for DS Wizard within MongoDB.

authEnabled¶

Type: Boolean
Default: false

Whether authentication is enabled on MongoDB server and is required to connect to the database.

username¶

Type: String
Default: "" (empty)

Username for authentication to database connection (if authEnabled is true).

password¶

Type: String
Default: "" (empty)

Password for authentication to database connection (if authEnabled is true).

RabbitMQ¶

Configuration of connection to RabbitMQ.

host¶

Type: String
Default: "rabbitmq"

Hostname or IP address of the server running RabbitMQ.

port¶

Type: Integer [0-65535]
Default: 5672

Port that is used for RabbitMQ on the server (usually 5672).

username¶

Type: String
Default: "guest"

Username for authentication to RabbitMQ connection.

password¶

Type: String
Default: "guest"

Password for authentication to RabbitMQ connection.

vhost¶

Type: String
Default: "/"

Virtual host on RabbitMQ server (see RabbitMQ docs).

Mail¶

Configuration for sending emails (such as registration activation or for forgotten password mechanism) from the DS Wizard using SMTP connection.

Tip

You should enable emails unless you test the application for yourself only. It is used for email confirmations and mechanism of resetting password.

enabled¶

Type: Boolean
Default: true

If emails will be sent and SMTP configured.

name¶

Type: String

Name of the DS Wizard instance that will be used as “senders name” in email headers.

email¶

Type: String
Default: "" (empty)

Email address from which the emails will be sent.

host¶

Type: String
Default: "" (empty)

Hostname or IP address of SMTP server.

port¶

Type: Integer [0-65535]
Default: 465 (or 25 based on ssl option)

Port that is used for SMTP on the server (usually 25 for plain or 465 for SSL).

ssl¶

Type: Boolean
Default: false

If SMTP connection is encrypted via SSL (we highly recommend this).

authEnabled¶

Type: Boolean
Default: false

If authentication using username and password should be used for SMTP.

username¶

Type: String
Default: "" (empty)

Username for the SMTP connection.

password¶

Type: String
Default: "" (empty)

Password for the SMTP connection.

Analytics¶

Configuration of analytic/informational emails for administrators, e.g., that a new user registered into the DS Wizard.

enabled¶

Type: Boolean
Default: false

If analytic emails should be sent.

email¶

Type: String
Default: "" (empty)

Target email address where analytics to which will be sent.

Integration¶

Integrations in the DS Wizard are using external APIs and you might need some configured variables such as API keys or endpoints. For example, integration with ID dbase might use following configuration.

integration.yml¶

dbase:
  apiKey: topSecretDBaseApiKey
  apiUrl: https://api.dbase.example:10666
  someConfig: someValue4Integration

There can be multiple integrations configured in single file. These can be used then when setting up the integration in the Editor as ${apiKey}, ${apiUrl}, etc. More about integrations can be found in separate Integrations documentation.

Note

Different knowledge models may use different variable naming, please read the information in README to find out what is required. We recommend authors to stick with apiKey and apiUrl variables as our convention.

Client¶

If you are running the client app using “Via Docker”, the all you need is to specify API_URL environment variable inside docker-compose.yml. In case you want to run the client locally, you need to create a config.js file in the project root:

config.js¶

window.dsw = {
    apiUrl: 'http://localhost:3000'
}

Client also provides wide variety of style customizations using SASS variables. Then you can mount it as volumes in case Docker as well:

volumes:
  # mount SCSS file
  - /path/to/extras.scss:/src/scss/customizations/_extras.scss
  - /path/to/overrides.scss:/src/scss/customizations/_overrides.scss
  - /path/to/variables.scss:/src/scss/customizations/_variables.scss
  # mount other assets, you can then refere them from scss using '/assets/...'
  - /path/to/assets:/usr/share/nginx/html/assets

_extras.scss = This file is loaded before all other styles. You can use it, for example, to define new styles or import fonts.
_overrides.scss = This file is loaded after all other styles. You can use it to override existing styles.
_variables.scss = A lot of values related to styles are defined as variables. The easiest way to customize the style is to define new values for these variables using this file.

For more information about variables and assets, visit Theming Bootstrap. The color of illustrations can be adjusted using $illustrations-color variable.

Document Worker¶

Configuration of document worker must match with server configuration. In docker, the default moint point is set to /app/config.yml but it can be overriden using environment variable DOCWORKER_CONFIG. Similarly, working directory can be changed from the default /tmp/docworker using DOCWORKER_WORKDIR.

docworker.yml¶

mongo:
  host: localhost
  port: 27017
  database: test_db
  collection: documents
  fs_collection: documentFs
  auth:
    username:
    password:
    database: (database)
    mechanism: SCRAM-SHA-256

mq:
  host: localhost
  port: 5672
  vhost: /
  queue: test_queue
  auth:
    username:
    password:

logging:
  level: INFO

externals:
  pandoc:
    executable: pandoc
    args: --standalone
    timeout:
  wkhtmltopdf:
    executable: wkhtmltopdf
    args:
    timeout:

mongo¶

host¶

Type: String
Default: "localhost"

Hostname or IP address of the server running MongoDB.

port¶

Type: Integer [0-65535]
Default: 27017

Port that is used for MongoDB on the server (usually 27017).

database¶

Type: String

Name of the database for DS Wizard within MongoDB.

collection¶

Type: String
Default: documents (optional)

Name of the collection for documents.

fs_collection¶

Type: String
Default: documentFs (optional)

Name of the collection for files.

templates_collection¶

Type: String
Default: templates (optional)

Name of the collection for templates.

assets_fs_collection¶

Type: String
Default: templateAssetFs (optional)

Name of the collection for template assets.

auth.username¶

Type: String
Default: None (optional)

Username for authentication to database connection (will be used if set).

auth.password¶

Type: String
Default: None (optional)

Password for authentication to database connection (will be used if set).

auth.database¶

Type: String
Default: <database> (optional)

Authentication database used for MongoDB, defaults to the same value provided in database option.

auth.mechanism¶

Type: String
Default: "SCRAM-SHA-256" (optional)

Authentication mechanism used for MongoDB.

mq¶

host¶

Type: String
Default: "localhost"

Hostname or IP address of the server running RabbitMQ.

port¶

Type: Integer [0-65535]
Default: 5672

Port that is used for RabbitMQ on the server (usually 5672).

vhost¶

Type: String
Default: "/"

Virtual host on RabbitMQ server (see RabbitMQ docs).

queue¶

Type: String

Name of queue used for passing document jobs (typically document.generation).

auth.username¶

Type: String
Default: None (optional)

Username for authentication to RabbitMQ connection (if any).

auth.password¶

Type: String
Default: None (optional)

Password for authentication to RabbitMQ connection (if any).

logging¶

level¶

Type: String
Default: "WARNING"

Name of logging level (use names of Python logging levels).

pandoc¶

executable¶

Type: String
Default: "pandoc"

Executable command for running Pandoc (can be aliased or defined with absolute path).

args¶

Type: String
Default: "--standalone"

Default arguments used for all Pandoc calls.

timeout¶

Type: Float
Default: None (optional)

Timeout for Pandoc subprocess call.

wkhtmltopdf¶

executable¶

Type: String
Default: "pandoc"

Executable command for running WkHtmlToPdf (can be aliased or defined with absolute path).

args¶

Type: String
Default: "" (empty)

Default arguments used for all WkHtmlToPdf calls.

timeout¶

Type: Float
Default: None (optional)

Timeout for WkHtmlToPdf subprocess call.

DMP templates¶

You can freely customize and style templates of DMPs (filled questionnaires). HTML and CSS knowledge is required and for doing more complex templates that use some conditions, loops, or macros, knowledge of Jinja templating language (pure Python implementation) is useful. On startup, DSW in Docker can load templates to database from /application/engine-wizard/templates.

To load custom templates from the file system, Docker container must have set environment variables ENABLE_TEMPLATE_LOAD `` to ``1 and SERVICE_TOKEN according to the configuration.

Template files¶

The basic structure is following:

templates/dmp/my-template/template.json = metadata about the template (must be named template.json)
templates/dmp/my-template/... = other template files (and sub-directories)

Templates allow you to iterate through questions and answers and find what you need to compose some output. For example, you can generate longer text based on answers of various questions by knowing its texts or UUIDs. To the template, object ctx (document context) is injected and can be used as variable - for information about its structure, browse current default template or visit source code.

You can have multiple DMP templates and users will be able to pick one of them when exporting a filled questionnaire. Each template must have its metadata JSON file that contain id (composed of organizationId, templateId, and version), name to be displayed when picking a template, description, readme, and license:

template.json¶

{
  "id": "dsw:default:1.0.0",
  "organizationId": "dsw",
  "templateId": "default",
  "version": "1.0.0",
  "name": "Common DSW Template",
  "description": "This is a default template",
  "readme": "# Default template",
  "license": "Apache-2.0",
  "recommendedPackageId": "dsw:root:2.2.0",
  "allowedPackages": [
      {
        "orgId": null,
        "kmId": null,
        "minVersion": null,
        "maxVersion": null
      }
  ],
  "formats": [
    {
      "uuid": "d3e98eb6-344d-481f-8e37-6a67b6cd1ad2",
      "name": "JSON Data",
      "shortName": "json",
      "icon": "far fa-file",
      "color": "#f15a24",
      "steps": [
        {
          "name": "json",
          "options": {}
        }
      ]
    },
    {
      "uuid": "a9293d08-59a4-4e6b-ae62-7a6a570b031c",
      "name": "HTML Document",
      "shortName": "html",
      "icon": "far fa-file-code",
      "color": "#f15a24",
      "steps": [
        {
          "name": "jinja",
          "options": {
            "template": "default.html.j2",
            "content-type": "text/html",
            "extension": "html"
          }
        }
      ]
    },
    {
      "uuid": "68c26e34-5e77-4e15-9bf7-06ff92582257",
      "name": "PDF Document",
      "shortName": "pdf",
      "icon": "far fa-file-pdf",
      "color": "#f15a24",
      "steps": [
        {
          "name": "jinja",
          "options": {
            "template": "default.html.j2",
            "content-type": "text/html",
            "extension": "html"
          }
        },
        {
          "name": "wkhtmltopdf",
          "options": {
            "args": ""
          }
        }
      ]
    },
    {
      "uuid": "f4bd941a-dfbe-4226-a1fc-200fb5269311",
      "name": "MS Word Document",
      "shortName": "docx",
      "icon": "far fa-file-word",
      "color": "#f15a24",
      "steps": [
        {
          "name": "jinja",
          "options": {
            "template": "default.html.j2",
            "content-type": "text/html",
            "extension": "html"
          }
        },
        {
          "name": "pandoc",
          "options": {
            "from": "html",
            "to": "docx"
          }
        }
      ]
    }
  ]
}

For allowedKMs, you can specify a list of knowledge models that the template can be used with (for example, when it is bound to its questions). You can even bound minimal and maximal version or let it unbound using null value. Using wkhtmltopdf and pandoc, you can specify template-related extra arguments for calls of those commands in case of document conversion.

Each template has information what formats and how are provided. A format has its own uuid, name, and icon for UI. Then there is a list of steps how to produce the output. Types of steps:

json = only dumps document context as JSON; no options; must be first step; produces a JSON document
jinja = uses Jinja2 templates to produce a document starting by root file specified in options.template (full Jinja2 can be used), options.content-type and options.extension must be used to specify type of the output; must be first step
wkhtmltopdf - runs wkhtmltopdf; options.args can be used for additional arguments and options to run it; must be after step producing a HTML document; produces a PDF document
pandoc - runs Pandoc; options.from and options.to define from which and to which type the transformation is used (use names according to docs), additionally options.args can be used for additional arguments and options to run it; must be used after step producing a document conforming options.from; produces a document according to options.to
rdflib-convert - uses rdflib to load graph from format specified by options.from and then serialize it to format specified by options.to; supported formats are: ttl, n3, rdf (RDF/XML), nt, trig, and jsonld

Graphics and scripts¶

If you want to include some graphics or JavaScript, we recommend you to put it directly into the HTML template file. In case of graphics, use base64 encoded content (suitable for smaller images like icons and logos):

<img src="data:image/png;base64, iVBORw0KGgoAAAANSU
 hEUgAAAAUAAAAFCAYAAACNbyblAAAAHElEQVQI12P4//8/w38GI
 AXDIBKE0DHxgljNBAAO9TXL0Y4OHwAAAABJRU5ErkJggg=="
 alt="Red dot" />

Alternatively, you can of course reference picture that is accessible online. For JavaScript, again you can put there directly some script or reference it, for example, from some CDN:

<style type="text/javascript" src="https://code.jquery.com/jquery-3.3.1.min.js"></style>
<style type="text/javascript">
  jQuery(".btn").click(function(){
    jQuery(this).toggleClass(".clicked");
  });
</style>

You can split your template code into multiple files and the use include directive that opens the file and inserts its content where the directive is placed - like we do for including CSS style in HTML template (only one complex HTML file is generated in the end):

<head>
  <title>Data Management Plan</title>
  <meta charset="utf-8">
  <style>{% include "root.css" %}</style>
</head>

Docker deployment¶

If you deploy the DS Wizard using Docker, you can mount custom files to templates/dmp folder and overwrite default template within docker-compose.yml:

server:
  image: datastewardshipwizard/wizard-server
  restart: always
  ports:
    - 3000:3000
  volumes:
    - /dsw/server/application.yml:/application/engine-wizard/config/application.yml
    - /dsw/templates/dmp:/application/engine-wizard/templates/dmp:ro
  # ... (continued)

Email templates¶

Similarly to DMP templates, you can customize templates for emails sent by the Wizard located in templates/mail folder. It also uses Jinja templating language (for email templates we use its implementation called Ginger). And you can create HTML template, Plain Text template, add attachments, and add inline images (which can be used inside the HTML using Content-ID equal to the filename).

Templates structure¶

The structure is following:

templates/mail/_common = layout, styles, common files
templates/mail/_common/attachments = attachments for all emails
templates/mail/_common/images = inline images for all emails
templates/mail/<name> = templates specific for this email type, should contain message.html.j2 and message.txt.j2 files (or at least one of them, mail servers prefer both variants)
templates/mail/<name>/attachments = attachments specific for email type
templates/mail/<name>/images = inline images specific for email type

All attachments are loaded from the template-specific and common folders and included to email with detected MIME type. It similarly works for inline images but those are not displayed as attachments just as related part to HTML part (if present). We highly recommend to use ASCII-only names without whitespaces and with standard extensions. Also, sending minimum amount of data via email is suggested.

Templates variables¶

All templates are provided also with variables:

appTitle = from the configuration appTitle
clientAddress = from the configuration clientUrl
mailName = from the configuration name
user = user (subject of an email), structure with attributes accessible via . (dot, e.g. user.name)

Email types¶

Currently, there are following types of mail:

registrationConfirmation = email sent to user after registration to verify email address, contains activationLink variable
registrationCreatedAnalytics = email sent to address specified in the configuration about registration of a new user (see Analytics config)
resetPassword = email sent to user when requests resetting a password, contains resetLink variable

Docker deployment¶

Including own email templates while using dockerized Wizard is practically the same as for DMP templates. You can also bind whole templates/mail folder (or even templates if want to change both):

server:
  image: datastewardshipwizard/wizard-server
  restart: always
  ports:
    - 3000:3000
  volumes:
    - /dsw/application.yml:/application/engine-wizard/config/application.yml
    - /dsw/templates/mail:/application/engine-wizard/templates/mail:ro
# ... (continued)