lora mox documentation

LoRa MOX DocumentationRelease 1.3.0

Magenta ApS

Oct 08, 2021

User Documentation

1 Introduction 1

2 Licensing 3

3 Content 5

4 Indices and tables 69

HTTP Routing Table 71

Index 73

i

CHAPTER 1

Introduction

This project contains an implementation of the OIO object model, used as a standard for data exchange by theDanish government, for use with a MOX messaging queue.

You can find the current MOX specification here:

http://www.kl.dk/ImageVaultFiles/id_55874/cf_202/MOX_specifikation_version_0.PDF

As an example, you can find the Organisation hierarchy here:

http://digitaliser.dk/resource/991439

In each installation of MOX, it is possible to only enable some of the hierarchies, but we provide the followingfour OIO hierarchies by default:

• Klassifikation

• Sag

• Dokument

• Organisation

1.1 Documentation

The official location for the documentation is:

• http://mox.readthedocs.io/

Please note that as a convention, all shell commands have been prefixed with a dollar-sign, or $, representing aprompt. You should exclude this when entering the command in your terminal.

1.2 Audience

This is a technical guide. It is split in two:

User documentation Contains documentation for the REST API and setup documentation. You are not expectedto have a profound knowledge of the system as such, but you do have to know your way in a Bash prompt— you should be able to change the Apache or Nginx configuration and e.g. disable or change the SSLcertificate on your own.

1

http://www.kl.dk/ImageVaultFiles/id_55874/cf_202/MOX_specifikation_version_0.PDF

http://digitaliser.dk/resource/991439

http://mox.readthedocs.io/

LoRa MOX Documentation, Release 1.3.0

Developer documentation Contains documentation for developers of the mox codebase and building and testinginstructions.

2 Chapter 1. Introduction

CHAPTER 2

Licensing

The MOX messaging queue, including the ActualState database, as found in this project is free software. You areentitled to use, study, modify and share it under the provisions of Version 2.0 of the Mozilla Public License asspecified in the LICENSE file.

This software was developed by Magenta ApS. For feedback, feel free to open an issue in the GitHub repository.

3

https://www.mozilla.org/MPL/2.0/

http://www.magenta.dk

https://github.com/magenta-aps/mox


4 Chapter 2. Licensing

CHAPTER 3

Content

3.1 High-level overview

On a high level the MOX actual state database consists of three server processes and several agents joining themtogether.

3.1.1 Server processes

PostgreSQL Database server providing the storage of the bi-temporal actual state database as well as validationand verification of the basic constraints.

Gunicorn WSGI server for the oio rest api. Ideally used with a frontend HTTP proxy in production.

RabbitMQ AMQP message broker providing interprocess communication between the various components.

3.2 MOX - infrastruktur for udveksling af offentlige data

MOX er en mekanisme, der vil tillade fagsystemer, der anvendes af det offentlige at udveksle data uden at behøveat integrere direkte til hinanden. I stedet behøver hvert fagsystem blot at integrere til en mox-agent, der ved atsende en besked til en såkaldt beskedfordeler kan sørge for, at interesserede fagsystemer kan bede om at få enopdateret kopi af de pågældende data.

MOX er som koncept opfundet og beskrevet af Kommunernes Landsforening i samarbejde med en lang rækkekommuner, og der er blandt andet en gennemgang af det her:

http://beta.rammearkitektur.dk/index.php/MOX_konceptet

Dette notat vil omhandle det MOX-system, som Magenta har udviklet i samarbejde Kommunernes Landsforening,Sager på Tværs og Frederiksberg Kommune, og forudsætter, at læseren selv i fornødent omfang gør sig bekendtmed det overordende MOX-koncept.

3.2.1 Overordnet arkitektur

MOX-systemet består grundlæggende af fem komponenter:

5

http://beta.rammearkitektur.dk/index.php/MOX_konceptet


• En MOX-agent, der kan modtage besked om ændring af data fra et fagsystem og sende ændringen videre tilen beskedfordeler.

• En beskedfordeler, der primært er et kø-interface, der for eksempel kan være (og i vores tilfælde er) baseretpå AMQP.

• En “OIO-REST”-agent, der lytter på en af beskedfordelerens køer og sender data-ændringerne videre til etserviceinterface, der vedligeholder de faktiske data.

• Et service-interface, der implementerer en standard-komponent fra OIOXML-standarden. Dette service-interface har form af et REST-interface, der er implementeret i Python. I øjeblikket understøtter MagentasMOX-system OIOXML-standarderne Organisation, Klassifikation, Sag og Dokument.

• En database, der opbevarer de faktiske data. Databasen vedligeholder dataenes integritet med hensyn tiltidsmæssige bindinger, herunder en fuld log af alle ændringer. Dette indgår som en del af OIOXML-standarden, men forklares i større detalje herunder. En trigger sørger for at give abonnerende fagsystemerbesked om alle ændringer i data. Denne database er implementet i PostgreSQL.

Som systemet er implementeret, har det følgende features/egenskaber:

Autentificering

Denne foregår ved hjælp af SAML-tokens. Når MOX-agenten får besked om en ændring af fagsystemets data,sørger den for, at der genereres en SAML-token på den indloggede brugers vegne. Denne SAML-token repræsen-terer den indloggede brugers digitale signatur af de pågældende ændringer. I REST-interfacet autentificeresbrugerens identitet op mod en SAML Identity Provider, og de pågældende ændringer logges med brugerensbrugerid (UUID).

Autorisation

Efter validering af SAML token kalder REST-interfacet en selvstændig rettighedsservice, der vil fortælle, ombrugeren har ret til at udføre den pågældende handling.

Fuld implementation af OIOXML-standarderne

REST-interfacet giver (sammen med databasen, som det kører overfor) en fuld implementation af version 1.1 afOIOXML-standarderne Klassifikation og Organisation, Dokument version 1.1.1 og Sag version 1.2 - det vil sigede nugældende standarder som defineret her:

https://digitaliser.dk/resource/444163

Hvis du gerne vil se en skematisk oversigt over, hvad en sådan service mere konkret indeholder, kan du f.eks. se“Informations- og meddelelsesmodeller for Organisation” for enden af dette link:

https://www.digitaliser.dk/resource/991439/artefact/Informations-ogmeddelelsesmodellerforOrganisation%5bvs.1.1%5d.pdf?artefact=true&PID=1602187

Når nu formatet netop hedder OIO XML, bør det bemærkes, at Digitaliseringsstyrelsen har defineret en rækkeXSD’er (XML-skemaer) for de modeller, der beskrives i standarderne - men disse har vi i samråd med kundenvalgt ikke at understøtte. I stedet kommunikerer REST-interfacet med en JSON-repræsentation af objekterne,der er lettere at arejde med. Det vil imidlertid være meget let at konvertere mellem dette JSON-format og etXML-format.

Hvis en kommune har brug for en service, der understøtter den gældende version af en eller flere af disse firestandarder, indgår de altså som en del af Magentas nuværende implementation af MOX.

Repræsentation af OIOXML-objekter

Databasen implementerer OIOXML-objekter som standardformat og har funktioner (stored procedures) dertillader oprettelse, import, opdatering, passivering og sletning af objekter efter forskrifterne i standarden.

6 Chapter 3. Content

https://digitaliser.dk/resource/444163




Dette betyder blandt andet, at hvert objekt beskrives som en samling af attributter, tilstande og relationer til andreobjekter. Hver OIOXML-standard består netop af en definition af en række objekter (klasser) og deres relation tilandre OIOXML-objekter.

Fig. 1: Som eksempel vises her objektmodellen for OIOXML-klassen Bruger.

Både databasen og REST-interfacet implementerer OIOXML-objekter på en standardiseret måde, der gør det letat tilføje nye objekter - og dermed at implementere nye standarder.

Bitemporalitet

Et aspekt af OIOXML-standarden er bitemporalitet. Dette indebærer, at hvert objekt kan være forsynet med totidsaspekter:

• Hver eneste version af objektet udgør en registrering. En registrering er altid gældende fra et start-tidspunktog til et slut-tidspunkt, hvor slut-tidspunktet kan være “uendelig”. Hver gang, der foretages en ændring afobjektet, dannes en ny registrering, som er en fuld kopi af objektet med de pågældende ændringer indført.Den registrering har altid start- og slut-tid fra opdateringstidspunktet til uendelig. Ved samme lejlighedopdates slut-tidspunktet på den seneste registrering, ligeledes til opdateringstidspunktet.

• Objektets attributter, tilstande og relationer er omfattet af en virkning. Denne virkning er essentielt ettidsinterval, der angiver, hvornår den pågældende værdi er gældende. Et Dokument kunne for eksempelhave tilstanden “Ikke-Publiceret” frem til 1. januar 2015 og “Publiceret” herefter. Det vil derfor have tovirkningsperioder for tilstanden PubliceretStatus.

3.2. MOX - infrastruktur for udveksling af offentlige data 7


Fleksibilitet og ensartethed

De p.t. fire understøttede OIO-standarder kommunikerer via et helt standardiseret REST-interface. Hvert objekt“bor” på stien

https://<service-URL>/<standard>/<klasse>/<uuid>

• altså for eksempel

https://mox.magenta-aps.dk/organisation/bruger/de305d54-75b4-431b-adb2-→˓eb6b9e546014

Som det er standard i REST-interfaces, kan man oprette, opdatere og læse objekter ved hjælp af HTTP-protokollens kommandoer PUT, PATCH, POST og GET.

Kildekoden understøtter alle ud af boksen og ved en standard-installation alle fire services, men det er muligt atlave en installation, hvor de ikke alle er aktive - ligesom det vil være muligt at lade flere forskellige tjenester kørepå forskellige server og lade dem interagere via MOX-agenter og beskedfordeler.

3.3 Installation

All the actively developed source code of mox is a Python package located in oio_rest. You can run it in as yousee fit. Two methods are described below. Use the docker image or a Python package in a virtual environment.

Tip: TL;DR: To get a running development environment with postgres and mox, run:

git clone https://github.com/magenta-aps/mox.gitcd moxdocker-compose up -d --build mox

Todo: Document how to install test dependencies when it is formalized in #28489.

3.3.1 Docker

The repository contains a Dockerfile. This is the recommended way to install mox both as a developer and inproduction.

All releases are pushed to Docker Hub at magentaaps/mox under the latest tag. The dev-latest tag containsthe latest build from the development branch.

To run mox in docker you need a running docker. To install docker we refer you to the official documentation.

Database

The container requires a connection to a postgres database. Read Database for a complete description.

Remember if you set DB_HOST to localhost the container will try to connect to a database in the samecontainer , not the host machine.

You will need to initialize the database, user and extensions. The docker image contains a few scripts to do thiseasily. They are designed to work with the official postgres docker image. You can copy the scripts from thedocker image by running:


https://hub.docker.com/r/magentaaps/mox

https://docs.docker.com/install/

https://hub.docker.com/_/postgres


# To a docker volume named postgres-initdb.ddocker run \

--rm \-v postgres-initdb.d:/postgres-initdb.d \--entrypoint="" \--user=root \magentaaps/mox:latest \cp -r /code/docker/postgres-initdb.d/. /postgres-initdb.d

# To a directory named postgres-initdb.ddocker run \

--rm \-v $PWD/postgres-initdb.d:/postgres-initdb.d \--entrypoint="" \--user=root \magentaaps/mox:latest \cp -r /code/docker/postgres-initdb.d/. /postgres-initdb.d

These scripts can now easily be mounted into the postgres docker image. They require the environment variablesDB_NAME, DB_USER and DB_PASSWORD to be set. It is also recommended to set POSTGRESS_PASSWORDfor the SUPERUSER in postgres. To start a postgres docker image run:

docker run \-v postgres-initdb.d:/docker-entrypoint-initdb.d \-e DB_NAME=mox \-e DB_USER=mox \-e DB_PASSWORD=mox \-e POSTGRES_PASSWORD=mox \postgres:9.6

The mox docker image will automatically initialize the database objects for on the first startup.

Run the container

When you have initialized the database with database, user and extensions, you can start a the container with:

docker run \-p 8080:8080 \-e DB_HOST=<IP of DB host> \-e DB_USER=mox \-e DB_PASSWORD=mox \magentaaps/mox:latest

This will pull the image from Docker Hub and start a container in the foreground. The -p 8080:8080 bindsport 8080 of the host machine to port 8080 on the container. The -e sets the environment variables in container.

If successful you should see the container initializing database and finally

when the gunicorn server starts up. You should now be able to reach the server from the host at http://localhost:8080.

For other setting you can set, see Settings.

Todo: Document testing dependencies.

Logs

The gunicorn access log is output on STDOUT and error log is output on STDERR. They can be inspected withdocker logs.

3.3. Installation 9

https://docs.docker.com/engine/reference/commandline/run/#publish-or-expose-port--p---expose

https://docs.docker.com/engine/reference/commandline/run/#publish-or-expose-port--p---expose

https://docs.docker.com/engine/reference/commandline/run/#set-environment-variables--e---env---env-file


File upload

/var/mox is the default FILE_UPLOAD_FOLDER. It is created as an anonymous volume. It can be named ifneeded.

You can also bind it, just make sure the folder on the host system have the owner set to the same UID/GID as theuser that runs the application in the Dockerfile.

User permissions

The Dockerfile creates and runs the application as the mox user. This user will own all the files generated bythe application. This user has a UID and GID of 72010.

If you want to use another UID/GID, you can specify it as the --user=uid:gid overwrite flag for the dockerrun command or in docker-compose. If you change the UID/GID, the /var/mox volume may not have theright permissions. It is recommended to only use bind if you overwrite the user and set the same user as owner ofthe directory you bind.

3.3.2 Docker-compose

You can use docker-compose to start up mox and postgres.

A docker-compose.yml for development is included. It automatically starts up postgres. It sets the environ-ment variables to connect them.

It also mounts the current directory in the container and automatically restarts the server on changes. This enablesyou to edit the files in oio_rest and the server will be reloaded automatically.

To pull the images and start the three service run:

docker-compose up -d --build mox

The -d flag move the services to the background. You can inspect the output of them with docker-composelogs <name> where <name> is the name of the service in docker-compose.yml. The --build flagbuilds the newest docker image for oio_rest from the local Dockerfile.

To stop the service again run docker-compose stop. This will stop the services, but the data will persist. Tocompletely remove the containers and data run docker-compose down.

The docker-compose.yml file contains a service named mox-cp. Its purpose is to copy the files needed toinitialize the database, user and extensions to a volume. This volume can then be mounted to the postgres imageto automatically initialize the database. This functionality is not needed by default because the needed files aremounted directly from the host. It is included as an example when you want to use an environment closer toproduction.

3.3.3 From source

All the relevant code is in a Python package located in oio_rest.

Prerequisites

The oio_rest package requires a few system dependencies. It requires:

• python >=3.5

• pip >=10.0.0

• setuptools >=39.0.1

• wheel


https://docs.docker.com/storage/bind-mounts/

https://docs.docker.com/engine/reference/run/#user

https://docs.docker.com/compose/compose-file/#domainname-hostname-ipc-mac_address-privileged-read_only-shm_size-stdin_open-tty-user-working_dir

https://docs.docker.com/storage/bind-mounts/

https://hub.docker.com/_/postgres


• git for installing some requirements from requirements.txt and

• libxmlsec1-dev for the Python package xmlsec.

Mox needs to connect to postgres9.6. mox can be configured with DB_HOST to connect to any machine.You can install postgres9.6 on the same machine and leave DB_HOST as the default value of localhost.

Installation

When the prerequisites are met, you can install mox from a clone of the git repository.

git clone https://github.com/magenta-aps/mox.gitcd mox/oio_restpip install .

Configuration

Look through the Settings and configure the one you need either as environment variables or as a config file. Themost likely changes are properly to DB_HOST, DB_USER and DB_PASSWORD.

Database initialization

Todo: Refer to relavant section in Database when it is written in #30317.

Run

When the database is initialized you can access the cli with python3 -m oio_rest <command>. To runthe development server run python3 -m oio_rest run. Alternatively, use gunicorn to run a server withgunicorn oio_rest.app:app.

3.4 Database

Todo: Document the database in #30317.

3.4.1 Database, user and extensions initialization

3.4.2 Object initialization

To initialize the database, run:

python -m oio_rest initdb

For more info run python -m oio_rest initdb --help.

3.4. Database 11

https://redmine.magenta-aps.dk/issues/30317

https://redmine.magenta-aps.dk/issues/30317


3.5 Objects

The database contains a large number of datatypes. To illustrate the basic operations of the API we introduce asubset here. Specifically a subset of Organisation.

The complete reference documentation for the fields can be found in Generelle egenskaber for services på sags- ogdokumentområdet with more info for those specifically for Organisation in : Specifikation af serviceinterfacefor organisation.

Note: As an example, the REST interface for Organisation is specified here: http://info.rammearkitektur.dk/index.php/LoRA_Organisationsservice

Please note that in comparison with this official specification, our system currently does not support the parameters-miljø and -version.

As regards the parameter -miljø (which could be -prod, -test, -dev, etc.) we have been trying to convincethe customer that we do not recommend running test, development and production on the same systems, so wewould prefer not to support that parameter.

As regards the parameter -version, we have deferred support for it until we actually have more than one versionof the protocol to support.

3.5.1 Some Organisation objects

organisation

Located at the endpoint /organisation/organisation. An organisation is a legal organisation. Agood example of this a municipality.

Listing 1: A small organisation with all the required fields high-lighted.

{"attributter": {"organisationegenskaber": [{

"brugervendtnoegle": "magenta-aps","organisationsnavn": "Magenta ApS","virkning": {

"from": "2017-01-01","to": "2019-03-14"

}}]

},"tilstande": {"organisationgyldighed": [{

"gyldighed": "Aktiv","virkning": {

"from": "2017-01-01","to": "2019-03-14"

}}

]}}

The fields used in the example organisation are the following:

organisationegenskaber→brugervendtnoegle An ID for the user. Semantically, this should beunique, but we don’t enforce that. It is intended for the user to recognise the object. Required.

organisationegenskaber→organisationsnavn The official name of the organisation.


https://www.digitaliser.dk/resource/1567464/artefact/Generelleegenskaberforservicesp%c3%a5sags-ogdokumentomr%c3%a5det-OIO-Godkendt%5bvs.1.1%5d.pdf?artefact=true&PID=1763377


https://www.digitaliser.dk/resource/1569113/artefact/Specifikationafserviceinterfacefororganisation-OIO-Godkendt%5bvs.1.1%5d.pdf?artefact=true&PID=1569586

https://www.digitaliser.dk/resource/1569113/artefact/Specifikationafserviceinterfacefororganisation-OIO-Godkendt%5bvs.1.1%5d.pdf?artefact=true&PID=1569586

http://info.rammearkitektur.dk/index.php/LoRA_Organisationsservice

http://info.rammearkitektur.dk/index.php/LoRA_Organisationsservice


organisationegenskaber→virkning The period when the above attributes is valid. See Valid time.Required.

organisationgyldighed→gyldighed Whether the organisation is active or not. Can take the valuesAktiv and Inaktiv. Required.

organisationgyldighed→virkning The period when the above gyldighed is valid. See Valid time.Required.

organisationenhed

Located at the endpoint /organisation/organisationenhed. An organisationenhed is an organ-isational unit. This could be a department, section, office, committee, project group, class, team or similar.

Listing 2: A small organisationenhed with all the required fieldshighlighted.

{"attributter": {"organisationenhedegenskaber": [{

"brugervendtnoegle": "copenhagen","enhedsnavn": "Copenhagen","virkning": {

"from": "2017-01-01","to": "2019-03-14"

}}]

},"tilstande": {"organisationenhedgyldighed": [{


"from": "2017-01-01","to": "2019-03-14"

}}]

},"relationer": {"overordnet": [{"uuid": "6ff6cf06-fa47-4bc8-8a0e-7b21763bc30a","virkning": {

"from": "2017-01-01","to": "2019-03-14"

}}],"tilhoerer": [{

"uuid": "6135c99b-f0fe-4c46-bb50-585b4559b48a","virkning": {

"from": "2017-01-01","to": "2019-03-14"

}}]

}}

The fields used in the example organisationenhed are the following:

organisationenhedegenskaber→* and organisationenhedgyldighed→* Similar toorganisation.

relationer→tilhoerer This is the root organisation which the organisationenhed is part of.This is usally set on all organisationenhed to the single organisation in the mox instance.

3.5. Objects 13


relationer→overordnet The parent organisationenhed.

3.5.2 Bitemporality

The database is a Bitemporal Database with valid time and transaction time.

Valid time

All attributes and relations have a valid time period associated as virkning. It is the time period during whichthe fact is true in the real world.

Listing 3: A sample virkning with required fields highlighted.

{"from": "2017-01-01","from_included": true,"to": "2025-12-31","to_included": false

}

The fields used in the example are the following:

from The time when this fact starts to be true in the real world. Date and time input is accepted in almost anyreasonable format, including ISO 8601. Required.

from_included Whether the from timestamp is closed or open. Default true.

to The time when this facts stop to be true in the real world. Date and time input is accepted in almost anyreasonable format, including ISO 8601. Required.

to_included Whether the to timestamp is closed or open. Default false.

Transaction time

All transactions also have a transaction time as registreret. This records the time period during which agiven fact is stored in the database. With the query parameters to a Read, List or Search operation it can give youa view of the state of the database at given time in the past or future.

3.5.3 Integrationdata

The integrationdata field is different and its uniqueness is documented here.

Integrationdata

integrationsdata is a regular text field intended for information from extern systems so they can be kept insync. This field should only include values that do not belong in another field. Do not overuse it. It is not part ofthe OIO standard.

Usage

The intention is that integrationsdata always contains a JSON-valid piece of text. Applications that usethe field should parse it as JSON and send valid JSON in return.

Warning: When you want to add or update a field in integrationsdata, please make sure to send allthe existing fields along, so they are not accidentally removed. This is easy to mess up, so take care.


https://en.wikipedia.org/wiki/Temporal_database


Listing 4: A small organisation with integrationsdata.

{"attributter": {"organisationegenskaber": [{"brugervendtnoegle": "magenta-aps","organisationsnavn": "Magenta ApS","virkning": {

"from": "2017-01-01","to": "2019-03-14"

},"integrationsdata": "{\"sdløn\": \"id5402349\", \"IB\": 4}"

}]},"tilstande": {"organisationgyldighed": [{


"from": "2017-01-01","to": "2019-03-14"

}}

]}}

Notice that the quotes (") in integrationsdata have been escaped as \".

3.6 REST API

This page will give you a overview of the REST API.

3.6.1 Self-documentation

The API serves some documentation for the services, objects and fields it contains. The following urls are avail-able:

GET /site-mapReturns a site map over all valid urls.

Status Codes

• 200 OK – No error.

GET /(service)/classesReturns a JSON representation of the hierarchy’s classes and their fields

Status Codes


GET /(service)/object/fields Returns a list of all fields a given object have.

Status Codes


GET /(service)/object/schema Returns the JSON schema of an object.

Status Codes


3.6. REST API 15

https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.2.1





Caution: The structure of each class is not completely analogous to the structure of the input JSON as it usesthe concept of “overrides”. This should also be fixed.

3.6.2 API tutorial

The following small exercises can be used as an inspiration to getting to know LoRa’s REST API. Read or skimthe pages about the Objects and the different Operations before moving on.

Warning: The exact end-date for the date ranges are not important, it is however important that they are inthe future when you do this tutorial. If they are not the Read operation, List operation and Search operationwill not return the object unless the virkning*-parameters is set.

1. Create an organisation called e.g. “Magenta” valid from 2017-01-01 (included) to 2025-12-31 (ex-cluded).

2. Make a query searching for all organisation in LoRa - confirm that Magenta exists in the system.

3. Create an organisationenhed called “Magenta” (which should be a subunit to the organisationMagenta) active from 2017-01-01 (included) to 2024-03-14 (excluded). Consider which attributes andrelations to set.

4. Create an organisationenhed called “Copenhagen” (which should be a subunit to theorganisationenhed Magenta) active from 2017-01-01 (included) to 2024-03-14 (excluded). Considerwhich attributes and relations to set.

5. Create an organisationenhed called “Aarhus” (which should be a subunit of theorganisationenhed Magenta) active from 2018-01-01 (included) to 2025-09-01 (excluded).Consider which attributes and relations to set.

6. Make a query searching for all organisationenhed in LoRa - confirm that Magenta, Copenhagen andAarhus exist in the system.

7. Add an address to the organisationenhed Aarhus (valid within the period where theorganisationenhed is active).

8. Fetch the organisationenhed Aarhus and verify that the newly added address is present in theresponse.

9. Add another address to the organisationenhed Aarhus (valid in a period exceeding the periodwhere the organisationenhed is active). What happens in this case?

10. Remove all address from the organisationenhedAarhus and confirm that they are gone afterwards.

11. Make a small script capable of adding n new organisationenhed (e.g. where 10 < n < 20) namedorgEnhed1, orgEnhed2, orgEnhed3,. . . These organisationenhed should all be subunits of theorganisationenhed Copenhagen and they should be active in random intervals ranging from 2017-01-01 (included) to 2025-12-31 (excluded).

12. Find all active organisation (if any) in the period from 2017-12-01 to 2025-06-01.

13. What are the names of the organisationenhed from above?

3.6.3 Operations

Read operation

GET /(service)/object/uuid The Read operation obtains an entire object as JSON.



Default is to return the object as it is currently seen but can optionally be constrained by virking* validtime and/or registrering* transaction time to give an older view.

Example request for GET /organisation/organisation/(uuid):

GET /organisation/organisation/5729e3f9-2993-4492-a56f-0ef7efc83111 HTTP/1.1Accept: */*Host: example.com

Example response for GET /organisation/organisation/(uuid):

HTTP/1.0 200 OKContent-Length: 744Content-Type: application/jsonDate: Tue, 15 Jan 2019 12:27:16 GMTServer: Werkzeug/0.14.1 Python/3.5.2

{"5729e3f9-2993-4492-a56f-0ef7efc83111": [{"id": "5729e3f9-2993-4492-a56f-0ef7efc83111","registreringer": [{

"attributter": {"organisationegenskaber": [{

"brugervendtnoegle": "magenta-aps","organisationsnavn": "Magenta ApS","virkning": {

"from": "2017-01-01 00:00:00+00","from_included": true,"to": "2019-03-14 00:00:00+00","to_included": false

}}]},"brugerref": "42c432e8-9c4a-11e6-9f62-873cf34a735f","fratidspunkt": {

"graenseindikator": true,"tidsstempeldatotid": "2019-01-15T10:43:58.122764+00:00"

},"livscykluskode": "Importeret","tilstande": {

"organisationgyldighed": [{"gyldighed": "Aktiv","virkning": {


}}]},"tiltidspunkt": {

"tidsstempeldatotid": "infinity"}}]}]}

Query Parameters

• registreretFra (datetime) – Transaction time ‘from’ timestamp.

• registreretTil (datetime) – Transaction time ‘to’ timestamp.

• registreringstid (datetime) – Transaction time ‘snapshot’ timestamp.

• virkningFra (datetime) – Valid time ‘from’ timestamp.

• virkningTil (datetime) – Valid time ‘to’ timestamp.

• virkningstid (datetime) – Valid time ‘snapshot’ timestamp.

All the registeret* and virkning* accept a value representing a specific date and time. Input isaccepted in almost any reasonable format, including ISO 8601, SQL-compatible, traditional POSTGRES,

3.6. REST API 17


and others. The accepted values are the Date/Time Input from PostgreSQL.

Response Headers

• Content-Type – application/json

Status Codes


• 400 Bad Request – Malformed JSON or other bad request.

• 404 Not Found – No object of a given class with that UUID.

• 410 Gone – The object has been deleted.

The Read operation is known as the Læs operation in the specification.

List operation

GET /(service)/object?uuid=uuid The List operation returns one or more whole objects as JSON.

The List operaation is similar to the Read operation, but uses a slightly different syntax. The UUID is givenas a parameter. With this syntax is is possible to list more than one UUID.

Note: GET /(service)/(object) can also be a Search operation depending on parameters. Withany the of uuid, virking* and registeret* parameters, it is a List operation. Given any otherparameters the operation is a Search operation and will only return a list of UUIDs to the objects.

Default is to return the object(s) as currently seen, but can optionally be constrained by virking* validtime and/or registrering* transaction time to give a past or future view.

There is no built-in limit to how many objects can be listed in this way, but both the HTTP-server andgunicorn may apply limits to the length of URIs. RFC 7230#section-3.1.1 recommends that all HTTPsenders and recipients support, at a minimum, request-line lengths of 8000 octets, but some may not.

List example request for GET /organisation/organisationenhed>:

GET /organisation/organisationenhed?uuid=74054d5b-54fc-4c9e-86ef-790fa6935afb&→˓uuid=ccfd6874-09f5-4dec-8d39-781f614bb8a7 HTTP/1.1Accept: */*Host: example.com

List example response for GET /organisation/organisationenhed:

HTTP/1.0 200 OKContent-Length: 2150Content-Type: application/jsonDate: Thu, 17 Jan 2019 14:49:31 GMTServer: Werkzeug/0.14.1 Python/3.5.2

{"results": [[{"id": "74054d5b-54fc-4c9e-86ef-790fa6935afb","registreringer": [{

"attributter": {"organisationenhedegenskaber": [{


"from": "2017-01-01 00:00:00+00","from_included": true,"to": "2019-03-14 00:00:00+00",

(continues on next page)


https://www.postgresql.org/docs/9.5/datatype-datetime.html#DATATYPE-DATETIME-INPUT

https://tools.ietf.org/html/rfc7231#section-3.1.1.5






https://tools.ietf.org/html/rfc7230.html#section-3.1.1


(continued from previous page)

"to_included": false}}]},

"brugerref": "42c432e8-9c4a-11e6-9f62-873cf34a735f","fratidspunkt": {

"graenseindikator": true,"tidsstempeldatotid": "2019-01-11T10:10:59.

→˓430647+00:00"},"livscykluskode": "Opstaaet","relationer": {

"overordnet": [{"uuid": "66e8a55a-8c61-4d33-b244-574c09ef41f7

→˓","virkning": {


}}],"tilhoerer": [{

"uuid": "66e8a55a-8c61-4d33-b244-574c09ef41f7→˓",

"virkning": {"from": "2017-01-01 00:00:00+00","from_included": true,"to": "2019-03-14 00:00:00+00","to_included": false

}}]},"tilstande": {

"organisationenhedgyldighed": [{"gyldighed": "Aktiv","virkning": {



"tidsstempeldatotid": "infinity"}}]},

{"id": "ccfd6874-09f5-4dec-8d39-781f614bb8a7","registreringer": [{

"attributter": {"organisationenhedegenskaber": [{

"brugervendtnoegle": "aarhus","enhedsnavn": "Aarhus","virkning": {


}}]},"brugerref": "42c432e8-9c4a-11e6-9f62-873cf34a735f","fratidspunkt": {

"graenseindikator": true,"tidsstempeldatotid": "2019-01-11T10:10:59.

→˓688454+00:00"},"livscykluskode": "Rettet","relationer": {


3.6. REST API 19



"overordnet": [{"uuid": "66e8a55a-8c61-4d33-b244-574c09ef41f7

→˓","virkning": {


}}],"tilhoerer": [{

"uuid": "66e8a55a-8c61-4d33-b244-574c09ef41f7→˓",

"virkning": {"from": "2018-01-01 00:00:00+00","from_included": true,"to": "2019-09-01 00:00:00+00","to_included": false

}}]},"tilstande": {




"tidsstempeldatotid": "infinity"}}]}]]}

Query Parameters

• uuid (uuid) – The UUID of the object to receive. Allowed multiple times in Listoperation.







All the registeret* and virkning* take a datetime. Input is accepted in almost any reasonableformat, including ISO 8601, SQL-compatible, traditional POSTGRES, and others. The accepted values arethe Date/Time Input from PostgreSQL.

All string parameters match case insensitive. They support the wildcard operators _ (underscore) to matcha single character and % (percent sign) to match zero or more characters. The match is made with ILIKEfrom PostgresSQL.

Response Headers


Status Codes





https://www.postgresql.org/docs/9.5/functions-matching.html#FUNCTIONS-LIKE








The List operation is known as the List operation in the specification.

Search operation

GET /(service)/object The Search operation returns a list of UUIDs of the objects that fit the parameters.

Note: GET /(service)/(object) can also be a List operation depending on parameters. With onlythe uuid, virking* and registeret* parameters, it is a List operation and will return one or morewhole JSON objects. Given any other parameters the operation is a Search operation.

Default is to return the object(s) as currently seen but can optionally be constrained by virking* validtime and/or registrering* transaction time to give an older view.

Search example request for GET /organisation/organisationenhed:

GET /organisation/organisationenhed?overordnet=66e8a55a-8c61-4d33-b244-→˓574c09ef41f7 HTTP/1.1Accept: */*Host: example.com

Search example response for GET /organisation/organisationenhed:

HTTP/1.0 200 OKContent-Length: 94Content-Type: application/jsonDate: Thu, 17 Jan 2019 15:02:39 GMTServer: Werkzeug/0.14.1 Python/3.5.2

{"results": [["74054d5b-54fc-4c9e-86ef-790fa6935afb","ccfd6874-09f5-4dec-8d39-781f614bb8a7"

]]}

Query Parameters

• uuid (uuid) – The UUID of the object to receive. Allowed once in Search operation.

• brugervendtnoegle / bvn (string) – Match text in thebrugervendtnoegle-field.

• vilkaarligattr (string) – Match text values of any attributter-field.

• vilkaarligrel (uuid) – Match values of any relationer.

• livscykluskode (enum) – Matches the livscykluskode-field. Can be one ofOpstaaet, Importeret, Passiveret, Slettet or Rettet.

• brugerref (uuid) – Match the brugerref-field. The (system) user who changedthe object.

• notetekst (string) – Match the notetekst-field in virkning. (Not to beconfused with the note-field.)

• foersteresultat (int) – The first result in a Paged search. Sorts the result bybrugervendtnoegle.

• maximalantalresultater (int) – The maximal number of results in a Pagedsearch. Sorts the result by brugervendtnoegle.

3.6. REST API 21











All the registeret* and virkning* take a datetime. Input is accepted in almost any reasonableformat, including ISO 8601, SQL-compatible, traditional POSTGRES, and others. The accepted values arethe Date/Time Input from PostgreSQL.

All string parameters match case insensitively. They support the wildcard % (percent sign) to match zero ormore characters.

If you want to match a litteral percentage-sign % you have to escape it with backslash. E.g. abc\%defwould match the value abc%def.

Contrary to typical SQL LIKE syntax, the character _ (underscore) matches only the underscore character(and not “any character”).

Attention: The URI should always be percent-encoded as defined in RFC 3986. Not doing so can leadto unexpected results when you use the % wildcard.

The percent-encoding of % is %25. E.g. a search query for a bvn with the string abc%123 would looklike ?bvn=abc%25123.

See Wildcards for an in depth explanation.

In addition to the above general query parameters, each object also has specialized parameters based onits field. The endpoints GET /(service)/(object)/fields lists the fields which can be used forparameters for a Search operation.

Response Headers


Status Codes





The Search operation is known as the Søg operation in the specification.

Paged search

The search function supports paged searches by adding the parameters maximalantalresultater (maxnumber of results) and foersteresultat (first result).

Since pagination only makes sense if the order of the results are predictable the search will be sorted bybrugervendtnoegle if pagination is used.

Note: If queries using pagination are becoming a performance bottleneck, you can try to optimise them by set-ting enable_hashagg = False and enable_sort = False in postgres.conf. This may decreaseperformance for other queries.



https://tools.ietf.org/html/rfc3986.html#section-2








Advanced search

It is possible to search for relations (links) as well by specifying the value, which may be either an UUID or aURN. E.g., for finding all instances of organisationenhed which belongs to Direktion:

GET /organisation/organisationenhed?tilknyttedeenheder=urn:Direktion HTTP/1.1

When searching on relations, one can limit the relation to a specific object type by specifying a search parameterof the format:

&<relation>:<objecttype>=<uuid|urn>

E.g. if you want to search on an opgave relation with "objekttype":"lederniveau" you make a querylike this: ?opgave:lederniveau=5cc827ba-6939-4dee-85be-5c4ea7ffd76e.

Note that the objecttype parameter is case-sensitive.

It is only possible to search on one DokumentVariant and DokumentDel at a time. For example, if

&deltekst=a&underredigeringaf=<UUID>

is specified, then the search will return documents which have a DokumentDel with deltekst="a" andwhich has the relation underredigeringaf=<UUID>. However, if the deltekst parameter is omitted, e.g.

&underredigeringaf=<UUID>

Then, all documents which have at least one DokumentDel which has the given UUID will be returned.

The same logic applies to the varianttekst parameter. If it is not specified, then all variants are searchedacross. Note that when varianttekst is specified, then any DokumentDel parameters apply only to thatspecific variant. If the DokumentDel parameters are matched under a different variant, then they are not includedin the results.

Searching on Sag-JournalPost-relations

To search on the sub-fields of the JournalPost relation in Sag, requires a special dot-notation syntax, due topossible ambiguity with other search parameters (for example, the titel parameter).

The following are some examples:

&journalpostkode=vedlagtdokument&journalnotat.titel=Kommentarer&journalnotat.notat=Læg+mærke+til&journalnotat.format=internt&journaldokument.dokumenttitel=Rapport+XYZ&journaldokument.offentlighedundtaget.alternativtitel=Fortroligt&journaldokument.offentlighedundtaget.hjemmel=nej

All of these parameters support wildcards (%) and use case-insensitive matching, except journalpostkode,which is treated as-is.

Note that when these parameters are combined, it is not required that the matches occur on the sameJournalPost relation.

For example, the following query would match any Sag which has one or more JournalPost relations whichhas a journalpostkode = "vedlagtdokument" AND which has one or more JournalPost relationswhich has a journaldokument.dokumenttitel = "Rapport XYZ"

&journalpostkode=vedlagtdokument&journaldokument.dokumenttitel=Rapport+XYZ

3.6. REST API 23


Create operation

POST /(service)/object The Create operation creates a new object from the JSON payload. POST the JSON representationof its attributes, states and relations to the URL of the class. The Create operation returns a newly generatedUUID for the created object.

The data can be supplied directly in the request if Content-Type: application/json is set.

Alternatively the the data can be supplied as form-data in the json-field with Content-Type: multipart/form-data. This allows seperate file form-data.

Example request for POST /organisation/organisationenhed:

POST /organisation/organisationenhed HTTP/1.1Content-Type: application/jsonHost: example.com



"from": "2017-01-01","to": "2019-03-14"

}}]},"relationer": {

"overordnet": [{"uuid": "6ff6cf06-fa47-4bc8-8a0e-7b21763bc30a","virkning": {

"from": "2017-01-01","to": "2019-03-14"

}}],"tilhoerer": [{

"uuid": "6135c99b-f0fe-4c46-bb50-585b4559b48a","virkning": {

"from": "2017-01-01","to": "2019-03-14"

}}]},"tilstande": {


"from": "2017-01-01","to": "2019-03-14"

}}]}}

Example response for POST /organisation/organisationenhed>:

HTTP/1.0 201 CREATEDContent-Length: 48Content-Type: application/jsonDate: Mon, 21 Jan 2019 09:12:00 GMTServer: Werkzeug/0.14.1 Python/3.5.2

{"uuid": "14b2abd4-ae3c-4a0f-b530-7a93443d729d"

}

Request Headers

• Content-Type – application/json or multipart/form-data

Status Codes


https://tools.ietf.org/html/rfc7231#section-4.3.3





• 201 Created – Object was created.


The Create operation is known as the Opret operation in the specification.

Update operation

PATCH /(service)/object/uuid

The Update operation applies the JSON payload as a change to the object. Returns the UUID ofthe object.

How the changes are applied are described in the following pages. For the logic of merging seeMerging logic. To delete part of an object see Deleting attributes, states and relations.

The data can be supplied directly in the request if Content-Type: application/json is set.

Alternatively the the data can be supplied as form data in the json-field with Content-Type:multipart/form-data. This allows seperate file form data.

Merging logic

When preforming a Update operation some logic is applied to merge the new data with the existing data. Thispage describes this logic.

It is worth noting, that the current implementation of the REST-api and the underlying DB procedures as a generalrule merges the incomming registration with the registration currently in effect for all virkning-periods notexplictly covered by the incomming registration.

Exceptions to this rule

• Deleting Attributes / States / Relations by explicitly specifying an empty list / object (see section belowregarding clearing/deleting Attributes/States/Relations)

• When updating relations with unlimited cardinality (0..n) you always have to supply the full list of all therelations of that particular type. No merging with the set of relations of the same particular type of theprevious registration takes place. However, if you omit the particular type of relation entirely, when you’reupdating the object - all the relations of that particular type of the previous registration, will be carried over.

• The relations in the services and object classes Sag, Aktivitet, Indsats and Tilstand have indicesand behave differently - this will be described below.

Example of updating attributes

As an example (purely made up to suit the purpose), lets say we have a Facet object in the DB, where the currentfacetegenskaber looks like this:

..."facetegenskaber": [

{"brugervendtnoegle": "ORGFUNK","beskrivelse": "Organisatorisk funktion æ","plan": "XYZ","opbygning": "Hierarkisk","ophavsret": "Magenta","supplement": "Ja","virkning": {


3.6. REST API 25








"from": "2014-05-19","to": "infinity","aktoerref": "ddc99abd-c1b0-48c2-aef7-74fea841adae","aktoertypekode": "Bruger","notetekst": "Adjusted egenskaber"

}}

]...

Let’s say we now supply the following fragment as part of the JSON body to the Update operation:


{"supplement": "Nej","virkning": {

"from": "2015-08-27","to": "2015-09-30","aktoerref": "ddc99abd-c1b0-48c2-aef7-74fea841adae","aktoertypekode": "Bruger","notetekst": "Adjusted supplement"

}}

]...

The resulting facetegenskaber of the Facet would look like this:


{"brugervendtnoegle": "ORGFUNK","beskrivelse": "Organisatorisk funktion æ","plan": "XYZ","opbygning": "Hierarkisk","ophavsret": "Magenta","supplement": "Ja","virkning": {

"from": "2014-05-19","to": "2015-08-27","aktoerref": "ddc99abd-c1b0-48c2-aef7-74fea841adae","aktoertypekode": "Bruger","notetekst": "Adjusted egenskaber"

}},{

"brugervendtnoegle": "ORGFUNK","beskrivelse": "Organisatorisk funktion æ","plan": "XYZ","opbygning": "Hierarkisk","ophavsret": "Magenta","supplement": "Nej","virkning": {

"from": "2015-08-27","to": "2015-09-30","aktoerref": "ddc99abd-c1b0-48c2-aef7-74fea841adae","aktoertypekode": "Bruger","notetekst": "Adjusted supplement"

}





},{"brugervendtnoegle": "ORGFUNK","beskrivelse": "Organisatorisk funktion æ","plan": "XYZ","opbygning": "Hierarkisk","ophavsret": "Magenta","supplement": "Ja","virkning": {

"from": "2015-09-30","to": "infinity","aktoerref": "ddc99abd-c1b0-48c2-aef7-74fea841adae","aktoertypekode": "Bruger","notetekst": "Adjusted egenskaber"

}}

]...

As we can se, the update operation will merge the incoming fragment with the facetegenskaber of the currentregistration according to the virkning-periods stipulated. The facetegenskaber-fields not provided in theincomming fragment, will be left untouched. If you wish to clear/delete particular facetegenskaber-fields,see Delete operation.

Example of updating states

Lets say we have a Facet-object, where the state facetpubliceret look likes this in the DB:

..."tilstande": {

"facetpubliceret": [{"publiceret": "Publiceret","virkning": {

"from": "2014-05-19","to": "infinity","aktoerref": "ddc99abd-c1b0-48c2-aef7-74fea841adae","aktoertypekode": "Bruger","notetekst": "Publication Approved"

}}]

},...

Lets say that we now, provide the following fragment as part of the JSON body to the Update operation of theREST-api:

..."tilstande": {

"facetpubliceret": [{"publiceret": "IkkePubliceret","virkning": {

"from": "2015-01-01","to": "2015-12-31","aktoerref": "ddc99abd-c1b0-48c2-aef7-74fea841adae","aktoertypekode": "Bruger","notetekst": "Temp. Redacted"

}


3.6. REST API 27



}]

},...

The resulting facetpubliceret state produced by the Update operation, would look like this:

..."tilstande": {

"facetpubliceret": [{"publiceret": "Publiceret","virkning": {

"from": "2014-05-19","to": "2015-01-01","aktoerref": "ddc99abd-c1b0-48c2-aef7-74fea841adae","aktoertypekode": "Bruger","notetekst": "Publication Approved"

}},{

"publiceret": "IkkePubliceret","virkning": {

"from": "2015-01-01","to": "2015-12-31","aktoerref": "ddc99abd-c1b0-48c2-aef7-74fea841adae","aktoertypekode": "Bruger","notetekst": "Temp. Redacted"

}},{

"publiceret": "Publiceret","virkning": {

"from": "2015-12-31","to": "infinity","aktoerref": "ddc99abd-c1b0-48c2-aef7-74fea841adae","aktoertypekode": "Bruger","notetekst": "Publication Approved"

}}]

},...

Hopefully it can be seen, that the Update operation will merge the incoming fragment with thefacetpubliceret state of the current registration according to the virkning-periods stipulated. If youwish to clear/delete particular states, see Delete operation.

Example of updating relations

As described in the top section we differentiate between relations with cardinality 0..1 and 0..n.

Lets say we have an Facet-object in the database, which has the following ansvarlig (cardinality 0..1) rela-tion in place:

..."relationer": {

"ansvarlig": [{

"uuid": "ddc99abd-c1b0-48c2-aef7-74fea841adae","virkning": {





"from": "2014-05-19","to": "infinity","aktoerref": "ddc99abd-c1b0-48c2-aef7-74fea841adae","aktoertypekode": "Bruger","notetekst": "Initial Responsible Set"

}}

]}

...

Lets say we now provide the following fragment as part of the incoming JSON body sent to the Update operation:

..."relationer": {

"ansvarlig": [{

"uuid": "ef2713ee-1a38-4c23-8fcb-3c4331262194","virkning": {

"from": "2015-02-14","to": "2015-06-20","aktoerref": "ddc99abd-c1b0-48c2-aef7-74fea841adae","aktoertypekode": "Bruger","notetekst": "Change of responsible"

}}]

}...

The resulting ansvarlig-relation of the Facet-object would look like this:

..."relationer": {

"ansvarlig": [{


"from": "2014-05-19","to": "2015-02-14","aktoerref": "ddc99abd-c1b0-48c2-aef7-74fea841adae","aktoertypekode": "Bruger","notetekst": "Initial Responsible Set"

}},{


"from": "2015-02-14","to": "2015-06-20","aktoerref": "ddc99abd-c1b0-48c2-aef7-74fea841adae","aktoertypekode": "Bruger","notetekst": "Change of responsible"

}},{


"from": "2015-06-20","to": "infinity","aktoerref": "ddc99abd-c1b0-48c2-aef7-74fea841adae",


3.6. REST API 29



"aktoertypekode": "Bruger","notetekst": "Initial Responsible Set"

}}

]}

...

As it can be seen, the Update operation has merged the incoming relation with the ansvarlig-relation of theprevious registration.

If you wish to delete / clear relations, see the section regading Delete operation.

If we want to update relations of a type with unlimited cardinality, we need to supply the full list of the relationsof that particalar type to the Update operation. Lets say we have a Facet-object in the DB with the followingredaktoerer-relations in place:

..."relationer": {

"redaktoerer": [{


"from": "2014-05-19","to": "infinity","aktoerref": "ddc99abd-c1b0-48c2-aef7-74fea841adae","aktoertypekode": "Bruger","notetekst": "First editor set"

}},

{"uuid": "ddc99abd-c1b0-48c2-aef7-74fea841adae","virkning": {

"from": "2015-08-20","to": "infinity","aktoerref": "ddc99abd-c1b0-48c2-aef7-74fea841adae","aktoertypekode": "Bruger","notetekst": "Second editor set"

}}

]}

...

Lets say we now provide the following fragment as part of the JSON body sent to the Update operation:

..."relationer": {

"redaktoerer": [{


"from": "2015-08-26","to": "infinity","aktoerref": "ddc99abd-c1b0-48c2-aef7-74fea841adae","aktoertypekode": "Bruger","notetekst": "Single editor now"

}}

]}

...



The resulting redaktoerer-part of the relations of the Facet-object, will look like this:

..."relationer": {

"redaktoerer": [{


"from": "2015-08-26","to": "infinity","aktoerref": "ddc99abd-c1b0-48c2-aef7-74fea841adae","aktoertypekode": "Bruger","notetekst": "Single editor now"

}}

]}

...

As we can see no merging has taken place, as we in this example are updating relations of a type with unlimitedcardinality (0..n).

As explained above, this works differently for “new-style” relations, i.e. relations with indices - specifically, theobject classes Sag, Indsats, Aktivitet and Tilstand.

Also see the section named Delete operation for info regarding clearing relations.

Relations of type Sag, Indsats, Tilstand and Aktivitet

The relations with unlimited cardinality (0..n) of the Sag, Indsats, Tilstand and Aktivitet-objects aredifferent from the relations of the other object types, as they operate with an ‘index’ field. This means that youcan update relations with unlimited cardinality without specifying the full list of the relations of the given type.You can update a specific relation instance, making use of its index value.

Lets say that you have a Sag-object with the following andrebehandlere-relations in place in the DB:

..."relationer": {

"andrebehandlere": [{"objekttype": "Bruger","indeks": 1,"uuid": "ff2713ee-1a38-4c23-8fcb-3c4331262194","virkning": {

"from": "2014-05-19","to": "infinity","aktoerref": "ddc99abd-c1b0-48c2-aef7-74fea841adae","aktoertypekode": "Bruger","notetekst": "As per meeting d.2014-05-19"

}},{

"objekttype": "Organisation","indeks": 2,"uuid": "ddc99abd-c1b0-48c2-aef7-74fea841adae","virkning": {

"from": "2015-02-20","to": "infinity","aktoerref": "ddc99abd-c1b0-48c2-aef7-74fea841adae","aktoertypekode": "Bruger","notetekst": "As per meeting 2015-02-20"

},}


3.6. REST API 31



]}...

Lets say you now provide the following fragment as part of the JSON body provided to the Update operation ofthe Sag-object:

..."relationer": {"andrebehandlere": [

{"objekttype": "Organisation","indeks": 2,"uuid": "ddc99abd-c1b0-48c2-aef7-74fea841adae","virkning": {

"from": "2015-05-20","to": "2015-08-20","aktoerref": "ddc99abd-c1b0-48c2-aef7-74fea841adae","aktoertypekode": "Bruger","notetekst": "As per meeting d.2015-02-20"

},},{

"objekttype": "Organisation","uuid": "ef2713ee-1a38-4c23-8fcb-3c4331262194","virkning": {


},},

]}...

The result would be the following:

..."relationer": {"andrebehandlere": [

{"objekttype": "Bruger","indeks": 1,"uuid": "ff2713ee-1a38-4c23-8fcb-3c4331262194","virkning": {

"from": "2014-05-19","to": "infinity","aktoerref": "ddc99abd-c1b0-48c2-aef7-74fea841adae","aktoertypekode": "Bruger","notetekst": "As per meeting d.2014-05-19"

},},{"objekttype": "Organisation","indeks": 2,"uuid": "ddc99abd-c1b0-48c2-aef7-74fea841adae","virkning": {

"from": "2015-05-20","to": "2015-08-20",





"aktoerref": "ddc99abd-c1b0-48c2-aef7-74fea841adae","aktoertypekode": "Bruger","notetekst": "As per meeting d.2015-02-20"

},},{

"objekttype": "Organisation","indeks": 3,"uuid": "ef2713ee-1a38-4c23-8fcb-3c4331262194","virkning": {


},},

]}...

As can be seen, the relation with "indeks": 2 has been updated and a new relation with "indeks": 3has been created. The relation with "indeks": 1 has been carried over from the previous registration. Pleasenotice, that in the case of relations of unlimited cardinality for the Sag-object, there is no merge logic regardingvirkning-periods.

To delete / clear a relation with a given indeks, you specify a blank uuid and/or a blank urn for that particularindeks.

Please notice, that for the Update operation, Create operation and Import operation of the Sag-object, the rule is,that if you supply an indeks-value that is unknown in the database, the specified indeks-value will be ignored,and a new relation instance will be created with an indeks-value computed by the logic in the DB-server. Forthe Create operation and Import operation, this will be all the specified index values.

Updating relations with cardinality 0..1 of the Sag-object is done similarly to updating relations of objects of othertypes. Any specified indeks-values are ignored and blanked by the logic of the Update operation. Otherwiseconsult Example of updating relations for examples and more info regarding this.

Deleting attributes, states and relations

This page describes deleting attributes, states and relation with an Update operation.

Note: Special consideration for Dokument and related objects are described on Deleting Dokument and Doku-ment Variant.

Example of deleting attributes

To clear / delete a previously set attribute value – lets say the egenskab supplement of a Facet-object – specifythe empty string as the attribute value in the JSON body:

..."attributter": {

"facetegenskaber": [{"supplement": "","virkning": {


3.6. REST API 33



"from": "2014-05-19","to": "infinity","aktoerref": "ddc99abd-c1b0-48c2-aef7-74fea841adae","aktoertypekode": "Bruger","notetekst": "Clearing supplement, defined by a mistake."

}}

]},

...

To delete all previously set attribute values of a specific kind - for all virkning-periods - you may simply specifyan empty list for the given type of attribute. Eg. to clear all egenskaber for a Facet - for all virkning-periods, you should do this:

..."attributter": {

"facetegenskaber": []

},...

Please notice, that this is different than omitting the list completely, in which case, the specific attributes will notbe updated at all. Eg. if you omit the facetegenskaber-key in the attributter-object in the JSON bodysupplied to the Update operation, all the facetegenskaber of the previous registration will be carried overuntouched.

..."attributter": {

},...

Example of deleting states

Similar to the procedure stated above for the attributes - clearing/deleting previously set states is done be supplyingthe empty string as value and the desired virknings period. Eg. to clear state publiceret of a Facet-object,the relevant part of the JSON body should look like this:

..."tilstande": {

"facetpubliceret": [{"publiceret": "","virkning": {

"from": "2014-05-19","to": "infinity","aktoerref": "ddc99abd-c1b0-48c2-aef7-74fea841adae","aktoertypekode": "Bruger","notetekst": "Clearing publiceret, defined by a mistake."

}}]

},...

You can clear all states of a specific kind, by explicitly specifying a completely empty list. Eg. to clearfacetpubliceret for all virkning-periods, the specific part of the JSON body should look like this:

..."tilstande": {





"facetpubliceret": []

},...

Please notice, that this is different than omitting the list completly, in which case, the specific state will not beupdated at all. Eg. if you omit the facetpubliceret-key in the tilstande-object in the JSON bodysupplied to the Update operation, all the facetpubliceret state values of the previous registration will becarried over untouched.

..."tilstande": {

},...

Example of deleting relations

Again, similar to the procedure stated above for the attributes and states, clearing a previously set relation withcardinality 0..1 is done by supplying empty strings for both uuid and urn of the relation. Eg. to clear a previouslyset the ansvarlig of a Facet-object, the specific part of the JSON body would look like this:

..."relationer": {

"ansvarlig": [{

"uuid": "","urn" : "","virkning": {

"from": "2014-05-19","to": "infinity","aktoerref": "ddc99abd-c1b0-48c2-aef7-74fea841adae","aktoertypekode": "Bruger","notetekst": "Nothing to see here!"

}}]

}...

When updating relations with unlimited cardinality (0..n), you have to supply the full list - that is, all the relationsof the particular type - and clearing a particular relation of a given type is accordingly done by supplying thefull list sans the relation, that you wish to clear. (The exception to this is when updating the Sag-object, whereyou can specify an indeks of the relation to only update a particular relation). To delete all the relations of aparticular type with unlimited cardinality (0..n) you must use the same procedure as described above for relationswith cardinality 0..1, where you specify a single relation of the given type with an empty string for uuid and urnand with a virkning-period as desired.

Specifying an explicitly empty object will clear all the relations of the object. Eg.:

..."relationer": {}

...

Notice, that this is different than omitting the relationer-key entirely, which will carry over all the relationsof the registration untouched.

3.6. REST API 35


Passivate operation

PATCH /(service)/object/uuid The Passivate operation is a special Update operation with a JSON payload containinglivscyklus: "Passiv". When an object is passive, it is no longer maintained and may not beupdated. The object will afterwards not show up in searches and listings. The operation returns the UUIDof the object.

The payload may contain an optional note field.

Example request for PATCH /organisation/organisationenhed/(uuid):

PATCH /organisation/organisationenhed/862bb783-696d-4345-9f63-cb72ad1736a3→˓HTTP/1.1Content-Type: application/jsonHost: example.com

{"note": "Passivate this object!","livscyklus": "Passiv"

}

Example response for PATCH /organisation/organisationenhed/(uuid):

HTTP/1.0 200 OKContent-Length: 48Content-Type: application/jsonDate: Mon, 21 Jan 2019 12:40:36 GMTServer: Werkzeug/0.14.1 Python/3.5.2

{"uuid": "862bb783-696d-4345-9f63-cb72ad1736a3 HTTP/1.1"

}

Request Headers


Status Codes

• 200 OK – Object was updated or passivated.



The Passivate operation is known as the Passiver operation in the specification.

Delete operation

DELETE /(service)/object/uuid The Delete operation deletes the object and return its UUID.

After an object is deleted, it cannot be retrieved by Read, List and Search Operations unless theregistreretTil and/or registreretFra indicate a period where it did exist.

The Delete operation deletes the whole object. To delete part of an object see Deleting attributes, states andrelations.

Example request for DELETE /organisation/organisationenhed/(uuid):

DELETE /organisation/organisationenhed/5fc97a7c-70df-4e97-82eb-64dc0a0f5746→˓HTTP/1.1Host: example.com








Example response for DELETE /organisation/organisationenhed/(uuid):

HTTP/1.0 202 ACCEPTEDContent-Length: 48Content-Type: application/jsonDate: Mon, 21 Jan 2019 16:47:00 GMTServer: Werkzeug/0.14.1 Python/3.5.2

{"uuid": "5fc97a7c-70df-4e97-82eb-64dc0a0f5746"

}

Status Codes

• 202 Accepted – Object was deleted.



The Delete operation is known as the Slet operation in the specification.

Import operation

PUT /(service)/object/uuid The Import operation creates or overwrites an object from the JSON payload and returns theUUID for the object. The Import operation is similar to a Create operation, but you specify at which UUID.

The Import operation creates a new object at the specified UUID if no object with the UUID exists or theobject with that UUID have been deleted or passivated.

If an object with the UUID does exist the Import operation completely overwrites the object including allvirkning periods. This is useful if you want to change the virking periods.

The JSON payload must contain a complete object in exactly the same format as for the Create operation.

If there are no object with the UUID exist the Import operation sets livscykluskode:"Importeret".

If an object is overwritten the Import operation sets livscykluskode: "Rettet".

Example request for PUT /organisation/organisationenhed/(uuid):

PUT /organisation/organisationenhed/841190a7-0e70-468a-bd63-eb11ed615337 HTTP/→˓1.1Content-Type: application/jsonHost: example.com



"from": "2017-01-01","to": "2019-03-14"

}}]},"relationer": {

"overordnet": [{"uuid": "6ff6cf06-fa47-4bc8-8a0e-7b21763bc30a","virkning": {

"from": "2017-01-01","to": "2019-03-14"

}}],


3.6. REST API 37







"tilhoerer": [{"uuid": "6135c99b-f0fe-4c46-bb50-585b4559b48a","virkning": {

"from": "2017-01-01","to": "2019-03-14"

}}]},"tilstande": {


"from": "2017-01-01","to": "2019-03-14"

}}]}}

Example response for PUT /organisation/organisationenhed/(uuid):

HTTP/1.0 200 OKContent-Length: 48Content-Type: application/jsonDate: Mon, 21 Jan 2019 10:17:19 GMTServer: Werkzeug/0.14.1 Python/3.5.2

{"uuid": "841190a7-0e70-468a-bd63-eb11ed615337"

}

Request Headers


Status Codes

• 200 OK – Object was created or overwritten.


The Import operation is known as the Importer operation in the specification.

3.6.4 Document etc.

Document and related objects have some special considerations.

File operations

File upload

When performing an Import operation, Create operation or Update operation on a Dokument, it is possible(if desired) to simultaneously upload files. These requests should be made using multipart/form-data-encoding. The encoding is the same that is used for HTML upload forms.

The JSON input for the request should be specified in a form-field called json. Any uploaded files should beincluded in the multpart/form-data request as separate form-fields.

The indhold-attribute of any DokumentDel may be a URI pointing to one of these uploaded file “fields”. Inthat case, the URI must be of the format:

field:myfield







where myfield is the form`-field name of the uploaded file included in the request that should be referenced bythe DokumentDel.

It is also possible to specify any URI (e.g. http://...., etc.) as the value of the indhold-attribute. In thatcase, the URI will be stored, however no file will be downloaded and stored to the server. It is then expected thatthe consumer of the API knows how to access the URI.

File download

When performing a Read operation or List operation on a Dokument, the DokumentDel-subobjects returnedwill include an indhold attribute. This attribute has a value that is the “content URI” of that file on the OIOREST API server. An example:

"indhold": "store:2015/08/14/11/53/4096a8df-ace7-477e-bda1-d5fdd7428a95.bin"

To download the file referenced by this URI, you must construct a request similar to the following:

http://localhost:5000/dokument/dokument/2015/08/14/11/53/4096a8df-ace7-477e-bda1-→˓d5fdd7428a95.bin

Deleting Dokument and Dokument Variant

Dokument and Dokument Variant have a few special considerations. This page show some examples.

Deleting varianter of a Dokument-object

To clear/delete a specific Dokument-object Variant you need to need to clear all the varianter egenskaberand Variant dele explicitly. Eg to clear the offentliggørelsesvariant of a Dokument-object you shouldsupply the specific part of the JSON body to the Update operation like this:

..."varianter": [

{"varianttekst": "offentliggørelsesvariant",

"egenskaber": [],"dele": []},

...]...

To delete / clear all the varianter of a Dokument-object, you should explicitly specify an empty list in theJSON body. Eg.

..."varianter": [],...

And again, please notice that this is different, than omitting the varianter-key completely in the JSON body,which will carry over all the Dokument varianter of the previous registration untouched.

Deleting Dokument-Del of a Dokument Variant

To clear / delete a specify Dokument Del of a Dokument Variant you should clear all the DokumentDel ‘egenskaber’ and Dokument Del relations explicitly. Eg. to clear the ‘Kap. 1’ Del of theoffentliggørelsesvariant, you should supply the specific part of the JSON body to the update Doku-ment operation like this:

3.6. REST API 39


..."varianter": [

{"varianttekst": "offentliggørelsesvariant","dele": [

"deltekst": "Kap. 1","egenskaber": [],"relationer": []

]}

]...

To clear / delete all the dele of a Variant, you should explicitly specify an empty list. Eg. for Del Kap. 1 of aoffentliggørelsesvariant, it would look like this:

..."varianter": [

{"varianttekst": "offentliggørelsesvariant","dele": []

}]...

Deleting egenskaber of a Dokument Del

To clear all egenskaber of a Dokument Del for all virkning-periods, you should explicitly specify an emptylist. Eg. to clear all the egenskaber of a Kap. 1-Del of a Dokument Variant it would look this:

..."varianter": [


"deltekst": "Kap. 1","egenskaber": []

]}

]...

To clear some or all the egenskaber of a Dokument Del for a particular virkning period, you shoulduse the empty string to clear the unwanted values. Eg. to clear lokation-egenskab value of Kap. 1 of aoffentliggørelsesvariant for the year 2014 the particular part of the JSON body would look like this:

..."varianter": [


"deltekst": "Kap. 1","egenskaber": [{"lokation": """virkning": {

"from": "2014-01-01","to": "2015-01-01","aktoerref": "ddc99abd-c1b0-48c2-aef7-74fea841adae","aktoertypekode": "Bruger",





"notetekst": "Clearing lokation for 2014"}

}],

]}

]...

Deleting relations of a Dokument Del

To clear all the relations of a particular Dokument Del, you should explictly specify an empty list. Eg. to clear allthe relations of the Kap. 1 Dokument Del of the offentliggørelsesvariant Variant, the specific partof the JSON body would look like this:

..."varianter": [


"deltekst": "Kap. 1","relationer": []

]}

]...

The delete / clear a specific relation of a Dokument Del you have to specify the full list of the relations of theDokument Del sans the relation, that you wish to remove. In general, when updating the Dokument Del relations,you have to specify the full list of relations.

3.7 Request For Comments: Specifikation af serviceinterface forLogHændelse

3.7.1 Introduktion

LoRas log-tjeneste er ikke en OIO-standard og er heller ikke som Tilstand, Indsats og Aktivitet lavet på bag-grund af et forslag om en sådan. Log-tjenesten er specificeret af Magenta på bestilling fra Naturstyrelsen. Dettedokument er den funktionelle specifikation, der lå til grund for den nuværende implementation.

3.7.2 Overordnet sammendrag

Tjenesten LogHændelse bruges til at implementere en benyttelseslog for rammearkitekturservices som Organisa-tion, Klassifikation, Sag, Dokument, Tilstand, Aktivitet og Indsats - i det hele taget alle rammearkitektur-services,som følger OIO- og MOX-protokollerne og udsender standardiserede notifikationer om tilgang til databaserneover AMQP.

Tjenesten kan også bruges af services uden for rammearkitekturen; den eneste betingelse er, at de skal sende log-oplysningerne i et standardiseret format, specifikt som AMQP-beskeder til en nærmere angivet kø eller exhange.

Logningen foregår ved, at logservicen lytter på den AMQP-exchange eller -kø, hvortil notifikationerne fra detjenester, der skal logges, bliver sendt.

Hvert objekt i tjenesten LogHændelse indeholder samtlige oplysninger, der skal logges i forbindelse med benyt-telsen af tjenesterne.

3.7. Request For Comments: Specifikation af serviceinterface for LogHændelse 41


Det drejer sig først og fremmest om

• service-ID (kan være tjenestens navn - er det ikke entydigt nok, kan det være UUID eller brugervendt nøglefor det IT-System, der repræsenterer servicen).

• klasse - den type objekt, som hændelsen gælder.

• tidspunkt for den hændelse, der logges.

• operation, der udføres - for rammearkitekturen søg, læs, ret, slet eller passivér.

• objekttype - altså navnet på den type, for hvilken hændelsen logges.

• objekt, for hvilket logningen foretages. Dette kan sammen med objekttypen og tjenestens navn eller IDentydigt bruges til at identificere og fremfinde det objekt, som logningen gælder.

• bruger - den bruger, der foretog den handling, der logges.

• brugerrolle - den brugerrolle, som brugeren har benyttet for at få adgang til at udføre den pågældendehandling.

• returkode - svarkoden fra servicen, altså f.eks. 200, 404, 403, 500.

• returtekst - fejlmeddelelse fra serveren i tilfælde af, at noget gik galt.

• note - såfremt notefelterne på eventuelle nye registreringer eller virkningsperioder er udfyldt, kan dissekopieres til notefeltet på LogHændelsen. Det vil selvfølgelig være helt op til de services, der brugerLogHændelse, hvad der skal stå i dette notefelt.

3.7.3 Autentificering

Log-servicen kunne i princippet genbruge SAML-tokens fra de oprindelige hændelser og oprette loghændelsernepå vegne af denne bruger.

Logning må imidlertid opfattes som et særskilt ansvarsområde, og det vil derfor være passende, om denne funktionvaretages af en bruger med en særlig rolle, som vi kunne kalde Logningsansvarlig.

Autentificering foregår altså ved, at der for hver af de services, der har adgang til at bruge LogHændelse, oprettesen bruger, der har rollen Logningsansvarlig for netop denne hændelse. Når en tjeneste ønsker at logge en hændelse,skal den danne et SAML-token for dens Logansvarlige bruger og medsende som autentikering.

Når LogHændelse modtager log-beskeden vil den først validere, at det indkommende token er gyldigt, at det erudstedt for IdP’en for den service, der logges for, og at den bruger, der autentikeres, faktisk er Logningsansvarlig.

Et interessant aspekt af denne protokol er, at LogHændelse-tjenesten ved logning ikke autentikerer en af sine egnebruger, men en af klienttjenestens. Dette volder dog ingen vanskeligheder og kræver ikke en fælles brugerdatabase;det kræver kun, at den pågældende IdPs offentlige nøgle registreres på LogHændelse-tjenesten sammen med enidentifikation af servicen.

Af hensyn til performance og fleksibilitet vil det være en stor fordel, om brugerrollerne medtages i de SAML-tokens, der genereres, så log-tjenesten ikke behøver at “kalde tilbage” for at verificere, at brugeren har den rigtigerolle.

3.7.4 Attributter

En LogHændelse har følgende attributter:

• service

• klasse

• tidspunkt (bemærk, dette kan være tidligere end virkning og registrering for log-objektet)

• operation

• objekttype



• returkode

• returtekst

• note

3.7.5 Tilstande

Umiddelbart kan det antages, at en LogHændelse skabes én gang, hvorefter den aldrig ændres. Objektet har derformåske ikke brug for tilstande.

Omvendt kan der måske være brug for at slette hændelser, der ved en fejltagelse er logget for f.eks. en forkertservice, eller for at ændre et notefelt, hvis der ved en fejltagelse er kommet fortrolige data med ud.

Vi har derfor én tilstand:

• Gyldighed - en angivelse af, om LogHændelsen er blevet ændret, siden den blev registreret. Værdimængdener “Ikke rettet” (default) og “Rettet”.

3.7.6 Relationer

En LogHændelse har relationer til ganske få andre objekter.

Dybest set repræsenterer en LogHændelse en registrering af en handling udført af en bruger, der har fået lov ellerer blevet afvist i forhold til en brugerrolle.

Handlingen er én af de operationer, der er beskrevet under de generelle egenskaber for OIO-standardernes objekter.Den implementeres ikke som en objektreference, men som et tekstfelt, hvis værdimængde i praksis er begrænsettil en simpel enumeration af de mulige operationer. Der foretages ingen validering af dette, eftersom det også skalvære muligt at logge hændelser for services, der ikke er en del af OIO-systemet. Handlingen kunne med andre ordhave været en reference til Klasse, men implemtenteres i stedet som tekstfeltet operation.

Brugeren er en reference til Bruger-objektet i Organisation.

Brugerrollen er en reference til Klassifikation. I nogle tilfælde kan brugerens adgangsrettigheder være beskrevetaf roller, der er oprettet ad hoc på IdP’en og ikke er mappet i klassifikation, og i sådanne tilfælde kan rollen angivesved en URN på formen “rolle:oprindelse:rollenavn”. Hvis rollerne senere oprettes i Klassifikation, skal URN’enkunne bruges til at slå rollen op dér.

LogHændelse har dermed følgende til-én-relationer:

• objekt - det objekt, som logningen gælder. UUID eller URN.

• bruger - den bruger, der har udført den handling, der logges. UUID eller URN. Reference til typen Brugeri Organisation.

• brugerrolle - den rolle, som er brugt til at give brugeren adgang (eller ej). Reference til Klassifikation.Såfremt brugerrollerne er ad hoc og ikke oprettet i Klassifikation, hvilket p.t. er tilfældet på reference-data.dk, kan angives en URN.

3.7.7 Operationer

Log-servicen vil tilbyde de samme operationer som de øvrige LoRa-services. Det kan dog overvejes, om ret-tighedsstyringen skal begrænse skriveoperationerne, så det kun er de operationer, der giver mening for dennetjeneste, der skal tillades.

3.7. Request For Comments: Specifikation af serviceinterface for LogHændelse 43


3.8 MOX Agents

Warning: This is outdated. It describes agents which were part of an older iteration of the LoRa stack.

Within the context of the Mox Messaging Service, agents are small pieces of software which either listen on anAMQP queue and perform operations on the incoming data, or expose certain operations as a web service.

The default installation includes the following agents:

MoxDocumentDownload Web service for exporting actual state contents as Excel spreadsheets.

MoxDocumentUpload Web service for importing data from Excel spreadsheets into the actual state database.

MoxRestFrontend AMQP agent bridging the REST API.

MoxTabel AQMP worker agent MoxDocumentDownload & MoxDocumentUpload.

3.8.1 python_agents folder

This directory contains the Notify to AMQP service, which serves as a working example of a relay of the internalPostgresSQL notifications to a publicly available notification service.

Legacy agents

Besides, this directory contains the following 2 (legacy) mox agents:

MOX Advis mox_advis.pyMOX Elk Log mox_elk_log.py

These agents were part of the early iterations of the Lora stack. However, these agents are not used in the currentrelease (may be added in the forseeable future).

The two agents are currently unsupported and not part of the default installation procedure.

Update/Dependencies

MOX Advis & MOX Elk Log have previouly been coupled with the OIO Rest package. The agents have nowbeen de-coupled and do not depend on external settings and python dependencies.

However the agents depend on a local OIO Rest module: saml2.py (Saml2_Assertion class). This dependency isnot Python 3 compatible and will be removed or refactored when the code is moved to Python 3.

Installation

A installation strategy (formula) has been created, however as previously described the agents have unresolveddependencies.

As such they are not automatically installed as part of the stack.

Note: Once the dependencies have been resolved, it should be relatively easy to adjust the installation strategy.



3.9 Authentication

Warning: This document is outdated.

Todo: When updating this as part of #25911. Also update Authentication.

SAML token authentication is enabled by default. This requires that you have access to a SAML Identity Provider(IdP) which provides a Security Token Service (STS). We currently support two types:

• Active Directory Federation Services

• WSO2

3.9.1 Using Active Directory Federation Services

In order to use AD FS as the Security Token Service, you first need an endpoint configured in ADFS. You shouldname this endpoint corresponding to the designated name of the box running LoRA, for example:

https://lora.magenta.dk

As for the attributes to send, select the following:

LDAP Attribute Outgoing Claim TypeobjectGUID PPIDUser-Principal-Name NameIDToken-Groups (Unqualified Names) Group

Please note that you should configure AD FS to sign, but not encrypt, its assertions.

Then configure the following fields in oio_rest/oio_rest/settings.py:

Setting DescriptionSAML_MOX_ENTITY_IDIn this case, “https://lora.magenta-aps.dk”.SAML_IDP_ENTITY_IDThe name of your ADFS.SAML_IDP_URL The URL where your ADFS may be reached.SAML_IDP_TYPE "adfs"USE_SAML_AUTHENTICATIONTrueSAML_USER_ID_ATTIBUTE"http://schemas.xmlsoap.org/ws/2005/05/identity/claims/

privatepersonalidentifier"

You should now be able to test the basic configuration, and extract the signing certificate:

$ cd /path/to/mox$ ./auth.sh --cert-onlyUser: user@domainPassword: <enter password here>

Now save the results to a file, e.g. adfs-cert.pem, and set that as SAML_IDP_CERTIFICATE. You may getan SSL error, in that case, you should add your certificate authority to the system. Alternatively, you can pass the--insecure option to auth.sh temporarily bypass the error.

3.9. Authentication 45


3.9.2 Using WSO2 for testing

The open source identity provider WSO2 is useful for testing. Download the binary and follow the instructions torun it.

In the folder wso2/ you can find an example init file for running the WSO2 Identity Server as a daemon.

To configure a STS, follow the instructions on https://docs.wso2.com/display/IS500/Configuring+the+Identity+Server+to+Issue+Security+Tokens (skip the part about Holder of Key).

Restart the WSO2 server! The STS endpoint simply did not work until I restarted the WSO2 server.

Setting up users on the IDP

This is for testing with the WSO2 Identity Server as described above - we assume that this is not the configurationwhich the municipalities want to use in a production setting.

Log in to the IDP with the credentials provided. The IDP could, e.g., be located at https://moxtest.magenta-aps.dk:9443/.

To create a new user, enter the “Configure” tab and select “Users and roles”. Enter the user’s first name, last nameand email address.

Important: In the URL field, enter the user’s (OIO) UUID. The URL field is currently used to map between theIDP and the OIO’s user concept. If the UUID is not specified, it will not be possible to authorize users correctly,nor will it be possible to make any changes to the database.

OIO-REST SAML settings

The default IdP entity ID is called “localhost”. If your IdP has a different entity ID, you must change theSAML_IDP_ENTITY_ID setting to reflect your IdP’s entity ID.

For testing purposes, WSO2’s IdP public certificate file is included in the distribution.

When configuring the REST API to use your IdP, you must specify your IdP’s public certificate file by setting insettings.py:

export SAML_IDP_CERTIFICATE=/path/to/idp_certificate.pem

In settings.py, SAML authentication can be turned off by setting:

USE_SAML_AUTHENTICATION = False

3.9.3 Requesting a SAML token using the OIO REST service

The OIO REST service provides a convenience method for requesting a SAML token in the correct base64-encoded gzipped format for use with the API.

Visit the following URL of the OIO REST server:

http://localhost:8080/get-token

Alternatively, you can run the following command locally on the server:

$ ./auth.sh -u <username> -p

You will be presented with a form with a username/password field. Optionally, you can specify the STS addressto use. This will request a token from the STS service using the given username and password. It will return thevalue that should be used for the HTTP “Authorization” header. If it fails due to invalid username/password, anerror message will be returned.

This value can then be included in the HTTP “Authorization” header, like the following:


http://wso2.com/products/identity-server

https://docs.wso2.com/display/IS500/Configuring+the+Identity+Server+to+Issue+Security+Tokens

https://docs.wso2.com/display/IS500/Configuring+the+Identity+Server+to+Issue+Security+Tokens

https://moxtest.magenta-aps.dk:9443/

https://moxtest.magenta-aps.dk:9443/


Authorization: <output of get-token>

For testing purposes, we recommend curl or the browser extensions Advanced REST client for Chrome or RESTEasy for Firefox.

3.9.4 Requesting a SAML token manually

Note: This section only applies covers using the WSO2 IdP.

Although the Java MOX agent does this automatically, it can be useful to request a SAML token manually, fortesting purposes.

To request a SAML token, it is useful to use SoapUI.

Download SoapUI and import the project provided in oio_rest/test_auth_data/soapui-saml2-sts-request.xml.

Navigate to and double-click on:

"sts" -> "wso2carbon-stsSoap11Binding" -> "Issue token - SAML 2.0"

Note: The value of <a:Address> element in <wsp:AppliesTo> must match yourSAML_MOX_ENTITY_ID setting. Change as needed.

The project assumes you are running the IdP server on https://localhost:9443/ (the default).

Execute the SOAP request. You can copy the response by clicking on the “Raw” tab in the right side of thewindow and then selecting all, and copying to the clipboard. Paste the response, making sure that the originalwhitespace/indentation is preserved. Remove all elements/text surrounding the <saml2:Assertion>..</saml2:Assertion> tag. Save to a file, e.g. /my/saml/assertion.xml.

After requesting a SAML token, to make a REST request using the SAML token, you need to pass in an HTTPAuthorization header of a specific format:

Authorization: saml-gzipped <base64-encoded gzip-compressed SAML assertion>

A script has been included to generate this HTTP header from a SAML token XML file. This file must onlycontain the <saml2:Assertion> element.

To run it:

$ python oio_rest/oio_rest/utils/encode_token.py /my/saml/assertion.xml

The output of this script can be used in a curl request by adding the parameter -H, e.g.:

$ curl -H "Authorization saml-gzipped eJy9V1................." ...

to the curl request.

Alternately, if using bash shell:

$ curl -H "$(python oio_rest/oio_rest/utils/encode_token.py" /my/saml/assertion.→˓xml) ...

3.10 Deployment

3.10.1 System requirements

LoRA currently supports Ubuntu 16.04 (Xenial). We recommend running it on a VM with the following allocation:

3.10. Deployment 47

https://chrome.google.com/webstore/detail/advanced-rest-client/hgmloofddffdnphfgcellkdfbfbjeloo

https://addons.mozilla.org/da/firefox/addon/rest-easy/

https://addons.mozilla.org/da/firefox/addon/rest-easy/

http://www.soapui.org/

https://localhost:9443/


CPU Memory Storage Disk typeMinimal 1 core 2 GB 15 GB any (SSD or HD)Test & development 2 cores 4 GB 30 GB SSD (recommended)Production 4 cores 8 GB 60 GB SSD

You should initially provision all the storage space you expect to use, as adjusting it is somewhat cumbersome.By comparison, increasing or decreasing CPU and memory is trivial.

3.10.2 Getting started

Warning: FIXME Fillout this section.

In a production environment, it is recommended to bind the oio_rest service to a unix socket, then expose thesocket using a HTTP proxy of your own choosing.

(Recommended: Nginx or Apache)

More on how to configure in the advanced configuration document. Link: Document is currentlybeing written

3.11 Settings

Most configurable parameters of oio_rest can be injected with environment variables. Alternatively, you mayset the parameters in a file.

Environment variables overwrite default values. The settings file overwrites both default values and environmentvariables.

CONFIG_FILEDefault: None

Setting the environment variable CONFIG_FILE to a path to a JSON file containing a dictionary of"<setting>": <value> pairs will load the settings specified in the file.

The settings from the file have precedence over any environment variables and will overwrite their values.

BASE_URLDefault: "" (Empty string)

Prefix for all relative URLs. A value of "/MyOIO" will give API endpoints such as http://example.com/MyOIO/organisation/organisationenhed.

3.11.1 Database

PostgreSQL

DB_HOSTDefault: "localhost"

The host to use when connecting to the database.

DB_PORTDefault: None

The port to use when connecting to the database specified in DB_HOST.



DB_USERDefault: "mox"

The username to use when connecting to the database.

DB_PASSWORDDefault: "mox"

The password to use when connecting to the database.

DATABASEDefault: "mox"

The name of the database to use.

DB_MIN_CONNECTIONSDefault: 0

Per-process lower limit on the amount of database connections. Setting it to a non-zero value ensures thatthe web application opens this amount at load, failing if the database isn’t available.

DB_MAX_CONNECTIONSDefault: 10

Per-process upper limit on the amount of database connections.

DB_STRUCTUREDefault: "oio_rest.db.db_structure"

The structure of the whole database. Overwrite this if you want to extend the database with additional fieldson the objects.

3.11.2 File upload

FILE_UPLOAD_FOLDERDefault: "/var/mox"

This path is where file uploads are stored. It must be readable and writeable by the system user running theREST API server. This is used in the Dokument hierarchy.

3.11.3 Audit log

An audit log is published as AMQP messages and written to a dedicated queue.

LOG_AMQP_SERVERDefault: ""

The AMQP server used to publish the audit log. If empty, audit logging is off.

Not to be confused with the AMQP service used by /python_agents/notification_service/notify_to_amqp_service.py.

MOX_LOG_EXCHANGEDefault: "mox.log"

The AMQP exchange used for the audit log.

MOX_LOG_QUEUEDefault: "mox.log_queue"

The AMQP queue used for the audit log.

3.11. Settings 49


3.11.4 Authentication

Todo: Fix this whole section as part of #25911.

LoRa has two independent ways to use SAML. An older one from the file mox/oio_rest/oio_rest/auth/saml2.py and a newer one from the package flask_saml_sso. Only use one of them at a time. They are bothdisabled by default. For an overview of how mox/oio_rest/oio_rest/auth/saml2.py works, see Au-thentication.

SAML from mox/oio_rest/oio_rest/auth/saml2.py

USE_SAML_AUTHENTICATIONDefault: False

Whether to enable SAML authentication from mox/oio_rest/oio_rest/auth/saml2.py.

SAML_MOX_ENTITY_IDDefault: "https://saml.local'"

The Endpoint specified in the AppliesTo element of the STS request. This will be used to verify theAudience of the SAML Assertion.

SAML_IDP_ENTITY_IDDefault: "localhost"

The Entity ID of the IdP. Used to verify the token Issuer specified in AD FS as the Federation Serviceidentifier.

Example: "http://fs.contoso.com/adfs/services/trust"

SAML_IDP_URLDefault: "https://localhost:9443/services/wso2carbon-sts.wso2carbon-stsHttpsEndpoint"

The URL on which to access the SAML IdP.

Example: "https://fs.contoso.com/adfs/services/trust/13/UsernameMixed"

SAML_IDP_TYPEDefault: "wso2"

We currently support authentication against wso2 and adfs.

SAML_IDP_CERTIFICATEDefault: "test_auth_data/idp-certificate.pem"

The public certificate file of the IdP, in PEM-format.

SAML_USER_ID_ATTIBUTEDefault: "http://wso2.org/claims/url"

SAML user ID attribute. Default is for WSO2

Example: "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/privatepersonalidentifier"

SAML from flask_saml_sso

Refer to the readme for flask_saml_sso for these settings.

SAML_AUTH_ENABLEDefault: False

Enables SAML authentication from flask_saml_sso.


https://github.com/magenta-aps/flask_saml_sso

https://github.com/magenta-aps/flask_saml_sso


SAML_IDP_METADATA_URLDefault: "https://172.16.20.100/simplesaml/saml2/idp/metadata.php"

SAML_IDP_INSECUREDefault: False

SAML_REQUESTS_SIGNEDDefault: False

SAML_KEY_FILEDefault: None

SAML_CERT_FILEDefault: None

SQLALCHEMY_DATABASE_URIDefault: "postgresql://sessions:[email protected]/sessions"

SESSION_PERMANENTDefault: True

PERMANENT_SESSION_LIFETIMEDefault: 3600

3.11.5 Restrictions

Todo: When writing authentication documentation #25911, include a section on restrictions and link to it fromhere.

DO_ENABLE_RESTRICTIONSDefault: False

Whether authorization is enabled and restrictions can be used. If not, the AUTH_RESTRICTION_MODULEis not called.

AUTH_RESTRICTION_MODULEDefault: "oio_rest.auth.wso_restrictions"

The module which implements the authorization restrictions. Must be present in sys.path.

AUTH_RESTRICTION_FUNCTIONDefault: "get_auth_restrictions"

The name of the function which retrieves the restrictions. Must be present inAUTH_RESTRICTION_MODULE and have the correct signature.

3.12 Database structure

The db/ folder contains the SQL source code for initializing the underlying database for LoRA.

3.12.1 Testing

You can run the database tests with the following command:

$ pg_prove --dbname mox --username mox --schema test

Alternatively, you can run them within the Python test harness:

(python-env) $ python -m unittest -vbf tests.test_sql

3.12. Database structure 51


The tests are written in PL/pgSQL using the pgTAP framework and located in tests/.

3.12.2 Organisation

The templates are organised as follows:

basis Initial definitions, structures, extensions, etc.

funcs/pre Initial function definitions.

templates The above-mentioned templates specifying core types, tables, functions, etc. for each

funcs/post Function definitions requiring the core tables and types.

tests The above-mentions PL/pgSQL tests; not loaded into a production database.

3.12.3 Templates

We generate many PostgreSQL functions and definitions from Jinja2 templates:

• ./templates/ contains the Jinja2 templates.

• ../oio_rest/tests/fixtures/db-dump.sql contains an up-to-date version of the output.

After updating the templates, run the tests mentioned above. One of the tests compares the rendered templates toa fixture, and will fail if the changes affect the resulting SQL. To fix this, replace the old results with the new ones:

$ mv ../oio_rest/tests/fixtures/db-dump.sql.new \> ../oio_rest/tests/fixtures/db-dump.sql

Then, inspect the changes using e.g. git diff.

3.13 Wildcards

When making a Search operation with wildcards, a lot happens. Going from the bottom of the stack the followinghappens:

In SQL the match between two strings is made with ILIKE from PostgresSQL. It is a case-insensitive string matchthat supports the wildcard operators _ (underscore) to match a single character and % (percent sign) to match zeroor more characters. To match a litteral underscore or percentage sign you must escape it with a single backslashas \_ or \%

In /oio_rest/oio_rest/utils/build_registration.py all incomming underscores _ are re-placed by an escaped underscore \_. This means it is not possible for the REST API user to send an underscorewildcard operator to SQL. This may change in the future.

Flask et. al. take care of decoding RFC 3986 percentage-encoded URIs. This means a percentage sign followedby two case insensitive hexidecimal signs (0-F) is decoded into the corresponding ASCII symbol. E.g. a %45in an URI is decoded to E in SQL and %25 is decoded to %. However a percentage sign followed by somethingwhere the first character is anything other than a hexidecimal sign is ignored. E.g. %g4 in an URI also remains%g4 in SQL and %magenta remains %magenta.

These three mechanisms combined results in some very novel behavior. See the table below:


https://pgtap.org

https://palletsprojects.com/p/jinja/




URI Query PGSQL RegEx Matchesjkl jkl ^jkl$ the string jkl_jkl \_jkl ^_jkl$ a litteral underscore followed by jkl%jkl/%25jkl %jkl ^.*jkl$ zero or more of any characters followed by jkl%abc c ^c$ the character 0xAB followed by c%afdeling deling ^deling$ the character 0xAF followed by delingE6/%456 E6 ^E6$ the string E6%25456 %456 ^.*456$ zero or more of any characters followed by 456\_jkl \\_jkl ^\\_jkl$ a \ followed by any character followed by jkl

The best way to avoid most of the confusion is to always percentage-encode your URI and never write \_ in yourquery.

Attention: Always use RFC 3986 percentage-encoding for your search URI!

3.14 Testing

OIO Rest has its own unit test suite:

$ cd oio_rest$ ./run_tests.sh

This will automatically generate a virtual environment, and run the tests in it.

3.15 Systematic testing of the OIO REST interface

Note: This document is work in progress.

The OIO REST interface (or parts of it) will be systematically tested (kind of) using Equivalence Class Partition-ing and Myers Heuristics (a good reference describing these techniques can be found here). Please note, that theequivalence classes (ECs) used here are a little fuzzy and the union of them do NOT form the full input set to theinterface.

3.15.1 OrganisationOpret

Equivalence class partitioning:

3.14. Testing 53


http://www.baerbak.com/


Condition Invalid ECs Valid ECsNote Note not a string [1] Zero notes [2], One note [3]Attr, BVN BVN missing [4], BVN not a string [5] Exactly one BVN [6]Attr, BVN (BVN consists of special characters [7])Attr, OrgName OrgName not string [8] No OrgName [9], OrgName string [10]Attr, Virkning Virkning missing [11], Virkning malformed

[12]Virkning correct [13]

Attr, No of attrs OrgEgenskaber missing [14] Two OrgEgenskaber present (no overlaps)[15]

Attr, Virkning Different OrgNames for overlapping virkn-ings [16]

Empty org Empty org [17]Attr Attr missing [18], Two attr objects [40]Tilstand, num-ber

Tilstand missing [19], Two tilstande objects[41]

Tilstand,orgGyld

OrgGyldighed missing [20] One valid OrgGyld [21], Two validOrgGyld [22]

Tilstand,gyldigh

Gyld not aktiv or inaktiv [23], gyld missing[26]

gyldighed aktiv [24], gyldighed inaktiv [25]

Tilstd, virkning Virkning missing [27], Virkning malformed[28]

Virkning valid [29]

Tilstd, virkning Different gyldighed for overlapping virkning[30]

Rel, object Two relationobjects present [39] Relation object missing [38]Rel, number Zero allowed [31], one allowed [32], two

[33]Rel, number Specific relation list empty [42]Rel, reference Reference not an UUID [34] Reference is an UUID [35]Rel, name Invalid relation name not allowed [36] All valid relation names allowed [37]

3.15.2 FacetOpret (JSON validation)

Equivalence class partitioning:



Condition Invalid ECs Valid ECsNote Note not a string [43] No notes [44], Note a string[45]Attr, BVN BVN missing [46], BVN not a string [47] BVN a string [48]Attr,Facetbeskriv

Not a string [49] Missing [80], beskrivelse is a String [50]

Attr, Facetplan Not a string [78] Missing [79], beskrivelse is a String [81]Attr, Facetop-byg


Attr, Face-tophavs


Attr, Facetsuppl Not a string [88] Missing [89], beskrivelse is a String [90]Attr, Retsklide Not a string [91] Missing [92], beskrivelse is a String [93]Attr, Virkning Virkning missing [51], Virkning malformed

[52]Virkning correct [53]

Attr, No of attrs Egenskaber missing [54] One egenenskaber [77],two egenskaberpresent [55]

Attr, unknown Unknown key [94]Empty facet Empty facet [56]Attr Attr missing [57]Tilstand, num-ber

Tilstand missing [58]

Tilstand,FacetPub

Missing [60] One valid FacetPubl [61], Two validFacetPub [62]

Tilstand,FacetPub

Pub not valid enum [61], pub missing [62] Pub = pub [63], pub = IkkePub [64]

Tilstd, virkning Virkning missing [65], Virkning malformed[66]

Virkning valid [67]

Tilstd, un-known

Unknown key [95]

Rel, object Relation object missing [68]Rel, number Zero allowed [69], one allowed [70], two

[71]Rel, number Two references in nul til en relation [96] Specific relation list empty [72]Rel, reference Reference not an UUID [73] or URN [114] Reference is UUID [74], reference is URN

[112]Rel, reference UUID and URN not allowed simultane-

ously [113]Rel, name Invalid relation name not allowed [75] All valid relation names allowed [76]Virkning, from missing [97], not string [98] String [101]Virkning, to missing [99], not string [100] String [102]Virk, aktoerref not UUID [103] Is UUID [104], missing [108]Virk, aktoer-type

Not string [105] String [106], missing [109]

Virk, notetekst Not string [107] String [110], missing [111]

More cases to come. . .

3.15.3 Myers Heuristics

The test cases will be constructed using Myers Heuristics following (in general) these rules (taken from the abovereference):

1. Until all valid ECs have been covered, define a test case that covers as many uncovered valid ECs as possible.

2. Until all invalid ECs have been covered, define a test case whose element only lies in a single invalid EC.

3.15. Systematic testing of the OIO REST interface 55


3.15.4 Boundary conditions

Check virkning. . .

3.15.5 TODO

Test registrations. . . Test virkning. . .

3.16 Building documentation

This README file is part of our documentation, and an HTML version can be obtained by the following commandin a command prompt:

$ make -C doc

Note that this requires Sphinx to be installed — on Ubuntu or Debian, this can be done with the following com-mand:

$ sudo apt install python-sphinx

3.17 Sending Messages on the Beskedfordeler

3.17.1 Using the OIO Mox Library

Caution: This section is currently out of date!!! Please refer to the examples above for more up to datedetails.

This is located in the folder agent/ in the Mox source code repository.

The library is built with Apache Maven - see pom.xml for Maven dependencies.

To send a command through the message queue, you first need a ObjectType representing the type of objectyou want to manipulate.

A collection of these can be defined in a properties file and loaded with

Map<String, ObjectType> objectTypes = ObjectType.load(File propertiesFile)

or

Map<String, ObjectType> objectTypes = ObjectType.load(Properties properties).

The properties must contain a set of keys adhering to the format:

type.[name].[operation].method = [method]type.[name].[operation].path = [path]

For example:

type.facet.create.method = POSTtype.facet.create.path = /klassifikation/facet



The default agent.properties file defines all of the classes from the OIOXML hierarchies Klassifikation, Organisa-tion, Sag and Dokument.

You can then get your ObjectType by calling get(String name) on the returned collection.

If you instead want to create your ObjectType yourself, you can create a new ObjectType(String name)and add operations to it with

addOperation(String name, ObjectType.Method method, String path)

where

• name denotes the type of operation (usually “create”, “update”, “passivate” or “delete”, but you can specifyyour own)

• method denotes the HTTP method to use when connecting to the REST interface. Available are: GET,POST, PUT, PATCH, DELETE and HEAD)

• path denotes the REST path, e.g. “/klassifikation/facet/[uuid]”, and [uuid] will be replaced with a uuidyou specify when calling the operation

You also need a MessageSender object, which can be created with:

new MessageSender(String queueInterface, String queue);

where

• queueInterface is a hostname/port combination to the RabbitMQ instance, e.g. “localhost:5672”, and

• queue is the RabbitMQ queue name, e.g. “incoming”.

The queue name and interface port must match what the queue listener is set up to use; the oio_moxagent listeneris currently configured to use the queue “incoming” for the RabbitMQ service on port 5672.

Now that you have an ObjectType instance and a MessageSender, you can call any of the following methods:

create

Future<String> create(MessageSender sender, JSONObject data)Future<String> create(MessageSender sender, JSONObject data, String authorization)

Sends a ‘create’ operation to the message queue, provided that a ‘create’ operation has been defined in theObjectType. Put your JSON document in the data field, and include an optional authorization token for theREST interface. The demonstration class already contains example code on how to obtain such a token (seethe getSecurityToken() method in Main.java) The function immediately returns a Future<String>handle, which can be used to obtain the server response. Calling the get() method on this handle blocks until aresponse is ready, and then returns it in a String.

update

Future<String> update(MessageSender sender, UUID uuid, JSONObject data)Future<String> update(MessageSender sender, UUID uuid, JSONObject data, String→˓authorization)

Sends an ‘update’ operation to the message queue, provided that an ‘update’ operation has been defined in theObjectType. Add the document UUID to be updated, as well as the JSON document you’re updating with.

passivate

Future<String> passivate(MessageSender sender, UUID uuid, String note)Future<String> passivate(MessageSender sender, UUID uuid, String note, String→˓authorization) (continues on next page)

3.17. Sending Messages on the Beskedfordeler 57



Sends a ‘passivate’ operation to the message queue, provided such an operation has been defined in the Object-Type. Add the document UUID to be passivated, as well as a note to go with the passivate operation (may benull).

delete

Future<String> delete(MessageSender sender, UUID uuid, String note)Future<String> delete(MessageSender sender, UUID uuid, String note, String→˓authorization)

Sends a ‘delete’ operation to the message queue, provided such an operation has been defined in the ObjectType.Add the document UUID to be deleted, as well as a note to go with the delete operation (may be null).

sendCommand

Future<String> sendCommand(MessageSender sender, String operationName, UUID uuid,→˓JSONObject data)Future<String> sendCommand(MessageSender sender, String operationName, UUID uuid,→˓JSONObject data, String authorization)

Sends a custom operationName (useful if you added an operation other than create, update, passivate or delete).Add a UUID and a JSON Object as needed by the operation.

This is the more general function, which is used to implement the other operations.

3.17.2 Using AMQP Messages

If you do not wish to use the Java library described above, you can send messages directly to the AMQP queuewhere the message handler is running.

The message handler will recognize four AMQP headers when sending Mox messages:

• “autorisation” - must contain the SAML token as described above.

• “objektID” - must contain the UUID of the object to manipulate; not used with create operations.

• “objekttype” - i.e., OIO class, e.g. “Facet”.

• “operation”, the action to be performed. Must be one of “create”, “update”, “passivate” or “delete”.

Import operations can be performed with the “update” command - but note that it’s also possible to map newcommands by editing the agent.properties file as described above. This could also be used to specify readoperations with GET, if so desired.

The content of the commands, i.e. the actual data, are send as the payload of the messages. Note that while it ispossible to specify a URL when uploading a document, it is currently not possible to upload the binary contentsof a document through the message queue - for this, the REST interface must be used directly.

For an example of how to create and send Mox messages with Java, please see the file ObjectType.java in agent/src/main/java/dk/magenta/mox/agent.

Notification Messages

Each time a write operation (create/import/passivate/update/delete) is performed, an internal notification messagesis sent out in the PostgreSQL database, using the Notify system. These messages can be read from PostgresSQLand relayed to other services as needed. An example of this is provided in the form of notify_to_amqp_service.py



and the corresponding systemd rules which allows the program to run as a service. This service will be installedpr default by the installer. The serivice will create an AMQP message exchange called “mox.notifications”.

To query the status of the service, run the command:: sudo systemctl status notification

systemctl can also be used to start and stop the service.

The notification message consists of a JSON-string with the following keys:

• “beskedtype” - always contains the value ‘Notification’

• “objektID” - contains the UUID of the object.

• “objekttype” - i.e., OIO class, e.g. “Facet”.

• “livscykluskode” - i.e. ‘Opstaaet’, ‘Importeret’, ‘Passiveret’, ‘Slettet’ or ‘Rettet’

3.18 Version history

3.18.1 Version 1.3.0, 27 May 2019

This release further introduces Docker support.

• The initdb functionality has been reimplemented in python.

• Outdated sections have been removed from the documentation.

• Docker support has been further expanded with a copy service responsible for copying Postgres-specificfiles out of the image, for initializing a Postgres container.

3.18.2 Version 1.2.0, 27 May 2019

This feature introduces Docker support. A Dockerfile has been added for creating a Docker-container con-taining the oio_rest application.

A docker-compose file has been added for setting up a full development environment, including database.

New features:

• Docker support, with docker-compose for development setups.

• AMQP audit logging is now disabled by default.

3.18.3 Version 1.1.0, 26 March 2019

New features:

• Python dependencies updated.

• Delay import of DB_STRUCTURE file until after configuration file is read.

• Improve speed and configurability of test databases. Disable fsync for speed and allow using long direc-tory names without failing.

• Fix calculation of BASE_DIR in tests.

• Make API endpoints trailing slash agnostic.

• Clear caches in test code when patching DB structures.

• Simplified validation of objects with additional attributes.

• Searching for/filtering on Boolean attributes fixed.

• Proof-of-concept infrastructure for Gitlab Runners added.

3.18. Version history 59


• New dummy SQL fixtures for integration with the MO test suite.

Documentation improved:

• API reference documentation vastly improved.

• API tutorial updated and made more user friendly.

• Behaviour of wildcard searches properly documented.

• Proper documentation of settings.py.

3.18.4 Version 1.0.0, 28 January 2019

New in this version:

• Fixes to JSON validation.

• Installation script was refactored to reflect that we currently don’t support the AMQP agents.

• Use the OS2MO test suite as part of the LoRa test suite.

• Consolidate the REST API into one package, including the database generation.

• Abolish patch system for database variations - all SQL is now generated by Jinja templates.

• JSON schemas exposed in new end point in the OIO REST API.

• Fix JSON validation to reflect discrepancy between the Dokument standard and the current implementation.

• Proper use of SAML for authentication.

• Fix bug in search results that gave multiple instances of the same hit in some cases.

• Fix overly-aggressive validation of URL parameters.

• Security fix: All Python dependencies upgraded.

• Restructuring, refactoring and removal of old test files.

• New “integration_data” property added to all objects in LoRa.

• Change (refactor) overall LoRa configuration.

• Add MPL license boiler plate to all source files.

• Restructured and improved Sphinx documentation.

• Complete overhaul of REST API documentation.

• Database must use a Unix socket rather than TCP whenever possible.

• Make tests pass regardless of time zone on server.

• Fix bug that meant database installation code would sometimes be run as superuser rather than designateddatabase user.

• Database generation no longer ignores “mandatory” metadata field.

3.18.5 Version 0.9.2, 13 December 2018

Hotfix:

• Upgrade Requests to version 1.21.4.

• Upgrade Flask to version 1.0.2.


https://requests.readthedocs.io/

https://palletsprojects.com/p/flask/


3.18.6 Version 0.9.1, 30 August 2018

Hotfix:

• Added support for verifying SAML2 assertions already wrapped in responses.

3.18.7 Version 0.9.0-post3, 18 June 2018

Fix issue in installer related to permissions


Fix installer, accidentally broken in previous post-release.


This hotfix contains no code changes, but updates the version metadata for the oio_rest package.

3.18.10 Version 0.9.0, 7 May 2018

This is a major version, including non-backwards-compatible changes to the REST protocol, parameter checks,JSON input validation, unit tests and integration tests.

Backwards incompatible changes:

• Allow replacing an object with PUT - updates now use PATCH (non- compatible with previous versions).

• Return an explicit error (410 Gone) on attempts to access a deleted object.

• Validate query parameters given during search, return an error when given unrecognised or unsupportedarguments.

• JSON Schema validation when creating new objects.

New features:

• Comprehensive test suite, including:

– Unit tests of our REST API.

– Integration tests based on the standards.

– End-to-end integration tests of the REST API and database layer.

– Continuous integration infrastructure that runs our test suite on each push to GitHub.

• Allow requesting objects at a certain registration or validity time. Previously, we either allowed searchingwithin an interval or the current time.

• Migration to Python 3; minimum version required is now Python 3.5 rather than 2.7.

• New and improved installer based on SaltStack to enable provisioning of client installations.

Bug fixes:

• An update with an empty list of relations deleted all relations.

• Fixed semantics for DELETE, so that we no longer merge old entries when reviving an object.

3.18.11 Version 0.3.1.1, 4 September 2017

This hotfix adds a missing import to ‘settings.py.base’; other than that, there is change of functionality.



3.18.12 Version 0.3.1, 23 August 2017

This is a minor update that fixes searching by validity.

3.18.13 Version 0.3.0.1, 28 March 2017

This quick hotfix addresses a missing variables that broke installation.

3.18.14 Version 0.3.0, 28 March 2017

This is a major release, adding four new services:

• Tilstand

• Indsats

• Activity

• Log

As a result of these changes, you’ll need to add the configuration for the new Log service to ‘settings.py’. Pleasenote that the installer does not add these new services to the database automatically.

In addition, it fixes the following bugs:

• Use DMY date order in this file.

• Fix running interface_tests on Darwin, i.e. macOS.

• Fix searching for document attributes and relations.

• Update the README, and factor out API documentation to a separate file.

• Reduce size of settings.py by moving the database structure definition to a separate Python module

• Install mox_advis by default

3.18.15 Version 0.2.17, 8 February 2017

This version contains various installer cleanups, including:

• Don’t prompt for WSO installation during install - it’s broken

• Consolidate all Python virtual environemnts into one

• Add support for Ubuntu 16.04 Xenial Xerus

• Fix agents by using ‘localhost’ for AMQP queues

• Install the headless JDK

• Fix installing with recommended dependancies turned off system-wide

• Fix initial install – don’t assume users exist

• Suppress prompt for resetting the database, and factor out doing to a separate script

• Don’t overwrite pre-existing configurations when re-installing

• Handle SSL errors gracefully in ‘auth.sh’

In addition, the README was updated to document how to set up AD FS authentication.



3.18.16 Version 0.2.16.1, 12 January 2017

Hotfix:

• Fix check for SAML authentication in get-token template

• Fix reading user name from prompt in ‘auth.sh’ script

3.18.17 Version 0.2.16, 10 January 2017


• Minor bug fixes for installer

• Factor out JDK installation to a separate script

• Consolidate WSGI webapp installers



• Converted spreadsheet download to a python Flask webservice

• Converted spreadsheet upload to a python Flask webservice

• Stability, configuration and verbosity update to moxrestfrontend

• Consolidated common classes & utilities to share between agents

• Simplified apache installation & configuration

• Created common install & config utilities, to avoid the same boilerplate code in install files

• Refactored get-token to support authentication against WSO2 and AD FS.

3.18.19 Version 0.2.14.1, 30 June 2016

Hotfix:

• Fix buggy Apache configuration.

• Commit new configuration to git.

3.18.20 Version 0.2.14, 28 June 2016


• Service to extract data to csv files

• Enhanced upload of spreadsheets, where multiple update rows merge into one update

• Bugfix: Tolerate ods files that Apache ODF Toolkit can’t parse

• Bugfix: Parse excel numbers as strings, not doubles (to avoid scientific notation)

• Configuration using environment-specific files and symlinks

• User documentation added with instructions for user management in WSO2

• Technical documentation updated with LIST operation

• Role-based access control implemented in WSO2

• Thorough documentation of how to use REST interface (examples in curl)



3.18.21 Version 0.2.13.3, 27 April 2016

Hotfix:

• Fix README and installation procedure.

3.18.22 Version 0.2.13.2, 19 April 2016

Hotfix:

• Place Tomcat dependencies where the installer can find them

• Create settings.py soft link before running database installation.

3.18.23 Version 0.2.13.1, 19 April 2016

Hotfix:

• Fix installation order of Java components (dependencies).

3.18.24 Version 0.2.13, 3 March 2016


• Reorganize Agents into distinct entities, with reusable classes defined in depencency modules

• Put server-specific config (development, testing, production) in separate files, and symlink to them as needed

• Set up demonstration servlet to receive file uploads

• Rename message queues by their recipient

3.18.25 Version 0.2.12.1, 15 February 2016

Hotfix:

• Mox Advis should not crash if receiving one UUID as string.

3.18.26 Version 0.2.12, 4 January 2016


• Read operation now supports registreringFra/Til parameters.

• Update README documentation to fix typo and to explain that the date range filters use the overlap oper-ator.

• Registrering JSON results now include the “TilTidspunkt” date range. IMPORTANT: The script indb/updates/update-2016-01-04.sh should be run (from the same directory) to update the database for thischange.

• Java components split into modules and ordered under that folder

• Servlet architecture set up

• Spreadsheet servlet begun





• Mox agent Mox Advis.

• Display JSON for class structures at e.g. /sag/classes

• Bug in Update Klassifikation due to wrong formatting of empty array.

3.18.28 Version 0.2.10, 3 November 2015


• aktoerref and notetekst should not be mandatory in Virkning.

3.18.29 Version 0.2.9, 26 October 2015


• Enhanced logging for java mox listener

3.18.30 Version 0.2.8, 7 October 2015


• AMQP listener now accepts mixed-case values for headers objectType and operation

• AMQP listener throws more error messages back through the defined response channel, rather than stayingsilent.

3.18.31 Version 0.2.7, 23 September 2015


• AMQP interface for read, search and list operations.

• Refactored agent.properties settings with standardized naming.



• Bugfix: For LIST operation, virkning parameters default to the current date/time.

• Improved documentation of search/list operation virkning/registrering parameters.



• Added support for RabbitMQ credentials ‘queueUsername’ and ‘queuePassword’ When specifying a user,please make sure that he is created in the RabbitMQ server, and that he has access to /





• Output Authorization header in easier-to-copy-and-paste format than the previous JSON output.

• Close the agent.sh process in /get-token after opening it.

• Better error-handling in /get-token callback for invalid passwords.

• Fix: Java agent’s “gettoken” command did not use the supplied username/password, but instead read themfrom the agent.properties file.

• Fix security vulnerability: /get-token callback did not escape command arguments to agent.sh script.



• Fix for bug in previous hotfix related to /get-token script.



• Fix for /get-token script to take into account proper location of agent.sh script.



• REST Interface implements a form for requesting SAML token from at the URL “/get-token”.

• Java agent client supports getting token via command-line, using “gettoken <username>” command.

• Updated sample SOAP project to request the SAML token to include the “URL” claim, which is needed inthe test setup, as it supplies the user’s UUID to the REST API.

• Fix parsing of MOX agent “-D” parameters.

• Add WSO2’s nexus repository to Java agent Maven project.



• REST interface for the OIO services Sag, Dokument, Organisation and Klassifikation.

• Database implementing the same hierarchies.

• Complete redesign of database.

• Support for authentication with SAML tokens.

3.18.39 Version 0.1.1, 9 March 2015


• Added missing classes from the Organisation hierarchy.



3.18.40 Version 0.1.0, 23 February 2015

Initial release.

• Status is “alpha”

• First version of ActualState database has been handed over to KL and Frederiksberg Kommune for testing.


CHAPTER 4

Indices and tables

• genindex

• modindex

• search

69


70 Chapter 4. Indices and tables

HTTP Routing Table

/(service)GET /(service)/(object), 21GET /(service)/(object)/(uuid), 16GET /(service)/(object)/fields, 15GET /(service)/(object)/schema, 15GET /(service)/(object)?uuid=(uuid),

18GET /(service)/classes, 15POST /(service)/(object), 24PUT /(service)/(object)/(uuid), 37DELETE /(service)/(object)/(uuid), 36PATCH /(service)/(object)/(uuid), 25

/site-mapGET /site-map, 15

71


72 HTTP Routing Table

Index

AAUTH_RESTRICTION_FUNCTION (built-in vari-

able), 51AUTH_RESTRICTION_MODULE (built-in variable),

51

BBASE_URL (built-in variable), 48

CCONFIG_FILE (built-in variable), 48

DDATABASE (built-in variable), 49DB_HOST (built-in variable), 48DB_MAX_CONNECTIONS (built-in variable), 49DB_MIN_CONNECTIONS (built-in variable), 49DB_PASSWORD (built-in variable), 49DB_PORT (built-in variable), 48DB_STRUCTURE (built-in variable), 49DB_USER (built-in variable), 48DO_ENABLE_RESTRICTIONS (built-in variable), 51

FFILE_UPLOAD_FOLDER (built-in variable), 49

LLOG_AMQP_SERVER (built-in variable), 49

MMOX_LOG_EXCHANGE (built-in variable), 49MOX_LOG_QUEUE (built-in variable), 49

PPERMANENT_SESSION_LIFETIME (built-in vari-

able), 51

RRFC

RFC 3986#section-2, 22, 52, 53RFC 7230#section-3.1.1, 18

SSAML_AUTH_ENABLE (built-in variable), 50SAML_CERT_FILE (built-in variable), 51SAML_IDP_CERTIFICATE (built-in variable), 50SAML_IDP_ENTITY_ID (built-in variable), 50SAML_IDP_INSECURE (built-in variable), 51SAML_IDP_METADATA_URL (built-in variable), 50SAML_IDP_TYPE (built-in variable), 50SAML_IDP_URL (built-in variable), 50SAML_KEY_FILE (built-in variable), 51SAML_MOX_ENTITY_ID (built-in variable), 50SAML_REQUESTS_SIGNED (built-in variable), 51SAML_USER_ID_ATTIBUTE (built-in variable), 50SESSION_PERMANENT (built-in variable), 51SQLALCHEMY_DATABASE_URI (built-in variable),

51

UUSE_SAML_AUTHENTICATION (built-in variable),

50

73

lora mox documentation

Documents