Niord System User Manual for Sysadmins

The sysadmin can add a new message series by clicking the "New Message Series" button, or edit an existing message series by clicking the pencil symbol next to the series.

This will open the Message Series Edit Page:

The message series attributes:

[ cols="30,70", options="header" ]

A message series can be deleted by clicking the trash icon next to them. However, this only works if the message series being deleted has never been assigned to a message.

If a message series has indeed been assigned to a message, the sysadmin should instead remove the message series from the relevant domain.

The sysadmin can add a new domain by clicking the "New domain" button, or edit an existing domain by clicking the pencil symbol next to the domain.

This will open the Domain Edit Page:

The domains attributes:

[ cols="30,70", options="header" ]

A domain can be deleted by clicking the trash icon next to them. However, this only works if there is not related data associated with the domain.

Instead the system administrator can choose to in-activate the domain.

The system administrator can export and import domains from the action menu.

The export/import file format is based on a JSON representation of the DomainVo class.

Example:

Importing a domain JSON file will trigger the domain-import batch job. Batch jobs can be monitored and managed by system administrators.

As an alternative to manually uploading a domain import JSON file on the Domains sysadmin page, the file can be copied to the $NIORD_HOME/batch-jobs/domain-import/in folder.

The sysadmin can add a new instance of a promulgation type by clicking the "Add Promulgation Type" button, which will allow the sysadmin to select which promulgation type to add:

Similarly, an existing promulgation type can be edited by clicking the pencil symbol next to the promulgation type.

Adding or editing a promulgation type will open the Promulgation Type Edit Page:

All editable attributes, except the bottom-most one, are common to all promulgation types:

[ cols="30,70", options="header" ]

A promulgation type can be deleted by clicking the trash icon next to them. However, this only works if there is not related data associated with the promulgation type.

Instead the system administrator can choose to in-activate the promulgation type.

The system administrator can export and import promulgation types from the action menu.

The export/import file format is based on a JSON representation of the PromulgationTypeVo class.

Example:

Please note, this data format does not include support for the promulgation type-specific attributes such as the NAVTEX transmitters. These attributes must be handled manually.

Importing a promulgation type JSON file will trigger the promulgation-type-import batch job. Batch jobs can be monitored and managed by system administrators.

As an alternative to manually uploading a promulgation type import JSON file on the Promulgation Types sysadmin page, the file can be copied to the $NIORD_HOME/batch-jobs/promulgation-type-import/in folder.

Before configuring a schedule on the Firing Schedules system admin page, the Niord system must be configured with the proper firing areas base data and the domains for firing areas and exercises.

The overarching idea is as follows:

The schedules configured on the Firing Schedules system admin page provide the underlying engine that generates firing exercise message based on firing areas and firing area schedules.

The steps outlined above are described in more details below.

A prerequisite for the firing exercise-generating mechanism to work, and for the schedule page to become available in the "Firing Areas" domain, is that a firing schedule has been defined on the Firing Schedules system admin page.

The sysadmin can add a new schedule by clicking the "New Firing Schedule" button, or edit an existing schedule by clicking the pencil symbol next to the schedule.

This will open the Firing Schedule Edit Page:

The schedule attributes:

[ cols="30,70", options="header" ]

A firing schedule can be deleted by clicking the trash icon next to them.

Niord defines a set of standard dictionaries, that are initially loaded from resource bundles in the Niord distribution. The standard dictionaries are:

[ cols="30,70", options="header" ]

Niord also supports importing extended dictionaries, as detailed in the [Importing and Exporting Dictionaries] section.

The entries of these dictionaries have an extra translatable long value field, which can be viewed and edited by checking the "Show Extended" option, next to the dictionary menu:

One such dictionary is the "aton" dictionary, which contain parameter list entries used in the message template function - please refer to the Message Template Execution chapter.

The sysadmin can add a new dictionary entry to the currently selected dictionary by clicking the "New Entry" button, or edit an existing dictionary entry by clicking the pencil symbol next to the entry.

This will open the Dictionary Entry Edit Page:

The standard dictionary entry attributes:

[ cols="30,70", options="header" ]

For extended dictionaries - i.e. when the "Show Extended" option is checked - there are a couple of additional attributes. These entries are typically used in parameter lists when executing message templates - please refer to the Message Template Execution chapter.

[ cols="30,70", options="header" ]

A dictionary entry can be deleted by clicking the trash icon next to them.

Be careful that the dictionary entry is not used in on of the [Parameter Types] definitions.

The system administrator can export and import dictionaries from the action menu.

The export/import file format is based on a JSON representation of the ExportedDictionaryVo class.

Example:

Importing a dictionary JSON file will trigger the dictionary-import batch job. Batch jobs can be monitored and managed by system administrators.

As an alternative to manually uploading a dictionary import JSON file on the _Dictionaries sysadmin page, the file can be copied to the $NIORD_HOME/batch-jobs/dictionary-import/in folder.

As mentioned in the [Standard Dictionaries] section, the standard dictionaries are initially loaded from resource bundles in the Niord distribution.

These dictionaries can be re-loaded from the resource bundles by enacting the "Reload from system" function in the action menu.

The sysadmin can add a new parameter type by clicking the "New Parameter Type" button, and then select either "List Parameter Type" or "Composite Parameter Type":

Similarly, the sysadmin can edit an existing list or composite (but not standard) parameter type by clicking the pencil symbol next to the entry.

This will open either the List Parameter Type Edit Page or the Composite Parameter Type Edit Page:

A parameter type can be deleted by clicking the trash icon next to them. However, the sysadmin should first check that the parameter is not used in either message template definitions (see the [Categories] chapter) or in composite parameter types.

A parameter type can be duplicated by clicking the copy icon next to them.

The system administrator can export and import parameter types from the action menu.

The export/import file format is based on a JSON representation of the ParamTypeVo class and its sub-classes: StandardParamTypeVo, ListParamTypeVo and CompositeParamTypeVo.

Example:

Importing a parameter type JSON file will trigger the param-type-import batch job. Batch jobs can be monitored and managed by system administrators.

As an alternative to manually uploading a parameter type import JSON file on the Parameter Types sysadmin page, the file can be copied to the $NIORD_HOME/batch-jobs/param-type-import/in folder.

The script resources are used throughout Niord for operations such as generating PDF print reports, generating mails, etc. An overview of functions using script resources is found in the table below:

[ cols="30,70", options="header" ]

Niord has added some extensions to both types of the script resources.

Firstly, the script resources are always loaded from the database, not the file system. However, if a script resource, as given by its path, does not exist in the database, then Niord will check to see if the resource can be loaded from the class-path and persisted to the database, thence commencing operations.

This means that Niord developers can develop and maintain the initial batch of script resources as project files and use a professional code editor for editing the scripts.

Another Niord extension has to do with script resource inclusion. File system-based Freemarker Templates can include other Freemarker templates, and similarly for JavaScript files; a trait that facilitates code reuse and libraries. Niord has extended the script resource inclusion concept, so that it works with the database-backed scripts as well. Subsequent sections will give examples of script resource inclusion.

An introduction to Nashorn JavaScript can be found at http://winterbe.com/posts/2014/04/05/java8-nashorn-tutorial/.

The scripts are executed server-side, which means that you can e.g. call Java Enterprise Beans and load and manipulate data.

The following example illustrates how to perform a message search for published firing exercises:

The documentation for Freemarker templates are found at http://freemarker.org.

The Freemarker Templates are typically used to convert a data model, such as a message list, into HTML, and sometimes into plain text or XML.

The following simple example, "templates/geometry/text.ftl", is instantiated with a "geometry" (position list) and a "format", and formats the positions as a comma-separated list of lat-lon positions.

Freemarker templates are used for generating an intermediate XML-based Message Template Format. Message templates are used so extensively, that the format merits its own description below. Please also refer to the [Categories] and Message Template Execution chapters.

The data passed along to the Freemarker template being executed is

[ cols="30,70", options="header" ]

The output of the Freemarker templates should be an intermediate XML format, which contain a list of <field-template> elements for each field that should be updated of the message. The snippet below illustrates what the resulting XML may look like:

The "field" attribute (which is actually a JavaScript expression), is used to point out which message attribute to update with the contents of the <field-template>.

The "update" attribute indicates whether to replace or append the content to the attribute.

The "format" attribute indicates the format of the <field-template> contents. If the format is e.g. "text", then the contents is cleaned up as a single line of text.

Unlike most other types of script resources, which gets instantiated with a specific language and locale, the ones used for message templates should handle multiple languages themselves. The currently used language can be changed using the following snippet:

The sysadmin can add a new Freemarker Template or JavaScript file by clicking the "Add Script Resource" button, and select the type to add:

Similarly, the sysadmin may edit an existing script resource by clicking the pencil icon next to the script resource.

The editor for the script being added or edited, contains two fields; the script resource path and the actual script:

The script editor has syntax coloring to aid and facilitate the editing process.

Script resources are versioned, in the sense that every time the sysadmin edits and saves a script resource, a new revision is stored. The revisions of a script resource can be inspected by clicking the "History" button, which opens the Script Resource History dialog:

An earlier revision of the script can be inspected by clicking the checkbox next to the revision. If two revisions are selected (as above), the dialog will display the differences using color coding.

A script resource can be deleted by clicking the trash icon next to them. However, the sysadmin should first carefully check if the script resource is referenced in mailing lists, print reports, message templates, or indeed included in other script resources.

As described in the [Script Resource Extensions] chapter, script resources may originate from files in the Niord class-path, before being loaded into the database. Any subsequent changes to a script resources will (naturally) only update the version in the database, which may thus come out of sync with the class-path file.

If the sysadmin enacts the "Reload files" function from the action menu, all script resources in the database will be overridden with any corresponding version in the class-path.

The system administrator can export and import script resources from the action menu.

The export/import file format is based on a JSON representation of the ScriptResourceVo class.

Example:

Importing a script resource JSON file will trigger the script-resource-import batch job. Batch jobs can be monitored and managed by system administrators.

As an alternative to manually uploading a script resource import JSON file on the Script Resource sysadmin page, the file can be copied to the $NIORD_HOME/batch-jobs/script-resource-import/in folder.

The user can add a new category by clicking the "Add" button, or edit a category by selecting it in the category tree.

When adding a new category, it will be a child-category of the currently selected category in the category tree. If no category is selected, the new category will become a new root category.

The details of the new or edited category will be present in the Category Editor Panel next to the category tree (see screenshot above).

The common editable attributes for both types of categories are listed below:

[ cols="30,70", options="header" ]

A category of type "Template" is executable, as described in the Message Template Execution chapter.

This section will detail the attributes that may be edited for templates in addition to the common attributes editable for all categories and detailed in the previous section.

The hierarchical order of the categories are updated in two ways:

A category, including all its sub-categories, can be deleted by selecting it and then click the "Delete" button above the category tree. However, this only works if neither the selected category, nor any of its sub-categories, have ever been assigned to a message.

If the category has indeed been assigned to a message, the user should instead deactivate the category.

The administrator can export and import categories from the category action menu.

The export/import file format is based on a JSON representation of the SystemCategoryVo class. The data file should define root categories at the top level, and the recursively define sub-categories in the children field of the parent category.

The imported data is merged into the existing category tree. If an MRN is defined for an imported category, this is used to search for existing categories in the category tree. Alternatively, the name of the category (in all supported languages) is used to search for existing matches.

Example:

Importing a category JSON file will trigger the category-import batch job. Batch jobs can be monitored and managed by system administrators.

As an alternative to manually uploading a category import JSON file on the Categories admin page, the file can be copied to the $NIORD_HOME/batch-jobs/category-import/in folder.

Niord will always ensure the availability of two standard reports available to all domains.

[ cols="30,70", options="header" ]

Generating PDFs using print reports is actually a two-phase process. First, the Freemarker template associated with the print report is used to convert a list of message into a HTML representation. Then, the HTML is converted into a PDF and returned to the user.

When creating and editing print reports, is important to understand how CSS Paged Media can be used to control such things as page size, pagination, headers and footers in the resulting PDF. Thus, the sysadmin is advised to acquaint herself with relevant documentation, such as:

The sysadmin can add a new print report by clicking the "New Report" button, or edit an existing print report by clicking the pencil symbol next to the domain.

This will open the Print Report Edit Page:

The print report attributes:

[ cols="30,70", options="header" ]

A print report can be deleted by clicking the trash icon next to them.

The system administrator can export and import print reports from the action menu.

The export/import file format is based on a JSON representation of the FmReportVo class.

Example:

Importing a print report JSON file will trigger the report-import batch job. Batch jobs can be monitored and managed by system administrators.

As an alternative to manually uploading a report import JSON file on the Print Reports sysadmin page, the file can be copied to the $NIORD_HOME/batch-jobs/report-import/in folder.

When a mail is submitted to the mail queue, it will have a pending status. Every minute, Niord will process pending mails and attempt to send them. If a mail is successfully sent to the SMTP-server, it will be assigned the sent status. If an error occurs, however, the error message will be stored in the mail, and it will be re-scheduled to be sent after a set number of minutes (which increases with each attempt). After 5 failed attempts to send a mail, it will be be assigned the error status, after which, no more attempts will be made to send the mail.

A system administrator should thus periodically check for error-status mails in the Mail Queue, and take appropriate actions (such as updating an invalid user mail address, etc).

The mail queue may contain thousands of mails. Hence the list of mails is paginated with 10 mails per page.

The fields above the mail list are used for filtering the mail list. The available filter fields are:

[ cols="30,70", options="header" ]

Clicking the letter-icon next to a mail, will open the Mail Details dialog:

This will display the mail in its entirety, including the date, address fields, subject and mail body.

The Legacy NW Import integration tab will only be present when the currently selected domain is "Legacy NW". It supports importing NWs from the current Danish NW production system (until Niord takes over this role completely, that is).

The panel allows the system administrator to specify the target message series to use for local and coastal legacy NW messages, and also to specify the date from which to import old legacy NW messages, and whether to auto-import NW messages from the legacy system or not.

The Legacy Firing Area Import integration tab will only be present when the currently selected domain is "Firing Areas". It supports importing firing areas from the current Danish NW production system (until Niord takes over this role completely, that is).

The tab also supports importing the firing schedules associated with the firing areas from the legacy NW system, and there is a function for generating firing area template messages from the firing areas.

Please refer to the [Firing Schedules] chapter for a discussion of firing areas and schedules in Niord.

The Legacy AtoN Import integration tab supports importing Danish Aids to Navigation seamarks. The files to upload must be Excel sheets generated from the Danish Afmærkningsregister system.

Please note, the AtoNs Niord module is currently not enabled in production, since the quality of the AtoN data is not good enough. However, in the future it is hoped that a more extensive AtoN integration will be implemented.

A setting in Niord is more that just a key-value pair. It is comprised of the following attributes:

[ cols="20,80", options="header" ]

Note that for settings of type "Password", the associated value never leaves the Niord backend. If the Editable flag is set, the sysadmin can set a new password on the Settings page, but she can never view the old password.

Whenever Niord starts up, it will look for a "niord.json" file in Niords home folder (itself controlled with a settings).

Example:

When the Niord app starts up, settings may be instantiated using environment variables.

Example: In order to specify the "niord.home" system setting, instantiate the app with the argument "-Dniord.home=/home/enav/niord".

If settings are defined from environment variables, they are essentially read-only. A sysadmin cannot override these settings on the Settings page.

Settings can also be defined and instantiated with default values directly in Niord java code.

Example:

The sysadmin can edit a system setting by clicking the pencil symbol next to the setting.

This will open the Setting Editor Page:

The only setting attribute that can be edited is the value of the setting. The editor widget used for editing the value depends on the type of the setting, with checkboxes used for "boolean" settings, a date picker used for settings of type "Date", etc.

The system administrator can export and import system settings from the action menu.

The export/import file format is based on a JSON representation of the SettingVo class.

Example:

Importing a settings JSON file will trigger the settings-import batch job. Batch jobs can be monitored and managed by system administrators.

As an alternative to manually uploading a settings import JSON file on the Settings sysadmin page, the file can be copied to the $NIORD_HOME/batch-jobs/settings-import/in folder.

Batch Jobs can either be executed from the UI (e.g. by uploading an AtoN Excel file from Danish [Integration] sysadmin page), or by copying the same file to a batch job folder on the Niord server.

The path to the batch job file system folder is: $NIORD_HOME/batch-jobs/batch-job-type/in. The screenshot below shows the batch job folder structure, with the folders of the "dk-aton-import" batch job fully expanded:

When the batch job is executed, the batch job file is copied to the "execution" sub-folder with the format $NIORD_HOME/batch-jobs/batch-job-type/execution/year/month/execution-no, along with the log files produced whilst executing the batch job.

The batch job architecture in Niord is implemented using the Java EE 7.0 Batch Processing Architecture - please refer to http://www.oracle.com/technetwork/articles/java/batch-1965499.html. This means that the batch jobs are typically divided into three sub-tasks for reading, processing and writing data, each producing their own log file in the execution folder.

Also, each batch job processes a certain batch job-specific number of records in each transaction, which means that the batch job can be stopped during execution, restarted or abandoned.

As an example, the "dk-aton-import", which is a fairly long-running batch job, will first have the "started" status, and appear with a progress indicator on the Batch Jobs page:

If the batch job fails during execution, if will stop executing, and enter the "failed" status. Similarly, if the user clicks the "Stop" button, it will enter the "stopped" status:

All database transactions will be rolled back to the last successful commit before the batch job failed or was stopped. The system administrator can now click the "Abandon" button to completely abandon the batch job, or she can click "Restart" to attempt to restart the batch job. If the batch job failed, the sysadmin may e.g. attempt to update the batch job file to fix the problem causing it to fail, and then restart.

The sysadmin can also inspect the log files generated by the sub-tasks of the batch job, by clicking the "logs…​" link, which opens the Batch Job Logs dialog:

All associated log files are listed in the top part of the dialog, and the contents of a log file can be inspected by clicking on the associated log file name.

The "Upload Batch Set" button in the top-right part of the Batch Jobs page, allows the system administrator to upload a zip-archive containing multiple batch job files.

The zip archive must also contain a special file, batch-set.json, which enlists and schedules all included batch jobs.

Example:

The scheduling mechanism (i.e. the "delay" attribute), can be used for handling inter-dependencies between imported data. If e.g. the areas.json file depends on the presence of the domains defined in the domains.json file, they can be scheduled as seen in the example above.

In order to avoid getting the Niord file system filled up with batch job execution files and folders, old files and folders will automatically be deleted after a certain amount of time, as dictated by the "batchFileExpiryDays" system setting.

As mentioned above, subscription of the mailing lists is detailed in the Mailing Lists chapter of the administrator manual.

The sysadmin can add a new mailing list by clicking the "New mailing list" button, or edit an existing mailing list by clicking the pencil symbol next to the mailing list.

This will open the Mailing List Edit Page:

The basic mailing list attributes:

[ cols="30,70", options="header" ]

Mailing list triggers are described separately below.

A mailing list can be deleted by clicking the trash icon next to them.

As an alternative, the system administrator can choose to in-activate the mailing list.

The system administrator can export and import mailing lists from the action menu.

The export/import file format is based on a JSON representation of the MailingListVo class.

Example:

Importing a mailing list JSON file will trigger the mailing-list-import batch job. Batch jobs can be monitored and managed by system administrators.

As an alternative to manually uploading a mailing list import JSON file on the Mailing Lists sysadmin page, the file can be copied to the $NIORD_HOME/batch-jobs/mailing-list-import/in folder.

Domains in Niord are detailed in the [Domains] chapter. For each domain in Niord, there will be bearer only application client in Keycloak.

As an example, for the "NM" Niord domain, there is a corresponding "niord-nm" application client in Keycloak.

Also, there is a single public application client for the actual Niord web application.

All users in Niord are created and maintained in Keycloak, although Niord also provides a UI for managing users on top of Keycloak, as detailed in the Users & Contacts chapter of the administrator guide.

In turn, Keycloak may be configured to use other identity providers. For instance, in the set-up used by the Danish Maritime Authority (DMA), Keycloak is integrated with the DMA Active Directory Federation Service, and hence, the first time a DMA employee clicks the "DMA" link on the log-in page and logs in using her usual work credentials, she will automatically be created as a user in Keycloak.

Keycloak can also be integrated with other Keycloak services, Active Directory or other LDAP provides, etc.

The list of editor fields being displayed when a user edits a message in Niord is highly configurable. There are settings to define a base set of fields, a message main-type (NW,NM) specific set, plus the list can be customized depending on the message series, areas and categories assigned to the message.

It is also possible to customize the list of editor fields per user in Keycloak, or indeed, per user group.

Step 1 is for the system administrator to define two User Attribute Mappers for the "niord-web" application client; one for the "editorFieldsNw" token claim and one for the "editorFieldsNm" token claim. Example:

Step 2 is for the system administrator to assign the relevant "editorFieldsNw" and "editorFieldsNm" user attributes either directly to the user, or - more likely - to the relevant user groups. The format of the attribute values should be a JSON structure that enables and disables specific editor fields. Example:

All access to the Niord backend happens via calls to REST endpoints. The obvious example is the actual Niord web application which makes Ajax calls to the Niord backend.

The Niord backend has been integrated with Keycloak, so that all calls to Niord resolves the roles of the callee in the context of the current domain (the current domain is specified via a request header).

Hence, access to Niord can be checked using standard Java EE security checks. See the (simplified) example below, where annotating a REST endpoint with "@RolesAllowed(Roles.ADMIN)" will validate that the user has the Admin role in the current domain, or throw an exception.