Copyright (C) 2019-2022 The Open Library Foundation

This software is distributed under the terms of the Apache License, Version 2.0. See the file "LICENSE" for more information.

The purpose of this edge API is to bridge the gap between self service circulation and patron services stations and FOLIO by allowing these systems to issue requests and receive responses in Standard Interchange Protocol v2 (SIP2). These requests will be serviced by FOLIO APIs and the responses will be based on FOLIO results for all supported (TBD) SIP2 commands.

• edge-sip2
  • Overview
  • Table of Contents
  • Configuration
    • Tenant configuration located in AWS S3
  • FOLIO Configuration
    • Tenant Properties
      • supportedMessages object properties
      • Example configuration object
    • Kiosk (service point) Properties
      • Example configuration object
    • FOLIO Provided Properties
      • Example configuration object
  • Implemented Messages
  • Security
  • Permissions
    • Security concerns for developers
  • Health check
  • Metrics
    • Enabling Vert.x metrics
    • Available metrics
    • Building with metrics
    • Launching with the community Docker image
  • Setting Up SIP2 for Multi-tenant usage
    • Summary
    • Multi-tenant configuration resolvers
      • Ip Subnet
      • Port
      • Location Code
      • Username Prefix
  • Common Problems
    • "Unable to find all necessary configuration(s). Found <N> of <M>"
    • "Configuration error: please add a value to Location Code."
  • Additional information
    • Issue tracker
    • Other documentation

The edge-sip2 module can be launched via edge-sip2-fat.jar as follows:

$ java -jar edge-sip2-fat.jar -conf sip2.conf

The -conf option can either specify the filename of the configuration or inline JSON. Here is a sample sip2.conf file:

{ 
  "port": 6443,
  "okapiUrl": "https://folio-testing-okapi.dev.folio.org",
  "haProxy": false,
  "tenantConfigRetrieverOptions": {
    "scanPeriod": 300000,
    "stores": [{
      "type": "file",
      "format": "json",
      "config": {
        "path": "sip2-tenants.conf"
      },
      "optional": false
    }]
  }
}

For inline JSON, the format is:

-conf '{"port":1234,"okapiUrl":"https://folio-snapshot-okapi.dev.folio.org".....}'

On Windows, inline JSON configuration is in double quotes and the inner double quotes should be escaped, for example:

-conf "{\"port\":1234,\"okapiUrl\":\"https://folio-snapshot-okapi.dev.folio.org\"....}"

One option is to mount the configuration files to the Docker container and provide command line arguments to point it to the right path (e.g. -conf /path/to/config).

Here is a sample sip2-tenants.conf file:

{
  "tenant": "default_single_tenant",
  "errorDetectionEnabled": true,
  "messageDelimiter": "\r",
  "fieldDelimiter": "|",
  "charset": "ISO-8859-1",
  "scTenants": [
    {
      "scSubnet": "11.11.00.00/16",
      "tenant": "multi-tenant1",
      "errorDetectionEnabled": true,
      "messageDelimiter": "\r",
      "fieldDelimiter": "|",
      "charset": "ISO-8859-1"
    },
    {
      "scSubnet": "22.22.00.00/16",
      "tenant": "multi_tenant2",
      "errorDetectionEnabled": true,
      "messageDelimiter": "\r",
      "fieldDelimiter": "|",
      "charset": "ISO-8859-1"
    }
  ]
}

Edge-sip2 supports various locations for sip2-tenants.conf tenant configuration. Additionally, it supports S3 config. To include vertx-config-s3 libraries when building edge-sip2, include the maven profile command:

mvn -P vertx-config-s3

Here is a sample sip2.conf for storing tenant config in S3:

    {
    "port": 6443,
    "okapiUrl": "https://folio-testing-okapi.dev.folio.org",
    "tenantConfigRetrieverOptions": {
      "scanPeriod": 300000,
      "stores": [{
        "type": "s3",
        "format": "json",
        "config": {
          "region": "my-region",
          "bucket": "my-bucket",
          "key": "sip2/sip2-tenants.conf"
        },
        "optional": true
      }]
    }
  }

Certain properties are retrieved from FOLIO configuration once a user has logged in via SIP2. There are properties at the tenant level and properties per kiosk, which is defined as a FOLIO service point. Properties are stored as JSON via the configuration module. Missing configuration properties will lead to edge-sip2 runtime failures. The edge-sip2 properties listed below must be manually created via the POST /configurations/entries API using a tool such as curl or postman.

{
  "module": "edge-sip2",
  "configName": "acsTenantConfig",
  "enabled": true,
  "value": "{\"supportedMessages\": [{\"messageName\": \"PATRON_STATUS_REQUEST\",\"isSupported\": \"N\"},{\"messageName\": \"CHECKOUT\",\"isSupported\": \"Y\"},{\"messageName\": \"CHECKIN\",\"isSupported\": \"Y\"},{\"messageName\": \"BLOCK_PATRON\",\"isSupported\": \"N\"},{\"messageName\": \"SC_ACS_STATUS\",\"isSupported\": \"Y\"},{\"messageName\": \"LOGIN\",\"isSupported\": \"Y\"},{\"messageName\": \"PATRON_INFORMATION\",\"isSupported\": \"Y\"},{\"messageName\": \"END_PATRON_SESSION\",\"isSupported\": \"Y\"},{\"messageName\": \"FEE_PAID\",\"isSupported\": \"N\"},{\"messageName\": \"ITEM_INFORMATION\",\"isSupported\": \"N\"},{\"messageName\": \"ITEM_STATUS_UPDATE\",\"isSupported\": \"N\"},{\"messageName\": \"PATRON_ENABLE\",\"isSupported\": \"N\"},{\"messageName\": \"HOLD\",\"isSupported\": \"N\"},{\"messageName\": \"RENEW\",\"isSupported\": \"N\"},{\"messageName\": \"RENEW_ALL\",\"isSupported\": \"N\"}, {\"messageName\": \"REQUEST_SC_ACS_RESEND\",\"isSupported\": \"Y\"}],\"statusUpdateOk\": false,\"offlineOk\": false,\"patronPasswordVerificationRequired\": true}"
}

{
  "module": "edge-sip2",
  "configName": "selfCheckoutConfig.e0ab8c91-2a4a-433d-a3cf-1837053c89a8",
  "enabled": true,
  "value": "{\"timeoutPeriod\": 5,\"retriesAllowed\": 3,\"checkinOk\": true,\"checkoutOk\": true,\"acsRenewalPolicy\": false,\"libraryName\": \"Datalogisk Institut\",\"terminalLocation\": \"e0ab8c91-2a4a-433d-a3cf-1837053c89a8\"}"
}

{
  "module": "ORG",
  "configName": "localeSettings",
  "enabled": true,
  "value": "{\"timezone\":\"America/New_York\",\"currency\":\"USD\"}"
}

Currently, edge-sip2 implements select SIP messages. Below, is the list of all implemented messages.

The SIP protocol does not consider security apart from providing the possibility of SC/ACS negotiated encryption algorithm for the password and user ID, which seems unlikely to be used. All communication over the wire, via TCP, is plain text. This may be fine in an environment that is locked down in some way. However, FOLIO, along with this module, can be hosted in the cloud and plain text communication is not acceptable.

We strongly recommend securing communication from the SC to edge-sip2. To this end, edge-sip2 can be configured to use TLS to encrypt communication. This requires the SC to communicate with TLS as well. It is our understanding that most self service kiosks do not have this ability natively and third party solutions must be employed. One such solution is stunnel.

A typical stunnel deployment will involve installing the stunnel service either on the SC or a machine that is locked down with the SC and provides a port that the SC will be configured to connect to for SIP communication. The SIP commands are then sent to and received from the stunnel port unencrypted. Communication from the stunnel to the TLS termination end point will be encrypted. There are several ways to terminate TLS and the advantages/disadvantages of these is out of scope for this document. Here, we will focus on enabling TLS termination via the edge-sip2 module. This is done via simple launch configuration options.

stunnel can be downloaded here: https://www.stunnel.org/downloads.html

Example FOLIO configuration section for stunnel.conf:

[FOLIO]
key = stunnel.pem
cert = stunnel.pem
client = yes
accept = 127.0.0.1:5555
connect = sip2.example.com:6443

Example edge-sip2 configuration:

$ java -jar edge-sip2-fat.jar -conf '{"port":1234,"okapiUrl":"https://folio-snapshot-okapi.dev.folio.org","tenant":"diku","netServerOptions":{"ssl":true,"pemKeyCertOptions":{"certPaths":["cert.crt"],"keyPaths":["cert.key"]}}}'

All permission associated with edge-sip2

    circulation.check-in-by-barcode.post
    circulation.check-out-by-barcode.post
    circulation.requests.collection.get
    search.instances.collection.get
    circulation.loans.collection.get
    configuration.entries.collection.get
    configuration.entries.item.get
    manualblocks.collection.get
    manualblocks.item.get
    accounts.collection.get
    accounts.item.get
    users.collection.get
    users.item.get
    patron-blocks.automated-patron-blocks.collection.get
    inventory.items.collection.get
    circulation.renew-by-barcode.post
    usergroups.collection.get
    users-bl.item.get
    usergroups.item.get
    usergroups.collection.get
    inventory-storage.holdings.item.get
    inventory.instances.item.get
    feefines.collection.get
    patron-pin.validate

For local development, there is no requirement to encrypt communications from a SIP2 client to edge-sip2. Unencrypted TCP sockets are the default when launching edge-sip2 as described in the Configuration section. Encrypted communication from a SIP2 client is only required when explicitly configured via the above options and is up to the developer to provide that secure connection for edge-sip2.

A GET /admin/health request sent to health check port gets a response with 200 HTTP status code. The value of the health check port is defined by the environment variable HEALTH_CHECK_PORT or defaults to 8081 if not set.

This module makes use of Micrometer to collect SIP2, Vert.x and JVM metrics. The metrics need to be collected by a monitoring system backed. This is where Micrometer provides flexibility, by allowing the module to code to the Micrometer interface, which is vendor neutral. Once determined, the vendor specific backend binding is provided runtime and can be easily replaced.

By default, metrics are disabled. To enable Vert.x metrics pass the following Java argument:

-Dvertx.metrics.options.enabled=true

With metrics enabled, configuration must be supplied to the verticle. This can be done as follows:

 -options '{"metricsOptions":{"labels":["LOCAL","REMOTE","HTTP_PATH","HTTP_METHOD","HTTP_CODE","CLASS_NAME"],"enabled":true,"prometheusOptions":{"enabled":true,"startEmbeddedServer":true,"embeddedServerOptions":{"port":8081}}}}'

The metricsOptions here indicate that the verticle should collect metrics with the supplied list of labels. Some of these labels, like REMOTE, may lead to high cardinality metrics. The default labels list is "HTTP_METHOD", "HTTP_CODE", "POOL_TYPE" and "EB_SIDE".

Also specified here are prometheusOptions. In this case, the backend will be Prometheus. Prometheus "scrapes" metrics via HTTP at a specified interval. The options specified here allow Vert.x to create an HTTP server to handle metrics scraping. Prometheus setup is outside the scope of this document. Other bindings could be used, like InfluxDB.

For a list of Vert.x metrics (HTTP Client and Net Server are the primary sources for metrics in this module) see: Vert.x core tools metrics

The following metrics are supplied by this module:

JVM metrics (memory, GC, threads, etc.) are supplied as well.

The Maven pom.xml contains 2 profiles, metrics-prometheus and metrics-influxdb. Building with either or both of these profiles active will include the appropriate dependencies required to use metrics with that registry.

$ mvn install -P metrics-prometheus

If metrics need to be enabled, it is probably best to add any required runtime binding jars to the fat jar as part of a build. If this is not possible, the module can still be launched via the community Docker image. N.B., we may find that this approach cumbersome and may need to come up with an alternative approach.

Copy the appropriate runtime binding jars to a directory:

$ cp micrometer-registry-prometheus-1.1.5.jar simpleclient_common-0.5.0.jar simpleclient-0.5.0.jar /my/metrics/libs

Then run a container from the FOLIO docker hub image (either snapshot folioci/edge-sip2 or released folioorg/edge-sip2):

$ docker run -v /my/metrics/libs:/metrics -p 6443:6443 --expose 8081 -p 8081:8081  -e JAVA_OPTIONS="-Dvertx.metrics.options.enabled=true " -e JAVA_CLASSPATH=/metrics/*:/usr/verticles/edge-sip2-fat.jar -e JAVA_MAIN_CLASS=io.vertx.core.Launcher folioci/edge-sip2 run org.folio.edge.sip2.MainVerticle -conf '{"port":6443,"okapiUrl":"https://folio-okapi.example.com","tenant":"diku","messageDelimiter":"\r","errorDetectionEnabled":true,"charset":"ISO-8859-1"}' -options '{"metricsOptions":{"labels":["LOCAL","REMOTE","HTTP_PATH","HTTP_METHOD","HTTP_CODE","CLASS_NAME"],"enabled":true,"prometheusOptions":{"enabled":true,"startEmbeddedServer":true,"embeddedServerOptions":{"port":8081}}}}'

This example shows how to launch with the Prometheus binding. Since Prometheus needs to scrape the metrics, we need to expose port for the HTTP server.

Edge-sip2 supports multi-tenant environments by dynamically resolving the correct tenant configuration for each SIP2 client connection. This is achieved through a set of pluggable Tenant Resolver classes, each designed to identify the tenant using different connection or protocol attributes. The supported resolvers and their resolution strategies are described below.

Tenant resolvers are configured with system property (sip2TenantResolvers) or environment variable (SIP2_TENANT_RESOLVERS), default value is PORT, IP_SUBNET. It is a comma-separated values with tenant resolver names.

There are 2 tenant detection phases:

1. Connection Phase (CONNECT): This phase occurs when a SIP2 client establishes a TCP connection with the edge-sip2 server. During this phase, the Tenant Resolver inspects connection-level attributes, such as the client's IP address or the port number used for the connection, to determine the appropriate tenant. This phase is crucial for initial tenant identification, especially in environments where multiple tenants share the same SIP2 server instance.

   Note: This phase is essential and required to set the following parameters:

   • message delimiter (messageDelimiter) (default is \r)
   • encoding charset (charset) (default is IBM850)
   • error detection (errorDetectionEnabled) (default is false)
   • field delimiter (fieldDelimiter) (default is |)

   If these values are not specified in the multi-tenant configuration, the values from the single tenant configuration will be used, if they are not found either, then the default values will be applied.

2. Protocol Phase (LOGIN): This phase takes place when the SIP2 client sends a LOGIN message after establishing the connection. The Tenant Resolver analyzes protocol-level attributes, such as the "Location Code" or Username field in the LOGIN message, to further refine or confirm the tenant identification. This phase is essential for scenarios where the initial connection attributes may not be sufficient for accurate tenant resolution.

   Note: This phase provides only a capability to change tenant. Connection details, such as message delimiter, field delimiter, error detection, and encoding charset, cannot be changed.

Configuration example:

{
  "tenant": "default_single_tenant",
  "errorDetectionEnabled": true,
  "messageDelimiter": "\r",
  "fieldDelimiter": "|",
  "charset": "ISO-8859-1",
  "scTenants": [
    {
      "scSubnet": "11.11.00.00/16",
      "tenant": "multi-tenant1",
      "errorDetectionEnabled": true,
      "messageDelimiter": "\r",
      "fieldDelimiter": "|",
      "charset": "ISO-8859-1"
    },
    {
      "scSubnet": "22.22.00.00/16",
      "tenant": "multi_tenant2",
      "errorDetectionEnabled": true,
      "messageDelimiter": "\r",
      "fieldDelimiter": "|",
      "charset": "ISO-8859-1"
    },
    {
      "scSubnet": "1001:db8:1::/48",
      "tenant": "multi_tenant3",
      "errorDetectionEnabled": false,
      "messageDelimiter": "\r\n",
      "fieldDelimiter": "~",
      "charset": "UTF-8"
    }
  ]
}

To configure sip2 for a port dedicated to a specific tenant, two modifications are necessary:

1. Modify the existing port config property in the sip2.conf from an integer value to an array of integer values, as demonstrated below:

   {
     "port": [ 6443, 6444, 6445 ],
     "okapiUrl": "https://folio-testing-okapi.dev.folio.org",
     "tenantConfigRetrieverOptions": {
       "scanPeriod": 300000,
       "stores": [
         {
           "type": "file",
           "format": "json",
           "config": {
             "path": "sip2-tenants.conf"
           },
           "optional": false
         }
       ]
     }
   }

2. In the sip2-tenants.conf, include a new property named 'port' and assign it the dedicated port value as indexed in the array from the previous conf file, as shown below:

   {
     "scTenants": [
       {
         "scSubnet": "11.11.00.00/16",
         "port": "6443",
         "tenant": "test_tenant1",
         "errorDetectionEnabled": true,
         "messageDelimiter": "\r",
         "fieldDelimiter": "|",
         "charset": "ISO-8859-1"
       },
       {
         "scSubnet": "22.22.00.00/16",
         "port": "6444",
         "tenant": "test_tenant2",
         "errorDetectionEnabled": true,
         "messageDelimiter": "\r",
         "fieldDelimiter": "|",
         "charset": "ISO-8859-1"
       },
       {
         "scSubnet": "33.33.00.00/16",
         "port": "6445",
         "tenant": "test_tenant3",
         "errorDetectionEnabled": true,
         "messageDelimiter": "\r",
         "fieldDelimiter": "|",
         "charset": "ISO-8859-1"
       }
     ]
   }

Note: Only 1 tenant can be assigned to a specific port. If multiple tenants are configured with the same port, only the first tenant found will be used.

sip2-tenants.json example (if the same connection parameters can be applied to multiple tenants):

{
  "errorDetectionEnabled": true,
  "messageDelimiter": "\r",
  "fieldDelimiter": "|",
  "charset": "ISO-8859-1",
  "scTenants": [
    {
      "tenant": "test_tenant1",
      "locationCodes": [
        "f79ed33a-d39e-4f39-a6b2-8d43504cfb1c",
        "961b41ba-9bbd-4781-a735-2e1cf9a9c899"
      ]
    },
    {
      "scSubnet": "22.22.00.00/16",
      "tenant": "test_tenant2",
      "locationCodes": [
        "c3da3e63-edb0-4704-b5f9-f675f8756077",
        "6326e2ed-12ba-40b0-ae65-ac7414ef7157"
      ]
    }
  ]
}

sip2-tenants.json example (if any of the tenants require special charset, message delimiter, etc.):

Note: Note that IP subnet being used to identify connection parameters in this example, and tenant will be defined during the login operation.

{
  "scTenants": [
    {
      "scSubnet": "11.11.00.00/16",
      "errorDetectionEnabled": true,
      "messageDelimiter": "\r",
      "fieldDelimiter": "|",
      "charset": "ISO-8859-1"
    },
    {
      "scSubnet": "22.22.00.00/16",
      "errorDetectionEnabled": true,
      "messageDelimiter": "\r",
      "fieldDelimiter": "|",
      "charset": "UTF-8"
    },
    {
      "tenant": "test_tenant1",
      "locationCodes": [
        "f79ed33a-d39e-4f39-a6b2-8d43504cfb1c",
        "961b41ba-9bbd-4781-a735-2e1cf9a9c899"
      ]
    },
    {
      "scSubnet": "22.22.00.00/16",
      "tenant": "test_tenant2",
      "locationCodes": [
        "c3da3e63-edb0-4704-b5f9-f675f8756077",
        "6326e2ed-12ba-40b0-ae65-ac7414ef7157"
      ]
    }
  ]
}

This log message happens when one or more of the FOLIO configuration key/value maps are missing when retrieved. Ensure that each set of properties is stored in FOLIO configuration and that the service point UUID for the kiosk configuration matches the "location code" in the SIP "Login" message. Another problem could be that the tenant locale settings may not be saved in the database. On the initial deployment of FOLIO, as of Edelweiss, the locale settings are defaulted by the UI and not stored in the database until the "save" button is pressed. Since the UI defaults to usable settings for many, it may be misleading that these settings are present for backend modules, like edge-sip2, to consume.

This log message happens when the location code (CP) is missing when SC status command is attempted then the error message is printed in the logs.

SIP2 Specification

See project SIP2 at the FOLIO issue tracker.

Other modules are described, with further FOLIO Developer documentation at dev.folio.org