Pipeline
Pipeline exports do not act as an intermediary for a datastore with a known protocol. This export type is designed to act as one component in a data pipeline. It takes a collection of records, evaluates each one, and then can return or forward some/all/none of the input content along with policy-driven knowledge about the records.
Export Configuration#
Exporting a Pipeline service via a Tranquil Data peer extends the configuration object with the following fields:
| Field Name | Type | Required | Default |
|---|---|---|---|
| includeValues | IncludeValues | ||
| destination | string |
The value of type must be pipeline.
If destination is absent or empty then the results are returned to the caller. Otherwise, the value of destination is interpreted as an HTTP endpoint and interpreted as detailed below.
IncludeValues#
Include values represent the set of data that is included in each result.
| Field Name | Type | Required | Default |
|---|---|---|---|
| advice | array of QualifierMatch | ||
| decision | boolean | false |
|
| input | boolean | false |
|
| obligations | array of QualifierMatch | ||
| properties | boolean | false |
|
| retention | boolean | false |
|
| score | boolean | false |
|
| subject | boolean | false |
|
| tags | boolean | false |
|
| tracker | boolean | false |
For advice and obligations the parameters control whether or not any (respectively) Advice or Obligation values resulting from policy evaluation are inluded. If either field is absent, then the associated type is not included. Otherwise, any qualifier that matches against any QualifierMatch is included using the Attribute Identifier as the field name an array of string for all resolved values.
If the value of decision is true then a field will be included specifying one of Permit, Deny, Indeterminate, or NotApplicale based on policy evaluation.
if the value of input is true then the initial input to this service will be treated as a record, and the resulting fields will be included. For instance, if the service is running in validate mode then then the entire input will be included, but if the service is running in redact mode then some fields may be removed from the result.
If the value of properties is true then each context property will be included as a separate field.
If the value of retention is true then a field named "retention" will be added that specifies the retention date for the input in ISO-8601 format.
If the value of score is true then a field named "score" will be added with the numeric score for the input as computed during evaluation.
If the value of subject is true then a field named "subject" will be added that names the Resource Subject associated with the input.
If the value of tags is true then a field named "tags" will be added that has, in an array, any tags that were associated with the input.
If the value of tracker is true then a field named "tracker" will be added that has a string of the form "PEER_ID:TRACE_ID:EVAL_ID" that uniquely identifies the decision associated with evaluation and the resulting change in the context graph.
QualifierMatch#
Qualifier matches are used to
| Field Name | Type | Required | Default |
|---|---|---|---|
| qualifierId | string | ||
| attributeId | string |
If qualifierId and/or attributeId contains a non-empty string value then any qualifier with the same identifier(s) are considered a match. An empty identifier matches any value. An empty QualifierMatch structure matches all qualifiers.
Export Interaction#
The endpoint for a Pipeline export accepts an HTTP PUT, POST, or GET.
For PUT or POST a JSON-Lines body is expected. Each line is evaluated, and the resulting output (based on the value of includeValues) is either forwarded to a remote service or returned to the caller, depending on the value of destination.
For GET the value of destination must be a remote HTTP service that will be invoked with the full URI provided. The result of the remote GET must be a JSON-Lines body. Each line is evaluated, and the resulting output (based on the value of includeValues) is returned to the caller
If expected content is not valid JSON-Lines then a 400 is returned with an error message in the body. If there is any error computing the result then a 500 is returned with with an error message in the body. If the service fails in a call to a remote service then a 502 is returned with no body.
Context Interaction#
All queries are treated as READ operations in-context so that the value of anonymous determines whether any resulting context forms. Each query, regardless of the number of records, is evaluated as a single session.