Mapping APIs

Mapping User Interface

There is a Web UI (by default at http://localhost:8890/mapping/) for creating and managing mapping groups that may be helpful as a visualization and debugging too for API users, or as a production interface for managing mapping in a deployed product.

The mapping APIs are used to map from the human-readable categories and fields defined in configuration to the specific schema of a datastore, file format, or API structure. The details of how mapping works are described elsewhere. This section shows a few simple examples of how to automate mapping definition with the APIs.

Creating a Mapping

As Tranquil Data is used to interact with new databases, transform flows, CSV formats, etc. you will define new mappings for each of these structures. You can do this through the UI or the API, with the latter providing a simple way to automate this task when mappings are created or updated. In this example, mappings are defined for both a Postgres export and an API-based transform:

Mapping for a Postgres ExportMapping for a Transform

curl -X PUT -H "Content-Type: application/json" \
    "http://localhost:8890/mapping/groups/" \
-d '{
        "name" : "postgres_prod",
        "fieldMappings" : [ {
            "recordField" : "users.id",
            "categoryId" : "contact",
            "fieldId" : "contact.email",
            "resourceContext" : true
        }, {
            "recordField" : "users.name",
            "categoryId" : "contact",
            "fieldId" : "contact.name"
        }, {
            "recordField" : "payment.ccnumber",
            "categoryId" : "financial",
            "fieldId" : "financial.ccnumber"
        } ]
    }'

curl -X PUT -H "Content-Type: application/json" \
    "http://localhost:8890/mapping/groups/" \
-d '{
        "name" : "Transform",
        "fieldMappings" : [ {
            "recordField" : "Email",
            "categoryId" : "contact",
            "fieldId" : "contact.email",
            "resourceContext" : true
        }, {
            "recordField" : "Name",
            "categoryId" : "contact",
            "fieldId" : "contact.name"
        }, {
            "recordField" : "CCNumber",
            "categoryId" : "financial",
            "fieldId" : "financial.ccnumber"
        } ]
    }'

In this example, the user's email address is being as used as their unique identifier in the SQL schema. A better-practice is to use some opaque identifier so that email addresses aren't scattered across your system. If you created users with an opaque identifier that was also used to uniquely identify users in your schema, but never wanted to share that identifier externally, then your Postgres mapping might evolve to look like this:

curl -X PUT -H "Content-Type: application/json" \
        "http://localhost:8890/mapping/groups/" \
-d '{
        "name" : "postgres_prod",
        "fieldMappings" : [ {
            "recordField" : "users.id",
            "resourceContext" : true
        }, {
            "recordField" : "contact.name",
            "categoryId" : "contact",
            "fieldId" : "contact.name"
        }, {
            "recordField" : "contact.email",
            "categoryId" : "contact",
            "fieldId" : "contact.email"
        }, {
            "recordField" : "payment.ccnumber",
            "categoryId" : "financial",
            "fieldId" : "financial.ccnumber"
        } ]
    }'

This tells Tranquil Data that the value of users.id can be used to resolve a user, but does not represent any configured category's field, and therefore would never be returned to a caller via evaluation.

Disambiguating Fields

The previous examples show another use for mapping: explicitly disambiguating fields that exist in multiple locations for different purposes. A common example of this is email which might be a way to contact someone, a login string, a unique identifier that shouldn't be used outside of a database, etc. By clearly defining which meaning is assumed for a given purpose, it's possible to show that the terms a user agreed to are being met without putting burden on developers or data users to understand this nuance.

In the example configuration, a field named Email appears in both Contact and Credentials categories. In the calls above, the mapping is always using the version from Contact so that a login string that is also an email address doesn't get mis-used. Another mapping might explicitly reference the field in Credentials to allow use for the purpose of (e.g.) creating an account or auditing use.