Transformers

Introduction

Transformers are re-usable functions that manipulate data in UserClouds. They allow you to minimize the data that you pass or store for each use case. This is key for complying with the data minimization principles in regulations like GDPR.

Transformers allow you to retain select structure and information from the raw data for different use cases, like sorting alphabetically, zip code analysis or simply flowing through your systems without triggering validation errors. For example, if you want to conduct analysis assessing the differences in behavior between children and adults, you may use a data transformer to pass a string indicating child or adult, instead of pulling raw Date of Birth from the store.

Transformers are executed in two places:

• Each accessor (read path) is associated with a transformer per column, dictating how the data in that column is obscured or minimized when the accessor is called
• When a token is created, a data transformer is specified to dictate how the token should be generated from the raw data (and therefore what information, if any, should be retained in the token)

In addition, column default transformers provide a way to consistently apply transformations across all accessors that extract data from specific columns:

• Column Default Transformers: These transformers are applied by default to all reads on specific columns. For example, you might apply a masking transformer to the SSN column to obscure the data by default in all read queries. Learn more here.

Examples of Transformers

Transformers range from the simplest UUID function that replaces any given data with a unique identifier, to custom-written Javascript that runs in a carefully-controlled sandbox.

Let’s look at four possible transformers, to see how transformers work.

Types of Transformers

There are four types of transformers:

• PassThrough Transformers pass through the raw data, unchanged
• Data Transformers mask, categorize, inject noise or group raw data. They are used to retain the minimum characteristics of a piece of raw data for a particular task, without being resolvable back into the raw data.
• Tokenize By Value Transformers create a resolvable token with an associated access policy. If the value of the raw data later changes, the token will resolve to the value of the data at the point of transformation.
• Tokenize By Reference Transformers create a resolvable token with an associated access policy. If the value of the raw data later changes, the token will resolve to the latest value of the data. Tokens generated by reference also respect user consent changes.

Structure of a Transformer

A transformer consists of:

• Name
• Description
• id - a unique transformer identifier, which can be used as a reference when creating accessors and tokens
• Input Type
• Output Type
• Transform Type - as described above
• Transform Function - a transform function with the signature func(data Object, parameters Object) (Token | error)
• Transform Parameters - a static JSON object (not containing un-encoded PII) that is available at runtime, allowing you to parameterize and reuse functions like "obfuscate all but the first X letters of these emails"

Managing Transformers

UserClouds has several off-the-shelf transformers for common use cases, like masking emails, national ID numbers and credit card details. However you can also create custom functions, in three ways:

1. Call the CreateTransformer API
2. Create functions in the UserClouds UI

For more info on creating transformers, see our How To Guide on Creating a Transformer.