Transformers

Introduction

Transformers are re-usable JavaScript functions that manipulate data in UserClouds. They allow you to minimize the data that you pass or store for each use case. This is key for complying with the data minimization principles in regulations like GDPR.

Transformers can be arbitrarily smart about how they preserve and obscure the information and structure of the data they encode.

Transformers allow you to retain select structure and information from the raw data for different use cases, like sorting alphabetically, common domain analysis or simply flowing through your systems without triggering validation errors. For example, if you want to conduct analysis assessing the differences in behavior between children and adults, you may use a data transformer to pass a string indicating child or adult, instead of pulling raw Date of Birth from the store.

Transformers are used in two places:

Each accessor (read API) is associated with a transformer per column, dictating how the data in that column is obscured or minimized when the accessor is called
When a token is created, a data transformer is specified to dictate how the token should be generated from the raw data (and therefore what information, if any, should be retained in the token)

Examples of Transformers

Transformers range from the simplest UUID function that replaces any given data with a unique identifier, to custom-written Javascript that runs in a carefully-controlled sandbox.

One simple transformer receives raw data, and creates a UUID token.

Let’s look at four possible transformers, to see how transformers work.

Function	Example Data	Example Output
Replace with UUID	[email protected]	6ac33e45-1117-457f-b610-b49afd0bd551
Retain email structure, first three letters & common domain names	[email protected]	[email protected]
Show year of birth	2/24/1955	1955
Show country from phone	+44 7899 123456	United Kingdom

Types of Transformers

There are four types of transformers:

PassThrough Transformers pass through the raw data, unchanged
Data Transformers mask, categorize, inject noise or group raw data. They are used to retain the minimum characteristics of a piece of raw data for a particular task, without being resolvable back into the raw data.
Tokenize By Value Transformers create a resolvable token with an associated access policy. If the value of the raw data later changes, the token will resolve to the value of the data at the point of transformation.
Tokenize By Reference Transformers create a resolvable token with an associated access policy. If the value of the raw data later changes, the token will resolve to the latest value of the data. Tokens generated by reference also respect user consent changes.

Structure of a Transformer

A transformer consists of:

Name
Description
id - a unique transformer identifier, which can be used as a reference when creating accessors and tokens
Input Type
Output Type
Transform Type - as described above
Transform Function - a transform function with the signature func(data Object, parameters Object) (Token | error)
Transform Parameters - a static JSON object (not containing un-encoded PII) that is available at runtime, allowing you to parameterize and reuse functions like "obfuscate all but the first X letters of these emails"

The PreserveCommonValue parameter allows you to optionally preserve common email domains (like gmail.com). Rare domains, like userclouds.com, will be obscured.

Managing Transformers

UserClouds has several off-the-shelf transformers for common use cases, like masking emails, national ID numbers and credit card details. However you can also create custom functions, in three ways:

Call the CreateTransformer() API
Create functions in the UserClouds UI

For more info on creating transformers, see our How To Guide on Creating a Transformer.