Skip to main content

System Transformers

Introduction​

Husonym ships with 40+ System Transformers to give you an easy way to get started with anonymizing or generating data. There are two kinds of System Transformers:

  1. Generators - these begin with Generate and do not anonymize existing data. They only generate net-new synthetic data.
  2. Transformers - these begin with Transform and can anonymize existing data or generate new-new synthetic data based on your configuration.

Reference​

NameTypeDescription
Generate CategoricalstringRandomly selects a value from a defined set of categorical values
Generate EmailstringGenerates a new randomized email address.
Generate BooleanbooleanGenerates a boolean value at random.
Generate Card Numberint64Generates a card number.
Generate CitystringRandomly selects a city from a list of predefined US cities.
Generate CountrystringRandomly selects a country and returns it as a 2-letter country code or the full country name
Generate E164 Phone NumberstringGenerates a Generate phone number in e164 format.
Generate First NamestringGenerates a random first name.
Generate Float64float64Generates a random float64 value.
Generate Full AddressstringRandomly generates a street address.
Generate Full NamestringGenerates a new full name consisting of a first and last name.
Generate GenderstringRandomly generates one of the following genders: female, male, undefined, nonbinary.
Generate JavascriptanyExecutes provided javascript code in the transformer for every row
Generate Int64 Phone Numberint64Generates a new phone number of type int64 with a default length of 10.
Generate Random Int64int64Generates a random integer value with a default length of 4 unless the Integer Length or Preserve Length parameters are defined.
Generate Last Nameint64Generates a random last name.
Generate SHA256 HashstringSHA256 hashes a randomly generated value.
Generate SSNstringGenerates a completely random social security numbers including the hyphens in the format xxx-xx-xxxx.
Generate StatestringRandomly selects a US state and returns the two-character state code.
Generate Street AddressstringRandomly generates a street address.
Generate String Phone NumberstringGenerates a Generate phone number and returns it as a string.
Generate Random StringstringCreates a randomly ordered alphanumeric string with a default length of 10 unless the String Length parameter are defined.
Generate Unix Timestampint64Randomly generates a Unix timestamp.
Generate UsernamestringRandomly generates a username
Generate UTC TimestamptimeRandomly generates a UTC timestamp.
Generate UUIDuuidGenerates a new UUIDv4 id.
Generate Zip codestringRandomly selects a zip code from a list of predefined US zip codes.
Transform EmailstringTransforms an existing email address.
Transform E164 Phone NumberstringTransforms an existing E164 formatted phone number.
Transform First NamestringTransforms an existing first name
Transform Float64float64Transforms an existing float value.
Transform Full NamestringTransforms an existing full name.
Transform Int64 Phone Numberint64Transforms an existing phone number that is typed as an integer
Transform Int64int64Transforms an existing integer value.
Transform JavascriptanyExecutes user-provided javascript in on the input value.
Transform Last NamestringTransforms an existing last name.
Transform Phone NumberstringTransforms an existing phone number that is typed as a string.
Transform PII TextstringTransforms free-form text using PII analyzers
Transform StringstringTransforms an existing string value.
Transform Character ScramblestringTransforms an existing string value by scrambling the characters while maintaining the format.
PassthroughstringPasses the input value through to the destination with no changes.
NullstringInserts a null string instead of the source value.
Use Column DefaultanyApplies the predefined default value of a column as specified in the SQL database schema
Scramble Identityint64Designed for auto-incrementing columns. Takes an input value and scrambles it (ensures the output is not the same). This transformer handles UNIQUE and Primary Key auto-incrementing values.

Generate Categorical​

Randomly selects a value from a defined set of categorical values.

Configurations

NameDescriptionDefaultExample Output
CategoriesList of comma-separated string values that Husonym will randomly sample.value1,value2value1

Examples

CategoriesExample InputExample Output
falsered,blue,greengreen
falsered,blue,greenred

Generate Email​

Generates a new randomized email address.

Configurations

NameDescriptionDefaultExample Output
Email TypeProvides a way to generate unique email addresses by appending a UUID to the domainuuid_v4ab6b676b-0d0e-4e38-b98a-3935a832da7d@gmail.com

Examples

Example Output
ab6b676b-0d0e-4e38-b98a-3935a832da7d
jFrankd@msn.com

Generate Boolean​

Generates a random boolean value.

Configurations

There are no configurations for this transformer.

Examples

Example Output
true
false

Generate Card Number​

Generates a 16 digit card number that is valid by Luhn valid by default.

Configurations

NameDescriptionDefaultExample Output
Valid LuhnGenerate a card number that can pass luhn validation.true6621238134011099

Examples

Valid LuhnExample Output
false3627893345931223
true6621238134011099

Generate City​

Randomly selects a city from a list of predefined US cities.

Configurations

There are no configurations for this transformer.

Examples

Example Output
Louisville
Manchester

Generate Country​

Randomly selects a country and by default, returns it as a 2-letter country code.

Configurations

NameDescriptionDefaultExample Output
Full NameIf true, the transformer will return the full country namefalseAL

Examples

Example Output
Canada
AL

Generate E164 Phone Number​

Generates a new random international phone number including the + sign and no hyphens.

Configurations

NameDescriptionDefaultExample Output
MinMin is the lower range of the a length value.9+29748675
MaxMax is the upper range of a length value.12+2037486752

Examples

MinMaxExample Output
912+5209273239
1014+5209273239
1212+52092732323

Generate First Name​

Generates a random first name between 2 and 12 characters long.

Configurations

There are no configurations for this transformer.

Examples

Example Output
Benjamin
Gregory
Ashley

Generate Float64​

Generates a random floating point number with a max precision of 17. Go float64 adheres to the IEEE 754 standard for double-precision floating-point numbers.

Configurations

NameDescriptionDefault
RandomizeSignRandomly sets the sign of the value.false
MinSpecifies the min range for a generated float value.1.00
MaxSpecifies the max range for a generated float value.100.00
PrecisionSets the precision of the value. For example, a precision of 6 will generate a value such as 42.9812.6

Examples

Note: if randomize sign has been selected, this may cause the generated numbers to exist outside of the configured min/max range.

RandomizeSignMinMaxPrecisionExample Output
true2.0035.004-23.84
true4.1219.4337.28
false-30.002.008-15.54219
false-20.0030.00420.19

Generate Full Address​

Generates a randomly selected real full address that exists in the United States.

Configurations

There are no configurations for this transformer.

Examples

Example Output
509 Franklin Street Northeast Washington, DC, 20017
14 Huntingon Street Manchester, CT 06040

Generate Full Name​

Generates a new full name consisting of a first and last name.

Configurations

There are no configurations for this transformer.

Examples

Example Output
Edward Marshall
Dante Mayer
Gregory Johnson

Generate Gender​

Randomly generates one of the following genders: female (f), male (m), undefined (u), nonbinary (n).

Configurations

NameDescriptionDefaultExample Output
AbbreviateAbbreviate will abbreviate the output gender so that it is only one character.falseu

Examples

AbbreviateExample Output
falsemale
truef
falsenonbinary

Generate Int64 Phone Number​

Generates a new int64 phone number with a default length of 10.

Configurations

There are no configurations for this transformer.

Examples

Example Output
5209273239
9378463528

Generate Javascript​

Allows the user to define Javascript code and then execute that on every row that they are generating. This transformer supports JS ECMA 5 and most of the standard JS lib. It does not currently support importing third party libraries.

Configurations

NameDescriptionDefaultExample Output
CodeThe javascript code that will be executedreturn 'test'test

Example

CodeExample Output
return 'test'test

Generate Random Int64​

Generates a random int64 value with a default length of 4.

Configurations

NameDescriptionDefault
RandomizeSignRandomly sets the sign of the generated value.false
MinSpecifies the min range for a generated int64 value.1
MaxSpecifies the max range for a generated int64 value.40

Examples

Note: if randomize sign has been selected, this may cause the generated numbers to exist outside of the configured min/max range.

RandomizeSignMinMaxExample Output
true1040-23
true-20-3-14
false210054

Generate Last Name​

Generates a random last name.

Configurations

There are no configurations for this transformer.

Examples

Example Output
Dach
Sporer
Rodriguezon

Generate SHA256 Hash​

Generates a random SHA256 hash and returns it as a string.

Configurations

There are no configurations for this transformer.

Examples

Example Output
1b836d86806e16bd45164b9d665ce86dac9d9d55bf79d0e771265da9fbf950d1
c74f07e35232c844a6639fef0c82ad093351cfa1a1b458fd2afc44cc19211508C
18dd7ce288e5538fbdd9c736aa9951ddb317e42c0b928e2b4db672414a82f811

Generate SSN​

Generates a random social security numbers including the hyphens in the format xxx-xx-xxxx.

Configurations

There are no configurations for this transformer.

Examples

Example Output
892-23-7634
479-82-3224

Generate State​

Randomly selects a US state and by default, returns it as a 2-letter state code.

Configurations

NameDescriptionDefaultExample Output
Generate Full NameSet to true to return the full state name with a capitalized first letter. Returns the 2-letter state code by default.falseCA

Examples

State CodeExample Output
FalseCA
TrueCalifornia

Generate Street Address​

Randomly generates a street address.

Configurations

There are no configurations for this transformer.

Examples

Example Output
509 Franklin Street Northeast
14 Huntingon Street

Generate String Phone Number​

Generates a random 10 digit phone number and returns it as a string with no hyphens.

Configurations

NameDescriptionDefaultExample Output
MinMin is the lower range of the a string length. The minimum range is 9.9234232974865
MaxMax is the upper range of a length value. The maximum range is 15.152037486752

Examples

MinMaxExample Output
10145209273239
1115520927323912
1212520927323969

Generate Random String​

Generates a random string of alphanumeric characters.

For example: h2983hf28h92

By default, the random string transformer generates a string of 10 characters long.

Configurations

NameDescriptionDefaultExample Output
MinSpecifies the min range for a generated int64 value. This supports negative values.2jn9d
MaxSpecifies the max range for a generated int64 value. This supports negative values.7hd9n93

Examples

MinMaxExample Output
27je7R6
12029h23rega9
55h2dni

Generate Unix Timestamp​

Randomly generates a Unix timestamp that is in the past.

Configurations

There are no configurations for this transformer.

Examples=

Example Output
2524608000
946684800

Generate Username​

Randomly generates a username

Configurations

There are no configurations for this transformer.

Examples

Example Output
jdoe
lsmith

Generate UTC Timestamp​

Randomly generates a UTC timestamp. in the past.

Configurations

There are no configurations for this transformer.

Examples

Example Output
July 8, 2025, 00:00:00 UTC
September 21, 1989, 00:00:00 UTC

Generate UUID​

Generates a new UUID v4.

For example: 6d871028b072442c9ad9e6e4e223adfa

Configurations

NameDescriptionDefaultExample Output
Include hyphensIncludes hyphens in the output UUID. Note: For some databases, such as Postgres, it will automatically convert any UUID without hyphens to having hyphens.false6817d39c-ea4f-45ec-a81c-bc0801355576

Examples

Include HyphensExample Output
false53ef1f16cd0d455cb45c4cb6434ae807
trueab6b676b-0d0e-4e38-b98a-3935a832da7d

Generate Zipcode​

Generates a randomly selected US zipcode.

Configurations

There are no configurations for this transformer.

Examples

Example Output
27563
90237

Transform Email​

Anonymizes and transforms an existing email address.

Configurations

NameDescriptionDefaultExample InputExample Output
Preserve LengthPreserve Length will ensure that the output email is the same length as the inputfalsejohn@gmailwilliam@gmail.com
Preserve DomainPreserve Domain will ensure that the output email domain is the same as the input email domainfalsejohn@gmailwilliam@yahoo.com
Excluded DomainsTakes in a list of comma separated domains and excludes them from the transformer[]@gmailwilliam@gmail.com
Email TypeProvides a way to generate unique email addresses by appending a UUID to the domainuuid_v4john@gmailab6b676b-0d0e-4e38-b98a-3935a832da7d@gmail.com
Invalid Email ActionProvide a way for Husonym to handle email strings that are not formatted as valid email addressesrejectjohn@gmail..comnull

Examples

There are several ways you can mix-and-match configurations to get different potential email formats. Note that the Excluded Domains List pertains to the transformed emails. If the PreserverDomain is true and there is a domain in the Excluded Domains list, then the default logic would be to not preserve the domain but since we are excluding those domains from the transformation logic via the exclude list, then those domains will be excluded.

PreserveLengthPreserveDomainExclude DomainsExample InputInvalid Email ActionExample Output
falsetruegmail.comjohn@gmail.comrejectf9nkeuh@ergwer.wewe
falsetruegmail.comjohn@gmail....comreject``
truefalsegmail.comf98723uh@gmail.comnullf98723uh@gmail.com
truetruegmail.comjohn@gmail.compassthroughf98723uh@weefw.wefw
falsefalsegmail.comjohn@gmail.compassthroughf98723uh@gmail.com

Transform E164 Phone Number​

Anonymizes and transforms an existing E164 formatted phone number.

Configurations

NameDescriptionDefaultExample InputExample Output
Preserve LengthPreserve Length will ensure that the output e164 phone number is the same length as the input phone number. If set to false, it will generate a value between 9 and 15 characters long.false+8923786243+290374867526

Examples

Preserve LengthExample InputExample Output
false+2890923784+520927392
true+62890923784+52092732393

Transform First Name​

Anonymizes and transforms an existing first name.

Configurations

NameDescriptionDefaultExample InputExample Output
PreserveLengthPreserve Length will ensure that the output first name is the same length as the input first name. The preserveLength config only preserves names up to 12 characters long.falseJohnathanBill

Examples

PreserveLengthExample InputExample Output
falseJoeBenjamin
trueFrankDante

Transform Float64​

Anonymizes and transforms an existing float value.

Configurations

NameDescriptionDefault
Relative Range MinSets the lower bound of a range for the input value.20.00
Relative Range MaxSets the upper bound of a range for the input value.50.00

Examples

Randomization Range MinRandomization Range MaxExample InputExample Output
102027.2317.248
14292.344-9.24
510-8.23-4.19

Transform Full Name​

Anonymizes and transforms an existing full name.

Configurations

NameDescriptionDefaultExample InputExample Output
PreserveLengthPreserve Length ensures that the output full name is the same length as the input full name by preserving the length of the first and last names. It only preserves first and/or last names up to 12 characters long. So the max full name length is 25 characters.falseJohnBill

Examples

PreserveLengthExample InputExample Output
falseJoe SmithEdward Marshall
trueFrank JonesDante Mayer
falseAidan BellGregory Johnson

Transform Int64 Phone Number​

Anonymizes and transforms an existing int64 phone number.

#Configurations

NameDescriptionDefaultExample InputExample Output
PreserveLengthPreserve Length will ensure that the output phone number is the same length as the input phone number.false8923877862432903748675392

Examples

PreserveLengthExample InputExample Output
false28909237845209273239
true38393049576395745371

Transform Int64​

Anonymizes and transforms an existing int64 value.

Configurations

NameDescriptionDefault
Relative Range MinSets the lower bound of a range relative to the input value.20
Relative Range MaxSets the upper bound of a range relative to the input value.50

Examples

Relative Range MinRelative Range MaxExample InputExample Output
551015
10702391
20903027

Transform Javascript​

Executes user provided code on the incoming input value. It's a very flexible way to transform the incoming input value however you'd like.

For example:

function append(value) {
return value + 'this_is_transformed';
}

return append(value);

Configurations

NameDescriptionDefault
CodeThe user provided codereturn value + test

Examples

Relative Range MinRelative Range MaxExample InputExample Output
551015
10702391
20903027

Transform Last Name​

Anonymizes and transforms an existing last name.

Configurations

NameDescriptionDefaultExample InputExample Output
PreserveLengthPreserve Length will ensure that the output last name is the same length as the input last name. The preserveLength config only preserves names up to 12 characters long.falseHillsKunze

Examples

PreserveLengthExample InputExample Output
falseLeiDach
trueLittelSporer
falseJohnsonstonRodriguezon

Transform Phone Number​

Anonymizes and transforms an existing phone number that is typed as a string.

Configurations

NameDescriptionDefaultExample InputExample Output
PreserveFormatKeeps the prefix (0 and the next digit, or +/00, the country code and the next digit), every separator and the length; replaces the other digits with a keyed permutation.true+33 6 12 34 56 78+33 6 48 34 17 66
PreserveLengthPreserve Length will ensure that the output phone number is the same length as the input phone number. Implied by PreserveFormat.false892387243786243290374867526392

With PreserveFormat, two distinct numbers never give the same output, so a unique column stays unique, and a number written nationally or internationally keeps the same digits. Under the Athanor engine, the key is derived from the job's consistency scope: the same number gives the same output across tables and runs. Under Benthos, the key is drawn when the worker starts.

The default applies to mappings created from now on. A mapping saved before PreserveFormat existed keeps its behavior.

Examples

PreserveFormatPreserveLengthExample InputExample Output
true-06 12 34 56 7806 48 34 17 66
true-+33 6 12 34 56 78+33 6 48 34 17 66
true-+1 (555) 123-4567+1 (556) 887-3834
falsefalse2890923784520927323239
falsetrue28909237845209223539

Transform PII Text​

Anonymizes free text: a note, a comment, the body of a message. A Presidio analyzer finds the personal data the text carries, and Husonym rewrites each finding where it stands. The rest of the text is left exactly as it is.

It needs a license and PRESIDIO_ANALYZER_URL (see Environment Variables). Only the analyzer is used: Husonym does not call a Presidio anonymizer. The license is needed however the transformer is reached: mapped to a column, stored in a user-defined transformer, or called from a script with husonym.transformPiiText.

Configurations

NameDescriptionDefault
ScoreThresholdThe confidence, from 0 to 1, the analyzer must reach for a finding to count. A finding scored exactly at the threshold counts. 0 keeps everything the analyzer suspects.0.5
LanguageThe two-letter language of the text.PRESIDIO_DEFAULT_LANGUAGE, else en
DefaultAnonymizerWhat to do with a finding whose entity has no anonymizer of its own.Replace with the entity type
EntityAnonymizersWhat to do with the findings of one entity (PERSON, PHONE_NUMBER, the name of a deny list…). An entity listed here gets this anonymizer and not the default.
AllowedEntitiesThe entities to look for. Empty, all of them. The names of the deny lists below are always looked for.
DenyRecognizersLists of words to treat as personal data, each list an entity of its own, named after the list. They are looked for in the language of the text.
AllowedPhrasesPhrases to leave as they are. A finding is left when its whole text is one of them: the case counts, a phrase that is only part of a finding does not count. An accented letter matches however it is encoded.

Anonymizers

AnonymizerWhat stands in place of a findingJörg Müller, found as PERSON
ReplaceThe given value. Without one, the entity type between angle brackets.<PERSON>
RedactNothing: the finding is removed.
MaskThe finding with chars_to_mask of its characters overwritten by masking_char, from its start, or from its end with from_end. Without chars_to_mask, every character is overwritten. masking_char is one character.**** Müller (4 characters), *********** (no count)
HashA keyed hash of the exact text of the finding, in lowercase hexadecimal: 64 characters for SHA256, 128 for SHA512, 32 for MD5 or no algorithm. Every length is computed with HMAC-SHA-2; the algorithm chooses the length of the output.3f1c… (64 characters)
TransformThe output of another transformer, which is handed the text of the finding. Without a transformer, one is chosen from the entity: a name for PERSON, a phone number for PHONE_NUMBER, an e-mail address for EMAIL_ADDRESS, a card number for CREDIT_CARD, a social security number for US_SSN, an IP address for IP_ADDRESS, and a random SHA-256 hash for anything else.Lena Sporer

A job that sets no language is analyzed in PRESIDIO_DEFAULT_LANGUAGE, else in en. Under the Benthos engine, a job that sets no language is always analyzed in en.

The transformer of a Transform anonymizer, like every anonymizer, works on the text its finding owns once overlaps are settled (see below): two findings joined into one are handed over as one text, and a finding that shares characters with another is handed the characters that belong to it.

Hashes

The same text gives the same hash wherever the key is the same, and the text cannot be recovered from its hash without the key. The case, the spaces and the encoding of the text count: Bob and bob have two hashes. The entity does not count: a text has one hash, whatever it was found as.

  • In a run of the Athanor engine, the key is derived from the job's consistency scope: the same text has the same hash in every table of the run. With the scope job or account it keeps that hash from one run to the next; with the default scope run, every run gives new hashes.
  • Everywhere else — a call to AnonymizeSingle or AnonymizeMany, the preview of a column, a run of the Benthos engine — the key belongs to the account and to the API process: it is derived, for each account, from a key drawn when the API starts. Two accounts never share it. Hashes are stable while that API process runs. They change when it restarts, and two replicas of the API give two different hashes for the same text. The preview of a column therefore never shows the hash a run will write.

Findings that overlap

Each character of the text belongs to one finding at most. Findings of the same entity that overlap, or that only spaces separate, are rewritten as one. A finding inside another is rewritten with it. Where findings of two entities overlap, each character goes to the finding the analyzer is most confident in.

Failures

A value is rewritten exactly or not at all. When the analyzer does not answer or refuses the text, when its answer does not fit the text, or when the transformer of a Transform anonymizer fails, the value fails: it is never returned half rewritten, and a run stops on it. An empty value and a null value are returned as they are, without calling the analyzer.

Transform String​

Anonymizes and transforms an existing string value.

Configurations

NameDescriptionDefaultExample InputExample Output
Preserve LengthPreserves the length of the input integer to the output string.falsehello9Fau3

Examples

PreserveLengthExample InputExample Output
falsebillje7R6
trueloremi1NMe

Transform Character Scramble​

Anonymizes and transforms an existing string value by scrambling the characters while maintaining the format based on a user provided regular expression. Letters will be replaced with letters, numbers with numbers and non-number or letter ASCII characters such as "!&*" with other characters.

Configurations

NameDescriptionDefaultExample InputExample Output
RegexA regular expression to match a part of the string to scramble. If no regex is specified that we will scramble the entire string.emptyellohello -> hmato

Examples

RegexExample InputExample Output
helloworlddidlenfioq
elloHello World!Hjiqs World!
Hello 12Hello 1234Praji 2834

Transform Scramble Identity​

Anonymizes and transforms integers in a unique way. Designed for auto-incrementing primary key values. The input will never equal the output.

Configurations

There are no configurations for this transformer at this time.

Examples

Example InputExample Output
112
100123
525

Passthrough​

Passes the input data out to the output without making any modifications to it. This is useful in many circumstances but cautious of accidentally leaking sensitive data through this transformer.

Null​

Simply returns a null value. This may be useful if you a column that can't be null but don't have a specific value that you want to insert.

Configurations

There are no configurations for this transformer.

Examples

Here are some examples of what an output null value may look like.

Example InputExample Output
N/Anull

Use Column Default​

Automatically applies the predefined default value of a column as specified in the SQL database schema. This means whenever new data is added without specifying a value for this column, the system will insert the default value set for that column in the database. This is typically used in conjunction with GENERATED columns.

Configurations

There are no configurations for this transformer.

Examples

Example InputExample Output
11

Works-in-progress​

This section is for detailing what we have limited or in-progress support for in regards to data transformations.

Array data types​

Husonym has limited support for array data types today.

  1. Passthrough - Array data types can be set to passthrough and have their values directly passed through from the source to the destination.
  2. Null - If the column is nullable, this transformer can be safely set.
  3. Default - If the column has a default set, this transformer can be set to allow the column be set to its default state.
  4. Javascript Code - The JS code transformer has support for array data types as the logic is effectively up to the function's implementation.