Computed fields - Forest Documentation

Forest allows creating new fields on any collection, either computationally, by fetching data on an external API, or based on other data available on the connected data sources. By default, the fields that you create will be read-only, but make them filterable and sortable, and writable by using the relevant methods.

How does it work?

When creating a new field you will need to provide:

Field	Description
`columnType`	Type of the new field (any primitive or composite type)
`dependencies`	List of fields needed from the source records and linked records to run the handler
`getValues`	Handler which computes the new value for a batch of records
`enumValues` (optional)	When `columnType` is `Enum`, you must specify the values that the field will support

Examples

Adding a field by concatenating other fields

This example adds a user.displayName field, which is computed by concatenating the first and last names.

// "user" Collection has the following structure: { id, firstName, lastName }
agent.customizeCollection('user', collection => {
  collection.addField('displayName', {
    // Type of the new field
    columnType: 'String',

    // Dependencies which are needed to compute the new field (must not be empty)
    dependencies: ['firstName', 'lastName'],

    // Compute function for the new field
    // Note that the function computes the new values in batches: the return value
    // must be an array which contains the new values in the same order than the
    // provided records.
    getValues: (records, context) =>
      records.map(r => `${r.firstName} ${r.lastName}`),
  });
});

# "user" Collection has the following structure: { id, firstName, lastName }
ForestAdmin.customize do
  customize_collection('user') do |collection|
    collection.add_field('displayName', ComputedDefinition.new(
      column_type: 'String',
      dependencies: ['firstName', 'lastName'],
      values: proc { |records| records.map { |r| "#{r['firstName']} #{r['lastName']}" } }
    ))
  end
end

# "user" Collection has the following structure: { id, firstName, lastName }
@create_agent.collection :user do |collection|
  collection.computed_field :displayName,
    type: 'String',
    depends_on: [:firstName, :lastName] do |records|
      records.map { |r| "#{r['firstName']} #{r['lastName']}" }
    end
end

Adding a field that depends on another computed field

This example adds a user.displayName field, then another that capitalizes it.

// "user" Collection has the following structure: { id, firstName, lastName }
agent.customizeCollection('user', collection => {
  collection
    // Create a field which is computed by concatenating the first and last names
    .addField('displayName', {
      columnType: 'String',
      dependencies: ['firstName', 'lastName'],
      getValues: (records, context) =>
        records.map(r => `${r.firstName} ${r.lastName}`),
    })

    // Create another field which is computed by uppercasing the first field
    .addField('displayNameCaps', {
      columnType: 'String',
      dependencies: ['displayName'], // You can depend on other computed fields
      getValues: (records, context) => records.map(r => r.displayName.toUpperCase()),
    });
});

# "user" Collection has the following structure: { id, firstName, lastName }
ForestAdmin.customize do
  customize_collection('user') do |collection|
    collection
      # Create a field which is computed by concatenating the first and last names
      .add_field('displayName', ComputedDefinition.new(
        column_type: 'String',
        dependencies: ['firstName', 'lastName'],
        values: proc { |records| records.map { |r| "#{r['firstName']} #{r['lastName']}" } }
      ))
      # Create another field which is computed by uppercasing the first field
      .add_field('displayNameCaps', ComputedDefinition.new(
        column_type: 'String',
        dependencies: ['displayName'], # You can depend on other computed fields
        values: proc { |records| records.map { |r| r['displayName'].upcase } }
      ))
  end
end

# "user" Collection has the following structure: { id, firstName, lastName }
@create_agent.collection :user do |collection|
  # Create a field which is computed by concatenating the first and last names
  collection.computed_field :displayName,
    type: 'String',
    depends_on: [:firstName, :lastName] do |records|
      records.map { |r| "#{r['firstName']} #{r['lastName']}" }
    end

  # Create another field which is computed by uppercasing the first field
  collection.computed_field :displayNameCaps,
    type: 'String',
    depends_on: [:displayName] do |records| # You can depend on other computed fields
      records.map { |r| r['displayName'].upcase }
    end
end

Adding a field that depends on a many-to-one relationship

We can improve the previous example by adding the city of the user to the display name.

// Structure:
// User    { id, addressId, firstName, lastName }
// Address { id, city }

agent.customizeCollection('user', collection => {
  collection.addField('displayName', {
    columnType: 'String',

    // We added 'address:city' in the list of dependencies,
    // which tells forest to fetch the related record
    dependencies: ['firstName', 'lastName', 'address:city'],

    // The address is now available in the parameters
    getValues: (records, context) =>
      records.map(r => `${r.firstName} ${r.lastName} (from ${r.address.city})`),
  });
});

# Structure:
# User    { id, addressId, firstName, lastName }
# Address { id, city }

ForestAdmin.customize do
  customize_collection('user') do |collection|
    collection.add_field('displayName', ComputedDefinition.new(
      column_type: 'String',

      # We added 'address:city' in the list of dependencies,
      # which tells forest to fetch the related record
      dependencies: ['firstName', 'lastName', 'address:city'],

      # The address is now available in the parameters
      values: proc { |records|
        records.map { |r| "#{r['firstName']} #{r['lastName']} (from #{r['address']['city']})" }
      }
    ))
  end
end

# Structure:
# User    { id, addressId, firstName, lastName }
# Address { id, city }

@create_agent.collection :user do |collection|
  collection.computed_field :displayName,
    type: 'String',
    # We added 'address:city' in the list of dependencies,
    # which tells forest to fetch the related record
    depends_on: [:firstName, :lastName, 'address:city'] do |records|
      # The address is now available in the parameters
      records.map { |r| "#{r['firstName']} #{r['lastName']} (from #{r['address']['city']})" }
    end
end

Adding a field that depends on a one-to-many relationship

Let’s add a user.totalSpending field by summing the amount of all orders.

// Structure
// User  { id }
// Order { id, customer_id, amount }

agent.customizeCollection('user', collection => {
  collection.addField('totalSpending', {
    columnType: 'Number',
    dependencies: ['id'],
    getValues: async (records, context) => {
      const recordIds = records.map(r => r.id);

      // We're using Forest's query interface
      // (use an ORM or a plain SQL query)
      const filter = {
        conditionTree: { field: 'customer_id', operator: 'In', value: recordIds },
      };
      const aggregation = {
        operation: 'Sum',
        field: 'amount',
        groups: [{ field: 'customer_id' }],
      };
      const rows = await context.dataSource
        .getCollection('order')
        .aggregate(filter, aggregation);

      return records.map(record => {
        const row = rows.find(r => r.group.customer_id === record.id);
        return row?.value ?? 0;
      });
    },
  });
});

# Structure
# User  { id }
# Order { id, customer_id, amount }

ForestAdmin.customize do
  customize_collection('user') do |collection|
    collection.add_field('totalSpending', ComputedDefinition.new(
      column_type: 'Number',
      dependencies: ['id'],
      values: proc { |records, context|
        record_ids = records.map { |r| r['id'] }

        # We're using Forest's query interface
        # (use an ORM or a plain SQL query)
        filter = { condition_tree: { field: 'customer_id', operator: 'In', value: record_ids } }
        aggregation = { operation: 'Sum', field: 'amount', groups: [{ field: 'customer_id' }] }
        rows = context.datasource.get_collection('order').aggregate(filter, aggregation)

        records.map do |record|
          row = rows.find { |r| r[:group]['customer_id'] == record['id'] }
          row ? row[:value] : 0
        end
      }
    ))
  end
end

# Structure
# User  { id }
# Order { id, customer_id, amount }

@create_agent.collection :user do |collection|
  collection.computed_field :totalSpending,
    type: 'Number',
    depends_on: [:id] do |records, context|
      record_ids = records.map { |r| r['id'] }

      # We're using Forest's query interface
      # (use an ORM or a plain SQL query)
      filter = { condition_tree: { field: 'customer_id', operator: 'In', value: record_ids } }
      aggregation = { operation: 'Sum', field: 'amount', groups: [{ field: 'customer_id' }] }
      rows = context.datasource.get_collection('order').aggregate(filter, aggregation)

      records.map do |record|
        row = rows.find { |r| r[:group]['customer_id'] == record['id'] }
        row ? row[:value] : 0
      end
    end
end

Adding a field fetching data from an API

Let’s imagine that we want to check if the email address of our users is deliverable. We can use a verification API to perform that work.

const emailVerificationClient = require('@sendchimplio/client');
emailVerificationClient.setApiKey(process.env.SENDCHIMPLIO_API_KEY);

// "User" Collection has the following structure: { id, email }
agent.customizeCollection('user', collection => {
  collection.addField('emailDeliverable', {
    columnType: 'Boolean',
    dependencies: ['email'],
    getValues: async (records, context) => {
      // Call the API to verify emails
      const response = await emailVerificationClient.verifyEmails(
        records.map(r => r.email),
      );

      // Return values in the same order than the source records
      return records.map(r => {
        const check = response[r.email];
        return check.domainValid && (!usernameChecked || usernameValid);
      });
    },
  });
});

# "User" Collection has the following structure: { id, email }
ForestAdmin.customize do
  customize_collection('user') do |collection|
    collection.add_field('emailDeliverable', ComputedDefinition.new(
      column_type: 'Boolean',
      dependencies: ['email'],
      values: proc { |records, context|
        # Call the API to verify emails
        response = EmailVerificationClient.verify_emails(records.map { |r| r['email'] })

        # Return values in the same order than the source records
        records.map do |r|
          check = response[r['email']]
          check[:domain_valid] && (!check[:username_checked] || check[:username_valid])
        end
      }
    ))
  end
end

# "User" Collection has the following structure: { id, email }
@create_agent.collection :user do |collection|
  collection.computed_field :emailDeliverable,
    type: 'Boolean',
    depends_on: [:email] do |records, context|
      # Call the API to verify emails
      response = EmailVerificationClient.verify_emails(records.map { |r| r['email'] })

      # Return values in the same order than the source records
      records.map do |r|
        check = response[r['email']]
        check[:domain_valid] && (!check[:username_checked] || check[:username_valid])
      end
    end
end

Performance

When adding many fields, keep in mind that:

You should refrain from making queries to external services
- Use relationships in the dependencies array when that is possible
- Use batch API calls instead of performing requests one by one inside of the records.map handler
Only add fields you need in the dependencies list
- This will reduce the pressure on your data sources (fewer columns to fetch)
- And increase the probability of reducing the number of records passed to your handler (records are deduplicated)
Do not duplicate code between handlers of different fields: fields can depend on each other (no cycles allowed)

​How does it work?

​Examples

​Adding a field by concatenating other fields

​Adding a field that depends on another computed field

​Adding a field that depends on a many-to-one relationship

​Adding a field that depends on a one-to-many relationship

​Adding a field fetching data from an API

​Performance