Stage Builder and Attributes System

The Stage Builder and Attributes system is a powerful abstraction for building MongoDB aggregation pipelines in a type-safe and maintainable way. This guide explains how to use these systems and how to implement them for new entities.

Overview

The system consists of two main parts:

Attributes System: Defines the schema and behavior of entity fields
Stage Builder: Builds MongoDB aggregation pipelines based on attributes

Attributes System

What are Attributes?

Attributes define the schema and behavior of entity fields. They provide:

Type information
Filtering capabilities
UI component mapping
Relationship definitions
Selection behavior

Types of Attributes


type Attribute =
  | StringAttribute
  | NumberAttribute
  | BooleanAttribute
  | RelationAttribute
  | EnumAttribute<EnumType>
  | I18nStringAttribute
  | ImageAttribute
  | ExistenceAttribute
  | SelectAttribute;

Each attribute type has specific properties:


interface BaseAttribute {
  name: string;
  description: string;
  type: "string" | "number" | "boolean" | "enum" | "select" | "range" | "relation";
  entity: "card" | "variant" | "inventory" | "product" | "price";
  array: boolean;
  filter?: FilterComponentType;
  selectable?: boolean;
  singleChoice?: boolean;
}

Creating Attributes

Example of defining attributes for a new entity:


const entityAttributes: Attributes = {
  name: {
    name: "Name",
    description: "Entity name",
    type: "string",
    entity: "product",
    array: false,
    filter: FilterComponentType.Input
  },
  status: {
    name: "Status",
    description: "Entity status",
    type: "enum",
    entity: "product",
    array: false,
    filter: FilterComponentType.Select,
    values: {
      active: { label: "Active" },
      inactive: { label: "Inactive" }
    }
  }
};

Stage Builder System

The Stage Builder system provides a modular way to construct MongoDB aggregation pipelines. It consists of several components:

1. Base Stage Builder

The BaseStageBuilder class provides common pipeline stage building functionality:

Pagination (skip/limit)
Sorting
Common interface implementation


abstract class BaseStageBuilder<TQuery extends PaginationQuery & SortQuery> {
  abstract buildMatch(query: TQuery): Promise<PipelineStage.Match | undefined>;
  abstract buildLookupStages(query: TQuery): PipelineStage[] | undefined;
  abstract buildProjection(query: TQuery): Promise<PipelineStage.Project | undefined>;
  abstract buildSearch(query: TQuery): PipelineStage.Search | undefined;
  abstract buildGroupByReplace(query: TQuery): PipelineStage[] | undefined;
  
  // Implemented methods for common operations
  buildLimit(query: TQuery): PipelineStage.Limit | undefined;
  buildSkip(query: TQuery): PipelineStage.Skip | undefined;
  buildSort(query: TQuery): PipelineStage.Sort | undefined;
}

2. Entity-Specific Stage Builder

For each entity that needs pipeline building capabilities, create a class extending BaseStageBuilder:


@injectable()
class EntitySearchQueryStageBuilder extends BaseStageBuilder<EntitySearchQuery> {
  constructor(
    @inject(GameConfigurationService)
    private readonly gameConfigurationService: GameConfigurationService,
  ) {
    super();
  }
 
  public async buildMatch(query: EntitySearchQuery): Promise<PipelineStage.Match> {
    const stage: PipelineStage.Match = { $match: {} };
    
    // Implement entity-specific match logic
    if (query.filters) {
      const { attributes, entityMappings } = await this.getAttributesAndMappings();
      const nexusQueryBuilder = new NexusPipelineMatchBuilder(
        entityMappings,
        attributes
      );
      nexusQueryBuilder.addFilters(stage, query.filters);
    }
 
    return stage;
  }
 
  // Implement other abstract methods...
}

MongoDB Atlas Search Indexes

When using the Stage Builder system with MongoDB Atlas Search capabilities, it’s important to note that certain field types require manual index configuration. The system uses compound search stages in the aggregation pipeline, which rely on properly configured Atlas Search indexes.

Required Index Types

For optimal search functionality, you need to configure the following index types in your MongoDB Atlas Search indexes:

String Enum Fields: Fields containing string enums (like status, type, etc.) should be indexed as token type
ObjectId Fields: Fields containing MongoDB ObjectIds (like references to other collections) should be indexed as objectId type
Nested Object Fields: For nested objects containing searchable fields, use the document type with appropriate field mappings

Example Index Configuration

Here’s an example of how to configure the indexes in your MongoDB connection:


await mongoose.connection.db?.collection("products").createSearchIndex({
  definition: {
    mappings: {
      dynamic: true,
      fields: {
        color: {
          type: "token",
        },
        expansion: {
          type: "objectId",
        },
        finish: {
          type: "token",
        },
        rarity: {
          type: "token",
        },
        type: {
          type: "token",
        },
      },
    },
  },
  name: "default",
  type: "search",
});

Important Notes

Indexes must be created before using search functionality
The index configuration should match your entity’s attribute types
Changes to index configuration require reindexing the collection
Search functionality is only available when ATLAS_FEATURES_ON environment variable is enabled

Implementation Example

Here’s how to implement the system for a new entity:

Define Entity Attributes:


// packages/shared/attributes/src/entity-attributes.ts
export const entityAttributes: Attributes = {
  // Define entity-specific attributes
};

Note: Game-specific attributes live in @repo/game-configuration at packages/games/game-configuration/src/lib/attributes/. These include attributes that vary per game, such as card-specific fields, rarities, and other game-dependent properties.

Create Entity Mappings:


const entityMappings: EntityTableMapping = {
  relation1: "field1",
  relation2: "field2"
};

Implement Stage Builder:


@injectable()
class EntitySearchQueryStageBuilder extends BaseStageBuilder<EntitySearchQuery> {
  constructor(
    @inject(GameConfigurationService)
    private readonly gameConfigurationService: GameConfigurationService,
  ) {
    super();
  }
 
  // Implement abstract methods
}

Create Service:


@injectable()
export class EntityService {
  private readonly pipelineBuilder: PipelineBuilder<
    EntitySearchQuery,
    typeof EntityModel,
    Entity
  >;
 
  constructor(
    @inject(EntitySearchQueryStageBuilder)
    entitySearchQueryStageBuilder: EntitySearchQueryStageBuilder,
  ) {
    this.pipelineBuilder = new PipelineBuilder(
      EntityModel,
      entitySearchQueryStageBuilder
    );
  }
 
  // Implement service methods using pipelineBuilder
}