Adding Enrichment Vendors

This guide covers the end-to-end process of adding a new third-party data vendor to the CRED platform. Adding a vendor requires changes across multiple projects.

Overview

When adding a new enrichment vendor (e.g., Apollo, Lusha, Cognism), you need to update:

Project	Repository	Purpose
CRED Model	cred-model	Database tables for storing provider data
DBT	cred-dbt	DataDescription exposure and field mappings
Model API	cred-model-api	API integration and data fetching
Commercial	cred-api-commercial	UI integration, waterfall configuration

Process Flow

┌─────────────┐    ┌─────────────┐    ┌─────────────┐    ┌─────────────┐
│ CRED Model  │ →  │    DBT      │ →  │  Model API  │ →  │ Commercial  │
│  (Tables)   │    │  (Schema)   │    │ (Endpoints) │    │    (UI)     │
└─────────────┘    └─────────────┘    └─────────────┘    └─────────────┘

Quick Start Summary

Step	Project	Repository	What to Do	Key Files
1	CRED Model	cred-model	Create migration + sync config	migrations/*.ts, sync-providers.config.ts
2	DBT	cred-dbt	Add to provider tables	data_description_providers_tables.sql
3	Model API	cred-model-api	Domain, services, adapter, factory, GraphQL	~15 files across layers
4	Commercial	cred-api-commercial	Enum, feature, seed, provider list, resilience, validation	6-7 files

Naming Conventions

Critical: Use Consistent Naming Across All Projects

All projects are interrelated. Using inconsistent names will break the integration.

Element	Convention	Example
Abbreviation	UPPERCASE, no spaces	AP, COG, LUSHA, ROCKETREACH
Table Name (Person)	{ABBR}Person	APPerson, CGPerson, LSPerson, RRPerson
Table Name (Company)	{ABBR}Company	APCompany
Provider Unique ID	providerUniqueId	providerUniqueId (consistent across all)
Feature Constant	FEATURE_WATERFALL_DATASOURCE_{ABBR}	FEATURE_WATERFALL_DATASOURCE_AP

When adding a new vendor, decide on:

1. Abbreviation (e.g., NV for "NewVendor") - use this EVERYWHERE. Check for collisions with existing abbreviations in sync-providers.config.ts and src/data/models/ before choosing.
2. Full name (e.g., "NewVendor") - for display purposes only

Cross-Repo Schema Contract

When implementing across multiple repos in parallel, the database column names defined in CRED Model are the source of truth. The Model API entity interfaces, repository queries, and cache keys must use identical field names to the database columns. Never invent provider-specific identifier names — always use providerUniqueId. If you are creating sub-issues for parallel execution, copy the exact column schema into every sub-issue that touches the data layer.

Cross-Reference Table:

Project	Repository	Where Abbreviation is Used
CRED Model	cred-model	Table name: NVPerson, column: providerUniqueId
DBT	cred-dbt	dataSourceAbbreviation: "NV", tableName: "NVPerson"
Model API	cred-model-api	ProviderDataSourceAbbreviation.NV, folder: newvendor/
Commercial	cred-api-commercial	CustomFieldRecordDataSourceEnum.NV, feature constant

---

1. CRED Model (Database Tables)

Repository: cred-model

Adding a new provider table (e.g., NVPerson, NVCompany) uses a single migration that handles everything: table creation, seeding, and replication setup.

Single Migration Pattern

File: src/data/migrations/{timestamp}-create-nv-person.ts

import * as Knex from "knex";
import addSetUpdatedAtTrigger from "./utils/add-set-updated-at-trigger";
import addToReplicationSet from "./utils/add-to-replication-set";
import addToStitchSync from "./utils/add-to-stitch-sync";
import seedIfEmpty from "./utils/seed-if-empty";

export async function up(knex: Knex): Promise<void> {
  // 1. Create table
  await knex.schema.createTable("NVPerson", (table) => {
    table.increments("id").notNullable().primary();
    table.string("providerUniqueId").notNullable().unique();
    table.integer("personId"); // or "companyId" for company tables
    table.foreign("personId").references("Person.id").onDelete("cascade");

    // Provider-specific columns...
    table.string("firstName");
    table.string("lastName");
    table.jsonb("someJsonField");

    // Standard timestamps
    table.dateTime("createdAt").notNullable().defaultTo(knex.raw("NOW()"));
    table.dateTime("updatedAt").notNullable().defaultTo(knex.raw("NOW()"));
    table.dateTime("deletedAt");

    // Recommended indexes
    table.index("personId"); // or "companyId" for company tables
    table.index("updatedAt");
    table.index("deletedAt");
  });
  await addSetUpdatedAtTrigger(knex, "NVPerson");

  // 2. Insert seed row with ALL columns populated
  await seedIfEmpty(
    knex,
    "NVPerson",
    {
      providerUniqueId: "seed-nv-person-001",
      personId: null,
      firstName: "Seed",
      lastName: "Record",
      someJsonField: { example: "data" },
      deletedAt: null,
    },
    "providerUniqueId",
    "seed-nv-person-001"
  );

  // 3. Add to pglogical replication
  await addToReplicationSet(knex, "NVPerson");

  // 4. Enable Stitch sync for BigQuery
  await addToStitchSync("NVPerson", "updatedAt");
}

export async function down(knex: Knex): Promise<void> {
  await knex.schema.dropTable("NVPerson");
}

What Each Step Does

Step	Utility	Purpose
1	addSetUpdatedAtTrigger()	Auto-updates updatedAt column on row changes
2	seedIfEmpty()	Inserts seed row if table is empty (idempotent). Seed row has ALL columns populated so DBT can detect the full schema.
3	addToReplicationSet()	Adds table to pglogical replication set with full replica synchronization (see details below). Skips in local dev where pglogical is not installed.
4	addToStitchSync()	Enables Stitch replication via API for BigQuery sync. Configures incremental sync using the specified replication key (default: updatedAt). Skips gracefully when Stitch credentials are not configured.

addToReplicationSet() Behavior

When addToReplicationSet(knex, "NewTable") is called, it performs the following:

1. Adds the table to pglogical replication set
2. Waits up to 5 minutes for the table to appear on all 3 replicas (dev, staging, prod)
3. Creates on each replica:
4. The id sequence
5. createdAt/updatedAt default values
6. table_update trigger
7. All indexes
8. Throws and blocks the deploy if any replica fails

Deploy Blocking

If replication to any environment fails, the migration will throw an error and block the deploy. This ensures data consistency across all environments.

Prerequisite: The table must be listed in PROVIDER_TABLES in sync-providers.config.ts. The ensureReplicaSequences function reads this list to process tables.

Seed Row Requirements

Requirement	Description
ALL columns populated	Required for DBT schema detection
Foreign keys = null	e.g., personId: null, companyId: null
Recognizable seed ID	e.g., "seed-nv-person-001"
deletedAt = null	If soft-delete column exists

Benefits

Benefit	Description
Single source of truth	Migration in CRED Model creates everything
No seeding in model-api	Data replicates via pglogical automatically
Guaranteed replica setup	Deploy fails if replicas aren't fully synchronized
Works locally	Replication step skips gracefully if pglogical not installed
Idempotent	Safe to run multiple times

Add Sync Provider Configuration

Required BEFORE running migration

The table must be added to PROVIDER_TABLES before deploying the migration. The addToReplicationSet() function uses ensureReplicaSequences which reads this list to set up replicas.

File: src/data/config/sync-providers.config.ts

export const PROVIDER_TABLES: ProviderTableConfig[] = [
  // ... existing providers ...

  {
    tableName: "NVPerson", // Must match migration table name
    primaryKey: "providerUniqueId",
    description: "NewVendor person data",
  },
];

Required Elements

Element	Required	Purpose
id (auto-increment)	✅	Internal primary key
providerUniqueId	✅	Unique ID from provider for sync
createdAt	✅	Timestamp
updatedAt	✅	Required for sync time filtering
deletedAt	✅	Soft delete support
addSetUpdatedAtTrigger()	✅	Auto-updates updatedAt on changes
seedIfEmpty()	✅	Seeds ALL columns for DBT detection
addToReplicationSet()	✅	Enables pglogical replication (waits for all replicas)
addToStitchSync()	✅	Enables Stitch sync for BigQuery
Sync config entry	✅	Prerequisite for addToReplicationSet()

Recommended Elements

Element	Notes
personId / companyId FK	Links to main Person or Company table
Index on personId/companyId	Foreign key lookups
Index on updatedAt	Required for efficient incremental sync
Index on deletedAt	Required for efficient soft-delete filtering
Index on email	Person tables — used for matching (if applicable)

What You Do NOT Need

Not Required	Reason
❌ TypeORM entity files	Tables use raw Knex queries
❌ Dedicated service files	Data accessed via sync mechanism
❌ Manual SQL sync code	System auto-discovers columns
❌ Seeding in model-api	Seed row replicates via pglogical automatically

Single Migration = Everything

One migration file handles table creation, seeding, and replication setup. Data automatically syncs to model-api via pglogical.

---

2. DBT (DataDescription)

Repository: cred-dbt

To expose a new enrichment source table in DataDescription, you need to add it to the provider tables configuration. The enrichment source tables are PostgreSQL tables stored in the credmodel_google schema.

Step 1: Ensure Source Table Exists

Prerequisite

The PostgreSQL table must exist in the credmodel_google schema and be defined in sources_datasets/credmodel_google.yml.

Examples of existing tables:

Table	Description
APPerson	Apollo Person
CGPerson	Cognism Person
RRPerson	RocketReach Person
LSPerson	Lusha Person

Step 2: Add Entry to Provider Tables List

File: models/credentity/data_description/intermadiate/data_description/data_description_providers_tables.sql

Add a new entry to the providerTables list:

{"tableName": "NVPerson", "parentDataName": "Person", "displayName": "NewVendor Person", "dataSourceAbbreviation": "NV"},

Parameters:

Parameter	Description	Example
tableName	Exact table name from CRED Model migration	NVPerson (must match exactly!)
parentDataName	Top-level entity: "Person" or "Company"	"Person"
displayName	Human-readable name	"NewVendor Person"
dataSourceAbbreviation	System abbreviation (same across projects)	"NV"

Example - Adding a new source:

{% set providerTables = [
    {"tableName": "APPerson", "parentDataName": "Person", "displayName": "Apollo Person", "dataSourceAbbreviation": "AP"},
    {"tableName": "RRPerson", "parentDataName": "Person", "displayName": "RocketReach Person", "dataSourceAbbreviation": "ROCKETREACH"},
    {"tableName": "CGPerson", "parentDataName": "Person", "displayName": "Cognism Person", "dataSourceAbbreviation": "COG"},
    {"tableName": "LSPerson", "parentDataName": "Person", "displayName": "Lusha Person", "dataSourceAbbreviation": "LUSHA"},
    {"tableName": "NVPerson", "parentDataName": "Person", "displayName": "NewVendor Person", "dataSourceAbbreviation": "NV"},  # ← Add here
] %}

Step 3: Verify Integration

After adding the entry, the provider table will automatically:

1. Generate an ID: Using to_hex(md5(concat('PROVIDER-TABLE-', tableName)))
2. Link to Parent Entity: Via data_description_cred_entity
3. Appear in DataDescription: Available for metadata queries

Step 4: Add Metadata (Optional)

If you want to add display metadata for the provider table, add an entry to the DataDescriptionMetadata seed table in BigQuery:

Location: cred-1556636033881.cred_seed.DataDescriptionMetadata

Add a row with:

• id: The generated ID (calculate: to_hex(md5(concat('PROVIDER-TABLE-', 'NVPerson'))))
• name: Display name (e.g., "NewVendor Person")
• description: Description of the table
• tooltip: Tooltip text

ID Generation

The ID is auto-generated from the table name. For NVPerson, the ID would be the MD5 hash of 'PROVIDER-TABLE-NVPerson'.

Testing

# Run the dbt model
dbt run --select data_description_providers_tables

# Verify the table appears in output
# Check it's linked to correct parent entity
# Verify it appears in DataDescription model

Current Provider Tables

Table Name	Parent Entity	Display Name	Abbreviation	Notes
APPerson	Person	Apollo Person	AP	Also has APCompany
RRPerson	Person	RocketReach Person	ROCKETREACH	Person only
CGPerson	Person	Cognism Person	COG	Person only
LSPerson	Person	Lusha Person	LUSHA	Person only

Pattern

Notice how abbreviations match: AP → APPerson table, CustomFieldRecordDataSourceEnum.AP, ProviderDataSourceAbbreviation.AP

---

3. Model API

Repository: cred-model-api

The Provider Enrichment system provides a unified interface for enriching entity data (Person, Company) from multiple external data providers.

Architecture Overview

┌────────────────────────────────────────────────────────────────────────────┐
│                      ProviderEnrichmentOrchestrator                        │
│                    (Entry point for all enrichment requests)               │
└─────────────────────────────────────┬──────────────────────────────────────┘
                                      │
                                      ▼
┌────────────────────────────────────────────────────────────────────────────┐
│                            ProviderFactory                                  │
│              createProvider(providerAbbreviation, entityType)              │
└─────────────────────────────────────┬──────────────────────────────────────┘
                                      │
                    ┌─────────────────┴─────────────────┐
                    ▼                                   ▼
           ┌────────────────┐                  ┌────────────────┐
           │  PERSON        │                  │  COMPANY       │
           │  Adapters      │                  │  Adapters      │
           └────────┬───────┘                  └────────┬───────┘
                    │                                   │
    ┌───────┬───────┼───────┬───────┐                   │
    ▼       ▼       ▼       ▼       ▼                   ▼
┌──────┐┌──────┐┌──────┐┌──────┐              ┌────────────────┐
│Apollo││Lusha ││Cognis││Rocket│              │ Apollo Company │
│Person││Person││Person││Reach │              │   Adapter      │
└──────┘└──────┘└──────┘└──────┘              └────────────────┘

Supported Providers & Entity Types

Provider	Abbreviation	Person	Company
Apollo	AP	✅	✅
Lusha	LUSHA	✅	❌
Cognism	COG	✅	❌
RocketReach	ROCKETREACH	✅	❌

Component Responsibilities

Component	Responsibility
ProviderEnrichmentOrchestrator	Entry point, routes requests to appropriate provider
ProviderFactory	Creates adapter based on providerAbbreviation + entityType
Enrichment Adapters	Implement IProviderEnrichmentService, adapt provider to unified interface
Default Services	Cache-first lookup, falls back to API, persists results
API Services	Pure HTTP calls to external APIs, response mapping
Repositories	Redis caching + PostgreSQL persistence

Data Flow

Request: enrichById(companyId=123, provider=AP, entityType=COMPANY)
                           │
                           ▼
             ┌─────────────────────────┐
             │ ProviderFactory         │
             │ → ApolloCompanyAdapter  │
             └───────────┬─────────────┘
                         │
                         ▼
             ┌─────────────────────────┐
             │ DefaultApolloCompany    │
             │ Service                 │
             └───────────┬─────────────┘
                         │
        ┌────────────────┴────────────────┐
        ▼                                 ▼
┌─────────────────┐               ┌─────────────────┐
│ Cache/DB Lookup │──── HIT ────▶│ Return cached   │
│ (Repository)    │               │ data            │
└────────┬────────┘               └─────────────────┘
         │ MISS
         ▼
┌─────────────────┐
│ ApiApolloCompany│
│ Service (HTTP)  │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│ Persist to DB   │
│ (Repository)    │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│ Return enriched │
│ data            │
└─────────────────┘

Files Structure

src/
├── domain/
│   ├── {vendor}/
│   │   ├── entity/
│   │   │   ├── {abbr}-person.ts
│   │   │   └── {abbr}-company.ts
│   │   ├── repository/
│   │   │   ├── {vendor}-person-repository.ts
│   │   │   └── {vendor}-company-repository.ts
│   │   └── service/
│   │       ├── {vendor}-person-service.ts
│   │       └── {vendor}-company-service.ts
│   └── provider-enrichment/
│       ├── entity/provider-types.ts
│       └── service/provider-enrichment-service.ts
├── data/repos/{vendor}/
│   ├── db-{vendor}-person-repository.ts
│   ├── cache-{vendor}-person-repository.ts
│   ├── db-{vendor}-company-repository.ts
│   └── cache-{vendor}-company-repository.ts
└── services/
    ├── {vendor}/
    │   ├── api-{vendor}-person-service.ts
    │   ├── default-{vendor}-person-service.ts
    │   ├── api-{vendor}-company-service.ts
    │   └── default-{vendor}-company-service.ts
    └── provider-enrichment/
        ├── provider-factory.ts
        ├── provider-enrichment-orchestrator.ts
        └── {vendor}-enrichment-adapter.ts

Step 1: Define the Domain Layer

Create the domain entities and service interface in src/domain/newvendor/.

a) Create Entity Types

Critical: Field names must match database columns

The entity interface field names must exactly match the column names from the CRED Model migration. The unique identifier column is always providerUniqueId — do NOT use provider-specific names like attioId, lushaPersonId, etc. Those are legacy patterns from older providers.

File: src/domain/newvendor/entity/nv-person.ts

import { EntityData } from "../../base/entity";

export interface NVPerson extends EntityData {
  providerUniqueId: string; // MUST match column name in CRED Model migration
  personId?: number | null;
  name?: string;
  // ... vendor-specific fields (must also match column names exactly)
}

b) Create Repository Interface

File: src/domain/newvendor/repository/newvendor-person-repository.ts

export default interface NewVendorPersonRepository {
  lookupPerson(linkedinUrl: string): Promise<NVPersonCreateData | null>;
  // ... other repository methods
}

c) Create Service Interface

File: src/domain/newvendor/service/newvendor-person-service.ts

export type NewVendorPersonInput = {
  personId?: number;
  linkedinUrl?: string;
  cacheOnly?: boolean;
  freshOnly?: boolean;
};

export type NewVendorPersonCreateData = {
  personData: Omit<NVPerson, "id" | "createdAt" | "updatedAt">;
  // ... nested data
};

export default interface NewVendorPersonService {
  lookupPerson(
    input: NewVendorPersonInput
  ): Promise<NewVendorPersonCreateData | null>;
}

---

Step 2: Implement the Services Layer

Create services in src/services/newvendor/.

a) Create API Service (external API calls)

File: src/services/newvendor/api-newvendor-person-service.ts

export default class ApiNewVendorPersonService
  extends RestService
  implements NewVendorPersonService
{
  constructor(private apiKey: string, private storage: CacheStorage) {}

  async lookupPerson(
    input: NewVendorPersonInput
  ): Promise<NewVendorPersonCreateData | null> {
    // Call external vendor API
    // Map response to internal data structure
  }
}

b) Create Default Service (orchestrates cache + API)

File: src/services/newvendor/default-newvendor-person-service.ts

export default class DefaultNewVendorPersonService
  implements NewVendorPersonService
{
  private static instance: DefaultNewVendorPersonService;

  static getInstance(
    personService: PersonService
  ): DefaultNewVendorPersonService {
    // Singleton pattern
  }

  async lookupPerson(
    input: NewVendorPersonInput
  ): Promise<NewVendorPersonCreateData | null> {
    // Check cache first, then call API, persist results
  }
}

---

Step 3: Create the Provider Enrichment Adapter

This is the key integration point for the unified waterfall logic.

File: src/services/provider-enrichment/newvendor-enrichment-adapter.ts

import { BaseProviderEnrichmentService } from "../../domain/provider-enrichment/service/provider-enrichment-service";
import {
  ProviderDataSourceAbbreviation,
  ProviderOptions,
} from "../../domain/provider-enrichment/entity/provider-types";

export class NewVendorEnrichmentAdapter extends BaseProviderEnrichmentService<
  NewVendorPersonCreateData,
  NVPerson
> {
  constructor(
    private readonly newvendorService: NewVendorPersonService,
    dataDescriptionService: DataDescriptionService,
    cacheStorage?: CacheStorage
  ) {
    super(
      ProviderDataSourceAbbreviation.NV, // Must match enum value
      dataDescriptionService,
      cacheStorage
    );
  }

  protected validateOptions(_options: ProviderOptions): string | null {
    // Return null if valid, error message if invalid
    return null;
  }

  protected async lookupPersonData(
    personId: number,
    options: ProviderOptions
  ): Promise<NewVendorPersonCreateData | null> {
    return this.newvendorService.lookupPerson({
      personId,
      freshOnly: options.freshOnly,
    });
  }

  protected extractPersonData(
    result: NewVendorPersonCreateData
  ): NVPerson | null {
    return result.personData as NVPerson;
  }

  protected getProviderName(): string {
    return "NewVendor";
  }
}

---

Step 4: Register the Provider in the Factory

a) Add to ProviderDataSourceAbbreviation enum

File: src/domain/provider-enrichment/entity/provider-types.ts

export enum ProviderDataSourceAbbreviation {
  AP = "AP",
  LUSHA = "LUSHA",
  COG = "COG",
  ROCKETREACH = "ROCKETREACH",
  NV = "NV", // ← Add new vendor (must match Commercial abbreviation)
}

b) Update ProviderFactory

File: src/services/provider-enrichment/provider-factory.ts

export class ProviderFactory {
  constructor(
    private readonly apolloService: ApolloPersonService,
    private readonly lushaService: LushaPersonService,
    private readonly cognismService: CognismPersonService,
    private readonly rocketreachService: RocketReachPersonService,
    private readonly newvendorService: NewVendorPersonService, // ← Add
    public readonly dataDescriptionService: DataDescriptionService,
    private readonly cacheStorage?: CacheStorage
  ) {}

  createProvider(
    providerAbbreviation: ProviderDataSourceAbbreviation
  ): IProviderEnrichmentService {
    switch (providerAbbreviation) {
      // ... existing cases
      case ProviderDataSourceAbbreviation.NV:
        return new NewVendorEnrichmentAdapter(
          this.newvendorService,
          this.dataDescriptionService,
          this.cacheStorage
        );
      default:
        throw new Error(`Unknown provider type: ${providerAbbreviation}`);
    }
  }
}

---

Step 5: Register in the Container Layer

a) Update types.ts

File: src/container/types.ts

export type ApiServicesInput = {
  // ... existing services
  newvendor?: NewVendorPersonService;
};

b) Update services.ts

File: src/container/services.ts

export type ApiServices = {
  // ... existing services
  newvendor: NewVendorPersonService;
};

const createServices = (...) => {
  // ... existing service creation
  const newvendor = input?.newvendor ?? DefaultNewVendorPersonService.getInstance(person);

  const providerFactory = new ProviderFactory(
    apollo,
    lusha,
    cognism,
    rocketreach,
    newvendor,  // ← Add new vendor
    dataDescription,
    cacheStorage
  );

  const services: ApiServices = {
    // ... existing services
    newvendor,
  };

  return services;
};

---

Step 6: Create GraphQL API Layer

Create the GraphQL types and resolver in src/graphql-api/newvendor/.

a) Create Types

File: src/graphql-api/newvendor/types/newvendor-person.ts

@ObjectType("NewVendorPerson")
export class TypeNewVendorPerson { ... }

b) Create Input Types

File: src/graphql-api/newvendor/input/newvendor-person-lookup.ts

@InputType("InputNewVendorPersonLookup")
export class InputNewVendorPersonLookup { ... }

c) Create Resolver

File: src/graphql-api/newvendor/resolvers/newvendor-resolver.ts

@Resolver()
export class NewVendorResolver {
  @Query(() => TypeNewVendorPerson, { nullable: true })
  async newVendorPersonLookup(...) { ... }
}

d) Register Resolver

File: src/graphql-api/schema-resolvers.ts

import { NewVendorResolver } from "./newvendor/resolvers/newvendor-resolver";

export default [
  // ... existing resolvers
  NewVendorResolver,
];

---

Step 7: Create Data Repository Layer

If the vendor data needs to be persisted, create repositories in src/data/repos/newvendor/.

File: src/data/repos/newvendor/db-newvendor-person-repository.ts

export default class DbNewVendorPersonRepository
  extends DbRepository<NVPerson>
  implements NewVendorPersonRepository { ... }

File: src/data/repos/newvendor/cache-newvendor-person-repository.ts

export default class CacheNewVendorPersonRepository
  extends CacheDbRepository
  implements NewVendorPersonRepository { ... }

---

Model API Summary

Step	Layer	Files
1	Domain	src/domain/newvendor/entity/, repository/, service/
2	Services	src/services/newvendor/api-*.ts, default-*.ts
3	Provider Adapter	src/services/provider-enrichment/newvendor-enrichment-adapter.ts
4	Factory	provider-types.ts, provider-factory.ts
5	Container	src/container/types.ts, services.ts
6	GraphQL	src/graphql-api/newvendor/, schema-resolvers.ts
7	Data	src/data/repos/newvendor/ (if persistence needed)

---

4. Commercial (Waterfall Integration)

Repository: cred-api-commercial

Prerequisite

The vendor must be integrated into the Model API first, exposing a providerEnrichment endpoint that returns standardized field values.

Step 1: Add Data Source Abbreviation Enum

File: src/domain/custom/entity/types.ts

Add the vendor abbreviation to CustomFieldRecordDataSourceEnum:

export enum CustomFieldRecordDataSourceEnum {
  // ... existing values
  /** Enrichment Providers */
  AP = "AP",
  LUSHA = "LUSHA",
  COG = "COG",
  ROCKETREACH = "ROCKETREACH",
  NV = "NV", // ← Add here (must match Model API abbreviation)
}

Step 2: Add Feature Constant

File: src/domain/feature/entity/feature.ts

Add a feature ID constant:

export const FEATURE_WATERFALL_DATASOURCE_NV = 99; // Use next available ID

Step 3: Add Feature Seed Entry

File: src/data/seeds/003-feature.ts

Add the feature configuration:

{
  id: 99,  // Must match the constant from Step 2
  name: "Waterfall datasource (NewVendor)",
  description: "NewVendor as data source for waterfall processing",
  transactionType: FeatureTransactionType.CREDIT,
  amount: 1,  // Credit cost per enrichment
  parentId: 24,  // FEATURE_CUSTOM_FIELD_WATERFALL
  dataSourceAbbreviation: "NV",  // Must match enum abbreviation
},

Step 4: Register in DataSource Enrichment Provider

File: src/domain/enrichment/provider/datasource-enrichment-provider.ts

Add the vendor abbreviation to the supported providers list:

const PROVIDER_ENRICHMENT_ABBREVIATIONS: CustomFieldRecordDataSourceEnum[] = [
  CustomFieldRecordDataSourceEnum.AP,
  CustomFieldRecordDataSourceEnum.LUSHA,
  CustomFieldRecordDataSourceEnum.COG,
  CustomFieldRecordDataSourceEnum.ROCKETREACH,
  CustomFieldRecordDataSourceEnum.NV, // ← Add here
];

Step 5: Add to Vendor Resilience Config

File: src/domain/enrichment/service/vendor-resilience-config.ts

Add the vendor abbreviation to the EXTERNAL_VENDORS array. This enables circuit breaker and rate limiting protection for the new vendor.

const EXTERNAL_VENDORS = [
  CustomFieldRecordDataSourceEnum.AP,
  CustomFieldRecordDataSourceEnum.LUSHA,
  // ... existing vendors
  CustomFieldRecordDataSourceEnum.NV, // ← Add here
] as const;

Optionally, add custom rate limit or circuit breaker settings in VENDOR_CONFIGS if the vendor has specific API limits:

const VENDOR_CONFIGS: Partial<
  Record<VendorProvider, Partial<VendorResilienceConfig>>
> = {
  // ... existing configs
  [CustomFieldRecordDataSourceEnum.NV]: {
    maxRequestsPerMinute: 60,
    maxRequestsPerDay: 10_000,
  },
};

Step 6: Add to Validation

File: src/domain/custom/usecase/field/create-or-update-many-custom-field-data-source.ts

Import the new feature constant and add it to the CRED_API_FEATURES array:

import {
  // ... existing imports
  FEATURE_WATERFALL_DATASOURCE_NV,
} from "../../../feature/entity/feature";

const CRED_API_FEATURES = [
  FEATURE_WATERFALL_DATASOURCE_CRED,
  FEATURE_WATERFALL_DATASOURCE_AP,
  // ... existing features
  FEATURE_WATERFALL_DATASOURCE_NV, // ← Add here
];

Step 7: Add Default Configurations (Optional)

File: src/data/seeds/005-custom-field-waterfall-config.ts

If the vendor should be a default source for certain fields in new workspaces:

{
  templateName: "email",
  entityType: EntityTypeEnum.CONTACT,
  priority: 3,
  featureId: FEATURE_WATERFALL_DATASOURCE_NV,
  isEnabled: false,  // Disabled by default
  credDataDescriptionId: "xxx",  // Get from DBT DataDescription
  dataDescriptionEntityType: EntityTypeEnum.PERSON,
},

Commercial Summary

Step	File	Change
1	types.ts	Add to CustomFieldRecordDataSourceEnum
2	feature.ts	Add FEATURE_WATERFALL_DATASOURCE_* constant
3	003-feature.ts	Add feature seed entry
4	datasource-enrichment-provider.ts	Add to PROVIDER_ENRICHMENT_ABBREVIATIONS
5	vendor-resilience-config.ts	Add to EXTERNAL_VENDORS (+ optional config)
6	create-or-update-many-custom-field-data-source.ts	Add to CRED_API_FEATURES array
7	005-custom-field-waterfall-config.ts	Add default configs (optional)

---

5. Adding BYOAPI (BYOK) Support for a New Provider

Once a provider is integrated via the steps above, you can enable Bring Your Own API Key (BYOAPI/BYOK) support so customers can use their own vendor credentials. This requires changes in Model API and Commercial.

Model API (cred-model-api) — 3 Steps

Step 1: Extend Provider Input Type

Ensure the provider's input type extends BaseProviderInput from provider-enrichment/entity/provider-input.ts. This gives it apiKey and freshOnly for free.

File: src/domain/provider-enrichment/entity/provider-input.ts

// BaseProviderInput provides apiKey and freshOnly
export interface BaseProviderInput {
  apiKey?: string;
  freshOnly?: boolean;
}

File: src/domain/newvendor/service/newvendor-person-service.ts

import { BaseProviderInput } from "../../provider-enrichment/entity/provider-input";

export type NewVendorPersonInput = BaseProviderInput & {
  personId?: number;
  linkedinUrl?: string;
  cacheOnly?: boolean;
};

Step 2: Update Concrete API Service

In the provider's API service, update the HTTP request method(s) to prefer the customer-supplied key over the platform default.

File: src/services/newvendor/api-newvendor-person-service.ts

export default class ApiNewVendorPersonService
  extends RestService
  implements NewVendorPersonService
{
  constructor(private apiKey: string, private storage: CacheStorage) {}

  async lookupPerson(
    input: NewVendorPersonInput
  ): Promise<NewVendorPersonCreateData | null> {
    const authKey = input.apiKey ?? this.apiKey; // ← Customer key takes priority
    // Use authKey when setting the authorization header/param
  }
}

Step 3: Add to BYOK Providers Set

Add the new provider abbreviation to the BYOK_PROVIDERS set so the platform knows this provider supports customer-provided API keys.

File: src/services/provider-keys/provider-key-resolver.ts

export const BYOK_PROVIDERS: ReadonlySet<string> = new Set<string>([
  ProviderDataSourceAbbreviation.ROCKETREACH,
  ProviderDataSourceAbbreviation.AP,
  ProviderDataSourceAbbreviation.LUSHA,
  ProviderDataSourceAbbreviation.COG,
  ProviderDataSourceAbbreviation.SR,
  ProviderDataSourceAbbreviation.HG,
  ProviderDataSourceAbbreviation.PDL,
  ProviderDataSourceAbbreviation.NV, // ← Add new vendor
]);

Commercial (cred-api-commercial) — 2 Steps

Step 1: Register BYOK Provider

Add the abbreviation to the ENRICHMENT_PROVIDERS array. BYOK_PROVIDERS is composed from ENRICHMENT_PROVIDERS, LLM_PROVIDERS, and REPORTING_PROVIDERS — so you add to ENRICHMENT_PROVIDERS, not BYOK_PROVIDERS directly. The secret type used in the GraphQL API and storage is the abbreviation itself (e.g., "NV"), not a value like NV_API_KEY.

File: src/domain/user-secret/entity/user-secret.ts

export const ENRICHMENT_PROVIDERS = [
  "ROCKETREACH",
  "AP",
  "LUSHA",
  "COG",
  // ... existing providers
  "NV", // ← Add new vendor abbreviation
] as const;

// BYOK_PROVIDERS is auto-composed — no changes needed here
export const BYOK_PROVIDERS = [
  ...ENRICHMENT_PROVIDERS,
  ...LLM_PROVIDERS,
  ...REPORTING_PROVIDERS,
] as const;

Secret Naming Convention

Secrets are stored in Secret Manager with the format ${abbreviation}-${companyId} (e.g., NV-450931).

Step 2: Add to Billing Map

Add entries linking the provider's direct enrichment feature IDs to the secret type (abbreviation). This enables automatic credit reduction when a customer uses their own key.

File: src/domain/user-secret/entity/initialize-feature-secret-map.ts

export function initializeFeatureSecretMap(): void {
  // ... existing mappings

  // NewVendor
  FEATURE_TO_SECRET_TYPE[FEATURE_NV_EMAIL_ENRICHMENT] = "NV";
  FEATURE_TO_SECRET_TYPE[FEATURE_NV_PHONE_ENRICHMENT] = "NV"; // if applicable
}

Use Direct Enrichment Features

Map direct enrichment features (e.g., FEATURE_NV_EMAIL_ENRICHMENT), not waterfall datasource features. The secret type is the abbreviation string (e.g., "NV"), not NV_API_KEY.

BYOAPI Summary

Step	Project	File	Change
1	Model API	provider-input.ts	Extend BaseProviderInput in provider's input type
2	Model API	api-{vendor}-person-service.ts	Use input.apiKey ?? this.apiKey for auth
3	Model API	provider-key-resolver.ts	Add abbreviation to BYOK_PROVIDERS Set
4	Commercial	user-secret.ts	Add abbreviation to ENRICHMENT_PROVIDERS array
5	Commercial	initialize-feature-secret-map.ts	Map enrichment feature IDs to abbreviation

---

6. BYOK Credential Validation Endpoint

Validates a customer-provided API key against the vendor's API before persisting it to Secret Manager, so invalid or expired keys are rejected at save time.

Reference Issue

See COM-31691 for implementation details.

Architecture

┌─────────────────────────────────────────────────────────────────────────────┐
│                              Frontend                                        │
│                         "Test Key" Button                                    │
└─────────────────────────────────┬───────────────────────────────────────────┘
                                  │
                                  ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│                        CREDCommercial                                        │
│  • validateVendorApiKey query (for frontend)                                 │
│  • addUserSecret mutation (validates before storing)                         │
│  └─── Both delegate to ModelService.validateProviderApiKey()                 │
└─────────────────────────────────┬───────────────────────────────────────────┘
                                  │
                                  ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│                         CREDModelApi                                         │
│  validateProviderApiKey(provider, apiKey) GraphQL query                      │
│  └─── Returns { valid: boolean, message: string, provider: string }          │
└─────────────────────────────────┬───────────────────────────────────────────┘
                                  │
                                  ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│                      Vendor API (Ping Call)                                  │
│  Lightweight validation endpoint per provider                                │
└─────────────────────────────────────────────────────────────────────────────┘

Supported Providers

Provider	Abbreviation	Validation Call	Auth Method
RocketReach	ROCKETREACH	GET /api/v2/account	Api-Key header
Apollo	AP	GET /v1/auth/health	X-Api-Key header
Lusha	LUSHA	GET /account	api_key header
Cognism	COG	POST /search/contact/enrich (empty body)	Authorization: Bearer
Semrush	SR	GET /users/countapiunits.html?key=...	Query param
HG Insights	HG	GET /data-api/v1/company/match?name=test	Authorization: Bearer
PeopleDataLabs	PDL	GET /v5/person/enrich?profile=...	X-Api-Key header

Adding Validation for a New Provider

Model API (cred-model-api) — 2 Steps

Step 1: Create Validator Function

Create a new validator function following existing patterns. Validators are implemented as functions, not classes.

File: src/services/provider-keys/validators/{provider}-validator.ts

import { ValidationResult } from "../provider-key-validator";

export async function validateNewVendor(
  apiKey: string
): Promise<ValidationResult> {
  try {
    const response = await fetch("https://api.newvendor.com/health", {
      method: "GET",
      headers: {
        "X-Api-Key": apiKey,
      },
    });

    if (response.ok) {
      return { valid: true, message: "API key is valid" };
    }

    return {
      valid: false,
      message: `Invalid API key: ${response.status}`,
    };
  } catch (error) {
    return {
      valid: false,
      message: `Validation failed: ${error.message}`,
    };
  }
}

Step 2: Register Validator

Add the validator function to the registry map, keyed by the provider abbreviation (not *_API_KEY).

File: src/services/provider-keys/provider-key-validator.ts

import { validateNewVendor } from "./validators/newvendor-validator";

const PROVIDER_VALIDATORS: Readonly<Record<string, ProviderValidator>> = {
  [ProviderDataSourceAbbreviation.ROCKETREACH]: validateRocketReach,
  [ProviderDataSourceAbbreviation.AP]: validateApollo,
  [ProviderDataSourceAbbreviation.LUSHA]: validateLusha,
  [ProviderDataSourceAbbreviation.COG]: validateCognism,
  [ProviderDataSourceAbbreviation.SR]: validateSemrush,
  [ProviderDataSourceAbbreviation.HG]: validateHGInsights,
  [ProviderDataSourceAbbreviation.PDL]: validatePDL,
  [ProviderDataSourceAbbreviation.NV]: validateNewVendor, // ← Add new vendor
};

Commercial (cred-api-commercial) — No Additional Steps

No Changes Needed

Once the abbreviation is in BYOK_PROVIDERS (from Section 5) and there is a corresponding validator in Model API, validation is automatically supported. The secret type used is the abbreviation itself (e.g., "NV").

Validation Summary

Step	Project	File	Change
1	Model API	validators/{provider}-validator.ts	Create validator function
2	Model API	provider-key-validator.ts	Add to PROVIDER_VALIDATORS keyed by abbreviation
—	Commercial	—	No changes (uses BYOK_PROVIDERS from Section 5)

---

Cross-Project Verification

Before testing, verify naming consistency across all projects:

Check	CRED Model (cred-model)	DBT (cred-dbt)	Model API (cred-model-api)	Commercial (cred-api-commercial)
Abbreviation	Table: NVPerson	dataSourceAbbreviation: "NV"	ProviderDataSourceAbbreviation.NV	CustomFieldRecordDataSourceEnum.NV
Table Name	NVPerson in migration	tableName: "NVPerson"	Entity: NVPerson	N/A
Provider ID	Column: providerUniqueId	N/A	Entity field: providerUniqueId	N/A

!!! danger "Common Mistakes" - Using different abbreviations across projects (e.g., NV in Model API but NEW_VENDOR in Commercial) - Mismatched table names between CRED Model migration and DBT config - Forgetting to add the sync config entry in CRED Model - Not adding the provider to ProviderFactory switch statement - Using provider-specific identifier names (e.g., attioId, closeId) instead of the standard providerUniqueId column. The Model API entity field name must match the database column name from the CRED Model migration. All new providers use providerUniqueId — never invent a provider-specific name. - Abbreviation collisions with existing providers. Always check sync-providers.config.ts, src/data/models/, and cred-dbt models before choosing an abbreviation. Known legacy abbreviations: PD (PeopleData), PB (PitchBook), CL (Clearbit), AT (Apptopia), AL (AngelList).

---

Checklist

Use this checklist when adding a new vendor:

• CRED Model: Database migration and sync config created
• DBT: Provider table added to DataDescription
• Model API: Full integration (domain, services, adapter, factory, GraphQL)
• Commercial: Waterfall integration (enum, feature, seed, provider list, resilience config, validation)
• BYOAPI (optional): Input type extends BaseProviderInput, API service uses input.apiKey ?? this.apiKey, abbreviation added to BYOK_PROVIDERS (Model API) and ENRICHMENT_PROVIDERS (Commercial), billing map updated
• BYOK Validation (optional): Create validator function in Model API, register in PROVIDER_VALIDATORS keyed by abbreviation
• Naming Consistency: Same abbreviation used in ALL projects
• Testing: End-to-end tested in development
• Documentation: Vendor added to relevant docs