Skip to content

CRED Agent AI - Complete Technical Documentation

Table of Contents

  1. Project Overview
  2. System Architecture
  3. Core Modules
  4. Data Ingestion Controllers
  5. LLM Processing System
  6. File Processing & Import System
  7. Communication Platform Integrations
  8. Database & Data Management
  9. Authentication & Security
  10. API Endpoints
  11. Deployment & Infrastructure
  12. Configuration & Environment

Project Overview

CRED Agent AI is a sophisticated NestJS-based API system designed to process and analyze messages from various communication platforms using advanced AI capabilities. The system provides a unified interface for handling text, files, and images with intelligent processing powered by Anthropic's Claude AI and OpenAI's Whisper for audio transcription.

Key Features

  • Multi-platform Integration: Seamless connectivity with Slack and Microsoft Teams
  • Advanced AI Processing: Leverages Claude AI for intelligent message understanding and response generation
  • File Processing: Comprehensive support for multiple file formats (PDF, CSV, Excel, JSON, HTML, images, audio)
  • Audio Transcription: OpenAI Whisper integration for voice message processing
  • File Import System: Automated file format conversion with template-based data extraction
  • Conversation Management: Persistent conversation history with thread support
  • MCP Integration: Model Context Protocol for enhanced AI capabilities
  • GraphQL API: Modern API interface for web applications
  • Callback System: Asynchronous response handling for long-running operations

System Architecture

The system follows a modular, microservices-oriented architecture with clear separation of concerns:

┌─────────────────────────────────────────────────────────────────┐
│                        CRED Agent AI                            │
├─────────────────────────────────────────────────────────────────┤
│  ┌─────────────────┐  ┌─────────────────┐  ┌─────────────────┐  │
│  │   Slack Module  │  │   Teams Module  │  │  GraphQL API    │  │
│  │   (Controller)  │  │   (Controller)  │  │   (Controller)  │  │
│  └─────────┬───────┘  └─────────┬───────┘  └─────────┬───────┘  │
│            │                    │                    │          │
│            └────────────────────┼────────────────────┘          │
│                                 │                               │
│  ┌─────────────────────────────────────────────────────────────┐ │
│  │              Message Processing Layer                       │ │
│  │  ┌─────────────────┐  ┌─────────────────┐  ┌─────────────┐ │ │
│  │  │ Messages Service│  │ LLM Processor   │  │ File        │ │ │
│  │  │                 │  │ Service         │  │ Processor   │ │ │
│  │  └─────────┬───────┘  └─────────┬───────┘  └─────────────┘ │ │
│  └────────────┼─────────────────────┼───────────────────────────┘ │
│               │                     │                           │
│  ┌────────────▼─────────────────────▼───────────────────────────┐ │
│  │              AI & Processing Layer                           │ │
│  │  ┌─────────────────┐  ┌─────────────────┐  ┌─────────────┐ │ │
│  │  │ Claude          │  │ MCP Connector   │  │ OpenAI      │ │ │
│  │  │ Connector       │  │                 │  │ Connector   │ │ │
│  │  └─────────────────┘  └─────────────────┘  └─────────────┘ │ │
│  └─────────────────────────────────────────────────────────────┘ │
│                                                                 │
│  ┌─────────────────────────────────────────────────────────────┐ │
│  │              Data & Storage Layer                           │ │
│  │  ┌─────────────────┐  ┌─────────────────┐  ┌─────────────┐ │ │
│  │  │ PostgreSQL      │  │ Google Cloud    │  │ File        │ │ │
│  │  │ Database        │  │ Storage         │  │ Import      │ │ │
│  │  └─────────────────┘  └─────────────────┘  └─────────────┘ │ │
│  └─────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘

Core Design Principles

  1. Modularity: Each platform integration and processing capability is isolated in its own module
  2. Scalability: Stateless design with external storage for conversation persistence
  3. Extensibility: Easy addition of new communication platforms and file processors
  4. Reliability: Comprehensive error handling and retry mechanisms
  5. Security: JWT-based authentication and secure token management

Core Modules

1. Messages Module (src/messages/)

Purpose: Central message processing orchestrator that coordinates between different platforms and AI services.

Key Components:

  • MessagesService: Main orchestrator for message processing
  • TextProcessor: Handles text content preprocessing
  • DatabaseConversationHistoryAdapter: Manages conversation persistence

Key Features:

  • Debug mode support ([debug], [info] flags)
  • Conversation history management
  • Message threading support
  • Token usage tracking

2. LLM Processor Module (src/llm-processor/)

Purpose: Handles all AI-related processing including text generation, file analysis, and audio transcription.

Key Components:

  • LLMProcessorService: Main LLM processing coordinator with retry logic
  • FileProcessorService: Processes various file types for AI consumption
  • CSVProcessorService: Specialized CSV file processing using LangChain
  • ExcelProcessorService: Excel file processing capabilities
  • JSONProcessorService: JSON file processing
  • HTMLProcessorService: HTML content processing
  • AudioProcessorService: Audio transcription using OpenAI Whisper

Key Features:

  • Automatic retry with exponential backoff for API overloads
  • Support for multiple file formats
  • Streaming response handling
  • Token usage optimization

3. Claude Connector (src/connectors/claude/)

Purpose: Direct integration with Anthropic's Claude AI for advanced text processing and generation.

Key Features:

  • Claude Sonnet 4 model integration
  • Tool calling capabilities with MCP integration
  • Streaming response processing
  • Debug mode with detailed request/response logging
  • System prompt customization per platform

4. MCP Connector (src/connectors/mcp/)

Purpose: Model Context Protocol integration for enhanced AI capabilities and external tool access.

Key Features:

  • Multi-server MCP client support
  • JWT-based authentication
  • Tool discovery and execution
  • HTTP transport with custom headers

Data Ingestion Controllers

1. Slack Controller (src/slack/slack.controller.ts)

Endpoints:

  • GET /slack/install - OAuth installation flow initiation
  • GET /slack/oauth/callback - OAuth callback handler
  • POST /slack/message - Webhook for incoming Slack messages

Key Features:

  • OAuth 2.0 flow management
  • Message event processing
  • Thread support for conversations
  • File attachment handling
  • Audio transcription integration
  • User validation and system integration

Message Flow:

  1. Receives Slack webhook events
  2. Validates user against system database
  3. Processes files and audio attachments
  4. Sends to Messages Service for AI processing
  5. Returns responses in appropriate Slack format

2. Teams Controller (src/teams/teams.controller.ts)

Endpoints:

  • POST /teams/message - Webhook for incoming Teams messages

Key Features:

  • Microsoft Bot Framework integration
  • Azure AD user authentication
  • Audio message transcription
  • File attachment processing
  • Message fragmentation handling

Message Flow:

  1. Receives Teams bot framework messages
  2. Extracts user information via Microsoft Graph API
  3. Processes audio and file attachments
  4. Accumulates responses to prevent fragmentation
  5. Sends complete response back to Teams

3. File Importer Controller (src/file-importer/file-importer.controller.ts)

Endpoints:

  • POST /file-importer - File upload and conversion endpoint

Key Features:

  • JWT authentication required
  • Multi-format file support
  • Template-based data extraction
  • Download or content response options
  • Import ID tracking for async processing

Supported Templates:

  • COMPANY_OPPORTUNITY: Company/account data extraction
  • CONTACT: Contact/person data extraction
  • PRODUCT: Commercial product data extraction
  • CUSTOMER: Customer data extraction

4. Health Controller (src/health/health.controller.ts)

Endpoints:

  • GET / - Health check endpoint

Purpose: Simple health monitoring for load balancers and monitoring systems.

5. Callback Controller (src/callback/callback.controller.ts)

Endpoints:

  • POST /callback/:token - Asynchronous callback handler

Key Features:

  • JWT token validation
  • Database record retrieval
  • Platform-specific response routing
  • Error handling and logging

LLM Processing System

Claude Integration

The system uses Anthropic's Claude Sonnet 4 model as the primary AI engine:

Configuration:

  • Model: claude-sonnet-4-20250514
  • Temperature: 0.1 (for consistent responses)
  • Max Tokens: 4096
  • Streaming: Enabled

System Prompts:

  • Slack: Optimized for Slack communication style
  • Teams: Optimized for Microsoft Teams environment
  • Default: General-purpose AI assistant

Tool Integration:

  • MCP tools for external data access
  • Web search capabilities
  • File processing tools
  • Custom business logic tools

Message Processing Flow

  1. Input Validation: Validates message format and user permissions
  2. Content Processing: Processes text, files, and audio content
  3. Context Building: Retrieves conversation history and user context
  4. AI Processing: Sends to Claude with appropriate system prompts
  5. Response Generation: Streams response back to user
  6. Persistence: Saves conversation and usage data

Debug Modes

  • [debug]: Full request/response logging with tool call details
  • [info]: Basic tool call notifications
  • Default: Standard processing without debug output

File Processing & Import System

File Type Support

Text Files:

  • PDF: Native Claude processing
  • TXT: Direct text processing
  • HTML: Structured content extraction

Structured Data:

  • CSV: LangChain CSVLoader with column support
  • Excel: XLSX/XLS processing with sheet support
  • JSON: Structured data processing

Media Files:

  • Images: JPG, PNG, GIF, WebP, BMP, TIFF
  • Audio: MP3, WAV, M4A, OPUS (with transcription)

Documents:

  • DOC/DOCX: Microsoft Word documents
  • PPT/PPTX: PowerPoint presentations

File Import Workflow

  1. Upload: File uploaded via multipart form data
  2. Validation: File type and size validation
  3. Storage: Upload to Google Cloud Storage
  4. Processing: AI-powered format conversion
  5. Template Application: Structured data extraction using templates
  6. Output Generation: Convert to requested format
  7. Storage: Save processed file
  8. Registration: Register with commercial API for import
  9. Response: Return content or download URL

Template System

Templates provide structured field mappings for improved data extraction:

interface Template {
  name: string;
  entityType: string;
  description: string;
  requiredFields: string[];
  fields: string[];
}

Communication Platform Integrations

Slack Integration

Authentication:

  • OAuth 2.0 flow with workspace installation
  • Bot token management with caching
  • User email mapping to system users

Features:

  • Real-time message processing
  • Thread conversation support
  • File attachment handling
  • Audio message transcription
  • Shared content processing
  • User validation and error messaging

OAuth Scopes:

  • chat:write: Send messages
  • users:read: Access user information
  • users:read.email: Access user emails
  • channels:read: Access channel information
  • im:read, im:write: Direct message support
  • files:read: Access file attachments

Microsoft Teams Integration

Authentication:

  • Microsoft Bot Framework integration
  • Azure AD application registration
  • Graph API access for user information

Features:

  • Bot framework message handling
  • Audio message transcription
  • File attachment processing
  • Message fragmentation prevention
  • Tenant-specific user lookup

API Integration:

  • Microsoft Graph API for user data
  • Bot Connector Service for messaging
  • Azure AD for authentication

Database & Data Management

Database Schema

Core Tables:

  1. LLMConversation: Conversation metadata

  2. id, userId, userCompanyId, title, entityType

  3. sessionId, conversationSource, createdAt, updatedAt

  4. LLMConversationMessage: Individual messages

  5. id, llmConversationId, externalMessageId, parentMessageId

  6. userPrompt, structuredUserPrompt, llmParsedOutput

  7. llmTaskId, entityType, entityId, language

  8. LLMTask: AI processing tasks

  9. id, status, llmSettingsId, llmRawPrompt, llmStructuredPrompt

  10. llmRawOutput, llmStructuredOutput, totalInputTokens

  11. totalOutputTokens, totalTokens, errorMessage

  12. LLMAgentImport: File import tracking

  13. id, userId, userCompanyId, inputFileName, inputFileUrl

  14. outputFormat, outputFileUrl, outputContent, processingStatus

  15. errorMessage, commercialImportId

  16. LLMAgentInstallation: Platform installations

  17. id, companyId, platform, enabled, settings

  18. createdAt, updatedAt

  19. User: System users

  20. id, email, name, companyId, role

Data Flow

  1. Message Reception: Platform-specific webhooks receive messages
  2. User Validation: Verify user exists in system database
  3. Conversation Management: Find or create conversation records
  4. Message Processing: Process content through AI pipeline
  5. Response Generation: Generate and send responses
  6. Persistence: Save conversation history and usage metrics

Authentication & Security

JWT Authentication

Token Structure:

interface JWTPayload {
  userId: number;
  email: string;
  companyId: number;
  role: string;
  iat: number;
  exp: number;
}

Usage:

  • API endpoint protection
  • MCP server authentication
  • Commercial API integration
  • Callback token generation

Security Measures

  1. Request Validation: All inputs validated using class-validator
  2. File Validation: File type and size restrictions
  3. User Authorization: Role-based access control
  4. Token Management: Secure JWT generation and validation
  5. Environment Security: Sensitive data in environment variables

Platform Security

Slack:

  • Signing secret validation
  • OAuth token management
  • User email verification

Teams:

  • Bot framework authentication
  • Azure AD integration
  • Tenant isolation

API Endpoints

Public Endpoints

  • GET / - Health check
  • GET /slack/install - Slack OAuth initiation
  • GET /slack/oauth/callback - Slack OAuth callback

Webhook Endpoints

  • POST /slack/message - Slack message webhook
  • POST /teams/message - Teams message webhook
  • POST /callback/:token - Async callback handler

Authenticated Endpoints

  • POST /file-importer - File upload and processing (JWT required)

GraphQL Endpoint

  • POST /graphql - GraphQL API for web applications

Deployment & Infrastructure

Container Deployment

Docker Configuration:

  • Multi-stage build for optimization
  • Node.js 18+ runtime
  • Production-ready configuration
  • Health check integration

Google Cloud Run:

  • Serverless deployment
  • Automatic scaling
  • Environment variable configuration
  • Public endpoint access

CI/CD Pipeline

GitHub Actions:

  • Automated testing
  • Docker image building
  • Google Cloud deployment
  • Environment-specific deployments

Deployment Strategy:

  • develop branch → Development environment
  • main branch → Production environment
  • Automated releases with commit SHA tagging

Monitoring & Logging

Logging:

  • Structured logging with Winston
  • Request/response tracking
  • Error monitoring
  • Performance metrics

Health Monitoring:

  • Health check endpoint
  • Database connectivity monitoring
  • External service status tracking

Configuration & Environment

Required Environment Variables

Application:

PORT=8080
ENV=development

Database:

DATABASE_URL=postgresql://user:password@host:port/database
PGSSLMODE=require

AI Services:

ANTHROPIC_API_KEY=your-anthropic-key
OPENAI_API_KEY=your-openai-key

Slack Integration:

SLACK_CLIENT_ID=your-client-id
SLACK_CLIENT_SECRET=your-client-secret
SLACK_REDIRECT_URI=https://your-domain.com/slack/oauth/callback
SLACK_SCOPES=chat:write,users:read,users:read.email,channels:read,im:read,im:write,files:read

Teams Integration:

TEAMS_APP_ID=your-app-id
TEAMS_APP_SECRET=your-app-secret

MCP Integration:

MCP_SERVER_URL=https://your-mcp-server.com

Authentication:

JWT_SECRET=your-jwt-secret

Storage:

GOOGLE_CLOUD_STORAGE_BUCKET=your-bucket-name
GOOGLE_APPLICATION_CREDENTIALS=path/to/service-account.json

Commercial API:

COMMERCIAL_API_URL=https://commercial-api-dev.credplatform.com/graphql

Development Setup

  1. Prerequisites:

  2. Node.js 18+

  3. PostgreSQL 13+
  4. Google Cloud account
  5. Slack workspace

  6. Microsoft Teams tenant

  7. Installation:

git clone <repository>
cd cred-agent-ai
npm install
cp .env.example .env
# Configure environment variables
npm run start:dev

  1. Database Setup:

  2. Run migrations

  3. Configure connection string

  4. Set up SSL if required

  5. Platform Configuration:

  6. Configure Slack app in developer portal
  7. Set up Teams bot in Azure
  8. Configure OAuth redirects

Technical Specifications

Performance Characteristics

  • Response Time: < 2 seconds for text processing
  • File Processing: Variable based on file size and type
  • Concurrent Users: Supports multiple simultaneous conversations
  • Token Usage: Optimized for cost efficiency
  • Storage: Efficient file storage with automatic cleanup

Scalability Considerations

  • Stateless Design: No server-side session storage
  • Database Connection Pooling: Optimized connection management
  • Caching: Redis-like caching for frequently accessed data
  • Load Balancing: Ready for horizontal scaling
  • Resource Management: Automatic cleanup of temporary files

Error Handling

  • Retry Logic: Exponential backoff for API failures
  • Graceful Degradation: Fallback mechanisms for service failures
  • User Feedback: Clear error messages for users
  • Logging: Comprehensive error tracking and monitoring
  • Recovery: Automatic recovery from transient failures

Future Enhancements

Planned Features

  1. Additional Platforms: Discord, WhatsApp integration
  2. Advanced AI Models: Support for multiple LLM providers
  3. Real-time Collaboration: WebSocket support for live updates
  4. Advanced Analytics: Usage metrics and conversation analytics
  5. Custom Workflows: User-defined processing pipelines
  6. Multi-language Support: Internationalization and localization

Technical Improvements

  1. Performance Optimization: Response time improvements
  2. Security Enhancements: Advanced threat detection
  3. Monitoring: Enhanced observability and alerting
  4. Testing: Comprehensive test coverage
  5. Documentation: API documentation and developer guides

This documentation provides a comprehensive overview of the CRED Agent AI system, covering all major components, integrations, and technical details. The system is designed to be robust, scalable, and maintainable while providing powerful AI-driven communication capabilities across multiple platforms.