Priority Pipeline
Monthly LinkedIn profile scraping pipeline using BrightData for priority profiles (executives, directors, key decision-makers).
Documentation
- API Reference - API endpoints and request/response formats
- Architecture - How the system works
- Development - Local setup and development workflow
- Deployment - Deployment process and environments
Overview
This pipeline scrapes priority LinkedIn profiles on a monthly schedule using a webhook-based architecture for reliable data delivery.
Quick Links
| Component | URL/Resource |
|---|---|
| API Service | priority-pipeline-api (Cloud Run) |
| Webhook Service | priority-pipeline-webhook (Cloud Run) |
| Coordinator Job | priority-pipeline-coordinator (Cloud Run Job) |
| BigQuery Table | linkedin.LinkedinApiCall |
| GCS Backup Bucket | brightdata-monthly-priority-people |