feat: expand sales opportunity workflow and metrics APIs

This commit is contained in:
2026-03-15 23:38:56 -05:00
parent 33b34d08a7
commit e764932c39
55 changed files with 3425 additions and 157 deletions
+36
View File
@@ -8,6 +8,8 @@ This document describes the caching layer used in the Optima API, covering the R
The API caches expensive ConnectWise (CW) API responses in **Redis** to reduce latency and avoid CW rate limits. The primary cache layer is the **opportunity cache** (`src/modules/cache/opportunityCache.ts`), which proactively warms data for all non-closed opportunities on a background interval.
The API also maintains a Redis-backed **sales member metrics cache** (`src/modules/cache/salesOpportunityMetricsCache.ts`) refreshed every 5 minutes. It precomputes per-member dashboard/reporting figures (pipeline revenue, won/lost counts, win rate, avg days to close, and related metrics) for fast reads from `/v1/sales/opportunities/metrics`.
### Key design principles
- **Adaptive TTLs** — cache durations are computed dynamically based on how "hot" an opportunity is (recently updated = shorter TTL = fresher data).
@@ -38,6 +40,14 @@ Inventory-adjustment-driven catalog sync adds a targeted product cache:
| ------------------------ | ---------------------------------------------------------- | -------------------------------------------------------------------------------------------- |
| `catalog:item:cw:{cwId}` | Full CW catalog item + computed `onHand` + DB row snapshot | `GET /procurement/adjustments` + `GET /procurement/catalog/:id` + catalog inventory endpoint |
Sales opportunity metrics caching adds member-focused keys:
| Cache Key Pattern | Data | Source |
| ------------------------------------- | -------------------------------------- | ------------------------------------------------------------------------------------- |
| `sales:metrics:members:all` | Envelope of all active-member metrics | Precomputed from active CW members + assigned opportunities + products cache/CW fetch |
| `sales:metrics:member:{cwIdentifier}` | One member's computed metrics snapshot | Same as above |
| `sales:metrics:oppRevenue:{cwOppId}` | Per-opportunity computed revenue blob | Metrics refresh lookups (products cache-first, then manager/controller fallback) |
---
## TTL Algorithms
@@ -172,6 +182,31 @@ The thunk pattern is critical. Previously, tasks were pushed as already-executin
At these settings, a full sweep of ~500 expired keys completes in ~1-2 minutes with zero CW errors and ~230ms median latency.
### Sales metrics refresh job
**Function:** `refreshSalesOpportunityMetricsCache()` in `src/modules/cache/salesOpportunityMetricsCache.ts`
**Interval:** Every 5 minutes, triggered from `src/index.ts`.
**Startup behavior:** On app startup, the refresh is invoked once with `forceColdLoad=true`, which clears metrics-owned Redis keys and bypasses metrics/product cache reuse for that initial rebuild. Subsequent interval runs use the normal warm path.
Refresh flow:
1. Fetch all active CW members (`inactiveFlag=false`).
Source: local `CwMember` table (kept in sync by the existing members refresh job).
2. Query DB opportunities assigned to those members (primary or secondary rep), scoped to open opportunities plus YTD-closed opportunities.
3. For each opportunity, compute revenue cache-first from `sales:metrics:oppRevenue:{cwOppId}` then `opp:products:{cwOpportunityId}`, and fallback through the manager/controller path (`opportunities.fetchRecord(...).fetchProducts()`) on miss.
4. Aggregate member metrics (pipeline revenue, won/lost MTD+YTD counts, avg days to close, weighted pipeline, win/loss rates, and related KPIs).
5. Write per-opportunity revenue blobs plus all-member and per-member snapshots to Redis with a 10-minute TTL.
Safety controls:
- **Single-flight lock** prevents overlapping refresh runs if a prior run is still in progress.
- **Per-opportunity timeout guard** ensures slow CW product lookups degrade to zero-revenue fallback instead of stalling the full refresh.
- **Force-cold-load mode** clears `sales:metrics:*` runtime state owned by the metrics cache before rebuilding startup data.
This cache-first model prioritizes metrics-owned opportunity revenue keys first, then opportunity product cache entries, and only reaches CW when needed.
---
## Retry Logic (`withCwRetry`)
@@ -343,6 +378,7 @@ src/index.ts
| `src/modules/cw-utils/cwApiLogger.ts` | Axios interceptor for JSONL call logging |
| `src/modules/cw-utils/fetchCompany.ts` | Company fetch with retry |
| `src/modules/cw-utils/procurement/listenInventoryAdjustments.ts` | Adjustment listener for targeted catalog-item cache + DB sync |
| `src/modules/cache/salesOpportunityMetricsCache.ts` | 5-minute active-member opportunity metrics cache |
| `src/constants.ts` | CW Axios instance config (timeout, logger) |
| `src/index.ts` | Refresh interval registration |
| `debug-scripts/analyze-cw-calls.py` | CW API call analysis script |