- Add Redis-backed opportunity cache with background refresh (30s interval) - Fix concurrency bug: use lazy thunks instead of eager promises for batching - Add withCwRetry utility with exponential backoff for transient CW errors - Add adaptive TTL algorithms (primary, sub-resource, products) based on opportunity activity - Add include query param on GET /sales/opportunities/:id (notes,contacts,products) - Add opt-in CW API logger (LOG_CW_API env var) with timestamped files in cw-api-logs/ - Add debug-scripts/analyze-cw-calls.py for API call analysis - Add computeSubResourceCacheTTL and computeProductsCacheTTL algorithms with tests - Increase CW API timeout from 15s to 30s - Unblock cache refresh from startup chain (remove await) - Prioritize recently updated opportunities in refresh cycle - Add CACHING.md documentation - Update API_ROUTES.md with caching details and include param - Update copilot instructions to require CACHING.md sync - Add dev:log script for CW API call logging during development
15 KiB
Copilot / AI Agent Instructions for optima-api
Purpose: make AI coding agents immediately productive in this repository by describing architecture, conventions, workflows, and helpful code pointers.
Big picture
This is a TypeScript API service (runs on Bun) using the Hono framework. The HTTP surface is implemented in src/api where small route-handler files are mounted on a versioned router in src/api/server.ts (see the /v1 mount). The typical request flow is:
server.ts → routers/<domain>Router.ts → api/<domain>/*.ts (route handlers) → managers/<domain>.ts → controllers (domain models) / modules / generated/prisma
Keep each layer focused:
- Route handler files (
src/api/<domain>/*.ts) — define individual endpoints using thecreateRoute()utility, handle request validation with Zod, call managers, and returnapiResponse.*results. - Routers (
src/api/routers/*) — aggregate route handler files from a domain'sindex.tsand re-mount them under a single prefix. Mounted insrc/api/server.ts. - Managers (
src/managers/*) — thin domain/persistence layers that wrapgenerated/prismacalls and other I/O. Managers instantiate and return controller instances as domain objects. - Controllers (
src/controllers/*) — domain model classes (e.g.,CompanyController,CredentialController,UserController) that encapsulate entity state and domain methods. They are NOT request handlers. Managers create and return controller instances. - Modules (
src/modules/*) — shared utilities, external API clients (ConnectWise, UniFi, Microsoft), credential helpers, permission utilities, and tools.
Runtime / tooling
The project runs on Bun exclusively — always use bun commands, never npm, npx, or yarn. DB tooling uses Prisma; the generated client lives under generated/prisma (do NOT edit generated files). Test preloads are configured in bunfig.toml so bare bun test works. Key scripts in package.json:
dev—NODE_ENV=development bun --watch src/index.ts(start dev server with hot reload)test—bun test(runs all tests with preload frombunfig.toml)db:gen—prisma generatedb:push—prisma migrate dev --skip-generateutils:dev—docker compose -f .docker/docker-compose.yml up --buildutils:gen_private_keys—bun ./utils/genPrivateKeysutils:create_admin_role—bun ./utils/createAdminRoleutils:assign_user_role—bun ./utils/assignUserRole
Data layer
Prisma schema is at prisma/schema.prisma. The app imports the generated Prisma client from generated/prisma/client.ts (or generated/prisma/browser.ts for browser type contexts). The shared prisma instance is exported from src/constants.ts. Always run bun run db:gen after updating schema.prisma.
Shared constants (src/constants.ts)
This file exports critical shared instances used across the codebase:
prisma— the PrismaClient instance (via@prisma/adapter-pg)PORT, session/token durations, private/public keys for JWT signingmsalClient— Microsoft MSAL client for OAuthconnectWiseApi— Axios instance for ConnectWise APIunifi—UnifiClientinstance for UniFi controller interaction
Route handler pattern (createRoute)
Every route handler file uses the createRoute() utility from src/modules/api-utils/createRoute.ts. It creates a self-contained Hono sub-app for a single endpoint:
export default createRoute(
"get", // HTTP method
["/companies"], // path(s)
async (c) => {
/* handler */
}, // request handler
authMiddleware({ permissions: ["company.fetch.many"] }), // middleware (spread)
);
Route files live in src/api/<domain>/*.ts. Each domain folder has an index.ts that re-exports all route modules. Router files (src/api/routers/<domain>Router.ts) import from the domain's index and auto-mount all routes:
import * as companyRoutes from "../companies";
const companyRouter = new Hono();
Object.values(companyRoutes).map((r) => companyRouter.route("/", r));
export default companyRouter;
src/api/server.ts then mounts each router under /v1:
v1.route("/company", require("./routers/companyRouter").default);
Routing & domain organization
The server.ts file mounts these routers under /v1:
/teapot— health check/auth— Microsoft OAuth flow (src/api/auth/*)/user— user routes (src/api/user/*) including@mesub-routes/company— company routes (src/api/companies/*)/credential— credential routes (src/api/credentials/*)/credential-type— credential type routes (src/api/credential-types/*)/role— role management (src/api/roles/*)/permissions— permission node queries (src/api/permissions/*)/unifi— UniFi integration (src/api/unifi/*withsites/andsite/sub-folders)
API layout & conventions (how to add a new endpoint)
- Add route handler files: Create
src/api/<domain>/<action>.tsfiles, each exporting a defaultcreateRoute(...)call. Validate input with Zod inside the handler, call managers for business logic, and return responses viaapiResponse.*. - Add domain index: Create
src/api/<domain>/index.tsthat re-exports all route modules from the folder. - Add router: Create
src/api/routers/<domain>Router.tsthat imports all routes from the domain's index and mounts them. Mount it insrc/api/server.tsunderv1. - Add manager: Create
src/managers/<domains>.ts(plural filename) for persistence/domain logic. Use theprismainstance fromsrc/constants.ts; do not import Prisma directly in route handlers. Managers should instantiate and return controller instances when the domain warrants it. - Add controller (if needed): Create
src/controllers/<Domain>Controller.ts(singular filename) as a class that encapsulates entity state, domain methods, and atoJson()serializer. Controllers are domain model objects — they do NOT handle HTTP requests. - Add modules/types: If needed, add helpers to
src/modules/*and runtime types tosrc/types/*. - Middleware & auth: Use
authMiddleware()fromsrc/api/middleware/authorization.tsas the last argument tocreateRoute(). It accepts{ permissions?: string[], scopes?: string[], forbiddenAuthTypes?: string[] }. Follow existing token/session patterns fromsrc/controllers/SessionController.tsandsrc/Errors/*. - Error handling: Throw repository-specific errors from
src/Errors/*(includestatus,name,message, optionalcause) and letsrc/api/server.tsmap them viaapiResponse.error. - Naming conventions: plural manager filenames (
companies.ts), singular controller class names (CompanyController.ts), descriptive route handler filenames (fetchAll.ts,create.ts,update.ts).
Examples & notable files
src/api/server.ts— mountsv1, registerscors, centralonErrorhandler andnotFoundresponse.src/api/companies/fetchAll.ts— canonical example of a route handler file usingcreateRoute(),authMiddleware(), managers, andapiResponse.src/api/credentials/create.ts— example of Zod validation inside a route handler.src/api/routers/companyRouter.ts— canonical router that auto-mounts all route modules from a domain folder.src/api/user/@me/*— nested sub-route example (user's own profile endpoints).src/controllers/CompanyController.ts— example domain model class with methods likerefreshFromCW(),fetchCwData(), andtoJson().src/controllers/CredentialController.ts— example of a richer domain model class with field validation, secure value handling, and sub-credential support.src/managers/companies.ts— example manager calling Prisma and returningCompanyControllerinstances.src/modules/api-utils/createRoute.ts— thecreateRoute()utility used by every route handler.src/modules/cw-utils/*— external ConnectWise API integrations; keep interfaces stable and return domain objects consumed by managers.src/modules/unifi-api/UnifiClient.ts— UniFi controller API client class with methods for sites, WLANs, devices, networks, etc.src/modules/credentials/*— credential field validation, secure value encryption/decryption, and type definitions.src/constants.ts— shared instances (prisma, API clients, keys, durations).
Validation & errors
Zod is used for input validation inside route handler files (not controllers). Zod errors are handled centrally in src/api/server.ts via apiResponse.zodError. Application errors use custom error classes in src/Errors/* (e.g., AuthenticationError.ts, GenericError.ts, AuthorizationError.ts, InsufficientPermission.ts). When creating errors, follow the shape used in existing errors (include status, name, message, and optional cause).
Response pattern
Use the apiResponse helpers in src/modules/api-utils/apiResponse.ts for formatting all HTTP responses:
apiResponse.successful(message, data?, meta?)— 200apiResponse.created(message, data?)— 201apiResponse.error(err)— readsstatusfrom the errorapiResponse.internalError()— 500apiResponse.zodError(err)— 400
Auth & external integrations
Microsoft OAuth flow is under src/api/auth/* and src/modules/fetchMicrosoftUser.ts. If touching authentication, follow existing redirect/refresh patterns in src/api/auth and preserve token-refresh semantics.
ConnectWise integration
Utilities for ConnectWise interactions live in src/modules/cw-utils/* (e.g., configurations/fetchCompanyConfigurations.ts, fetchCompany.ts, fetchAllCompanies.ts). These modules call external APIs via the connectWiseApi Axios instance from src/constants.ts and return domain data; preserve the module contracts (input types and returned shapes) when refactoring.
UniFi integration
The UnifiClient class in src/modules/unifi-api/UnifiClient.ts wraps all UniFi controller API interactions (login, sites, WLANs, devices, networks, AP groups, WLAN groups, speed profiles, PPSKs). The shared instance is exported from src/constants.ts as unifi. UniFi route handlers live in src/api/unifi/ with sites/ (multi-site operations) and site/ (single-site operations) sub-folders.
Generated files and CI
generated/ is a build artifact. Do not modify. When updating Prisma models, run npm run db:gen and commit changes to generated/prisma only if that's the established workflow.
Coding conventions & patterns specific to this repo
- Prefer the existing layered architecture: route handlers → managers → controllers (domain models) / modules.
- Use the
createRoute()utility for all route handler files. - Use the
apiResponsehelpers for all HTTP responses. - Throw or propagate repository-specific custom errors (from
src/Errors/*) rather than returning bare objects. - Keep TypeScript types in
src/types/*and use Zod for runtime checks inside route handlers. - Avoid
elsestatements — prefer ternary operators, early returns, or other control flow patterns. Only useelseif there is absolutely no other way. - ES module syntax (
export default,import) is used throughout. Therequire()calls inserver.tsare for lazy loading but all modules useexport default.
Local dev / quick checks
- Start dev server:
bun run dev - Run tests:
bun test - Regenerate Prisma client:
bun run db:gen - Apply DB migrations locally:
bun run db:push - Docker dev utilities:
bun run utils:dev - Generate private keys:
bun run utils:gen_private_keys - Create admin role:
bun run utils:create_admin_role - Assign user role:
bun run utils:assign_user_role
When editing generated or infra files
If you need to change generated/prisma/* (rare), explain why in the PR and show commands used to regenerate.
Documentation sync (IMPORTANT)
Whenever you add, remove, or modify API routes or permission nodes, you must update all three of the following files to keep them in sync:
src/types/PermissionNodes.ts— the single source of truth for all permission node definitions, categories, descriptions, andusedInreferences.PERMISSIONS.md— human-readable documentation of all permission nodes; must strictly reflect the data inPermissionNodes.ts.API_ROUTES.md— comprehensive documentation of all API routes, including method, path, auth requirements, permissions, request/response examples.
Additionally, whenever you add, remove, or modify caching logic (TTL algorithms, cache key patterns, background refresh mechanics, retry settings, or invalidation behavior), you must update:
CACHING.md— comprehensive documentation of the Redis-backed opportunity cache, TTL algorithms, background refresh mechanics, retry logic, and debugging tools.
Always verify that new routes have their required permissions listed in PermissionNodes.ts, that PERMISSIONS.md tables match the TS file exactly, that API_ROUTES.md includes full documentation for every mounted route, and that CACHING.md accurately reflects any caching changes. Run through all relevant files at the end of any route, permission, or caching change to catch discrepancies.
Field-level permission gating (processObjectValuePerms)
Some routes use processObjectValuePerms from src/modules/permission-utils/processObjectPermissions.ts to filter response objects on a per-field basis. When this pattern is used, every key of the response object becomes a permission node in the form <scope>.<field> (e.g., unifi.site.wifi.read.passphrase). Only fields whose corresponding permission the user holds are included in the response.
There is also processObjectPermMap in the same file, which returns a Record<key, boolean> indicating which fields the user has permission for (useful for UI gating).
When documenting a route that uses field-level gating, you must:
- Note in
API_ROUTES.mdthat the route uses field-level gating, explain the behaviour, and list every<scope>.<field>permission node in a collapsible table. - Add a
unifi.site.wifi.read-style parent permission node inPermissionNodes.tswith afieldLevelPermissionsarray listing every<scope>.<field>node. - Add matching rows/notes to
PERMISSIONS.mdincluding the full list of field-level nodes.
Current routes using field-level gating:
GET /v1/unifi/site/:id/wifi— scopeunifi.site.wifi.read, gates every field on theWlanConfobject.
If anything here is unclear or you'd like more examples (e.g., a walk-through adding a route handler + manager + controller), tell me which area to expand and I'll iterate.