fix: fix several different data parsing issues

This commit is contained in:
2026-04-18 14:47:06 +00:00
parent 5141ed20f9
commit f91d8debcb
16 changed files with 632 additions and 49 deletions
@@ -0,0 +1,5 @@
-- Fix schema drift: the Opportunity.typeId column was added as NOT NULL but the
-- Prisma schema declares it as Int? (nullable). Drop the NOT NULL constraint so
-- the DB matches the schema and so that creating an opportunity without a type
-- (e.g. when CW uses no default type) no longer fails with P2011.
ALTER TABLE "Opportunity" ALTER COLUMN "typeId" DROP NOT NULL;
+258
View File
@@ -0,0 +1,258 @@
SSUUMMMMAARRYY OOFF LLEESSSS CCOOMMMMAANNDDSS
Commands marked with * may be preceded by a number, _N.
Notes in parentheses indicate the behavior if _N is given.
A key preceded by a caret indicates the Ctrl key; thus ^K is ctrl-K.
h H Display this help.
q :q Q :Q ZZ Exit.
---------------------------------------------------------------------------
MMOOVVIINNGG
e ^E j ^N CR * Forward one line (or _N lines).
y ^Y k ^K ^P * Backward one line (or _N lines).
f ^F ^V SPACE * Forward one window (or _N lines).
b ^B ESC-v * Backward one window (or _N lines).
z * Forward one window (and set window to _N).
w * Backward one window (and set window to _N).
ESC-SPACE * Forward one window, but don't stop at end-of-file.
d ^D * Forward one half-window (and set half-window to _N).
u ^U * Backward one half-window (and set half-window to _N).
ESC-) RightArrow * Right one half screen width (or _N positions).
ESC-( LeftArrow * Left one half screen width (or _N positions).
ESC-} ^RightArrow Right to last column displayed.
ESC-{ ^LeftArrow Left to first column.
F Forward forever; like "tail -f".
ESC-F Like F but stop when search pattern is found.
r ^R ^L Repaint screen.
R Repaint screen, discarding buffered input.
---------------------------------------------------
Default "window" is the screen height.
Default "half-window" is half of the screen height.
---------------------------------------------------------------------------
SSEEAARRCCHHIINNGG
/_p_a_t_t_e_r_n * Search forward for (_N-th) matching line.
?_p_a_t_t_e_r_n * Search backward for (_N-th) matching line.
n * Repeat previous search (for _N-th occurrence).
N * Repeat previous search in reverse direction.
ESC-n * Repeat previous search, spanning files.
ESC-N * Repeat previous search, reverse dir. & spanning files.
ESC-u Undo (toggle) search highlighting.
ESC-U Clear search highlighting.
&_p_a_t_t_e_r_n * Display only matching lines.
---------------------------------------------------
A search pattern may begin with one or more of:
^N or ! Search for NON-matching lines.
^E or * Search multiple files (pass thru END OF FILE).
^F or @ Start search at FIRST file (for /) or last file (for ?).
^K Highlight matches, but don't move (KEEP position).
^R Don't use REGULAR EXPRESSIONS.
^W WRAP search if no match found.
---------------------------------------------------------------------------
JJUUMMPPIINNGG
g < ESC-< * Go to first line in file (or line _N).
G > ESC-> * Go to last line in file (or line _N).
p % * Go to beginning of file (or _N percent into file).
t * Go to the (_N-th) next tag.
T * Go to the (_N-th) previous tag.
{ ( [ * Find close bracket } ) ].
} ) ] * Find open bracket { ( [.
ESC-^F _<_c_1_> _<_c_2_> * Find close bracket _<_c_2_>.
ESC-^B _<_c_1_> _<_c_2_> * Find open bracket _<_c_1_>.
---------------------------------------------------
Each "find close bracket" command goes forward to the close bracket
matching the (_N-th) open bracket in the top line.
Each "find open bracket" command goes backward to the open bracket
matching the (_N-th) close bracket in the bottom line.
m_<_l_e_t_t_e_r_> Mark the current top line with <letter>.
M_<_l_e_t_t_e_r_> Mark the current bottom line with <letter>.
'_<_l_e_t_t_e_r_> Go to a previously marked position.
'' Go to the previous position.
^X^X Same as '.
ESC-M_<_l_e_t_t_e_r_> Clear a mark.
---------------------------------------------------
A mark is any upper-case or lower-case letter.
Certain marks are predefined:
^ means beginning of the file
$ means end of the file
---------------------------------------------------------------------------
CCHHAANNGGIINNGG FFIILLEESS
:e [_f_i_l_e] Examine a new file.
^X^V Same as :e.
:n * Examine the (_N-th) next file from the command line.
:p * Examine the (_N-th) previous file from the command line.
:x * Examine the first (or _N-th) file from the command line.
:d Delete the current file from the command line list.
= ^G :f Print current file name.
---------------------------------------------------------------------------
MMIISSCCEELLLLAANNEEOOUUSS CCOOMMMMAANNDDSS
-_<_f_l_a_g_> Toggle a command line option [see OPTIONS below].
--_<_n_a_m_e_> Toggle a command line option, by name.
__<_f_l_a_g_> Display the setting of a command line option.
___<_n_a_m_e_> Display the setting of an option, by name.
+_c_m_d Execute the less cmd each time a new file is examined.
!_c_o_m_m_a_n_d Execute the shell command with $SHELL.
|XX_c_o_m_m_a_n_d Pipe file between current pos & mark XX to shell command.
s _f_i_l_e Save input to a file.
v Edit the current file with $VISUAL or $EDITOR.
V Print version number of "less".
---------------------------------------------------------------------------
OOPPTTIIOONNSS
Most options may be changed either on the command line,
or from within less by using the - or -- command.
Options may be given in one of two forms: either a single
character preceded by a -, or a name preceded by --.
-? ........ --help
Display help (from command line).
-a ........ --search-skip-screen
Search skips current screen.
-A ........ --SEARCH-SKIP-SCREEN
Search starts just after target line.
-b [_N] .... --buffers=[_N]
Number of buffers.
-B ........ --auto-buffers
Don't automatically allocate buffers for pipes.
-c ........ --clear-screen
Repaint by clearing rather than scrolling.
-d ........ --dumb
Dumb terminal.
-D xx_c_o_l_o_r . --color=xx_c_o_l_o_r
Set screen colors.
-e -E .... --quit-at-eof --QUIT-AT-EOF
Quit at end of file.
-f ........ --force
Force open non-regular files.
-F ........ --quit-if-one-screen
Quit if entire file fits on first screen.
-g ........ --hilite-search
Highlight only last match for searches.
-G ........ --HILITE-SEARCH
Don't highlight any matches for searches.
-h [_N] .... --max-back-scroll=[_N]
Backward scroll limit.
-i ........ --ignore-case
Ignore case in searches that do not contain uppercase.
-I ........ --IGNORE-CASE
Ignore case in all searches.
-j [_N] .... --jump-target=[_N]
Screen position of target lines.
-J ........ --status-column
Display a status column at left edge of screen.
-k [_f_i_l_e] . --lesskey-file=[_f_i_l_e]
Use a lesskey file.
-K ........ --quit-on-intr
Exit less in response to ctrl-C.
-L ........ --no-lessopen
Ignore the LESSOPEN environment variable.
-m -M .... --long-prompt --LONG-PROMPT
Set prompt style.
-n -N .... --line-numbers --LINE-NUMBERS
Don't use line numbers.
-o [_f_i_l_e] . --log-file=[_f_i_l_e]
Copy to log file (standard input only).
-O [_f_i_l_e] . --LOG-FILE=[_f_i_l_e]
Copy to log file (unconditionally overwrite).
-p [_p_a_t_t_e_r_n] --pattern=[_p_a_t_t_e_r_n]
Start at pattern (from command line).
-P [_p_r_o_m_p_t] --prompt=[_p_r_o_m_p_t]
Define new prompt.
-q -Q .... --quiet --QUIET --silent --SILENT
Quiet the terminal bell.
-r -R .... --raw-control-chars --RAW-CONTROL-CHARS
Output "raw" control characters.
-s ........ --squeeze-blank-lines
Squeeze multiple blank lines.
-S ........ --chop-long-lines
Chop (truncate) long lines rather than wrapping.
-t [_t_a_g] .. --tag=[_t_a_g]
Find a tag.
-T [_t_a_g_s_f_i_l_e] --tag-file=[_t_a_g_s_f_i_l_e]
Use an alternate tags file.
-u -U .... --underline-special --UNDERLINE-SPECIAL
Change handling of backspaces.
-V ........ --version
Display the version number of "less".
-w ........ --hilite-unread
Highlight first new line after forward-screen.
-W ........ --HILITE-UNREAD
Highlight first new line after any forward movement.
-x [_N[,...]] --tabs=[_N[,...]]
Set tab stops.
-X ........ --no-init
Don't use termcap init/deinit strings.
-y [_N] .... --max-forw-scroll=[_N]
Forward scroll limit.
-z [_N] .... --window=[_N]
Set size of window.
-" [_c[_c]] . --quotes=[_c[_c]]
Set shell quote characters.
-~ ........ --tilde
Don't display tildes after end of file.
-# [_N] .... --shift=[_N]
Set horizontal scroll amount (0 = one half screen width).
--file-size
Automatically determine the size of the input file.
--follow-name
The F command changes files if the input file is renamed.
--incsearch
Search file as each pattern character is typed in.
--line-num-width=N
Set the width of the -N line number field to N characters.
--mouse
Enable mouse input.
--no-keypad
Don't send termcap keypad init/deinit strings.
--no-histdups
Remove duplicates from command history.
--rscroll=C
Set the character used to mark truncated lines.
--save-marks
Retain marks across invocations of less.
--status-col-width=N
Set the width of the -J status column to N characters.
--use-backslash
Subsequent options use backslash as escape char.
--use-color
Enables colored text.
--wheel-lines=N
Each click of the mouse wheel moves N lines.
---------------------------------------------------------------------------
LLIINNEE EEDDIITTIINNGG
These keys can be used to edit text being entered
on the "command line" at the bottom of the screen.
RightArrow ..................... ESC-l ... Move cursor right one character.
LeftArrow ...................... ESC-h ... Move cursor left one character.
ctrl-RightArrow ESC-RightArrow ESC-w ... Move cursor right one word.
ctrl-LeftArrow ESC-LeftArrow ESC-b ... Move cursor left one word.
HOME ........................... ESC-0 ... Move cursor to start of line.
END ............................ ESC-$ ... Move cursor to end of line.
BACKSPACE ................................ Delete char to left of cursor.
DELETE ......................... ESC-x ... Delete char under cursor.
ctrl-BACKSPACE ESC-BACKSPACE ........... Delete word to left of cursor.
ctrl-DELETE .... ESC-DELETE .... ESC-X ... Delete word under cursor.
ctrl-U ......... ESC (MS-DOS only) ....... Delete entire line.
UpArrow ........................ ESC-k ... Retrieve previous command line.
DownArrow ...................... ESC-j ... Retrieve next command line.
TAB ...................................... Complete filename & cycle.
SHIFT-TAB ...................... ESC-TAB Complete filename & reverse cycle.
ctrl-L ................................... Complete filename, list all.
@@ -0,0 +1,35 @@
column_name | data_type | is_nullable | column_default
---------------------+-----------------------------+-------------+--------------------
id | integer | NO |
uid | text | NO |
name | text | NO |
notes | text | YES |
oppNarrative | text | YES |
typeId | integer | NO |
stageId | integer | YES |
statusId | integer | YES |
interest | USER-DEFINED | YES |
probability | double precision | NO | 0
source | text | YES |
primarySalesRepId | text | YES |
secondarySalesRepId | text | YES |
companyId | integer | YES |
contactId | integer | YES |
siteId | integer | YES |
customerPO | text | YES |
locationId | integer | YES |
departmentId | integer | YES |
expectedCloseDate | timestamp without time zone | YES |
pipelineChangeDate | timestamp without time zone | YES |
dateBecameLead | timestamp without time zone | YES |
closedDate | timestamp without time zone | YES |
closedFlag | boolean | NO | false
closedById | text | YES |
productSequence | ARRAY | YES | ARRAY[]::integer[]
updatedBy | text | NO |
eneteredBy | text | NO |
createdAt | timestamp without time zone | NO | CURRENT_TIMESTAMP
updatedAt | timestamp without time zone | NO |
taxCodeId | integer | YES |
(31 rows)
+2
View File
@@ -18,6 +18,7 @@ export default createRoute(
const includePrimaryContact =
c.req.query("includePrimaryContact") === "true";
const includeAllContacts = c.req.query("includeAllContacts") === "true";
const includeAllAddresses = c.req.query("includeAllAddresses") === "true";
console.log(company.toJson({ includeAddress, includePrimaryContact, includeAllContacts }));
@@ -49,6 +50,7 @@ export default createRoute(
includeAddress,
includePrimaryContact,
includeAllContacts,
includeAllAddresses,
});
const gatedData = await processObjectValuePerms(
companyData,
@@ -118,6 +118,7 @@ export default createRoute(
);
}
console.error("[Opportunity Create] DB write failed after CW create:", err);
throw new GenericError({
status: 500,
name: "OpportunityCreateError",
+39 -8
View File
@@ -288,19 +288,50 @@ export class CompanyController {
includeAddress?: boolean;
includePrimaryContact?: boolean;
includeAllContacts?: boolean;
includeAllAddresses?: boolean;
}) {
const cw_Data: Record<string, unknown> = {};
if (opts?.includeAddress && this.cw_Data) {
const addr = this.cw_Data.company;
cw_Data.address = {
line1: addr.addressLine1 ?? null,
line2: addr.addressLine2 ?? null,
if (opts?.includeAddress) {
if (this.cw_Data) {
const addr = this.cw_Data.company;
cw_Data.address = {
line1: addr.addressLine1 ?? null,
line2: addr.addressLine2 ?? null,
city: addr.city ?? null,
state: addr.state ?? null,
zip: addr.zip ?? null,
country: addr.country?.name ?? "United States",
};
} else if (this._defaultAddress) {
const addr = this._defaultAddress;
cw_Data.address = {
line1: addr.addressLine1 ?? null,
line2: addr.addressLine2 ?? null,
city: addr.city ?? null,
state: addr.state ?? null,
zip: addr.zipCode ?? null,
country: addr.country ?? "United States",
};
}
}
if (opts?.includeAllAddresses) {
cw_Data.allAddresses = this._addresses.map((addr) => ({
id: addr.id,
uid: addr.uid,
name: addr.name,
description: addr.description ?? null,
defaultFlag: addr.defaultFlag,
inactiveFlag: addr.inactiveFlag,
addressLine1: addr.addressLine1 ?? null,
addressLine2: addr.addressLine2 ?? null,
city: addr.city ?? null,
state: addr.state ?? null,
zip: addr.zip ?? null,
country: addr.country?.name ?? "United States",
};
zip: addr.zipCode ?? null,
country: addr.country ?? null,
phone: addr.phone ?? null,
}));
}
if (opts?.includePrimaryContact) {
+5 -2
View File
@@ -1476,14 +1476,17 @@ export class OpportunityController {
public async resequenceProducts(
orderedIds: number[]
): Promise<ForecastProductController[]> {
// Validate all IDs exist in the local ProductData table (the IDs the UI works with)
// Validate all IDs exist in the local ProductData table (the IDs the UI works with).
// Fall back to productSequence for items that were just added and haven't been
// synced to productData yet — appendProductSequenceIds writes them immediately.
const existingRows = await prisma.productData.findMany({
where: { opportunityId: this.cwOpportunityId },
select: { id: true },
});
const existingIds = new Set(existingRows.map((r) => r.id));
const sequenceIds = new Set(this.productSequence);
for (const id of orderedIds) {
if (!existingIds.has(id)) {
if (!existingIds.has(id) && !sequenceIds.has(id)) {
throw new GenericError({
status: 404,
name: "ForecastItemNotFound",
+59 -2
View File
@@ -46,7 +46,18 @@ export const opportunities = {
// Resolve optional local FKs — nullify any that don't exist locally yet
// (the sync may be behind; these are all nullable in the schema)
const [companyExists, contactExists, siteExists, typeExists] = await Promise.all([
const [
companyExists,
contactExists,
siteExists,
typeExists,
stageExists,
statusExists,
locationExists,
departmentExists,
primaryRepExists,
secondaryRepExists,
] = await Promise.all([
cwData.company?.id
? prisma.company.findFirst({ where: { id: cwData.company.id }, select: { id: true } })
: null,
@@ -59,21 +70,67 @@ export const opportunities = {
mapped.typeId != null
? prisma.opportunityType.findFirst({ where: { id: mapped.typeId }, select: { id: true } })
: null,
mapped.stageId != null
? prisma.opportunityStage.findFirst({ where: { id: mapped.stageId }, select: { id: true } })
: null,
mapped.statusId != null
? prisma.opportunityStatus.findFirst({ where: { id: mapped.statusId }, select: { id: true } })
: null,
mapped.locationId != null
? prisma.corporateLocation.findFirst({ where: { id: mapped.locationId }, select: { id: true } })
: null,
mapped.departmentId != null
? prisma.internalDepartment.findFirst({ where: { id: mapped.departmentId }, select: { id: true } })
: null,
mapped.primarySalesRepId != null
? prisma.user.findFirst({ where: { cwIdentifier: mapped.primarySalesRepId }, select: { cwIdentifier: true } })
: null,
mapped.secondarySalesRepId != null
? prisma.user.findFirst({ where: { cwIdentifier: mapped.secondarySalesRepId }, select: { cwIdentifier: true } })
: null,
]);
const companyId = companyExists?.id ?? null;
const contactId = contactExists?.id ?? null;
const siteId = siteExists?.id ?? null;
const typeId = typeExists?.id ?? null;
const stageId = stageExists?.id ?? null;
const statusId = statusExists?.id ?? null;
const locationId = locationExists?.id ?? null;
const departmentId = departmentExists?.id ?? null;
const primarySalesRepId = primaryRepExists?.cwIdentifier ?? null;
const secondarySalesRepId = secondaryRepExists?.cwIdentifier ?? null;
// Strip fields returned by mapCwToDb that are not columns in the Prisma schema
// (ratingName, ratingCwId, campaignName, primarySalesRepName, primarySalesRepIdentifier,
// secondarySalesRepName, secondarySalesRepIdentifier, cwLastUpdated).
// Prisma will throw a validation error if unknown fields are passed to create().
const {
ratingName: _ratingName,
ratingCwId: _ratingCwId,
campaignName: _campaignName,
primarySalesRepName: _primarySalesRepName,
primarySalesRepIdentifier: _primarySalesRepIdentifier,
secondarySalesRepName: _secondarySalesRepName,
secondarySalesRepIdentifier: _secondarySalesRepIdentifier,
cwLastUpdated: _cwLastUpdated,
...dbFields
} = mapped;
const record = await prisma.opportunity.create({
data: {
id: cwData.id,
...mapped,
...dbFields,
typeId,
stageId,
statusId,
locationId,
departmentId,
companyId,
contactId,
siteId,
primarySalesRepId,
secondarySalesRepId,
},
include: {
company: { include: { contacts: true, companyAddresses: true } },
+17
View File
@@ -0,0 +1,17 @@
import { Client } from "pg";
const url = process.env.DATABASE_URL ?? "postgresql://optima:123web123@localhost:5432/optima";
const c = new Client(url);
await c.connect();
const r = await c.query(
"SELECT column_name, is_nullable, column_default FROM information_schema.columns WHERE table_name = 'Opportunity' ORDER BY ordinal_position"
);
console.log("Opportunity columns:");
for (const row of r.rows) {
const nullable = row.is_nullable === "YES" ? "nullable" : "NOT NULL";
console.log(` ${row.column_name}: ${nullable}${row.column_default ? ` (default: ${row.column_default})` : ""}`);
}
await c.end();