Forum Discussion

Jonathan_Gibson's avatar
Jonathan_Gibson
Qrew Cadet
5 days ago

Accessing previous field values after an import mapping failure

I have a bulk event-triggered pipeline firing whenever records are modified, leading to an Import to Quickbase step. The import stores current and previous field values on another table.

Working on error handling for this import, I would like to log the attributes of the intended "change record" it was attempting to create, i.e. current and previous field values of the source record. If the error was an "import error" this is easy: I can access any of the target field values inside the `import_error.record` container.

But if the error was a "mapping error" then it seems only the source's current fields are accessible through the `mapping_error.item` container, with apparently no metadata like previous field values.

I don't think I can reference the original record in this context because it's a completely different object with a different index (since the import failures are obviously a subset of the original record set). So this leaves me without a way to store previous field values if there is a mapping error that I can see, unless I restructure the pipeline to process the records serially, which I'd prefer not to do if I can avoid it as the trigger event can potentially be quite large (thousands of changes at a time). 

Am I missing something?

Or should I just not concern myself with auto-handling mapping errors in production, because the data types should never be allowed to change in the first place? Should I just make sure to throw a bright red alert if it does ever happen?

1 Reply