Forum Discussion

Curtis_M's avatar
Curtis_M
Qrew Cadet
21 hours ago

Pipeline Bulk Upsert Unique Values Only

Hi,

I have a table of Account_Templates. It is a many to many table where many Templates have many Accounts. The Account has an associated lookup value, we'll call Question. Many accounts might have the same value for this Question field.

I want my pipeline to search these Account-Templates and create a record on table Template_Questions. A new record should be created for only the unique combinations of field Template and field Question ([Template]-[Question]). My pipeline is currently structured as:


1. Prepare Bulk Upsert to Template_Questions
2. Search Account_Templates for all records
Loop{
3. For each search record found, add an upsert row where the key field equals "[Template]-[Question]".
}Loop End
4. Commit Upsert

When I run this pipeline, I get an error of duplicate values trying to be created.

Is there a way to trim the upsert of duplicates before committing it?
Alternative ideas are welcome.

Thank you,
Curtis

Pipelines

1 Reply

  • MarkShnier__You's avatar
    MarkShnier__You
    Icon for Qrew Legend rankQrew Legend
    11 hours ago

    I suggest using the new Bulk Trigger step type.

    do the search,

    Then do the build trigger.  It should be able to take the results of the search and then in a one by one sequential process, within the For Each loop, to do a search to see if the Parent Template exists, and branch on the result set empty to create it.  Don't put a Stop step in the logic flow.  There is a bug to be fixed that will stop all subsequent processing of you have a stop step.