Forum Discussion

AishwaryaJinde1's avatar
AishwaryaJinde1
Qrew Member
3 years ago

Check for unique records and import them while reject others in bulk import

Hello team,

I have a table called Products. Here, there is a column called Supplier-Product number which is formula field and I have made it unique according to the requirement.

When I am bulk importing the records from spreadsheet, if there are any duplicate records those are getting rejected with an error message. However, the unique records from the spreadsheet are also not getting uploaded.
Is there a way, to reject only duplicates and upload only unique records?


Thanks & Regards,
Aishwarya Jinde


------------------------------
Aishwarya Jinde
------------------------------
Data import and export
Pipelines

3 Replies

Sort By
  • MattHardy's avatar
    MattHardy
    Qrew Trainee
    3 years ago
    Could you create a copy of the table without the unique requirement, import into that, then run a pipeline to copy the data to the working table? The pipeline will fail on each of the non-unique records, but will continue on all unique. Then use the same pipeline to remove all records from the import table

    ------------------------------
    Matt Hardy
    ------------------------------
    • AishwaryaJinde1's avatar
      AishwaryaJinde1
      Qrew Member
      3 years ago
      Hello Matt, this solution works great. Thank you!

      Above you have said that using the same pipeline to remove all records from the import table
      Here, the import table refers to the copy table where the unique property is not present, right?

      ------------------------------
      Aishwarya Jinde
      ------------------------------
  • MarkShnier__You's avatar
    MarkShnier__You
    Icon for Qrew Legend rankQrew Legend
    3 years ago
    The record could be imported to a temporary scratch table. Then us a formula query to detect duplicates and only flag the 1st "duplicate" as good to import.

    When I get a chance I will post a link to how to detect duplicates.


    ------------------------------
    Mark Shnier (YQC)
    mark.shnier@gmail.com
    ------------------------------