Forum Discussion

Qrew Cadet

2 months ago

Pipeline to Search 130k Records in Another App and Create if the Record Does Not Yet Exist

I'm trying to optimize my pipeline and I'm having trouble deciding on the best way. I have 2 apps in the same realm. App "A" has approx. 30k records, and app "B" has approx. 130k records. When a ne...

Pipelines

ToddMolino

Qrew Cadet

2 months ago

I need to search through over 130k records to see if it exists before adding the new record and that takes so much time.

MarkShnier__You
Qrew Legend
2 months ago
I don't see why you need to do a search. My post above says to create a bulk upsert, add a single row, and commit the upsert. It will Merge in if the Login ID exists and hence have no effect. If it's not there, it will add it. I think that you are over thinking this.