Forum Discussion
Mark,
Thanks for the great explanation on Bulk Upsert. I've been trying to understand what it is and when to use it.
So do I understand correctly:
This WOULD work by simply doing a search in the master table, and the For Each: Create the child record. But that would take system resources as 50 create APIs will run?
However, if I do the Bulk Upsert technique, in the For Each loop it only performs one API call?
------------------------------
Mike Tamoush
------------------------------
Mike, exactly.
The other thing you will observe if you watch the pipeline run if you try to add the records individually is that the For Each loop fires asynchronously. What that means is basically it hammers the application with 50 API requests all at the same time. Because QuickBase cannot do 50 things at once it rejects many of the API requests. Now, fortunately the pipeline re-Queues those requests automatically and then fires the same request after a short delay.
I think I recall seeing an indication that it was willing to try the same request up to 15 times before it gives up.
But imagine a use case where you were adding say 2000 records. So you're app gets hammered with 2000 instantaneous API calls and I think that there's a risk that some of them could timeout because every time it goes to retry a rejected request the app would still be busy processing others of those 2000 API's.
Also, if you had 2000 individual request I think your users would definitely feel like the app became unresponsive.
So sure, in practice, probably 50 individual requests you could get away with easily and nobody would really notice but it's just basic good practice if you are going to add dozens of records to take the extra time to create that bulk upsert and do it "right ". One API call to Query and one API call to Upsert.
------------------------------
Mark Shnier (YQC)
mark.shnier@gmail.com
------------------------------