Sync table and pipelines
Hello,
I have a sync table to salesforce, and I have another table called projects.
I've created a pipeline when the trigger is record added to SF table, then it must create a record into projects table.
I tested this in sandbox and it works, but I manually created the record in SF tables (trying to mimic the behavior in live). So, I created another pipeline for "live", but this doesn't trigger. Now I know that sync table doesn't works the same as a normal table, so I need to change the logic of my trigger.
SF table refresh the data once every day. I was thinking in add a "schedule", and search when "Creation Date" is the date of "today", like:
If CreationDate = today AND flag = unchecked
Create record in projects
flag = Check
End if
But I don't know how accurate this logic is. I want to create a record in project with the new sync data and not with old values.
Any suggestion will be appreciated :)
Since it is just a few records per day, then you can just directly do the search and then the For Each loop will add the required records.
I was just offering a more generalized response for the situation when there are 1,000s of records to be added. There is a step in a Pipeline called a Bulk Upsert.
So what happens is you have a step to create essentially an empty box with identified fields in it. That is the Bulk Upsert container.
Then the For Each loop will add Rows to the container (there is a pipeline step type for that), and then outside the loop you "Commit" the upsert. (also a Pipeline step type)
The effect of that is that it has the least impact on the performance of your app. There is just one query to identify the records and then in a separate process run by the pipeline and not impacting your app at all, it fills up the box with the records that are going to need to be added. Then the Bulk Upsert box is imported all at once, which is very efficient for Quickbase to do.
The alternative if you just do it by brute force would be to do a search that returns say 1000 records and then the For Each loop would add those records one by one. That places a lot of load on the app as you are basically asking the app to do 1000 individual activities on your app. So it would slow down the app a little bit for your regular users until those all completed and it also introduces the possibility that the app gets so busy that some of the records failed to get added, and the pipeline has a hard error for some of the records.
In your case, the more complicated method is not necessary, so I just offered that by way of education for maybe a future situation where there are thousands of records to be added or merged into a table.
Mark
