Forum Discussion

MikeTamoush's avatar
MikeTamoush
Qrew Commander
2 years ago

Deleting bulk records with pipeline

I know using a bulk record upsert is a more efficient way to add or modify records using pipelines. Is there a best practice when deleting? Or just use the search function, and then a for each loop?



------------------------------
Mike Tamoush
------------------------------
  • It looks like you use an Advanced Query to define the records to be deleted.  You might not want to "blow" the Query Syntax and accidentally zap all your data, so be careful.  I imagine that it's super fast to purge the records and much less load on your app than one by one.

    https://helpv2.quickbase.com/hc/en-us/articles/4418021154708-Quickbase-August-2021-Release-Notes



    Channel enhancement - Remove records in bulk

    We have introduced a new step which speeds up the delete process for a large number of records. This can help you out when you're migrating automations.






    ------------------------------
    Mark Shnier (Your Quickbase Coach)
    mark.shnier@gmail.com
    ------------------------------
    • MarkShnier__You's avatar
      MarkShnier__You
      Icon for Qrew Legend rankQrew Legend

      By coincidence I had an opportunity to want to purge child records when a parent was deleted. I used the remove record step and based on the timestamp of the pipeline steps, the pipeline triggered and then one second later 608 children were deleted.  



      ------------------------------
      Mark Shnier (Your Quickbase Coach)
      mark.shnier@gmail.com
      ------------------------------
    • MikeTamoush's avatar
      MikeTamoush
      Qrew Commander

      I found that feature and have cautiously tried it (first in a test table of course). It seems to work using the same syntax as a formula query. It zapped all my records (thank goodness the correct ones) very quickly. Only thing I found is the pipeline never says 'pipeline finished', so i am having customer service look at that part. It instantly works as expected, then just spins...and never says finished. I am trying to find out if that is taking up any resources. 

      I can't help but laugh a little at your scenario. I once deleted a table by messing up an automation, but instead of zapping instantaneously, every time I refreshed the page less records where there. After about 100 deleted records my mind understood what was happening (my mistake in my filter) and I was screaming at the computer 'STOP AUTOMATION PLEASE STOP'. Needless to say, the automation did not stop.



      ------------------------------
      Mike Tamoush
      ------------------------------
      • MarkShnier__You's avatar
        MarkShnier__You
        Icon for Qrew Legend rankQrew Legend

        A suggestion to ourselves would be that when setting the Advanced Query to delete, to come up with a native Quickbase checkbox field which does any calculations on eligibility so that the syntax for the pipeline Query is simple stupid.  



        ------------------------------
        Mark Shnier (Your Quickbase Coach)
        mark.shnier@gmail.com
        ------------------------------