Automation that deletes old data from prior automation runs

Question

I am running an automation that dumps data from one table in to another table. When I run an automation I need to have another automation delete all prior data in the table so that only data from the new automation run exists. I only want the data from the most recent run to exist in the table. What does the delete data automation look like?

------------------------------
George Bramhall
------------------------------

markshnier__you · Answer

Will there always be less than 1,000 records to be deleted?

------------------------------
Mark Shnier (YQC)
Quick Base Solution Provider
Your Quick Base Coach
http://QuickBaseCoach.com
mark.shnier@gmail.com
------------------------------

georgebramhall2 · Answer

Hi Mark,If I trigger the Automation, then less than 1000 records need to be deleted. I have not tried this automation in a daily schedule for all records in the parent table, but is quite likely at some point we could be deleting over 1000 records at once in the receiving table.------------------------------George Bramhall------------------------------

essenceqb_admin · Answer

Hey Mark, Do you have a solution for both? I just ran into the 1000 record restriction and need ideas for solving this issue.As for the deletion automation, IF it is less than 1000 records, and your automation is running on a set schedule, you could run the deletion automation based on the date the records were created a set number of days in the past. It would look like:
Date Created -&gt; Is Equal To -&gt; The Date Value -&gt; &lt;input number of days&gt; -&gt; Day(s) in the pastIf it is more than 1000 records, I would love to here some solutions.------------------------------Ray MossEssence QB AdminEssence Cabinets Inc.Edmonton, AB, CAN------------------------------

georgebramhall2 · Answer

Thanks Essence. Since my users can run a triggered automation more than once in a day the solution above does not work here. I have to have the latest automation records in the table only. I can't use the Max Summary (date/time) to filter because the records added by the automation potentially can run across a different time stamp (usually only by a minute or so).

------------------------------
George Bramhall
------------------------------

markshnier__you · Answer

Deleting using an Automation is not practical for high record counts unless high means less that 1,000 and its in the middle of the night.&nbsp; They delete at a slow rate of about 5 per second at most.&nbsp; So deleting 1,000 records would take 13 minutes.&nbsp; &nbsp;If a user is expecting a report to be empty after a button push, then that is too slow.We have an additional tool now in Piplines.&nbsp; We still cannot deleted a single record at a time as it would probably be too slow, but more importantly we ned to think about Step Runs. No sense to blow 1,000 step run on deleting one by one.So I would have the Pipeline call a webhook which would use the API Purge records&nbsp; API.&nbsp; I would probably make a relationship between the old and new table so I'm 100% sure that I'm not deleting records which were not successfully copied across.I would also make a nice clean checkbox field in Base called [OK to Purge], and then allow any logic in the pipelin to be as stupid simple as possible.&nbsp;So yes it does mean the need now to learn Pipelines. I have just started to use them successfully in the last 2-3 weeks, including for two real customer situations to send timed, mid-day emails., so not just practice., Fortunately there are some session at next week's EMPOWER on Pipelines "101 and 201", ie "intro to" and then a bit more advanced.------------------------------Mark Shnier (YQC)Quick Base Solution ProviderYour Quick Base Coachhttp://QuickBaseCoach.commark.shnier@gmail.com------------------------------

Forum Discussion

Automation that deletes old data from prior automation runs

Related Content

Delete Formula Button Error

Automating Manager Approval via Email

Pipelines automation to change record owner

Deleting Orphaned Children (records)

Update on the Retirement of Automations, Actions, and Webhooks features