Forum Discussion
MarkShnier__You
Qrew Legend
5 years agoWill there always be less than 1,000 records to be deleted?
------------------------------
Mark Shnier (YQC)
Quick Base Solution Provider
Your Quick Base Coach
http://QuickBaseCoach.com
mark.shnier@gmail.com
------------------------------
------------------------------
Mark Shnier (YQC)
Quick Base Solution Provider
Your Quick Base Coach
http://QuickBaseCoach.com
mark.shnier@gmail.com
------------------------------
EssenceQB_Admin
5 years agoQrew Member
Hey Mark, Do you have a solution for both? I just ran into the 1000 record restriction and need ideas for solving this issue.
As for the deletion automation, IF it is less than 1000 records, and your automation is running on a set schedule, you could run the deletion automation based on the date the records were created a set number of days in the past. It would look like:
------------------------------
Ray Moss
Essence QB Admin
Essence Cabinets Inc.
Edmonton, AB, CAN
------------------------------
As for the deletion automation, IF it is less than 1000 records, and your automation is running on a set schedule, you could run the deletion automation based on the date the records were created a set number of days in the past. It would look like:
Date Created -> Is Equal To -> The Date Value -> <input number of days> -> Day(s) in the past
If it is more than 1000 records, I would love to here some solutions.
If it is more than 1000 records, I would love to here some solutions.
------------------------------
Ray Moss
Essence QB Admin
Essence Cabinets Inc.
Edmonton, AB, CAN
------------------------------
- GeorgeBramhall25 years agoQrew CadetThanks Essence. Since my users can run a triggered automation more than once in a day the solution above does not work here. I have to have the latest automation records in the table only. I can't use the Max Summary (date/time) to filter because the records added by the automation potentially can run across a different time stamp (usually only by a minute or so).
------------------------------
George Bramhall
------------------------------- MarkShnier__You5 years ago
Qrew Legend
Deleting using an Automation is not practical for high record counts unless high means less that 1,000 and its in the middle of the night. They delete at a slow rate of about 5 per second at most. So deleting 1,000 records would take 13 minutes. If a user is expecting a report to be empty after a button push, then that is too slow.
We have an additional tool now in Piplines. We still cannot deleted a single record at a time as it would probably be too slow, but more importantly we ned to think about Step Runs. No sense to blow 1,000 step run on deleting one by one.
So I would have the Pipeline call a webhook which would use the API Purge records API. I would probably make a relationship between the old and new table so I'm 100% sure that I'm not deleting records which were not successfully copied across.
I would also make a nice clean checkbox field in Base called [OK to Purge], and then allow any logic in the pipelin to be as stupid simple as possible.
So yes it does mean the need now to learn Pipelines. I have just started to use them successfully in the last 2-3 weeks, including for two real customer situations to send timed, mid-day emails., so not just practice., Fortunately there are some session at next week's EMPOWER on Pipelines "101 and 201", ie "intro to" and then a bit more advanced.
------------------------------
Mark Shnier (YQC)
Quick Base Solution Provider
Your Quick Base Coach
http://QuickBaseCoach.com
mark.shnier@gmail.com
------------------------------- GeorgeBramhall25 years agoQrew CadetSince deleting old data dumps by an automation is not an option, my alternative is to filter my data in the receiving table by only showing the last automation run leaving old automation runs from the old automation in the table but filtered out.
Problem with this is that my automation data dumps take so long that the each of the record the "date created" fields span as much as a few minutes apart so using the filter Max summary date created (look up) = date created does not pick up all of the data in the most recent scheduled automation run. Anyone have any tricks for a filter that will pick up all of the records in the most recent scheduled automation run?
------------------------------
George Bramhall
------------------------------