Skip to main content
Question

Pipeline error: Formula Query Execution took too long

  • Sep 19, 2023 5:24 PM UTC
  • 0 replies
  • 538 views

Forum|alt.badge.img+6

I found some posts similar to this regarding reports but none for Pipelines so here is my situation.  I've also run the Performance Analyzer, Performance Insights, and Performance Optimizer but not have provided any advice other than "optimize your query". 

When running pipelines, I get 3 outcomes:  Success without issue, Success with Heavy Traffic notifications, Failure due to the Formula Query Execution taking too long.

This happens on the exact same table, using the exact same pipeline (no differences between runs).

Success without issue happens on the weekends or middle of the night.

Heavy Traffic notifications seems to be hit or miss during the day on weekdays.

Failure due to Formula Query Execution happens during the weekdays during typical office hours.

Based on these scenerios, I am guessing it is a Quickbase "resource availability" issue????

The table is ~ 4000 records with 57 fields. 

Below is the Pipeline activity output for one of the failures.  Sometimes it makes it through nearly all records and then dies.  Other times it dies after the first few hundred.  This particular pipeline exports 20 fields but it also dies on similar pipelines for this table that export only 5 fields. 

The pipeline grabs all records in the table and shifts data between the 20 fields exported as part of this pipeline (data in fields 11-15 move to 16-20, fields 6-10 move to 11-15, fields 1-5 move to 6-10) 



------------------------------
Aaron B
ab1692@att.com
------------------------------
This topic has been closed for replies.

MarkShnierYou
Forum|alt.badge.img+24

What if you try a pipeline step to copy the value of the formula query into a scalar field.  Then use the scalar field in the real Pipeline. The copy could be done by making a bulk upsert and adding the Bulk upsert rows one by one, then commit the Pipeline.  maybe that way it only has to calculate the formula queries record by record and it won't time out.

You may be able to copy your existing pipeline and then use the new pipeline designer interface to insert steps before your main pipeline steps.



------------------------------
Mark Shnier (Your Quickbase Coach)
mark.shnier@gmail.com
------------------------------

MarceloBenavide
Forum|alt.badge.img+8

You are using a for-each to loop through the 4k records and make changes to each of the 20 fields in each iteration?



------------------------------
Marcelo Benavides
------------------------------

MarceloBenavide
Forum|alt.badge.img+8

 are you using a do-loop to loop through the 4k records and make changes to each of the 20 fields in each iteration?



------------------------------
Marcelo Benavides
------------------------------

MarceloBenavide
Forum|alt.badge.img+8

HI Aaron, I apologize, I was  not clear. My question is:  Are you  using a  do-loop (pipeline function) to  through the 4k records and make changes to each of the 20 fields in each iteration?

Thank you



------------------------------
Marcelo Benavides
------------------------------