Forum Discussion

Re: Pipelines recently very very slow?

Hi Everyone,

In the last week we saw an uptick in performance pain and performance cases around Pipelines and wanted to provide some information here in this thread. It can absolutely be frustrating to have your automated workflows delayed and it is something Quickbase takes very seriously when we get that feedback and see cases of customers in pain. Over the last 12 months the usage of Pipelines has grown by over 5,000%, which is great cause it means we have opportunity to pour even more innovation and enhancement into it but has come with some growing pains. The good news I wanted to pass on is that, as we speak, our Product team is working on several major enhancements to increase the performance & execution of Pipelines.

The biggest improvement is focused on moving all customers to a new behind-the-scenes engine (named "Blaze Engine") that processes the rules and logic of each Pipeline up to 5x faster. We expect to have this new engine rolled out to all customers by our September 2021 product release. For those of you who are still having performance pains right now if you haven't submitted a case yet with our Care team we do recommend you do so as they can look at your specific experience and ideally provide you some steps to help with making your experience better today. There are often things we can do today to help alleviate slowness.

For those of you working with Pipelines and looking for a little help on Best Practices to set them up with an eye to performance we do have some resources available when building Pipelines, especially when building Loops or handling lots of records in bulk. We recommend reviewing our free on-demand video content from Empower 2021 about Optimizing your Pipelines (9 minutes) and Troubleshooting your Pipelines (40 minutes). When you're trying to do bulk-record operations (something to more than 100 records), you should make sure you use these 4 steps, in this order: 

 A. Prepare Bulk UpsertB. Search RowsC. Add Row to Bulk Upsert, and D. Commit Upsert

This optimizes for the best performance and we are working up a guide that will go over setting this up in more depth right now. Many of these Best Practices will also appear on the "Pipelines Certification" we are launching later this year... We hope this information helps you, and we look forward to continuing to improve your Quickbase experience each and every day. I want to thank everyone for sharing their experience and being open with your frustrations. We want to make sure we can handle not just this massive growth but another 5,000% increase and that is only possible by hearing your feedback and incorporating it into our plans.

------------------------------
Evan Martinez
Community Marketing Manager
Quickbase
------------------------------

9 Replies

  • MichaelTamoush's avatar
    MichaelTamoush
    Qrew Captain
    Evan,

    This seems like a boiler plate response with no real solution. They have moved a number of my pipelines to the new Engine, and I see marginal or no improvements. Telling developers to try to build more efficient Pipelines is great, but when a single 'if' statement takes 2-5 minutes to process, it doesn't matter how efficient the pipeline is, it will be so slow it is unusable.

    I could understand if the pipelines were previously taking 5 seconds, and are now taking 45 seconds. The solutions you propose would be fine. But that is not the case. Pipelines that previously took 5 seconds are taking in some cases over an hour!!!

    Even if you achieved 5 times faster that is still 12 minutes+ for one run!

    You are telling us Pipelines has grown too fast for you to keep up with, yet also are forcing everyone to move their automations to Pipelines.

    If other developers are reading this and share my frustration, does anyone have ideas of what we can do to combat this?

    ------------------------------
    Mike Tamoush
    ------------------------------
    • EvanMartinez's avatar
      EvanMartinez
      Qrew Elite
      Hi Mike,

      I'm sorry that you are still feeling frustrated and that is absolutely understandable given the slowness you are experiencing. You are correct that in some instances there are Pipelines still experiencing slowness right now and that is something that needs to be adresses. I'm hoping it might help if I clarify a bit on the engine and why it might help a little right now but will help more in the long term. It actually would have the advantage long term of not just speeding up individual Pipelines so that they themselves run faster but we are also working to address how many can run concurrently.

      You are probably familiar with the concept that Quickbase apps often exist in server with several other applications and on occasion an application can feel pain when a lot of requests get lined up and have to be processed in order. While the engine that runs Pipelines is more complex than one line it does still have a limit on its queuing and essentially how many Pipelines get attention at once, in the case of our growing environment and high traffic that can result in slowdowns like we are seeing this last week. We aren't just going to get you through the lines faster by speeding up your individual Pipelines we are also working on opening up more lanes so that you feel less impact due to traffic. So the engine itself will help things run more quickly, best practices will help them be more efficient with that time, and then increased capacity and load balancing improvements will help open up access to resources and more smartly share them where the need arises.

      Being able to really focus on this work long term is part of the reason for the Automations end of life program, trying to make both platforms meet their growing demands would have meant not being able to commit more to one without bigger sacrifices in customer experience. Before the announcement since Automations was very familiar to builders its usage was also growing alongside Pipelines meaning both platforms would have needed this kind of performance attention to avoid slowdown and other issues. 

      It is important for us to hear about your frustrations and experiences though to help push those priorities and innovations. When we hear this dissatisfaction we want to make sure we are communicating what we are doing about it even, even when parts of it are more long term.

      ------------------------------
      Evan Martinez
      Community Marketing Manager
      Quickbase
      ------------------------------
      • BusinessIT's avatar
        BusinessIT
        Qrew Member
        Hi Evan,

        Thanks for the detailed update and explanation. 

        For my pipeline, the move to the new engine resulted in great performance. I am seeing it run in 12-15 seconds which is fantastic. Thanks for the continued focus on performance as the usage grows. 

        Robert

        ------------------------------
        Robert
        ------------------------------