Forum Discussion

AaronB's avatar
AaronB
Qrew Trainee
9 months ago

Back-to-Back Pipleine calls: Do they run in parallel or sequentially?

I am trying to simplify some of my more lengthy pipelines by breaking them up into logical sections and then calling each one separately.  Basically:

  • Parent Pipeline
    • call pipeline 1
    • call pipeline 2
    • call pipeline 3

They need to run sequentially. However, I just did my first run and based on the view windows for the pipelines involved, it appears they run in parallel.  As soon as the parent pipeline kicks off pipeline 1, it immediately kicks of pipeline 2, and then pipeline 3, without waiting for the previous pipeline to finish.

I could not find any documentation or previous posts on how these callable pipelines run so was wondering if anybody else could shed some light on this. 

Thanks....Aaron



------------------------------
Aaron B
ab1692@att.com
------------------------------

3 Replies

  • They will only run  sequentially if you have the last step of the first pipeline call the second pipeline. Calling a Pipeline will cost you a step run, so unless you don't care about step runs based on your plan, or you don't need to pass any values from one to the other, you can just say have a date time field on an admin record and use that to trigger the pipelines. 



    ------------------------------
    Mark Shnier (Your Quickbase Coach)
    mark.shnier@gmail.com
    ------------------------------
    • AaronB's avatar
      AaronB
      Qrew Trainee

      Thanks for the clarification Mark.

      What's interesting is this......based on what I was seeing during all the pipeline tracing text as it was getting spit out to me, I was expecting the data to be total trash.  Surprisingly, the data looks pretty good even though I am seeing a few minor glitches.

      This is just a theory but.....even though the pipeline tracing text makes it look like things are getting executed out of sequence, the database record 'writes' may be happening much faster in the background and are only minimally impacted by the timing/sequence.  Again, just a theory that may be totally incorrect.

      It hadn't even crossed my mind to have pipeline 2 call pipeline 3 (instead of having it called from pipeline 1) so that will be helpful.

      One last question.....You mentioned..."....or you don't need to pass any values from one to the other".   This is exactly the case for what I am doing.  How does this not count against our step runs?

      Thanks....Aaron



      ------------------------------
      Aaron B
      ab1692@att.com
      ------------------------------
      • MarkShnier__You's avatar
        MarkShnier__You
        Icon for Qrew Legend rankQrew Legend

        Sorry, I realize that they will fire sequentially, but I have no idea if it will wait for the first one to complete before firing the second one.

        As for my comment, I was just meaning to say that a Callable Pipeline does have the feature to be able to pass values along to the next "called" Pipeline.  



        ------------------------------
        Mark Shnier (Your Quickbase Coach)
        mark.shnier@gmail.com
        ------------------------------