Forum Discussion

TJTaylor's avatar
TJTaylor
Qrew Member
2 months ago
Solved

Pipeline Help

I am working on connecting MethodCRM to Quickbase so that I can pull field data from a MethodCRM form and replicate it in a Quickbase form. I have attached images showing my current pipeline setup. The challenge is building the URL for retrieving records. Right now, the URL will pull the first 100 records from MethodCRM, but I have to manually change the URL to get the next batch of records. Example: rest.method.me/api/v1/tables/Contacts?skip=10&top=10

This works, but after pulling the first 10 records, I want the pipeline to automatically update so the next run changes top=10 to top=20 (and so on), allowing the pipeline to continue pulling subsequent batches without manual edits.

Pipelines
  • hueyal's avatar
    hueyal
    2 months ago

    I'm not sure if I am totally following the flow; however, in your callable pipeline, can you add skipValue and topValue arguments to the call definition?  Then in the URL, use those argument values:

    rest.method.me/api/v1/tables/Contacts?skip={{a.skipValue}}&top={{a.topValue}}

    You would need to keep track of where you are in the process to pass those values to the called pipeline.

3 Replies

  • hueyal's avatar
    hueyal
    Qrew Cadet
    2 months ago

    I'm not sure if I am totally following the flow; however, in your callable pipeline, can you add skipValue and topValue arguments to the call definition?  Then in the URL, use those argument values:

    rest.method.me/api/v1/tables/Contacts?skip={{a.skipValue}}&top={{a.topValue}}

    You would need to keep track of where you are in the process to pass those values to the called pipeline.

    • TJTaylor's avatar
      TJTaylor
      Qrew Member
      2 months ago

      This seemed to have worked thank you for the input!

  • Mez's avatar
    Mez
    Qrew Captain
    2 months ago

    How is your pipeline called, and what are the inputs, if any? 

    I looked at the documentation real quick. What you could do is if there is a "nextLink" value in the response, then pass this to this pipeline for processing. You may have to use jinja to extract or you could blindly trust the url and just use it.