Discussions

 View Only
  • 1.  Pipeline Jinja Issues

    Posted 03-13-2023 11:54

    Hello Quickbase Community!

    I'm having issues with a pipeline. 

    Step A: Quickbase Channel - Quickbase API - Make a Request

    Essentially, I query data in a table and get back some JSON data. This step works completely fine. 

    Step B (the problem step): Quickbase Channel - Quickbase API - Make a Request

    This request is to post data to a test table for now. I'm trying to run a for loop on the data. I'm eventually going to incorporate the myTimeStop variable, but it can be ignored for now. I'm trying to iterate through the data, adding a line for each one, and keeping a running total for row '33'.

    This works, but it's still resetting the runningTotal to 0.0 in each loop for some reason. Table ID redacted.

    {% set myDataSet = "myTableID" %}
    {% set myTimeStop = 5.0 | float %}
    {% set runningTotal = 0.0 | float %}
    {
        "to": "{{myDataSet}}",
        "data": [
            {% for row in a.json.data %}
            {% set runningTotal = runningTotal + row['33'].value %}
            {
                "6": {
                    "value": "{{row['36'].value}}"
                },
                "7": {
                    "value": "{{row['8'].value}}"
                },
                "8": {
                    "value": "{{row['13'].value}}"
                },
                "9": {
                    "value": "{{row['33'].value}}"
                },
                "10": {
                    "value": "{{runningTotal}}"
                }
            }
            {% if not loop.last %},{% endif %}
            {% endfor %}
        ]
    }

    I've tried this, but it gives me a vague error. It's pretty much the same as the top, but I'm appending each line to the list as later I want to iterate through the data until I hit the timestop variable and then add that list to Quickbase. 

    {% set myDataSet = "myTableID" %}

    {% set myTimeStop = 5.0 | float %}
    {% set runningTotal = 0.0 | float %}

    {% set data = [] %}

    {% for row in a.json.data %}
        {% set runningTotal = runningTotal + row['33'].value %}
        {% set item = {
            "6": {"value": row['36'].value},
            "7": {"value": row['8'].value},
            "8": {"value": row['13'].value},
            "9": {"value": row['33'].value},
            "10": {"value": runningTotal}
        } %}
        {% do data.append(item) %}
    {% endfor %}

    {
        "to": "{{myDataSet}}",
        "data": {{data|to_json}}
    }

    I'm just not understanding why it doesn't like the second variation. It passes the test initially, but when run gives a vague "Something went wrong executing a step. Try modifying your pipeline."

    Also, not understanding why in the first one it's not actually keeping a running total. 



    ------------------------------
    Dwight Munson
    ------------------------------


  • 2.  RE: Pipeline Jinja Issues

    Posted 03-13-2023 12:01

    For the second bit, I've also tried:
    {% set data = data.append(item) %}

    and

    {% set _ = data.append(item) %}

    and

    {{data.append(item)}}

    I've also tried: 

    {
        "to": "{{myDataSet}}",
        "data": {{data|tojson}}
    }

    and

    {
        "to": "{{myDataSet}}",
        "data": {{ data | tojson }}
    }



    ------------------------------
    Dwight Munson
    ------------------------------



  • 3.  RE: Pipeline Jinja Issues

    Posted 03-13-2023 16:52
    Edited by Don Larson 03-13-2023 16:53

    Dwight,

    I am no Jinja or Python expert, however I remember reading somewhere in a Python resource about declaring variables inside and outside of a loop.   You have defined Running Total twice and there is a voice in my head saying that your problem lies there.

    The two best resources for Jinja here are @Doug Henning or @Prashant Maheshwari 



    ------------------------------
    Don Larson
    ------------------------------



  • 4.  RE: Pipeline Jinja Issues

    Posted 03-13-2023 17:01

    This worked for me for the running total. Now I'm trying to figure out a while loop for it to only iterate up to the myTimeStop.

    {

    {% set myDataSet = "myTableID" %}

    {% set myTimeStop = 5.0 %}

    {% set runningTotal = namespace(val=0.0) %}

        

    "to": "{{myDataSet}}",

    "data":[

     

        {% for row in (a.json.data) %}

        {

            {% set runningTotal.val = runningTotal.val + row['33'].value %}

            "6":{          

                    "value": "{{row['36'].value}}"

            },

            "7":{          

                    "value": "{{row['8'].value}}"

            },

            "8":{           

                    "value": "{{row['13'].value}}"

            },

            "9":{          

                    "value": "{{row['33'].value}}"

            },

            "10":{          

                    "value": "{{runningTotal.val}}"

            }

        }

        {% if loop.last == false %},{% endif %}

        {%- endfor %}

    ]

    }



    ------------------------------
    Dwight Munson
    ------------------------------



  • 5.  RE: Pipeline Jinja Issues

    Posted 03-13-2023 17:03

    In the AM I'm going to try something like this: 

    {

    {% set myDataSet = "myTableID" %}

    {% set myTimeStop = 5.0 %}

    {% set runningTotal = namespace(val=0.0) %}

       

    "to": "{{myDataSet}}",

    "data":[

        {% set i = 0 %}

        {% while runningTotal.val <= myTimeStop and i < a.json.total_records %}

        {% set row = a.json.data[i] %}

        {% set remainingTime = myTimeStop - runningTotal.val %}

        {% set currentValue = row['33'].value if row['33'].value <= remainingTime else remainingTime %}

        {% set runningTotal.val = runningTotal.val + currentValue %}

        {

            "6":{          

                "value": "{{row['36'].value}}"

            },

            "7":{          

                "value": "{{row['8'].value}}"

            },

            "8":{          

                "value": "{{row['13'].value}}"

            },

            "9":{          

                "value": "{{currentValue}}"

            },

            "10":{          

                "value": "{{runningTotal.val}}"

            }

        }

        {% set i = i + 1 %}

        {% if loop.last == false and runningTotal.val <= myTimeStop %},{% endif %}

        {%- endwhile %}

    ]

    }



    ------------------------------
    Dwight Munson
    ------------------------------



  • 6.  RE: Pipeline Jinja Issues

    Posted 03-13-2023 22:47

    You are too kind @Don Larson !! 

    Doug Hemming is the absolute beast in Jinja !

    Dwight , you seem to have figured namespace which I would have recommended. Below is from on my notes I made few months back

    You cannot use a simple counter variables inside ninja loops. Case in Point
    ```
    {% set x = 0 %}
    Counter Value before loop {{x}}
    {% for myloop in range(6) %}
    {% x=x+1 %}
    {%end for%}
    Counter Value after loop {{x}} 
    ```
    You would expect the output  to be 
    ```
    Counter value before loop 0
    1
    2
    3
    4
    5
    Counter value after loop 5
    ```
    Instead what we get it is 
    Counter value before loop 0
    1
    1
    1
    1
    1
    1
    
    Counter value after loop 0
    Everything inside loop gets a fresh value of our counter X in case
    ---
    
    
    After some painful searching I stumbled upon a Ninja function called namespace for solving exactly the same issue
    ```
    {% set x = namespace(value=0) %}
    Counter Value before loop {{x}}
    {% for myloop in range(6) %}
    {% x.value=x.value+1 %}
    {%end for%}
    Counter Value after loop {{x}} 
    ```
    Now the output is
    ```
    Counter value before loop 0
    1
    2
    3
    4
    5
    Counter value after loop 5
    ```


    ------------------------------
    Prashant Maheshwari
    ------------------------------



  • 7.  RE: Pipeline Jinja Issues

    Posted 03-14-2023 15:40

    Thanks everyone! Great information in here!



    ------------------------------
    Dwight Munson
    ------------------------------