Undocumented Pipeline yaml functionality

Question

Pipeline yaml files can apparently use at least some of yaml's advanced features.&nbsp; I was able to use "yaml references" in my pipeline spec to avoid repeating long strings that might be prone to typos. specifically:

- IF:
&nbsp; - AND:
&nbsp; &nbsp; - a&lt;&gt;body_params.partner starts c1
&nbsp; - THEN:
&nbsp; &nbsp; - ACTION quickbase[abcdefg] bulk_record_set csv_import -&gt; c:
&nbsp; &nbsp; &nbsp; &nbsp; inputs-meta:
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; csv_url: '{{b.file_transfer_handle}}'
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; header_row: &amp;headers A,REALLY,LONG,LIST,OF,HEADERS

- IF:
&nbsp; - AND:
&nbsp; &nbsp; - a&lt;&gt;body_params.partner starts shi
&nbsp; - THEN:
&nbsp; &nbsp; - ACTION quickbase[bcdefgh] bulk_record_set csv_import -&gt; e:
&nbsp; &nbsp; &nbsp; &nbsp; inputs-meta:
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; csv_url: '{{b.file_transfer_handle}}'
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; header_row: *headers

The&nbsp;&amp;headers creates a yaml reference to whatever follows.&nbsp; *headersuses that reference.

Yaml references are kind of obscure and a bit tricky to use in my past experience, but they do work.&nbsp; Unfortunately, the references are not preserved: When you export the pipeline after using them, the export will have A,REALLY,LONG,LIST,OF,HEADERShard coded wherever the&nbsp;*headers reference was used.

It would be nice if QB would publish a spec for these yaml files along the lines of Gitlab/Github/AWS CodeBuild/ etc.------------------------------Dan Locks------------------------------

danlocks · Answer

A best practice across all programming disciplines is to avoid repeating identical lines or values when writing code. In my case, I have 3 tables with identical columns but differing data. The "import with csv" action looks *almost* the same for each table.&nbsp; This yaml syntax allows me to write the value for `header_row:` once and define a reference to that value in-line. Any time I need that same value, I can use the reference instead of the actual value.
Keep in mind this is a purely textual modification to the yaml, and after the yaml is loaded, the resulting values are hard coded again.&nbsp; After loading a pipeline with yaml references, you could edit the pipeline in the UI to have different `header_row` values.
I am using yaml references to define the fields i import, however this simple reference is insufficient to create a valid "import with csv" block.&nbsp; Each field must also have mapping in the form of `qb_field_for_a` then `qb_field_for_really`, then `qb_field_for_long`... one for each field listed in `header_row`.&nbsp; The value of the mapping is the field ID for the destination table.&nbsp; For me, the destination field ID is not consistent, so I can't simply repeat all the `qb_field_for` using yaml references, because the values differ.

Honestly, it's not clear how useful it is, but I figured someone might get some use.------------------------------Dan Locks------------------------------

justintorrence · Answer

Nice find!I'm curious if the references are preserved in the "Pipeline Summary" in the Activity Log.
Also, I fear that they might also be lost if you use the Refresh Schemas button:
------------------------------Justin TorrenceQuickbase Expert, Jaybird Technologiesjtorrence@jaybirdtechnologies.comhttps://www.jaybirdtechnologies.com/#community-post------------------------------

donlarson · Answer

Dan,

Would you please expand on the functionality this provides.&nbsp; &nbsp;

I think you are using it to define the fields that you are importing to with the CSV.
------------------------------Don Larson------------------------------

donlarson · Answer

Dan,

Thanks for introducing something new.&nbsp; You have me thinking about it.

Don------------------------------Don Larson------------------------------

danlocks · Answer

There are a few other automation systems out there that use yaml files for specifications (kubernetes, AWS CloudFormation, Terraform, Gitlab, AWS Codebuild, probably others) Most of them seem to use templating to pre-process the yaml similar to what I suggest here.&nbsp; Some use jinja for their templating system.&nbsp; These systems publish a rigorous spec for the yaml documents and don't provide a pretty GUI.
It would be great if QB would publish a spec and treat the yaml as the primary source for pipelines.&nbsp; I think the GUI gets in the way of managing pipelines of any complexity at all.&nbsp; Granted, I am quite new to QB, so take my opinions as uninformed.------------------------------Dan Locks------------------------------

Forum Discussion

Undocumented Pipeline yaml functionality

5 Replies