Keeping a Production and QA/Staging App in Sync?

Question

Hi everyone,&nbsp;
So in my world we have a concept of Production and Staging.
In production, changes are done using the Sandbox, while new pipelines are built in staging (since pipelines can't be used in the sandbox).&nbsp;
Main issue I've run into , that cause me some headaches:&nbsp;

I can't take my schema changes from the production app sandbox and duplicate them in staging. I have to manually make the same schema changes in 2 apps to maintain consistency during development to ensure people can test the changes End to end.
Developing pipelines in the "staging" app, and moving to a production pipeline (that works against the production app!) requires a deep audit of the pipeline, since the staging and production apps aren't identical in field ids (sometimes) , and the account slugs / table ids between Production and Staging are different. Generally easy to change via the YAMLs, so far less of a headache then&nbsp;#1

I've experimented with a couple things, but feel I am missing a key component Quickbase has that helps keep 2 apps in sync?&nbsp;

I don't want to marry prod data with Staging - so connected tables I feel won't work...?&nbsp;
Copy table from prod to staging, won't work since staging has table in identical names (so I can delete and copy from production? that doesn't sound desirable)
Neither those table movement solutions keep table to table-relationships&nbsp;

What seems like the easiest solution albeit not easy is:

Copy the production app (without data) after every release, and make a new staging, I still have the issue of having to update all the pipeline slugs, and tables referenced in my old "staging" pipelines....while losing all the staging test data. This gets cumbersome with over a dozen pipelines. But seems like the best solution for now.....

Thanks for pointing me to any viable alternative solution, or at the very least sharing some ways to accomplish this.&nbsp;------------------------------Jamie Gray------------------------------

chayceduncan · Answer

Open forum to everyone else's best practices, but in general I have the same approach and general annoyance with what you're describing. For the bulk of our production apps I follow the same approach you have in that staging is a copy of the production app and all development done in Prod or approved from Staging must be duplicated/reworked in the other application. I almost never use the out of the box sandbox that QB provides given the lock that it places on Prod and the short term / all or nothing nature of the environment.&nbsp;
Our implementation process is to develop everything in staging first so the two environments stay in sync and have minimal deviation and then port to production once approved and validated.&nbsp;
To my knowledge there is no way to truly do a prod/staging/sandbox setup in the classical IT way of thinking about it.&nbsp;
Typically our sandbox/development environments live for 3-6 months so while its a headache, I've found that being diligent about keeping the two environments in sync allows that timeline to extend, but this is a core feature that needs some focus for larger/more controlled enterprises.&nbsp;------------------------------Chayce Duncan------------------------------

adamkrzyzanek · Answer

I use similar approach but we have 3 instances:
1) Production
2) QA - Ideal copy of production, after Development completed in DEV we first move it to QA to see if all steps were captured during development, we test it here and only after success move it to PROD.&nbsp;
3) Dev&nbsp;- Fresh copy of App (sometimes it is multiple Development Environments)
It created more work as we need to move all Developments and Pipelines multiple times but at least it is save for end user.Challenges are not only for Pipelines but also for Formula Queries, as field ID may vary.
I am not using Sandbox as it block production from any editing, and before I do big development, I may have a need for smaller releases 😀------------------------------Adam Krzyzanek------------------------------

chayceduncan · Answer

This isn't much help - but if you have a robust and interconnected environment, there are some helper tools that you can setup to make it easier as you transition between new staging environments.&nbsp;

I think there is an app on the exchange, or if not its not hard to build a simple searchandreplace setup where you can copy YAML and plug in the old / new params and spit out the new one as a way to eliminate potential manual error.&nbsp;

If you've got apps that have connected tables that you need to remain in sync in staging, its simple enough to build a data model that you can run through a pipeline where you basically recreate the idea of a native QB Connected Table in its components. Just have table of 'syncs' where you tell it what DB to pull from, the clist for fields and query param and then where / what fields to load it to and you can make a simple Pipeline to do a genresults csv pull and load it. This helps when you QB converts the connected table to basic setup when you make copies.&nbsp;

------------------------------Chayce Duncan------------------------------

gary1 · Answer

My hope is that the newly released Solutions module for versioning of an environment (apps, pipelines) will eventually allow you to deploy a new instance of a Solution. Right now Solutions are basically snapshots of an environment that allow you to rollback, but it shouldn't be a far reach to use the Solution for deployment.

------------------------------
gary
------------------------------

chayceduncan · Answer

I agree with Gary's sentiment, it's a step in the right direction but still a long way off from a true staging environment with whats been released so far.

------------------------------
Chayce Duncan
------------------------------

Forum Discussion

Keeping a Production and QA/Staging App in Sync?