Discussions

 View Only

Getting App Data (Not Files) To S3: How Would You Approach?

  • 1.  Getting App Data (Not Files) To S3: How Would You Approach?

    Posted 04-29-2021 15:44
    We have a reasonably sizable QB app with tables having >200K records.
    One of the things we need to do is take some incoming data from multiple sources and then based on what we have in QB, decide what we need to add (or modify).

    In short, if I could get data into storage like Amazon S3, we know what to do from there.  This transfer would run once per day.
    But, only need a few fields out of each table. 

    Is there a clean way to get data over there without having to write custom code that grabs data via the API (but has to deal with record size limits), and then transfer to Amazon S3?   If you can point me to some top level concepts to look at, then I am all ears.

    Thank you.

    ------------------------------
    Chris Anderson
    ------------------------------