Discussions

 View Only
Expand all | Collapse all

Find and append new unique values to secondary table

  • 1.  Find and append new unique values to secondary table

    Posted 12-14-2017 21:29
    I have an app where I run a daily upload of data (much of it duplicate due to the nature of the source), but sometimes my daily upload includes a new record that needs to be added to a secondary table within the app. The key field in the secondary table matches one of the fields in my daily upload. I have set up a QB action that should identify new unique values in the field related to the secondary table's key and copy these records. But I am running into a problem. Sometimes my upload will include several rows with the same value in the field that I want to use to populate the secondary table's key field . When this happens, my action is registering an error. I only need the value in one of the duplicate lines to write to the secondary table, but I need to preserve all of the lines in the primary table. Any suggestions?


  • 2.  RE: Find and append new unique values to secondary table

    Posted 02-16-2018 22:22
    If you have the ability, I'd use a webhook to a script that will query the table, and then do a single import based on a the combined information, thus limiting the duplicates.


  • 3.  RE: Find and append new unique values to secondary table

    Posted 03-23-2020 05:53

    Did you manage to solve this problem?
    I have similar one while using Automations.

    What I want to do on a daily basis run automation to copy all not existing data in Table 2 from Table 1


    Daily automation to copy Unique value from Table 1 to Table 2


    ------------------------------
    Adam Krzyzanek
    ------------------------------



  • 4.  RE: Find and append new unique values to secondary table

    Posted 03-23-2020 06:58
    I too am looking to so,be that same problem which is a feature needed in so many applications where we need to auto create parents.  I'm hoping that there is a solution using Pipelines, but so far I have not been able to crack it.

    ------------------------------
    Mark Shnier (YQC)
    Quick Base Solution Provider
    Your Quick Base Coach
    http://QuickBaseCoach.com
    mark.shnier@gmail.com
    ------------------------------



  • 5.  RE: Find and append new unique values to secondary table

    Posted 03-23-2020 07:36
    If there is a way to do this with out scripts, I would love to see it.  I am sure there would be many other uses.

    ------------------------------
    Don Larson
    Paasporter
    Westlake OH
    ------------------------------



  • 6.  RE: Find and append new unique values to secondary table

    Posted 03-25-2020 06:40
    @Evan Martinez - Maybe any solution for us?

    Please vote here:

    https://quickbase.uservoice.com/forums/111823-quick-base-product-feedback/suggestions/40015141-copy-unique-data-to-other-table

    @Don Larson, @Mark Shnier (Your Quickbase Coach)​​​

    ------------------------------
    Adam Krzyzanek
    ------------------------------



  • 7.  RE: Find and append new unique values to secondary table

    Posted 03-25-2020 08:42
    I gave it 3 votes.  There are a couple of Pipeline Webinars coming up, plus another opportunity to learn more about Pipelines later in April.  Maybe I will be smarter after I attend them :).  But I'm concerned that even if there is a Pipelines solution it may be too "step intensive .. we will see.

    ------------------------------
    Mark Shnier (YQC)
    Quick Base Solution Provider
    Your Quick Base Coach
    http://QuickBaseCoach.com
    mark.shnier@gmail.com
    ------------------------------



  • 8.  RE: Find and append new unique values to secondary table

    Posted 03-26-2020 20:50
    Adam,

    I have voted!   

    Lets see if the development team picks it up.

    ------------------------------
    Don Larson
    Paasporter
    Westlake OH
    ------------------------------



  • 9.  RE: Find and append new unique values to secondary table

    Posted 03-30-2022 18:36
    Did anyone ever solve this issue??

    ------------------------------
    Mike Tamoush
    ------------------------------



  • 10.  RE: Find and append new unique values to secondary table

    Posted 03-30-2022 19:30
    You can use a formula query to detect duplicates in the upload table and only upload the first duplicate.  So you would import in a two step process. First step is into a temporary scratchpad type import file.  Then run the import.

    Make an admin record with one record on the table.

    Make a button there to clear the scratch pad (API_PurgeRecords.  Make a summary field of the # of records in the scratchpad.  Show the button if there are records to clear.

    Make a button to call the import page.
    urlroot() etc .......?a=ImportExport

    https://yqc.quickbase.com/db/bh4pwztea?a=ImportExport​

    Make the formula query to flag duplicates which are not the first duplicate.

    Make a saved T2T import

    Make a button to run the import  API_RunImport&ID=10.

    Since the scratchpad fields are totally in your control, th field names will match the import sheet exactly

    Workflow;
    1. Click to clear
    2. Click to initiate import screen and fields line up so click to import
    3. Click to "commit" the scratchpad into the main table.

    ------------------------------
    Mark Shnier (YQC)
    mark.shnier@gmail.com
    ------------------------------



  • 11.  RE: Find and append new unique values to secondary table

    Posted 03-30-2022 19:44

    I like this, but the trouble I am running into is querying for duplicates. I keep getting the 'this process will take too long'. Perhaps it is because my table has 50,000 records and growing....

    If I could query, i feel like i could somehow just get a list of all record IDs that are duplicates, and have it find the minimum record ID. Then formula checkbox, is this the minimum record ID. Then only that one is checked and can T2T import off checked boxes, and where parent does not exist.

    My current solution is a pipeline that runs on create record. It does a quick search to see if the parent exists (i cant rely solely on a field showing it exists because everything fires so quickly, so it's a race - one of them hits and creates first, the rest just say oh, it exists, ill do nothing).

    This works for me, but I suspect it is because my imports won't be super huge, maybe only 100. Not sure how many pipeline calls i can trigger all at once with an import...?



    ------------------------------
    Mike Tamoush
    ------------------------------