Forum Discussion

MikeTamoush's avatar
MikeTamoush
Qrew Commander
2 years ago

Querying a table with over 60k records

Is there simply no way to use a formula query on a table with 50k+ records? I've tried having a first query that would elimante all but a handful of options but I still get the 'will take too long error'.

Are queries just limited to small tables or is there a trick I am missing?

My use case is I need to see if a certain entry in one table, exists in another (by matching a couple fields).

I can't use a combined text because those are limited to a string of 150, and my string will be about 1500 entries, and i need to search through those 1500 specific entries to see if there is a match. I could use like 15 combined text fields and break it up but that sounds like too much work.

I could use a query but my query will need to parse 60k+ records and start eliminating.

------------------------------
Mike Tamoush
------------------------------
formula queries

4 Replies

Sort By
  • PaulPeterson1's avatar
    PaulPeterson1
    Qrew Assistant Captain
    2 years ago

    What queries have you tried so far?  What is the field type you are trying to query?

    The answer I always get from QB about formula queries is they don't work well for large data sets or tables with a large number of relationships.



    ------------------------------
    Paul Peterson
    ------------------------------
    • MikeTamoush's avatar
      MikeTamoush
      Qrew Commander
      2 years ago
      I have 3 fields I can use in my query. An 'Amount' field, which if I queried should probably return a max of 100 results, a project ID field which could return up to 1200 results, and another text field that would return like 20k results. But all 3 should only return a couple results max. So i tried to query the Amount first, then the project ID. I shouldnt even need the last field to be honest.  I also tried the reverse, query project ID then AMount.

      I am trying to query on a synced table, so I am thinking maybe that is it? But I thought synced tables were just seen as straight data, so maybe its just the 60k records.

      I cannot find a work around though for my issue. I might start a new thread with it.

      ------------------------------
      Mike Tamoush
      ------------------------------
      • PaulPeterson1's avatar
        PaulPeterson1
        Qrew Assistant Captain
        2 years ago

        There is a workaround that I used for a similar issue before formula queries were introduced.  I created a text field to mirror the numeric field and created a combined text field using the mirror.  It's not the most elegant solution, but it worked in the instance I used it.

        I haven't tried a formula query on a Sync table, and you're right, that may be part of the issue.



        ------------------------------
        Paul Peterson
        ------------------------------