Quick Base Memory Calculations

I've consistently been plugging the QuickBase Under the Hood session from Empower 2018 to builders looking to learn more about what is happening behind the scenes with Quick Base. The feedback from attendees is that they enjoyed learning about what makes Quick Base tick. With how successful it was, we have continued to expand on that content by sharing use cases, examples, and learnings in our blog and the Quick Base Community. Continuing to look for new opportunities to share that deeper insight, we thought it might be helpful to go through a thought exercise on a sample application as it relates to resource utilization.


To give some background, Quick Base is an in-memory database. This makes it unbelievably powerful doing complex calculations on operational data in milliseconds. What this means is, rather than Quick Base having to talk to a hard disk to retrieve data like a traditional database, it reads and writes from/to RAM. Let's use an analogy. On your computer, you might have a hard drive, with say 500gb, that has many programs, pictures, videos, etc. Now, if you decide to open a large program or intense game, it might take anywhere from 2 to 10 seconds for your computer to read all this information and process it. Now, once that application/game is in memory, you can generally minimize and restore it freely and quickly in far less than a second. That is why Quick Base is so fast it has all your application's data in memory and can quickly recall it, including calculated data. This isn't a perfect analogy when comparing the power of servers to personal computers but the principle is still the same.

Let's look at an example. These are some fake numbers and data structure for the sake of simplicity and to illustrate the scenario.



So, looking at the above, we can see the text entered into the "Notes" field is consuming about 100mb of data.

Side note: If we were looking to reduce how much data was taken on the Projects table, we could look at some tactics such as programmatically moving the notes data to a file attachment field.

Aside from that ~100mb of data consuming "storage", we also will keep it in memory. Now, this is Quick Base so there is a good chance the customer is going to use this field in some logic, lookups, summaries, formulas and reports down the line to get even more value out of it. With that end goal in mind let's add a formula text field that previews the first 100 characters in the field. This enables reports to have a field that shows a portion of the notes, without having the whole text there which might make the report unwieldy.




Now, this is where things get tricky. If we were to pull a report of all 50,000 records and included the "Notes Preview" field, roughly 7.5mb of memory would be used at face value. But for us to calculate the length and parse the characters, we need to read in the entire Notes data at least temporarily.

What if we then pass in the "Notes" and the "Notes Preview" into a child table, called Tasks.




Side note: This is memory (RAM) usage, not app space. Therefore, it does not count towards any table size limits or your account limits. As well, while there are warnings and thresholds behind the scenes that our great operations group use to ensure application and platform stability, these are far higher than most of our customers will ever hear about, let alone be impacted by.

We are now talking potentially about 1gb of RAM used solely to handle these project notes and this was a relatively simple example. It wouldn't be uncommon in Quick Base for us to see these notes going through multiple relationships, used in many more formulas, summary fields and even permissions. For example, say you only wanted to grant permissions to a certain role when the notes contained a keyword. Quick Base lets you do that, orders of magnitude simpler than other technologies. You can see how much memory QuickBase will need to use to accomplish these requirements.

So why are we sharing all of this? We don't want our app builders counting kilobytes or megabytes. That is overkill. What we think is important is that Quick Base is providing an unparalleled amount of flexibility and value with relatively little effort and without the deep technical understanding that other platforms or tools would require. But, with great power comes great responsibility. Adding these lookups, formulas and other complex calculations is extremely powerful. Understanding, at a very high level, what comes along with that can help builders make informed decisions during application design and development and can help during regular maintenance and clean-up. 



Recommended Reading: https://community.quickbase.com/quickbase/topics/quick-base-makes-complex-logic-simple