Recap

Quickbase takes reliability and performance very seriously. We know your applications are used around the world to help do mission critical work. To that end, we are always improving our infrastructure to support continued growth of the platform. In order to support the increased volume of traffic to our RESTful APIs, we began a migration in December to a newer and more scalable infrastructure. The new infrastructure has already improved performance by an average of ~30%, helping customers do more in less time.

We want to take a moment to offer some clarity on a few issues we have had so far and how we are rectifying them in order to keep our migration going as smoothly as possible. We make infrastructure changes, security upgrades, and conduct maintenance regularly with minimal disruption and our plan was for this migration to be the same - only improving performance. Extensive automated and manual testing was done in development and pre-production environments, with slow production rollouts starting with our internal realms first.

Rollout

Throughout the rollout, monitoring was in place to review an increase or decrease in specific traffic patterns. This monitoring was leveraged during the migration to detect issues and proactively implement resolutions before any customer impact was noticed. Early in the migration, a very limited number of customers had traffic inconsistently detected by our web application firewall, and additional security layers, as malicious. Our engineering and site reliability teams quickly validated that the traffic did not pose any threats, and manually removed the offending rules.

During the last phase of our rollout on the /records endpoint, our tech support team began getting cases that were difficult to troubleshoot and inconsistent with our testing and logging.

After deep investigation, we determined that the newer infrastructure was more strictly adhering to the API specifications than the previous infrastructure it was replacing. This was an unintentional change that we've traced back to a specific piece of technology now in use. While our testing covers many success and failure states, the extent of combinations of how the API can be consumed and the way a Quickbase application can be configured was not sufficiently covered. For example, a customer passing in a null value for a text field or a numeric field may expect that to blank out a value from a field, others expect it to do nothing. Similarly, the translation between integers and text in JSON cannot always be mapped 1 to 1 with Quickbase fields.

Identified Issues

Below is a list of issues relating to increased validation against our API specifications that we have identified and/or been made aware of and have already relaxed to match the previous behavior:

• null can be passed as a sortBy and groupBy parameters in queries.

• A trailing semicolon can be passed as part of the content-type header.

• https can precede the hostname, like https://foo.quickbase.com.

• null can be passed as a field value. The object will be ignored.

• If an array of records is submitted in the /records POST, and one or more objects is null, we will ignore those objects.
• mergeFieldID will accept a numeric value passed as text.

Please note: while the above are not part of the API spec, we are leaving these changes while we complete the migration. If we choose to roll them back to be more strict, those changes will be communicated within our release notes so builders and integrators have time to accommodate the changes. If we find additional areas that we determine need to be updated, we will communicate them here.

Additionally, the following bugs have been resolved in the new infrastructure to match our API spec:

• The content-type is automatically inferred as application/json when not passed. This was not being inferred after migration. Previously, customers received an unsupported media type error.

• Temporary tokens were unable to retrieve file downloads. This has been resolved.

Next Steps

We will be completing the migration of all endpoints and customers to the newer infrastructure over the coming days. We take seriously the quality and reliability of these services and are implementing further automated testing that increases our coverage of both successful and non-successful transactions. Specifically, we will be adding more combinations of field types, empty and null values, and missing headers. This increased coverage will allow us to better monitor any unintentional breaking changes to the API.

Getting Help

If you are experiencing an issue, or would like support on any topic surrounding APIs, please open a support ticket including an HTTP request/response (sanitizing all sensitive information and authentication), in line with our Extending Quickbase support article. Including the qb-api-ray (returned in the HTTP Response Headers), and the source IP address your calls are coming from, will greatly expedite investigation and resolution.

For more questions or further discussion, please reach out to your Quickbase account team.

------------------------------
Harrison Hersch
------------------------------