Forum Discussion

hhersch's avatar
hhersch
Quickbase Staff
5 years ago

Updates on our RESTful APIs

Fine print: this post discusses Quick Base features and functions that are currently in development. Quick Base has no obligation to deliver these features and they may be unstable.
If you are not yet involved in our beta program for the RESTful APIs, please make sure you register based on the instructions in our release notes. We are slowly inviting people in. If you are exceptionally passionate about working with us on this topic and being part of the cutting edge in Quick Base, please reach out to your account team at Quick Base so that they get in touch with the right people on our product team to advocate for you.

Thanks to those of you that have been participating in this limited early access. We wanted to provide a brief update as to the progress.

First, I thought it was important to clarify that this is still a beta and not all features are there. We will be considering this production-ready in probably 6-8 weeks. That means the most commonly used APIs will be present to start and we will slowly burn down more and more based on our prioritization process.  The large bucket of APIs we will tackle as a fast follow to GA will be around user/role/group management. Our main focus though is quality and depth. We are not only making sure these APIs are ready for the future, but also looking back at nearly 20 years of feedback in the form of cases and bugs to make decisions on improvements.

Throughout the beta on these APIs, we have received some great feedback. So, first, I’ll address the quality of life items we have improved as a direct result of what you all told us. :

  • It was screaming loud and clear that we needed to make it simpler to deal with how you enter your realm. We will soon accept either "demo" or "demo.quickbase.com".
  • The middle pane bounces back to the top of the page as you click to other pages.
  • A few people got confused with the Content-Type header so we simplified it and just insert it for you if it isn't included.
  • You told us you needed more clarity on what page you were visiting in the portal. Done!
Additionally, it drove most of us bonkers that we had to type back in our testing values (like Authorization) on the right as we chose different API calls. A few of you mentioned that too, so we added some persistence there to make testing even easier.

Since our first post on the APIs, we added the ability to get field usage summary data back. And with the upcoming release, we are making those API calls significantly faster. For the past few weeks, the team has had two parallel efforts:

  1. More quality of life improvements: this means things like more detailed error handling, bug squashing, etc.
  2. Added functionality: net new endpoints.

Coming in, roughly, the next two weeks are the following goodies.

  1. The quality of life improvements and bug fixes mentioned above.
  2. Create, edit, and delete fields.
  3. Create, edit, and delete tables.
  4. A brand new temporary authentication mechanism that allows you to securely use the APIs from JavaScript without hardcoding credentials.
  5. (...drum roll...) Downloading batches of audit logs with some limited filtering.

One additional thing that is a new concept for us - we are exploring adding some new response headers that will indicate our rate-limiting policies and how many calls are left for what time period for the submitter. While we have always had rate limiting to protect our platform and our customers, we want to be transparent about those and help our builders create awesome experiences that factor in industry best practices. That includes retry and backoff processes. Rather than waiting for an error from us (an HTTP 429), you could monitor these headers and slow down calls if needed. You can even use these for logging and analytics. In practice, we expect these to help:

  1. ...our builders to be able to go the extra mile for an awesome user experience, using this meta-data to drive messaging to users in a custom interface.
  2. ...our builders create integrations and extensions that utilize the information to self-throttle and send alerts and alarms to support staff to better manage issues proactively.
Please note that this is only regarding our technical rate limiting and is mutually exclusive from any billing entitlements or limits you may have.

Up next, we will likely be shifting focus back onto production readiness. That means making sure these APIs can handle the hundreds of millions of calls we know you are ready to throw at them, going through some final pre-flight security checks, training all of our teams and planning for our post-GA roadmap.

Please keep beating these up. The feedback has been great, and we can't wait to hear more from you.

------------------------------
Harrison Hersch
------------------------------
APIs and custom code
  • DerekHutchinson's avatar
    DerekHutchinson
    Qrew Cadet
    5 years ago
    Great update. I am super excited about the audit log functionality it has been a pain point for us for a while and currently download the audit log one week at a time before we load it back into quickbase.

    ------------------------------
    Derek Hutchinson
    Business Process Manager
    PSG Dover
    Grand Rapids, MI
    269-779-5748
    ------------------------------
  • SamJones1's avatar
    SamJones1
    Qrew Cadet
    5 years ago
    Thanks for sharing, Harrison! Tell the dev teams to keep up the good work!

    ------------------------------
    - Sam

    ______________________________________________
    Sam Jones
    Vice President, Product and Technology
    he/him

    The Data Collaborative, Inc.
    sjones@datacollaborative.com
    366 Massachusetts Ave, Suite 203 | Arlington, MA 02474
    ------------------------------
  • AustinK's avatar
    AustinK
    Qrew Commander
    5 years ago
    In the last post about this it mentioned you guys streamlining your API from 60 calls to roughly 10 calls that focus on the fundamentals. Does this mean some functionality is being lost here?

    Also wondering how long the old API calls will be supported. Will there be 2 sets of API we can work with or are you planning on only having the new API up and running? I assume the plan is to get to one API eventually.

    Does this have any affect on the SDKs you guys have? Will those get a large overhaul at some point? I know you really don't support them currently but trying to find out if things will drastically change and what to expect here.
    • hhersch's avatar
      hhersch
      Quickbase Staff
      5 years ago

      Hi Austin! Good questions. I'll try to clarify here. Classic disclaimer applies here - this is all still under investigation and references beta features that are not yet committed to production. Finally, we acknowledge the road ahead on making sure these messages are clear and I'm working closely with @Adam Hoover to make that happen. You just happen to be a few weeks ahead of us with your questions - which is not a bad thing!

      First, we intend to offer "Parity+" for the high value and most commonly used APIs. What that means is that we have to be diligent about what we bring over and what we don't. Let's start with things we may not migrate. An example is API_GetRecordInfo. We don't intend on offering a comparable API for that. It is more streamlined, simple and easily understandable to simply hit our new records endpoint and filter to the record you want.

      Another example might be API_GenResultsTable (delivering HTML) or API_GetRecordasHTML. Through research and usage patterns, we know that most use cases where builders want to pull record info into a custom script involves their own styling. It doesn't mean that there aren't cases where someone is happy to take our default HTML, though. When you combine that with our UI refresh and the increased customization builders desire, it makes it difficult to support a dedicated endpoint to get HTML, which may not even fully represent what that builder is looking for.

      One that is more interesting, and we are still gathering info on is API_CopyMasterDetail. Much more advanced workflows are possible in Pipelines. So that is what we mean when we talk about being diligent, and ensuring that our investments are going into areas that provide a lot of value for the widest audience of people.

      Now, when we look at the base operations - that starts with making sure we have a great experience for CRUD on records. The "+" in "Parity+" refers to the fact that our new reports endpoint allows you to do things like get the report schema and also run summary reports. These are things that weren't available in the XML APIs. Loosely, the plan is Records -> Schema -> Users/Groups/Roles.

      That is some background. To answer your questions directly:


      1. The comment about 60 endpoints to 10 endpoints is around the fact that a single endpoint, called Records, now handles the equivalent functionality of API_AddRecord, API_EditRecord, API_ImportFromCSV, API_DoQuery, API_GetRecordInfo, and most of API_GenResultsTable. It mostly also replaces API_GetNumRecords and API_DoQueryCount. If you count API_DeleteRecord and API_PurgeRecords, since those are also record operations, we have simplified a lot of functionality. That means less friction for you to get up and running, reduced risk on our end maintaining multiple endpoints, and more deep investment in specific areas. The "60" to "10" was an estimate, and we also learned we needed to break a couple of things out that we initially thought might be able to be in one call. We get a lot of inquiries today around which API to use when. So much so, I even held a session at Empower a few years ago that compared API_GenResultsTable and API_DoQuery.
      2. The existing XML APIs will be around for a long time (measured in years). It would not be fair for us to ask builders to dig up their scripts from years old and replace everything when hundreds of millions of requests hit the XML API every month. That being said, we hope to provide enough value and incentives that folks are interested in converting to the RESTful APIs.
      3. Finally, with regard to an SDK - I'd love to hear your input. We are still researching this one. I think there is a decent chance we do something along the lines of publishing an OpenAPI spec (no promises yet). To be transparent, the feedback around an SDK has been almost 50/50 on "nope, don't need an SDK" vs "of course I need an SDK". What are you looking to achieve out of one? Is it just easy implementation into a toolkit, or were you hoping we maintained some decorations and enhancements?

      Hope this helps. Keep the questions coming.
      ​

      ------------------------------
      Harrison Hersch
      ------------------------------
      • AustinK's avatar
        AustinK
        Qrew Commander
        5 years ago
        Thank you for the information. I am preparing to switch to the new API at some point and just trying to see how much of a pain that is going to be. I'm glad to hear the old API will be supported for a while to come.

        As far as the SDK goes there is generally a pretty heavy split there. Some like them while others do not. When working with JavaScript and Quick Base I find using an SDK to be very valuable and it makes everything easier. I don't use one that is made by you guys though, there is a node version someone else released. Switching to a RESTful API will make it easier to work without an SDK so maybe one isn't needed. We currently use a heavily modified version of the c sharp SDK you have posted to Github. Comparing the calls we normally use to what is available in the new API there are several that are currently missing that we would want to see according to other users here I spoke with. I don't have a list in front of me though.

        One other thing I noticed is I saw the original page speaking about moving the SDKs to Github. https://www.quickbase.com/blog/quickbase-sdks-now-on-github It said "I encourage developers to contribute to the repository and make it even richer than it is today." and I would love to see that. If you do decide to have a new SDK it would be awesome if it was more of a community driven thing. If a pull request is put in where the community is trying to improve it it would be great to see them used. I mention this because looking at all the SDKs on there right now many have pull requests from 4+ years ago that are still not merged. The community wants to help but I'm sure there aren't many new pull requests because they can see the old ones are generally ignored.

        I guess my main point there is that if it isn't maintained at all then it is kind of pointless to have an SDK. They are good bases to get started even now but many(all possibly?) of them have some huge issues at this point if a new user were to try and use it. It is almost a detriment to have them available because of this instead of any kind of help which is a real shame.