Forum Discussion

MikeTamoush's avatar
MikeTamoush
Qrew Commander
2 months ago

Bringing attention to the End of Life for Webhooks

*Note, my title says End of Life, but should read End of Sale (webhooks will still work, just can't add any more). However, this posting still reads the same.

Hoping to get more eyes on this blog post, and if there is enough uproar, maybe Quickbase will reconsider.

The comments on the post are spot on (I would add a comment, but for the life of me cannot figure out how). In addition to the current comments, webhooks are great for very simple tasks (if a new record is created, create record in my combined table...), keeping the number of pipelines down to a manageable amount. If I had to convert my webhooks to pipelines, I would suddenly have an extra 100 pipelines, making it nearly impossible to search through. I understand that my existing webhooks will stay, but in the future who wants to search through thousands of pipelines?

Let your voice be heard if you think the removal of webhooks is a terrible idea!

https://community.quickbase.com/blogs/brian-cafferelli1/2024/03/15/creation-of-new-qb-actions-and-webhooks-will-be-lo



------------------------------
Mike Tamoush
------------------------------

14 Replies

  • I understand why Quickbase would want to get rid of the webhook feature, but it is definitely a step in the wrong direction for several reasons.

    1. Lack of Collaboration and Visibility: Pipelines lack the cross-team collaboration and visibility inherent in webhooks. To replace webhooks, pipelines need shared access among admins and a comprehensive view of related pipelines for efficient maintenance.

    2. Synchronous Execution: Webhooks offer synchronous execution, ensuring instant response to events, unlike pipelines which might introduce delays.

    3. Efficient Bulk Updates: Webhooks excel in handling bulk updates efficiently, minimizing overhead by repeating actions in a single api call.

    4. Slow Pipeline Building: Pipeline construction can be sluggish compared to webhooks, potentially hindering productivity in complex workflows. The number of seconds waiting for a dropdown to load or a field to populate really hinders the builders' flow. 

    4. Cost Considerations: Webhook calls to outside servers incur fewer costs compared to pipelines, where every HTTP call uses an account step. 

    5. API Integrations: Webhooks have a limited, but useful API control via the legacy XML API. There is no such API control for pipelines.

    In essence, while Quickbase's decision to phase out webhooks may seem logical from a business perspective, it overlooks crucial functionalities vital for efficient workflow management. Enhancing pipeline capabilities to match the efficiency and performance of webhooks should be prioritized before ending webhook support.



    ------------------------------
    Bradley
    ------------------------------
  • Webhooks are faster and easier to setup than Pipelines 

    When writing a Webhook, the process looks like this. Open the Webhook, write the Webhook, test the Webhook. Takes no time and little effort.

    With Pipelines users have to log on to a service account in incognito mode. Then go through an arduous process of waiting for drop down lists to load and dragging little images around on a screen, etc etc... it's too many layers of abstraction for anyone who knows how to write a Webhook. 

    Until the Pipelines process is as fast or faster than the Webhooks process, there is little to no value in using Pipelines.

    Webhooks still have great value and it is a mistake to remove them from the rich feature set Quickbase has to offer.



    ------------------------------
    Jim Harrison
    transparency = knowledge + understanding : The Scrum Dudes
    ------------------------------
  • Brian Wrote:
    "Also, Jim you mentioned that the process of setting up a new pipeline feels slow to you. Can you tell me know about what you're seeing? I'm curious whether you have tried out our new pipeline designer. You may also be able to accelerate the setup by importing a pipeline via YAML."

    First you have to log on to a service account using incognito mode. When you log in to the service account, you have to wait for the verification code from Quickbase. The verification code may or may not show up in your inbox in the ten minutes you have to enter the verification code.

    Once logged in there are about 400 Pipelines and you can only see the top 10 or so. Unless you know the exact name of the pipeline or it is tagged properly, there is no easy way to find the pipelines associated with a table or App. There is no way to organize the Pipelines into folders, which may feel archaic but it is the best way to organize large amounts of files. (at this point you are already testing the Webhook.)

    If you are adding a new pipeline, the first step wants you to pick an app. So the step has to query your realm and get a list of apps, this can take about 15 seconds. Then once you have selected the App, you have to select the table. The table load query can also take about 15 seconds.

    From there forward the process is easy. 

    Now you want to test. How do I test this thing without it changing every record in the table? How do I stop the Pipeline once it is started? I turned the pipeline on and it is slowly changing all the data in my table because I made a mistake and there is no way to stop the pipeline. 

    The pipeline environment needs to store all Apps and tables and fields in an organized visual manner. The visibility can be provisioned via User_Token. User_Tokens should also provision Users or Groups. I am a Realm Admin. I create a User_Token and add three builders to it, who can also see all the Pipelines in a folder structure in the Pipeline UI.

    This way if I am in the Pipelines environment and someone has added a pipeline with the agreed upon User_Token, I too can see that Pipeline. Maybe that user can set the User_Token to r/w or r only so it is under each builders control to hold or share.

    Finally Troubleshooting. The audit log is a bog. Each Pipeline needs its own Audit log container. The UI of the audit log needs to take up less space on the page. Users who are looking at the Audit log need to be able to see as much information in one screen as possible. The way it is currently set up, it looks like a social media page. You have to dig and expand to get to the details of each step. 

    In Pipelines, it's been long enough now that I have lost the will to live. My Webhook was done about an hour ago and I have moved on. Hopefully my pipeline doesn't fail because I will have no idea if it did. The email sent only displays a long number without any information. In a Webhook, I get an error. I know the App, the table and the error message. I can't clear the error but at least I know where to look. 

    Brian I hope this is enough information to answer your question. If you would like to meet and record my observations for further analysis feel free. I am certain Edi is aware of all these problems and is working his team into the ground to solve them but it's not ready. 

    As a reminder the New Style Reports still aren't working. New Style forms aren't working. Taking things that are mostly working away from your customers is questionable. It just seems to me that this decision should be reconsidered until the customers agree it is good enough to replace Webhooks. 

    We will have to write Webhooks into code pages so we can trigger Workato.



    ------------------------------
    Jim Harrison
    transparency = knowledge + understanding : The Scrum Dudes
    ------------------------------
  • In my experience a feature that goes EoS eventually makes its way to EoL.
    Moving away from Webhooks, a fairly simple and expedient way to trigger workflows, and into Pipelines (more complex), does not seem like a lean solution. If the focus is on removing duplication within your own product offering, then this makes sense.  If the focus is on offering your clients flexibility to integrate QB into their overall tech stack, this doesn't make sense.  Looking for QB to provide the user base more background as to why this is a good thing for users.

    Also, will QB be offering a migration tool/automation to move our existing webhooks and QB Actions into Pipelines, similar to what was offered for EoL of QB Automations?

    Our team has done some testing with building the same workflow in Pipelines vs Workato, and so far we're sticking with Workato as our primary automation platform.



    ------------------------------
    Jon Froderberg
    ------------------------------
    • Gary1's avatar
      Gary1
      Qrew Cadet

      @Jon - I agree with everything you said. 

      We're in the same boat. Workato is our primary automation platform. This update means that Workato will completely lose the ability to programmatically create table webhooks for real-time recipe triggers. 

      This is a huge loss of functionality and convenience that can't be understated. Routing will either have to go through manually created webhooks, a Pipeline message queue, or become timed queries.

      I'm interested to know if you have any other ideas on how to work around the loss of webhooks for Workato integrations. Would be happy to compare notes. 



      ------------------------------
      gary
      ------------------------------
      • BrianCafferelli's avatar
        BrianCafferelli
        Qrew Captain

        Hi Gary, I wanted to clarify that this change will not disrupt workato workflows. I will add that to the community post.



        ------------------------------
        Brian Cafferelli
        Product Marketing Manager | Quickbase
        ------------------------------
  • My biggest concern is the inability to create Pipelines programmatically. The legacy XML API allows for CRUD of legacy webhooks, but there is no way to do this with a Pipeline webhook in either API.  

    It's probably just a matter of time until they cut off the XML API as well, which would be another huge mistake without ensuring all commonly-used legacy functionality is replicated in the new API.



    ------------------------------
    gary
    ------------------------------
    • BrianCafferelli's avatar
      BrianCafferelli
      Qrew Captain

      Hi there, so we do not currently have any plans to retire the XML API. While the RESTful API offers more modern functionality, we understand that the XML API plays a slightly different role in Quickbase (for example, when you create a relationship, the Add Child Record button uses the XML API).

      Also, with our upcoming Solutions API capability, you will be able to programmatically create and manage Pipelines so that is something we are currently working on.



      ------------------------------
      Brian Cafferelli
      Product Marketing Manager | Quickbase
      ------------------------------

  • Adding my take (previously shared with Quickbase directly).



    At this point, we (the developer community) do not feel there is parity between table webhooks and pipeline webhooks.

    I would say the top 3 reasons are:

    • Access: Not all users have access to Pipelines or it's often easier to create table webhooks with shared access which is not currently an option in Pipelines.
    • Speed: Table webhooks run faster than you can blink even for large sets of data.
    • Bulk Triggers: Today a table webhook triggered off a bulk action (grid edit or import) creates a single trigger that can either be used to create a single action like add record OR a bulk action with ImportFromCSV. In pipelines any time a bulk action (grid edit or import) triggers a pipeline it files individually which can cause issues with large updates that may trigger 1000+ instances.
      Here are some feedback cases on this particular issue:
      https://feedback.quickbase.com/app/#/case/359112
      https://feedback.quickbase.com/app/#/case/160597


    Since the retirement of Automations I've found myself relying more heavily on table webhooks when the use case is simple and the above are necessary.

    Without parity it would be DISASTROUS and expensive to rework the apps where extensive table webhooks have been used... and I'm sure many partners feel the same.

     

    In addition to the above, I agree with others who have also mentioned the often painfully slow process of building Pipelines compared to the speedy development of table webhooks as well as the difficulty searching and organizing Pipelines.

    -Sharon



    ------------------------------
    Quick Base Junkie
    Sharon Faust
    https://quickbasejunkie.com
    ------------------------------

  • ben_simon's avatar
    ben_simon
    Community Manager

    Thanks for getting this discussion started Mike. 

    For anyone that cannot Add a comment to the blog announcing these changes, please share any feedback or concerns here in this discussion. 

    There's an issue with some of our community members have that is a security setting behind our SSO config that I will not be able to fix until April 1. So we'll need to use a discussion thread like this instead.



    ------------------------------
    Ben Simon
    bsimon@quickbase.com

    Do you have Feedback on how to make Qrew Discussions a better experience? Let's chat!
    https://calendly.com/bsimon-2
    ------------------------------
    • MikeTamoush's avatar
      MikeTamoush
      Qrew Commander

      In my opinion, you would have to accomplish 3 things to get rid of webhooks (these are what affect me and I suspect the masses, others may have different requirements).

      1. Make the Pipelines Gui work, and work smoothly. Right now (see my other post), it is barely functional. Goes offline, doesn't save, is unthinkably slow. It is almost unusable. The top priority MUST be making Pipelines actually work. Seeing as how it has been like this since inception, it does not give us developers optimism that this is going to happen.
      2. Make Pipelines fire as fast as Webhooks. Meaning, they run instantly before the refresh of the page. This is paramount. My concern is that QB keeps talking about making them faster etc, but I don't think they mean Webhooks fast. We developers need some automations to fire essentially, instantaneously. So until a Pipeline fires in less than half a second, please do not get rid of Webhooks.
      3. A better organization structure. Perhaps 'folders' that you can put Webhooks in? The tags work OK, but so many webhooks are quick hitters, and some people will have 100, 200, or 300+ extra pipelines to accomplish fairly menial tasks. Having a list of 400 pipelines is hard for organization.

      To me, numbers 1 and 2 are required, number 3 would be really nice.



      ------------------------------
      Mike Tamoush
      ------------------------------
      • FraidyJacoby's avatar
        FraidyJacoby
        Qrew Trainee

        Mike, I couldn't have said it better myself. I agree that these are the top three concerns with phasing out actions and webhooks and having total reliance on pipelines going forward. It's great that pipelines are constantly being improved on, but are they up to replacing actions and webhooks? Not quite.



        ------------------------------
        Fraidy Jacoby
        ------------------------------