Webhooks vs API Polling
As APIs become more widespread, developers are now looking to receive real-time event data from their API providers. Two common solutions are webhooks and API polling. If you prefer to watch a video, you can checkout out our video on YouTube.
TLDR: Webhooks are send by the source of a data update whenever there is a change in the data. API Polling involves the receiver of data updates sending API requests to the data source and comparing responses to detect if anything changed.
What is a Webhook?
A webhook is a reverse API, or server-to-server push notifications. They work by sending out an HTTP request to a specified endpoint when an event is triggered. This makes webhooks ideal for integrating two applications.
If you're not familiar with webhooks, you can learn more about them in this article: What is a webhook?.
What is API Polling?
API polling is the process of repeatedly sending requests to the same endpoint and comparing subsequent responses you receive to determine if there is updated information.
Webhooks vs Polling - What's the Difference?
Here is a quote from Zapier on API polling:
“Polling is the process of repeatedly hitting the same endpoint looking for new data. We don't like doing this (it's wasteful), vendors don't like us doing it (again, it's wasteful) and users dislike it (they have to wait a maximum interval to trigger on new data). However, it is the one method that is ubiquitous, so we support it.”
If an API provider does not offer webhooks, polling can be your only option.
An Analogy
Imagine the classic situation of a family on a road trip. Polling is analogous to the kids constantly asking, “Are we there yet?” and the parents replying “no” over and over. Webhooks would be the scenario where the kids sleeping and are woken up by their parents when they arrive at the destination.
Webhooks remove the need to send repeated requests. Instead, the service provider sends updated data whenever it's available to a pre-determined URL.
Request Origin
Polling requests are made by the the receiver of the data updates while webhook requests are made by the source of the data. Webhooks are also automatically triggered when an event occurs, whereas polling is set up to run at fixed intervals and runs whether there is a new event or not.
Resource Efficiency
Polling can be resource-intensive and you need to make calls on whether the efforts will be fruitful or not. This is not the case for webhook requests, which are only made when there is new information.
For example, if 10,000 users poll the API every 5 seconds, your API will have to be able to handle up to 10,000 requests per second, depending on the timing of the requests.
Zapier estimates that only 1.5% of polling requests find an update. That implies that with a webhook solution, you would only need to send up to 150 responses per second while your users don't have to send any requests at all. That's less than 1% of the resources needed to sustain the polling solution (10000 requests + 10000 responses).
Real-Time Updates
With polling, there will always be an interval between requests that creates a delay in receiving an update. Approximating real time feedback with polling would require a lot of resources and incur high costs.
Webhooks are generated by the server that observes the triggering event so updates happen as the events happen.
When to use polling
Generally speaking, you should only use polling if you meet two conditions:
- You don't need updates in real time
- Updates are very frequent
When both of these are true, polling is actually a very good solution because you won't have wasted API calls and sending webhooks for every event would be highly resource intensive.
For example, a high growth consumer social media app wanting to report on its user base would get drowned in webhook messages because so many users are signing up. Instead, they can poll an API to get this information periodically as they don't need to update the number of users in real time.
When to use webhooks
If you need updated data in real time or updates are not very frequent, webhooks are the perfect solution.
A popular example of webhooks is Discord bots. A common use case for webhooks in Discord is a bot to automatically post in a channel when a specific Twitter account sends a tweet. This is an excellent use case for webhooks as you want the post to happen right away (real time), only when someone sends a tweet (event driven), Twitter doesn't need a response (one way), and only the bot needs to receive the information (one to one).