ReadMe Webhook Documentation Review

This post is a part of our "Documentation Review" series, where we collaborate with different companies to review, analyze and improve their docs. You can find the rest of the posts in the series here.

We power the webhooks for many customers across a variety of industries, which means we get to see a lot of different webhooks docs. The documentation we've seen generally has these five elements: signature verification, event catalog, retry policy, troubleshooting and failure recovery, and testing tools.

Today, we're taking a look at ReadMe's webhook docs. ReadMe is a popular tool for transforming your API docs into interactive hubs to help developers succeed so naturally we wanted to see how they wrote their webhook docs. Thanks to Kanad Gupta, Developer Advocate @readme for collaborating with us on this post.

TL;DR:
Signature Verification: A-
Event Catalog: B+
Retry Policy: N/A
Troubleshooting and Failure Recovery: A
Testing Tools: A

Signature Verification

Signature Verification is a critical security feature of webhooks. It lets users verify that a webhook was actually sent from the expected source. The signature verification section of webhook documentation should explain the signature scheme (how the signature is created) and give instructions on how to verify the signature with code samples.

Although ReadMe's docs don't explain the signature scheme, there's a good reason why. They provide a verifyWebhook function in their SDKs. This makes things super simple for developers. Dealing with signatures and raw payloads is one of the most common issues we see for webhook consumers so a built in function that abstracts away these challenges makes for a great developer experience. For those that want to know exactly how the signature is generated, you can dive into their open source SDKs (example in Node).

The only confusing part of their verification instructions is that their secret is stated to be the project's API key but in the code sample is set to “JWTSECRET”. Do we need to create a JWT using the api key or just use the actual api key? We would assume the latter because there are no instructions on how to generate the JWT and the full code example pop up sets the secret to “YOUR_SECRET_HERE”.

We really like that ReadMe offers a built-in verification function (we also offer this at Svix). We don't see it often so major points here. We would like a bit more clarity/information on how the signature is generated and whether a JWT is actually required or not for the signing secret. This would let users know it's secure and reliable.

Grade: A-

Event Catalog

The Event Catalog documents the event types users can subscribe to. They denote the type of message being sent and are the primary way for webhook consumers to configure what events they are interested in receiving. The best event catalogs list every available event type with sample payload schemas and description of what actions trigger the event.

ReadMe's webhook is specifically for enabling personalized documentation and is only triggered upon a user login. There is an example HTTP request to demonstrate the expected headers and payload body but there is no specified event type. We'd suggest including an event type in the payload if they expand the webhook functionality to offer subscriptions to events other than a login.

Grade: B+

Retry Policy

Typically, retries are a crucial component of a reliable webhook service.

However, ReadMe's webhook feature specifically enabling customized documentation makes retries less important. It's not so useful to retry the login notification 5 minutes after the user is trying to read the documentation. Perhaps 1-2 retries within a few seconds in case there was a fluky error could be useful but delaying the rendering of the documentation for retries is a worse user experience than simply showing a generic example.

Grade: N/A

Troubleshooting and Failure Recovery

It's common for webhook docs to give some guidance on how to troubleshoot failing endpoints and how to get them back online. ReadMe addresses this in their FAQ with a link to instructions on how to integrate their Metrics SDK to log endpoints and surface any errors in the project dashboard.

They also do a great job giving troubleshooting tips within the webhook setup UI itself. You can check out a video demo of this feature here.

A common practice for many webhook providers is to automatically disable endpoints that fail repeatedly. They don't want to retry messages indefinitely.

Because retries aren't useful for ReadMe, they don't offer them and in turn do not disable failing endpoints. They state in the FAQ that if this policy were to change in the future, that users will be notified well in advance.

Grade: A

Testing Tools

There are two types of tools that can be very useful for testing webhooks. One is a tool like Svix Play that gives users an endpoint to send test events to. This helps them see how your webhook service works and actually see it in action without needing to deploy their own endpoint.

This isn't super useful for ReadMe users because the endpoint would actually need to respond with some user information to get the customized docs to work.

The second tool is for sending test events. This is extremely useful when you're setting up an endpoint and running into issues (e.g. with verifying the signature). Instead of needing to trigger an actual event (in the case of ReadMe you would need to login => logout => login to generate login events) you can just click a button and send a test event.

ReadMe's docs mention the ability to test endpoints in passing under the configuration & testing section so we dug into the product to check it out (video demo by ReadMe). After you input an HTTP endpoint and a user email, ReadMe sends you a test event so you can see the personalized docs in action.

We'd love to see this get a bit more attention in the docs because it's really a great feature.

Overall we're happy to see the testing feature as it's another thing missing from most webhook implementations.

Grade: A

Summary

We wanted to share ReadMe's docs as an example of a specialized webhook implementation. They offer webhooks to enable one specific feature, so certain aspects of webhook docs that are critical in most cases aren't as important for ReadMe's users (mainly the retry policy and event catalog). There are some potential improvements in adding some details around the signature scheme and some guidance on troubleshooting and when endpoints may be disabled/how to re-enable them.

ReadMe does a great job offering features like verification in the SDK and the ability to fire off test events that make developer's lives easier when setting up webhooks. These are missing from a lot of other implementations so we love to see it!

If you're looking to implement world class webhooks, consider trying Svix, the enterprise ready webhooks service. Svix makes it super easy to build a secure, reliable, and scalable webhook solution through an API.