Why Lob switched their webhooks to Svix

Lob is on a mission to be the world's largest sender of direct mail. They are already well on their way, serving over 12,000 customers, including Expedia, Twitter, and Gusto.

Lob were already sending tens of millions of webhooks a month when they first came to Svix, though they were feeling the pain of their existing system. They hit its limits in terms of scale, it was expensive to run, it lacked key features that their customers required, and it was painful for their team to maintain. They now send many more webhooks a month, and these are no longer a concern.

We are incredibly happy we chose Svix. The best benefit: we can basically forget webhooks exist!

Reliable webhooks are important for business

Lob's product automates snail mail and makes it more intelligent. Businesses usually have to undergo an arduous process where they negotiate with printers to get their postcards, letters, and other mail products proofed and then printed. With Lob, you can use either their API or dashboard to create highly targeted (customizable per recipient) mail at scale and track attribution using QR codes on these campaigns.

Mail is a very asynchronous system. Lob's customers use webhooks to get updates on their mail pieces as they go through Lob's systems, get delivered by the postal service, and are read by customers. For example, Lob sends them events when a letter is printed, a postcard is delivered, a check is returned to sender, or a QR code is viewed.

We asked Krys Flores, a Staff Engineer at Lob, and the person in charge of their webhooks overhaul project, what made them first consider Svix; she explained that while they already had an internal webhooks system, it was incredibly brittle, and it was hard to track an event through the system and understand what was going on. They were also missing important features which meant their team often had to manually help their customers.

So when one day one of their big customers said they needed a specific feature by a certain date, they decided to buckle down and search for a proper solution. At this point, it wasn't yet clear to them whether the solution would be internal or external, but they knew they needed to act.

I did the classic “build vs. buy” research and “buy” really was best for Lob. Thank you, Svix!

Svix ticked all the boxes

We asked Krys to share a story from the earliest days of evaluating Svix, this is what she had to say:

I think the following story is very telling: In order for me to feel comfortable using a new company (Svix was a young company when we started talking), I had to be sure they could withstand our load. I had two very eager interns so I threw them the problem “Try to break Svix.” (Sorry, not sorry!)

The interns load- and stress-tested the system, though the Svix servers easily scaled to the additional load. Svix also knew exactly when we were testing, a testament to their monitoring. I know that because they would often ask: “Are you load testing us right now?” I’d wait until the load testing was done and then reply, “Why yes! We were.”

Expert customer support that never sleeps

When we asked her what she liked the most:

It must be the ease of integration and the fantastic customer support.

To which Andrew Reagan, the Senior PM working on their webhooks replied:

Totally! They were responsive, knowledgeable about our implementation and quick to help on any issues.

The Svix engineering team executes more quickly than I've ever seen any engineering team in my career. We tested Svix a lot before deciding to sign a contract. When we found something that was missing, they said, “Not a problem.” Two days later it was up in prod and working smoothly.

One example for that was when a customer told Lob that they weren't getting webhooks delivered. Many of Lob's customers rely on the webhooks for their business operations. So when a customer said they weren't getting some webhooks, Lob immediately contacted the Svix team to help with the investigation.

After a quick investigation it was found that the customer has made a change on their side that didn't roll out as planned. They were responding to webhooks with 202 ACCEPTED (a success response) even though their processing was actually failing.

Krys further elaborated:

Svix were very quick in identifying the issue on the customer side, notifying us, and enabling us to respond to our customer promptly. Since these were “successful” events in a webhook system, we normally shouldn’t need to replay them; however, once the customer fixed the issue on their end, they asked us to replay them. Svix had a simple endpoint that made it EASY for us to do this request.

The End Result

In their blog post about switching to Svix, where they summarize the key benefits they got from the switch:

• Simplified webhooks infrastructure and greater scalability.
• The engineers' lives are much easier and they can focus on business goals.
• Lower and more predictable costs that decrease as they scale.
• The new dashboard is feature-rich: customers can now easily self-serve and do much more.