Node and Express: How to implement basic webhook server

Question

I'm having a surprisingly hard time finding tutorials. I'm new to webhooks and have not used or seen them, beyond some basic descriptions of how they're supposed to work.

Our use-case for this is updating users of our APIs when there are new records. Since we're using Kafka and have settled on "eventual consistency" , another use might be to notify them of errors when records could not be read/written properly from the Kafka stream.

So, the basic concept as far as I can see:

const express = require("express");
const router = express.Router();

const processSomething = callback => {
    setTimeout(callback, 20000);
}

router.post("/hook", (req, res, next) => {
    processSomething(() => {
        res.status(200).send({
            id: "ABC123",
            message: "New record added!"
        });
    });
});

module.exports = router;

Is this essentially what I'm shooting for? Would users post to this endpoint, await a response, then post again upon getting the response so as to re-subscribe? Is there a problem with this running for a long period or even indefinitely?

I could really use more sophisticated examples but I'm just not finding them. Most of what you find when Googling this involves integrating third-party webhooks, like Github, Slack, etc., not custom which is what I'm needing to build.

I'm not opposed to another approach entirely, either. Just searching for the best means of notifying API users of updates and other important info.

Solution 1

Webhooks try to notify users with delayed events but they do it in asynchronous way. Your code is synchronous that means users need to wait for reply. I think it should looks more like this

const express = require("express");
const router = express.Router();

const processSomething = callback => {
  setTimeout(callback, 20000);
}

router.post("/hook", (req, res, next) => {
  processSomething(() => {
    const webhookUrl = req.params.url;

    /**
     * Your Kafka action or something else. There
     * you should collect info about success or
     * fail of client's action.
     */

    /** 
     * Your API call to webhookUrl with 
     * your defined body about status of event
     */
  });

  res.status(200).send('OK')
});

module.exports = router;

where function processSomething do your action in asynchronous way and call client's API with response but your route still answer with response - 200 OK means you got the message from client.

Also you can delegate notyfing client about success/fail message to another place of your code but you need to save webhookUrl for that. In events system, you can create event (or simple message) where one of attributes is client's url.

Solution 2

You're headed in the right direction, but I think you are just looking to build a RESTful API. Here is a good article from Scotch.io that talks about setting up a simple express API. This is a good example because they talk about routing, middleware and even give you an example of working with a database. They use PostMan as well which is a handy API testing tool.

EDIT: Op indicated that he was indeed looking for docs on web hooks and not just a RESTful API...

In that case I would recommend a pub sub pattern. I use this pattern because web hooks like Github's web hook API or GCP's cron job API aren't really concerned about the response from your API. Therefore, what I will usually do is have my web hook routes in my app simply emit an event and then resolve the API request send from the web hook. Then the event emitter will trigger a listener and the desired process will kick off. Something like this...

const Router = require('express').Router();
const eventEmitter = require('./eventEmitter');
// sets event listener
const listener = require('./eventListener');

Router.post('/webhook', (req, res) => {
    eventEmitter.emit('pubsub', req.body);
    res.status(200).send('success');
});

module.exports = Router;

And then in eventListener.js...

const emitter = require('./eventEmitter');

emitter.on('pubsub', function(requestBody) {
  // Do what you want
});

One last thing, this is a great pubsub npm that I use on other projects. What is great is that you can create subscriptions using dot notation.

Solution 3

As far as I understand, a hookback is just an endpoint that an automated service will POST to in an automated fashion, you handle the data and then send a response, I don't think it is any different than a REST api, just has a funky name.

Source: Pretty sure I built a webhook API a couple years ago and that's what I discoevered

Comments

[Tsar Bomba]

Appreciate that, but I've built many APIs. What I'm looking for specifically, is providing webhooks as API callbacks. A common pattern but I don't see many examples. Not new to any of these technologies, just this pattern. Thanks.

[Max Baldwin]

@TsarBomba apologies! I updated my answer accordingly

[Tsar Bomba]

This might be exactly what I'm after, thank you. It was the process/flow that I wasn't sure of. This seems to make the most sense. Will give it a shot!

[daihovey]

Lol at the source :)

[B L Λ C K]

@Anthony thanks for source :P but jokes aside. I was confused before starting to create my own webhook demo because it exactly looks like rest api and I couldn't convince myself, there is a slight difference. Anyway fast forward, I am still confused about it. if the client is keep posting to rest end point every 10sec then how it is different. Please add some more explanation or provide some "source" where I can learn. I end up here doing my own research. I know it is a old answer but I hope to see your reply. be safe :)