How should I organize multiple Express servers on the same system?

Question

I'm using one server to host multiple Node.js web apps, which are distributed across multiple domains. My current practice is to run an Express server for each app on a different port, and to run a base server that simply routes (redirects) requests to the correct port/Express server. This works, but it means that my base server is routing every single HTTP request (and by manually redirecting it), and that my users see my apps as hosted at [hostname.com]:8000.

After a bit of research, I've found that I can use http-proxy for my routing needs, but I'd still like to know if there's a best practice for running multiple Express servers on the same system. Here's how I'm planning on doing it:

Each web app will have its own folder, with a complete Express folder structure (app.js, routes, views, etc.) Apps will be grouped by domain, so an example folder structure would be:

    hostname.com/
        app.js
        routes/
        views/
        ...
        app1/
            app1.js
            routes/
            views/
            ...
        app2
        ...
    hostname2.com/
        app.js
        routes/
        views/
        ...

I'll have to run each app.js separately with node (or with forever, which I'm currently using), and each one will have to use a different port internally, with cross-app redirects being pointed at the port of the target app.

So, that's my current plan. What are the problems with it, and what pitfalls should I try to avoid? Most importantly, is there an established solution to this problem - the problem of hosting multiple web apps on the same system with Node.js/Express?

EDIT: I do plan to eventually use WebSockets and HTTPS, and the amount of bandwidth my setup can support is of little importance to me - this is a development server (at least for now). Thanks to David Ellis for bringing up the issue of WebSockets.

SECOND EDIT: Thanks to both EhevuTov and David Ellis for their answers, both of which helped greatly. I'm still settling on an overall structure for my application, and it looks like that question is addressed in some detail by this StackOverflow question

THIRD EDIT: I've come a ways since posting this question (though I have much further to go). Check out this file in my GitHub repository, which leverages what I learned from the answers to this question!

Solution 1

Since Express uses Connect, I'm pretty sure you can use Connect's virtual host middleware. It operates similar to other vhost modules on other products. I don't have multiple domains to test and show you proper code, but I would think it's something like this:

express.createServer()
.use(express.vhost('hostname1.com', require('/path/to/hostname1').app)
.use(express.vhost('hostname2.com', require('/path/to/hostname2').app)
.listen(80)

If you get to the point where one Express server isn't enough, then look into using the Node.Cluster from the API. If that also isn't enough, then the current practice is to put a asnyc reverse proxy such as Nginx in front of your Express servers and point the proxies to your Express servers.

Solution 2

If you don't need to use WebSockets (or any HTTP 1.1 feature, really), you can use NginX as your proxy instead.

The advantage is the total load NginX can handle versus Node is higher (being statically compiled and specialized for this sort of thing, basically), but you lose the ability to stream any data (sending smaller chunks at a time).

For a smaller site, or if you're unsure what features you'll need in the future, it's probably better to stick with node-http-proxy and only switch to NginX if you can demonstrate the proxy is the bottleneck on your server. Fortunately NginX isn't hard to set up if you do need it later.

Comments

[Aaron]

I do plan to eventually use WebSockets and HTTPS, and the amount of bandwidth my setup can support is of little importance to me - this is a development server (at least for now). Thanks for bringing up the issue of WebSockets. However, my question is about the organization of my system one step after reverse proxy server - how should I set up and organize Express servers for my multiple web apps?

[Admin]

I'd say that Node.js is young enough that there is no established pattern for multiple Express apps on the same server. Node.js shares more in common with Tomcat/Getty and other Java-based web servers; one app per server, while hosting multiple websites has traditionally been done with a single web server (Apache/NginX/Lighttpd/etc). Each Node.js webapp must be configured to not use a TCP port used by another webapp, so there's no "drag-n-drop" way to get it set up.

[Admin]

To add on that, because a Node.js HTTP server can be set up to listen on a port only for a specific hostname, you could have the whole thing set up as one Node.js process running multiple Express servers simultaneously. Of course that makes little sense when these servers never touch code between each other and the Node.js process overhead is only a handful of megabytes, but it's also an "option."

[Aaron]

Thanks for the comment. I've started using vhost (with the help of some helpful IRC users in #express), and it's working like a charm. Also thanks for the heads up the possibility of moving up to Node.Cluster.

[Aaron]

Thanks for pointing out that listen() takes an optional hostname argument! Right now I'm using a single Node.js process with five vhosts, but using listen(port, hostname) I'll be able to run apps independently of each other!

[Matt]

@Aaron Do you have some code you could share to show us how you do it?

[Aaron]

@Matt Yeah, just check out my 3rd edit in the post. I added a link to my GitHub repo.

[Aaron]

As I mentioned in my question: "My current practice is to run an Express server for each app on a different port, and to run a base server that simply routes (redirects) requests to the correct port/Express server. This works, but it means that my base server is routing every single HTTP request (and by manually redirecting it), and that my users see my apps as hosted at [hostname.com]:8000." I was looking for a way to have a single entry point (a proxy) to multiple apps, and as I mentioned in my answer above, express.vhost did the trick.

[aakash4dev]

o sorry. you can try fs, ``` const fs = require('fs'); fs.readdir(testFolder, (err, files) => { files.forEach(file => { console.log(file); }); }); ``` get the file names of html files folder. above codes will return something like ["index.html","about.html","blog.html"]. make server with forEach loop as per file names

[Aaron]

I appreciate your desire to help, but you're suggesting inapplicable answers to an already-solved question from over 10 years ago. I'm good, thanks 👍🏼