Strategies for server-side rendering of asynchronously initialized React.js components

Question

One of the biggest advantages of React.js is supposed to be server-side rendering. The problem is that the key function React.renderComponentToString() is synchronous which makes it impossible to load any asynchronous data as the component hierarchy is rendered on the server.

Let's say I have a universal component for commenting which I can drop pretty much anywhere on the page. It has only one property, some kind of identifier (for example id of an article below which the comments are placed), and everything else is handled by the component itself (loading, adding, managing comments).

I really like the Flux architecture because it makes a lot of things much easier, and its stores are perfect for sharing state between server and client. Once my store containing comments is initialized, I can just serialize it and send it from server to client where it is easily restored.

The question is what is the best way to populate my store. During past days I've been googling a lot and I've come across few strategies, none of which seemed really good considering how much this feature of React is being "promoted".

1. In my opinion, the simplest way is to populate all my stores before the actual rendering begins. That means somewhere outside of the component hierarchy (hooked to my router for example). The problem with this approach is that I would have to pretty much define the page structure twice. Consider a more complex page, for example a blog page with many different components (actual blog post, comments, related posts, newest posts, twitter stream...). I would have to design the page structure using React components and then somewhere else I would have to define the process of populating each required store for this current page. That doesn't seem like a nice solution to me. Unfortunately most isomorphic tutorials are designed this way (for example this great flux-tutorial).

2. React-async. This approach is perfect. It lets me simply define in a special function in each component how to initialize the state (doesn't matter whether synchronously or asynchronously) and these functions are called as the hierarchy is being rendered to HTML. It works in a way that a component is not rendered until the state is completely initialized. The problem is that it requires Fibers which is, as far as I understand, a Node.js extension that alters the standard JavaScript behavior. Although I really like the result, it still seems to me that instead of finding a solution we changed the rules of the game. And I think we shouldn't be forced to do that to use this core feature of React.js. I'm also not sure about the general support of this solution. Is it possible to use Fiber on standard Node.js web hosting?

3. I was thinking a little on my own. I haven't really thought trough the implementation details but the general idea is that I would extend the components in similar way to React-async and then I would repeatedly call React.renderComponentToString() on the root component. During each pass I would collect the extending callbacks and then call them at the and of the pass to populate the stores. I would repeat this step until all stores required by current component hierarchy would be populated. There are many things to be solved and I'm particularly unsure about the performance.

Did I miss something? Is there another approach/solution? Right now I'm thinking about going the react-async/fibers way but I'm not completely sure about it as explained in the second point.

Related discussion on GitHub. Apparently, there is no official approach or even solution. Maybe the real question is how the React components are intended to be used. Like simple view layer (pretty much my suggestion number one) or like real independent and standalone components?

Solution

If you use react-router, you can just define a willTransitionTo methods in components, which gets passed a Transition object that you can call .wait on.

It doesn't matter if renderToString is synchronous because the callback to Router.run will not be called until all .waited promises are resolved, so by the time renderToString is called in the middleware you could have populated the stores. Even if the stores are singletons you can just set their data temporarily just-in-time before the synchronous rendering call and the component will see it.

Example of middleware:

var Router = require('react-router');
var React = require("react");
var url = require("fast-url-parser");

module.exports = function(routes) {
    return function(req, res, next) {
        var path = url.parse(req.url).pathname;
        if (/^\/?api/i.test(path)) {
            return next();
        }
        Router.run(routes, path, function(Handler, state) {
            var markup = React.renderToString(<Handler routerState={state} />);
            var locals = {markup: markup};
            res.render("layouts/main", locals);
        });
    };
};

The routes object (which describes the routes hierarchy) is shared verbatim with client and server

Comments

[phtrivier]

Just to get things : the asynchronous calls would happen on the server-side, too ? I don't understand the benefits in this case as opposed to rendering the view with some parts left empty, and filling it as the results from asynchronous response arrive. Probably missing something, sorry !

[tobik]

You must not forget that in JavaScript even the simplest query to database to fetch latest posts is asynchronous. So if you're rendering a view, you have to wait until the data is fetched from the database. And there are obvious benefits to rendering on server-side: SEO for example. And also it prevents the page from flickering. Actually server-side rendering is the standard approach that most websites still use.

[phtrivier]

Sure, but are you trying to render the whole page (once all the asynchronous db queries have responded) ? In which case, I would have naïvely separated it as 1/ fetching all data asynchronously 2/ when done, pass it to a "dumb" React View, and responds to the request. Or are you trying to do both server-side rendering, then client-side with the same code (and you need the async code to be close to the react view ?) Sorry if that sounds silly, I'm just not sure I get what you're doing.

[tobik]

No problem, perhaps other people have also problems to understand :) What you just described is the solution number two. But take for example the component for commenting from the question. In common client-side application I could do everything in that component (loading/adding comments). The component would be separated from the outer world and the outer world wouldn't have to care about this component. It would be completely independent and standalone. But once I want to introduce server-side rendering, I have to handle the asynchronous stuff outside. And that breaks the whole principle.

[phtrivier]

Just to be clear, I'm not advocating using fibers, but just doing all the asyncs calls, and after they're all finished (using promise or whatever), render the component on the server side. (So the react components would not know at all about the asynchronous stuff.) Now, that's only an opinion, but I actually like the idea of completely removing anything related to server communication from React components (which are really only here to render the view.) And I think that's the philosophy behind react, which might explain why what you're doing is a bit complicated. Anyway, good luck :)

[tobik]

Sorry I might have confused you, you were describing not solution number two but number one (not fibers). My opinion is, and I'm not alone from what I've noticed, that React can be so much more. It's the way you build the website. You start with small reusable components and combine them into complete component hierarchy which creates the actual page. And if I had to fetch all information outside of the components, I would have to recreate the whole hierarchy. I would be doing the same thing twice. And I would reduce the React layer to simple view layer which would be a waste of potential.

[phtrivier]

Ok, I'm probably biased on the "Reducing React to a view layer" part, since most of what I've done with React was through clojuscript and om.

[tobik]

This is what I have in mind: facebook.github.io/react/tips/initial-ajax.html The beauty of it is that I can drop this component anywhere and that's all. But I can't simply take it and render it to string.

[phtrivier]

react-router seems to have an AsyncState mixin, to get initialState asynchronously, would that be adaptable ? github.com/rackt/react-router/blob/master/docs/api/mixins/… - Still searching (I'm with you on this !)

[tobik]

Thanks :) But do you think it will work when the component is server-side rendered? I've been keeping my eye on react-router for quite time and there are two long threads on github where the guys are discussing how to implement server-side rendering for react-router. However, I wasn't able to make up from it how they intend to get around the limitation of renderComponentToString() being synchronous.

[phtrivier]

Yes, at some point I think the real solution to you problem will be hacking React to make all the component lifecyle methods return Promise, a.k.a, ahem, "rewrite the whole thing" ;)

[Mohamed El Mahallawy]

Any luck or answer to this?

[tobik]

I'm not aware of any break-through. Probably the best way to go right now is to use react-router (which is the strategy number one, only in a nicer react-like wrapper). I've sort of come to a conclusion that I perhaps expected too much from React and there is no perfect strategy, at least not in the current version. The problem is simply too complex and there will always be significant trade-offs. One day I would like to see a complete isomorphic Wordpress-like CMS written purely in React/JavaScript.

[tobik]

Thank you. I get the idea, but it's really not what I want. Let's say I want to build some more complex website using React, like bbc.com. Looking at the page, I can see "components" everywhere. A section (sport, business...) is a typical component. How would you implement it? Where would you prefetch all the data? To design such a complex site, components (as a principle, like little MVC containers) are very good (if maybe the only) way to go. The component approach is common for many typical server-side frameworks. The question is: can I use React for that?

[phtrivier]

You'll prefetch the data on the server-side (as it's probably done in this case, before passing it to a "traditional" server-side template system) ; just because the display of the data benefits from being modular, does it mean the computation of the data necessarilly has to follow the same structure ? I'm playing devil's advocate a bit here, I had the same trouble you have when checking out om. And I sure hope someone has more insights on this then I do - seamlessly composing stuff on any side of the wire would help a lot.

[tobik]

By where I mean where in the code. In the controller? So the controller method handling bbc's home page would contain like dozen of similar queries, for each section one? That's imho a way to hell. So yes, I do think that computation should be modular as well. Everything packed in one component, in one MVC container. That's how I develop standard server-side apps and I'm pretty confident that this approach is good. And the reason why I'm so excited about React.js is that there's a great potential for using this approach on both client and server side to create awesome isomorphic apps.

[tobik]

Thanks. The thing is that as far as I know, only route components support this willTransitionTo method. Which means that it's still not possible to write completely standalone reusable components like the one I described in the question. But unless we're willing to go with Fibers, this is probably the best and the most react way to implement server-side rendering.

[Hyra]

This is interesting. How would an implementation of the willTransitionTo method look like to have async data loaded?

[tobik]

You will get the transition object as a parameter, so you will simply call transition.wait(yourPromise). That of course means that you have to implement your API to support promises. Another disadvantage of this approach is that there is no simple way to implement a "loading indicator" on client side. The transition will not switch to the route handler component until all promises are resolved.

[tobik]

But I'm actually not sure about the "just-in-time" approach. Multiple nested route handlers can match one url which means that multiple promises will have to be resolved. There is no guarantee they will all end at the same time. If the stores are singletons, it can cause conflicts. @Esailija could you maybe explain your answer a little?

[Esailija]

I have automatic plumbing in place that collects all the promises that .waited for a transition. Once all of them are fulfilled, the .run callback is called. Just before .render() I collect all the data together from the promises and set the singelton store states, then on the next line after the render call I initialize the singleton stores back. It's pretty hacky but it all happens automatically and the component and store application code stays virtually the same.

[Esailija]

@tobik I also don't use transitionTo in client side at all since it is not called when dynamic segment changes or query string changes etc... so there is not many use cases for it at least imo

[tobik]

How do you collect the data and pass it to the stores? The way I use Flux is that I collect data and then I dispatch an action with the data as payload and that will update the stores. For every component, there might be a different set of data and different action. How do you deal with that? The only solution that comes to my mind is that the resolved promise would return a function, you would gather all the functions and call them at once to update the stores. But that adds complexity...

[tobik]

Anyway, I would prefer if the stores were not singletons. And that wouldn't be that difficult to do, the only thing I need is that the component props (or React context) would be passed as another argument of willTransitionTo. I actually proposed a concrete code change to react-async guys (only one line of the code had to be changed at the time) but they said they had something better and more universal coming up. So who knows. Anyway, my goal is to write 100% isomorphic code, so if willTransitionTo is not called in those cases you described, it's probably not the way to go.

[tobik]

Update: willTransitionTo is now executed even when only part of the query is changed: github.com/rackt/react-router/commit/c6aa4d3

[Esailija]

Components pass promises for store state snapshots to transition .wait in the willTransitionTo method, these promises are collected and aggregated and when they are all ready, the snapshots are used to get all affected stores in the right state just before the render call

[tobik]

Okay, I think I understand. I don't know what kind of application you are developing but if I wanted to apply this approach to an isomorphic app, it would mean that I would be using two different ways (one for the server, one for the client) to do the same thing (init/update stores for given url).

[Esailija]

@tobik Well it's all automated and if willTransitionTo was called properly on client side, my component code would work exactly the same for both sides, but even with the bug it is good enough. The only difference is how store retrieves data , on client side one uses $.ajax and on server side you call dao or api server

[tobik]

I guess so. Btw there are ways to make even the data retrieving isomorphic. I use for example slightly modified yahoo/fetchr

[Federico]

On any site (large/small), you only have to server-side render (SSR) the current page with its init state; you dont need the init state for every page. The server grabs the init state, renders it, and passes it to the client <script type=application/json>{initState}</script>; that way the data will be in the HTML. Rehydrate/bind UI events to the page by calling render on the client. Subsequent pages are created by the client's js code (fetching data as needed) and rendered by client. That way any refresh will load fresh SSR pages & clicking on a page will be CSR. = isomorphic & SEO friendly