how to create a readable stream from a remote url in nodejs?
fs.createReadStream()
does not work with http URLs only file://
URLs or filename paths. Unfortunately, this is not described in the fs
doc, but if you got look at the source code for fs.createReadStream()
and follow what it calls you can find that it ends up calling fileURULtoPath(url)
which will throw if it's not a file:
URL.
function fileURLToPath(path) {
if (typeof path === 'string')
path = new URL(path);
else if (!isURLInstance(path))
throw new ERR_INVALID_ARG_TYPE('path', ['string', 'URL'], path);
if (path.protocol !== 'file:')
throw new ERR_INVALID_URL_SCHEME('file');
return isWindows ? getPathFromURLWin32(path) : getPathFromURLPosix(path);
}
It would suggest using the got()
library to get yourself a readstream from a URL:
const got = require('got');
const mp4Url = 'https://www.example.com/path/to/mp4Video.mp4';
app.get('/video', (req, res) => {
got.stream(mp4Url).pipe(res);
});
More examples described in this article: How to stream file downloads in Nodejs with Got.
You can also use the plain http/https
modules to get the readstream, but I find got()
to be generally useful at a higher level for lots of http request things so that's what I use. But, here's code with the https module.
const https = require('https');
const mp4Url = 'https://www.example.com/path/to/mp4Video.mp4';
app.get("/", (req, res) => {
https.get(mp4Url, (stream) => {
stream.pipe(res);
});
});
More advanced error handling could be added to both cases.
Martin Wittick
Updated on June 16, 2022Comments
-
Martin Wittick almost 2 years
on nodejs documentation, the streams section says I can do
fs.createReadStream(url || path)
. But, when I actually do that It tells meError: ENOENT: no such file or directory
. I just want to pipe the video from a readable to a writable stream, But I'm stuck on creating a readable one.my code:
const express = require('express') const fs = require('fs') const url = 'https://www.example.com/path/to/mp4Video.mp4' const port = 3000 app.get('/video', (req, res) => { const readable = fs.createReadStream(url) }) app.listen(port, () => { console.log('listening on port ' + port) })
the ERROR:
listening on port 3000 events.js:291 throw er; // Unhandled 'error' event ^ Error: ENOENT: no such file or directory, open 'https://www.example.com/path/to/mp4Video.mp4' Emitted 'error' event on ReadStream instance at: at internal/fs/streams.js:136:12 at FSReqCallback.oncomplete (fs.js:156:23) { errno: -2, code: 'ENOENT', syscall: 'open', path: 'https://www.example.com/path/to/mp4Video.mp4' }
PS: https://www.example.com/path/to/mp4Video.mp4 IS NOT THE ACTUAL URL
-
Martin Cup almost 3 yearsThe require('https') version worked very well without having to install another 3rd party package.
-
Optymystyc almost 3 yearsDoes
https.get
return the stream? So can I do something likeconst myStream = https.get(mp4Url)
? I'd like to eventually use this withfluent-ffmpeg
. -
jfriend00 almost 3 years@Optymystyc - As you can see in the last code block in my answer, it doesn't return the stream, it provides the stream to the callback you pass it.
-
JCutting8 about 2 yearsDon't forget to set your content type and also catch finish and error events.
-
JCutting8 about 2 yearsHow do you go about handling a 302 redirect response? Stream doesn't seem to have a status code
-
jfriend00 about 2 years@JCutting8 - Please write your own question, show your own code and describe your own specific circumstance. In this code, a 302 response from the
https.get()
would just get passed through to the response becausehttps.get()
does not automatically follow 302 responses.