Storing a file into MongoDB using Multer in Mongoose

Question

I have gotten this far to accept a file in my HTML form and post in with angular via an $http.post using the ng-file-upload module. Now I want to accept this file in Mongoose and store it into my NoSQL MongoDB hosted on MongoLab. I have read about this module called Multer and followed the basic documentation, but it only me as far. Before I explain the beginning of the problem let me post my Code:

---

My HTML form:

<form name="upForm">
   <fieldset>
      <legend>Upload files here</legend>
         <label>Insert File Here:</label>
         <input type="file" ngf-select ng-model="exFile" name="file" ngf-accept="'.cs'" required>
         <i ng-show="upForm.file.$error.required">*required</i>
         <div class="alert" ng-show="file.$error === 'pattern'">
              file type is not accepted
         </div>
         <br />
         <button ng-disabled="!upForm.$valid" ng-click="uploadExercise(exFile)" class="btn btn-default">Submit</button>
         <span class="progress" ng-show="picFile.progress >= 0">
         <div style="width:{{exFile.progress}}%" ng-bind="picFile.progress + '%'"></div>
         </span>
         <span ng-show="picFile.result">Upload Successful</span>
    </fieldset>
</form>

---

My Angular Code:

 $scope.uploadExercise = function (file) {

    console.log(file);
    var fd = new FormData();
    fd.append('file', file);
    $http.post(url+"/Upload", fd,{
        transformRequest: angular.identity,
        header:{'Content-Type': undefined},
        enctype:'multipart/form-data'
    }).success(function () { }).error(function () { });
    console.log(fd);
};

console logs return the correct file objects.

---

Mongoose so far:

var mongoose = require("mongoose");
var express = require("express");
var multer = require('multer');
var upload = multer({ dest: 'Uploads/' });
var bodyparser = require("body-parser");
var app = express();


mongoose.connect("connection-string");
app.use(bodyparser.json());

app.post('/Upload', upload.single('solution') ,function (req, res, next) {
    console.log(req.file);
});

This console.log keeps returning undefined. So something, somewhere went terribly wrong. Please help me out! I want to receive this file in my Mongoose and store it into the MongoDB, I have never done this before and can't seem to find any decent documentation for Multer or any decent explanation for storing files that is relevant for my case. What am I doing wrong? What should I be doing instead?

Solution 1

Man you are running on the wrong path. I have already explained in your previous question request that Multer is used to save files in file disk system and not to your database directly. For that you must use GRIDFS.

Coming to your current question.

app.post('/Upload', upload.single('solution') ,function (req, res, next) {
    console.log(req.file);
});

Here the upload.single('solution') - calls a function Upload and the file name passed is solution but it is obvious enough that it isn't available here.

use this type of format - documentation of Multer

var storage = multer.diskStorage({
  destination: function (req, file, cb) {
    cb(null, '/tmp/my-uploads')
  },
  filename: function (req, file, cb) {
    cb(null, file.fieldname + '-' + Date.now())
  }
})

var upload = multer({ storage: storage })

The Storage Part there is used to give path to where your file must be saved and the file name section is used to make changes to the file name that you would like to have.

Please read the documentation because that'll help. When we use third party modules we must acknowledge the information they have already given so that we can use their work easily.

Let me make it easier for you. Here is ready made code that works.

Multer throwing weird error while uploading file via ng-file upload

Go check that thread. The question was raised by me - the problem there was I was sending files in array format, as in multiple files at once. If you are not doing that just change ng-file-upload segment to use the single upload demo example and on server side nodejs code replace .array with .singleand things will work the way you want them to work - given that you want to use file disk system to store files.

I repeat that this method wont help you to save the file in mongodb directly.

Let me know if you need any further clarification.

Solution 2

After some research I found the answer to my problem. I'm able to store files into my MongoDB now. For those who are interested I'm posting my Mongoose code below with a small description.

---

Start by installing packages required for this operation. I used the following ones

npm install --save formidable

npm install --save gridfs

npm install --save fs

these are the packages I used ( Formidable, gridfs and fs). this is the fully working mongoose code for me relevant to this particular form and request.

var mongoose = require("mongoose");
var express = require("express");
var formidable = require("formidable");
var fs = require("fs");
var grid = require("gridfs-stream");
var bodyparser = require("body-parser");
var app = express();

mongoose.connect("**fill in your connection-url here**");
var conn = mongoose.connection;

app.use(bodyparser.json());

app.post('/Upload', function (req, res) {
    var form = new formidable.IncomingForm();
    form.uploadDir = __dirname+"/Uploads";
    form.keepExtensions = true;
    form.parse(req, function (err, fields, files) {
        if (!err) {
            console.log('Files Uploaded: ' + files.file)
            grid.mongo = mongoose.mongo;
            var gfs = grid(conn.db);
            var writestream = gfs.createWriteStream({
                filename: files.file.name
            });
            fs.createReadStream(files.file.path).pipe(writestream);
        }
    });
    form.on('end', function () {
        res.send('Completed ... go check fs.files & fs.chunks in mongodb');
    });
});

this worked for me! I now went to look in my mongoDB hosted on MongoLab and see that fs.chunks and fs.files collections were created and fs.files contains the correct data. So for those who have this problem, this is a solution and the documentation on http://excellencenodejsblog.com/gridfs-using-mongoose-nodejs/ helped me out a lot. Now that I have this working, I also want to download the file FROM mongoDB onto a chosen directory on my pc, is there anyone who can give me the answer to that? or is that just as simple as to create a readstream to a file-system?

Comments

[Tom Kustermans]

I understand the difference :) thanks! Can I also use Multer to store files that I want to "download"/"read" FROM my mongoDB into a chosen directory?

[Gandalf the White]

You want to save files in file disk system after retrieving them from Mongo DB?

[Tom Kustermans]

exactly. I noticed that in my code, GridFS will save my file into a directory form.uploadDir = __dirname+"/Uploads"; before streaming it into MongoDB. But I want to know how to receive a file FROM mongoDB and store it to a directory. Also, do you know how to clear this directory after the writestream is complete, otherwise this directory will get packed with uploaded files right ?

[Gandalf the White]

If you are using GRID FS to save files why would you like to save him anywhere else. And to be honest I haven't deleted sections automatically using node js in file system. But I can suggest one alternative. You can make a cron job that would run every day and using linux commands on your server you can clear a specific section. I am doing some research on your previous query, I'll let you know if I find something.

[Gandalf the White]

him* = it - in comment above

[ridoansaleh]

How could you installed fs since it is a core module of NodeJS ?