Preparation

  1. Download the StrongLoop Node.js distro for your platform
    http://strongloop.com/products
  2. Get your free copy of Node.js in Action
    http://strongloop.com/promotions/mongosv

(and then after my presentation)

  1. $ git clone git://github.com/mattpardee/mongosv.git restapp && cd restapp && slnode install

Building REST APIs

Using Node.js, and exposing data from MongoDB

by @matt_pardee from StrongLoop

StrongLoop

We are packaging a Node.js distribution for use in production. It is backed by professional support and consulting plans.

Enterprise -> community feedback loop.

We want Node to succeed.

StrongLoop

Founded by myself, Al Tsang (former VP ShutterFly), and Raymond Feng.

But, importantly, Bert Belder and Ben Noordhuis, who build libuv.

What this Workshop Covers

  1. Why?
  2. Design Approach
  3. The slnode utility
  4. Express
  5. MongoDB
  6. Mongoose
  7. Implementing API routes
  8. Passport
  9. Awesome new stuff from StrongLoop (?)

Why?

API tier: The Node.js sweet spot (one of many)

  • Airbnb
  • LinkedIn
  • Uber
  • eBay
  • Cloud9 IDE

Airbnb and Rendr

Airbnb's Problem

Airbnb's Problem: Solved

Design Approach

Two use cases:

  • Web
  • Mobile

Third use-case?

  • B2B

Design Approach

Provide an interface for rendering content and a REST API

OK. Let's get started.

Using the command-line utility slnode

  • scaffold apps
  • run tests
  • install SL-supported packages

e.g. RESTful app with Mongoose support $ slnode create web restapp -m -r
Command-line tool $ slnode create cli clitool

Express App Organization

  • public — static assets
  • routes — all the route logic
  • views — template files
  • app.js — application starting point

REST App Organization

Classic express structure with some added folders/files

  • db — config and convenience methods for mongo
  • models — the schemas/models for mongo data
  • routes — all the route logic
  • test — tests for our routing logic and database layer
  • app.js — application starting point

MongoDB :: our data foundation

Running npm test from the command-line is convenient and Just A Good Idea(TM)

package.json:

{ "name": "restapp", "description": "restapp", "private": true, "version": "1.0.0", "scripts": { "start": "node app", "test": "./node_modules/mocha/bin/mocha --timeout 30000 --reporter spec test/*.js --noAuth --noSetup" }, "dependencies": { "express": "latest", "ejs": "latest", "mongoose": "latest" }, "devDependencies": { "mocha": "latest", "should": "latest", "supertest": "latest", "underscore": "latest" } }

Test Data

What do our supertest+mocha-powered tests look like?

test/alldata.js

process.env.NODE_ENV = 'test'; require('should'); var app = require('../app.js'), request = require('supertest'), mongoose = require('mongoose'), _u = require('underscore'); assert = require('assert'); var json = JSON.stringify; before(function onBefore(done) { var connection = mongoose.connection; connection.on('open', function () { connection.db.dropDatabase(function () { console.log('dropped database [' + connection.name + ']'); done(); }); }); }); describe('rest', function () { describe('GET /rest/users', function () { it('should return empty list', function (done) { request(app).get('/rest/users') .expect('Content-Type', /json/) .expect(200) .end(function (err, res) { if (err) console.log('ERROR', arguments); res.should.have.property('body'); res.body.should.be.an.instanceOf(Array); res.body.should.have.length(0); done(); }); }); }); var id; describe('POST /rest/users', function () { it('should create a new user and return it', function (done) { request(app) .post('/rest/users') .set('Content-Type', 'application/json') .send(json({ username: 'mattpardee', first_name: 'Matt', last_name: 'Pardee', email: 'matt@strongloop.com', password: 'strongstrong' })).expect(200).end(function (err, res) { if (err) console.log('ERROR', arguments); res.should.have.property('body'); res.body.should.have.property('username', 'mattpardee'); res.body.should.have.property('first_name', 'Matt'); res.body.should.have.property('last_name', 'Pardee'); res.body.should.have.property('email', 'matt@strongloop.com'); id = res.body._id; done(); }); }); }); describe('GET /rest/blogs', function () { it('should return empty list', function (done) { request(app).get('/rest/blogs') .expect('Content-Type', /json/) .expect(200) .end(function (err, res) { if (err) console.log('ERROR', arguments); res.should.have.property('body'); res.body.should.be.an.instanceOf(Array); res.body.should.have.length(0); done(); }); }); }); describe('POST /rest/blogs', function () { it('should create a new blogs and return it', function (done) { request(app) .post('/rest/blogs') .set('Content-Type', 'application/json') .send(json({ author: 'superblogger', title:'Test Blog 1', body:'Some blogged goodness' })).expect(200).end(function (err, res) { if (err) console.log('ERROR', arguments); res.should.have.property('body'); res.body.should.have.property('author', 'superblogger'); res.body.should.have.property('title', 'Test Blog 1'); res.body.should.have.property('body', 'Some blogged goodness'); id = res.body._id; done(); }); }); }); });

Mongoose :: MongoDB interface for Node.js

mongoosejs.com

Demo

Expose our Mongo data to clients via REST API

Goals:

1. Cleanly map to Mongo

2. Make it extensible/reusable (DRY)

How routes should be programmed

A client can POST, GET, PUT, DELETE (CRUD)

Example A) GET /rest/users

  1. Our express app catches the request on app.get("/rest/:resource")
  2. Express turns the :resource into a variable on the request object: req.params.resource
  3. Get the associated Mongo model:
    var mongoModel = mongo.model(req.params.resource);
  4. The model is found, and the data is queried: mongoModel.find(...)
  5. Boom. Done. Results are returned to the client.

How routes should be programmed

Example B) POST /rest/users

  1. Our express app catches the request on app.post("/rest/:resource")
  2. The POST data is in req.body
  3. Express turns the :resource into a variable on the request object: req.params.resource
  4. Get the associated Mongo model:
    var mongoModel = mongo.model(req.params.resource);
  5. The model is found, and POSTed data is injected: mongoModel.create(req.body)

Expose our Mongo data to clients via REST API

Goals:

1. Cleanly map to Mongo

var mongoModel = mongo.model(req.params.resource);

2. Make it extensible/reusable (DRY)

app.verb("/rest/:resource")

Deeper Dive on the Code

routes/resource.js
/** * REST APIs for mongoose models that supports CRUD operations */ /** * Create a new entity */ exports.create = function(mongoose) { var mongo = mongoose; return function(req, res, next) { var mongoModel = mongo.model(req.params.resource); if(!mongoModel) { next(); return; } mongoModel.create(req.body, function (err, obj) { if (err) { console.log(err); res.send(500, err); } else { res.send(200, obj); } }); }; } /** * List all entities */ exports.list = function(mongoose) { var mongo = mongoose; return function(req, res, next) { var mongoModel = null; try { mongoModel = mongo.model(req.params.resource); } catch(err) { console.log(err); } if(!mongoModel) { next(); return; } var options = {}; if(req.query.skip) { options.skip = req.query.skip; } if(req.query.limit) { options.limit = req.query.limit; } mongoModel.find(null, null, options, function (err, objs) { if (err) { console.log(err); res.send(500, err); } else { res.send(200, objs); } }); }; } /** * Find an entity by id */ exports.findById = function(mongoose) { var mongo = mongoose; return function(req, res, next) { var mongoModel = mongo.model(req.params.resource); if(!mongoModel) { next(); return; } var id = req.params.id; mongoModel.findById(id, function (err, obj) { if (err) { console.log(err); res.send(404, err); } else { res.send(200, obj); } }); }; } /** * Delete an entity by id */ exports.deleteById = function(mongoose) { var mongo = mongoose; return function(req, res, next) { var mongoModel = mongo.model(req.params.resource); if(!mongoModel) { next(); return; } var id = req.params.id; mongoModel.findByIdAndRemove(id, function (err, obj) { if (err) { console.log(err); res.send(404, err); } else { res.send(200, obj); } }); }; } /** * Update an entity by id */ exports.updateById = function(mongoose) { var mongo = mongoose; return function(req, res, next) { var mongoModel = mongo.model(req.params.resource); if(!mongoModel) { next(); return; } var id = req.params.id; mongoModel.findByIdAndUpdate(id, req.body, function (err, obj) { if (err) { console.log(err); res.send(404, err); } else { res.send(200, obj); } }); }; } /** * Expose the CRUD operations as REST APIs */ exports.setup = function(app, options) { options = options || {}; mongoose = options.mongoose || require('../db/mongo-store').mongoose; var base = options.path || '/rest'; // Create a new entity app.post(base + '/:resource', exports.create(mongoose)); // List the entities app.get(base + '/:resource', exports.list(mongoose)); // Find the entity by id app.get(base + '/:resource/:id', exports.findById(mongoose)); // Update the entity by id app.put(base + '/:resource/:id', exports.updateById(mongoose)); // Delete the entity by id app.delete(base + '/:resource/:id', exports.deleteById(mongoose)); }

Implementing API Routes Pt. 2

Model files set up schemas and any specific CRUD logic.

For example models/user.js

/** * User model */ module.exports = function(options) { options = options || {}; var crypto = require('crypto'); var mongoose = options.mongoose || require('mongoose'); var Schema = mongoose.Schema; var UserSchema = new Schema({ username : { type : String, required : true, unique : true, index : true, display : { help : 'This must be a unique name' } }, first_name : { type : String }, last_name : { type : String }, email : { type : String }, password : { type : String }, created_at : { type : Date , default: new Date() }, modified_at : { type : Date , default: new Date() } }); function sha1b64(password) { return crypto.createHash('sha1').update(password).digest('base64'); } UserSchema.pre('save', function(next) { var _this = this; if (this._doc.password && this._doc.password != '_default_') { this.password = sha1b64(_this._doc.password) } if (this.isNew) this.created_at = Date.now(); else this.modfied_at = Date.now(); next(); }); UserSchema.pre('init', function(next, data) { delete data.password; next(); }); var User = mongoose.model("users", UserSchema); }

In app.js we load model files and pass them our mongoose instance:

options.mongoose = mongo.mongoose; require('./models/user')(options);

I promised you swagger

"Swagger sucks. That was my fault. I'm sorry."

- Swagger Developer

I promised you Auth with passport

That I do have.

Review

Node.js is very good at moving bits.

Like Backbone?
Airbnb's Rendr :: https://github.com/airbnb/rendr

THE END

Matt Pardee / StrongLoop