#sails.js

The Node Security project just released an advisory about the CORS implementation in Sails.

tldr; If you are using CORS in your Sails app, review your configuration to be sure it is secure.

If your app has vulnerable CORS configuration, there are two ways to resolve it:

Either replace origin: '*' with a specific set of whitelisted domains

Or set credentials: false

See Concepts > Security > CORS in the Sails docs for more information on how CORS works, and how to use it.

Note that you don't necessarily need to upgrade to v0.12.7 -- although if possible, it's a very good idea. As of 0.12.7, Sails will log a warning if you lift your app in production when vulnerable CORS settings are detected.

Just reached the next milestone on the road towards Sails v1: As of today, you can add a line of code to your HTML to directly expose data from your server-side view locals to client-side JavaScript, with built-in XSS attack prevention. This eliminates the need to hand-roll your own decoding/encoding (or worse, forget). And it means you don't need to send a bunch of extra AJAX requests to safely get the data you need onto the web page. (For more background on the use case, check out this 2012 post from Dan Webb, Twitter's former engineering manager: https://blog.twitter.com/2012/improving-performance-on-twittercom)

I've spent a lot of time testing Node.js/Sails.js apps and modules for memory leaks (with the help of @particlebanana, @sgress454, and others). Fortunately, true memory leaks are pretty rare.

But I have encountered memory leaks in some application-level code at least once. I say "at least" because the rest of the Sails.js team and I have found WAAAAAYY more things that we thought were memory leaks, but actually weren't.

Which is good, I guess. But possibly even more annoying.

So I wanted to jot down a bit about what I've learned about memory leaks over the past five years -- and specifically the process that I use to diagnose whether one exists in a Node.js/Sails.js app.

1. Eliminate variables

Do everything you can to simplify your test environment.

Before you get started trying to diagnose a memory leak, you should rule out as many problems as possible. In a Node.js/Sails.js app, that starts with configuring recommended production settings.

2. Assume you messed up

Accept the crushing reality that it's probably all your fault.

Next, check your app-level code (i.e. try to isolate the leak to a single endpoint). In my experience, it is very common w/ Node apps in general to end up with leaks from failing to handle errors (e.g. if you're using promises and forget to do a .catch()).

If you can't isolate the source of increasing memory usage to a particular endpoint or anything about your app code, then it's time to try replicating the leak in a brand new app running with the same recommended production settings, with no bells and whistles, with recommended production settings.

3. Remind yourself what a memory leak is, actually

Take a moment to mutter to yourself to make sure you remember what you're doing.

I have no idea if this is true for you or not; but when I'm working on a problem, I have to constantly remind myself of what I'm doing. Otherwise I forget. Or worse, I'll realize what I was doing didn't really make any sense anyway.

So let's take a second to do that.

"What are we doing?"

We're trying to figure out whether our Node process has a memory leak.

"Why are we doing it?"

Good question.

OK how do we know?!

A Node process will grow its memory usage until it decides it's time to run the garbage collector. Just because memory is going up continually does not mean there is a memory leak-- it means that the garbage collector has not run yet.

Sounds simple, right? But this tends to be where things break down. Until you've actually gone through the process of diagnosing a leak step by step, it's really hard to know wtf you're doing. And if you think you know wtf you're doing, you probably don't. (That was the case the first time I tried it anyway.)

So in the spirit of that, here are a few examples to give you some reps: (1 2 3 )

4. Run an experiment

Set up monitoring, recreate the supposedly leaky behavior, then use the garbage collector to verify that memory is never being reclaimed.

The only surefire way to diagnose a memory leak is to run your Node process with the --expose-gc flag enabled, to manually run the garbage collector , and then to compare the "valleys" of the chart. Here's a step by step guide:

4A. Expose gc endpoint

Expose a development-only endpoint that, when called, will run the garbage collector. For example: https://github.com/balderdashy/sails-hook-dev/blob/master/index.js#L190-L203 (you can also just install sails-hook-dev in your project).

4B. Monitor

Start up a program that monitors your Node process's memory. I recommend NodeSource.

4C. Lift

Lift your app with all recommended production settings (see deployment and scaling docs on sailsjs.org). But also make sure you tell Node.js to allow your code programmatic use of the garbage collector. For example, I like to do this by running:

NODE_ENV=production node --expose-gc app.js

4D. Do stuff or run load tests (round 1)

Now perform the behavior that you suspect will cause a memory leak. For example, that might be sending "POST" requests to a particular URL (use your app's UI or a tool like Postman). After about 30 minutes of this, if you take a look at the graphs, they might look slopey and bad, or they might flat and good. But that doesn't matter at all-- we have no idea about whether there are memory leaks at this point.

4E. Force garbage collector to run (round 1)

Now hit your endpoint that runs the garbage collector. You'll see the graphs drop dramatically after a moment. Take note of the lowest point in the graph (the "valley"). This is the amount of memory that the process is using, and that could not be reclaimed using the garbage collector. This is the memory where stuff you're actually using lives-- e.g. the require() cache, and local variables that are still in use. Be sure to take a screenshot of the graphs and memory usage in GB at this point.

4F. Do stuff or run load tests (round 2)

Now do exactly the same thing we did in step 4D again, for another 30 minutes.

4G. Force garbage collector to run (round 2)

Now do exactly the same thing we did in step 4E again. After a moment, if you notice that the "valley" in the graph is significantly higher than it was in step 4E, there might be something going on (there seems to be some amount of natural variation in what the garbage collector can actually reclaim-- in some cases this second "valley" is actually lower). Be sure to take a second screenshot of the graphs and memory usage in GB at this point.

Finally, if, after running through the steps above, it seems likely that there is a memory leak (i.e. the second "valley" was significantly higher than the first), then repeat steps 4F and 4G one more time to be sure.

5. Analyze your results

Check out the "valleys" in your memory usage graph (the spots right after the garbage collector ran). If they're getting higher and higher, you've got a memory leak.

If there is a memory leak, you can expect the second "valley" to be significantly higher than the first, and the third "valley" to be significantly higher than the second.

6. Cope

If there's a memory leak, fix it.

That's it. Whether there's a memory leak or not, you'll never get that hour back. But at least now you know.

-m

This is a patch release, so there shouldn't be any breaking changes. The release notes are included below for reference, and to point out what's changed and what's new. Big thanks to all of the folks who helped with this release, especially those of you who contributed to testing and documentation!

-Mike

P.S. If you're curious what's next, I posted a first look at the Sails v1.0 roadmap here.

v0.12.4 Release Notes

Originally posted on GitHub

A new patch release with bug fixes, enhancements, and upgrades to Sails' dependencies.

See also the recently-published improvements to the Sails documentation.

Changelog

From https://github.com/balderdashy/sails/blob/master/CHANGELOG.md#0124:

[INTERNAL] Upgrade Mocha to 3.0.0 to remove more of the deprecation notices when installing dependencies (see mocha:#2200)

[INTERNAL] Simplify config-merging code in captains-log. This also gets rid of more deprecation notices during install (see captains-log:49f433eff348c05115a2caf292b4da0db9499887)

[BUGFIX] Fix long-standing, low-priority (but super annoying) issue with logged dictionaries/arrays getting extra quote marks (due to a pecularity in the usage of util.format()) d67e9c8e6775

[ENHANCEMENT] Add sails.config.session.routesDisabled config option to specify routes that should not use session middleware c712acf

[ENHANCEMENT] Use res.forbidden() when denying access to a route via a policy. Thanks @wulfsolter! 3764

[ENHANCEMENT] Allow use of Express style path and RegExp in sails.config.csrf.routesDisabled. Thanks @bolasblack!

[BUGFIX] Fix for query / body params called length 3738

[BUGFIX] Fix view rendering when i18n hook is disabled. Thanks @mordred! 3741

[BUGFIX] Allow sails.renderView to work with globals turned off 3753 d3f634c

[BUGFIX] Fix typo which could cause crashing when attempting to serialize non-json-compatible output in the response to a socket request.

[UPGRADE] Update all Grunt dependencies 5f6be05

[UPGRADE] Update "connect" dependency to 3.4.1 1d3c9e6

[UPGRADE] Update "compression" dependency to version 1.6.2

[UPGRADE] Upgraded version of Consolidate to 0.14.1 a70623c

[UPGRADE] Upgraded version of grunt-contrib-watch to 1.0.0 3678

[INTERNAL] Use standalone CSRF package instead of using the one (formerly) bundled with Connect. Sails should be using all standalone middleware now. 1d3c9e6 98861ef

[INTERNAL] Add some assertions to ensure custom hooks don't use reserved properties 2e76dac

[INTERNAL] Update code that virtual response uses to read buffer to work with all Node versions a5ab134

Introduction

Sails.js is node.js framework that allows you to build enterprise-ready, custom MVC ( model, view, controller ) application on-the-go. Sails.js has built-in features such as an API creator, and its socket integration in every route and database ORM makes it very useful and helps speed up development.

Sails.js features

Sails.js has lots of features which will help you to build custom, enterprise-ready applications. Some of them are as follows:

Auto generated API

Database ORM

Inbuilt task runner

Security code

Built-in web sockets in routes.

Unlike MEAN stack, where you need to create API's manually and do same repetitive thing again and again, Sails.js allows you to automatically create skeleton APIs. Just a single command, and you will have block of code generated to get you started.

Additionally, Sails.js provides database drivers for every major database systems such as MySQL, MongoDB, PostgreSQL. Sails.js uses Grunt task runner as the default task runner, and also provides an effective way to manage custom grunt tasks.

Furthermore, common security parameters such as CORS and CSRF are already included in Sails.js project. All you need to do is to enable them from respective files and your application is secured with the latest security standards.

Sails.js uses Express to handle HTTP operations and Socket.io to handle web socket messages. It provides web socket messages to your routes without configuring it in back-end or front-end layer.