#securing npm

Earlier this year, we announced npm, Inc.’s acquisition of ^Lift Security and the Node Security Platform and promised to provide updates as our teams combined to protect the world’s largest JavaScript developer community. Today, we have some news about how the acquisition affects you.

With vulnerability scanning that was built into your workflow, and by maintaining JavaScript’s definitive listing of known package vulnerabilities, the Node Security Platform has kept thousands of developers safe from insecure code.

Since joining npm, we’ve worked to bring these protections to the larger developer community.

npm now automatically reviews every install request against the NSP vulnerability database and warns you if you try to use unsafe code. Additionally, beginning with npm@6, a new command, npm audit, recursively analyzes your dependency trees to identify specifically what’s insecure, recommend a replacement, or fix it automatically with npm audit fix.

These tools are faster and more directly integrated into the way you write JavaScript. They also form the backbone of powerful new security tools we have in store for the months ahead. Given this, it’s time to phase out the original Node Security Platform.

The Node Security Platform service will stop working on September 28, 2018.

We encourage you to update to npm@6 — just type npm i -g npm@latest — to take advantage of powerful, automatic protection built right into your workflow, along with a host of other enhancements and features. If you have been a subscriber to NSP’s pre-publication advisories, your account will cancel automatically (these advisories are now available to customers of any of npm’s paid services).

We've recently made some changes to how package naming works to better fight typosquatting, and help package authors pick names that stand out.

You might have read our post earlier about typosquatting on the npm registry. We responded to this incident with some internal tooling that identifies new packages with names similar to existing ones. We've been reviewing these by hand and taking action in a few cases. Some trends we have noticed in these packages led us to believe that we can encourage package naming that better suits the needs of both registry users and package authors. Specifically, you can no longer publish packages that differ from existing packages in punctuation only.

Package names can include punctuation like dots, dashes, and underscores (., -, _). Punctuation can make package names readable! For example react-native is a more readable name than reactnative would be. However, many typosquat names either accidentally or on purpose use punctuation differences to sneak through. For example, until this change it was possible to typosquat react-native by publishing reactnative. Under our new naming rules, however, this is disallowed.

Read on for details about the change.

The new rules

If you are publishing a new package—that is, a package that has not been in the registry before—we remove punctuation from its name and compare it to existing package names. If the names are identical without punctuation, we do not allow the package to be created. Instead, we suggest that you publish the package with that name under your own scope. You can, of course, also find a new name that’s sufficiently different from an existing package, but using your own scope is a fast way to do that.

Here are some examples of how this comparison works.

Because react-native exists, no one can publish variations like:

reactnative

react_native

react.native

Similarly, because jsonstream exists, no one can publish variations like:

json-stream

json.stream

json_stream

js-on-stream

Use a scope!

If you are prevented from publishing a package with a name too similar to an existing package, the easiest way to find a unique name is to use your scope. You can scope packages to your npm account by putting @ + your npm user name in front of the name. For example, my npm user name is ceejbot, so my scope is @ceejbot.

Publishing public packages to your own scope is free for everyone! Here’s how to do it:

Here I have been told that json-stream is too similar to an existing package, so I need to find a new name. First, I edit my package.json to add my npm user name as a scope:

{ "name": "json-stream" }

becomes

`{ "name": "@ceejbot/json-stream" } `

Next, I publish the package. Scoped packages default to private, so I pass the flag --access=public to make this package public:

> npm publish --access=public + @ceejbot/[email protected]

Everyone can publish public scoped packages to the npm registry, but to publish private packages you’ll need a paid subscription. For more information on working with scoped packages, check out our docs.

A little bit of history

The history of package names in the npm registry is a tale of cautiously-added restrictions over time. In its earliest days, npm allowed packages to have use any url-safe character in their names, including upper and lower case. You can’t create new packages with upper-case letters in the name any more, but packages with upper-case letters in their names are still in the registry and still in use. Package names that differ only in case were our first encounter with accidental typosquatting!

The example most people are likely to encounter is jsonstream. The packages JSONStream and jsonstream are different but difficult to distinguish. These packages also cause problems if you are installing them on case-insensitive filesystems, such as the file systems on most OS X computers.

Not every url-safe character is allowed any more. You can use the package validate-npm-package-name to check if your package name uses allowed characters. You’ll need to do a search of the registry to figure out if the name you’d like is already in use, however.

Good package names help everyone

Today, we are announcing two new ways to protect your npm account. Please read on to learn how you can use these security features to keep your code safe and increase everyone’s trust in the more than 550,000 packages of code in the npm Registry.

Two-factor authentication (2FA)

Now, you can sync your npm account with an authentication application like Google Authenticator or Authy. When you log in, you’ll be prompted for a single-use numeric code generated by the app.

2FA is another layer of defense for your account, preventing third parties from altering your code even if they steal or guess your credentials. This is one of the easiest and most important ways to ensure that only you can access to your npm account.

Read-only tokens

If your continuous integration / continuous deployment (CI/CD) workflow includes linking your npm account to tools like Travis CI with authentication tokens, you can now create read-only tokens for tools that don’t need to publish. You can also restrict tokens to work from only specified ranges of IP addresses.

Even if your token is compromised — for example, if you accidentally commit it to GitHub — no one else can alter your code, and only authorized CI servers will be able to download your code.

Learn more about tokens in this doc: 

Working with Tokens

Set these up now (please)

The npm community is now larger than the population of New York City, so it’s never been more important to reinforce trust and encourage collaboration. Every developer who secures their npm account with these new methods helps ensure the safety and integrity of the code we all discover, share, and reuse to build amazing things.

Learn how to activate 2FA in this doc:

Using Two-Factor Authentication

Watch this space

There has never been a major security incident caused by leaked npm credentials, but our security work is never finished. We work continuously to protect the npm Registry and detect and remove malicious code, and we try to keep you informed of our efforts.

If you ever believe you’ve encountered any malicious code on the Registry or in npm itself, contact us right away [using the npm website](lgt security contact form) or by emailing [email protected]. If you have any feedback or questions about what we’ve rolled out today, just contact [email protected].

Thanks for helping keep the npm community safe.

On August 1, a user notified us via Twitter that a package with a name very similar to the popular cross-env package was sending environment variables from its installation context out to npm.hacktask.net. We investigated this report immediately and took action to remove the package. Further investigation led us to remove about 40 packages in total.

On July 19 a user named hacktask published a number of packages with names very similar to some popular npm packages. We refer to this practice as “typo-squatting”. In the past, it’s been mostly accidental. In a few cases we’ve seen deliberate typo-squatting by authors of libraries that compete with existing packages. This time, the package naming was both deliberate and malicious—the intent was to collect useful data from tricked users.

All of hacktask’s packages have been removed from the npm registry.

Adam Baldwin of Lift Security also looked into this incident to see if there were any other packages, not owned by hacktask, with the same package setup code. He has every file in the public registry indexed by content hash to make scans like this possible. He did not find any other instances of that specific file with those contents.

exposure

Following is a list of hacktask’s packages, with a count of total downloads from 7/19 to 7/31.

Download counts for these packages are larger in the last two days because of public interest in the problem. The numbers from before exposure are more revealing of the effect of the malware. Note that 30-40 downloads is typical for any public package published to the registry, from registry mirrors automatically downloading copies. From this you can see that the real danger came from the crossenv package, which had nearly 700 downloads, with some secondary exposure from the jquery typosquats. But even in that case, most of the downloads come from mirrors requesting copies of the 16 versions of crossenv published. Our estimate is that there were at most 50 real installations of crossenv, probably fewer.

babelcli: 42 cross-env.js: 43 crossenv: 679 d3.js: 72 fabric-js: 46 ffmepg: 44 gruntcli: 67 http-proxy.js: 41 jquery.js: 136 mariadb: 92 mongose: 196 mssql-node: 46 mssql.js: 48 mysqljs: 77 node-fabric: 87 node-opencv: 94 node-opensl: 40 node-openssl: 29 node-sqlite: 61 node-tkinter: 39 nodecaffe: 40 nodefabric: 44 nodeffmpeg: 39 nodemailer-js: 40 nodemailer.js: 39 nodemssql: 44 noderequest: 40 nodesass: 66 nodesqlite: 45 opencv.js: 40 openssl.js: 43 proxy.js: 43 shadowsock: 40 smb: 40 sqlite.js: 48 sqliter: 45 sqlserver: 50 tkinter: 45

If you downloaded and installed any of these packages, you should immediately revoke and replace any credentials you might have had in your shell environment.

what we plan to do next

hacktask’s email address is banned from using npm. In this era of throwaway email addresses, that is not sufficient to prevent the human being behind it from trying again, but we felt it was a necessary gesture.

We are supporting ^Lift Security and the Node Security Project in their ongoing work to do static analysis of public registry packages, but this will not find every problem. Determining if a package contains malicious content when published is, of course, equivalent to the halting problem and therefore not something we can do.

We’re discussing various approaches to detecting and preventing publication—either accidental or malicious—of packages with names very close to existing packages. There are programmatic ways to detect this, and we might use them to block publication. We’re using the Smyte service to detect spam as it is published to the registry, and will be experimenting with using it to detect other kinds of violations of our terms of service.

Editor’s note: This is a guest post from Adam Baldwin of ^Lift Security and the Node Security Platform. As we discussed in earlier posts, Adam conducts constant security reviews of the Registry and its contents and keeps us appraised of anything that might compromise our security. 

Over the years I've spent a lot of time digging through the half million public packages on npm. There are millions of public tarballs to go with those public packages, which means there’s a lot of code to look through and experiment with. Buried in that code is a surprising amount of sensitive information: authentication tokens, passwords, and production test data including credit card numbers.

You, as a developer publishing to npm, want to avoid leaking your data like this. I’ll share some tips for how to control what you’re publishing and keep your secrets out of the public registry.

npm publish

Let's explore the behavior of npm publish, because understanding how it chooses which files to include is critical to controlling what gets published. If you want to dive in deeper, the npm documentation goes into more detail, but I’ll cover the important points here.

When you run npm publish, npm bundles up all the files in the current directory. It makes a few decisions for you about what to include and what to ignore. To make these decisions, it uses the contents of several files in your project directory. These files include .gitignore, .npmignore, and the files array in the package.json. It also always includes certain files and ignores others.

npm will always include these files in a package:

package.json

README and its variants like README.md

CHANGELOG and its variants

LICENSE and the alternative spelling LICENCE

npm will always ignore these file name patterns:

.*.swp

._*

.DS_Store

.git, hg , .svn, CVS version control directories

.npmrc

.lock-wscript

.wafpickle-*

config.gypi

npm-debug.log

.gitignore vs .npmignore

One of the most common ways to exclude files and folders is to specify them in a .gitignore file. This is because files you do not want to commit to your repository are also typically files you do not want to be published.

npm also honors a file called .npmignore, which behaves exactly the same as .gitignore. These files are not cumulative. Adding an .npmignore file to your project replaces .gitignore entirely. If you try to use both, you will inadvertently publish a file you thought you had excluded.

This is how that might happen:

You add your production.json configuration file to your .gitignore file because it contains sensitive information.

Then you add .npmignore but forget to add any files to it.

You run npm publish.

Because .npmignore exists, it is consulted instead of .gitignore, but it includes no files to ignore!

Your production.json file is therefore published, and your sensitive information is leaked.

Stick to using .gitignore if you can! If you are using a different version control system, use .npmignore. If you are using git and have an ignore file but wish to publish some of the files you’re not committing— perhaps the result of build steps— start by copying your .gitignore file to .npmignore. Then edit to remove the files you don’t want in git but do want in your package.

Whitelisting files with the files array

There’s an even better way of controlling exactly which files are published with your package: whitelisting with the files array. Only 57,000 packages use this method of controlling what goes into them, probably because it requires you to take inventory of your package. It’s by far the safest way to do it, though.

The files array specifies each file or directory to include in your publish. Only those files are included, plus the ones npm always includes no matter what (such as package.json), minus the ones denied by another rule.

Here’s a package.json file with a files array:

{  "name": "@adam_baldwin/wombats",  "version": "1.0.0",  "description": "",  "main": "index.js",  "scripts": {    "test": "echo \"Error: no test specified\" && exit 1"  },  "files": [   "index.js"  ],  "keywords": [],  "author": "Adam Baldwin <[email protected]> (https://liftsecurity.io)",  "license": "ISC" }

No matter what other files exist in this project directory during npm publish, only the index.js file will be packed up into the tarball (plus the readme and package.json, of course!)

Double-check your work

You can use the npm-packlist module to programmatically get a list of the files npm would include for a specific directory. You can also run npm itself to find what it would include. This command lists the files that would get packed up:

tar tvf $(npm pack)

Using read-only tokens

If you are using private modules in a continuous integration (CI) system you will need to provide that service with an authentication token. In the past you had to provide a regular authentication token out of your .npmrc, which gives your CI system the ability to do everything you can do with your npm account. Now it's possible to generate a read-only token that can limit the damage if a token is leaked via CI.

This feature isn’t yet supported by the npm CLI, but you can use the public registry API to generate a token by hand. Here’s an example with curl:

curl -u [USERNAME]:[PASSWORD] https://registry.npmjs.org/-/npm/v1/tokens \ -X POST -H 'content-type: application/json' \ -d '{"password":"[USERNAME]", "readonly": "true"}'

You can review and delete the tokens you have created via the npm website. under Your profile > Tokens.

What to do if you’ve published sensitive data

If you accidentally published a module containing sensitive information, you should consider that data compromised. It would be nice if you could just unpublish the module and hope that nobody saw the mistake, but the reality is as soon as you publish a module it’s replicated to all of the registry mirrors and and other third parties, like ^Lift. The only way to ensure that your services and data aren’t compromised is to invalidate and change any API keys or passwords that were published.

If a secret you can’t change has been leaked, your first step should be to unpublish the package to limit the damage, then take any other actions that are appropriate for the kind of data that leaked. The npm support team will help you unpublish any packages that are outside the 24 hour window for unpublication.

Try whitelisting!

I hope this has been a good refresher on how to help protect what you publish in the registry and ensure that you don't accidentally leak sensitive data. I encourage you to try using whitelists to control which files go into your next package.

You probably know ^Lift Security for its work as the Node Security Project, which reviews the most popular of the half-million packages in the npm Registry to find security vulnerabilities. However, you might not know that ^Lift also reviews the npm registry itself.

Since npm was founded, we have worked with Adam Baldwin and his team to conduct periodic security reviews of the code that we use to power the world’s largest software registry. Their methods include penetration tests and code audits of the contents of all private packages and of all of npm’s operations.

We work with ^Lift’s engineers to get a much-needed second opinion about our work. They clearly explain tradeoffs and priorities in their code reviews and give us actionable suggestions for mitigating risks.

Earlier this week, ^Lift completed another penetration test of the registry and I am currently reviewing their report of what they found. As always, they’ve shown us that we have things to do to tighten up our operations, including using HSTS and changing the ways some of our APIs operate.

In npm, Inc.’s three and a half years of operations, there has never been an incident in which a stranger exploited a vulnerability to steal user credentials — but our work to improve security is never done. Every change to a system can have security implications, and we’re constantly working on a lot of things! Every npm user and package on the npm registry is safer because of ^Lift’s reviews.

“We’re dedicated to creating good processes so that a solid foundation exists for new features to help users protect their accounts and the software they publish such as two factor authentication or package signing,” Adam said.

“Security is always at odds when it comes to other business objectives. From ^Lift’s perspective, npm does a great job prioritizing security when necessary and also taking it seriously. We don’t have to go to extreme lengths to convince them when something is an issue. npm takes the time to understand and make it a part of its overall business plans.”