#bodyparser

Aprender Node.js – Realizar el primer setup inicial

Toca renovarse. Después de muchos años usando una tecnología es normal que te hayas podido quedar un poquito estancado en cuanto a los referentes del mercado aunque por otro lado es muy seguro que hayas conseguido dominar esa tecnología y hoy vamos a empezar a aprender Node.js.

En el mundo tecnológico existe una máxima que dice que aproximadamente cada dos años se duplica el número de…

I recently spent a couple of days working with XDomainRequests. This is the legacy technology that allow you to do cross origin requests (CORS) on Internet Explorer 8 and 9. To explain CORS further: if you have a web app available through “myfirstdomain.com” and that web app, once it is loaded in the browser, sends a request to an API that is available on “myseconddomain.com”, that is CORS. Requests between different domains.

Now, this blogs title is “makeitnew.io”, and I will be the first to admit that this posts relevance might be missing. But as it took me a really long time to find a solution for this in my network, and on Google, I feel that I need to write this post to help the next person, implementing legacy browser support for his REST API, to get this problem solved more quickly.

The Problem

My issue manifested itself after deploying a piece of client side JavaScript that supported XDomainRequests to all our clients. Suddenly alerts about a huge spike in 400 errors filled our system health channel on slack. The number of errors, and the lack of decrease in HTTP response 200, makes it relatively clear that most of the extra traffic we were expecting to get from rolling out support for IE8 and IE9 are all coming in as errors.

I immediately rolled back the change. After the back end looks healthy again, I immediately started looking for reasons why this would happen. Former Microsoft employee Eric Law has written probably the most comprehensible guide to XDomainRequests, and I read through it with a looking glass to find possible explanations.

The first likely culprit is that XDomainRequests are supposed to send requests with the content-type “text/plain” in the header. Yes, I missed the note in Eric Law’s post saying that as of 2014 no content-type seems to be sent. So anyways, I open Postman and start sending requests with the culprit content-type and get 400s back. Time for a pull request on our collector service.

So after setting up everything I need to test locally, I add support for body-parser to parse text/plain on our Node.js / Express back end collection service. Still just 400 back. WAT.

So I am using Internet Explorer 11 in IE9 emulation mode for sending the requests. And the request bodies being sent appear empty. I enable some logging in the console. And, no, the request body is not at all empty. Weirdness is going on here. I get some logging going on my local fork of the collection service. And yes, the requests are in fact empty except for the header.

At this point I am confused and I start reaching out to both my network and Google to figure out if others have experienced the same problem. And there is not much help to be had. I find some colleagues that have experienced something similar and have solved it by creating a proxy for requests with the same domain as their service. Allowing them to use XmlHttpRequests in old Internet Explorer instead. As the script I am making is running across a lot of different domains that I do not control, this is not an alternative. So I keep on Googling.

My breakthrough comes when a colleague suggests that I use PyProxy. After firing up Wireshark (because that was what I had installed) and sending some packages, I discover two things. First, that there are infact a request body that looks healthy. And secondly that the content-type is missing.

The Solution

After realising the issue was actually the content-type missing, I was able to find a package called “express-content-type-override”. And while it was not perfect, it was enough to fix the problem for my case.

What the aforementioned package lacks to account for, is that the charset also goes in the content-type. That caused some initial problems that was quick to find and fix.

I also submitted a pull request for this: https://github.com/rbartoli/express-content-type-override/pull/1

What is important is that the content type is checked and modified before bodyParser is added to as middleware.

Example

This code is all in the server.js file. Which is where the routing happens in the mentioned application.

This is put in the top of the file, together with the rest of the required packages.

var contentTypeOverride = require('express-content-type-override');

The middleware is applied right before the routing happens:

// Handle XDomainRequests coming in without content-type header. app.use('/api/send', contentTypeOverride({ contentType: 'application/json', charset: 'utf-8' })); var apiSendRouter = require('./routes/api-send'); app.use('/api/send', apiSendRouter);

Parsing of the body happens inside the required file for the path (./routes/api-send)

About the author

New Post has been published on http://blog.giiko.net/2014/04/11/%e5%9c%a8-express-%e4%bd%bf%e7%94%a8-bodyparser-%e8%a6%81%e6%b3%a8%e6%84%8f%e7%9a%84%e4%ba%8b%e9%a0%85/

在 express 使用 bodyParser 要注意的事項

今天在修改程式時才發現，原來沒有使用 bodyParser 是沒辦法接收 client post 的資料？仔細看才知道，原來也不見得需要 bodyParser ，要的只是 json 和/或 urlencoded middleware ，所以也順便找了一下 bodyParser 的故事…

找到的竟然是 Do Not Use bodyParser with Express.js

也順便整理了一下相關的問題和解決方案：

背景：

其實根據 express 3.x 的官網文件說明：

express.bodyParser() = express.json() + express.urlencoded() + express.multipart()

這次真的出問題的，其實就是 express.multipart() 這個 connect 的 middleware ，問題在於，其實 multipart 這個 middleware 可以在沒有任何認證或是檢查的條件下，無限上傳檔案到 nodejs server 上，造成 temp 檔案過多的問題

解決方式：

有以下 5 點的建議參考解決方法：

使用 multipart 或是 bodyParser 時，自行手動清除 temp 檔案

因為問題在於過多的 temp 檔案，所以可以手動方式清除

使用工具清除 temp 檔案

可以考慮使用tmpwatch 或是 reap 設定自動清除 temp 檔案

除非你真的知道你在幹嘛，否則就別用 multipart / bodpParser 了

這…盡量

在 multipart 使用 defer 參數

使用 express.multipart("defer": true); 的參數，在傳輸期間之內，及可以呼叫 next() 的 callback function

使用其他的檔案上傳 parser 套件

例如可以使用：

multipart busboy parted ormidable

There are tons of express.js tutorials that use bodyParser as an express middleware.

bodyParser itself is not bad if you know how to use it. The problem is that almost none of these tutorials warns you that using bodyParser could be bad. In this blog post I'm gonna cover one security issue which may come with bodyParser.

As you may know, bodyParser parses file uploads as well.

Imagine we create an express.js application and we use something like

app.use(express.bodyParser());

What happened now is that you are telling express JS to use bodyParser for all requests for a client.

Now the security issue. Imagine I am a bad guy and I want to fill up all your disk space witch may result in worst scenario into app crash.

Even you don't handle file uploads in your server I can actually upload a file just by editing your post form. Every post endpoint where you don't handle req.files is probably vulnerable.

So What I as a bad guy is that I load a form in your app and I add a new line to your form. This new line is just an input type file. Nothing else. I click submit and bodyParser do the magic. bodyParser will save a temp file in your temp directory.

That's all. Really trivial 'error' but it could be fatal for your app.

It's not very hard to avoid this. In short, you should always check the req.files but you must always think about it. You should not use bodyParser as express middleware for all routes. If you must use it, use it just in the routes where you must. Do not use bodyParser at all, you can use express.json(), express.urlencoded() etc. and for routes which handles file uploads you could use express.multipart() and be sure to clean up all temp files.

More with TJ suggestions is described in this blogpost.

Edit: