Is the Laser Egg only accurate in Beijing?
A few months ago, Smart Air tested the Laser Egg, the Node, and the Dylos against official PM2.5 readings. The results were way better than I expected! All three machines correlated r = .90 or higher with the gold-standard numbers, which is very high.
But this test may have exaggerated their accuracy. See, the Laser Egg and the Node are probably calibrated to PM2.5 readings in Beijing, so their formulas work great here. But how well do they work in different environments?
Why Your Air Quality Monitor Might Not Be Accurate in Shanghai
Lots of environmental differences could make particle counters less accurate outside of Beijing. Two examples:
Beijing tends to be pretty dry, and we know humidity affects laser particle counts.
Some cities may also have more of certain size particles. For example, Beijing might have more particles from the steel factories in Hebei, whereas Shanghai might have more vehicle exhaust particles. If these particles tend to be different sizes, the formula converting particle counts to micrograms to would be less accurate.
Testing the Laser Egg in Guangzhou
Smart Air DIYer Nikki in Guangzhou teamed up with me to test her Laser Egg against the US Consulate in Guangzhou.
Nikki took 147 readings outside her apartment, which is 2.77 kilometers away from the consulate. This distance can make the readings less accurate, although my tests with the US Embassy in Beijing from 7 kilometers away showed results nearly identical to tests done 200 meters away.
The Laser Egg correlated highly with the US Consulate (r =.88), but much lower than in Beijing (r = .98). The Egg consistently overestimated PM2.5:
The Laser Egg was off by an average of 23.53 micrograms. That is higher than in Beijing, where it was off by just 6.5 micrograms. (Open-source raw data is free for all).
Is The Laser Egg Inaccurate Because of Humidity?
I tested whether humidity was the culprit by analyzing humidity readings. Humidity ranged from 36% to 89%, so there was wide variation in humidity.
Adjusting for humidity increased the accuracy of the Laser Egg readings. The model went from explaining 76.8% of the variance in PM2.5 to 79.4%. The Egg clearly had less error when humidity was low (left) than when humidity was high (right):
The graphs below are a bit complicated, so let me break them down. First, if the Laser Egg were perfectly accurate, 1 microgram on the Laser Egg would equal 1 microgram from the US Consulate (blue line):
But the Laser Egg isn’t 100% accurate. When humidity is low (< 50%), the actual relationship is off by a bit. In that case, 1 Egg microgram = .72 actual micrograms (middle line). And when humidity is high (> 50%), the actual relationship is even worse, 1 Egg microgram = .60 actual micrograms (right line).
The Laser Egg is less accurate in Guangzhou than in Beijing.
Humidity can explain at least some of this error.
If you’re in a more humid climate, actual PM2.5 is probably lower than what your Laser Egg is reading.
Is This Just a Laser Egg Problem?
I’ve also done lots of testing with the Dylos particle counter and found that high humidity makes it less accurate too. So have researchers at Drexel University.
So far, I’ve never seen a particle counter that takes into account humidity. Thus, I don’t think this is just an Egg problem, so I don’t mean to single out the Egg for criticism! Overall, the data has convinced me that it’s a great machine for 600 RMB, and that’s why I decided to carry it on our site.
Laser Egg Is Working on It
I talked about this data to Liam, who made the Laser Egg. Liam says the upcoming Laser Egg 2 [update: now released!] will address the humidity problem:
“We’ve been working on this for almost two years and can say with confidence that the results people will be seeing are waaaay past anything else out there. In the hardware, we’ve made a few improvements, including the addition of a temperature and relative humidity sensor (well calibrated), as well as better controlling the relative humidity inside the Laser Egg via a heated circuit board near the inlet. By heating the inlet (same thing most government monitors do) the relative humidity level can be reduced, ensuring the inside of the Laser Egg never exceeds 80% humidity. There’s a good study that shows the major effects of humidity on particle measurement take place over 80% humidity.”
This could make the Laser Egg more accurate in more cities, although I’ll still want to test it to make sure!
Open Data and a Correction Formula
As always, I’m posting the original data and more info on the test for fellow nerds, plus:
A correction formula that users in high humidity cities like Guangzhou can use
Is it really humidity? One reason it might be another variable.
One reason I might actually be underestimating the effect of humidity!
Egg users in humid climates can use this formula to correct their numbers:
PM2.5 ug/m3 =(Laser Egg micrograms x .981) + (Humidity x .052) + (Laser Egg micrograms xHumidity x -.006)
But remember this is a very provisional formula, based on one egg in one city.
Maybe It’s Not Actually Humidity?
Liam also pointed out a potential problem with analyzing humidity: changes in humidity may be confounded with other changes. For example, in Beijing, northern winds bring drier air, but they also probably bring different types of particles, such as the well-known spring sand storms. So when humidity drops, particle type also changes.
But the problem is, we can’t tell different types of particles apart, so we don’t know. All we have is humidity. And because we’re only measuring humidity, we’ll incorrectly conclude, “look at all this effect of humidity!” What’s more, if humidity is confounded with other variables, that would mean a humidity formula built on one city wouldn’t work as well in other cities.
Then again, perhaps I’m underestimating the effect of humidity. Surely there’s some error in our measurement of humidity, and that error might lead us to underestimate the effect of humidity.
Ultimately, I think most people would agree that there’s at least some effect of humidity on particle counters. If humidity is confounded with other particles, that would probably mean some effect of humidity is actually due to other variables. It’s highly unlikely that all of the effect of here is due to third variables.
Is Humidity Confounded with High AQI?
One potential problem with the analysis I did is if higher humidity tends to be accompanied by worse AQIs. Why? Because absolute error tends to be higher when readings are really high. For example, if the air is at 20 micrograms, it’s really hard to be off by more than 10 micrograms or so. But when the air is at 200 micrograms, it’s easy to be off by 20, 30, 40 micrograms. When concentration is high (according to the US Consulate), Egg error tends to be higher r = .45.
However, humidity was actually negatively correlated with PM2.5 (as measured by the Consulate) r = -.23. So if anything, my formula here might be underestimating how much we need to correct for humidity.
