#supplemental data

This article includes supplemental data for the main article “So You Thought Hong Kong’s Pollution Comes From the Mainland?”

This supplemental data will show how high and low wind speeds affect the PM2.5 levels and hence Hong Kong’s pollution. Ultimately, this supplemental article draws the same conclusion as the main article.

  Does Wind Speed Affect PM2.5?

What if particulate matter can only be transported at high speeds? In order to test if wind speed affects PM2.5, I repeated the analysis for PM2.5 data at wind speeds below average and above average.

The data shows that lower wind speeds in Hong Kong almost always correlated with lower PM2.5 levels, regardless of the wind direction.

However, this also shows that air pollution was significant even without strong winds blowing PM2.5 from Mainland China. This means that there are PM2.5 sources within Hong Kong as well.

When wind was blowing at higher speeds from the direction of Mainland China, the PM2.5 level was about 20% higher than average.

Get the latest clean air tips!

Data and Test Details

  Test Methods: These tests were identical to my previously published tests, details of which can befound here, here, and here.

Original Data: The original data for all 13 tests is available as an Excel file here

  Average Outdoor PM2.5 During Tests

It’s important to note the average outdoor PM2.5 during tests because purifiers achieve smaller percentage reductions when outdoor air is really bad. According to the US Embassy, outdoor PM2.5 was 128.2 micrograms during the Xiaomi tests and 108.4 during the Philips tests. The tests were done in the winter, so these averages about normal for Beijing.

  Theoretically, if the auto mode is working as it’s supposed to, outdoor air pollution shouldn’t matter. If air is bad, the machine should just spend more time in high mode.

  Fluctuations in Outdoor Pollution

If outdoor pollution gets a lot worse during the test, that could make it seem like the purifier isn’t working well (although again, if auto mode is working like it should, that shouldn’t matter, at least pollution exceeds the ability of the high setting). But to be conservative, I re-analyzed the data with fluctuations in mind.

  During the Philips tests, outdoor air was remarkably stable. The only large fluctuations worked in the Philips’s favor (falling during the test). During those two days, the Philips averaged 50.9% reductions in 0.5 micron particles, so no big difference.

  During the Xiaomi tests, outdoor air varied a lot. Outdoor PM2.5 fell on some days and rose on some days. The fluctuations didn’t seem to matter much. For example, on the most extreme day, outdoor air rocketed from 178 micrograms to 467, yet the percentage reduction was right near average (65% versus 61%).

  In sum, with the auto mode tests, I think outdoor fluctuations are less important than when the purifier is on a constant setting. Instead, a lot rides on whether the auto mode has decided to turn on at the moment the Dylos takes a reading. That introduces randomness into the results, although with enough tests, the randomness should smooth out.

  Converting Dylos 0.5 Microns to PM2.5: As in previous tests, I converted the Dylos 0.5 micron counts to PM2.5 ug/m3 by dividing them by 100. In prior tests, this semi-official formula from Dylos worked pretty well.

  Can you really not keep the Xiaomi 2 on high setting?

Apart from the tests we have done, we went to Xiaomi directly just to confirm on this. Here’s what they say:

  Translation:

Smart Air: Is it not possible to keep the purifier on Favourite mode?

Xiaomi: No, it’s not possible

Smart Air: I heard a friend say you can use the Xiaomi App to keep the purifier running on Favourite mode.

Xiaomi: It’s not possible to keep the purifier constantly running on favourite mode

This article includes supplemental data for the main article “Xiaomi Mi 2 Auto Mode Leaves Air Unsafe for 86% of hours | Review“

Data and Test Details

Settings

The app asks what size the room is. How Xiaomi uses that data is a little opaque, but it would be logical that settings for larger rooms will run the purifier harder, so we used the highest allowable setting, 34-37m2.

It should be noted that the Xiaomi also has a turbo mode, which I’ve heard people describe as “sounding like a jet taking off.” This really isn’t meant to be used while people are in the room. Instead, it’s meant to be used temporarily before returning home.

Tests in a Different Room

We’ve used this same bedroom in Chaoyangmen, Beijing for many of our tests—of the Original DIY, Cannon, Philips, Blue Air, and IQ Air. So I doubt there’s something weird about this room that somehow hurts the Xiaomi but not other machines.

But you never know! So to be sure, we conducted an overnight test in the Smart Air office near Sanlitun. The results are shown in the noise test above. They show the same pattern as the other room tests. Thus, the results don’t seem to be something weird happening with that particular test room.

Tests with a Different Particle Counter

We’ve used the Dylos Pro for most of our tests, so this is constant across tests. But particle counters can break or lose accuracy over time. Thus, we carried out the noise/particle test above with a different Dylos Pro. The fact that the pattern of the results is the same suggests the results are not because of any problems with the particle counter. (That conclusion is also hinted at by the fact that the Xiaomi performed well for the first three hours while on high.)

How Bad Was Outdoor Air?

It’s important to analyze how bad outdoor air was during the tests because my analyses have shown it’s harder to achieve a high percentage reduction when outdoor air pollution is bad. (Or put another way, it’s easier to achieve a high percentage reduction on relatively clean days.) For example, here is the relationship between effectiveness and outdoor air pollution for the Blue Air 203:

However, theoretically, the Xiaomi auto mode shouldn’t be affected by outdoor air pollution. If it can accurately detect air pollution levels and turn on the fan in response, the results shouldn’t be affected by outdoor air pollution levels (until we get to levels that are too high for even constant high mode to clean). But to be conservative, I analyzed hourly outdoor air pollution data from the US Embassy, about 7 kilometers from Anna’s home.

During the auto mode tests, outdoor air averaged 128 micrograms. During the high-mode tests, outdoor air averaged 246 micrograms. Both of these are higher than the average in Beijing (90-100 micrograms). So it’s worth seeing how the Xiaomi did on days with lower pollution.

I analyzed the four days with lowest outdoor air pollution (average 89 micrograms; 11/29, 12/2, 12/5, 12/7). On these days, the Xiaomi averaged a 56% reduction in 0.5 micron particles and 87% in 2.5 micron particles. Thus, the poor results in the main tests do not seem to be because of outdoor air pollution levels.

Outdoor Air Fluctuations

Besides the baseline level of outdoor air pollution, it’s also important in these real-world tests to look for large fluctuations in outdoor air pollution. If outdoor air gets a lot worse during the test, it can look like the purifier is not cleaning the air very well. And on the flip side, if outdoor air pollution goes down a lot during the test, it can look like the purifier did a great job.

First, I analyzed the data after removing any test day where outdoor air fluctuated more than 100 micrograms from beginning to end (4 days total). That left 8 test days. The result was very similar: on these stable days, the Xiaomi average a 67% reduction in 0.5 micron particles and 86% in 2.5 micron particles.

Next, I analyzed all days where outdoor PM2.5 micrograms changed no more than 50% from baseline at any point during the test. This “at any point” criterion is more stringent, and it left three test days (12/11, 12/12, 12/18). Again, the results were nearly identical: a 69% reduction in 0.5 micron particles and 83% in 2.5 microns. In sum, the poor results did not seem to be caused by fluctuations in outdoor air pollution.

% Hours of Unsafe Air

To calculate the percentage of unsafe air for the Blue Air 203/270E (3,600 RMB), and Philips AC4072 (3,000 RMB), I used the data in my previously published tests. For the Cannon, I used three series of tests: my original tests, tests where tested whether adding a pre-filter affects performance (below), and tests comparing performance after adding a carbon layer.

That gives a lot of data! Here’s exactly how many hours of data I had:

Purifier

# Hours of Unsafe Air (>25ug)

Average PM 2.5 During Tests (ug/m3)

Phillips

45 78

Blue Air

41

113

Xiaomi

93

209

Cannon

98

111

IQ Air 80

146

% Unsafe Hours: How bad was outdoor air?

It’s important to compare just how bad outdoor air was during the tests. If one machine has lots of hours where air was just above 25 micrograms, it would be a lot easier for the machine to clean the air. So, I calculated the average outdoor micrograms for all of these unsafe hours. For reference, Beijing’s air has averaged about 90-100 micrograms for the last 7 years according to the US Embassy.

In the chart above, I analyze what the average outdoor PM2.5 concentration was for the different tests. The Xiaomi really stands out, and that could unfairly lower its results. Thus, I re-analyzed data only looking at days with lower outdoor air pollution, with an average of 138 micrograms—lower than the IQ Air. The result was similar to the original analysis: 83% of hours were unsafe.

To be even more conservative, I analyzed the two days with the lowest outdoor concentration, averaging 108 micrograms. That is lower than all of the other machines except the Philips. On these days, 68% of hours were unsafe. Thus, the Xiaomi was leaving far too much unsafe air, even on days with lower outdoor AQI.

Nerd note: In calculating the percentage of time of unclean air in the main text, I accidentally excluded the shortest test day (12/11). So I re-did the analysis with this data included, and the new result was that 85% of hours were unsafe. This oversight makes little difference, but I think it’s important to note for completeness.

  The Xiaomi Test Compared to Other Recent Tests in the Same Room

I’ve published dozens and dozens of days of test data using a Dylos particle counter in this exact same room (1, 2, 3, 4, 5, 6). That makes me pretty confident these poor results are not some basic flaw of the test design. However, there is always the possibility that the particle counter will break or lose its accuracy slowly over time. Or maybe some neighbor is setting up a secret chuan’r stand nearby. Fortunately, I have data from just two weeks earlier in the same room with the same particle counter.

If something is wrong with the particle counter, we should get weird results for these tests. But over several tests with a new-and-improved Original DIY 1.1, the results are pretty much what I’d expect—a modest percentage higher than results for the DIY 1.0. Here’s a sample test day for the DIY 1.1 two weeks before the Xiaomi test:

That test data looks normal to me. And that suggests that there’s nothing strange going on with the room or the particle counter recently.

Converting Particle Counts to PM2.5 Micrograms

The tests used a Dylos Pro particle counter. To convert to PM2.5 micrograms, I took the 0.5 micron particles divided by 100. This formula comes from Dylos, and our prior tests show it’s pretty accurate compared to the US Embassy—a correlation of r = .90.

Original Data: Xiaomi

Below is the original data for the Xiaomi tests, including the outdoor PM2.5 levels as recorded by the US Embassy. My comparison data for the Blue Air, Philips, IQ Air, and DIY Cannon are available through in my earlier post. To request a copy of the original data in Excel format, send an email to [email protected].

  Original Data: Noise + Particle Count Test

Here is the original data for the test done in a different room with a different Dylos while decibel readings were taken:

Xiaomi Noise + Particle Count

Original Data: The Xiaomi Can’t Stay on High? 

Below are the transcripts from Xiaomi customer service where we asked whether or not the Xiaomi 2 can stay on high mode continuously.

Transcripts with Xiaomi Customer Service About Auto Mode

Original Data: Cannon with HEPA + Pre-Filter

Methods

Results were identical to my earlier tests of the Original DIY, Blue Air, IQ Air, Xiaomi, and Philips. Anna conducted the overnight tests in her 15m2 Beijing bedroom. Doors and windows were closed during the test (except for occasional opening during the first couple hours before bed—that random variation is one reason I use the average of the last four hours to calculate effectiveness).

Anna used the Dylos Pro DC1700 to take a baseline measurement of the particulate in her room before turning on the purifier. She then turned the purifier on and left it on all night. The Dylos takes measurements every hour in hourly mode. I calculated effectiveness as (baseline particulate) – (the average of the last four hours before waking up).

  Daytime Tests

It’s more convenient for Anna to run tests during the day, so she first ran 8 daytime tests. When I discovered this, I asked her to run nighttime tests instead because all of our previous tests were nighttime tests, and I didn’t want to unnecessarily change a variable. So Anna then ran 11 nighttime tests.

My worrying seems to have been unnecessary. The average of all tests combined was 84% of 0.5 micron particles and 93% of 2.5 micron particles. The average of the night tests alone was 85% and 92%. These averages are different from the averages in the main article because…

  Outdoor Pollution Fluctuations

I argued in the main article that room tests are a coarse method for testing for small differences in effectiveness between two machines. Random fluctuations in outdoor PM2.5 during the tests might add enough noise to the data to outweigh small differences in effectiveness. (But room tests do consistently reveal large differences in effectiveness. These room tests have routinely found differences in effectiveness between the Original DIY and the Cannon. They’ve also spotted consistent poor performers like this DIY knockoff with a low-performing HEPA and the Xiaomi’s auto mode.)

So to detect small differences, I put more weight on the CADR tests. And for that reason, I tried extra hard in this data set to isolate days with stable outdoor PM2.5.

First, I pulled out all of the test days where outdoor PM2.5 fluctuated no more than 50 micro-grams during the test. That left 6 tests with an average outdoor PM2.5 of 90 micro-grams, which is right near Beijing’s average for the last few years. I report this result in the main article: 86% and 94%.

Another way to do this is to pull out the day with the most stable outdoor PM2.5. The data from January 16th stick out for being remarkably stable, with an average of 122 micro-grams. The results from that day are almost identical to the average in the main article: 86% and 95%.

But wait, it’s not a fair comparison if I’m pulling out the very stable days for the DIY 1.1 and not the DIY 1.0. Indeed! I did the same analysis in my most recently published tests of the DIY 1.0 (tests of the slightly smaller 29mm HEPA). In that dataset, the results on the 8 stable days where outdoor PM2.5 fluctuated no more than 50 micrograms was 84% and 91%. That’s nearly identical to the 84% and 92% values for the DIY 1.0 here. Thus, I’m moderately confident that the room tests are picking up on a difference in effectiveness between the 1.0 and the 1.1 (although I give the final say to the CADR tests).

  But Wait, Don’t We Need a Control Condition?

How do we know particulate decreased because of the purifier? Maybe it’s because Anna was breathing in all that particulate. Maybe particulate goes down just because the doors and windows were closed. Maybe outdoor particulate went down during the test.

These are fantastic sources of skepticism. To test for these alternatives, I’ve run tests using a nearby control room, which rules out some of those alternative explanations. Some people suspect that PM2.5 is naturally lower at night (that’s what I always thought), but PM2.5 data in Beijing and other cities shows that the opposite is true. PM2.5 is on average higher at night. That means running overnight tests actually makes it harder to show an effect (on average). But the better way to answer that question is to look at data from shorter tests and tests with the control room rule out the idea that these reductions are because of changes in outdoor air pollution.

Finally, are humans good purifiers? I’ve analyzed particulate data from rooms with and without humans to test this question. The results are abysmal. I don’t recommend buying a human purifier.

  Average Outdoor PM2.5 During Tests

As I’ve argued before, it’s important to analyze the average outdoor PM2.5 during the tests. That’s because EVERY purifier I’ve tested achieves lower percentage reductions on days with high outdoor pollution. It’s harder to fight particulate when there is really thick particulate seeping into the room.

For the 6 stable AQI days, the average outdoor PM2.5 was 90 micro-grams, which is right around the average in Beijing for the last few years. Over all 19 tests, the average outdoor PM2.5 was 121 micro-grams. These values are similar to values during tests with the Blue Air (113 micrograms) and the Cannon (111 micrograms).

  Original data: DIY 1.1

The attached ZIP file below is the original data for the DIY 1.1 tests for different days, including the outdoor PM2.5 levels as recorded by the US Embassy. My comparison data for the Blue Air, Philips, IQ Air, and DIY Cannon are available through in my earlier post. To request a copy of the original data in Excel format, send an email to [email protected].

SICKPAPES SPECIAL EDITION ON SUPPLEMENTAL DATA!!!

Supplemental Text 1 from:

Deshpande, G., Zhou, K., Wan, J.Y., Friedrich, J., Jourjine, N., Smith, D., Schedl, P., 2013. The hedgehog Pathway Gene shifted Functions together with the hmgcr-Dependent Isoprenoid Biosynthetic Pathway to Orchestrate Germ Cell Migration. PLoS Genet. 9, e1003720.

Like many people, I have conflicted views about the reams of Supplemental Data that accompany the online versions of most papers published these days. At its worst, Supplemental Data is nothing more than a grainy video of a bat performing fellatio, overdubbed with thumping techno.  At it’s best, however, a truly great Supplemental Figure reminds me of the famous scene in When Harry Met Sally where Meg Ryan loudly fakes an orgasm in a crowded restaurant, except that for me, the orgasm is real, and in those moments of ecstasy I am neither woman nor man but purely divine flesh of the GodBody incarnate. This Supplemental Text Document right here is one of those good ones which make me very glad to be living in the “Era of Supplemental Data,” as it was recently dubbed by Francis, the first pro-Open Access Pope.

To be real with you, I’ve never seen a Supplemental Document quite like this one. While Supplemental Data most often includes extra methodological details, additional controls, large datasets, or tangential experiments requested by reviewers, this one is a long-form prose essay about a decade-old scientific disagreement between these authors and a different lab. We here at SickPapes don’t take sides in this quagmire - both of these labs are outrageously hot and time-tested - but we find it surprisingly compelling to read such an emotionally honest and open piece about a genuine scientific disagreement. 

Our story begins in 2001, when Paul Schedl’s lab published a paper in Cell providing evidence that a secreted protein called hedgehog is involved in guiding embryonic germ cells as they migrate towards the future gonad. In one key set of experiments, they ectopically expressed hedgehog in abnormal locations, and showed that germ cells migrated incorrectly. This was interpreted to suggest that hedgehog is sufficient to influence germ cell migration.

As far as the public was concerned, the next thing that happened was In 2007, when Ruth Lehmann’s lab published a rebuttal, with the unambiguous title “hedgehog does not guide migrating Drosophila germ cells.” In the Lehmann lab, the hedgehog ectopic expression simply did not affect germ cell migration as it did in the hands of the Schedl lab members. This was not a matter of subtle differences in methods, this was literally a direct repeat of a simple experiment, giving different answers. Both labs are highly respected, and this discrepancy was hard to explain without getting a tad bit disrespectful. 

Which brings us to explaining why this new Supplemental Essay from Schedl’s lab is so sick: it explains what was happening behind the scenes at Cell the whole time. First, they reveal that the Lehmann lab’s 2007 paper actually originated as a technical comment sent to Cell in 2002 in response to the original 2001 article. The Cell editors asked the Schedl lab if they wanted to retract their paper, and the Schedl lab said “No.” Instead, they came to an agreement with Cell that an outside, independent researcher would repeat the hedgehog experiments, communicating only with Cell and not with either of the labs in question (Shout out to Stephen DiNardo, stepping in to do a major solid for the scientific community without any personal glory).

After an extended drum-roll, Dr. DiNardo told Cell that his independent experiments confirmed the original results from the 2001 Schedl lab paper, not the Lehmann results. So, what does Cell do with this important finding? Nothing. Instead of publishing this informative back-and-forth, they didn’t make any of this public, and just let everyone gossip for a decade. And, according to the Schedl lab in this Supplemental Text, this gossip-filled silence has “undermined [their] credibility in the scientific community, jeopardizing [their] careers.” Damn, Cell - that is some cold shit. I wish more folks would publish Supplemental Emotionally Honest Essays detailing the strife that various journals put them through. At the least, this might add fuel to Randy Schekman’s dope protest against journals like Cell.