The thing about the Sokal 2 thing is that, out of the seven accepted articles, three of them had fabricated data. So the only conclusion I can really draw from those three is that these three journals donāt look too hard to see if data is fabricated:
Gender, Place and Culture
If you want another list of journals that donāt look too closely at whether the results are real, you could look at the list of retracted papers by Anil Potti.
Journal of Clinical Oncology
New England Journal of Medicine
The Nature MedicineĀ one was published 2006. It was retracted in 2011. Hereās the retraction statement:
We wish to retract this article because we have been unable to reproduce certain crucial experiments showing validation of signatures for predicting response to chemotherapies, including docetaxel and topotecan. Although we believe that the underlying approach to developing predictive signatures is valid, a corruption of several validation data sets precludes conclusions regarding these signatures. As these results are fundamental to the conclusions of the paper, we formally retract the paper. We deeply regret the impact of this action on the work of other investigators.
Why wasnāt it retracted following the letter to Nature Medicine by Coombes Wang and Baggerly, which they published in 2007? They showed that you canāt get any of the results in the paper by following the data analysis methodology described in the paper. Some quotes:
Baggerly, Wang and Coombes:
1. We cannot reproduce their selection of cell lines.Ā ā¦
2. The lists of genes initially reported in the supplementary information on the Nature Medicine website are wrong because of an āoff-by-oneā indexing error. ā¦Ā
3. Using their software and lists of cell lines, we reproduced their published heatmaps for six out of seven drugs. (We could not reproduce the heatmap for cytoxan.) However, after correcting for the off-by-one error, we matched the reported gene lists exactly for only three out of seven drugs (Supplementary Report 9). The other lists contain outliers. ā¦Ā
4. For docetaxel, their software yields only 31 of their 50 reported genes. ā¦
5. Their software does not maintain the independence of training and test sets, and the test data alter the model. ā¦
6. The interaction between point 4 (accidentally including genes from Chang et al.2 whose expression levels separate the test set responders from nonresponders) and point 5 (combining training and testing data when choosing gene weights) can produce 'better than chanceā predictions in the wrong direction. ā¦
I think the reason, I think what happened between 2007 when this letter was published and 2011 when the paper was retracted, is that it was revealed that Anil Potti had lied about being a Rhodes scholar on his job application. That was 2010.
I donāt think thereās any way to conclude from Sokal 2 that Gender, Place and Culture is a stupid journal but Nature Medicine is fine. They both publish made-up results and donāt look too hard at them. (and if you want to conclude anything else about a journal, you have to submit an article without made-up data, like the Sokal2 article published by Fat Studies, which was very silly.)