Do not look for the perfect Computer Science conference rankings
Publications are one of the most important Key Performance Indicator (KPI)Ā Ā metric for an academic in Computer Science. However, like in all aspects of life, not all conferences are considered to be the same. The range of goodnessĀ varies from excellent, to good, to very poor. Computer Science conferences also have various sub-categories including (but not limited to) Machine Learning, Natural Language Processing, Databases and Algorithms. The definition of an area itself is subjective and ephemeral, like most things in Computer Science. There are various websites which give ratings/rankings to such conferences across Computer Science. Some of such rankings areĀ
CORE Conference Ranking ExerciseĀ
Microsoft Academic Search
Top quality research labs like Microsoft Research, Yahoo! Research and IBM Research Labs also have their own internal ranking conference list. But, these rankings themselves are based on various metrics like impact factor, citations and some of them could be based on factors apart from research. Thus, no such list is perfect and/or comprehensive.Ā So then, how does one make sense of these rankings andĀ rank theĀ goodnessĀ of a conference? Here's my recommendation - Rather than look for the perfect conference ranking list look for evidences that suggest the standing of the conference to argue your case.Ā If there are multiple evidences, there's not much anyone can argue about it.
So, how do I do so? Before we proceed, let us agree that there are four broad categories of conferences - A* (excellent), A(very good), B(good) and C(average). Ā Lets take an example.Ā ICWSMĀ is a very good conference for researchers working in social media and social computing. However, surprisingly ICWSM is not even listed on CORE. Now, one could argue that ICWSM came into existence in 2007 and hence, is too early to be listed. But that theory doesn't hold true asĀ WSDMĀ which is almost as good as ICWSM is listed on CORE as B. This is enough to show there are inconsistencies.Ā So, does it then mean that ICWSM is a bad venue to publish? Evidences suggest quite the contrary. ICWSM isĀ ranked #10 under the Databases & Information Systems area on Google Scholar with an h5-index of 41 while WSDM ranks #7 on the same list with an h5-index of 48. Now, the category assigned to ICWSM (A or B) is a separate issue but it seems a bit surprising that it doesn't deserve a place in CORE despite being one of the Top #10. One would also argue that Google Scholar would be just a counter example but Microsoft Academic Research ranks it #14 under the broad Data Mining category.Ā Lets look at an example of another conference - CVPR which is listed on CORE but ranked as A. But, is this an accurate reflection of the standing of CVPR? I would respectfully disagree. Microsoft Academic Research ranks CVPR as the top 3 conferences in Computer Science (read that last phrase once again) and ranks it #1Ā under the Computer Vision sub-category. You will observe a similar pattern on Google Scholar and ArnetMiner too. CVPR makes a very strong case for an A* rating.Ā
So, let us summarize the process I suggest -Ā
Look for a comprehensive independent research rankings like CORE, MS Academic Research, Google Scholar and ArnetMiner. You may include internal lists of other research labs but I doubt they will differ as much
Look for inter-list evidences viz. multiple evidences to suggest the conference rating. If a conference has high standing in multiple lists, it is more likely to be widely acceptable as a good conference.
However, some other questions remain -
What if my conference/research area is new ?
Well, there's nothing much you can do about it. One way to understand if a conference isĀ potentially goodĀ is to look at the program committee in addition to the selection rate of papers. A strong PC will more likely drive the conference to be a very good one in the future. However, as of this moment the truth is what you see it.
But, I see top researchers publish in a conference which is not listed or ranked high
People publish at venues for various reasons and sometimes these may be completely unrelated to research purposes. Don't fall into that trap - look for strong PC's coupled with selection rates to decide the standing of the conference. Even so, I would still recommend you to use Google Scholar or MS Academic Research to make your case.
But, it is unfair to ask me to publish in a A* conference!
Unfortunately, we don't have any control over the rules of the game. We only play the game. The only people who have control over the rules of the game are the ones who have excelled at it.Ā
But, I know of people who don't have publications in A* but work at A* places.
Good for you. In the end, this is all about expectation maximizationĀ for the place you want to be. Different places will have different KPIs, its your job to find out the KPIs and deliver on them. That said, academics generally look for publications as their KPI and hence, there's no running around from that part in your PhD. Ā
Well, I have intentionally left them out in my blog. I haven't published in a journal and hence wouldn't like to comment on it. However, I would be surprised to find if things are very different for journal rankings. Metrics may vary but the process would roughly be similar.