At the 1863 exhibition of works from the Paris Salon, there was a room dedicated to paintings rejected by the jury. Inspired by this, Ule von Luxburg and Isabelle Guyon, this year’s NIPS program chairs, have invited authors of rejected papers to exhibit their work.
Quoting them:
As a result of the NIPS review process, we accepted less than 600 papers out of nearly 2500 submissions. With a review process of this scale it is unavoidable that among the rejected papers there are hidden gems that simply drowned in the review process. For this reason we propose an exhibition of rejected papers, in a spirit similar to the Salon de Refusés. Below you can find a list of those rejected NIPS submissions whose authors decided to take part in the exhibition. Neither NIPS nor the program committee provides any warranty about the quality of these papers.” Isabelle Guyon and Ulrike von Luxburg
Initially, Bob Williamson was kindly hosting these papers on his site, but to ensure a more viable long-term home I’ve put together a github-based home for them on salon-des-refuses.org. It should be easily expandable for any future exhibitions.
The NIPS 2016 exhibition is hosted here. Apparently the rejected papers of the Paris Salon attracted ridicule and scandal in 1863 … although many of them are now considered to be among the most important works of the time.
A few years ago Corinna Cortes and I performed the NIPS Experiment, which showed that the selection of papers can be inconsistent. It is unsurprising that this is the case, when only a few people are asked for their subjective opinion of a work. I don’t think I could have put it much better than Bob Williamson has done:
As the NIPS experiment showed, the consistency of the reviewing process is far from perfect. As Eric Price wrote in a blog post about the experiment: “In other words, most papers at NIPS would be rejected if one reran the conference review process .” A hallmark of every good scientist is to always recognise that they may have been mistaken. The list below of papers rejected by the review process is an admirable recognition of the cognitive limitation that we all share. No doubt many of these papers have flaws. But recognising several former NIPS program chairs as authors makes me certain that there are, as Isabelle and Ule say, likely to be some gems here Bob Williamson
You can also find details of our technical analysis of the NIPS experiment here.
Well done to Ule and Isabelle for undertaking this initiative.