Abstract

This paper summarises objectives,organisation,andresultsofthefirst news recommendation evaluation lab (NEWSREEL 2014). NEWSREEL targeted the evaluation of news recommendation algorithms in the form of a campaign-style evaluation lab. Participants had the chance to apply two types of evaluation schemes. On the one hand, participants could apply their algorithms onto a data set. We refer to this setting as off-line evaluation. On the other hand, participants could deploy their algorithms on a server to interactively receive recommendation requests. We refer to this setting as on-line evaluation. This setting ought to reveal the actual performance of recommendation methods. The competition strived to illustrate differences between evaluation with historical data and actual users. The on-line evaluation does reflect all requirements which active recommender systems face in practise. These requirements include real-time responses and large-scale data volumes. We present the competition’s results and discuss commonalities regarding participants’ approaches.

@inproceedings{Kille:2014,
  author    = {Benjamin Kille and Torben Brodt and Tobias Heintz and Frank Hopfgartner and Andreas Lommatzsch and Jonas Seiler},
  title     = {{NEWSREEL} 2014: Summary of the News Recommendation Evaluation Lab},
  booktitle = {Working Notes for {CLEF} 2014 Conference, Sheffield, UK, September
               15-18, 2014.},
  year      = {2014},
  pages     = {790--801}
 }