ICDAR 2017 Competition on Historical Document Writer Identification (Historical-WI)


Like for past writer identification competitions Historical-WI uses a leave-one-out strategy for evaluation. This means that a ranking according to the similarity of the handwriting has to be generated for a reference document. The performance measurement used is the mean average precision (map), which is a standard evaluation metric for information retrieval (Wikipedia) because it not only measures the precision, but also the ranking itself. If the other 4 pages of an author ranked at the first 4 places for all documents, a map of 100% is achieved. Also the precision (Top 1) is calculated and shown to the participants, but will not be taken into account for generating the final ranking. For submissions, the participants have to write a text file in which each line contains the basename of the reference image and then the images ranked according to the similarity of the handwriting.
[ref image],[most similar],[2nd most similar],…. [least similar]
So for images image1.jpg, image2.jpg, image3.jpg, image4.jpg, image5.jpg a possible submission file can look like this:
image1,image5,image3,image2,image4
image2,image3,image1,image5,image4
image3,image2,image4,image5,image1
image4,image5,image1,image3,image2
image5,image1,image4,image2,image3

Since this file is arround 60 mb you have to create a zip archive containing only this file. This archive is then uploaded to SciptNet and automatically analyzed and the participants see their results. Note that there is maximum number of submission per week. You are not able to see the results of the other participants.

News

Website online Dataset released change of the submission date

Important Dates

Release of the Dataset: 17.03.2017

Submission open 23.06.2017

Submission Deadline: 05.07.2017
12.07.2017





Organizers







Stefan Fiel

[TU Wien, Computer Vision Lab] 

Stefan Fiel works as research assistant at the Computer Vision Lab, TU Wien.

Basilis Gatos

[NCSR Demokritos / IIT / CIL] 

Researcher at the Institute of Informatics and Telecommunications of the National Center for Scientific Research "Demokritos", Athens, Greece.

Georgios Louloudis

[NCSR Demokritos / IIT / CIL] 

Research Associate at the Institute of Informatics and Telecommunications of the National Center for Scientific Research "Demokritos", Athens, Greece.

Nikolaos Stamatopoulos

[NCSR Demokritos / IIT / CIL] 

Research Associate at the Institute of Informatics and Telecommunications of the National Center for Scientific Research "Demokritos", Athens, Greece.

Markus Diem

[TU Wien, Computer Vision Lab] 

Markus Diem is a senior scientist at the Computer Vision Lab, TU Wien, Austria. His research interests are Cultural Heritage Applications and Document Analysis.

Florian Kleber

[TU Wien, Computer Vision Lab] 

Florian Kleber is currently a senior scientist at the Computer Vision Lab, Institute for Computer Aided Automation, TU Wien, Austria. His research interests are Cultural Heritage Applications and Document Analysis Applications.

Vincent Christlein

[FAU Erlangen-Nürnberg, Pattern Recognition Lab] 

Vincent Christlein is a PhD student at the Pattern Recognition Lab, FAU Erlangen-Nuremberg.