Stefan Fiel works as research assistant at the Computer Vision Lab, TU Wien.
ICDAR 2017 Competition on Historical Document Writer Identification (Historical-WI)
Like for past writer identification competitions Historical-WI uses a leave-one-out strategy for evaluation. This means that a ranking according to the similarity of the handwriting has to be generated for a reference document. The performance measurement used is the mean average precision (map), which is a standard evaluation metric for information retrieval (Wikipedia) because it not only measures the precision, but also the ranking itself. If the other 4 pages of an author ranked at the first 4 places for all documents, a map of 100% is achieved. Also the precision (Top 1) is calculated and shown to the participants, but will not be taken into account for generating the final ranking.
For submissions, the participants have to write a text file in which each line contains the basename of the reference image and then the images ranked according to the similarity of the handwriting.
[ref image],[most similar],[2nd most similar],…. [least similar]
So for images image1.jpg, image2.jpg, image3.jpg, image4.jpg, image5.jpg a possible submission file can look like this:
Since this file is arround 60 mb you have to create a zip archive containing only this file.
This archive is then uploaded to SciptNet and automatically analyzed and the participants see their results. Note that there is maximum number of submission per week. You are not able to see the results of the other participants.
NewsWebsite online Dataset released change of the submission date
Release of the Dataset: 17.03.2017
Submission open 23.06.2017