Congratulations to Michael Jungo, PhD student at iCoSys for winning the best student paper award at the International Conference on Document Analysis and Recognition – ICDAR 2023. The conference took place in San Jose, California, and is considered one of the top conference for the Document Analysis and Recognition – DAR community.
The paper, entitled “Character Queries: A Transformer-Based Approach to On-line Handwritten Character Segmentation“, is the result of a research collaboration between iCoSys and Google.
Paper citation: Jungo, M., Wolf, B., Maksai, A., Musat, C., Fischer, A. (2023). Character Queries: A Transformer-Based Approach to On-line Handwritten Character Segmentation. In: Fink, G.A., Jain, R., Kise, K., Zanibbi, R. (eds) Document Analysis and Recognition – ICDAR 2023. ICDAR 2023. Lecture Notes in Computer Science, vol 14187. Springer, Cham. https://doi.org/10.1007/978-3-031-41676-7_6
Paper abstract: On-line handwritten character segmentation is often associated with handwriting recognition and even though recognition models include mechanisms to locate relevant positions during the recognition process, it is typically insufficient to produce a precise segmentation. Decoupling the segmentation from the recognition unlocks the potential to further utilize the result of the recognition. We specifically focus on the scenario where the transcription is known beforehand, in which case the character segmentation becomes an assignment problem between sampling points of the stylus trajectory and characters in the text. Inspired by the k-means clustering algorithm, we view it from the perspective of cluster assignment and present a Transformer-based architecture where each cluster is formed based on a learned character query in the Transformer decoder block. In order to assess the quality of our approach, we create character segmentation ground truths for two popular on-line handwriting datasets, IAM-OnDB and HANDS-VNOnDB, and evaluate multiple methods on them, demonstrating that our approach achieves the overall best results.
