DeepArUco++: improved detection of square fiducial markers in challenging lighting conditions.

In this project, we develop a neural network-based framework for robust fiducial marker detection and decoding in challenging lighting conditions. Our approach combines deep learning models for detection, corner refinement, and decoding, outperforming state-of-the-art methods.

Abstract

Fiducial markers are a computer vision tool used for object pose estimation and detection. These markers are highly useful in fields such as industry, medicine and logistics. However, optimal lighting conditions are not always available, and other factors such as blur or sensor noise can affect image quality. Classical computer vision techniques that precisely locate and decode fiducial markers often fail under difficult illumination conditions (e.g. extreme variations of lighting within the same frame). Hence, we propose DeepArUco++, a deep learning-based framework that leverages the robustness of Convolutional Neural Networks to perform marker detection and decoding in challenging lighting conditions. The framework is based on a pipeline using different Neural Network models at each step, namely marker detection, corner refinement and marker decoding. Additionally, we propose a simple method for generating synthetic data for training the different models that compose the proposed pipeline, and we present a second, real-life dataset of ArUco markers in challenging lighting conditions used to evaluate our system. The developed method outperforms other state-of-the-art methods in such tasks and remains competitive even when testing on the datasets used to develop those methods.

Code

To support the research community and encourage exploration, we have provided access to the code used for the development and testing of our models through our GitHub repository. Also, you can easily try our methods with no installation required through our Google Colab demo.

Read our work

A preprint of our paper can be accessed through ArXiv: link

Citing

If you use this work in your research, you must cite:

  1. Rafael Berral-Soler, Rafael Muñoz-Salinas, Rafael Medina-Carnicer, Manuel J. Marín-Jiménez, DeepArUco++: Improved detection of square fiducial markers in challenging lighting conditions, Image and Vision Computing, Volume 152, 2024, 105313, ISSN 0262-8856, https://doi.org/10.1016/j.imavis.2024.105313
  2. Rafael Berral-Soler, Rafael Muñoz-Salinas, Rafael Medina-Carnicer, and Manuel J. Marín-Jiménez. 2023. DeepArUco: Marker Detection and Classification in Challenging Lighting Conditions. In: IbPRIA 2023. Lecture Notes in Computer Science, vol 14062. Springer, Cham. https://doi.org/10.1007/978-3-031-36616-1_16

Contact

If you have any further questions, please contact rberral@uco.es.