Attention: Answers.ros.org is deprecated as of August the 11th, 2023. Please visit robotics.stackexchange.com to ask a new question. This site will remain online in read-only mode during the transition and into the foreseeable future. Selected questions and answers have been migrated, and redirects have been put in place to direct users to the corresponding questions on Robotics Stack Exchange. Additional details are available here.
Ok, so I finally managed to succeed (after no less than 5 hours of trying). What I found out:
Try to get as good image quality as possible (major factor for me as I used a thermal cam)
Do not try to accumulate hundreds of images, it is very likely that calibration will never enable the calibration button if you have over a hundred or so. My successful try was with 59 samples. It is better to restart often than to stick to a failed calibration try.