By José Ignacio Orlando, PhD -Subject Matter Expert in AI/ML @ Arionkoder- and Nicolás Moreira, Head of Engineering @ Arionkoder.
Back in the 2018 FIFA World Cup in Russia, the International Federation of Football Associations started to employ Video Assisted Referee (or VAR), a team of on-side officials that helps referees make decisions during the match by analyzing video footage of the plays.
While this helped prevent mistakes that could ultimately influence results, manually analyzing video to confirm an event is particularly slow for both player’s and audience’s expectations, as it pauses the game for too long, taking an average of 70 seconds. FIFA committed to overcome this limitation through technology, making partnerships with universities and tech companies all over the world to design a tool to mitigate this issue.
The ultimate solution is the so-called semi-automated offside technology, a system that helps VAR make more precise and fast offside decisions. It consists of an AI tool that simultaneously processes synchronized data coming from a connected ball and optical sensors. This way, they’re able to track the relative position of all players in the pitch and detect offside incidents in real time. This tool also emits alerts to the VAR referees, and makes reproducible offside decisions in 20-25 seconds that are communicated to the audience in an easy-to-follow, straightforward way.
Let’s dive a bit deeper into the details. The first ingredient of this system is Adidas’ official match ball for Qatar 2022, Al Rihla, which contains an inertial sensor in its core that reports physical data at a rate of 500 times per second. This information is processed to detect the exact kick point in a more accurate way than using conventional video broadcast or optical tracking tools.
In parallel, 12 dedicated cameras mounted underneath the roof of each stadium automatically track both the ball and 29 limb points detected on each individual player. These last points correspond to the position of the players’ limbs that are relevant to score goals, and therefore for offside calls. To accurately detect them, cameras make use of Computer Vision algorithms for pose estimation, which are particularly useful for tasks such as motion transfer, augmented reality and human activity estimation.
All this information is fed to an AI model that can predict the exact relative position of players in the pitch at the exact moment the ball is played. By exploiting this information, the AI automatically triggers a visual alert to the video operation room when it detects an offside position. Then, the “semi-automated” part of the system gets involved, as human officials at the VAR have to ultimately decide whether the system was right or wrong. To this end, the system indicates in the exact frame of the video the kick point and an automatically created virtual offside line. Hence, officials analyze this information and communicate with the referee through a headset if the incident is confirmed.
At the same time that the VAR communicates with the referee, the very same limb information and offside lines used to make the call is used by the AI to create a 3D replay animation for in-stadium fans and television viewers, so that they can understand where and who was involved in the offside.
As the Chairman of FIFA’s Referees Committee, Pierlugi Collina, said at a FIFA’s Living Football episode, referees are human beings that can make mistakes, and this automated system is intended to reduce the possibility of them occurring. Real physical tests of this technology started back in 2021, and different providers and system configurations were evaluated in FIFA-sponsored events such as the Arab Cup or the Club World Cup. FIFA authorities agreed that this tool was “very reliable” and “very accurate”, but no numbers have been released to the public for third-party experts to assess.
So what if this AI system fails? FIFA claims that having humans in the loop to make the final decisions is what prevents said failure. But humans helped by computers can also make mistakes, especially due to the so-called confirmation bias: if everyone says that the AI model is so trustworthy, why should I, as human, contradict its output? Moreover, what if the outputs that the tool is providing coincide with the decision I made, but they are actually wrong? This last concern was recently raised by many football fans on Twitter, especially after one (particularly tight) decision made in the Argentina vs. Saudi Arabia match.
AI is a remarkable tool that we as humans are leveraging in a myriad of applications. But being able to carefully evaluate and audit it is a key aspect if we seek to ensure its safety and real-life applicability. Furthermore, paying particular attention to the potential impact of certain AI biases in our decision-making processes is a significant quality factor that everyone should consider. Perhaps football is not so risky, but what if money, health or even life were involved?
At Arionkoder we are committed to ensuring good-quality AI solutions that are accurate and safe for your business and decision-making. Reach out to us and discover the amazing things we can achieve together!