Research shows AI models struggle with two tasks | Tech News

Lead writer Rohit Saxena, a researcher on the University of Edinburgh, launched a assertion in regards to the findings, saying that the failures needed to be corrected.

“Most people can inform the time and use calendars from an early age. Our findings spotlight a important hole within the capacity of AI to hold out what are fairly primary abilities for people,” she mentioned.

“These shortfalls must be addressed if AI systems are to be successfully integrated into time-sensitive, real-world applications, such as scheduling, automation and assistive technologies.”

The researchers investigated AI’s timekeeping abilities by feeding a custom dataset of clock and calendar images into various multimodal large language models (MLLMs), which can process visual as well as textual information. They used the models Meta’s Llama 3.2-Vision, Anthropic’s Claude-3.5 Sonnet, Google’s Gemini 2.0 and OpenAI’s GPT-4o.

What resulted was a disappointing performance from AI, as more than half of the time the models were unable to identify the correct time from an image of a clock or the day of the week for a sample date.

According to the researchers, there is an explanation as to why AI’ is so poor at time-reading.

Saxena explained, “Early systems were trained based on labelled examples. Clock reading requires something different — spatial reasoning.”

“The model has to detect overlapping fingers, measure angles and navigate various designs like Roman numerals or stylized dials. AI recognizing that ‘that is a clock’ is less complicated than really studying it.”

As well as reading the time, dates were just as much of a problem. If the AI was asked “What day will the 153rd day of the yr be?” the failure rate was similarly high. The results compare as follows: AI systems read clocks correctly only 38.7% and calendars only 26.3%.

“Arithmetic is trivial for conventional computer systems however not for big language models. AI does not run math algorithms, it predicts the outputs based mostly on patterns it sees in coaching information,” Saxena mentioned.

“So while it may answer arithmetic questions correctly some of the time, its reasoning isn’t consistent or rule-based, and our work highlights that gap.”

This research project is the latest in a growing body of research that highlights the differences of understanding between the ways AI and humans.

AI Models get their answers from familiar patterns and work best when there are enough examples in their training data, but when they need to use abstract reasoning, they often fail.

“What for us is a very simple task like reading a clock may be very hard for them, and vice versa,” Saxena said.

“AI is powerful, but when tasks mix perception with precise reasoning, we still need rigorous testing, fallback logic, and in many cases, a human in the loop.”

There is clearly a lot more analysis needed to actually unlock the potential of Artificial Intelligence.