While Recent press appearanceSam Altman, Openai’s general manager, said he has observed AI’s “IQ” rapidly improved over the past several years.
“Very roughly, it seems to me – this is not scientifically accurate, this is just a vibration or spiritual answer – every year we move one standard deviation from IQ,” Altman said.
Altman is not the first to use IQ, an assessment of a human intelligence, as a reference for AI progress. AI -Influents On social media IQ testing and aligned the results.
But many experts say that IQ is a bad measure of the capabilities of a model – and misguided.
“It can be very tempting to use the same measures we use for people to describe skills or progress, but this is how to compare apples with oranges,” Sandra Wachter, a researcher studying technology and regulation at Oxford, told Techcrunch.
In his comments at the press, Altman equated IQ with intelligence. However IQ tests are relative – not objective – measures of sure kinds of intelligence. Is some Consent that IQ is an acceptable test of logical and abstract reasoning. But it doesn’t measure Practical Intelligence – knowing how to operate things – and it’s best to get caught.
“IQ is a tool for measuring human skills – disputed no less – based on what scientists look like,” Wachter noted. “But you cannot use the same measure to describe AI capabilities. A car is faster than humans, and a submarine is better diving. But that does not mean that cars or submarines exceed human intelligence. You equate one aspect of action with human intelligence, which is much more complex. ”
To stand out at an IQ test whose origins Some historians Track back to eugenics, the widely discredited scientific theory that people can be improved by selective reproduction, a test -a taker must have Strong working memory and knowledge of Western cultural standards. This invites the opportunity for bias, of course, so why One psychologist called IQ tests “Ideologically corrupt mechanical models” of intelligence.
That a model could act well with an IQ test indicates more about the defects of the test than the performance of the model, according to OS Keyes, a doctoral candidate at the University of Washington studying ethical AI.
“[These] Tests are easy enough to play if you have a virtually endless amount of memory and patience, “Keyes said.” IQ tests are a very limited way to measure cognitive, sentence and intelligence, something we have known from before the invention of the digital computer itself . “
AI probably has an unfair advantage of IQ tests, too, considering that models have massive amounts of memory and internalized knowledge at their disposal. Often, models are trained on public online data, and the site is full of sample questions made by IQ test tests.
“Tests tend to repeat very similar patterns – a pretty unwise way to raise your IQ is to practice doing IQ tests, which is essential what each [model] did, “said Mike Cook, a researcher at King’s College London specialized in AI. or signal. ”
Ultimately, IQ tests-spoken as they were designed for people, added chef-targeted as a way to evaluate general problems to solve problems. They are suitable for technology that is approaching problems solving very differently than humans.
“Crow could possibly use a tool to get candy from a box, but that doesn’t mean it can be registered at Harvard,” Cook said. “When I solve a problem. In other words, human brains dispute with much more things when they solve a problem – any problem at all, IQ tests or otherwise – and they do it with much less help [than AI.]”
All this notes the need for better AI testsHeidy Khlaaf, chief AI scientist at the AI Now Institute, told Techcrunch.
“In the history of computing, we did not compare computer skills with that of people exactly because the nature of computing means systems have always been able to perform tasks already outside the human ability,” Khlaaf said. “This idea that we directly compare the action of human skills systems is a recent phenomenon that is highly disputed, and what surrounds the controversy of the increasingly expanding-and moving references created to evaluate AI systems.”
