Francois Chollet, the creator of Keras framework, is holding a AGI (Automatic General intelligence) competition, and the prize money is 1,000,000 USD for the first to answer 85% of the questions correctly. So far the best AI result is 43%.
The test set is basically a series of graphical IQ tests. You are given 4 example questions and answers, and you have to extrapolate the rule, and determine what the answer for the 5th question is going to be.
Francois Chollet thinks that LLM is not going to bring AGI because it ultimately is a memorization based AI, and AGI should involve more reasoning and abstraction.
You can git clone his dataset repo
and open the testing_interface.html with a browser and try some of the questions yourself.
Some are easy, but some tests I couldn’t figure out the logic behind them.
This one for example.
I have one theory that works, but it seems too convoluted.