I think you’re leaning too doomer on AGI. The particulars of FOOM is not just whether the AI is connected to the internet, but whether it can spread onto other compute substrate. While most AI runs on GPUs, it requires extremely high memory bandwidth and precision networking. It’s not clear that a genius AI would be able to increase its intelligence by gobbling up more compute. Additionally, as of right now, the compute necessary for superintelligence would require improvements all the way down the stack of semi conductors and increased production, as well as an order of magnitude more electric power. There are dozens of limiting factors that depend on processes whose speed is dampened by reality (R&D, manufacturing, mining, etc.).
I'm sceptical. AI is only allowed to learn passively, not by interactions with humans, since we might turn it into a violent bigot. It's like shutting an infant in a room with a TV from birth and never having any humans talk to it, with eventually the TV showing more and more other children simply watching TV.
This severely limits it.
And even without that, well spreadsheets were meant to make book-keepers obsolete, since complex tasks were made simple. Instead, we took the opportunity to make the complex tasks more complex (taxation and corporate law) and this complexity of course gave more opportunity for mistakes and fraud - so while book-keeper jobs were lost, more accounting, auditing, management etc jobs were created.
And already most office workers report actually working for fewer than 3 hours a day. Humans are good at keeping themselves "busy".
If AI is as revolutionary as everyone thinks, then we'll see some crazy productivity increases too. Annual 5% real growth means a doubling in total income every 14 years. A quadrupling every generation. Owning capital would be massive. Then it's up to liberal democracies to share the wealth fairly. I personally favor the idea of the US government giving every American newborn $1000 worth of the SPX at birth (can't sell before 50 except maybe taking out 20% for college or to buy a house). And anyone can add X amount to it each year. Maybe make it more than $1000K/baby.
A Star Trek future of no scarcity seems fine so long as AI doesn't kill us.
I look forward to it! Not irrelevant, here is what I call Ximm's Law:
Ximm's Law: every critique of AI assumes to some degree that contemporary implementations will not, or cannot, be improved upon.
Lemma: any statement about AI which uses the word "never" to preclude some feature from future realization is false.
I find that many attempts to prognosticate about even the near future are faltering or showing widening error, because just as you enumerate the uptake of AI accelerates even as the technology continues to make nonlinear improvements in numerous metrics...
...things which we are largely incapable of intuiting or reasoning about.
In the case of the application of tools to consolidate and render unassailable the grotesque advantageous position they are already in, our nascent klept and their dynasties may fail individually; but the perverse system they have architected with little interruption since c. 1970 shows no signs of weakness.
There's a distribution of prospective outcomes but I don't see much chance of a utopian one.
But I am a curmudgeon brutalized psychically by our culture's compounding failures. I look forward to cause for optimism.
Re: UBI, my own protest sign slash mantra is ¡No AGI Without UBI!
I was telling my partner this weekend, the $1M question our civilization now faces appears to be:
assuming the permanent kleptocratic billionaire class uses first-mover advantage and already-amassed wealth to capture the gains (and tooling) of the immanent "fully automated economy,"
will they grudgingly, in service of social stability that lets them rule in peace, relinquish 2% of the total output of that economy as UBI for the bottom 99% of us?
Or will they make our lives something very close to a literal dystopian nightmare of slave-class indebtedness, using the twin superpowers of AI-backed total surveillance and pervasive sentiment steering, to keep them more like 0.2%?
I wish I was kidding. I'm not remotely kidding. I don't see how we get any other outcome... absent a "gray swan" disruptor such as e.g. runaway climatic change or a Carrington event.
Nice update! What are your thoughts on something like UBI? Have you ever pondered on what a post ASI economy looks like?
I think you’re leaning too doomer on AGI. The particulars of FOOM is not just whether the AI is connected to the internet, but whether it can spread onto other compute substrate. While most AI runs on GPUs, it requires extremely high memory bandwidth and precision networking. It’s not clear that a genius AI would be able to increase its intelligence by gobbling up more compute. Additionally, as of right now, the compute necessary for superintelligence would require improvements all the way down the stack of semi conductors and increased production, as well as an order of magnitude more electric power. There are dozens of limiting factors that depend on processes whose speed is dampened by reality (R&D, manufacturing, mining, etc.).
Damn, AI democratized being a hot girl
Crémieux says it’s hard to directly compare AI and human’s IQ : https://x.com/cremieuxrecueil/status/1766649068862730528 . But I agree with the general direction of the article.
I am reminded of the Arthur C. Clarke quote that “It has yet to be proved that intelligence has real survival value”.
Personally, I find the notion a super intelligent AI would present an existential threat to itself and humanity absolutely frightening.
Thank you Tomás, a great summary as always.
Just a minor update I found in An Qu's experience with Claude Opus: apparently it did have prior knowledge of Circassian... https://twitter.com/hahahahohohe/status/1765435151817830834
I'm sceptical. AI is only allowed to learn passively, not by interactions with humans, since we might turn it into a violent bigot. It's like shutting an infant in a room with a TV from birth and never having any humans talk to it, with eventually the TV showing more and more other children simply watching TV.
This severely limits it.
And even without that, well spreadsheets were meant to make book-keepers obsolete, since complex tasks were made simple. Instead, we took the opportunity to make the complex tasks more complex (taxation and corporate law) and this complexity of course gave more opportunity for mistakes and fraud - so while book-keeper jobs were lost, more accounting, auditing, management etc jobs were created.
And already most office workers report actually working for fewer than 3 hours a day. Humans are good at keeping themselves "busy".
https://venturebeat.com/ai/the-ai-feedback-loop-researchers-warn-of-model-collapse-as-ai-trains-on-ai-generated-content/
If AI is as revolutionary as everyone thinks, then we'll see some crazy productivity increases too. Annual 5% real growth means a doubling in total income every 14 years. A quadrupling every generation. Owning capital would be massive. Then it's up to liberal democracies to share the wealth fairly. I personally favor the idea of the US government giving every American newborn $1000 worth of the SPX at birth (can't sell before 50 except maybe taking out 20% for college or to buy a house). And anyone can add X amount to it each year. Maybe make it more than $1000K/baby.
A Star Trek future of no scarcity seems fine so long as AI doesn't kill us.
I look forward to it! Not irrelevant, here is what I call Ximm's Law:
Ximm's Law: every critique of AI assumes to some degree that contemporary implementations will not, or cannot, be improved upon.
Lemma: any statement about AI which uses the word "never" to preclude some feature from future realization is false.
I find that many attempts to prognosticate about even the near future are faltering or showing widening error, because just as you enumerate the uptake of AI accelerates even as the technology continues to make nonlinear improvements in numerous metrics...
...things which we are largely incapable of intuiting or reasoning about.
In the case of the application of tools to consolidate and render unassailable the grotesque advantageous position they are already in, our nascent klept and their dynasties may fail individually; but the perverse system they have architected with little interruption since c. 1970 shows no signs of weakness.
There's a distribution of prospective outcomes but I don't see much chance of a utopian one.
But I am a curmudgeon brutalized psychically by our culture's compounding failures. I look forward to cause for optimism.
Re: UBI, my own protest sign slash mantra is ¡No AGI Without UBI!
I was telling my partner this weekend, the $1M question our civilization now faces appears to be:
assuming the permanent kleptocratic billionaire class uses first-mover advantage and already-amassed wealth to capture the gains (and tooling) of the immanent "fully automated economy,"
will they grudgingly, in service of social stability that lets them rule in peace, relinquish 2% of the total output of that economy as UBI for the bottom 99% of us?
Or will they make our lives something very close to a literal dystopian nightmare of slave-class indebtedness, using the twin superpowers of AI-backed total surveillance and pervasive sentiment steering, to keep them more like 0.2%?
I wish I was kidding. I'm not remotely kidding. I don't see how we get any other outcome... absent a "gray swan" disruptor such as e.g. runaway climatic change or a Carrington event.