One needn’t go as far as souls anyway. Jefferson’s hypothesis—that there is some electrochemical basis to thought—is sufficient to solve the problem. Were it true, the reason computers seem fundamentally blocked from progress on the Turing Test would amount to the fact that they are wholly mechanical objects, while “thought” is as much a biological function as “digestion” or “copulation.”

Even if true, why couldn't the electrochemical processes be simulated too? I don't think it's necessary to strictly and completely reproduce a biological brain to produce thought in a computer, but even if it is, it's "just" a matter of scale. If you can increase the fidelity of the simulation with effectively infinite computing power, what would it be missing? It'd have to be something that can't be predicted, can't even have its unpredictability described with an equation (I don't know what any coin flip will turn up as, but I do know how to write a program that produces indistinguishable results from a real coin for a simulation), so it's just changing all the time and follows no rules whatsoever, but also you can't just write a program that does its own "random crap that can't be predicted" simulation because the real one is somehow also so precise that it's the only thing that makes consciousness work and a mechanical one isn't good enough?

[–] zogwarg@awful.systems 10 points 2 days ago (12 children)

Even if true, why couldn’t the electrochemical processes be simulated too?

You're missing the argument, that even you can simulate the process of digestion perfectly, no actual digestion takes place in the real world.
Even if you simulate biological processes perfectly, no actual biology occurs.
The main argument from the author is that trying to divorce intelligence from biological imperatives can be very foolish, which is why they highlight that even a cat is smarter than an LLM.

But even if it is, it’s “just” a matter of scale.

Fundamentally what the author is saying, is that it's a difference in kind not a difference in quantity.
Nothing actually guarantees that the laws of physics are computable, and nothing guarantees that our best model actually fits reality (aside from being a very good approximation).
Even numerically solving the Hamiltonians from quantum mechanics, is extremely difficult in practice.

I do know how to write a program that produces indistinguishable results from a real coin for a simulation.

Even if you (or anyone) can't design a statistical test that can detect the difference of a sequence of heads or tails, doesn't mean one doesn't exist.
Importantly you are also only restricting yourself to the heads or tails sequence, ignoring the coin moving the air, pulling on the planet, and plopping back down in a hand. I challenge you to actually write a program that can achieve these things.
Also decent random-number generation is not actually properly speaking Turing complete [Unless again you simulate physics but then again, you have to properly choose random starting conditions even if you assume you have a capable simulator] , modern computers use stuff like component temperature/execution time/user interaction to add "entropy" to random number generation, not direct computation.

As a summary,

When reducing any problem for a "simpler" one, you have to be careful what you ignore.
The simulation argument is a bit irrelevant, but as a small aside not guaranteed to be possible in principle, and certainly untractable with current physics model/technology.
Human intelligence has a lot of externalities and cannot be reduced to pure "functional objects".
- If it's just about input/output you could be fooled by a tape recorder, and a simple filing system, but I think you'll agree those aren't intelligent. The output as meaning to you, but it doesn't have meaning for the tape-recorder.

[–] chaos@beehaw.org -2 points 2 days ago (11 children)

(I'm going to say "you" in this response even though you're stating some of these as arguments from the author and not yourself, so feel free to take this as a response to the author and not you personally if you're playing devil's advocate and don't actually think some of these things.)

You're missing the argument, that even you can simulate the process of digestion perfectly, no actual digestion takes place in the real world.

But it does take place in the real world. Where do you think the computers are going to be? Computers can and do exist in and interact with the real world, they always have, so that box is already checked. You can imagine the computations as happening in a sort of mathematical void outside of the universe, but that's mostly only useful for reasoning about a system. After you do all that, you move electrons around in a box and see the effects with your own human senses.

The main argument from the author is that trying to divorce intelligence from biological imperatives can be very foolish, which is why they highlight that even a cat is smarter than an LLM.

Well, yeah, current LLMs are tiny and stupid. Something bigger, and probably not an LLM at all, might not be.

Nothing actually guarantees that the laws of physics are computable, and nothing guarantees that our best model actually fits reality (aside from being a very good approximation). Even numerically solving the Hamiltonians from quantum mechanics, is extremely difficult in practice.

It doesn't have to actually fit reality perfectly, and it doesn't have to be able to predict reality like a grand unified theory would. It just needs to behave similarly enough to produce the same effects that brains do. It hasn't been shown to be possible, but there's also no reason to think we can never get close enough to reproduce it.

Even if you (or anyone) can't design a statistical test that can detect the difference of a sequence of heads or tails, doesn't mean one doesn't exist.

Yes it does. If they're indistinguishable, there is no difference.

Importantly you are also only restricting yourself to the heads or tails sequence, ignoring the coin moving the air, pulling on the planet, and plopping back down in a hand. I challenge you to actually write a program that can achieve these things.

I don't have any experience writing physics simulators myself, but they definitely exist. Even as a toy example, the iOS app Dice by PCalc does its die rolls by simulating a tossed die in 3D space instead of a random number generator. (Naturally, the parameters of the throw are generated, the simulation is just for fun, but again, it's a distinction without a difference. If the results have the same properties, the mechanism doesn't matter.) If I give you a billion random numbers, do you think you could tell if I used the app or a real die? Even if you could, would using one versus the other be the difference between a physics simulation being accurate or inaccurate enough to produce consciousness?

certainly untractable with current physics model/technology.

Of course. This is addressing an argument made by the post that computers are inherently incapable of intelligence or consciousness, even assuming sufficient computation power, storage space, and knowledge of physics and neurology. And I don't even think that you need to simulate a brain to produce mechanical consciousness, I think there would be other, more efficient means well before we get to that point, but sufficiently detailed simulation is something we have no reason to think is impossible.

Human intelligence has a lot of externalities and cannot be reduced to pure "functional objects".

Why not? And even if so, what's stopping you from bringing in the externalities as well?

If it's just about input/output you could be fooled by a tape recorder, and a simple filing system, but I think you'll agree those aren't intelligent.

What are the rules of the filing system? If they're complex enough, and executed sufficiently quickly that I can converse with it in my lifetime, let me be the judge of whether I think it's intelligent.

[–] corbin@awful.systems 6 points 1 day ago (1 children)

I don’t have any experience writing physics simulators myself…

I think that this is your best path forward. Go simulate some rigid-body physics. Simulate genetics with genetic algorithms. Simulate chemistry with Petri nets. Simulate quantum computing. Simulate randomness with random-number generators. You'll learn a lot about the limitations that arise at each step as we idealize the real world into equations that are simple enough to compute. Fundamentally, you're proposing that Boltzmann brains are plausible, and the standard physics retort (quoting Carroll 2017, Why Boltzmann brains are bad) is that they "are cognitively unstable: they cannot simultaneously be true and justifiably believed."

A lesser path would be to keep going with consciousness and neuroscience. In that case, go read Hofstadter 2007, 'I' is a strange loop to understand what it could possibly mean for a pattern to be substrate-independent.

If they’re complex enough, and executed sufficiently quickly that I can converse with it in my lifetime, let me be the judge of whether I think it’s intelligent.

No, you're likely to suffer the ELIZA Effect. Previously, on Awful, I've explained what's going on in terms of memes. If you want to read a sci-fi story instead, I'd recommend Watts' Blindsight. You are overrating the phenomenon of intelligence.

[–] chaos@beehaw.org 2 points 1 day ago

I'm clearly failing to communicate my thoughts, and doing it in the wrong forum, but I appreciate the links, I'm excited to learn new things from them.

load more comments (9 replies)

load more comments (18 replies)