A computer simulation of water can easily convince the human eye it's water, both in terms of pixel perfect representation and simulated behaviour in simulated environments. Until they try to put it in a bottle and drink it.
Turns out that physics of what it actually is matters more than human observation that some of the pretty output patterns look identical or superior to the real thing.
(And aside from being physically very dissimilar, stuff like even attempting to model human sex drive is entirely superfluous to an LLM's ability to mimic human sexy talk, so we can safely assume that it isn't actually horny just because it's successfully catfishing us!)
I've interacted with many people online, only through text, and my life has significantly changed because of many of those interactions. The effect on my life would have been the same whether the entities typing were made out of silicon or carbon.
Sure, and I've been deeply affected by books, but I'm not going to start using that as a basis for an argument a book and a human think in exactly the same way
This was in response to your comment about how you can tell that a water simulation is fake by trying to dip a water bottle in there. The distinction between chemical and silicon doesn't matter when the output is text. There's no physical test you can perform in the text, like dipping a water bottle in water, to see if it's chemical or silicon.
If you test both on the same terms (i.e. only interaction via a remote terminal) then a decent simulation can entirely convince humans that a bottle has been dipped in it and water removed from it too. But it still doesn't have the essential properties of H20, it just looks like it in many ways to some imperfect observers.
Testing is a moot point when my original argument was that it there is no reason to assume that a converts-to-ASCII subset of i/o as it is perceived by a [remote] human observer other is the only differences between two dissimilar physical processes (one of which we know results in sensory experiences, self awareness etc). Takes a lot more belief that the human mind is special to believe that sensory experience etc resides not in physics but whether human observation deduces the entity has sensory experience.
Turns out that physics of what it actually is matters more than human observation that some of the pretty output patterns look identical or superior to the real thing.
(And aside from being physically very dissimilar, stuff like even attempting to model human sex drive is entirely superfluous to an LLM's ability to mimic human sexy talk, so we can safely assume that it isn't actually horny just because it's successfully catfishing us!)