I think i’ve only once flat out told one it was wrong about a specific assertion I quoted and it immediately was able to find its way to what I knew to be the correct claim.
I just wonder what would happen if i was in fact mistaken and I told it confidently it was wrong without elaborating
What’s the difference between converting the characters and numerals into a response based off their syntax vs understanding them? How can you establish, as the person submitting the notes, whether whoever is in the Chinese room is the former or the latter?
Intentionality is the key difference. You can eventually tell a Chinese room’s nature by giving it new variables that it hasn’t encountered before. New problems lead to algorithmic breakdown.
That said, there’s deeper conversations you can have about what consciousness truly is, of course. My personal view is that it requires a level of complexity that we are still very far away from architecturally, and a level of scalability that we may not even be able to support ecologically. This thought experiment is mainly to show you what the inner workings of a computerized process can look like, and works to provide a demystified perspective.