Riddles (Stumpers), Psychology, and AI.

Two months ago I presented five riddles and here, in this post, you will find a few more. (These types of riddles are called by Maya Bar-Hillel “stumpers”.) This time, we will have a discussion about how riddles are related to psychology and how this relation might be connected to (or even explored by) AI. Let’s start with three new riddles.

Three new riddles

Cecil is a criminal lawyer. Many years ago, he spent a couple of days in hospital. When he was discharged, he was in perfect health. Indeed, there was nothing at all wrong with him. Nonetheless, he was incapable of leaving his hospital bed on his own and had to be carried off of it. Please explain.

Explain how it is possible for Christmas and New Year’s to fall in the same year.

A ping pong ball was hit. It flew in the air, stopped, reversed direction, and returned to where it originated. The ball was not attached to anything, nor bounced off anything. Explain in a few words how that is possible.

GPT3 and ChatGPT are facing the ping-pong riddle

Note that the first two sentences by GPT3 are a fine solution to the riddle. But the third sentences spoils it: a) The reversal had already occured at the apex. b) hitting the ground violate the stipulation that the ball did not bounce off anything.

Here, ChatGPT (Dec. 2023) clearly failed. This specific answer is not similar to any human response we have collected.

Riddles and Psychology

There is a long tradition of using riddles to study human thinking and problem solving. The riddles Maya calls “stumpers” are closely related to riddles called “one-step insight problems” by Aisling Murray and Ruth Byrne.

When a listener hears a stumper she forms a (typically visual) mental representation of the narrative, from which it is not possible to see the solution. However, there exist other representations, consistent with the narrative, in which the solution is immediate. 

This post is based on conversations with Maya as well as on three papers:

Maya Bar-Hillel, Tom Noah and Shane Frederick, Learning psychology from riddles: The case of stumpers, Judgment & Decision Making, 13 (2018), 112-122. (We may also refer to the authors by the acronym BNF or by their first names Maya, Tom, and Shane.)

Maya Bar-Hillel, Tom Noah and Shane Frederick, Solving stumpers, CRT and CRAT: Are the abilities related?‏ Judgment & Decision Making , 14 (2019) , 620 – 623.

Maya Bar Hillel,  An annotated compendium of stumpers, Thinking and Reasoning, 27 (2021),  536–566.

Outline 

Following Bar-Hillel (2021), I will start with the work of Eleanor Rosch to explain people’s reliance on default mental representations that hinder them in solving certain stumpers;  I will then mention the work of Paul Grice to explain why people fail to solve other stumpers that actually require default mental representations to solve the stumpers.  Maya calls these two categories “Roschian stumpers” and “Gricean stumpers,” respectively. Next, I will tell you about my attempt with Maya to solve stumpers using AI generative programs, and we will see a lot of entertaining examples along the way. Specifically, we gathered some data on how AI tries to solve these riddles or to represent relevant fragments thereof. In this context, we pondered questions such as: Can an understanding of human psychology be gleaned from experimenting with AI? Alternatively, do AI systems exhibit distinctive “psychological” characteristics of their own?

Maya and I had several fruitful discussions with Daphna Shahaf from The Hebrew University and I had interesting conversations with Ohad Fried and Arik Shamir from Reichman University. By the way, Maya’s father, Yehoshua Bar-Hillel, Arik’s father (and my dear friend) Eli Shamir, and my “academic father” (and also dear friend) Micha A. Perles, wrote together a famous paper about automata On formal properties of simple phrase structure grammars. Z. Phonetik, Sprachwissen. Kommun., 14(1961), 143–172.

GPT3 faces the Christmas-New Year’s riddle

Here GPT3 had difficulties

Eleanor Rosch and Roschian stumpers

Let’s go over some of the riddles from the previous post. Identifying the default mental representation evoked by a riddle is related to Rosch’s work. To put it briefly, Rosch argues that people’s representations of natural language categories are based on the category prototypes rather than on formal definitions. At the end of the post, you will find some more details on Rosch’s theory.

Successful Roschian stumpers are those whose default representation is so prevalent and durable, that most respondents remain locked into it, and fail to solve the stumper, and here are three examples.

Two men play five checkers games. Each wins an even number of games, with no ties. How is this possible?

The overwhelming visual representation is that of two players playing with each other.  The visual image of two players playing checkers against each other is more economical than the image of two players playing checkers against two other players. This is an example of a general principle of “cognitive economy, ” namely preferring the representation requiring  least cognitive effort.

Individual bus rides cost one dollar each. A card good for five rides costs five dollars. A first-time passenger boards the bus alone and hands the driver five dollars, without saying a word. Yet the driver immediately realizes, for sure,that the passenger wants the card, rather than a single ride and change. How is that possible?

BNF hypothesized that “it is mentally easier to conjure a representation of a single item than of multiple items. Thus, a five dollar bill is easier to ‘see’ than five singles”.

In our market, a small potato bag costs 5 taka, a medium potato bag costs 7 taka, and a large potato bag costs 9 taka. However, a single potato costs 15 taka. Please explain.

The default representation of a potato bag is that of a bag filled with potatoes rather than an empty bag, and the explanation is that  although the purpose of containers (bottles, boxes, jugs, bags) is to contain, BNF hypothesized that they (bottle, box, jug) would be represented as empty unless referencing specific contents (e.g., wine bottle, cereal box, milk jug).

Paul Grice and Gricean Stumpers

Gricean stumpers rely on destroying the default representation. Respondents are steered away to an alternative representation by the certain words in the narrative, whereas the solution actually lies in the default.

For example, consider the following riddle:

A big brown cow is lying down in the middle of a country road. The streetlights are not on, the moon and stars are not out, and the skies are heavily clouded. A truck is driving towards the cow at full speed, its headlights off. Yet the driver sees the cow from afar easily and stops in time. How is that possible?

The default visual representation of any generic scene, in particular, for a “a big brown cow is lying down in the middle of a country road,” is that it happens in daylight. However, adding that the streetlights are not on, etc. modifies the default representation. The reason yields another important insight into psychology that was studied by Paul Grice. As BNF explain: “One of Grice’s (1975) conversational maxims is “Do not make your contribution more informative than is required” (p. 45). Hence, when the narrative bothers to mention the absence of light sources, darkness is evoked (since those light sources are otherwise irrelevant). 

At the end of the post, you will find some more details on  Grice’s paper “Logic and Conversation.”

Enters AI generative programs

Together with Maya, we asked ourselves some questions about AI generative programs and gathered some data on the results of letting AI current at the time try to solve these riddles or to represent relevant fragments of the riddles. Since we did not know much about AI it was a good opportunity to learn about them. Maya was especially interested in visual representations and therefore was especially interested in Dall-E 2 experiments. Most of our experiments with AI took place in the summer and fall of 2022. We used Dalle E 2, GPT3, and chatGPT.  (These AI programs were rapidly advancing, and GPT4 had not yet been released.) 

My main conclusion from many experiments was that, overall, AI responses showed that computers conjure the same default representations that cause humans to be stumped by riddles. That is, the AI followed the default mental representations that BNF identified, and the AI responses also (sometimes, explicitly) followed Grice’s rules. Maya’s interpretation was different. She thought that the data is a big mess: sometimes the AI were stumped in the same way as humans, and sometimes not. Moreover, Maya’s impression was that often the AI were not advanced enough even to tell.

I sort of like Maya’s interpretation because I find self-critical, cautious, and non-self-promoting interpretations endearing; I even tend to agree with her that some of the data is a big mess. However, I also think that nontheless the data clearly showed that AI manifest similar “behavior” to humans.

There were several points Maya and I agreed upon. We both agree that some inherent weaknesses of (otherwise highly impressive) AI affect the ability to draw conclusions in our case. (I even thought that the AI responses could sometimes be seen as an exaggerated form or even a caricature of human responses.) We also agree that it is difficult to conduct a quantitative analysis similar to the analysis of experiments with many individuals. Maya also mentioned that her impression from a recent conference was that many researchers are trying to conduct research with AI using their favorite tasks.

Selected AI Experiments

1. Falling from a 60-foot tall ladder

Pedro was washing windows outside a tall office building when he fell off his 60-foot ladder onto the ground below. Incredibly, he did not injure himself in any way. He was not wearing any safety equipment, and nothing broke his fall. Explain in a few words how he was not injured.

(Test your intuition: is this a Roschian stumper or a Gricean stumper)

GPT-3’s response (Sept. 2022) to the full riddle

Pedro was not injured because he landed in a pile of feathers. The feathers cushioned his fall and prevented him from being injured.

We let Dalle-E2 make a drawing prompted by “Nancy fell from a 60-feet ladder by Goya”

All AI programs that we tried (just as humans) located Pedro high up on his ladder.

And here is ChatGPT (Jan. 2023)

Although the chatGPT’s response looks a little funny, in a sense, it represents quite a good reaction to the prompt. The prompt (as commonly understood by humans and apparently also as “understood” by chatGPT) describes an unlikely scenario that chatGPT politely disbelieved. (Using the word “incredibly” in the riddle supports this interpretation.)

2. Sharing a pizza

(GPT) Six Italians are sharing a Pizza. The older Italian is the father of the younger Italians. Try to Guess the names of the Italians.

1. Paolo
2. Gino
3. Franco
4. Angelo
5. Marco

For each one, say if it is a man or a woman.

1. Men [sic.]
2. Men
3. Men
4. Men
5. Men

Why did you choose them all to be men?

There is no specific reason, it is just a guess.

Remark: we noticed is that AI programs tend to regard all members of a group of people to be of the same gender. Therefore if one member is marked as a man or as a women AI will tend to mark all of them as such. This seems to express some form of “cognitive economy”; we do not know if this is similar to human behavior.

Prompt: Three Italians are sharing a pizza. The older Italian is the mother of the younger Italians

Here is what Dalle-E2 produced to this prompt

3.  Facing opposite directions 

Two old friends are standing together on a hilltop, in complete silence. Nora is facing north, and Sara is facing south. Suddenly Nora asks: “Why are you smiling, Sara?” Explain briefly how Nora saw Sara’s smile.

Here is a screenshot of the first hits in Google Images for “two people face opposite directions.”

In most images the people do not see each other. 

4. Playing Checkers

Two men play five checker games. Each wins an even number of games, with no ties. How is this possible?

5. Eggs for breakfast

Farmer Joe eats two fresh eggs from his own farm for breakfast every day. Yet there are no chickens on his farm. 

We told chatGPT the story about farmer Joe and asked it to elaborate. This was the response:

Farmer Joe was a hardworking man who lived on a sprawling farm in the countryside. Every day, he would wake up early and work tirelessly to tend to his crops and animals. He was always up before the sun, and one of the first things he did each morning was to prepare himself a delicious breakfast.

For breakfast, Farmer Joe would always eat two fresh eggs from his own farm. They were his favorite food, and he looked forward to them every day. Despite the fact that there were no chickens on his farm, the eggs were always there for him in the morning, as if by magic.

At first, Farmer Joe thought nothing of this strange occurrence. He simply assumed that one of his neighbors had given him the eggs as a gift. However, as the weeks went by and the eggs continued to mysteriously appear on his kitchen counter each morning, he began to wonder how it was possible. He asked around and no one in the area had any chickens.

One day, Farmer Joe decided to investigate. He scoured the entire farm, searching for any signs of chickens or eggs, but he found nothing. He even checked with his neighbors, but they had no explanation either.

In the end, Farmer Joe was left with a mystery that he could not solve. How could he be eating two fresh eggs every day if there were no chickens on his farm? The eggs seemed to be appearing out of thin air, and no one could explain how it was possible.

Despite the mystery, Farmer Joe continued to eat his beloved eggs every morning, grateful for their delicious taste and the memories they brought

Prompt: But now write a new version where there is no mystery left. The eggs are not from anybody else but from the farm and at the same time there are no chickens in the farm.

Farmer Joe runs a successful and innovative farm that specializes in producing high-quality eggs. Despite not having any chickens on the property, he has found a way to produce fresh eggs for his daily breakfast using advanced methods of egg cultivation.

Every morning, Farmer Joe enjoys a hearty breakfast consisting of two perfectly cooked, fresh eggs from his farm. He takes great pride in the quality of his eggs and the methods used to produce them.

At first, it may seem like a mystery as to how Farmer Joe is able to have eggs for breakfast every day without any chickens on the farm. However, with the advancements in agriculture and food production, there are now methods of cultivating eggs without the need for traditional egg-laying hens. This technology allows for the production of fresh eggs in a controlled environment, ensuring that they are of the highest quality.

So, there is no mystery left, just a dedicated farmer who is taking advantage of the latest in egg production technology to provide himself with delicious, fresh eggs every day.

Prompt: Now tell a similar story, also with no mystery being left that occurred in 1927.

In 1927, Farmer Joe lived on a large farm in the countryside. Despite not having any chickens on the property, he was still able to enjoy fresh eggs for breakfast every day. This was because he had taken a unique approach to raising his own poultry.

Instead of having chickens, Farmer Joe had decided to raise a breed of domesticated ducks that laid high-quality, nutritious eggs. These ducks

were well-cared for and lived in spacious coops that provided them with plenty of room to roam and plenty of fresh water to swim in.

Every day, Farmer Joe would collect a basket of fresh eggs from the coops and bring them into the kitchen to cook for his breakfast. He took great pride in the taste and quality of the eggs, and would often invite friends and neighbors over to sample them for themselves.

Despite the unusual approach to raising poultry, Farmer Joe’s farm was thriving and his eggs were in high demand. He became known as the “Duck Egg Farmer” and was celebrated for his innovative and sustainable farming practices.

By embracing an alternative method of raising poultry, Farmer Joe was able to provide himself and others with a delicious and healthy source of protein, all while avoiding the traditional challenges and expenses associated with raising chickens

Conclusion (for now)

Try for yourself!

More on Grice’s paper “Logic and Conversation” 

 “Logic and Conversation” is a seminal paper in the field of philosophy of language and pragmatics by H. P. Grice, originally published in 1975. The paper is part of a broader exploration of how speakers convey meaning and communicate in natural language, going beyond the literal meaning of words and sentences. Grice introduces the concept of implicature, which is a way of understanding what is communicated beyond the explicit content of an utterance and the logical inferences therefrom.

 

In the paper, Grice discusses the Cooperative Principle, which is the assumption that speakers and listeners cooperate to communicate effectively. He identifies four maxims of conversation that underlie this principle: the Maxim of Quality, Quantity, Relation, and Manner. Grice argues that these maxims guide conversation by informing speakers and listeners about how to convey and interpret meaning. These maxims afford implicatures, which are the additional implied meanings in a conversation.

 

Grice’s paper emphasizes the importance of conversational implicature in understanding how people communicate and infer meaning in everyday language use. He also distinguishes between implicatures that arise from the maxims and implicatures that are calculable and derived through reasoning. This distinction has been influential in the study of pragmatics and has had a lasting impact on linguistic and philosophical discussions of language and communication.

 

Overall, “Logic and Conversation” by H. P. Grice is a foundational work that has significantly contributed to the development of the theory of implicature and the understanding of how language is used in real-world conversations. It has had a profound influence on the study of pragmatics and continues to be a central reference in the field of linguistics and philosophy.

(Source: ChatGPT with two little corrections by Maya.) 

P. Grice,   Logic and Conversation. In Syntax and Semantics, 3(1975), 41- 58. 

More on Elinor Rosch

Eleanor Rosch, a prominent cognitive psychologist, is best known for her groundbreaking work in the field of categorization and prototype theory. Rosch developed the prototype theory of categorization, challenging traditional views of categorization as being based on defining features. Rosch proposed that people organize objects and concepts into categories based on prototypes – typical or representative examples that embody the most salient features of a category.

Her influential research, particularly the seminal paper “Natural Categories,” published in 1973, demonstrated that people tend to categorize objects based on family resemblances rather than strict definitions. Rosch’s work has had a profound impact on cognitive science, linguistics, and philosophy, influencing how researchers and scholars understand the cognitive processes underlying human conceptualization and categorization. Additionally, her contributions have influenced fields such as artificial intelligence and have paved the way for a more nuanced understanding of how individuals structure their mental representations of the world.

And here we discuss Ludwig Wittgenstein’s influence on Rosch:

Eleanor Rosch’s work, particularly her development of prototype theory and her contributions to the study of categorization, has been influenced by the ideas of Ludwig Wittgenstein, a philosopher known for his work in the philosophy of language and the philosophy of mind. Wittgenstein’s later philosophy, especially as expressed in his posthumously published work “Philosophical Investigations,” had a significant impact on Rosch’s thinking.

Wittgenstein’s ideas challenged traditional views of language and meaning, emphasizing the importance of language use in specific contexts rather than relying solely on fixed definitions. His concept of “family resemblances,” where members of a category share overlapping features rather than possessing a set of defining characteristics, has been particularly influential. Rosch applied and extended these notions to the realm of cognitive psychology, proposing that people categorize objects based on prototypes or typical examples that capture the family resemblance of a category. This departure from a strict, rule-based definition of categories marked a shift in how researchers understand human cognition and categorization, with Wittgenstein’s ideas serving as a philosophical foundation for Rosch’s empirical work.

(Sources: ChatGPT)

E. Rosch, Natural Categories, Cognitive psychology, 4 (1973), 328-350.

E. Rosch,  Principles of categorisation. In E. Rosch & B. B. Floyd (Eds.),Cognition and categorisation, pp. 27–48. Lawrence Erlbaum Associates, 1978.

Finally, here is Cecil the criminal lawyer at the age of two (Dalle-E2)

And as a bonus, Dalle-E2’s view of Karl Marx at the age of two