As a human, I found your prompt confusing enough that I had to read it over a couple of times. I associate the letters 'A', 'B', and 'C' with alphabetical order strongly enough that swapping the order is a bit difficult. I can do it of course, but if I was tired and/or not paying attention... well, it's a mistake a human could make.
I also find it notable that when you asked for more detail, the AI was correct up until the last sentence.
It's not hard to get it to trip up with letters that correspond to the order:
Me: Imagine a set of 3 events (X, Y and Z) ordered with unique timestamps each. Event X is the first of all events to happen. Event Y is in between the other two. What is a valid bound for Event Z's timestamp?
ChatGPT: In this scenario, Event X must have the earliest timestamp, and Event Y must have a timestamp that falls between the timestamps of Events X and Z. Therefore, the valid bounds for the timestamp of Event Z would be the timestamp of Event X, and any timestamp that is greater than the timestamp of Event Y. This means that the valid range for the timestamp of Event Z would be the range from the timestamp of Event X to infinity.
Prompt: Imagine a set of 3 events (A, red and Q) ordered with unique timestamps each. Event A is the first of all events to happen. Event Q is in between the other two. What is a valid bound for Event red's timestamp?
ChatGPT: Since we are given that Event A happened first and that Event Q occurred in between Events A and red, a valid bound for the timestamp of Event red would be that it must have occurred after Event A and before Event Q. So, the valid bound for the timestamp of Event red would be the time range between the timestamp of Event A and the timestamp of Event Q.
---
I don't think event names matter at all here, it just trips up on some logical constructs.
---
EDIT: here's some reasoning!
Me: So the timestamp for Event Red must be after both the timestamp of Event A and the timestamp of Event Q. How can it be between the timestamps of Events A and Q?
ChatGPT: The timestamp for Event Red must be after the timestamp of Event A and after the timestamp of Event Q, but it does not necessarily have to be after both of them at the same time. The timestamp for Event Red can be between the timestamps of Events A and Q, as long as it is after the timestamp of Event A and before the timestamp of Event Q. This is because the timestamps for Events A and Q represent the time at which those events happened, and the timestamp for Event Red must be after the time at which Event A happened and before the time at which Event Q happened.
> I also find it notable that when you asked for more detail, the AI was correct up until the last sentence.
I actually think that makes the AI look worse. If it did the human mistake of alphabetically ordering things then it would be fine. But it shows that it clearly understood the prompt by repeating the ordering of events correctly but fails to answer the implication of that. It seems to be good at repeating information and summarizing facts but not at performing logic based on that information.
I had similar experience where it told me that a given prime number is not a prime number as it has two divisors, 1 and itself. Literally having given the reason for why it should be a prime and failing to get to the correct conclusion. And it was asked in plain language.
The question contains contradictory information. I suppose a smarter AI might have worked out the intended meaning, or asked for clarification. But how many humans would?
I believe the word “ordered” in the question caused the confusion for some of us humans and for the machine. It anticipates some information, that the events are ordered by their time stamps, but doesn’t yet provide the order.
I also find it notable that when you asked for more detail, the AI was correct up until the last sentence.