Update: After the embargo update (Oct 25): Now that I have some answers from the people involved let me make a quick update: 1) I still find the paper unconvincing, specifically, the few verifiable experiments (namely experiments that can be verified classically) cannot serve as a basis for the larger experiments that cannot be classically verified. 2) Many of my attempts to poke hole in the experiments and methodology are also incorrect. 3) In particular, my suggestion regarding the calibration That I express with much confidence at the beginning of the post goes against the basic strategy of the researchers (as they now clarified) and the detailed description in the supplement. 4) I will come back to the matter in a few weeks. Meanwhile, I hope that some additional information will become available. Over the comment section you can find a proposal for a “blind” test that both Peter Shor and I agree to. (If and when needed.) It would respond to concerns about “cross-talk” between the classical computations and quantum computation during the experiments, either via the calibration method or by other means.
Original post (edited):
Recall that a quantum supremacy demonstration would be an experiment where a quantum computer can compute in 100 seconds something that requires a classical computer 10 hours. (Say.)
In the original version I claimed that: “The Google experiment actually showed that a quantum computer running for 100 seconds PLUS a classic computer that runs 1000 hours can compute something that requires a classic computer 10 hours. (So, of course, this has no computational value, the Google experiment is a sort of a stone soup.)” and that: “The crucial mistake in the supremacy claims is that the researchers’ illusion of a calibration method toward a better quality of the quantum computer was in reality a tuning of the device toward a specific statistical goal for a specific circuit.” However, it turned out that this critique of the calibration method is unfounded. (I quote it since we discussed it in the comment section.) I remarked that “the mathematics of this (alleged) mistake seems rather interesting and I plan to come back to it (see at the end of the post for a brief tentative account), Google’s calibration method is an interesting piece of experimental physics, and expressed the hope that in spite of what appears (to me then) to be a major setback, Google will maintain and enhance its investments in quantum information research. Since we are still at a basic-science stage where we can expect the unexpected.”
Detection statistical flaws using statistics
Now, how can we statistically test such a flaw in a statistical experiment? This is also an interesting question and it reminded me of the following legend (See also here (source of pictures below), here, and here) about Poincaré and the baker, which is often told in the context of using statistics for detection. I first heard it from Maya Bar-Hillel in the late 90s. Since this story never really happened I tell it here a little differently. Famously, Poincaré did testify as an expert in a famous trial and his testimony was on matters related to statistics.
The story of Poincaré and his friend the baker
“My friend the baker,” said Poincaré, “I weighed every loaf of bread that I bought from you in the last year and the distribution is Gaussian with mean 950 grams. How can you claim that your average loaf is 1 kilogram?”
“You are so weird, dear Henri,” the baker replied, “but I will take what you say into consideration.”*
A year later the two pals meet again
“How are you doing dear Henri” asked the baker “are my bread loaves heavy enough for you?”
“Yes, for me they are,” answered Poincaré “but when I weighed all the loaves last year I discovered that your mean value is still 950 grams.”
“How is this possible?” asked the baker
“I weighed your loaves all year long and I discovered that the weights represent a Gaussian distribution with mean 950 grams truncated at 1 kilogram. You make the same bread loaves as before but you keep the heavier ones for me!”
“Ha ha ha” said the baker “touché!”** and the baker continued “I also have something that will surprise you, Henri. I think there is a gap in your proof that 3-manifolds with homology of a sphere is a sphere. So if you don’t tell the bread police I won’t tell the wrong mathematical proofs police :)” joked the baker.
The rest of the story is history, the baker continued to bake bread loaves with an average weight of 950 grams and Poincaré constructed his famous Dodecahedral sphere and formulated the Poincaré conjecture. The friendship of Poincaré and the baker continued for the rest of their lives.
(Added, Dec 27, 2019).
“What?” asked the superintendent
“I think that this was an innocent mistake in the calibration method of weighting the bread” said Henri, and continued: “I found a beautiful mathematical explanation for how this could happen.” Henri was happy that he could simultaneously help his friend the baker, and brag about his mathematical skills. He truly believed in his calibration theory.
“I am working at the bakery since the age of 12 and I am surprised how ignorant Henri is about what we do.” said the baker and continued “There was no such mistake in the calibration method. and no baker in the world can make such a mistake” .
“Oh Benoît, Benoît,… merde…” thought Henri (in French)
* “There are many bakeries in Paris” thought the baker “and every buyer can weight the quality, weight, cost, and convenience”.
** While the conversation was originally in French, here, the French word touché is used in its English meaning.
The mathematics of such a hypothetical calibration in a few sentences.
The ideal distribution for a (freezed) random circuit can be seen exponentially distributed probabilities depending on .
The first order effect of the noise is to replace by a convex combination with a uniform distribution . (For low fidelity is rather small.)
The second order effect of the noise is adding a Gaussian fluctuation described by Gaussian-distributed probabilities . Like these probabilities also depend on the circuit .
For low fidelity, as in our case, the calibration mainly works in the range where is dominant and the calibration (slightly) “cancels” this Gaussian fluctuation. This does not calibrate the quantum computer but rather tweaks it toward the specific Gaussian contribution that depends on the circuit .
Technical update (Nov 18): Actually, some calculation shows that even with a hypothetical calibration toward noisy , the contribution to the statistical test from instances that represent the Gaussian part of the noisy distribution is rather small. So (under the hypothetical no-longer-relevant calibration scenario that I raised) Peter Shor’s interpretation of a computationally-heavy proof-of-concept of some value (rather than a valueless stone soup), is reasonable. One more remark (Dec 1): The second order effect of noise mentioned here (and also in a toy model in a later post) may well play a role in understanding the Google supremacy demonstration.
Trivia question: what is the famous story where a mathematician is described as objecting to children dreaming.
Answer to trivia question (Nov 15, 2019).
The famous story is the Happy Prince by OSCAR WILDE
“He looks just like an angel,” said the Charity Children as they came out of the cathedral in their bright scarlet cloaks and their clean white pinafores.
“How do you know?” said the Mathematical Master, “you have never seen one.”
“Ah! but we have, in our dreams,” answered the children; and the Mathematical Master frowned and looked very severe, for he did not approve of children dreaming.
A famous sentence from the story is:
“Swallow, Swallow, little Swallow,” said the Prince, “will you not stay with me for one night, and be my messenger?