Source (Bluesky)
Transcript
Here’s an example that Google’s Josh Woodward, VP of the Gemini app, Google Labs, and AI Studio, shared in a blog post about how Personal Intelligence can work. Google also put together a similar example in a video that I’ve embedded below:
For example, we needed new tires for our 2019 Honda minivan two weeks ago. Standing in line at the shop, I realized I didn’t know the tire size. I asked Gemini. These days any chatbot can find these tire specs, but Gemini went further. It suggested different options: one for daily driving and another for all-weather conditions, referencing our family road trips to Oklahoma found in Google Photos. It then neatly pulled ratings and prices for each. As I got to the counter, I needed our license plate. Instead of searching for it or losing my spot in line to walk back to the parking lot, I asked Gemini. It pulled the seven-digit number from a picture in Photos and also helped me identify the van’s specific trim by searching Gmail. Just like that, we were set.
I ask a question, they give me the wrong answer. I tell them they gave me the wrong answer. They apologize, and then repeat the same mistake in the next answer. Or they give me a different wrong answer. I eventually give up and solve it with a web search.
I don't know if my questions are about really obscure stuff or what, but it's really annoying. Like, I know that they're only predicting tokens, but how hard is it to program them to go "Okay, we've already established that this pattern of tokens is wrong, so I'm not going to include it in the next answer".
They have no sense of "truth", it's a complex graph and weights that predict the most likely next token. You can change the output by doing training or adjusting the context by changing the prompt (also temperature that affects the randomness).
The training data affects what it will predict, but if the training data includes a debate, then both sides get encoded into the weights and the context is what determines what "side" of the debate your response gets. It can't determine the truth; the truth doesn't even factor in to what its output is (even if it "talks" about the truth in that output).