large language models

References

Summary

Probabilistic

LLMs produce probability distributions when given input

There's also a hardware aspect to this

AI currently has no verification step or confidence interval

The Chess Problem

[2026-04-13 Mon 23:00]

I just went through one of my games with Gemini.

It was unable to present the best tactic at the end of the game, in which White had to sacrifice a bishop on f6, to remove the pawn from g7, and then play Qg6+ on the king.

This isn't a difficult tactic (to be fair, I missed it, but I was playing in a bullet game; to a computer that's like a classical amount of time), and stockfish (and probably weaker engines) could find it immediately, just like any grandmaster (and any chess player around my level could easily find it if we knew we were looking for a tactic)

And yet the AI couldn't, even when I told it there were better moves on the board than what it suggested

This brings up an interesting question:

Deep Blue

Kasparov lost to Deep Blue in 1997

Neural networks took a long long time to even get competent at chess

It's not clear to me that NN models aren't just a waste of time, in comparison to searching for an explicit algorithm

Mixture of Experts

When the human brain works on a problem, only certain "parts" of your brain get involved

In a typical NN, all of the nodes are involved in everything