AI-generated summary: The University of Florida team applied the recently proven three-dimensional Kakeya conjecture to large model training, developing the GeoLAN method to address AI’s “black box” problem. This approach leverages the concept of “sticky Kakeya sets” derived from the proof of the Kakeya conjecture to impose geometric constraints on the semantic space of large models, clarifying concepts from the training stage and making AI reasoning paths traceable. Experiments on models such as Llama-3-8B and Gemma-3-4B demonstrated significant performance improvements. Meanwhile, Fields Medalist Jacob Tsimerman announced on the day of his award that he is joining OpenAI to work on AI safety, highlighting the unprecedented convergence of mathematics and AI.

Author and source: Leiphone

The mathematics community and the AI community have strongly collaborated.

Pure mathematical achievements at the highest level have already been directly applied to large model training.

A Xiaohongshu creator named Z9 discovered while reviewing papers that the recently viral 3D Kakeya conjecture has now been incorporated into neural network training workflows by a team from the University of Florida, applying it across disciplines to address the "black box" problem in large models.

Their paper is titled "GeoLAN: Geometric Learning of Latent Explanatory Directions in Large Language Models." At its core, it employs the key concept of "sticky Kakeya sets," derived from the proof of the Kakeya conjecture, to establish a systematic framework for organizing the internal semantic space of large models—straightening out entangled concepts during training to make AI's reasoning pathways truly traceable.

Indeed, ahead of them, mathematical experts pioneered the way; behind them, AI researchers reaped the rewards.

The underlying issue with large models: representation collapse

Let’s start with the “black box” issue that everyone can relate to.

Current large models have high accuracy in solving problems, writing code, and performing analysis, but when you ask, “How did you arrive at this step?”, they either can’t explain it clearly or fabricate a false rationale to deceive you.

The root of this issue lies in a fundamental characteristic known as representation collapse.

You can imagine the semantic space of a large model as a vast, thousands-layer intelligent warehouse that theoretically can categorically store countless concepts—logic, emotion, facts, reasoning, and more—each occupying its own space.

However, during training, Transformers tend to be particularly lazy—they often cram all the semantic information into a small area near the entrance of the warehouse, leaving the remaining 90% of the space completely unused.

As a result, all concepts are squeezed into a narrow, high-dimensional conical region, which academia refers to as "anisotropy."

Being crowded together can lead to many problems.

For example, concept entanglement forces unrelated content into the same direction, causing a single neuron to respond to both “cat” and “Bible verses.”

Moreover, you can't even determine which part of the representation corresponds to which logical step—the model itself cannot reconstruct the actual reasoning path.

When similar but distinct concepts are lumped together, it’s easy to get confused; a slight change in phrasing can cause the AI to make things up.

The paper provides a precise summary: a handful of "rogue dimensions" dominate most of the information variance, wasting the model's effective capacity.

This issue is a flaw in nearly all major Transformer architectures, from GPT and Llama to Gemma.

In response to this situation, the industry’s previous approach has largely been “reactive”—waiting until model training is complete and then using tools like sparse autoencoders to forcibly untangle intertwined concepts.

However, the results obtained through this method only appear reasonable and may not reflect the model’s actual internal logic, resulting in a consistent loss of fidelity.

Now, GeoLAN has shifted its approach entirely: instead of waiting for disorder to arise and then cleaning up, it establishes order from the start. Throughout training, geometric constraints are applied to the representation space, ensuring each concept finds its proper place.

How is the Kakeya conjecture related to AI?

To understand GeoLAN's concept, you first need to know what the Kakeya conjecture is about.

In 1917, mathematician Kakeya Sōichi posed a seemingly simple problem that puzzled mathematicians for half a century: What is the smallest possible area that a 1-centimeter-long needle can sweep out as it rotates completely on a flat surface?

You might think that going around in a circle would necessarily cover some area, but mathematicians have found that it can approach zero.

How is it done?

If you let a needle rotate while sliding, it can trace out a bizarre shape with an area nearly zero. In other words, in a two-dimensional space, you can fit all possible geometric paths into an extremely small corner.

The challenge escalates further—what if, in three-dimensional space, this needle can rotate in countless directions—up, down, left, right, and all other spatial orientations? What is the minimum size this space must be to accommodate all possible orientations of the needle?

According to two-dimensional experience, in three-dimensional space, the absolute volume of this shape can still be compressed to nearly zero.

But this raises a new puzzle: although this shape is hollow and has no volume, has it also shrunk or degraded at the microscopic level? Mathematicians introduced the concept of "fractal dimension" to measure the density of its structure.

Finally, Wang Hong and her team discovered that even when you squeeze the volume of the shape to be as hollow as possible, the internal interwoven framework still maintains a solid "three-dimensional fullness" to accommodate every directional needle, with no degradation at the microscopic level. This is precisely what makes Wang Hong’s resolution of the century-old problem so remarkable.

It was during this rigorous proof process that the extremely important concept of a "sticky Kakeya set" emerged.

“Stickiness” is essentially a set of anti-crowding rules designed to prevent lines and tubes from clustering together.

If a set of tubes follows these rules, they will spread out neatly, and the space will remain exactly as large as it should be; but if they don’t follow the rules and all crowd into a small corner, the “size” of the space will shrink, appearing squashed.

By now, you should be able to connect the dots: the representation collapse in large models is essentially "the semantic tube failing to satisfy the viscosity condition, with everything squeezed together."

What GeoLAN does is directly adopt the mathematical conclusion of the Kakeya conjecture as an engineering standard: since it has been rigorously proven in mathematics that "viscosity prevents spatial collapse," we apply the same viscosity constraint to the semantic space of large models, ensuring their representations naturally avoid clustering together.

To incorporate this rule into the training process, the core is to create a quantifiable penalty function—a rule the model can compute and understand how to adjust.

Then, GeoLAN designed two penalty rules that can calculate accounts, transforming the geometric constraints in the Gauss conjecture into training objectives. One is called KT-CW, specifically targeting "semantic clustering"; the other is called KT-Attn, specifically addressing "attention laziness."

In simple terms, KT-CW randomly samples various directions in the semantic space during training. If it detects that too much semantic information is clustered in one direction, the model is penalized. This forces the model to evenly distribute knowledge across every corner of the high-dimensional space, fully utilizing previously underused dimensions and fundamentally solving the problem of "clustering."

Another set, KT-Attn, specifically targets the slack behavior of attention mechanisms. In the later stages of training, attention heads in large models often become severely homogenized, with many heads fixating repeatedly on the same word or position, leaving very few heads actually contributing meaningfully. This rule enforces that each attention head must focus on distinctly different semantic regions, ensuring comprehensive, multi-directional coverage and completely resolving the rank collapse phenomenon in attention heads.

The impact of this series of actions has been immediate.

On Llama-3-8B, GeoLAN transformed the originally compressed conical representation space into a smooth spherical one, significantly reducing clustering and greatly improving uniformity across all directions, while maintaining task accuracy.

On Gemma-3-4B, the MMLU accuracy improved from 0.59 to 0.60, an increase of approximately 0.75 percentage points. TruthfulQA semantic stability significantly improved.

On the larger Gemma-3-12B model, the bias rate metric also showed a statistically significant decrease.

Interestingly, the paper also identified a fascinating phenomenon called the "Goldilocks zone."

Thus, the strict geometric constraint of uniform distribution becomes a disaster for models with too few parameters; small models inherently rely on conceptual entanglement to achieve extreme semantic compression, and forcibly dispersing them only severely degrades their internal geometric structure.

For overly large models, the extensive pre-training process has already spontaneously established highly complex manifold structures; adding this set of rules would not only be incompatible but also hinder overall performance.

Models with medium-scale parameters, such as four to eighty billion, are optimally suited to fully leverage the benefits of uniform distribution.

Moreover, the paper derives a sophisticated theorem, the "Semantic Stickiness Theorem": when representations satisfy the "anti-crowding rule," distinct semantic concepts automatically decompose into approximately orthogonal subspaces, which the paper vividly refers to as "grains." Each subspace can be independently interpreted and adjusted, thereby making the long-standing black-box problem in the industry transparent.

An ACL paper has pulled the Kakeya conjecture straight out of the realm of pure mathematics and into the domain of AI engineering.

Fields Medalist announces joining OpenAI that day

Another notable point is that Fields Medalist Jacob Tsimerman announced on the day of his award that he will soon be joining OpenAI to work on AI safety.

This highlights the close connection between cutting-edge mathematics and AI.

Just one year ago, he co-authored the paper "A Taxonomy of Omnicidal Futures Involving Artificial Intelligence" with Berkeley AI safety researcher Andrew Critch, using the utmost rigor of a mathematician to construct a meticulous classification system for AI risk pathways.

He himself is a major beneficiary of AI; in 2025, he published five mathematics papers on arXiv, openly stating that AI doubled his research productivity.

Even more dramatically, three days before the award ceremony, someone used Anthropic’s latest Claude Fable 5 to overturn the 87-year-old Jacobi conjecture overnight, shaking the global mathematics community. That person was his student, Levent Alpöge.

Faced with AI continuously approaching and even surpassing the intelligence of top mathematicians, renowned mathematician Timothy openly remarked: “This is the first time in my life that I’ve seen a large language model effortlessly solve a world-famous century-old problem in a field entirely unfamiliar to me.”

The relationship between mathematics and AI has long surpassed the description of being inseparable—they are merging at an astonishing pace.

Looking back at AI’s performance in the mathematical arena over the past year—from DeepMind’s AlphaProof winning a silver medal at the International Mathematical Olympiad, to OpenAI’s reasoning model finally resolving the eighty-year-old Erdős unit distance conjecture, to GPT-5.6 solving the cycle double cover conjecture in just one hour, and Claude Fable 5 dismantling the Jacobian conjecture overnight—AI has set multiple century-defining records within just two months, with the pace of breakthroughs accelerating exponentially.

In the AMS interview, Jacob bluntly stated, “There’s a lot of self-deception in the math community—people see that AI is currently not as good as mathematicians and assume it will never be.” He demonstrated through his own practice that although the model’s proofs are “incomplete and not rigorous,” they “often provide a broadly correct approach that can take you eighty percent of the way.”

He even believes that within two years, AI will surpass humans entirely in the task of mathematical proof.

So the question arises: Will there still be human recipients of the Fields Medal in 2030, four years from now?

The Fields Medal has a strict age limit: recipients must be under 40. This means that if, within the next four years, AI systematically surpasses human mathematicians in original discovery, the selection committee will face an unprecedented dilemma: are you awarding the prize to the person who has done the best mathematics, or to the best human mathematician?

More intriguing is the mutual acceleration between mathematics and AI. Just three months after the three-dimensional Kakeya conjecture was proven, GeoLAN emerged—a pure mathematical breakthrough directly transformed into a tool for training large models. What AI capabilities will emerge tomorrow from today’s breakthroughs in the André-Oort conjecture and Hilbert’s sixth problem, celebrated at the Fields Medal?

In the past, it was often said that mathematicians forged the way ahead, while AI followed behind to pick up the treasures. But now, the one doing the picking up has begun carving its own path.