Why AI Can Remember All Human Knowledge But Not 'Yesterday'

Author: Hugo, Robonaissance

Compiled by Deep潮 TechFlow

Shenchao Overview: From early Atari game agents to large language models with millions of context windows, artificial intelligence has always faced a fundamental physical contradiction when making sequential decisions across time: models can easily accommodate vast amounts of existing human static knowledge, yet in dynamic interactive environments, they cannot carry forward all past details without loss.

Under hard constraints of limited computational power and context window, accurately distilling the core states that determine the future while compressing the past is not only a technical breakthrough for resolving “perceptual confusion,” but also the foundational logic for autonomous agents to achieve true persistent memory.

This is Episode 3 of the series "Memory, Forgetting, and the Self in the Machine": How intelligence unfolds across time when agents cannot carry their entire past with them.

Try pausing any frame of the classic Pong paddle game.

The screen shows only two rackets, a ball, and the current score. At this moment, if you were asked to say which direction the ball is moving, you simply couldn’t tell—even if you showed the same static image to a machine, it would be equally unable to determine.

Scholars Matthew Hausknecht and Peter Stone explicitly stated in their 2015 paper on agents learning to play Atari games: "Current single-frame screens reveal only the positions of the paddle and the ball, but not the ball's velocity."

They then explained why this is critical: “Knowing the direction of the ball’s movement is essential to determining the optimal position of the racket.”

This still frame is precise, with every detail in its proper place. But this single frame alone is still not enough to play the game.

What the sensor captures at this very moment is merely an "observation."

The true global state upon which the agent's next action depends is the "system state (State)."

In the Pong game, observations and system states are disjointed on each individual frame slice. This is because the ball’s direction of motion depends on facts from several previous moments, and no single isolated moment can contain it alone.

When immediate input is insufficient for decision-making, a well-performing system must carry information from its past into the present.

But what it carries can never be the complete past. It carries only a summary, and the selection of that summary depends entirely on what it needs to predict next.

This trade-off determines the informational foundation available to subsequent moments. Any detail omitted in the summary will forever be unavailable for subsequent moments to respond to.

The system that Hausknecht and Stone responded to was the DQN agent introduced by Volodymyr Mnih and others at DeepMind in 2015 in the journal Nature.

DQN was able to learn how to play dozens of Atari games purely from screen pixels, but the paper’s authors encountered the problem of frozen frames early in their development.

In the methodology section of their paper, they wrote: "Since the agent can only observe the current screen, this task constitutes a partially observable state, with many simulator states being perceptually aliased."

In other words, the current screen display alone cannot provide the agent with a complete picture of its situation.

所谓“感知混淆”，是对日常困扰的一种精确学术描述。两个截然不同的情境（例如球正飞向你的球拍，或球正飞离你的球拍），在静态画面中会产生完全相同的图像。

A system that relies solely on images is forced to treat them as the same scenario; if these two scenarios require completely opposite actions, the system will inevitably make an error in one of them.

Their remedy is very straightforward: instead of receiving just one screen frame, the network now stacks the most recent few frames together as input.

By default, the value is set to 4 frames, although the algorithm is also adaptable to values of 3 or 5 frames.

When the four frames are placed side by side, the ball's motion immediately appears as a change in position. The direction of motion is no longer an abstract concept that must be remembered, but becomes a visual property that can be directly seen.

The paper does not refer to this stack of frames as memory; it is essentially just an input channel that is slightly wider than a single moment.

This input width has extremely rigid physical boundaries, which Hausknecht and Stone precisely defined.

They wrote: "Since DQN is trained using inputs composed of the agent's most recent four states, it will be unable to master a game that requires the player to remember events further back than the last four screens."

For DQN, the history beyond the fifth-to-last screen is not within its input range.

This fixed sliding window mechanism directly reveals the core contradiction in machine intelligence for time series processing: no matter how wide the window, information outside the boundaries is physically erased.