According to media reports, OpenAI’s upcoming Astra model employs a recurrent depth architecture (also known as a looped transformer).
Unlike conventional Transformers, which pass each generated token through all layers exactly once, Astra allows the same information to be repeatedly processed through the same set of layers over multiple rounds before output. ByteDance revealed a similar approach last year.
Seed team's Ouro is a Looped Language Model that similarly cycles a set of Transformer layers, embedding more computation within the model itself to enable smaller models to approach the performance of larger models by increasing inference computation.
This architecture raises security concerns: some reasoning occurs within internal hidden states, making it impossible for humans to see the full text record and harder to audit whether the model is violating rules through chain-of-thought analysis.
OpenAI has therefore limited Astra's use of recurrent depth to preserve readable chains of thought and plans to add additional CoT monitoring.