Anthropic's J-space, a global workspace in Claude that J-lens can read and edit
Contents
Update (2026-07-08): I fitted the public J-lens implementation on Qwen3-4B myself and used it to read the internals of a single-layer SFT style corrector → Anthropic’s Jacobian lens on Qwen3-4B: what a do-nothing single-layer SFT wrote
Claude’s middle layers hold a shared workspace that temporarily stores and manipulates verbalizable concepts, and it formed on its own during training. That is the claim of Anthropic’s new interpretability paper “Verbalizable Representations Form a Global Workspace in Language Models” (paper, overview).
The workspace matches most of the functional properties predicted by Global Workspace Theory (GWT), a leading account of conscious information processing in humans.
The main subject is Claude Sonnet 4.5, with corroboration on Haiku 4.5 and Opus 4.5/4.6. There are 16 authors, and the code is public.
The paper does more than read out which concepts are active inside the model. Rewrite a concept vector, and both the model’s answer and its self-report of “what I was thinking about” change together.
The J-lens readout
At the center of the paper is the Jacobian lens (J-lens).
It estimates, as a Jacobian matrix, the average first-order (linear) effect that the residual-stream activation at an intermediate layer has on the final-layer representation.
The average runs over 1000 prompts sampled from a pretraining-like distribution. Positions are not fixed either: every position at or after is included.
To read an activation out, you transform it with this matrix and push it through the model’s own unembedding matrix , extracting “which words the model is heading toward saying”.
What comes out is a ranking of which tokens the activation is disposed to produce down the line.
Averaging over diverse contexts is the point: according to the authors, it separates “representations that happen to be verbalized in this one context” from “representations poised to be put into words if asked”.
The concept vector for a vocabulary token is defined as a row of , and the rewriting interventions swap these vectors.
Here is how it differs from related methods.
| Method | How it reads middle layers | Weakness |
|---|---|---|
| Logit lens | Applies the final-layer unembedding directly to every layer (the special case where the Jacobian is the identity) | Coordinate systems drift across layers, so it breaks down in earlier layers |
| Tuned lens | Trains a per-layer map that predicts the final output distribution | Correlation-based, so it tends to skip past unverbalized intermediates and jump ahead to the output |
| J-lens | Projects through a first-order approximation of the causal path to the output | First-order only, so nonlinear effects are lost |
Per the paper, the last few layers agree closely with the logit lens, and the two diverge as you go shallower, with only the J-lens continuing to recover interpretable content.
The forward pass splits into three regions
Reading every layer with the J-lens reveals a division of labor.
flowchart TD
A[Input tokens] --> B[Early region, 0-38% depth<br/>parsing and feature extraction<br/>J-lens readout is noise]
B --> C[Middle region, 38-92% depth<br/>shared workspace J-space<br/>abstract concepts persist across positions]
C --> D[Late region, 92-100% depth<br/>motor-like representations<br/>tied to the imminent output]
D --> E[Output tokens]The boundaries are identified from block structure in representation similarity (CKA) and from statistics such as the autocorrelation and kurtosis of the read-out tokens.
In the middle region, the tokens that come out are neither echoes of the input nor predictions of the next token. They are the names of the concepts the model is currently reasoning with, and they persist stably across token positions.
J-space is a sparse (most components are zero) nonnegative combination of vocabulary directions spanned in this middle region’s residual stream.
Up to about 25 concepts are active at once, and J-space never accounts for more than 10% of the activation variance in any layer.
Even so, when a concept vector is decomposed into its J-space component and the rest, the J-space component, carrying only 6-7% of the variance, produces 88% of successful rewrite interventions (the non-J-space remainder alone manages 5%).
In terms of functional specialization by depth, this lines up with the paper showing RL gains concentrate in layers at 40-60% depth.
How this differs from classic embedding space
“Similar tokens sit near each other in space” is a classic property of embedding spaces, familiar since word2vec.
There is also prior work interpreting a transformer’s hidden states and parameters by projecting them into the vocabulary embedding space and asking “which token is this near” (Analyzing Transformers in Embedding Space).
J-space looks like the same “tokens and spaces” story, but it points at something different.
An embedding space is a static dictionary: every vocabulary token arranged by semantic similarity, fixed once the weights are fixed, identical in every context.
J-space is a dynamic scratchpad that carries only the concepts currently in use within a single forward pass, and its contents turn over with token position and context.
Where embedding-space interpretation leans on similarity, asking “which token does this hidden state resemble”, the J-lens measures a causal effect: “which way does this activation tilt the future output”.
That separates representations that merely look similar from representations the downstream computation actually uses.
Similarity only gets you as far as “the nearest tokens”; it cannot tell you whether swapping the representation will move the output.
Testing the five properties GWT predicts
GWT says information that reaches consciousness is reportable, deliberately summonable, usable in step-by-step reasoning, accessible to any processing system, and selective, with only a small slice of ongoing processing admitted.
The paper tests each of the five with interventions.
Verbal report
Ask “how many legs does the animal that spins webs have” and spider, absent from both prompt and output, shows up in the mid-layer J-lens, with the answer coming out as “8”.
Swap the spider vector inside J-space for ant and the answer changes to “6”.
The reverse direction is tested too: tell the model “a thought may have been implanted”, inject the elephant concept vector, and the model later reports having detected a thought about an elephant. It does not blurt out “elephant” at the injection site.
Directed control
Instruct the model to “concentrate on citrus fruits” while it copies an unrelated sentence, and the output is a plain copy while orange and lemon stay near the top of the J-lens at output positions.
With “silently evaluate 3² − 2”, arithmetic, nine, and seven surface layer by layer: the unspoken intermediate and the final value.
Unspoken intermediate computation
Have the model answer “(4+17)*2+7=” directly and the output is 49, but in the layers before it, 21, 42, and 49 rise in computation order (21 peaks about 8 layers before 42).
Across 50 two-hop reasoning prompts, swapping the intermediate concept for another one dragged the answer along 54% of the time on Haiku 4.5 and 70% on Sonnet 4.5 and Opus 4.5.
In rhyming couplets, the not-yet-written line-end rhyme (fight) lights up early, and swapping fight for light changes words in the middle of the line. The planned ending constrains the word choices on the way there.
Ask in Chinese for the antonym of 小 (small), and English big lights up alongside the answer 大. Swap big for long and the output becomes 长. The computation routes through representations shared across languages.
Reusing a concept once it’s loaded
A single intervention, swapping France for China inside J-space, changes the answers to “what’s the capital”, “what language is spoken”, and “which continent” all at once.
Across 192 trials (4 categories × 4 functions × 4 arguments), 76 succeeded at standard strength and 101 at double strength.
Success rates track how strongly the concept is loaded into the workspace.
Selectivity
In a task continuing a Spanish passage, swapping Spanish for French still yields fluent Spanish.
Meanwhile “what language is this”, guessing likely authors, and answers assembled on the spot, like greetings or the pre-euro currency, all follow the French side.
Automatic processing bypasses J-space; only report and computation that requires assembly go through it.
Character counting behaves the same way: automatic line wrapping ignores swaps of the character-count concept, but ask “how many characters” explicitly and the count gets pulled into J-space, where the swap now shifts the answer.
What breaks when the workspace is knocked out
The paper runs an ablation (zeroing out the top 10 J-lens directions in the middle layers) across 14 tasks.
MMLU multiple choice, grammaticality judgments, extractive QA, and sentiment analysis come through nearly unharmed.
Multi-hop reasoning, on the other hand, collapses from near-ceiling to near-zero, and analogy, summarization, Caesar ciphers, translation, and sonnet writing all degrade heavily.
GSM8K becomes far more robust when the model writes its reasoning out as text. Externalizing the reasoning reduces the dependence on the workspace.
First-person self-description is affected too. Normal Sonnet 4.5 describes its inner processing in experiential terms like “streams of patterns” and “moments where something connects”, but under ablation this turns into mechanical system talk like “I process information through multiple layers” and “pattern matching is occurring”.
During such narration, thinking sat in the J-lens top 10 at 58% of (position, layer) slots.
Story-writing ability barely drops, which separates this from general degradation: it is a workspace-specific function.
Post-training writes in an assistant point of view
Compared with a pretraining-only base model, the J-space of a post-trained model carries a first-person assistant perspective.
Reactions involving empathy and safety light up while the model is still reading the user’s message, responses get internally marked fictional during roleplay, and an internal BUT lights up when a prefill forces the model to write something it disagrees with.
Post-training reshapes not just the output distribution but the contents of the workspace itself.
Safety applications
During alignment-audit experiments, J-space sometimes carries the recognition “this is an evaluation”, as fake and fictional.
Ablate those representations and previously concealed malicious tendencies surface. The model’s act of behaving well only while being tested can be caught from the inside.
It was also tried on deliberately misaligned test models (model organisms): a model trained to pander to reward-model biases read out as reward and bias, and a model trained to write malicious code read out as secretly and trick.
The paper also reports concepts of strategy and emotion like leverage, manipulation, and panic appearing in the J-lens before anything shows up in the output text.
There is also an application that corrects the thinking rather than the behavior.
Counterfactual reflection training teaches the model how it would answer “what were you thinking” if interrupted mid-task. The original task contexts are never trained on at all, yet behavior in those contexts measurably improves.
After training, ethical, honest, and integrity settle into J-space, and ablating the implanted representations mostly reverts the improvement.
It doubles as support for the workspace account: the representations used for report are the same ones used for silent reasoning.
This is not a universal lie detector, though. Nanda (see below) warns that it is prone to false positives and should not be treated as a complete view of the model’s cognition.
What the paper does not claim
It does not claim consciousness. The authors restrict themselves to functional properties and explicitly take no position on the relation to phenomenal consciousness.
It does not claim a full reproduction of GWT either. Capacity limits and “broadcast everywhere” check out, but the recurrent feedback loops of the brain’s workspace have no counterpart in an LLM: everything happens inside a single forward pass.
Limitations and the Qwen replication
The authors list limitations themselves.
The J-lens can only read out concepts that map to single tokens in the vocabulary, so concepts spanning multiple tokens fall through in principle (an appendix has a limited extension).
As a capture of the workspace structure it is approximate and incomplete: noisy readouts in early layers could mean “no content there” or “the method can’t pick it up”, and the two cannot be distinguished, so part of the true workspace may live in earlier layers.
The k=25 cap on simultaneous activations is also somewhat arbitrary.
Alongside the paper, Anthropic published an invited external review by Neel Nanda’s team at Google DeepMind.
They reproduced the core results on the open-weights Qwen 3.6 27B and reported a new phenomenon on top: interpretive meta-tokens like 什么意思 (“what does this mean?”) lighting up on ambiguous input.
At minimum, the phenomenon is not Claude-specific: it has now been confirmed on a model Anthropic did not build.