Latent Reasoning with Normalizing Flows

Guancheng Tu^1*,Xiangjun Fu^1*,Suhao Yu¹,Yao Tang¹,Haoqiang Kang²,Lianhui Qin²,Yizhe Zhang³,Jiatao Gu¹

* Equal contribution

¹University of Pennsylvania²UC San Diego³Meta

Large language models often improve reasoning with explicit chain-of-thought (CoT), but expressing thoughts as text confines reasoning to a verbose, low-information-density channel. Latent CoT addresses this limitation by moving intermediate reasoning into continuous states. However, existing latent methods often fail to retain the core advantages of explicit CoT: native autoregressive generation, probabilistic sampling, and tractable likelihood estimation. We propose NF-CoT, a latent reasoning framework that preserves these advantages by modeling continuous thoughts with normalizing flows. NF-CoT realizes a TARFlow-style normalizing flow within the LLM backbone, defining a tractable probability model over compact continuous thoughts learned from explicit CoT. This yields tractable likelihoods for continuous thoughts, supports direct policy-gradient optimization in the latent reasoning space, and enables probabilistic left-to-right decoding with the original KV cache. On code-generation benchmarks, NF-CoT improves pass rates over explicit-CoT and prior latent-reasoning baselines while substantially reducing intermediate-reasoning cost.

Four Paradigms for Chain-of-Thought Reasoning

Comparing different approaches to intermediate reasoning in language models

discrete token hidden state diffusion latent Gaussian

noise

Method Overview

NF-CoT training and inference pipeline. Training uses a frozen VAE encoder and shallow flow blocks to convert discrete-CoT rationales into continuous-CoT states.

Key Insight: At inference, NF-CoT samples continuous thoughts left-to-right and continues answer generation in the same causal stream—without recomputing the prompt or running the VAE/shallow blocks.

Performance Results

Comprehensive evaluation on code generation benchmarks comparing NF-CoT with autoregressive, diffusion, and latent reasoning baselines.

Model / Method	Base Model	MBPP	MBPP+	HumanEval	HumanEval+	LCB v6	Avg.
Open-Sourced Models
Autoregressive Coding Models
Qwen 2.5 Coder*autoregressive	Qwen2.5-7B	75.9	62.9	66.5	60.4	26.8	59.9
OpenCoder*autoregressive	from scratch (8B)	79.9	70.4	66.5	63.4	29.6	62
OlympicCoder*autoregressive	Qwen2.5-7B	80	66.4	82.1	76.9	37.3	68.5
Seed-Coder*autoregressive	from scratch	82	69	77.4	68.3	28.4	65
Diffusion Language Models
Dream*diffusion	Qwen2.5-7B	68.7	57.4	56.7	50	18.6	—
LLaDA*diffusion	from scratch (8B)	50.1	42.1	35.4	30.5	12.4	—
Diffu-Coder*diffusion	Qwen2.5-Coder-7B	75.1	61.9	72	65.2	24.5	59.7
Dream-Coder*diffusion	Qwen2.5-Coder-7B	75.9	61.6	66.5	60.4	21.4	57.2
d1*diffusion	LLaDA+RL	39	—	45.5	—	—	—
Looped Latent Reasoning Models
Ouro*looped	from scratch (2.6B)	80.4	66.6	78.2	70.7	38.7	66.9
Method Comparison
Same Backbone Comparison (Qwen3-8B-Base)
Base Model	Qwen3-8B-Base	60.5	53.8	78.2	68.6	17.7	55.8
Standard SFT	Qwen3-8B-Base	63.3+2.8	52.7-1.1	84.6+6.4	69.5+0.9	—	—
Soft Thinking	Qwen3-8B-Base	64.2+3.7	53.1-0.7	85+6.8	71.2+2.6	—	—
TaH+	Qwen3-8B-Base	65.6+5.1	56.5+2.7	85.8+7.6	74.3+5.7	—	—
LaVAE	Qwen3-8B-Base	42-18.5	30.2-23.6	47.8-30.4	32.8-35.8	10.8-6.9	32.7-23
LaDiRdiffusion	Qwen3-8B-Base	66.8+6.3	59.5+5.7	87.4+9.2	73.2+4.6	21.3+3.6	61.6+5.9
NF-CoT (Dual-Path)normalizing flowOurs	Qwen3-8B-Base	77.5+17	66.2+12.4	82.9+4.7	77.7+9.1	21.6+3.9	65.2+9.4
NF-CoT (Unified)normalizing flowOurs	Qwen3-8B-Base	83.9+23.4	72.1+18.3	85.8+7.6	78.3+9.7	23.7+6	68.8+13
↳ + RLnormalizing flowOurs	Qwen3-8B-Base	85.4+24.9	73.3+19.5	86.7+8.5	80.2+11.6	25.1+7.4	70.1+14.3

autoregressiveStandard LLM decodingdiffusionIterative denoisingnormalizing flowTractable likelihood (Ours)

Pass@k Scaling

Pass@k measures the probability that at least one of k generated samples passes all test cases. NF-CoT's probabilistic sampling in continuous thought space enables diverse solution exploration, leading to consistent improvements over both the base model and LaDiR across all k values.

Figure: Pass@k on MBPP+ (left) and HumanEval+ (right). NF-CoT continues to improve with larger k, demonstrating effective sampling diversity from the learned continuous thought distribution.

Case Studies

Qualitative examples showing how different latent samples lead to different implementation strategies while still producing functionally correct programs.

Note: Decoded latent CoTs are qualitative probes, not faithful natural-language explanations of the model's internal reasoning.

HumanEval/63: fibfib64/64 passed

Problem

def fibfib(n: int):
    """The FibFib number sequence is a sequence similar to the 
    Fibonacci sequence that's defined as follows:
    fibfib(0) == 0
    fibfib(1) == 0
    fibfib(2) == 1
    fibfib(n) == fibfib(n-1) + fibfib(n-2) + fibfib(n-3).
    
    Please write a function to efficiently compute the n-th 
    element of the fibfib number sequence.
    >>> fibfib(1)
    0
    >>> fibfib(5)
    4
    >>> fibfib(8)
    24
    """

This task demonstrates how the same problem produces 3 passing answers with genuinely different implementation strategies.

Decoded Latent CoT:

To calculate the Fibonacci number sequence up to the n-th number, we need to follow these steps: ### Plan 1. **Understanding the Fibonacci Sequence**: The Fibonacci sequence is defined as: - F(1) = 0 - F(2) = 1 - F(n) = F(n-1) + F(n-2) 2. **Compressing the Formula for Efficiency**: We use a fixed amount of space to store the last three Fibonacci numbers and calculate the next. This optimizes, since we don't need to use n variables. 3. **Flow**: This calculation is done in constant O(1) extra space by using three variables.

Generated Answer:passed

def fibfib(n: int) -> int:
    """Compute the n-th element of the fibfib sequence."""
    # Base cases
    if n == 0:
        return 0
    elif n == 1:
        return 0
    elif n == 2:
        return 1
    # Initialize the first three elements
    a, b, c = 0, 0, 1
    # Compute the sequence iteratively
    for _ in range(3, n + 1):
        next_value = a + b + c
        a, b, c = b, c, next_value
    return c

HumanEval/154: cycpattern_check61/64 passed

HumanEval/89: encrypt64/64 passed

Key Takeaway

Across these examples, decoded latent CoTs consistently reflect the coarse algorithmic route followed by the final program. NF-CoT does not merely store a single hidden solution trace—instead, its latent space captures a distribution over plausible reasoning trajectories, allowing different samples to steer generation toward distinct but correct implementations.

Citation

BibTeX

@article{tu2026nfcot,
  title={Latent Reasoning with Normalizing Flows},
  author={Tu, Guancheng and Fu, Xiangjun and Yu, Suhao and Tang, Yao and Kang, Haoqiang and Qin, Lianhui and Zhang, Yizhe and Gu, Jiatao},
  journal={arXiv preprint arXiv:2506.XXXXX},
  year={2026}
}