Sizhuang He | Yale University

About Me

I’m a second-year Computer Science Ph.D. student at Yale University, advised by Dr. David van Dijk. Previously, I completed my Bachelor’s degree at University of Michigan, Ann Arbor, majoring in Honors Mathematics and minoring in Computer Science.

I work on the intersection of machine learning and biology:

Generative Modeling: Flow Matching, Diffusion, Discrete Diffusion
Operator Learning: Modeling Continuous Spatiotemporal Dynamics, Integral Equations
Computational Biology: Single-cell Transcriptomics Data Analysis
LLMs and Agentic AI: Autonomous Systems for Biological Discovery

Check out my CV here: Curriculum Vitae

News

[Sep 2025] Our paper Non-Markovian Discrete Diffusion with Causal Language Models is accepted to NeurIPS 2025 (San Diego, CA).
[Jul 2025] Our paper COAST: Intelligent Time-Adaptive Neural Operators is accepted to AI4MATH@ICML 2025 (Vancouver, Canada).
[Jan 2025] I received the Fan Family Fellowship of Yale University.
[Jan 2025] Our paper Intelligence at the Edge of Chaos is accepted to ICLR 2025 (Singapore).

Selected Publications

Cell2Sentence

C2S-Scale scales this framework to 27 billion parameters trained on a billion-token multimodal corpus—achieving state-of-the-art predictive and generative performance for complex, multicellular analyses. Read more about this work in our Google Research blog post. — bioRxiv

CaDDI

We introduce a novel approach to discrete diffusion models that conditions on the entire generative trajectory, thereby lifting the Markov constraint and allowing the model to revisit and improve past states. CaDDi treats standard causal language models as a special case and permits the direct reuse of pretrained LLM weights with no architectural changes. — NeurIPS 2025 (Poster)

CaLMFlow

We present Volterra Flow Matching, a novel generative modeling framework that reformulates ODE-based flow matching frameworks with Volterra Integral Equations, hence avoiding a core challenge in ODE-based methods, known as stiffness. We show the connection between Volterra Integral Equations and causal transformers, the backbone of modern Large Language Models and hence demonstrates that causal language models can be naturally extended to generative modeling over continuous data domains through the lens of Volterra Flow Matching. — arXiv

Intelligence at the Edge of Chaos

By training LLMs on elementary cellular automata rules of varying complexity, we pinpoint a 'sweet spot' of data complexity that maximizes downstream predictive and reasoning abilities. Our findings suggest that exposing models to appropriately complex patterns is key to unlocking emergent intelligence. — ICLR 2025 (Poster)

Complete Publication List

(*: Equal Contribution; Last Updated: Sep 2025)

Filter by Research Area

# Causal Language Models # Cellular Automata # Complexity Theory # Computational Biology # Diffusion Models # Flow Matching # Generative Modeling # Integral Equations # Large Language Models # Neural Operators # Numerical Analysis # Operator Learning # Single-Cell Analysis

2025

Scaling Large Language Models for Next-Generation Single-Cell Analysis

Syed Rizvi^*, Daniel Levine^*, Aakash Patel^*, Shiyang Zhang^*, Eric Wang^*, Sizhuang He and 17 more authors

bioRxiv

Abstract BibTex PDF Blog

# Large Language Models # Computational Biology # Single-Cell Analysis
Single-cell RNA sequencing has transformed our understanding of cellular diversity, yet current single-cell foundation models (scFMs) remain limited in their scalability, flexibility across diverse tasks, and ability to natively integrate textual information. In this work, we build upon the Cell2Sentence (C2S) framework, which represents scRNA-seq profiles as textual 'cell sentences,' to train Large Language Models (LLMs) on a corpus comprising over one billion tokens of transcriptomic data, biological text, and metadata. By scaling model size to 27 billion parameters, we observe consistent improvements in predictive and generative capabilities, as well as the capacity for advanced downstream tasks requiring synthesis of information across multicellular contexts. Through targeted fine-tuning supported by modern reinforcement learning techniques, our approach excels in tasks such as perturbation response prediction, natural language interpretation, and complex biological reasoning. By unifying transcriptomic and textual data at unprecedented scales, this approach not only surpasses both specialized single-cell models and general-purpose LLMs, but also establishes a powerful platform for next-generation single-cell analysis, paving the way for the development of 'virtual cells.'
Non-Markovian Discrete Diffusion with Causal Language Models

Yangtian Zhang ^*, Sizhuang He^*, Daniel Levine, Lawrence Zhao, David Zhang and 4 more authors

NeurIPS 2025 (Poster)

Abstract BibTex PDF

# Diffusion Models # Causal Language Models # Generative Modeling
Discrete diffusion models offer a flexible, controllable approach to structured sequence generation, yet they still lag behind causal language models in expressive power. A key limitation lies in their reliance on the Markovian assumption, which restricts each step to condition only on the current state, leading to potential uncorrectable error accumulation. In this paper, we introduce CaDDi, a discrete diffusion model that conditions on the entire generative trajectory, thereby lifting the Markov constraint and allowing the model to revisit and improve past states. By unifying sequential (causal) and temporal (diffusion) reasoning in a single non-Markovian transformer, CaDDi also treats standard causal language models as a special case and permits the direct reuse of pretrained LLM weights with no architectural changes. Empirically, CaDDi outperforms state-of-the-art discrete diffusion baselines on natural-language benchmarks, substantially narrowing the remaining gap to large autoregressive transformers.
TANTE: Time-Adaptive Operator Learning via Neural Taylor Expansion

Zhikai Wu, Sifan Wang, Shiyang Zhang, Sizhuang He, Min Zhu and 3 more authors

In Review

Abstract BibTex PDF

# Neural Operators # Operator Learning
Operator learning for time-dependent partial differential equations (PDEs) has seen rapid progress in recent years, enabling efficient approximation of complex spatiotemporal dynamics. However, most existing methods rely on fixed time step sizes during rollout, which limits their ability to adapt to varying temporal complexity and often leads to error accumulation. Here, we propose the Time-Adaptive Transformer with Neural Taylor Expansion (TANTE), a novel operator-learning framework that produces continuous-time predictions with adaptive step sizes. TANTE predicts future states by performing a Taylor expansion at the current state, where neural networks learn both the higher-order temporal derivatives and the local radius of convergence. This allows the model to dynamically adjust its rollout based on the local behavior of the solution, thereby reducing cumulative error and improving computational efficiency. We demonstrate the effectiveness of TANTE across a wide range of PDE benchmarks, achieving superior accuracy and adaptability compared to fixed-step baselines, delivering accuracy gains of 60–80% and speed-ups of 30–40% at inference time. The code is publicly available at https://github.com/zwu88/TANTE for transparency and reproducibility.
COAST: Intelligent Time-Adaptive Neural Operators

Zhikai Wu, Shiyang Zhang, Sizhuang He, Sifan Wang, Min Zhu and 3 more authors

AI4MATH Workshop at ICML 2025 (Poster)

Abstract BibTex PDF

# Neural Operators # Operator Learning
Operator learning for time-dependent partial differential equations (PDEs) has seen rapid progress in recent years, enabling efficient approximation of complex spatiotemporal dynamics. However, most existing methods rely on fixed time step sizes during rollout, which limits their ability to adapt to varying temporal complexity and often leads to error accumulation. To address this gap, we propose the Time-Adaptive Transformer with Neural Taylor Expansion (TANTE), a novel operator-learning framework that produces continuous-time predictions with adaptive step sizes. TANTE predicts future states by performing a Taylor expansion at the current state, where neural networks learn both the higher-order temporal derivatives and the local radius of convergence. This allows the model to dynamically adjust its rollout based on the local behavior of the solution, thereby reducing cumulative error and improving computational efficiency. We demonstrate the effectiveness of TANTE across a wide range of PDE benchmarks, achieving superior accuracy and adaptability compared to fixed-step baselines, delivering accuracy gains of 10-50% and speed‑ups of 30-80% at inference.

2024

CaLMFlow: Flow Matching using Causal Language Models

Sizhuang He^*, Daniel Levine^*, Ivan Vrkic, Marco Bressana, David Zhang and 4 more authors

arXiv

Abstract BibTex PDF

# Flow Matching # Causal Language Models # Generative Modeling # Integral Equations
We introduce CaLMFlow, a novel framework that recasts flow matching as a Volterra integral equation (VIE) while leveraging causal language models (CLMs) for continuous data generation. Although integral equations have previously been studied for nonlocal operator learning, we are the first to apply them explicitly to flow matching. By discretizing both time and space, CaLMFlow transforms continuous trajectories into a sequence-based representation, allowing standard large language model architectures to capture long-range dependencies and flexibly incorporate textual prompts. In experiments on synthetic benchmarks and a single-cell perturbation response task, CaLMFlow demonstrates improved stability and scalability over ODE-based methods in high-dimensional regimes. These findings suggest that combining an integral-equation formulation with CLMs provides a promising, context-aware paradigm for generative modeling.
Intelligence at the Edge of Chaos

Shiyang Zhang^*, Aakash Patel^*, Syed Rizvi, Nianchen Liu, Sizhuang He and 3 more authors

ICLR 2025 (Poster)

Abstract BibTex PDF

# Large Language Models # Complexity Theory # Cellular Automata
We explore the emergence of intelligent behavior in artificial systems by investigating how the complexity of rule-based systems influences the capabilities of models trained to predict these rules. Our study focuses on elementary cellular automata (ECA), simple yet powerful one-dimensional systems that generate behaviors ranging from trivial to highly complex. By training distinct Large Language Models (LLMs) on different ECAs, we evaluated the relationship between the complexity of the data generated by the rules and the models' ability to learn effective general representations, as reflected in their performance on downstream tasks. Our findings reveal that models trained on more complex data exhibit greater predictive ability, as demonstrated by their performance on reasoning and chess move prediction tasks. Both uniform and periodic systems, and often also highly chaotic systems, resulted in poorer downstream performance, highlighting a sweet spot of complexity conducive to intelligence. We conjecture that intelligence arises from the ability to predict complexity and that creating intelligence may require only exposure to complexity.

2023

Operator Learning Meets Numerical Analysis: Improving Neural Networks through Iterative Methods

Emanuele Zappala, Daniel Levine, Sizhuang He, Syed Rizvi, Sacha Lévy and 1 more authors

arXiv

Abstract BibTex PDF

# Numerical Analysis # Operator Learning # Integral Equations
Deep neural networks, despite their success in numerous applications, often function without established theoretical foundations. In this paper, we bridge this gap by drawing parallels between deep learning and classical numerical analysis. By framing neural networks as operators with fixed points representing desired solutions, we develop a theoretical framework grounded in iterative methods for operator equations. Under defined conditions, we present convergence proofs based on fixed point theory. We demonstrate that popular architectures, such as diffusion models and AlphaFold, inherently employ iterative operator learning. Empirical assessments highlight that performing iterations through network operators improves performance. We also introduce an iterative graph neural network, PIGN, that further demonstrates benefits of iterations. Our work aims to enhance the understanding of deep learning by merging insights from numerical analysis, potentially guiding the design of future networks with clearer theoretical underpinnings and improved performance.

Services

Conference Reviewer:
- ICLR
- AI4MATH@ICML2025

Apart from research

Beyond research, my biggest passions include:

Football 🏈: I am a dedicated Michigan Wolverines fan 〽️ (Go Blue!) in college football and root for the Detroit Lions in the NFL.
Traveling: I enjoy exploring different cities—my all-time top three are Tokyo, Taipei, and Singapore. When visiting new places, I particularly enjoy experiencing local train systems, with my favorite being the Yamanote Line 🟢 operated by JR East in Tokyo.