Ziteng Pang

ZitengPang2027@u.northwestern.edu

I’m a fourth-year Ph.D. student in the Statistics and Data Science department at Northwestern University under the supervision of Professor Bradly C. Stadie and Professor Han Liu. My research interests lie at the intersection of LLMs, reinforcement learning, and robotics, with a particular current focus on exploring generalist vision-language-action models. I have also worked on developing faithful medical LLMs through embedding models and reward models, as well as exploring the implications of using pretrained VLM backbones for robotic manipulation policies.

Prior to this, during my master’s at the University of Michigan, I was a core member of the Bayesian Light Source Separator (BLISS) project under the supervision of Professor Jeffrey Regier. We developed an end-to-end variational autoencoder with a Wake-Sleep algorithm to automatically catalog thousands of overlapping astronomical light sources in telescope images using variational inference. I also worked on machine olfaction research under the supervision of Professor Ambuj Tewari, focusing on using Graph Neural Network architectures to model molecular odor properties from chemical structure.

recent publications

TMLR

Thoughts and Lessons on Using Visual Foundation Models for Manipulation

Chen, Ryan, Pang, Ziteng, and Stadie, Bradly C.

Transactions on Machine Learning Research 2025

Abs PDF

Training vision-based robotic systems from scratch is both computationally expensive and memory intensive. To mitigate these challenges, recent approaches forgo end-to-end training in favor of adopting visual representations from visual foundation models – large scale models designed for broad task transferability. Recent years have seen numerous vision foundation models emerge, including several designed specifically for manipulation tasks. However, we still lack clear principles for what makes these models effective for robotics applications. To address this gap, we systematically evaluate vision foundation models to understand what makes them effective for offline robotic learning. We find that across eleven diverse vision encoders, a representation’s ability to reconstruct edges and predict keypoints strongly correlates with its performance on manipulation tasks. Extensive correlation analysis across 21 manipulation tasks consistently shows that representations preserving edge and keypoint information achieve the highest environment success rates. These findings appear to challenge conventional wisdom about holistic reconstruction-based pretraining and offer a new lens for understanding what makes vision representations effective for robotics.
NPJ Digit. Med.

Expert of Experts Verification and alignment (EVAL) framework for large language models safety in gastroenterology

Giuffrè, Mauro, You, Kisung, Pang, Ziteng, Kresevic, Simone, Chung, Sunny, Chen, Ryan, Ko, Youngmin, Chan, Colleen, Saarinen, Theo, Ajcevic, Milos, Crocè, Lory S, Garcia-Tsao, Guadalupe, Gralnek, Ian, Sung, Joseph J Y, Barkun, Alan, Laine, Loren, Sekhon, Jasjeet, Stadie, Bradly, and Shung, Dennis L

NPJ Digit. Med. May 2025

Abs PDF

Large language models generate plausible text responses to medical questions, but inaccurate responses pose significant risks in medical decision-making. Grading LLM outputs to determine the best model or answer is time-consuming and impractical in clinical settings; therefore, we introduce EVAL (Expert-of-Experts Verification and Alignment) to streamline this process and enhance LLM safety for upper gastrointestinal bleeding (UGIB). We evaluated OpenAI’s GPT-3.5/4/4o/o1-preview, Anthropic’s Claude-3-Opus, Meta’s LLaMA-2 (7B/13B/70B), and Mistral AI’s Mixtral (7B) across 27 configurations, including zero-shot baseline, retrieval-augmented generation, and supervised fine-tuning. EVAL uses similarity-based ranking and a reward model trained on human-graded responses for rejection sampling. Among the employed similarity metrics, Fine-Tuned ColBERT achieved the highest alignment with human performance across three separate datasets (ρ= 0.81-0.91). The reward model replicated human grading with 87.9% of cases across temperature settings and significantly improved accuracy through rejection sampling by 8.36% overall. EVAL offers scalable potential to assess accuracy for high-stakes medical decision-making.