Research Projects & Fellows
2026-2027
In the 2026-2027 academic year, CAIRFI is pleased to support six faculty-led research projects.
Understanding and Stabilizing Reinforcement Learning Dynamics in Post-Training Language Models
PI: Adam Block, Department of Computer Science
Post-training language models is an increasingly important but still poorly understood optimization problem. Compared with pre-training, post-training starts from a strong warm start, uses fewer tokens, and often optimizes noisy objectives whose gradients only indirectly reflect the final behavior we care about. These features make the training dynamics sensitive to seemingly small choices: the optimizer, the averaging rule, the decay schedule, the amount of regularization, and, in reinforcement learning, the mechanism used to keep the policy close to a reference model. The goal of this project is to develop a more principled account of these dynamics and to use that account to design interventions that stabilize and accelerate learning.
The project will study this problem using stochastic estimation and stochastic control. The starting point is PACE, a recent optimizer intervention we derived from an optimal-control formulation of stochastic optimization with an iterate-averaged returned model. The motivating question behind PACE is simple: if
the model returned at the end of training is an average of the training trajectory, how should we change the optimizer to improve that average? In a simplified stochastic quadratic model, the answer takes the form of a controller that pulls the live iterate toward the running average; a practical approximation gives a lightweight wrapper around AdamW that improves supervised fine-tuning and small-scale language-model pre-training. This project will test how far that idea extends by studying three settings in which the current method raises natural open questions: matrix-based optimizers, limited-data fine-tuning, and reinforcement learning post-training.
Secure CodeLLM with Reasoning (Renewal)
PI: Baishakhi Ray, Department of Computer Science
This project builds on Prof. Ray's prior year project to create a domain-specialized language model framework designed for high-stakes financial applications.
Proactive Defenses for Large Language Models
PI: Junfeng Yang, Department of Computer Science
As Large Language Models (LLMs) are integrated into critical financial and enterprise software, ensuring their safety is paramount. However, static safety alignments are increasingly insufficient: they provide a binary reward signal (Refusal vs. Compliance) that allows the attacker to compute a simulated gradient and iteratively strengthen their attack. As such, adversaries have evolved from using static prompts to deploying "iterative jailbreaks"—automated attacks where an adversary LLM analyzes a target’s refusal, refines its prompt, and retries until it bypasses safety guardrails. Current defenses remain largely reactive; they passively filter inputs or outputs, often failing to stop search-based attacks that can eventually find a loophole given enough attempts. To secure the next generation of AI agents, we require a paradigm shift from passive refusal to proactive disruption.
We propose a novel defense framework, PROACT, designed to "jailbreak the jailbreak." Our key insight is that the very reliance of attackers on feedback loops—specifically, their use of evaluator LLMs to judge attack success—is a vulnerability we can exploit. Rather than simply refusing a malicious query (which provides the attacker with a signal to refine their approach), our proposed system will intentionally inject "spurious responses." These responses are algorithmically crafted to appear as successful, harmful outputs to the attacker’s automated evaluator, but actually contain benign, meaningless, or non-functional content. By feeding false positive signals back into the attacker’s optimization loop, PROACT corrupts the binary reward signal of the static defense, and misleads the attacker’s internal evaluator into believing the attack succeeded, causing the adversarial search to terminate prematurely.
Fast Controllable Update Summarization
PI: Kathy McKeown, Department of Computer Science
We propose a new approach to generate fast, faithful, and controllable summaries by building an abstractive summarization pipeline within our framework for masked diffusion language models (MDLMs) (Horvitz et al 2025). We envision applying this model to the task of update summarization, where new updates are generated on an unfolding event as changes occur. The summary can be iteratively modified by our method in real-time, highlighting new information as it comes in. We imagine applying this to the generation of updates on events that may impact financial markets. For example, it may be used to track updates about companies (acquisitions or mergers), people of interest, policies (e.g., tariffs), and unfolding disasters.
Vibe-Working: Agentic Infrastructure for Humans to Become AI Supervisors for General Knowledge Work (Renewal)
PI: Lydia Chilton, Department of Computer Science
This project builds on Prof. Chilton's DoubleAgents project to create systems that allow AI agents to coordinate with humans in complex real-world environments.
Uncertainty Estimates for Black-Box API Language Models
PI: Micah Goldblum, Department of Electrical Engineering
Continual learning, updating an already-trained language model with new information, remains a major obstacle to keeping deployed LLMs current with evolving software, recent events, and individual users. Recent methods perform continual learning via self-distillation, distilling a teacher with new documents in its context into a student that does not. These works concentrate on the fine-tuning algorithm while synthesizing training data with simple, single-document question-answer pairs. We hypothesize that the data synthesis strategy, rather than the exact fine-tuning algorithm, is the primary driver of continual learning performance. We will develop rich synthetic data mixes that capture multi-document, multi-hop, belief-revising, and indirectly relevant queries, and we will pair them with existing training algorithms such as self-distillation and supervised fine-tuning. We hypothesize that
these mixes unlock favorable scaling laws: performance that keeps improving as more data is synthesized, in contrast to the rapid saturation of naive single-document pairs. We will open-source all models, our data-synthesis pipeline, and our evaluation suite.
2025-2026
In the 2025-2026 academic year, CAIRFI is pleased to support five faculty-led research projects.
Trustworthy Lending: Explaining Group Differences
PI: Elias Bareinboim, Associate Professor, Computer Science, Columbia Engineering
This project develops new methodologies and tools to explain why people from different demographic groups may receive different outcomes when applying for loans. By uncovering the causes underlying these disparities, the work aims to support more transparent and suitable lending practices that can benefit both banks and borrowers.
DoubleAgents: Using AI Agents for Task Completion and Evaluation
PI: Lydia Chilton, Assistant Professor, Computer Science, Columbia Engineering
DoubleAgents is a general-purpose AI agent system designed to coordinate and allocate human effort in complex, multi-person tasks—such as organizing events, making collective decisions, or managing team-based projects. The system uses AI agents both to complete coordination tasks (e.g., negotiating, emailing, scheduling) and to simulate human behavior for evaluation, allowing it to adapt to strategic, unpredictable, and socially nuanced interactions.
By learning from real-world feedback and simulation-based evaluation, DoubleAgents improves coordination outcomes while reducing the burden on human organizers. Beyond AI research, this work contributes to scalable human-AI collaboration tools that can improve productivity, equity, and responsiveness in domains like education, civic planning, and organizational management.
Reinforcement Learning for Training LLMs to Engineer Tabular Features
PI: Micah Goldblum, Assistant Professor, Electrical Engineering, Columbia Engineering
Data scientists spend an inordinate amount of their time engineering new features out of existing columns in tabular datasets. This procedure often leads to large performance boosts for downstream predictors trained on these features. Automating feature engineering would be valuable, but supervised learning approaches are difficult since we do not know the ground-truth best feature set for any given tabular dataset. We will design reinforcement learning algorithms for fine-tuning large language models (LLMs) to perform feature engineering. LLMs have a strong advantage over alternatives because they can read textual column headers and benefit from pretraining. This general approach may be applicable across data science applications broadly, and we plan to test it out in applications ranging from medical diagnosis to finance.
Foundation Model for Adaptive Experimentation
PI: Hongseok Namkoong, Assistant Professor; Decision, Risk, and Operations Division; Columbia Business School
The ability to navigate new environments and make robust decisions is a central hallmark of intelligence. This project advances AI’s societal impact by proposing a new approach to agentic systems that interact with the real world and continuously improve from feedback. We propose to develop techniques for serializing structured interaction data into natural language format, and build tabular foundation models capable of handling long experiences. Our framework provides the groundwork for AI agents that design their own learning curriculum: exploring the environment by quantifying its own uncertainty and taking actions to actively resolve it.
Secure CodeLLM with Reasoning
PI: Baishakhi Ray, Associate Professor; Computer Science, Columbia Engineering
We propose a secure, domain-specialized language model framework designed for high-stakes financial applications, where generic AI systems often fall short. It tackles sector-specific risks such as unauthorized transactions, compliance violations, and logic errors that can compromise critical systems in banking, asset management, and payments. By grounding its reasoning in executable financial logic and regulatory constraints, the model ensures outputs that are not only syntactically correct but also semantically aligned with institutional policies and legal requirements. A key innovation is its ability to handle both precisely defined rules (e.g., capital adequacy thresholds) and context-dependent principles (e.g., fair lending or risk appetite), blending formal verification with adaptive, judgment-based reasoning. The result is a robust and trustworthy foundation for deploying AI in financial workflows—one that prioritizes correctness, compliance, and security by design.
2024-2025
In the 2024-2025 academic year, supported two faculty-led research projects and two PhD Fellows. Read more about them and their work below.
User Session-Level Counterfactual Simulator
PI: Prof. Anish Agarwal, Industrial Engineering and Operations Research
About the Project
With AI being deployed to make decisions in critical areas, counterfactual reasoning through causal inference is crucial to personalized decision-making. In digital platforms such as Capital One, the most fine-grained data that is collected about users is “session-level trace data”. Such data records highly specific behavior of users such as which buttons they click on, what items they view and purchase etc. Session-level data also records the state of the digital platform as users navigate it, such as what ads/text/images they are shown on various parts of the platform. Toward the goal of truly personalized decision-making, the essential question at hand is as follows: “For a given set of interactions between a user and a platform thus far during a session, what is the predicted trajectory for a user if the platform intervened and changed a state of the system in some way?”. That is, in real-time during a session, could a platform create a “counterfactual” simulator of that user’s predicted trajectory (e.g., what features they click on, what their engagement will be) if a specific text/image is shown? Our goal is to build such a counterfactual simulator by effectively leveraging historical session-level trace data.
About Anish Agarwal
Most recently, Anish was a research scientist at Amazon, Core AI, before which he was a postdoctoral research fellow at the Simons Institute at UC Berkeley. He did his PhD at MIT EECS, where he was fortunate to be advised by Alberto Abadie, Munther Dahleh, and Devavrat Shah. He has spent time at Microsoft Research. He's also served as a technical consultant to TauRx Therapeutics and Uber Technologies on questions related to experiment design and causal inference. Before the PhD, he was a management consultant at Boston Consulting Group. He got his undergrad and master's degrees from Caltech under the supervision of Mani Chandy and Adam Wierman.
A Framework for Responsible LLM Deployment in a Changing World
PI: Prof. Richard Zemel, Computer Science
About the Project
Our world is open-ended, non-stationary, and constantly evolving; thus what we talk about and how we talk about it change over time. The inherently dynamic nature of language---constantly adapting to integrate new information and conditions---contrasts with the current static language modeling paradigm, which trains and evaluates models on utterances from overlapping time periods. Despite impressive progress, recent results have show that large language models (LLMs) perform worse in the realistic setup of predicting future utterances from beyond their training period, and that model performance becomes increasingly worse with time. A fundamental aim in deploying these LLMs is to obtain some performance guarantee.
While most techniques for evaluating these models focus on average performance on a validation set, this can lead to a deployment where unexpectedly poor responses are generated, which is especially dangerous in the financial services domain. Our team has developed a framework for deriving rigorous bounds on the worst- case performance of any AI model. We offer methods for producing bounds on a diverse set of metrics, including quantities that measure worst- case responses and disparities in model responses across the population of users. The focus of this proposal is to extend the underlying statistical techniques used to produce these bounds in order to accommodate distribution shifts in deployment, and demonstrate our framework's application to the important setting of temporal adaptation.
We will collaborate with the AI Foundations team at Capital One to explore tailoring these innovations to key issues arising in continual-learning LLMs developed in the financial setting.
About Rich Zemel
Richard Zemel is the Trianthe Dakolias Professor of Engineering and Applied Science in the Computer Science Department at Columbia University. He is the Director of the NSF AI Institute for Artificial Intelligence and Natural Intelligence (ARNI). He was the Co-Founder and inaugural Research Director of the Vector Institute for Artificial Intelligence. He is a Canadian Institute for Advanced Research AI Chair, an Amazon Scholar, and is on the Advisory Board of the Neural Information Processing Society. His research contributions include foundational work on systems that learn useful representations of data with little or no supervision; graph-based machine learning; and algorithms for fair and robust machine learning.
Leonardo Toso, PhD Fellow
Project: Bayesian Priors for Efficient Multi-task Representation Learning
About the Project
We propose to investigate the problem of learning latent representations from multi-task, non-i.i.d., and non-isotropic datasets while leveraging prior information on the local and global latent variables to enhance the recovery process. A fundamental idea underpinning recent advances in machine learning is the ability to extract shared features from diverse task data. Intuitively, utilizing all available data to unveil a latent representation across multiple tasks reduces computational complexity and enhances statistical generalization by minimizing the number of parameters that require fine-tuning for a specific task. This encompasses and is not limited to the setting where the objective is to make accurate financial portfolio recommendations based on the client's personal investment preferences. Since multiple clients in a database may share common interests, unveiling such features (i.e., learning a representation) is paramount to performing accurate and efficient predictions on the clients' predilections to meet their long-term financial objectives. Moreover, prior information on the representation (e.g., sparsity, low-rankness, structural information, engineer's knowledge, among others) is often available. Accurately handling such prior beliefs may be critical for a more efficient multi-task representation learning framework.
About Leonardo Toso
Leonardo is a second-year Ph.D. student in the Department of Electrical Engineering at Columbia University, where he is advised by Professor James Anderson. His research interests lie in the intersection between control theory, machine learning, and optimization. Prior to joining Columbia, he was an undergraduate research assistant in the Department of Engineering Science at the University of Oxford. Leonardo was awarded his M.S. in Control, Signal, and Image Processing from the University of Paris-Saclay and his M.Eng. in Electrical Engineering from CentraleSupélec. In addition, he received his B.Eng. in Electrical Engineering from the University of Campinas (Unicamp).
Sachit Menon, PhD Fellow
Project: Towards Trustworthy Decision Making in Artificial Intelligence
About the Project
While recent advances in artificial intelligence, such as large language models (LLMs), have enabled unprecedented capabilities, they can fail unpredictably and unsafely. This lack of trust makes them unsuitable for many applications in the real world, such as finance, where safety is critical. My research aims to bridge this gap by developing systems that justify their decisions, give reasons to diagnose failure, and – critically – provide recourse, exposing simple ways for users to prevent failures moving forward.
This work aligns closely with the mission of CAIRFI in multiple ways. First, I am developing more trustworthy techniques that will make use of AI technology in the financial services industry more feasible. Rather than trading off trust with performance, I am innovating on neurosymbolic methods to actually enhance existing capabilities while being inherently explainable. The applications of my research in computer vision can be used to understand economic trends from satellite, social media images, and other ground images, improving financial forecasting. My previous work has shown that these methods open up entirely new avenues to combat hallucination and failure in large models, with a notable application being correction of cross-cultural bias.
About Sachit Menon
Sachit Menon is a PhD student in Computer Science at Columbia University advised by Professor Carl Vondrick. His research centers around models trained at scale and ways to use them for novel tasks, such as using large language models to perform visual reasoning.