Zach Lawrence

Publications

SimulCost: A Cost-Aware Benchmark and Toolkit for Automating Physics Simulations with LLMs

Yadi Cao, Sicheng Lai, Jiahe Huang, Yang Zhang, Zach Lawrence, Rohan Bhakta, Izzy F. Thomas, Mingyun Cao, Chung-Hao Tsai, Zihao Zhou, Yidong Zhao, Hao Liu, Alessandro Marinoni, Alexey Arefiev, Rose Yu

International Conference on Machine Learning (ICML), 2026

Evaluating LLM agents for scientific tasks has focused on token costs while ignoring tool-use costs like simulation time and experimental resources. As a result, metrics like pass@k become impractical under realistic budget constraints. To address this gap, we introduce SimulCost, the first benchmark targeting cost-sensitive parameter tuning in physics simulations. SimulCost compares LLM tuning cost-sensitive parameters against traditional scanning approach in both accuracy and computational cost, spanning 2,643 single-round (initial guess) and 2,304 multi-round (adjustment by trial-and-error) tasks across 11 simulators from fluid dynamics, solid mechanics, and plasma physics, whose costs are analytically defined and platform-independent. A twelfth simulator, a production plasma code measurable only by wall clock, is reported separately. Frontier LLMs achieve 45–62% success rates in single-round mode, dropping to 34–50% under high accuracy requirements, rendering their initial guesses unreliable especially for high accuracy tasks. Multi-round mode improves rates to 66–81%, but LLMs are 1.5–2.7x slower than traditional scanning, making them uneconomical choices. We also investigate parameter group correlations for knowledge transfer potential, and the impact of in-context examples and reasoning effort, providing practical implications for deployment and fine-tuning. We open-source SimulCost as a static benchmark and extensible toolkit to facilitate research on improving cost-aware agentic designs for physics simulations, and for expanding new simulation environments.

Deep Learning for Modeling and Dispatching Hybrid Wind Farm Power Generation

Zach Lawrence, Jessica Yao, Chris Qin

IEEE International Conference on Data Mining Workshops (ICDMW), 2025

Wind farms with integrated energy storage, or hybrid wind farms, are able to store energy and dispatch it to the grid following an operational strategy. For individual wind farms with integrated energy storage capacity, data-driven dispatch strategies using localized grid demand and market conditions as input parameters stand to maximize wind energy value. Synthetic power generation data modeled on atmospheric conditions provide another avenue for improving the robustness of data-driven dispatch strategies. To these ends, the present work develops two deep learning frameworks: COVE-NN, an LSTM-based dispatch strategy tailored to individual wind farms, which reduced annual COVE by 32.3% over 43 years of simulated operations in a case study at the Pyron site; and a power generation modeling framework that reduced RMSE by 9.5% and improved power curve similarity by 18.9% when validated on the Palouse wind farm. Together, these models pave the way for more robust, data-driven dispatch strategies and potential extensions to other renewable energy systems.

Talks & Posters

  • Yadi Cao, Sicheng Lai, Jiahe Huang, Yang Zhang, Zach Lawrence, Rohan Bhakta, Izzy F. Thomas, Mingyun Cao, Chung-Hao Tsai, Zihao Zhou, Yidong Zhao, Hao Liu, Alessandro Marinoni, Alexey Arefiev, Rose Yu. “SimulCost: A Cost-Aware Benchmark and Toolkit for Automating Physics Simulations with LLMs.” Poster, International Conference on Machine Learning (ICML), 2026.
  • Zach Lawrence, Jessica Yao. “Deep Learning for Modeling and Dispatching Hybrid Wind Farm Power Generation.” Talk, ICDM Workshops, Washington, D.C., November 2025.
  • Zach Lawrence, Jessica Yao. “Deep Learning for Strategic Power Generation for Hybrid Wind Farms.” Poster, Summer Undergraduate Research Symposium, Vancouver, WA, August 2025.
  • Jessica Yao, Zach Lawrence. “Atmospheric Data-Driven Time Series Modeling of Wind Power Generation.” Poster, Summer Undergraduate Research Symposium, Vancouver, WA, August 2025.
  • Zach Lawrence, G. Wrye, A. Panthi, R. Rai. “Mangrove Monitoring.” Poster, Engineers for Exploration REU Symposium, Escondido, CA, August 2024.