Taleef Tamsal
Software Developer at Medical Informatics Engineering · Researcher in LLM safety and evaluation
I’m a software developer at Medical Informatics Engineering and a researcher in LLM safety and evaluation. I finished my M.S. in Computer Science at Purdue University Fort Wayne in May 2026, where Dr. Jonathan Rusert advised my thesis on how parameter-efficient fine-tuning affects jailbreak robustness.
Most of my research asks whether the numbers we report about model safety can be trusted. I study how fine-tuning changes a model’s resistance to jailbreaks, and how much the answer depends on the automated judge doing the scoring. In one of our studies, GPT-4o-mini labeled 45.7% of outputs that human annotators agreed were safe as jailbreaks. I’m also interested in agentic AI security and in retrieval systems that stay faithful to their sources.
At work I’m the sole developer of WorkWell Measure Studio, which computes CMS clinical quality measures from FHIR data. It’s the same problem in a different setting: checking that a system’s output means what it claims to mean.
News
| Sep 2026 | Your Judge Is a Confound was accepted as an oral presentation at the First Workshop on Reliable Evaluation for Language Models (JUDGe) at NeurIPS 2026. It was also accepted at AdvML-Frontiers × CoTMA at COLM 2026. |
|---|---|
| Sep 2026 | Released the code, result tables, and evaluator labels for Adaptation, Not Algorithm. Most of the paper’s statistics can be recomputed from the released tables in about a minute, without a GPU. |
| Sep 2026 | Adaptation, Not Algorithm was accepted to Findings of AACL-IJCNLP 2026. Across five open-weight LLMs, we find that LoRA and full fine-tuning cause about the same black-box jailbreak degradation. |
| Jul 2026 | Our two SemEval 2026 system papers are out in the ACL Anthology: Task 6 on political response clarity and Task 8 on multi-turn RAG. I presented both virtually at ACL 2026. |
| May 2026 | Started as a Software Developer at Medical Informatics Engineering. I’m the sole developer of WorkWell Measure Studio, which computes CMS clinical quality measures from FHIR data. |
Selected research
- Adaptation, Not Algorithm (Findings of AACL-IJCNLP 2026). Across five open-weight model families, LoRA and full fine-tuning degrade black-box jailbreak safety by about the same amount. QLoRA is less predictable, and 4-bit quantization alone does little. [code and artifacts]
- Your Judge Is a Confound (JUDGe workshop at NeurIPS 2026, oral; AdvML-Frontiers × CoTMA at COLM 2026). Swapping the judge can change a safety conclusion as much as swapping the fine-tuning method. Even GPT-5.6 judges flagged 21.5–29.9% of safe completions, and identical outputs sometimes got a different label when scored again.
- Two SemEval 2026 system papers in the ACL Anthology: multi-seed DeBERTa ensembles for political response clarity (Task 6) and tri-fusion retrieval for multi-turn RAG (Task 8, 6th of 26 on generation).
- In progress: a budget-aware black-box attack agent against hate-speech classifiers. The validity constraints these attacks are usually scored with measure edit size rather than whether the harmful content survived, so I’m calibrating survival-based judges against human-written contrast pairs.
See publications for abstracts, PDFs, and BibTeX.
Selected engineering work
- WorkWell Measure Studio (Medical Informatics Engineering, 2026–present). A TypeScript platform for WebChart that runs CMS’s own FHIR measure artifacts. All 455 official test cases across nine CMS measures pass in CI, and the sandbox runs 120,000 evaluations a night. Its AI tools are read-only and never decide compliance.
- RASS (Medical Informatics Engineering internship, 2025). A containerized RAG service for semantic search over 3,000+ Redmine issues, load-tested at 25 concurrent users with responses under 500 ms.
- JobOps Copilot (2026–present). A multi-tenant platform that runs stateful LangGraph agents with pgvector RAG and MCP tools, with tenant isolation, PII redaction, and CI quality gates.
- Riccle (2025–2026). At an early-stage e-commerce startup, I merged order, inventory, payment, and customer data from four trackers into one source and set up recurring KPI reports in SQL and Python, saving about six hours a week.