Isaac Kobby Anni

Trustworthy evaluation of LM-driven AutoML

• Evaluation framework that scores generated actions on six criteria: executability, incremental predictive value, stability, evaluator generality, robustness, and decision cost.
• Leakage-safe matched baseline–candidate experiments across 19 datasets using cross-validation, paired bootstrap analysis, and held-out evaluation.
• Evidence-tracking and output-validation workflows that make each keep/repair/reject decision reproducible. Code available on request.

Finding: none of the 14 generated features that passed cross-validation gates were confirmed on held-out data, and 6 of them (43%) caused measurable harm. Passing cross-validation was not enough evidence to trust a generated feature.

Language-model-driven automated feature engineering

• Workflow in which language models propose executable features that must pass controlled-execution, target-exclusion, leakage, and predictive-value checks.
• Specialist components for numerical, temporal, and categorical features, evaluated on 14 public classification datasets.

Result: improved TabPFN on 11 of 14 datasets (mean ROC-AUC gain +0.0038) and random forests on 12 of 14 (mean gain +0.0111).

Automated data cleaning and leakage detection

• Constrained proposal-and-validation pipeline for model-suggested cleaning actions, covering downstream evaluation, leakage prevention, and failure handling.

Result: detected train/eval duplicate leakage in 2 of 5 benchmark datasets, with up to 20% overlap between evaluation and training rows.

Human–AI collaboration

• Research on trust calibration, reliance, human oversight, and risks arising from AI-assisted decision-making.
• Co-author of a lifecycle taxonomy of sociotechnical risks in human–AI collaboration (arXiv:2608.05614).