AutoCompile
Graduation Research · Team ProjectA compiler-optimization research system that fine-tunes LLMs to predict LLVM pass sequences from IR in a single forward pass, without iterative compiler feedback during inference.
AutoCompile addresses the cost of per-program compiler autotuning by amortising an offline LLVM pass-sequence search into a language model trained with supervised fine-tuning. The five-stage pipeline — size-biased collection from IR-OptSet, bounded autotuner label generation (budget B=100), IR-fingerprint deduplication, stratified 80/10/10 splitting, and LoRA fine-tuning — produces a policy whose distribution contains pass sequences that match or exceed the offline teacher on virtually every evaluated program. Five controlled experiment variants (model-size ablation, pretraining-prior ablation, GRPO reinforcement learning, and win-focused conditioning) localise the deployable gap as a decoding-routing problem rather than a capacity limitation, and a feature-discriminability probe provides a mechanistic explanation for greedy template collapse.
Team & My Role
My Role: Co-developer and research co-author with shared responsibility across implementation, dataset preparation, experimentation, analysis, documentation, and presentation.
AutoCompile was developed collaboratively, with responsibilities shared across the research pipeline rather than assigned as isolated individual components.
- Size-biased collection filters IR-OptSet to 3,000 programs with post-Oz instruction counts in [300, 10,000], targeting the large-function tail where optimization headroom is measurable (66% winner fraction).
- Bounded autotuner generates teacher labels with budget B=100 opt evaluations per program, using a three-phase search: seed evaluation, random exploration, and hill climbing with mutation operators.
- IR-fingerprint deduplication collapses exact-IR duplicates via SHA-1 hashing (after stripping comments, blanks, and whitespace normalisation) before splitting, with zero cross-split overlaps verified.
- Stratified train/val/test split (80/10/10) allocates per OverOz bucket (baseline, small_win, medium_win, large_win) using the largest-remainder method to preserve headroom distribution.
- LoRA fine-tuning (rank 16, α=16) on all seven projection matrices with response-only loss masking, early stopping with patience 3, and hard-fail guards on chat-template delimiters.
- A five-stage offline SFT pipeline amortises a bounded per-program autotuner search (B=100) into a single forward-pass language model, eliminating inference-time compiler feedback.
- IR-fingerprint deduplication using SHA-1 after whitespace normalisation, combined with stratified OverOz-bucket splitting, achieved zero cross-split leakage across 3,000 programs.
- A feature-discriminability probe diagnosed greedy template collapse as a routing failure, with a random-forest classifier on 12 structural features reaching only 0.30 balanced accuracy for template prediction.
- Controlled experiments showed that the N=1 to best-of-N gap persists across backbone scale, pretraining domain, GRPO reinforcement learning, and conditioning SFT.
- A five-category reporting framework separates raw performance, guarded score, teacher reference, sampled capability, and correctness-verified results.
- Response-only loss masking with hard-fail template guards concentrates training signal on the pass list rather than the much larger LLVM IR prefix.
- Full five-stage pipeline: size-biased collection, bounded autotuner, IR-fingerprint deduplication, stratified splitting, and LoRA fine-tuning.
- Five controlled experiment variants: model-size ablation (3B/7B/14B), pretraining-prior ablation (Meta LLM Compiler), GRPO RL, and win-focused conditioning.
- Best-of-40 inference exceeds the +3.92% teacher ceiling at +4.08% on the held-out test set (n=303), with 47 programs beating their individual teacher label.
- Mechanistic explanation of greedy template collapse via Phase-0 feature-discriminability probe and input-information ceiling analysis.
- Research paper with fourfold contribution: pipeline, gap localisation, collapse diagnosis, and five-category reporting framework.