llm_merge_hpc: the 7B firm-up makes the Fisher-Muller sign decisive
Re-ran the specialist-merge experiment at a capable base (Qwen2.5-7B-Instruct, 200 tests/family) on one L40S GPU of Imperial's CX3 HPC (8 min walltime). The two caveats the 0.5B prototype left marginal are now resolved: - "exceeds every parent overall" is clean: both merges 0.87 vs best specialist 0.77 (+10 pts), and above every specialist on every family. - dilution vanishes: at 0.5B averaging diluted the lists-specialist (0.43->0.26); at 7B the merge beats it (0.62>0.57). Dilution was a small-model artefact -- a capable base composes rather than dilutes, which softens E4's "merge, don't average" once the parents are strong. The figure title is now data-driven (reports ">" for 7B, "~" for 0.5B). Adds the hpc/ smoke job script and the llm_merge walltime trim. Results synced to results/llm_merge_hpc/ (parquet gitignored per the reproducibility contract). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
38bb252c20
commit
585264d0b4
12 changed files with 140 additions and 22 deletions
14
hpc/llm_smoke.pbs
Normal file
14
hpc/llm_smoke.pbs
Normal file
|
|
@ -0,0 +1,14 @@
|
|||
#!/bin/bash
|
||||
# Smoke gate: run the small 0.5B merge experiment on an L40S to confirm the GPU pipeline works
|
||||
# on CX3 before the 4 h 7B run. ~5 min. submit: qsub hpc/llm_smoke.pbs
|
||||
#PBS -l select=1:ncpus=8:mem=48gb:ngpus=1:gpu_type=L40S
|
||||
#PBS -l walltime=00:20:00
|
||||
#PBS -N lam_llm_smoke
|
||||
|
||||
cd "$PBS_O_WORKDIR"
|
||||
export HF_HOME="$EPHEMERAL/hf_cache"
|
||||
export TOKENIZERS_PARALLELISM=false
|
||||
source .venv/bin/activate
|
||||
nvidia-smi --query-gpu=name,memory.total,driver_version --format=csv,noheader
|
||||
python -m llm.experiment configs/llm/merge.yaml # 0.5B, writes results/llm_merge/
|
||||
echo "done: $(date)"
|
||||
Loading…
Add table
Add a link
Reference in a new issue