giorgio/MachineSex
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Manuscript revision and pending experiment work, snapshot before restructuring

Browse source 
Clarity pass over the main text (36-item audit), Discussion rewrite and cut,
acknowledgements, Souly et al. as ref 62, lettered SI panels, model section
moved under Results; plus the untracked curriculum/society/compose/smol
configs, runners, figures, stats and tests that the SI already cites.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y64o8FKP7rCuXzC48pxpMm
This commit is contained in:
Giorgio Gilestro 2026-09-13 16:54:09 +01:00
parent e4804adabc
commit 84124de143
450 changed files with 52813 additions and 1202 deletions
Download patch file Download diff file Expand all files Collapse all files

53
results/llm_curriculum_v5_decor/README.md Normal file
View file

@ -0,0 +1,53 @@
# llm_curriculum_v5_decor — a curriculum that decouples partner complementarity from generation
Config `configs/llm/curriculum_v5_decor.yaml`. Same population as v5 (three lineages, six real-dataset
families, Qwen2.5-1.5B, rank-16 adapters continued from the parent) but every lineage starts with
mnli, then diverges maximally, then converges, so mean pairwise complementarity by generation is
0.00, 0.67, 0.70, 0.58, 0.33, 0.00 (Latin square: 1.00, 1.00, 0.80, 0.67, 0.33, 0.00). Arms:
`isolated` (never merge) and `society` with `allow_veto: true` (declinable merge). Pre-registered in
`tasks/prereg-llm-society-v4.md` §8h before running. Seed 1 local; seeds 2–3 on CX3
(`hpc/llm_curriculum_controls.pbs`). Layout: `results.parquet` (seed 1), `s2/`, `s3/`.
Why: in the Latin square, complementarity and generation number are collinear, so the veto's
acceptance curve (Fig. 4B, Spearman ρ = −0.57 with complementarity) could equally be a response to
adapter age. Pooling both curricula breaks the collinearity.
## Result (`figures/stats_llm_curriculum.py`)
Fraction of proposed merges declined (mean of 3 seeds) against complementarity:
| generation | Latin: declined | Latin: compl. | decor: declined | decor: compl. |
|---|---|---|---|---|
| 0 | 0.44 | 1.00 | 0.44 | 0.00 |
| 1 | 0.56 | 1.00 | 0.44 | 0.67 |
| 2 | 0.44 | 0.80 | 0.67 | 0.70 |
| 3 | 0.78 | 0.67 | 0.67 | 0.58 |
| 4 | 0.67 | 0.33 | 0.44 | 0.33 |
| 5 | 1.00 | 0.00 | 0.89 | 0.00 |
Pooled test (2 curricula × 6 generations × 3 seeds = 36 points):
| quantity | value |
|---|---|
| partial Spearman ρ(declined, complementarity \| generation) | −0.067, seed-clustered 95% CI (−0.211, +0.088) |
| partial Spearman ρ(declined, generation \| complementarity) | +0.31 |
| raw ρ(declined, complementarity) | −0.314 |
| raw ρ(declined, generation) | +0.458 |
Final accuracy (best lineage, all six families, generation 5):
| arm | s1 | s2 | s3 | mean |
|---|---|---|---|---|
| decor, declinable merge | 0.789 | 0.792 | 0.789 | 0.790 |
| decor, never merge | 0.786 | 0.792 | 0.792 | 0.790 |
Pre-registered reading (§8h): the modifier hypothesis (partial ρ with complementarity < 0, CI
excluding 0) is **not supported**; the adapter-age hypothesis (partial ρ with complementarity ≈ 0,
with generation > 0) fits. The Latin-square correlation of −0.57 was carried by generation. What
rises with generation in both curricula and is not separated here: the adapters' training age, the
number of skills each holds, and the presence of the two format-destroyer families (boolq,
winogrande), which every lineage carries by the late generations in both schedules. The declinable
merge again finished level with never merging (−0.000 ± 0.003 over seeds).
Falsifier status: the manuscript's reading of the acceptance curve as a recombination modifier
tracking complementarity (the reduction principle) is withdrawn; the curve tracks generation.

74
results/llm_curriculum_v5_decor/manifest.json Normal file
View file

@ -0,0 +1,74 @@
{
"experiment": "llm_curriculum_v5_decor",
"master_seed": 1,
"git_commit": "e4804adabcdce6d928c5e6b1e85429b2a6acf2fe",
"python": "3.14.7",
"libraries": {
"numpy": "2.5.0",
"scipy": "1.18.0",
"pandas": "3.0.3",
"pyarrow": "24.0.0",
"torch": "2.12.1",
"transformers": "5.16.1",
"peft": "0.20.0"
},
"rows": 439,
"results_sha256": "19c07b8460eabea024ee8a16a6ed98be2c771450e9f9b23f71a1e4f9e2573f07",
"layer": "2",
"tier": "llm",
"base_model": "Qwen/Qwen2.5-1.5B",
"hard": false,
"curriculum": {
"families": [
"mnli",
"arc",
"hellaswag",
"squad",
"boolq",
"winogrande"
],
"lineages": 3,
"generations": 6,
"arms": [
"isolated",
"society"
],
"baselines": [],
"n_new": 300,
"n_replay": 150,
"operator": "linear",
"ancestor_depth": null,
"lora": {
"r": 16,
"alpha": 32
},
"allow_veto": true,
"merge_until": null,
"orders": [
[
"mnli",
"arc",
"hellaswag",
"squad",
"boolq",
"winogrande"
],
[
"mnli",
"squad",
"boolq",
"winogrande",
"arc",
"hellaswag"
],
[
"mnli",
"winogrande",
"arc",
"hellaswag",
"squad",
"boolq"
]
]
}
}

BIN
results/llm_curriculum_v5_decor/partial_isolated_s1.parquet Normal file
View file

Binary file not shown.

BIN
results/llm_curriculum_v5_decor/partial_society_s1.parquet Normal file
View file

Binary file not shown.

65
results/llm_curriculum_v5_decor/resolved_config.yaml Normal file
View file

@ -0,0 +1,65 @@
experiment: llm_curriculum_v5_decor
seed: 1
n_replicates: 1
source_config:
experiment: llm_curriculum_v5_decor
kind: llm_curriculum
base_model: Qwen/Qwen2.5-1.5B
seed: 1
families:
- mnli
- arc
- hellaswag
- squad
- boolq
- winogrande
orders:
- - mnli
- arc
- hellaswag
- squad
- boolq
- winogrande
- - mnli
- squad
- boolq
- winogrande
- arc
- hellaswag
- - mnli
- winogrande
- arc
- hellaswag
- squad
- boolq
lineages: 3
generations: 6
arms:
- isolated
- society
baselines: []
allow_veto: true
n_new: 300
n_replay: 150
n_test: 60
n_val: 20
epochs: 3
lr: 0.0001
operator: linear
merge_weights:
- - 0.5
- 0.5
- - 0.3
- 0.7
- - 0.7
- 0.3
max_new_tokens: 48
batch_size: 24
train_batch_size: 2
train_max_len: 512
lora:
r: 16
alpha: 32
output:
dir: results/llm_curriculum_v5_decor
n_replicates: 1

74
results/llm_curriculum_v5_decor/s2/manifest.json Normal file
View file

@ -0,0 +1,74 @@
{
"experiment": "llm_curriculum_v5_decor",
"master_seed": 2,
"git_commit": null,
"python": "3.11.7",
"libraries": {
"numpy": "2.4.6",
"scipy": "1.17.1",
"pandas": "3.0.3",
"pyarrow": "24.0.0",
"torch": "2.12.1",
"transformers": "5.16.1",
"peft": "0.20.0"
},
"rows": 439,
"results_sha256": "a75595b9f4eeef9d0ddb7533cdeec3f225906c835e66c0b2215e8885080cb66b",
"layer": "2",
"tier": "llm",
"base_model": "Qwen/Qwen2.5-1.5B",
"hard": false,
"curriculum": {
"families": [
"mnli",
"arc",
"hellaswag",
"squad",
"boolq",
"winogrande"
],
"lineages": 3,
"generations": 6,
"arms": [
"isolated",
"society"
],
"baselines": [],
"n_new": 300,
"n_replay": 150,
"operator": "linear",
"ancestor_depth": null,
"lora": {
"r": 16,
"alpha": 32
},
"allow_veto": true,
"merge_until": null,
"orders": [
[
"mnli",
"arc",
"hellaswag",
"squad",
"boolq",
"winogrande"
],
[
"mnli",
"squad",
"boolq",
"winogrande",
"arc",
"hellaswag"
],
[
"mnli",
"winogrande",
"arc",
"hellaswag",
"squad",
"boolq"
]
]
}
}

BIN
results/llm_curriculum_v5_decor/s2/partial_isolated_s2.parquet Normal file
View file

Binary file not shown.

BIN
results/llm_curriculum_v5_decor/s2/partial_society_s2.parquet Normal file
View file

Binary file not shown.

65
results/llm_curriculum_v5_decor/s2/resolved_config.yaml Normal file
View file

@ -0,0 +1,65 @@
experiment: llm_curriculum_v5_decor
seed: 2
n_replicates: 1
source_config:
experiment: llm_curriculum_v5_decor
kind: llm_curriculum
base_model: Qwen/Qwen2.5-1.5B
seed: 2
families:
- mnli
- arc
- hellaswag
- squad
- boolq
- winogrande
orders:
- - mnli
- arc
- hellaswag
- squad
- boolq
- winogrande
- - mnli
- squad
- boolq
- winogrande
- arc
- hellaswag
- - mnli
- winogrande
- arc
- hellaswag
- squad
- boolq
lineages: 3
generations: 6
arms:
- isolated
- society
baselines: []
allow_veto: true
n_new: 300
n_replay: 150
n_test: 60
n_val: 20
epochs: 3
lr: 0.0001
operator: linear
merge_weights:
- - 0.5
- 0.5
- - 0.3
- 0.7
- - 0.7
- 0.3
max_new_tokens: 48
batch_size: 48
train_batch_size: 4
train_max_len: 512
lora:
r: 16
alpha: 32
output:
dir: results/llm_curriculum_v5_decor/s2
n_replicates: 1

74
results/llm_curriculum_v5_decor/s3/manifest.json Normal file
View file

@ -0,0 +1,74 @@
{
"experiment": "llm_curriculum_v5_decor",
"master_seed": 3,
"git_commit": null,
"python": "3.11.7",
"libraries": {
"numpy": "2.4.6",
"scipy": "1.17.1",
"pandas": "3.0.3",
"pyarrow": "24.0.0",
"torch": "2.12.1",
"transformers": "5.16.1",
"peft": "0.20.0"
},
"rows": 439,
"results_sha256": "e1edd757be0e27f489b5835e64fd7103f08b8f4d27482e00e751b4a43e6f8e6e",
"layer": "2",
"tier": "llm",
"base_model": "Qwen/Qwen2.5-1.5B",
"hard": false,
"curriculum": {
"families": [
"mnli",
"arc",
"hellaswag",
"squad",
"boolq",
"winogrande"
],
"lineages": 3,
"generations": 6,
"arms": [
"isolated",
"society"
],
"baselines": [],
"n_new": 300,
"n_replay": 150,
"operator": "linear",
"ancestor_depth": null,
"lora": {
"r": 16,
"alpha": 32
},
"allow_veto": true,
"merge_until": null,
"orders": [
[
"mnli",
"arc",
"hellaswag",
"squad",
"boolq",
"winogrande"
],
[
"mnli",
"squad",
"boolq",
"winogrande",
"arc",
"hellaswag"
],
[
"mnli",
"winogrande",
"arc",
"hellaswag",
"squad",
"boolq"
]
]
}
}

BIN
results/llm_curriculum_v5_decor/s3/partial_isolated_s3.parquet Normal file
View file

Binary file not shown.

BIN
results/llm_curriculum_v5_decor/s3/partial_society_s3.parquet Normal file
View file

Binary file not shown.

65
results/llm_curriculum_v5_decor/s3/resolved_config.yaml Normal file
View file

@ -0,0 +1,65 @@
experiment: llm_curriculum_v5_decor
seed: 3
n_replicates: 1
source_config:
experiment: llm_curriculum_v5_decor
kind: llm_curriculum
base_model: Qwen/Qwen2.5-1.5B
seed: 3
families:
- mnli
- arc
- hellaswag
- squad
- boolq
- winogrande
orders:
- - mnli
- arc
- hellaswag
- squad
- boolq
- winogrande
- - mnli
- squad
- boolq
- winogrande
- arc
- hellaswag
- - mnli
- winogrande
- arc
- hellaswag
- squad
- boolq
lineages: 3
generations: 6
arms:
- isolated
- society
baselines: []
allow_veto: true
n_new: 300
n_replay: 150
n_test: 60
n_val: 20
epochs: 3
lr: 0.0001
operator: linear
merge_weights:
- - 0.5
- 0.5
- - 0.3
- 0.7
- - 0.7
- 0.3
max_new_tokens: 48
batch_size: 48
train_batch_size: 4
train_max_len: 512
lora:
r: 16
alpha: 32
output:
dir: results/llm_curriculum_v5_decor/s3
n_replicates: 1