Methodology · v1 · 2026-05-04
A 450M-parameter vision-language model, fine-tuned on the crisis, beats its 200-billion-parameter teacher on five of ten fields.
AquaVeritas fine-tunes LFM2.5-VL-450M on 1,280 expert-annotated Sentinel-2 observations across twenty global freshwater bodies. The result is evaluated three ways — against the unmodified base model, the Claude Opus oracle that generated the training labels, and the fine-tune itself — on a held-out test split.
Base 18% → Fine-tuned 85%.
Within touching distance of Claude Opus (86%) at 444× fewer parameters.
Water clarity, shoreline encroachment, flood risk, crop stress type, cultivation expansion.
Headline comparison
All three models on the same thirty held-out tiles.
| Model | Params | Overall | Core zone (4) | Buffer zone (6) |
|---|---|---|---|---|
Claude Opus oracle | ~200B | 86% | 86% | 87% |
LFM2.5-VL-450M base | 450M | 18% | 15% | 20% |
AquaVeritas fine-tuned | 450M | 85% | 90% | 82% |
The fine-tuned model trails Claude by 0.9pp overall but beats it on the core zone (90% vs 86%) — the four fields describing the water body itself, where the model has the most visual signal to work with.
Approach
Why full weight fine-tuning, not LoRA.
LFM2.5-VL was pretrained on natural images — photographs of objects, scenes, faces. Sentinel-2 multispectral imagery is far outside that distribution. Sub-pixel water bodies. SWIR false colour. Hyperspectral signatures of cropland under drought. None of it looks like ImageNet.
LoRA freezes the multimodal projector. For our domain that is the wrong layer to freeze — the projector is exactly where satellite spectral signatures need to be remapped to meaningful visual tokens. Full weight fine-tuning was necessary to move the accuracy meaningfully.
The +67.4pp uplift validates that choice. A LoRA control is on the V2 roadmap to quantify the gap.
Training
Modal H100, 2.5 hours, final loss 0.011.
- Base model
- LiquidAI/LFM2.5-VL-450M
- Fine-tune type
- Full weights
- Epochs
- 3
- Learning rate
- 2.0e-5 · cosine · 5% warmup
- Effective batch
- 8 (2 × 4 grad accum)
- Precision
- bfloat16
- Compute
- Modal H100 · ~2.5 h
- Final training loss
- 0.011
- Framework
- leap-finetune
Dataset
1,656 captures, 376 discarded, 1,280 labelled.
01
Capture
DPhi SimSat fetches Sentinel-2 tiles for twenty monitored sites across 84 months of history. RGB + SWIR, 15 km core and 30 km buffer zone per observation.
02
Triage
Tiles with ≥65% cloud cover or featureless open water are rejected. 1,656 raw observations across collection runs reduced to 1,280 surviving quality triage.
03
Annotate
Claude Opus (claude-opus-4-5) labels every curated observation as the oracle teacher. Eleven structured fields per observation. Same model is also used for the oracle baseline at evaluation time — a known caveat (see Caveats below).
04
Split
Train: observations before 2024-01-01 (1,250 obs). Test: from 2024-01-01 onwards (30 obs). Time-based split so the model never sees a tile from the same site at a similar season during training.
Per-field accuracy
Every field broken out. The wins and the losses.
Accuracy is the fraction of observations where the model’s prediction exactly matches the oracle ground-truth label. All thirty observations evaluated across all ten fields for all three models.
Core zone — the water body
| Field | Claude | Base LFM | Fine-tuned | Δ vs base |
|---|---|---|---|---|
Water extent shrinking · stable · flooded · recovering · dry | 90% | 3% | 87% | +83.4 |
Flood risk none · elevated · active | 77% | 30% | 80%beats oracle | +50.0 |
Water clarity clear · turbid · heavily silted | 93% | 13% | 97%beats oracle | +83.4 |
Shoreline encroachment yes / no | 83% | 13% | 97%beats oracle | +83.4 |
Buffer zone — surrounding land
| Field | Claude | Base LFM | Fine-tuned | Δ vs base |
|---|---|---|---|---|
Agriculture present yes / no | 87% | 30% | 67% | +36.7 |
Crop stress level none · low · moderate · severe | 77% | 13% | 57% | +43.4 |
Crop stress type drought · flood damage · none | 70% | 0% | 83%beats oracle | +83.3 |
Cultivation → water yes / no | 93% | 17% | 97%beats oracle | +80.0 |
Settlement visible yes / no | 97% | 30% | 93% | +63.3 |
Bare soil expansion yes / no | 97% | 30% | 97% | +66.7 |
Caveats
Where these numbers should be read with care.
Self-referential oracle
Claude generated the ground-truth labels and is also evaluated against them. Non-determinism is why the oracle scores 86.3% and not 100%. Human annotation of a small validation subset is on the V2 roadmap to provide an independent anchor.
Crop stress level — 56.7%
The hardest field. Distinguishing none / low / moderate / severe from a 15 km tile is inherently ambiguous; the oracle itself scores only 76.7%. The ceiling here may reflect a label-quality limit rather than a model limit.
Agriculture present regression
The fine-tune underperforms the oracle on this field (66.7% vs 86.7%). Likely cause: training labels for arid sites (Aral Sea, Dead Sea) mark agriculture_present=false across many months, biasing the model toward under-detection in similar test imagery.
Test set size
Thirty observations is small. Each field-level percentage has a ±9pp uncertainty at 95% confidence. The headline numbers are real but should not be over-interpreted at single-point precision.
Reproduce
Weights, dataset, training code — all public.
- Fine-tuned weights (GGUF)
- Arty1001/aquaveritas-lfm-GGUF
- Training dataset
- devleks/aquaveritas-water-stress
- Fine-tune config
- configs/aquaveritas_finetune_modal.yaml
- Evaluation script
- scripts/evaluate.py
- Comparison report
- data/reports/AquaVeritas_Eval_Report_2026-05-04.md