Participant Results
Final results of the 2026 evaluation: the systems submitted via CodaBench and, for Task 1, further systems that the organisers ran and scored with the same CodaBench scoring program. The reference systems published before the competition are on the baselines page.
Task 1 — global alignment
Submitted and organiser-run systems
| # | System | Team | Micro-averaged, R⁺≈ (headline) | Micro-F1, R≈ | Source | Date | ||
|---|---|---|---|---|---|---|---|---|
| P | R | F1 | ||||||
| 1 | LogMapLLMFamily used to build the referenceLogMapLLM belongs to the LogMap family of matchers, one of the families whose consensus votes produced the partial reference alignments (before manual review and repair). Bear this in mind when comparing its scores with those of other systems. | diso-oaei | 0.967 | 0.791 | 0.870 | 0.816 | CodaBench | 2026-09-25 |
| 2 | LogMapFamily used to build the referenceLogMap belongs to the LogMap family of matchers, one of the families whose consensus votes produced the partial reference alignments (before manual review and repair). Bear this in mind when comparing its scores with those of other systems. | oaei-ml | 0.959 | 0.777 | 0.858 | 0.791 | CodaBench | 2026-09-25 |
| 3 | LogMapKGFamily used to build the referenceLogMapKG belongs to the LogMap family of matchers, one of the families whose consensus votes produced the partial reference alignments (before manual review and repair). Bear this in mind when comparing its scores with those of other systems. | — | 0.959 | 0.773 | 0.856 | 0.787 | Organiser run | 2026-10-01 |
| 4 | MOSAIC-cli | — | 0.831 | 0.432 | 0.569 | 0.570 | Organiser run | 2026-10-01 |
| 5 | MatchaFamily used to build the referenceMatcha belongs to the Matcha family of matchers, one of the families whose consensus votes produced the partial reference alignments (before manual review and repair). Bear this in mind when comparing its scores with those of other systems. | pedrocotovio | 0.586 | 0.518 | 0.550 | 0.509 | CodaBench | 2026-09-24 |
| 6 | LogMapLtFamily used to build the referenceLogMapLt belongs to the LogMap family of matchers, one of the families whose consensus votes produced the partial reference alignments (before manual review and repair). Bear this in mind when comparing its scores with those of other systems. | — | 0.778 | 0.424 | 0.549 | 0.645 | Organiser run | 2026-10-01 |
| 7 | Matcha_legacyFamily used to build the referenceMatcha_legacy belongs to the Matcha family of matchers, one of the families whose consensus votes produced the partial reference alignments (before manual review and repair). Bear this in mind when comparing its scores with those of other systems. | martasilva | 0.688 | 0.447 | 0.542 | 0.561 | CodaBench | 2026-09-29 |
| 8 | CHARM-localLLM | — | 0.736 | 0.412 | 0.528 | 0.532 | Organiser run | 2026-10-03 |
| 9 | OLaLaForOAEI | — | 0.797 | 0.385 | 0.519 | 0.495 | Organiser run | 2026-10-02 |
| 10 | SECEA | — | 0.642 | 0.297 | 0.406 | 0.384 | Organiser run | 2026-10-01 |
| 11 | LSMatch-sortedcp | — | 0.592 | 0.152 | 0.241 | 0.230 | Organiser run | 2026-10-02 |
| 12 | ParisMatcher | — | 0.496 | 0.121 | 0.194 | 0.191 | Organiser run | 2026-09-30 |
| 13 | TIM-sortedcp | — | 0.974 | 0.078 | 0.144 | 0.117 | Organiser run | 2026-10-01 |
P, R and F1 are micro-averaged over the six pairs against the repaired reference R⁺≈ (the headline); the last score column is micro-F1 against the unrepaired R≈. Date: submission date (CodaBench) or run date (organiser run). Columns sort on click.
Info icon: the system belongs to a matcher family (LogMap or Matcha) whose consensus votes produced the partial reference alignments; bear this in mind when reading the table. Hover over, focus or tap the icon for details.
Organiser run: not a CodaBench submission. The organisers ran the system on the six pairs (through MELT unless noted below) and scored its alignments with the unmodified CodaBench Task 1 scoring program. A suffix in a system's name marks a run that differs from the system as registered for OAEI 2026.
Notes on the organiser-run systems
- LogMapKG: The official OAEI 2026 LogMap package with its Knowledge Graph track parameters, run through MELT Web.
- MOSAIC-cli: Run through its own command-line script, because no MELT package has been published for it; not run through MELT.
- LogMapLt: The official OAEI 2026 LogMapLt package, run through MELT Web.
- CHARM-localLLM: CHARM with local open-weight models in place of the SAP AI Core models it was registered with: Mistral-Small-3.2-24B-Instruct for chat and Qwen3-Embedding-8B for embeddings.
- OLaLaForOAEI: OLaLa (an OAEI 2023 participant) as packaged in MELT, run unchanged with its built-in Llama-2-70B model loaded in 8-bit; not an OAEI 2026 participant.
- SECEA: The submitted package, run through MELT Web.
- LSMatch-sortedcp: The submitted LSMatch 2.0 image, with its library directory listed in alphabetical order: as submitted, it fails on every pair with a servlet-API class-path clash. Its Java payload ran unchanged on an arm64 recreation of the image.
- ParisMatcher: MELT's wrapper around PARIS; not an OAEI 2026 participant. PARIS is not deterministic: over seven runs its micro-F1 (R⁺≈) ranged from 0.165 to 0.202; the scored run is shown.
- TIM-sortedcp: The submitted image, with its library directory listed in alphabetical order: as built, it fails on every pair with a servlet-API class-path clash. It completes jc3iedm-facility only; the other five pairs fail inside TIM and score as empty alignments.
Task 2 — local equivalence ranking
Submitted systems
| # | System | Team | MRR | Hits@1 | Hits@3 | Hits@5 | Hits@10 | Submitted |
|---|---|---|---|---|---|---|---|---|
| 1 | Matcha | pedrocotovio | 0.725 | 0.597 | 0.834 | 0.848 | 0.871 | 2026-09-24 |
CodaBench scores are a snapshot of the CodaBench leaderboards taken on 2026-10-03, after the evaluation phase closed. Task 1 is headlined by micro-F1 against R⁺≈; Task 2 by macro-MRR. See evaluation metrics. Machine-readable: CodaBench snapshot and organiser runs.
Participate
How to submit
Work through the quickstart: download the data, produce one alignment RDF per Task 1 pair and/or one ranking JSONL per Task 2 pair, validate them with the provided scripts, and submit via CodaBench — Task 1 · Task 2. Results for previous editions will be archived on the baselines page as the track matures.