02 — Baseline: Lexical / Taxonomy Lookup Matching

The simplest possible approach to job-title matching: build a lookup table from ESCO's English titles (preferred + alternative labels), then for a query title, try (1) an exact normalized-string match, and (2) if that fails, a fuzzy string-similarity fallback.

This is deliberately the "dumb" baseline. Notebook 03 builds a multilingual embedding matcher that should beat it on paraphrases/synonyms; notebook 04 is where we actually measure that head-to-head, honestly, with real negatives. This notebook's job is just to build the lexical matcher and sanity-check it — the numbers here are exploratory, not the official evaluation.

Matcher logic lives in common.py (build_esco_en_candidates, lexical_match) so notebook 04 imports the exact same code rather than re-implementing it — avoiding silent drift between "what we built" and "what we evaluated."

import random

import matplotlib.pyplot as plt
import polars as pl
from rapidfuzz import fuzz, process

from common import build_esco_en_candidates, lexical_match, load_esco, load_onet, normalize, onet_all_titles

BLUE = "#2a78d6"
ORANGE = "#eb6834"
INK = "#0b0b0b"
SECONDARY_INK = "#52514e"
MUTED = "#898781"
GRID = "#e1e0d9"
SURFACE = "#fcfcfb"

plt.rcParams.update({
    "figure.facecolor": SURFACE, "axes.facecolor": SURFACE, "axes.edgecolor": GRID,
    "axes.labelcolor": SECONDARY_INK, "text.color": INK, "xtick.color": SECONDARY_INK,
    "ytick.color": SECONDARY_INK, "axes.grid": True, "grid.color": GRID,
    "grid.linewidth": 0.8, "font.size": 10,
})

RNG_SEED = 42

1. Build the ESCO English lexicon

We restrict this baseline to English (en + en-us ESCO labels) because our independent eval source, O*NET, is English-only — comparing English-to-English is the fairest apples-to-apples test of lexical matching specifically. (Cross-lingual matching is exactly what notebook 03's embedding approach is for — a lexical/fuzzy matcher has no real way to bridge languages at all, which is itself a finding worth stating plainly.)

esco_occ, esco_titles = load_esco()
onet_occ, onet_titles = load_onet()

candidates = build_esco_en_candidates(esco_titles)
n_dropped = candidates["n_dropped_ambiguous_norms"][0]
print(f"ESCO EN candidate lexicon: {candidates.height} unique normalized titles "
      f"(covering {candidates['uri'].n_unique()} distinct ESCO occupations)")
print(f"Dropped {n_dropped} normalized titles that were ambiguous (mapped to >1 ESCO "
      f"occupation) — we can't use those as unambiguous ground truth, so we exclude them "
      f"from the lexicon entirely rather than guessing which occupation they mean.")

exact_lookup = dict(zip(candidates["norm"].to_list(), candidates["uri"].to_list()))
choices = candidates["norm"].to_list()
uri_by_choice = exact_lookup  # same mapping, just named for clarity at the call site

ESCO EN candidate lexicon: 16919 unique normalized titles (covering 1695 distinct ESCO occupations)
Dropped 490 normalized titles that were ambiguous (mapped to >1 ESCO occupation) — we can't use those as unambiguous ground truth, so we exclude them from the lexicon entirely rather than guessing which occupation they mean.

2. A few concrete examples

Real O*NET titles, picked to show the three outcomes: a clean exact match, a fuzzy match that needed the fallback, and a genuine miss.

onet_all = onet_all_titles(onet_occ, onet_titles)
examples = ["Software Developer", "Marketing Manager, Digital", "Death Investigator",
            "Certified Wellness Program Coordinator", "MDI (Medicolegal Death Investigator)",
            "Penetration Testers"]
for ex in examples:
    n = normalize(ex)
    result = lexical_match(n, exact_lookup, choices, uri_by_choice, threshold=85)
    print(f"{ex!r:55s} -> matched={result['matched']!s:5} method={result['method']:6} "
          f"score={result['score']:5.1f}  matched_title={result['matched_title']!r}")

'Software Developer'                                    -> matched=True  method=exact  score=100.0  matched_title='software developer'
'Marketing Manager, Digital'                            -> matched=False method=none   score=  0.0  matched_title=None
'Death Investigator'                                    -> matched=False method=none   score=  0.0  matched_title=None
'Certified Wellness Program Coordinator'                -> matched=False method=none   score=  0.0  matched_title=None
'MDI (Medicolegal Death Investigator)'                  -> matched=False method=none   score=  0.0  matched_title=None
'Penetration Testers'                                   -> matched=False method=none   score=  0.0  matched_title=None

The pattern that shows up immediately: titles that are already a plain English occupation name ("Software Developer") tend to exact-match, because ESCO's own English lexicon contains that exact string somewhere among its 1699 occupations' preferred/alt labels. Real self-reported titles with extra qualifiers, abbreviations, or company-specific phrasing tend to need the fuzzy fallback or don't match at all — that's the actual matching problem this repo exists to study.

3. Picking a fuzzy-match threshold

rapidfuzz.fuzz.token_sort_ratio is order-invariant (sorts tokens before comparing), which matters because titles frequently reorder words across phrasings (e.g. "Manager, Marketing" vs "Marketing Manager"). We need a similarity cutoff below which we refuse to call it a match — otherwise every query gets "matched" to something, however unrelated, which would make the baseline look artificially complete instead of admitting when it doesn't know.

To choose the threshold, we sample a fixed set of O*NET titles (independent of ESCO), look at the distribution of each one's best fuzzy score against the ESCO lexicon, and look for a natural gap between "these are clearly the same job, just reworded" and "these just happen to share a few words."

random.seed(RNG_SEED)
onet_norm_unique = onet_all.with_columns(
    pl.col("title").map_elements(normalize, return_dtype=pl.Utf8).alias("norm")
)["norm"].unique().to_list()
sample_for_threshold = random.sample(onet_norm_unique, min(1500, len(onet_norm_unique)))
print(f"Sampling {len(sample_for_threshold)} of {len(onet_norm_unique)} unique normalized "
      f"O*NET titles (fixed seed={RNG_SEED}) to explore the fuzzy-score distribution — "
      f"this is a subsample for speed, not the full O*NET title set.")

best_scores = []
for q in sample_for_threshold:
    if q in exact_lookup:
        best_scores.append(100.0)
        continue
    hit = process.extractOne(q, choices, scorer=fuzz.token_sort_ratio)
    best_scores.append(hit[1] if hit else 0.0)

fig, ax = plt.subplots(figsize=(9, 4))
ax.hist(best_scores, bins=40, color=BLUE, edgecolor=SURFACE, linewidth=0.4)
ax.axvline(85, color=ORANGE, linewidth=1.5, linestyle="--", label="threshold = 85")
ax.set_xlabel("Best fuzzy score (token_sort_ratio, 0-100) against ESCO EN lexicon")
ax.set_ylabel("# O*NET titles (sampled)")
ax.set_title("Distribution of best-match score — picking a fuzzy-match threshold", loc="left", fontweight="bold")
ax.legend(frameon=False)
ax.spines[["top", "right"]].set_visible(False)
plt.tight_layout()
plt.savefig("../results/lexical_fuzzy_score_distribution.png", dpi=110)
plt.show()

Sampling 1500 of 47580 unique normalized O*NET titles (fixed seed=42) to explore the fuzzy-score distribution — this is a subsample for speed, not the full O*NET title set.

There's no crisp bimodal gap here — it's a fairly smooth distribution, which is itself informative: there's no score cutoff that cleanly separates "true match" from "coincidental overlap" for fuzzy string similarity on job titles. We pick 85/100 as the threshold: high enough that it requires almost all words to line up, used consistently for the rest of this notebook and in notebook 04's official evaluation. We are not tuning this against the eval set in notebook 04 — it's picked here, on this separate exploratory sample, and then held fixed.

4. Exploratory coverage over a broader O*NET sample

This is not the official evaluation (no ground truth, no negatives, no dedup-before-split rigor — that's notebook 04). It's a quick descriptive check: of a broad sample of real O*NET titles, what fraction get any match from this lexical approach at all, ignoring for now whether the match is correct.

random.seed(RNG_SEED)
coverage_sample = random.sample(onet_norm_unique, min(3000, len(onet_norm_unique)))
methods = []
for q in coverage_sample:
    r = lexical_match(q, exact_lookup, choices, uri_by_choice, threshold=85)
    methods.append(r["method"] if r["matched"] else "none")

method_counts = pl.DataFrame({"method": methods}).group_by("method").agg(pl.len().alias("n"))
method_counts = method_counts.with_columns((pl.col("n") / len(methods) * 100).round(1).alias("pct"))
print(f"Sample size: {len(methods)} unique normalized O*NET titles (subsample, seed={RNG_SEED})")
method_counts.sort("n", descending=True)

Sample size: 3000 unique normalized O*NET titles (subsample, seed=42)

shape: (3, 3)

method	n	pct
str	u32	f64
"none"	2414	80.5
"fuzzy"	416	13.9
"exact"	170	5.7

Summary

• The lexical/fuzzy matcher is a real, working baseline: exact match handles the (surprisingly common) case where an O*NET title happens to literally be one of ESCO's English labels; fuzzy match extends that to close variants.
• It has an obvious, structural ceiling: it cannot bridge languages (English query vs. French ESCO label = near-zero token overlap, whatever the threshold), and it cannot recognize a true synonym that doesn't share surface tokens (e.g. "Coder" vs. "Software Developer").
• The numbers above are exploratory (no ground truth). Notebook 04 evaluates this same matcher against an honest, deduped, negative-inclusive held-out set and reports precision/recall/F1 for real.