Scaling Author Identity Disambiguation to the World of Code: A Methodology

Source: arXiv:2607.06920 · Published 2026-07-08 · By Audris Mockus

TL;DR

This paper addresses the problem of author identity disambiguation (de-aliasing) at unprecedented scale—resolving roughly 107 million distinct author strings from approximately 6 billion commits across the World of Code (WoC) dataset. Prior work such as ALFAA and Fry et al. resolved identities on tens of millions of author IDs, but scaling to the entire WoC introduces new challenges, chiefly over-merge errors where unrelated persons are incorrectly grouped into mega-clusters due to bridge identities like bot accounts, placeholder emails, or multi-author commit fields. The authors document a comprehensive experimental campaign involving over twenty iterations exploring node-gating heuristics and learned edge classification to suppress these large erroneous clusters without sacrificing legitimate merges. The final production approach combines a betweenness-centrality based nodal graph cut, a classifier trained on 2.6 million GitHub no-reply ID labeled edges, and curated blocklists of problematic identity attributes. End to end, this shrinks the largest cluster from 170,431 to under 7,000 members while increasing gold-standard recall from 0.44 to 0.70 and maintaining high precision (~0.87 to 0.88). Independent evaluation on a 21M-alias GitHub ground truth confirms superior performance and robustness relative to prior state of the art. The work offers a unique deep experimental record and lessons about scale-specific identity resolution phenomena.

Key findings

• The unconditioned global union step results in mega-clusters of up to 170,431 members, causing severe over-merge errors.
• Node-level gating (information-score cutoff, project spread threshold, link degree threshold) individually reduce mega-clusters but cannot fully dissolve them without harming recall.
• A combined node-gate union (blocking identities with info-score cutoff ∼64, project spread ≥100, or degree ≥100) gates 42,110 nodes, reducing the largest cluster from 111,926 to 25,957 and preserving recall at the maximum two-member cluster count of 9.60M.
• A per-edge logistic regression classifier trained on 2.64 million GitHub no-reply ID labeled edges achieves 95.2% accuracy and AUC 0.956, effectively distinguishing true links from bridges.
• The classifier transfers well to human adjudicated labels with AUC of 0.99, matching ALFAA’s complex random forest without behavioral fingerprints.
• Structural graph cuts based on betweenness centrality and articulation points on the global union graph break the mega-cluster dramatically; cutting the top 2,000 betweenness nodes reduces cluster size from 170,431 to 7,268.
• Classifier-based pruning inside residual fragments further breaks them down, freeing ~46% of fragment IDs into small components ≤100 with minimal recall loss (~0.03%).
• Classifier-filtered expansion of dormant cross-project shingle groups at a threshold of 0.9 recalls 881K additional pairs, lifting recall from 0.44 to 0.70 while precision improves slightly to 0.88 and zero clusters exceed 10,000 members.

Threat model

The adversary includes automated bots, service/role accounts, placeholder emails, and multi-author commit strings that act as bridges linking distinct genuine author clusters, thus causing over-merge errors transitive through union operations. These identities are partly adversarial or unintended sources of ambiguity and may span multiple unrelated projects. The model assumes no ability to alter commit metadata upstream and no direct access to ground truth beyond mined public identity metadata. The defense mechanisms cannot prevent a malicious actor from deliberately creating many fake linked identities but aim to minimize propagation of such bridges in the absence of explicit labels.

Methodology — deep read

The authors tackle identity disambiguation over a massive dataset from the World of Code V2604 release, containing ~107 million distinct author strings sourced from ~6 billion commits. Their goal is to alias free-text identity strings (name, email, username, GitHub numeric ID) to canonical persons while minimizing over-merge errors that cause unrelated people to be grouped. The threat model involves adversarial identity strings such as role accounts, bots, placeholder emails, and multi-author commit fields that serve as bridges linking unrelated clusters. They assume no control over upstream identity data and no external ground truth beyond mined GitHub no-reply ID labels.

Data inputs include parsed identity fields split by author and project shards, totaling 49.76 million candidate links generated from within-project Phase-1 co-authorship/fingerprint linkage. Phase 2 globally unions these candidate pairs into clusters, selecting representatives with maximum information score based on rarity of component fields.

The core challenge is over-merge due to bridge identities that link distinct clusters. Initial node-level gating strategies include (a) information-score cutoffs to exclude low-information identities, (b) project-spread gating to exclude identities appearing across many deforked projects, and (c) link-graph degree gating excluding identities linked to many distinct partners. Extensive experimentation showed each gating strategy individually reduces mega-clusters but cannot fully dissolve them without sacrificing recall, and even combined gating cannot remove residual bridging by moderate-degree nodes.

To surpass node-gating limits, they develop a learned per-edge classifier using logistic regression trained on 2.64 million labels mined for free from GitHub no-reply email IDs indicating matching or conflicting numeric GitHub IDs. Features include endpoint degree, project spread, information scores, component matches, and rarity-weighted match scores. This classifier effectively differentiates true positive edges (same person) from false cross-cluster edges bridging distinct identities.

They complement classifier pruning with structural graph analyses: computing betweenness centrality and articulation points on the global union subgraph reveals bridge nodes whose removal breaks mega-clusters. Sampling top central nodes and gating them removes mega-clusters with minimal recall loss. Inside residual fragments, classifier-based edge pruning breaks homonym blocks.

Finally, they incorporate classifier-filtered expansion of cross-project shingle groups (rule-3 matches) preserving high-confidence edges previously missing from graph structures, recovering recall sacrificed by gating.

Training used logistic regression with roughly 300K training pairs and 2.3 million held-out test edges on standard computing clusters; no active labeling was needed. Evaluation included in-domain cross-validation against GitHub numeric ID labels, transfer to human adjudicated ALFAA gold, extensive ablation of gating heuristics, and external benchmarking on an independent 21M-alias GitHub ground truth. All maps are privacy-preserving, relying only on public commit metadata and no direct personal data.

A concrete example: a 170,431-member mega-cluster containing role bots and common placeholder emails is initially formed by transitive union. Node gating removes some bridges but leaves a Cloudflare relay email hash string welded cluster ⇒ the betweenness centrality computation identifies the top 2,000 bridge nodes which are then gated to shatter the mega cluster to 7,268. Residual fragments are pruned by classifier edges reducing largest fragment size further, resulting in no cluster > 7,000 members and recall gain from 0.44 to 0.70 using the full pipeline.

Technical innovations

• Identification and characterization of over-merge as the dominant scaling failure mode in global identity disambiguation, shifting the design focus from recall maximization to precision-centric bridge suppression.
• Development of a comprehensive multi-signal node gating strategy combining information score, project spread, and link degree thresholds tuned via cross-GitHub ID false link rates.
• Creation of a large-scale, zero-cost labeled dataset derived from mining GitHub no-reply numeric ID prefixes embedded in commit identities to supervise a per-edge logistic classifier for link precision.
• Use of graph structural metrics (betweenness centrality, articulation points) on the exact global union graph to identify and selectively gate bridge nodes dissolving mega-clusters efficiently.
• A composite identity resolution pipeline integrating gated union, learned edge classification, and classifier-filtered expansion of shingle groups, balancing precision and recall effectively at unprecedented scale.

Datasets

• World of Code V2604 — ~107 million distinct author strings, ~6 billion commits — public WoC corpus from open source repositories
• GitHub no-reply ID labels — 2.64 million labeled pairs mined from embedded numeric GitHub IDs in commit email strings — derived from public Github commit metadata
• ALFAA gold set — 469,369 human-adjudicated author identity pairs covering 2,345 IDs — public metadata from prior ALFAA study
• Bock et al. GitHub ground truth — 21 million alias handles owning ~21M single-author repository commits — independent published benchmark

Baselines vs proposed

• Ungated global union baseline: largest cluster size = 111,926 members vs proposed: ~7,000 members after gating and classifier pruning
• Information-score cutoff gating: dissolves mega-cluster at cutoff=64 but drops size-2 clusters from 9.60M to 2.14M (recall loss)
• Project-spread gating: reduces mega cluster size from 51,860 to 28,526 but requires gating many identities and cannot fully dissolve mega cluster
• Link-graph degree gating alone: largest cluster shrinks from 71,172 to 20,635 at degree threshold 50 but residual mega cluster remains >10k
• Combined gating (project spread ≥ 100 or degree ≥ 100): largest cluster drops from 111,926 to 25,957 while preserving all 9.60M size-2 clusters
• Per-edge logistic classifier: test AUC = 0.956, accuracy 95.2%, transfers to ALFAA human labels at AUC 0.987
• Betweenness structural cut gating (top 2,000 nodes): largest cluster size drops from 170,431 to 7,268 while recall remains above baseline
• Classifier-filtered shingle expansion at threshold 0.9: recall improves from 0.436 to 0.645 and precision rises from 0.873 to 0.882 without any >10k clusters
• On GitHub 21M-alias benchmark at max9 repo size, recall improves from 0.433 (V2604) to 0.589 (post expansion), exceeding predecessor maps and state of the art

Limitations

• Residual mega-clusters are caused by long tails of moderate-degree bridges invisible to node-level gates, requiring sophisticated per-edge pruning.
• Flat per-edge classifier thresholds cannot simultaneously dissolve mega clusters while preserving recall, necessitating structural graph cuts and iterative pruning.
• Model currently uses logistic regression; more complex models like gradient-boosted trees overfit or do not transfer well in out-of-distribution settings.
• The pipeline relies on mining GitHub no-reply numeric IDs embedded in emails, limiting applicability outside GitHub-sourced data or anonymized commits.
• Recall gains plateau due to limitations in the rules and feature sets, e.g., project co-occurrence and shingle groups inert for some missing merges.
• Evaluation bias: external benchmarks focus mostly on GitHub-based ground truth, which may not generalize to other ecosystems or anonymized identities.

Open questions / follow-ons

• How to extend de-aliasing methods to handle fully anonymized commits with no public ciphertext like GitHub no-reply IDs?
• Can more advanced graph neural network models overcome limitations of linear logistic classifiers for per-edge classification without overfitting?
• What strategies can ensure robust identity disambiguation in multi-repository, federated, or private code ecosystems where global union graphs are incomplete or unavailable?
• How to systematically integrate temporal and behavioral signals beyond static metadata for improving recall without sacrificing precision at scale?

Why it matters for bot defense

From a bot-defense and CAPTCHA engineering standpoint, this paper illustrates the challenges and methodologies for large-scale identity resolution in an adversarial, noisy metadata environment. Analogously, bot detection systems must disambiguate malicious actors who may use diverse but overlapping identifiers to avoid being flagged as unique adversaries. The approach of combining node-level heuristics, learned per-edge classifiers, and structural graph cuts can inspire multi-layered defenses to limit false merges that group distinct malicious and legitimate users, preserving accuracy while scaling to massive, real-world workloads. The insights on bridge entities—analogous to transient bot clusters or shared user proxies—highlight the necessity of fine-grained link-level discrimination and structural network analysis beyond thresholding heuristics. Furthermore, the work emphasizes the importance of combining complementary signals and verifying solutions against independent ground truth, a principle directly translatable to creating robust human verification systems that distinguish bots without penalizing diverse legitimate users.

Cite

@article{arxiv2607_06920,
  title={ Scaling Author Identity Disambiguation to the World of Code: A Methodology },
  author={ Audris Mockus },
  journal={arXiv preprint arXiv:2607.06920},
  year={ 2026 },
  url={https://arxiv.org/abs/2607.06920}
}

Read the full paper

• arXiv:2607.06920 (abstract)
• arXiv:2607.06920 (PDF)