Invariant pretraining: making code clone detection robust to equivalent rewrites

What happened on this date

At 21:01 UTC on 2026-08-15, arXiv received v1 of "Invariant Pretraining for Robust Code Representations" (primary category cs.LG), with a comment that it is to appear in LMPL 2026. The authors note that encoder-based code representation models are still widely used for clone detection and code classification because they are small and cheap to run, but they are fragile: on semantically equivalent programs written in different syntactic forms, the learned representations degrade substantially. The paper measures this gap across four encoder baselines, two downstream tasks and four datasets, and proposes InvPT, a code-only continued pretraining recipe that applies semantics-preserving transformations to the corpus and combines masked language modeling with multi-positive contrastive learning, treating all transformed versions of a function as positives without needing paired natural-language data. The abstract reports robustness gains on transformed test sets of up to 11 points for clone detection and 19 for code classification, while matching or improving standard accuracy.

Why it matters for people protecting app logic

Semantics-preserving rewriting is the cheapest way to disguise copied code, and having an LLM rewrite code now costs almost nothing. The paper is useful in two directions. If you want to find copies of your own code, it shows that many off-the-shelf clone detection models fail noticeably on rewritten code, so choosing a tool by its standard benchmark score is not enough. If you were counting on renaming and reordering to make code look different, it shows that robustness can be trained back in, so surface transformations may not hide identical logic for long. Note that the study concerns source-level encoder models, not binary analysis, and the gains come from the authors' datasets and transformation sets; the abstract makes no claim about other languages or more aggressive rewrites.

Concrete checks

First, when evaluating any clone detection tool, generate several semantically equivalent rewrites of your own code as a test set and measure how much recall drops. Second, do not treat identifier renaming, reformatting or statement reordering as ways to protect core logic; truly sensitive rules belong on the server. Third, keep commit history and release dates for your source as originality evidence, since detection results by themselves only indicate similarity. Fourth, follow robustness research on code representation models and periodically review whether the detection tools you use need updating. Record the tool versions and test sets you used so later comparisons stay meaningful.

Your checklist

• Test clone detectors with equivalent rewrites of your own code
• Do not rely on renaming or reformatting to protect core logic
• Keep commit history and release dates as originality evidence
• Periodically review the robustness of your detection tools