Predicates¶

Return the set of Unicode scripts present in text, in order of first appearance.

Examples:

>>> detect_scripts("Hello")
[Script.LATIN]
>>> detect_scripts("Hello Мир")
[Script.LATIN, Script.CYRILLIC]

Inspect how lang="auto" would resolve for the given text.

Use this to audit or log the detection decision made by the three-stage auto-detection pipeline.

Examples:

>>> inspect_auto_lang("Київ")["chosen_lang"]
'uk'
>>> inspect_auto_lang("Москва")["reason"]
'script_default'

True if text contains characters from more than one Unicode writing system.

Resolves the UTS #39 §5.1 augmented script sets (#776), so a script pair that one writing system uses is not "mixed":

================================== ============================ Han + Hiragana + Katakana Japanese Han + Hangul Korean Han + Bopomofo Chinese ================================== ============================

So 日本語テスト is one writing system, not three scripts. Anything without a writing system in common is still mixed, including a CJK script beside a non-CJK one — 例えa is Japanese and Latin, and that is the case this check exists for.

Examples:

>>> is_mixed_script("Hello")
False
>>> is_mixed_script("Hello Мир")  # Latin + Cyrillic
True
>>> is_mixed_script("日本語テスト")  # Han + Katakana, one writing system
False
>>> is_mixed_script("ひら한")  # Japanese + Korean, no set in common
True

True if text mixes strong left-to-right and strong right-to-left characters.

This is the precondition for Unicode Bidi display-reordering (UAX #9) — the structural signal behind "BiDi Swap"-style spoofs, where an LTR brand label sits beside an RTL domain (e.g. "varonis.com.ו.קום"). Unlike a bidi-override (U+202x) check, it fires on the real letters: Latin / Cyrillic / Greek / CJK are left-to-right; Hebrew / Arabic / Syriac / Thaana / N'Ko are right-to-left; digits, punctuation and combining marks are neutral and never create a conflict on their own.

A False result is not a safety guarantee.

has_bidi_conflict("hello שלום")            True
inspect_anomalies("hello שלום").kinds      []
has_bidi_conflict("helloשלום")             True
inspect_anomalies("helloשלום").kinds       ['bidi_mixed']

Examples:

>>> has_bidi_conflict("hello")
False
>>> has_bidi_conflict("helloא")  # Latin + Hebrew
True
>>> has_bidi_conflict("hello שלום")  # whole string, so the space is no barrier
True
>>> inspect_anomalies("hello שלום").kinds  # per token, so it is two clean words
[]
>>> has_bidi_conflict("invoice\u202Egpj.exe")  # RLO override, not letters
False
>>> inspect_anomalies("invoice\u202Egpj.exe").kinds  # this is the check
['bidi']

True if text contains characters confusable with target-script characters.

Examples:

>>> is_confusable("pаypal")  # Cyrillic а looks like Latin a
True
>>> is_confusable("paypal")  # all genuine Latin
False

Every upstream confusable source disarm's bundled table does not fold (#563).

Read this as exposure, not as a score. A tool at 95% per-source coverage is not 95% safe — it is one query away from the other 5%, and this set is where an adaptive attacker goes when the mapped sources stop working.

Most of the set is out of scope rather than missing: a source whose upstream target is non-Latin has no business in the to-Latin table. Cross-reference CONFUSABLES_VERSION and docs/provenance.md before reading any one codepoint as a defect.

The set includes five ASCII characters — %, 0, 1, I and m. TR39 is a skeleton transform (m→rn, I/1→l, 0→O), so those are upstream sources; disarm does not apply those rows because folding a legitimate ASCII m to rn corrupts prose. Nothing is filtered out here: a coverage report that quietly drops rows reads as coverage it does not have.

Examples:

>>> unmapped = unmapped_confusables()
>>> "а" in unmapped  # Cyrillic а IS folded, so it is not exposure
False
>>> "m" in unmapped  # TR39 skeleton source m→rn, deliberately not applied
True

Find confusable sources in text that disarm's table does not fold (#563).

The confusables analogue of find_untranslatable, and it follows the same convention: (character, byte_offset) pairs in order of appearance. This is what turns unmapped_confusables from a global number into something answerable against your own traffic.

Composition runs exactly as it does in normalize_confusables, so a decomposed homoglyph whose precomposed form is mapped counts as covered rather than as a gap — otherwise the report would disagree with what the transform does. Offsets are anchored in text, never in the composed intermediate.

Ordinary English will report the letter m; see unmapped_confusables for why that is deliberate.

Examples:

>>> find_unmapped_confusables("pаypal")  # Cyrillic а folds — covered
[]
>>> find_unmapped_confusables("hello")
[]

True if all characters are in U+0000–U+007F.

Examples:

>>> is_ascii("hello 123")
True
>>> is_ascii("café")
False

Which of values reduce to the same identity key (#620).

Every other disarm detector is a single-string predicate, and a collision is not a property of a single string — groß.txt is an ordinary German filename, and аdmin is only a problem next to admin. This is the set-shaped question: given these names, which of them are the same name?

That is what node-tar's PathReservations guard failed to ask before extracting two paths in parallel (CVE-2026-23950), and what a registry has to ask before accepting a second admin (CVE-2013-7236). The two want opposite policies from the same answer — one refuses the batch, the other refuses the registration — so this reports and decides nothing.

Choosing key is choosing the policy, and there is no default. Measured against the four collision CVEs in the validation matrix:

============================ ========== ========== ========= ========= key 2026-23950 2019-19844 2013-7236 2020-12063 ============================ ========== ========== ========= ========= "fold_case" yes -- -- -- "search_key" yes yes yes yes "catalog_key" yes yes yes yes "canonicalize" -- yes yes yes "canonicalize_strict" -- yes yes yes "normalize_confusables" -- yes yes yes ============================ ========== ========== ========= =========

A stronger key finds more collisions, including ones nobody attacked: search_key collides Muller with Müller and Ivan with Иван. That is not a false positive — they really are one key — it is the cost of the key you chose. sort_key is deliberately not offered: a sort key exists to collide, so reporting its collisions would be noise.

Reducing and grouping happen in one pass over one reducer, so the report cannot disagree with the collapse it describes. A group is returned only when it holds two or more distinct inputs — the same string twice is the same name twice, which a reservation table already handles.

The return is not a partition, and the two counts do not add (#763). A name that collides with nothing never appears, so the groups do not cover the input. The quantity a registry actually wants — after reduction, how many distinct identities does this batch hold? — has to be derived, and there is one correct spelling::

reduced = len(set(values)) - sum(len(g.values) for g in groups) + len(groups)

values and indices have different denominators by design (see KeyCollision), so they must never be arithmetically combined. Substituting g.indices for g.values above, or len(values) for len(set(values)), gives a formula that is right on every duplicate-free batch and wrong the moment an input repeats. Measured over 400 duplicate-free batches all four spellings agree with the truth; over 400 with one repeat injected, only this one does.

Examples:

>>> found = find_key_collisions(
...     ["groß.txt", "gross.txt", "other.txt"], key="fold_case"
... )
>>> found[0].key
'gross.txt'
>>> found[0].values
['groß.txt', 'gross.txt']
>>> found[0].indices
[0, 1]
>>> find_key_collisions(["a.txt", "b.txt"], key="fold_case")
[]

A repeated input — the shape every other example omits, and the only shape that separates the correct derivation from its three near-misses:

>>> names = ["admin", "admin", "Admin"]
>>> groups = find_key_collisions(names, key="fold_case")
>>> groups[0].values          # distinct inputs: two
['admin', 'Admin']
>>> groups[0].indices         # occurrences: three
[0, 1, 2]
>>> len(set(names)) - sum(len(g.values) for g in groups) + len(groups)
1

Three names, one identity. The three near-misses give 2, 0 and 1 — the last by cancellation rather than by construction.

True if text is a stable identity key under case folding.

Answers fold_case(text) == text.lower(). A False result says some other string folds to the same value, so a table keyed on this one can collide — groß.txt and gross.txt are the pair node-tar collided on (CVE-2026-23950), and ſtraße/straße and file/file are the same shape. Roughly 2,000 code points behave this way, including every Latin ligature, ẛ, the micro sign, and all of Cherokee (whose fold direction runs small→capital, so both cases move).

This is a fact about the string, not an accusation. groß is an ordinary German word, so a False here is not a report of an attack and the predicate is deliberately kept out of has_anomalies. What to do about it is the caller's decision: reserve both forms, reject the name, or key the table on fold_case rather than str.lower().

str.lower() is the correct comparison basis and str.casefold() is not: casefolding performs the very transform under test, so a predicate written against it is True everywhere.

Answers about disarm's own folding table (Unicode 16.0), so it also reports False for characters your Python's str.lower() knows about and that table does not — which is a collision hazard for the same reason.

A True result is not a uniqueness guarantee: two distinct stable strings can still collide under some other normalization.

Examples:

>>> is_case_fold_stable("gross.txt")
True
>>> is_case_fold_stable("groß.txt")
False
>>> is_case_fold_stable("ΟΔΟΣ")  # Greek final sigma: οδος vs οδοσ
False

True if text is already in the specified normalization form.

Examples:

>>> is_normalized("café")  # NFC by default
True
>>> is_normalized("e\u0301", form="NFC")  # NFD decomposed
False

Detect whether text contains zalgo-style combining mark abuse.

Returns True if any base character has more than threshold consecutive combining marks in NFD decomposition.

Examples:

>>> is_zalgo("café")
False
>>> is_zalgo("Việt Nam")
False
>>> is_zalgo("ḧ̸̡̢̧̛̗̱̜̼̯̞̙́̑̾̊̿̏̒̓̕ě̵̢̧̛̗̱̜̼̯̞̙̈́̑̾̊̿̏̒̓̕l̸̡̢̧̛̗̱̜̼̯̞̙̈́̑̾̊̿̏̒̓̕l̸̡̢̧̛̗̱̜̼̯̞̙̈́̑̾̊̿̏̒̓̕ơ̵̢̧̗̱̜̼̯̞̙̈́̑̾̊̿̏̒̓̕")
True

Flag a hostname as suspicious for Unicode homoglyph spoofing.

Returns (suspicious, analysis) where analysis is a HostnameAnalysis with attributes:

• suspicious: bool — True if a problem was detected (mixed-script, a bundled-table confusable, or a bidi-direction conflict). Because the confusable check is an any-character screen, this flags essentially every hostname with a non-Latin letter — legitimate (москва.рф) as well as spoofs — so it is a maximally conservative screen, not a precise verdict.
• scripts: list[str] — Unicode scripts found across all labels.
• mixed_script: bool — True if any single label contains more than one script.
• has_confusables: bool — True if confusable homoglyphs found. Read after the UTS #46 mapping and NFKC, so it cannot see a compatibility form by construction: ｇoogle.com is already google.com by the time this is computed, and False is the correct answer — after mapping there is no confusable left. Seeing canonical differ from the input while this stays False means compat_fold, not a defect.
• bidi_conflict: bool — True if the decoded hostname mixes strong left-to-right and strong right-to-left characters (the "BiDi Swap" reorder precondition). Folded into suspicious.
• bidi_control: bool — True if the decoded hostname carries a UAX #9 bidi control character: an override (U+202D/U+202E), embedding (U+202A–U+202C), isolate (U+2066–U+2069) or directional mark (U+200E/U+200F/U+061C). Disjoint from bidi_conflict, which reads strong-direction letters only and is therefore blind to the RLO extension spoof. IDNA2008 disallows every character in the set, so this is folded into suspicious and the characters are stripped from canonical.
• has_invisible: bool — True if the decoded hostname carries an invisible character of any class: zero-width (U+200B-U+200D, U+2060-U+2064, U+FEFF, U+180E), tag (U+E0000-U+E007F), variation selector (U+FE00-U+FE0F, U+E0100-U+E01EF), noncharacter (U+FDD0-U+FDEF and the last two of every plane), or private use (U+E000-U+F8FF, planes 15 and 16). Disjoint from bidi_control — these carry no direction at all, so neither bidi field can see them. RFC 5892 puts the tag, variation-selector, noncharacter and private-use classes in DISALLOWED outright, which is what justifies including private use and variation selectors here. U+200C/U+200D are the exception — CONTEXTJ, so conditionally permitted; the screen flags them anyway as a deliberate fail-closed policy. Folded into suspicious. They are removed per label before any other field is computed, so a hostname whose only non-ASCII is an invisible no longer reports a phantom script (U+FEFF sits in the Arabic Presentation Forms block, U+FDD0 in its range).
• compat_fold: bool — True if any label carried a Unicode compatibility form before normalization: fullwidth (ｇoogle), ligature (file), Roman numeral (ⅠBM), mathematical alphanumeric (𝗀𝗈𝗈𝗀𝗅𝖾), circled, superscript, and the rest of the compatibility repertoire. The predicate is RFC 5892 §2.1's, applied per code point: a character c where toNFKC(c) != c is DISALLOWED in an IDN label, so IDNA2008 disallows the whole set and this is folded into suspicious on the same footing as bidi_control and has_invisible. The threat is a blocklist bypass rather than a lookalike: ｅvil.com is absent from a blocked set, screens clean, and resolves to evil.com. Tested per character rather than "NFKC changed the label", which would fire on decomposed input that is entirely valid (한국.kr written with conjoining jamo). Read per label, not over the whole hostname: three of the four UTS #46 label separators carry a compatibility decomposition (U+FF0E and U+FF61 do, U+3002 does not), and a separator is structure rather than label content. This is the one field read from the raw input — every other field is computed after normalization, which is what makes them work and also what erases this evidence.
• cross_label_script: bool — True if the labels span more than one distinct script. Broader and noisier than bidi_conflict (it fires on benign IDN ccTLDs like google.рф), so it is not folded into suspicious; exposed for caller policy.
• label_scripts: list[list[str]] — per-label resolved scripts, left to right.
• whole_script_confusable: bool — True if any label is a whole-script confusable: single-script, non-Latin, whose confusable skeleton is entirely Latin (e.g. Cyrillic аррӏе → apple). A graded signal, not a verdict — on its own it fires on short non-Latin ccTLDs (ру→py) and on real words (оса→oca), so it is not folded into suspicious.
• label_whole_script_confusable: list[bool] — per-label flags, parallel to label_scripts, so a caller can exclude the TLD label. The precise, low-false-positive policy is wsc(non-TLD label) and TLD-is-Latin (plus a caller-supplied protected-name list for the irreducible оса-style case).
• canonical: str — Latin-normalized form of the hostname.

A hostname is flagged suspicious if any single label is mixed-script (draws on more than one Unicode script, excluding Common/Inherited), contains confusable homoglyphs, or has a bidi-direction conflict (bidi_conflict), carries a bidi control character (bidi_control), or carries a zero-width/invisible character (has_invisible), or carries a compatibility form (compat_fold). The mixed-script rule is conservative and fails closed: it flags benign combinations such as Latin+CJK as well as spoofing ones, so a caller wanting a more permissive policy can inspect the mixed_script and scripts fields and decide for itself.

A False (not-suspicious) result is not a safety guarantee. It means only that no mixed-script label and no confusable from the bundled TR39 table was found. Confusables outside the bundled table are not detected and report not-suspicious. Base allow/deny decisions on the granular findings (including whole_script_confusable) plus your own policy — a detector can attest the presence of a problem, never the absence of all problems.

Examples:

>>> suspicious, analysis = is_suspicious_hostname("google.com")
>>> suspicious
False
>>> analysis.canonical
'google.com'
>>> _s, a = is_suspicious_hostname("arnazon.com", contractions=True)
>>> a.canonical
'amazon.com'