Writing YARA Rules That Don't Melt Your Scanner

Why rule performance matters

A YARA rule that runs fine on one sample can bring a scanning pipeline to its knees when you point it at millions of files or every process on a fleet. The difference between a fast rule and a slow one is rarely the logic. It is how the rule interacts with the matching engine.

If you already know how to write YARA, this is the next layer: making rules that stay fast at scale. To do that, you need to understand what the engine actually does under the hood.

How YARA matches: atoms

YARA does not scan the input once per string. It uses a two-stage model built around atoms.

An atom is a short byte sequence, up to 4 bytes, that YARA extracts from each of your strings. During compilation the engine pulls the most selective atom it can from every string and regex. At scan time it runs an Aho-Corasick automaton to find all atoms in a single pass over the data. Only when an atom matches does YARA perform the expensive, full verification of the string at that offset.

The consequence is the single most important performance rule in YARA:

The quality of the atoms YARA can extract from your strings determines how fast your rule runs.

A rule with strong, rare atoms barely does any verification work. A rule whose atoms are short or common triggers verification constantly, and verification is where time goes.

Atom quality and short strings

When YARA compiles a rule, it warns you about strings that yield weak atoms:

warning: rule "Bad_Rule": string "$s1" may slow down scanning

Take that warning seriously. It almost always means the atom is too short or too common.

Consider a two-byte string:

$s1 = "MZ"

The atom is 4D 5A. That byte pair appears at the start of every Windows executable and countless other places. The automaton flags a match on essentially every PE file, and YARA verifies each one. Now imagine that string across a million-file corpus. The engine is doing verification work constantly for a string that tells you almost nothing.

Longer, rarer strings produce better atoms:

// weak: 2-byte atom, matches everywhere
$bad = "MZ"

// strong: 4-byte selective atom, rarely matches by accident
$good = "sekurlsa::logonpasswords"

The $good string gives YARA a 4-byte atom of seku (or a rarer slice), and full verification only runs on the handful of files that actually contain it.

Practical rule: avoid strings shorter than 4 bytes, and prefer strings with distinctive, non-repeating content. If you must match something short, gate it behind a cheaper condition so the short string is only checked when it matters.

The cost of case-insensitive and wide

Modifiers multiply the atoms YARA has to track.

• nocase forces the engine to generate atom variants for every combination of upper and lower case in the atom bytes. A 4-byte nocase atom can expand into 16 atoms.
• wide interleaves null bytes, which can shorten the effective atom and make it less selective.
• xor across the full key range generates 255 variants of the atom.

None of these are forbidden. They are all useful. The point is to apply them deliberately, not reflexively. Do not slap nocase wide ascii xor on every string out of habit. Use the modifier the sample actually requires.

Regex is expensive

Regular expressions are the most common cause of pathological YARA performance. The engine still tries to extract an atom from a regex, but some patterns give it nothing to work with.

Patterns that start with an unbounded or highly general term force YARA to fall back to slow, byte-by-byte evaluation across the whole input:

// terrible: no usable atom, evaluated everywhere
$re = /.*evil.*/

// terrible: leading quantifier over any char
$re = /[a-z]+\.exe/

// better: a fixed, selective prefix gives a strong atom
$re = /evil_[a-f0-9]{16}\.exe/

Two habits keep regex fast:

1. Give the regex a fixed, distinctive substring so YARA can extract a real atom from it.
2. Avoid unbounded quantifiers (.*, .+) at the start of the pattern, and bound your ranges where you can.

And remember: many things people write as regex are better written as a plain string or a hex pattern. If you are matching a fixed byte sequence, use a hex string. It is faster and clearer.

Condition short-circuiting

YARA evaluates conditions left to right and short-circuits boolean and and or. That means the order of your terms is a performance decision.

Put the cheapest, most selective checks first so that expensive checks never run on files that were going to fail anyway.

// BAD: entropy is computed for every file before the cheap filter
condition:
    math.entropy(0, filesize) > 7.0 and uint16(0) == 0x5A4D

// GOOD: the near-free magic-byte check eliminates non-PE files first
condition:
    uint16(0) == 0x5A4D and math.entropy(0, filesize) > 7.0

A rough cost ordering, cheapest first:

Lead with filesize and magic bytes. Gate the expensive stuff behind them.

Module and loop overhead

Modules like pe, math, and hash do real parsing work. That work is only wasted if you invoke it on files that cannot match.

• Guard module calls. pe.is_pe and ... ensures the PE structure is only walked for actual PE files.
• Guard hash calls hard. hash.md5(0, filesize) reads and hashes the entire file. Never make it the first term; put it behind size and magic filters, and ideally behind a string match.
• Bound your loops. A for any i in (0..pe.number_of_sections - 1) loop runs its body per section. Keep the body cheap and make sure the loop only runs on files that already passed earlier filters.

Benchmarking and profiling

Do not guess which rule is slow. Measure.

YARA's command line can report per-rule timing. Run your ruleset against a representative corpus and see where the time goes:

# Show scanning statistics and slowest rules
yara --print-stats -r rules.yar /corpus/

# Warn on strings likely to slow scanning at compile time
yara -w rules.yar /dev/null

# Multi-threaded scan for realistic throughput numbers
yara -p 8 -r rules.yar /large/corpus/

# Cap per-file time so one pathological rule cannot stall the run
yara --timeout=60 -r rules.yar /corpus/

Build a benchmark corpus that reflects reality: a mix of the file types you actually scan, at the scale you actually scan them. A rule that looks fine against 100 files can reveal its cost against 100,000.

When you author rules in a browser workbench like hunt.mlab.sh, the compile-time warnings surface immediately, so you catch weak atoms and slow strings before the rule ever reaches a production scanner or the file-scanning path on mlab.sh.

Before and after

Here is a slow rule and its optimized twin.

// BEFORE: no magic filter, short strings, greedy regex, entropy first
rule Suspicious_Slow {
    strings:
        $a = "MZ"
        $b = "dll" nocase
        $re = /.*VirtualAlloc.*/
    condition:
        math.entropy(0, filesize) > 6.5 and $a and $b and $re
}

// AFTER: cheap filters first, selective strings, bounded regex
import "pe"
import "math"

rule Suspicious_Fast {
    strings:
        $api1 = "VirtualAllocEx"
        $api2 = "WriteProcessMemory"
        $re   = /CreateRemoteThread(Ex)?/
    condition:
        uint16(0) == 0x5A4D and
        filesize < 5MB and
        pe.is_pe and
        2 of ($api*, $re) and
        math.entropy(0, filesize) > 6.5
}

The rewrite keeps the intent but changes the economics. Magic bytes and size eliminate most files for free, the strings carry strong atoms, the regex has a fixed anchor, and entropy runs last on the tiny set of survivors.

Closing

Fast YARA is not about writing less, it is about knowing what the engine pays for. Give it strong atoms, order your conditions cheapest-first, keep regex anchored, and measure before you blame the logic. Do that and your rules stay sharp at a million files instead of buckling at a thousand.

The engine rewards selectivity. Write for the atom and the scanner keeps up.