Today's Plan (3 Hours)

Learning outcome (LO.3): write obfuscation as a security rule · state each technique's cost and limit · measure a protection and justify it

What Is Source Code?

• The human-readable program text that you write (e.g., a C file).
• Example: int topla(int a, int b) { return a + b; }

Source code is compiled and turned into the form the machine runs.

Compiler and Binary

• Compiler: the program that translates source code into machine code (gcc, clang).
• Binary: the result of compilation; the file the computer runs directly (.exe, .so).

Machine Code and Assembly

• Machine code: the numeric instructions the processor understands.
• Assembly: a somewhat more human-readable form of machine code (mov, cmp, jmp).

This is what you see when you open a binary.

What Is Reverse Engineering?

Reverse engineering: looking at a binary and trying to work out what the program does.

This is the attacker's basic job.

Decompiler

• Decompiler: a tool that turns a binary back into a form close to readable code.
• Examples: Ghidra, IDA.

Goal: someone with no access to the source can infer the logic just by looking at the binary.

Bit, Byte, Hexadecimal

• Bit: 0 or 1.
• Byte: 8 bits.
• Hexadecimal (hex): written with a 0x prefix; 0x2A = 42.

We will see values like 0x5A in the code; these are just numbers.

What Is XOR?

• XOR (^): 1 if the two bits differ, 0 if they are the same.
• Property: a ^ b ^ b == a → it undoes itself.

c = a ^ 0x5A;   /* şifrele */
a = c ^ 0x5A;   /* geri çöz */

It is heavily used in obfuscation because it is reversible.

What Is Entropy (Randomness)?

• Entropy: how "random" a piece of data looks.
• Cryptographic keys look high-entropy (irregular bytes).

If an attacker sees a high-entropy block in a binary, they say "there might be a key here."

Function, Branch, Condition

• Function: a block of code that does one job (erisim_ver).
• Branch: a fork in the road, like an if.
• Condition: the expression that decides which way the branch goes.

Basic Block

• Basic block: a sequence of instructions that runs straight through, with no branch.
• When an if is reached the block ends, and two new blocks begin.

Program = basic blocks wired together.

Control Flow Graph (CFG)

• CFG (Control Flow Graph): a diagram that makes basic blocks its nodes and the transitions between them its edges.
• It is the program's "road map."

This Week's Question in One Sentence

I have delivered the program to the user; the attacker now owns it. How do I make it harder
for them to read and modify my code?

Answer: code obfuscation rules. Let's begin.

First, a Question

You have delivered a mobile app to the user.

• Whose machine is the code running on now?
• Are your server-side security checks still valid here?
• How many times can the attacker try?

MATE: the Attacker at the End

Man-At-The-End = an attacker who possesses the program itself.

• The binary is in their hands
• They can read and modify memory
• They have a decompiler and a debugger in hand
• They can try as many times as they like

This is also called the white-box model (week 1).

The MATE Attacker Model — Diagram

The Honest Truth

An attacker with enough time, skill, and motivation breaks every obfuscation.

Cookbook (Recipe 12.1): anti-tampering is not "unbreakable."

So what is the goal, then?

The Goal: Make Breaking It Uneconomical

Obfuscation is a tool for raising cost, not for keeping a secret.

• Make breaking it expensive
• Make automating it harder
• Buy time for the other layers

Let's turn these into three rules.

Rule 1 — Push the Cost Above the Value

The attack must be more expensive than the protected asset.

• Protecting a 10 TL secret with 1000 TL of effort: a waste
• Protecting a 1000 TL secret with 10 TL of effort: insufficient

First determine the asset's value, then choose the protection.

Rule 2 — Break Automation

An attack that works on one copy should not work on every copy.

• This is called diversification (section 5 of this week)
• With tooling in week 14 (Tigress --Seed)

One crack should not open every door.

Rule 3 — Buy Time for the Other Layers

Obfuscation is not a defence on its own; it is a layer.

• RASP (week 6) protects it while it is running
• Integrity checking, server-side checks
• Holding out until the key/version is renewed is enough

What Does Obfuscation Hide?

Collberg's classification: we split obfuscation by what it hides.

There are five families. Let's go through them in order.

Virtualisation — Diagram

Five Families — Potency ↑ Cost ↑

In the Field: All Together

The native countermeasures listed in a certified product cover this entire map:

symbol hiding · name obfuscation · arithmetic transform · string encoding · opaque boolean · bogus operation · dead branch · flattening · random exit · stripping logging

None of them is strong alone; strength comes from togetherness.

What Obfuscation GIVES

• A delay against static analysis (strings, symbols, decompilation)
• Resistance against automated attack (via diversification)
• Time for the other layers
• A measurable increase in cost

What Obfuscation DOES NOT Give

• The secrecy of a secret → that is cryptography's job
• Protection against reading memory → RASP, whitebox
• Permanent security → there is no such thing as "unbreakable"
• Free protection → every obfuscation comes with a cost

What Obfuscation Gives and Does Not Give — Diagram

⚠️ The Most Common Mistake

Mistaking obfuscation for storing a key.

• An embedded/hidden key is still there
• The program has to use it while it runs → it is exposed in memory
• The decoding logic is inside too

Secrecy is cryptography's job, not obfuscation's. (week 3)

The "Protection Rule" Template

KURAL K-xx: <teknik>
  Neyi korur?  : hangi varlık / kod bölümü
  Hangi tehdit?: statik / dinamik / otomatik / kurcalama
  Nasıl?       : bir cümle (kavram)
  Maliyet      : boyut, hız, bakım, hata ayıklama
  Sınır        : neyi korumaz
  Ölçüm        : etkinlik nasıl doğrulanır

Why Is the "Measurement" Line Required?

Obfuscation is not "present/absent."

• The evaluator (week 13) asks how much each layer delays the attack
• Protection without measurement = a claim, not evidence

In the project you will write S9 using this template.

Four Metrics — Diagram

Section 2 — Quick Check

1. How does the MATE attacker differ from a black box?
2. What were obfuscation's three rules?
3. Does obfuscation protect a key? Why?

The Purpose of This Section

We treat the CFG (already defined in the background section) as this section's skeleton; the attacker's first job is to extract that skeleton.

In week 4 we introduced flattening.

In this section we add the rules that strengthen it:

K-01 opaque predicate · K-02 arithmetic encoding · K-03 bogus/dead · K-04 flattening (in depth) · K-05 random exit · K-06 call hiding

All examples are synthetic, built around a single check: erisim_ver.

Opaque Predicate — Diagram

K-01 · What Does It Protect?

• Which condition a branch is taken under
• How many times a loop runs

Idea: branch on an expression whose value the programmer knows but the tool cannot easily resolve.

K-01 · Opaque Predicate Example

/* x*(x+1) daima çifttir → sonuç daima true */
static int opak_dogru(unsigned x) {
    return ((x * (x + 1)) & 1u) == 0;
}

The programmer knows: it is always true. Static analysis cannot easily prove this.

K-01 · How Is It Used?

if (opak_dogru(sayac)) {
    durum = gercek_adim(durum);   /* daima buraya girer */
} else {
    durum = sahte_adim(durum);    /* ölü yol: hiç çalışmaz */
}

The analyst thinks both paths are "real"; they spend their time on the fake path.

K-01 · Opaque Loop

The guide makes loops a separate rule:

• for/while are never shown directly
• Hidden inside flattening
• The body is split into small blocks
• Iteration is driven by a state variable

The information "this loop runs N times" cannot be read off the CFG.

K-02 · What Does It Protect?

Constants and simple computations.

It hides directly readable hints like "this number is 0x1F, this operation is an XOR."

K-02 · Equivalent Expressions (MBA)

Replace an operation with an equivalent one that produces the same result but looks complex:

a + b   ≡   (a ^ b) + 2*(a & b)
x * 2   ≡   x << 1

Same result, complex appearance. (MBA = mixed boolean-arithmetic)

K-02 · Hide the Constant

/* 0x2A doğrudan yazılmaz; iki parçadan üretilir */
static uint8_t esik(void) {
    uint8_t a = 0x37, b = 0x1D;
    return (uint8_t)(a ^ b);   /* = 0x2A */
}

Searching the binary for 0x2A returns nothing.

Bogus Operation ≠ Dead Branch — Diagram

K-03 · The Two Are Different!

The guide makes these separate rules. The difference is critical:

• Bogus operation: runs, does not change the result
• Dead branch: never runs (protected by an opaque predicate)

K-03 · Bogus Operation Example

crc = crc32_guncelle(crc, veri, n);
crc ^= (sabit ^ sabit);   /* = crc; sonuç aynı */

sabit ^ sabit == 0 → the CRC does not change, but the code gets more crowded.

K-03 · Dead Branch Example

if (opak_yanlis()) {        /* daima false */
    sahte_kontrol();        /* hiç çalışmaz */
}

The analyst treats this block as "real" and examines it; they waste time.

K-04 · What Does It Protect?

The algorithm's structure:

• The natural adjacency of blocks
• The information "which block leads to which"

K-04 · Idea: Vertical → Horizontal

• Normal: blocks follow one another (vertical flow)
• Flattened: all blocks sit in a single switch dispatcher (horizontal)
• The order can only be read from a state variable

K-04 · Template

int durum = BASLA;
for (;;) switch (durum) {
  case BASLA: durum = ADIM1; break;
  case ADIM1: durum = kosul ? ADIM2 : HATA; break;
  case ADIM2: durum = BITIR; break;
  case HATA:  return RED;
  case BITIR: return IZIN;
  default:    return RED;   /* rastgele çıkış (K-05) */
}

K-04 · Weak on Its Own

We said back in week 4: an experienced analyst can solve this template fairly quickly.

Strength comes from reinforcements. In order:

K-04 · Reinforcement 1 — State Values (K-02)

• Scattered, arithmetically produced values instead of case 1, 2, 3
• The sequence is invisible
• Which step follows which cannot be read

K-04 · Reinforcement 2 — Bogus/Dead (K-03)

• Bogus cases are added
• Telling which case is real becomes harder

K-04 · Reinforcement 3 — Random Exit (K-05)

• Errors exit through the default branch with an unpredictable state
• Where the failure happened cannot be read

K-04 · Reinforcement 4 — Opaque Transitions (K-01)

• case transitions depend on opaque conditions, not fixed values
• Static tracing becomes even harder

K-05 · What Does It Protect?

Where a check fails.

The attacker typically:

• Finds the "failure branch"
• Turns it into the "success branch"

K-05 · How?

if (!imza_gecerli(p)) {
    durum = kararsiz_deger();  /* switch'te tanımsız → default */
    break;
}

There is no direct return RED; the state goes to an unpredictable value.

K-05 · Success Should Exit the Same Way Too

• A successful exit uses the same method too
• So success and failure look alike in the flow
• The analyst cannot say "this is the accept branch"

K-06 · What Does It Protect?

Hints like: "this function calls memcmp, so it must be doing a comparison."

Standard library calls tell the attacker what you are doing.

K-06 · How?

• Your own constant-time esit_mi instead of memcmp on the critical path
• No recognisable name appears to the linker
• Bogus parameters: add parameters that are never used → the signature becomes misleading

K-06 · ⚠️ Do Not Break Constant-Timeness

When you write your own comparison, make sure it stays constant-time (week 3).

• An early-exit comparison → opens a side channel
• Do not create a new hole while adding a protection

Section 3 — Summary

• K-01 opaque predicate/loop · K-02 arithmetic encoding
• K-03 bogus operation ≠ dead branch
• K-04 flattening (with reinforcements)
• K-05 random exit · K-06 call hiding

None is sufficient on its own; together and measured.

Section 3 — Quick Check

1. What is the difference between a bogus operation and a dead branch?
2. What are the 4 rules that strengthen flattening?
3. Which attack does random exit make harder?

Data Obfuscation Rules K-07–K-09 — Diagram

Control Flow ≠ Data

• Control flow (section 3): hides the algorithm's structure
• Data obfuscation (this section): hides the values the program processes

Strings, constants, tables, variables.

K-07 · What Does It Protect?

The readable text inside a binary.

The cheapest attack step: strings.

"Lisans gecersiz" or a URL → tells the attacker exactly where to look.

K-07 · Idea

• Sensitive strings are encoded before compilation (e.g., XOR)
• They stay encoded in the binary
• Decoded at the moment of use
• Wiped the instant the job is done

K-07 · Example (Synthetic)

static const uint8_t GIZLI[] = {0x3B,0x2A,0x2E,0x2E,0x2D};
void kullan(void) {
    char tmp[sizeof GIZLI];
    for (size_t i=0;i<sizeof GIZLI;i++)
        tmp[i] = GIZLI[i] ^ 0x5A;      /* çöz */
    isle(tmp, sizeof GIZLI);
    memset_s_benzeri(tmp, sizeof tmp); /* hemen sil */
}

K-07 · ⚠️ Critical Limit

• The decoded string is exposed in memory while running
• The decoding key is also in the binary
• This is a static-scan countermeasure

In the field: the decoding key is split and distributed, and the decoding function gets extra checks.

K-07 · String Obfuscation ≠ Key Storage

Again: string obfuscation stops strings, it does not protect a key.

• For the key → whitebox (week 11) or hardware
• Measurement: the sensitive string must not be found in strings output

K-08 · Problem: Plain 0/1 Is Dangerous

Returning a check's result as a plain 0/1:

• The attacker flips a single byte
• "Failure" becomes "success"

K-08 · Idea

Tie the true/false value to:

• Multiple values
• In a way a single-byte patch cannot flip

Functions return an opaque return code instead of a plain boolean.

K-08 · Example (Concept)

typedef struct { uint32_t a, b; } Karar;
static Karar izin_ver(void) {
    return (Karar){ 0xA3C1u, 0x5C3Eu };  /* a ^ b == 0xFFFF */
}
static int karar_izin_mi(Karar k) {
    return (k.a ^ k.b) == 0xFFFFu;
}

The calling side verifies both fields.

K-09 · What Does It Protect?

The recognisable footprint of a sensitive variable in memory.

Patterns like "a 32-byte block = an AES key" point the attacker straight at it.

K-09 · How?

• Split a variable (produce a 32-bit value from two 16-bit shares)
• Merge several variables into a single word
• Rearrange arrays
• Change buffer size/order/layout
• Fill with random values after use

K-10 · Virtualisation (Concept)

What does it protect? A function's machine code.

How? The function → a custom VM's bytecode; the bytecode and a small interpreter are embedded in the binary.

The attacker does not see familiar code — they see a custom instruction set they must first solve.

K-11 · Compiler-Based (O-LLVM)

What does it protect? The rules above, applied through a compiler pass, not by hand.

• LLVM-based obfuscators: flattening, bogus control flow, instruction-substitution passes
• The source stays readable; protection is a compile option
• A different seed per build → easy diversification

K-12 · Dynamic Encryption (Concept)

What does it protect? The most sensitive code sections, by keeping them encrypted in the binary.

How? The section stays encrypted → decoded only right when it is about to run → then re-encrypted/wiped.

K-12 · ⚠️ Risk

• Directly conflicts with OS memory protections (DEP/NX, W^X)
• Done wrong, it both crashes and opens a new hole
• The code sits exposed in memory for a moment while running → a memory dump can catch it

K-12 · Rule

If adding one protection turns off another, measure the net gain and record the trade-off.

In practice it is used very selectively, on small sections.

The Java/Managed Side (from Week 5)

• This week's rules are for native (C/C++)
• The managed-language equivalent: ProGuard/R8 name obfuscation + dead code elimination + shrinking
• APIs reached via -keep and reflection

The Two Sides Work Together

• The Java side is obfuscated with R8
• The native side, with this week's rules
• Cross-check: native verifies Java's integrity, Java verifies native's (week 6, RASP)

Section 4 — Summary

• K-07 string encoding · K-08 opaque boolean · K-09 variable splitting
• K-10 virtualisation (expensive) · K-11 compiler-based (O-LLVM) · K-12 dynamic (risky)

Data obfuscation complements control-flow obfuscation.

Section 4 — Quick Check

1. Does string obfuscation protect the key?
2. Why does an opaque boolean make a single-byte patch harder?
3. What are virtualisation's two biggest costs?

Diversification — Diagram

Diversification: Recall Rule 2

An attack that works on one copy should not work on every copy.

If the attacker can crack one copy and distribute the patch/script to everyone, one crack opens every door.

What Is Diversification?

Producing binaries from the same source that are behaviourally equivalent, structurally different.

Opaque predicates, bogus blocks, state values change from copy to copy.

Two Kinds of Diversification

• In space: every build/distribution gets a different seed
  • An attack written for one copy does not work on another
• In time: every release gets a new arrangement
  • An attack written for the old release breaks in the new one

Works together with key/version renewal (week 10).

What Does Diversification Prevent?

• It does not raise obfuscation's potency (a single copy can still be cracked)
• But it prevents scaling

Week 14: with tooling, via Tigress --Seed, RandomFuns.

Diversification Effectiveness: a Simple Metric

• Compile the same source with two different seeds
• Measure the byte difference of the protected functions
• The higher the difference, the lower the odds that an attack on one works on the other

Write this metric into S9.

Why Measure?

To defend a protection decision.

Collberg's framework: four dimensions.

These four are the foundation of week 13's attack potential scoring.

Metric 1 — Potency

Question: how incomprehensible is it to a human?

How: complexity metrics — CFG node/edge count, cyclomatic complexity, nesting depth (before/after).

Metric 2 — Resilience

Question: how much does it withstand an automated tool?

How: how long/successfully a deobfuscation tool (symbolic execution, a simplifier) takes to undo the protection.

Metric 3 — Stealth

Question: does the obfuscation give itself away?

How: is obfuscated code statistically distinguishable from normal code?

If it is distinguishable, the attacker knows where to look.

Metric 4 — Cost

Question: how much does it cost us?

How: size increase, runtime, memory, maintenance, debugging difficulty.

Before Flattening (Concept)

Few nodes, readable flow.

After Flattening (Concept)

Many nodes, a single hub; "which block comes next" cannot be read.

Cost Measurement Example

size ./program_temiz ./program_gizli   # boyut
# süre: aynı girdiyle N kez çalıştır, ortalama

Interpreting the Measurement

• Potency ↑ (34 blocks) and cost ↑ (28% size, 1.7× time)
• This trade-off is expected
• Decision: is the gain acceptable given the asset's value?

Do not say "strong"; show these numbers.

Decision Rule (Write into S9)

Why Should We Know the Other Side's Tools?

To measure resilience correctly.

We learn these to test our own defence — not to attack.

The Other Side's Tools — Diagram

Tool 1 — Symbolic Execution (KLEE)

• Program paths are solved as mathematical constraints
• Opaque predicates and flattening can be undone this way

Resilience rule: tie opaque predicates to structures that are expensive to solve (factorization, hashing), grow the state space — but measure the cost.

Tool 2 — MBA Simplifier

• Undoes arithmetic encoding (K-02)

• Countermeasure: also tie encoded constants to opaque predicates — layers that depend on each other are harder to solve separately

Rule: do not count arithmetic encoding as a secrecy layer on its own; combine it with control flow.

Tool 3 — Pattern Recognition

• Recognises known library/pass patterns

Rule: diversify; do not rely on a single opaque predicate pattern or a single VM design.

(in space: several patterns/seeds · in time: change the patterns every release)

Main Takeaway

Banescu et al. (Tigress + KLEE): how much each transform withstands symbolic execution is measured.

Resilience is not a claim, it is a measured quantity.

Do not say "strong"; produce a number: "this transform pipeline did not let this problem, of this size, be solved in this much time, against this tool."

Obfuscation in Layered Defence — Diagram

An Observation from the Field

In a certified product, the native countermeasures cover all of this week's rules.

But the document notes, next to every one of them: "not strong on its own."

What convinces an evaluator: togetherness + diversification + measured delay.

The Goal of This Example

We take a single synthetic check (erisim_ver) and protect it layer by layer.

At every step: what was added · what we gained · what we paid.

Step 0 · Unprotected Start

int erisim_ver(const char *jeton) {
    if (jeton_gecerli(jeton))
        return IZIN;      /* tek dal, tek dönüş */
    return RED;
}

Weakness: a single branch, strings, bypassed with a single-byte patch.

Step 0 · What Does the Attacker Do?

• strings → IZIN/RED and the related strings
• Decompile → a single if, two returns
• Flip the failure branch to success (a single byte)

Time: minutes. Skill: low.

Step 1 · K-07 Encode the String/Constant

/* IZIN/RED ve dizgeler artık düz görünmez */
static int izin_kodu(void){ return 0x9E ^ 0x9F; } /* =1 */

• Gain: strings does not find the sensitive text
• Cost: very low

Step 1 · Measurement

The first layer is the cheapest and gives the highest return.

Step 2 · K-02 Arithmetic Encoding

/* karşılaştırma karmaşık ama denk bir ifadeye döner */
int esik = ((a ^ b) + 2*(a & b));   /* = a + b */

• Gain: the constant/computation is not directly readable
• Cost: low

Step 3 · K-04 Flatten

int durum = BASLA;
for(;;) switch(durum){
  case BASLA: durum = kontrol()?ADIM:HATA; break;
  case ADIM:  durum = BITIR; break;
  case HATA:  return RED;
  case BITIR: return IZIN;
}

No single if anymore; the order lives in the state variable.

Step 3 · What Changed?

• The CFG is now "horizontal" — the blocks sit in one switch
• Which branch means "success" cannot be read directly
• Cost: medium (more blocks/branches, a bit slower)

Step 4 · K-05 Random Exit

if (!kontrol()) { durum = kararsiz_deger(); break; } /* →default */

• Where the error was caught cannot be read from the flow
• A single-byte patch is no longer enough

Step 5 · K-03 Bogus + Dead

crc ^= (s ^ s);            /* sahte işlem: sonuç aynı */
if (opak_yanlis()) sahte(); /* ölü dal: çalışmaz */

• The real logic sits in a crowd
• Cost: medium

Step 6 · K-08 Opaque Boolean

Karar k = izin_ver();               /* a^b==0xFFFF */
if (karar_izin_mi(k)) uygula();

• The result is not a plain 0/1 → a single byte cannot flip it
• Flipping it requires multiple points

Step 7 · K-10 Virtualisation?

• Only for the most critical function
• Cost: high (tens of times slower)
• Unnecessary for most checks

Rule: do not go this far if the value is low.

End to End: Takeaway

• Every layer made the attack a little more expensive
• Every layer added a cost
• Where you stop depends on the asset's value and on what you measured

This table is exactly the protection table you will put into S9.

Project · S9 (Advanced Hardening) — 1

1. Protection table

For 2–3 critical sections (license, key derivation, integrity), fill in the rule template:

what it protects · threat · how · cost · limit · measurement

Project · S9 — 2

2. Measurement

For at least one technique, before/after:

• Count of sensitive strings in strings output
• CFG node count
• Binary size, the duration of an operation

Project · S9 — 3, 4

3. Diversification decision: yes/no + justification

4. Limit and remaining risk: what does each protection not protect?

"This string obfuscation does not protect the key; for the key see S8 whitebox/hardware."

Later: S15 (week 14, putting obfuscation into the build pipeline) · S16 (week 12, testing that behaviour is unchanged).

Through an Evaluator's Eyes

What S9 is graded on is not "I used a lot of techniques."

It is graded on: the justification and measurement of every technique.

An unmeasured protection = a claim. Not evidence.

Sources

• Cookbook (Viega & Messier) Chapter 12 — anti-tampering
• Secure programming technical guide — native countermeasures
• Collberg & Nagra, Surreptitious Software — taxonomy, measurement
• Schrittwieser et al. (ACM CSUR 2016) — overview, deobfuscation
• Banescu et al. — Tigress + KLEE resilience measurement
• Obfuscator-LLVM — compiler-based