Tags: engineering xor reverse 

Rating:

### **high-level overview**

we are given a binary called **a.out** and an assignment which hints towards XOR (Exclusive OR) operations. from the Break the Syntax 2025 CTF, this was the rev task with the least amount of solves, which is surprising due to its simple nature, as opposed to other rev tasks.

### **control flow architecture**

(i used Binja with Pseudo C decompilation)

the binary uses a tightly-scoped entrypoint, passing all logic through a single function:

```
int main(int argc, char** argv) {
sub_40119d(argv[1]);
return 0;
}
```

this is a single entry point into decryption and validation logic.
the function ```sub_40119d``` sets up the xor-based stage exec.

### **sub_40119d**

inside of the `sub_40119d` function, we observe :

1. the first input byte is extracted and used as a decryption key` input[0]`
2. the function `sub_401149 ` is called to xor-decrypt a 0xc7-byte block at `0x4073d2`
3. the binary then enters a loop, calling each entry in a function pointer table stored at `0x404c60 `, incrementing through the table

simplified , cleaned up pseudocode (i renamed the function `sub_401149` that was mentioned in step 2 as ` xor_decrypt`

```
void sub_40119d(char* input) {
uint8_t xor_key = input[0];
xor_decrypt(0x4073d2, 0xc7, xor_key);

void (**table)() = (void**)0x404c60;
size_t i = 1;
while (1) {
table[i - 1](input[i]);
i++;
}
}
```

### **xor decryption logic**

` sub_401149` , or as we renamed it, ` xor_decrypt` , applies a basic in-place xor over a fixed-size buffer:

```
void xor_decrypt(uint8_t* data, size_t len, uint8_t key) {
for (size_t i = 0; i < len; ++i) {
data[i] ^= key;
}
}
```

### **function pointer chain**

starting at `0x404c60` , the binary defines a flat array of 48 function pointers. each points to an encrypted memory section (e.g., `.f_0_section`, .`f_1_section`, ...). these regions contain encrypted functions that validate one byte of input each.

#### pointer layout:

```
0x404c60 → 0x4073d2
0x404c68 → 0x40749b
...
```

none of these function blocks are valid code until decrypted. after decryption, the first few bytes conform to a typical function **prologue**. this is the key to decryption, remember

### **decrypting stage 0: key discovery**

to decrypt the first block at `0x4073d2` , we brute-force the xor key until we find a valid x86 function signature. one such example ->

```
55 push rbp
48 89 e5 mov rbp, rsp
```

== mentioned prologue.

only one key yields this result: `0x42` (ascii 'B'). so the correct first byte of input is `B`.

### **flag recovery**

each stage uses a predictable function header and the validation logic is simple. so our plan is to :

1. extract the encrypted block for each function pointer
2. brute force xor keys 0x00-0xff until the decrypted version starts with :
```
55 48 89 e5
```
3. record the key — it corresponds to one flag byte.

this approach yields a sequence of 48 xor keys, each independently discoverable :

}3M1T_4_T4_R0X_3NO_ZSU3D4T_N4P_GN1TPYRC3D{FTCStB

oh wait, its reversed? O_o (think about why)

final flag :

**BtSCTF{D3CRYPT1NG_P4N_T4D3USZ_ON3_X0R_4T_4_T1M3}**

solved by tlsbollei