Rating: 3.3
Full writeup: https://lkmidas.github.io/posts/20210228-aeroctf2021-writeups/
tl;dr
1. Analyze the main function -> see 3 functions, the 1st one seems to be encrypted.
2. Analyze the 3rd function -> learn that our given dump file is actually the .text, .data and the heap of the executing process that is dumped out.
3.Analyze the 2nd function -> learn that it encrypts the first function by xoring it with 32 bytes that are read from an unknown stream, also there seems to be some trash data that got insert after the real key.
4. Use the knowledge that the new ELF file format always include the endbr64 instruction at the start of each compiled function -> xor its opcode with the first 4 bytes of the encrypted code to get the first 4 bytes of the xor key -> search for it in the memory and decrypt the code.
5. Analyze the 1st function -> learn that it reads the flag into the beginning of the data dump, then encrypts the flag using AES-CBC, the key and IV are also stored somewhere in the data dump the same as the xor key before.
6. Use the knowledge that the code for AES-CBC actually hard copy the key and IV into a AES_ctx struct, also stored somewhere in the dump, to search for duplicates byte string that is 16 bytes in length -> find the key and IV to decrypt the flag.
