First, download the `math` file. It's likely a Linux executable, since this is a reversing challenge, but let's run `file ./math` to be sure:
```
./math: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=71fce45eb1092a34d4e5f72881177f3d14d22be1, for GNU/Linux 3.2.0, not stripped
```
We know that this is a executable now. What does it do? Let's try it:
```
Question #1: What is 1+1?
Question #2: How many sides does a square have?
Question #3: What is 8*30?
Question #4: What is the remainder when 39 is divided by 4?
Question #5: What is the 41st fibbonaci number?
Question #6: How many questions are on this test?
Question #7: What number am I thinking of?
Question #8: What is the answer to this question?
Question #9: Prove that the answer to the previous question is correct.
Question #10: Give me an integer.
You got [NUMBER OF CORRECT ANSWERS] out of 10 right!
If you get a 10 out of 10, I will give you the flag!
```
Note that I've ommitted my answers and the number of correct answers, which I've replace with `[NUMBER OF CORRECT ANSWERS]`. (Partly because the results are different for different people, and partly because I don't want people seeing my score.)

Let's look at what the assembly instructions are with`objdump -Mintel -D ./math`.
Once we have it disassembled, we can look at the assembly code and the data inside the executable. Let's start from `main`:
```assembly
0000000000001385 <main>:
1385: 55 push rbp
1386: 48 89 e5 mov rbp,rsp
1389: 48 83 ec 10 sub rsp,0x10
138d: 48 8d 05 5c 0e 00 00 lea rax,[rip+0xe5c] # 21f0 <_IO_stdin_used+0x1f0>
1394: 48 89 c7 mov rdi,rax
1397: e8 b4 fc ff ff call 1050 <puts@plt>
139c: 48 8d 05 9d 0e 00 00 lea rax,[rip+0xe9d] # 2240 <_IO_stdin_used+0x240>
13a3: 48 89 c7 mov rdi,rax
13a6: e8 a5 fc ff ff call 1050 <puts@plt>
13ab: c7 45 fc 00 00 00 00 mov DWORD PTR [rbp-0x4],0x0
13b2: eb 6e jmp 1422 <main+0x9d>
13b4: 8b 45 fc mov eax,DWORD PTR [rbp-0x4]
13b7: 83 c0 01 add eax,0x1
13ba: 89 c6 mov esi,eax
13bc: 48 8d 05 b4 0e 00 00 lea rax,[rip+0xeb4] # 2277 <_IO_stdin_used+0x277>
13c3: 48 89 c7 mov rdi,rax
13c6: b8 00 00 00 00 mov eax,0x0
13cb: e8 90 fc ff ff call 1060 <printf@plt>
13d0: 8b 45 fc mov eax,DWORD PTR [rbp-0x4]
13d3: 48 98 cdqe
13d5: 48 8d 14 c5 00 00 00 lea rdx,[rax*8+0x0]
13dc: 00
13dd: 48 8d 05 1c 2f 00 00 lea rax,[rip+0x2f1c] # 4300 <questions>
13e4: 48 8b 04 02 mov rax,QWORD PTR [rdx+rax*1]
13e8: 48 89 c7 mov rdi,rax
13eb: e8 60 fc ff ff call 1050 <puts@plt>
13f0: 8b 45 fc mov eax,DWORD PTR [rbp-0x4]
13f3: 48 98 cdqe
13f5: 48 8d 14 c5 00 00 00 lea rdx,[rax*8+0x0]
13fc: 00
13fd: 48 8d 05 7c 2f 00 00 lea rax,[rip+0x2f7c] # 4380 <submitted>
1404: 48 01 d0 add rax,rdx
1407: 48 89 c6 mov rsi,rax
140a: 48 8d 05 75 0e 00 00 lea rax,[rip+0xe75] # 2286 <_IO_stdin_used+0x286>
1411: 48 89 c7 mov rdi,rax
1414: b8 00 00 00 00 mov eax,0x0
1419: e8 52 fc ff ff call 1070 <__isoc99_scanf@plt>
141e: 83 45 fc 01 add DWORD PTR [rbp-0x4],0x1
1422: 8b 05 38 2c 00 00 mov eax,DWORD PTR [rip+0x2c38] # 4060 <NUM_Q>
1428: 39 45 fc cmp DWORD PTR [rbp-0x4],eax
142b: 7c 87 jl 13b4 <main+0x2f>
142d: b8 00 00 00 00 mov eax,0x0
1432: e8 93 fe ff ff call 12ca <grade_test>
1437: 90 nop
1438: c9 leave
1439: c3 ret
143a: 66 0f 1f 44 00 00 nop WORD PTR [rax+rax*1+0x0]
```
We can see that it does a lot of stuff, which is probably asking the question since we see a lot of `printf` and `scanf`s, then calls `grade_test` and exits.
Now we want to look at `grade_test` since it probably has information about the answers (unless it's a decoy by the organizers, which it probably isn't):
```assembly
00000000000012ca <grade_test>:
12ca: 55 push rbp
12cb: 48 89 e5 mov rbp,rsp
12ce: 48 83 ec 10 sub rsp,0x10
12d2: c7 45 fc 00 00 00 00 mov DWORD PTR [rbp-0x4],0x0
12d9: c7 45 f8 00 00 00 00 mov DWORD PTR [rbp-0x8],0x0
12e0: eb 3d jmp 131f <grade_test+0x55>
12e2: 8b 45 f8 mov eax,DWORD PTR [rbp-0x8]
12e5: 48 98 cdqe
12e7: 48 8d 14 c5 00 00 00 lea rdx,[rax*8+0x0]
12ee: 00
12ef: 48 8d 05 8a 30 00 00 lea rax,[rip+0x308a] # 4380 <submitted>
12f6: 48 8b 14 02 mov rdx,QWORD PTR [rdx+rax*1]
12fa: 8b 45 f8 mov eax,DWORD PTR [rbp-0x8]
12fd: 48 98 cdqe
12ff: 48 8d 0c c5 00 00 00 lea rcx,[rax*8+0x0]
1306: 00
1307: 48 8d 05 72 2d 00 00 lea rax,[rip+0x2d72] # 4080 <answers>
130e: 48 8b 04 01 mov rax,QWORD PTR [rcx+rax*1]
1312: 48 39 c2 cmp rdx,rax
1315: 75 04 jne 131b <grade_test+0x51>
1317: 83 45 fc 01 add DWORD PTR [rbp-0x4],0x1
131b: 83 45 f8 01 add DWORD PTR [rbp-0x8],0x1
131f: 8b 05 3b 2d 00 00 mov eax,DWORD PTR [rip+0x2d3b] # 4060 <NUM_Q>
1325: 39 45 f8 cmp DWORD PTR [rbp-0x8],eax
1328: 7c b8 jl 12e2 <grade_test+0x18>
132a: 8b 45 fc mov eax,DWORD PTR [rbp-0x4]
132d: 89 c6 mov esi,eax
132f: 48 8d 05 34 0e 00 00 lea rax,[rip+0xe34] # 216a <_IO_stdin_used+0x16a>
1336: 48 89 c7 mov rdi,rax
1339: b8 00 00 00 00 mov eax,0x0
133e: e8 1d fd ff ff call 1060 <printf@plt>
1343: 83 7d fc 0a cmp DWORD PTR [rbp-0x4],0xa
1347: 75 2a jne 1373 <grade_test+0xa9>
1349: 48 8d 05 37 0e 00 00 lea rax,[rip+0xe37] # 2187 <_IO_stdin_used+0x187>
1350: 48 89 c7 mov rdi,rax
1353: e8 f8 fc ff ff call 1050 <puts@plt>
1358: 48 8d 05 45 0e 00 00 lea rax,[rip+0xe45] # 21a4 <_IO_stdin_used+0x1a4>
135f: 48 89 c7 mov rdi,rax
1362: e8 e9 fc ff ff call 1050 <puts@plt>
1367: b8 00 00 00 00 mov eax,0x0
136c: e8 58 fe ff ff call 11c9 <generate_flag>
1371: eb 0f jmp 1382 <grade_test+0xb8>
1373: 48 8d 05 3e 0e 00 00 lea rax,[rip+0xe3e] # 21b8 <_IO_stdin_used+0x1b8>
137a: 48 89 c7 mov rdi,rax
137d: e8 ce fc ff ff call 1050 <puts@plt>
1382: 90 nop
1383: c9 leave
1384: c3 ret
```
Now we see a reference to `answers`, which sounds like it might have some interesting information. If we follow the assembly code, we can see that it's loading the pointer to `answers` into `rax`, and then setting rax to the value at the address `rax + rcx`.

So what is this offset `rcx`? Well, if we look at location `12ff`, and the context (i.e. the instructions before it), we can see that it's the loop counter times 8. (If you don't get this, follow the program and see what numbers are being assigned to the registers and memory and what the program is doing with them.)

Now why would it be multiplied by 8? Well this might signal that the answers are 8 bytes long integers. So let's examine what `answers` exactly is:
```assembly
0000000000004080 <answers>:
4080: 02 00 add al,BYTE PTR [rax]
4082: 00 00 add BYTE PTR [rax],al
4084: 00 00 add BYTE PTR [rax],al
4086: 00 00 add BYTE PTR [rax],al
4088: 04 00 add al,0x0
408a: 00 00 add BYTE PTR [rax],al
408c: 00 00 add BYTE PTR [rax],al
408e: 00 00 add BYTE PTR [rax],al
4090: f0 00 00 lock add BYTE PTR [rax],al
4093: 00 00 add BYTE PTR [rax],al
4095: 00 00 add BYTE PTR [rax],al
4097: 00 03 add BYTE PTR [rbx],al
4099: 00 00 add BYTE PTR [rax],al
409b: 00 00 add BYTE PTR [rax],al
409d: 00 00 add BYTE PTR [rax],al
409f: 00 6d 8d add BYTE PTR [rbp-0x73],ch
40a2: de 09 fimul WORD PTR [rcx]
40a4: 00 00 add BYTE PTR [rax],al
40a6: 00 00 add BYTE PTR [rax],al
40a8: 0a 00 or al,BYTE PTR [rax]
40aa: 00 00 add BYTE PTR [rax],al
40ac: 00 00 add BYTE PTR [rax],al
40ae: 00 00 add BYTE PTR [rax],al
40b0: 87 15 59 00 00 00 xchg DWORD PTR [rip+0x59],edx # 410f <flag+0x2f>
40b6: 00 00 add BYTE PTR [rax],al
40b8: 49 1e rex.WB (bad)
40ba: a1 06 00 00 00 00 5f movabs eax,ds:0x60e95f0000000006
40c1: e9 60
40c3: 20 00 and BYTE PTR [rax],al
40c5: 00 00 add BYTE PTR [rax],al
40c7: 00 09 add BYTE PTR [rcx],cl
...
```
Now you might see some instructions on the right side of this disassembly, but if you look at them what they're doing doesn't make sense. That's because the label (which is `answers`) isn't supposed to be instructions! They're just random instructions that just happen to correspond to the data in `answers` (there's a reason it's in the label `.data`!).

This is because we used the `-D` flag which disassembles everything, instead of `-d` which disassembles executable sections, but doesn't show data sections. (There are flags to make `objdump` with `-d`, but imo it's simpler to just use `-D` here. Your choice)

Ok, so back to what we were doing. Let's look at the data inside `answers`, We see `2`, a bunch of `0` bytes, then `4`... well, if we look at the first few questions, we can obviously see that the answers are `2` and `4`. (unless you're a toddler. but then how are you reading this? pls don't spam comments thx)

Now we can see that each answer is 8 bytes long because there are a bunch of `0`s after the first few answers, and `0` wouldn't make sense as answers. Also, we saw that the loop count was being multiplied by `8` earlier, so it makes sense.

Now we can find the answers! They're just 8 byte long integers. But we have to make sure to get the bytes in the right order, since x86 is little endian. So we just need to reverse the byte order for each integer.

Doing this, we get the answers:
1. `2`
2. `4`
3. `0xf0=240`
4. `3`
5. `0x09de8d6d=165580141`
6. `0x0a=10`
7. `0x0x591587=5838215`
8. `0x06a11e49=111222345` (this is one of those things where you feel really stupid after seeing the answer)
9. `0x2060e95f=543222111`
10. `9` (there aren't any `0`s after this so we can just assume that there's nothing else)

Now when we enter the answers (as decimal), we get the flag! It's `LITCTF{y0u_must_b3_gr8_@_m4th_i_th0ught_th4t_t3st_was_imp0ss1bl3!}`.
Note that we couldn't have gotten the flag using purely math to answer the questions. (Well, technically everything is applied math, but I'm saying pure math here.)

You might ask, why couldn't we have just edited the code to generate the flag directly or faked it by using a debugger and modifying stuff to pass the checks?
Well, if we look inside the `generate_flag` function, we see that it's computing the flag, and something named `submitted` is involved. Well, if we look through, we find out that this is the place where your inputted answers are stored! So the flag DEPENDS on the answers, and you can't get it without finding the answers in some way.