Tags: rev

Rating:

# xHell

We are given an Excel spreadsheet with around 80 rows and five
columns. The values in the A column are parameters (we will see that
these are instructions later), and columns B to E are computed using a
large formula, except in the first row, where they are user-provided
parameters.

The formula is (example from D14, line breaks and indentation for
clarity):


=IF(MID($A14,4,1)=TEXT(COLUMN()-1,"0"), IF(MID($A14,1,1)="1",
MOD(SUM(
INDIRECT(ADDRESS(ROW()-1,1+MID($A14,2,1))), INDIRECT(ADDRESS(ROW()-1,1+MID($A14,3,1)))),256),
IF(MID($A14,1,1)="2", MOD( SUM(INDIRECT(ADDRESS(ROW()-1,1+MID($A14,2,1))),
-INDIRECT(ADDRESS(ROW()-1,1+MID($A14,3,1)))),256), IF(MID($A14,1,1)="3",
MOD(PRODUCT(
INDIRECT(ADDRESS(ROW()-1,1+MID($A14,2,1))), INDIRECT(ADDRESS(ROW()-1,1+MID($A14,3,1)))),256),
IF(MID($A14,1,1)="4", MOD(MOD( INDIRECT(ADDRESS(ROW()-1,1+MID($A14,2,1))),
INDIRECT(ADDRESS(ROW()-1,1+MID($A14,3,1)))),256), IF(MID($A14,1,1)="5",
MOD(BITAND(
INDIRECT(ADDRESS(ROW()-1,1+MID($A14,2,1))), INDIRECT(ADDRESS(ROW()-1,1+MID($A14,3,1)))),256),
IF(MID($A14,1,1)="6", MOD(BITOR( INDIRECT(ADDRESS(ROW()-1,1+MID($A14,2,1))),
INDIRECT(ADDRESS(ROW()-1,1+MID($A14,3,1)))),256), IF(MID($A14,1,1)="7",
IF(INDIRECT(ADDRESS(ROW()-1,1+MID($A14,2,1))) =INDIRECT(ADDRESS(ROW()-1,1+MID($A14,3,1))),
1,
0),
"X"))))))),
D13)


We can see that this reads an instruction from the A column (a
four-digit number), splits it into digits and uses them to compute the
values in that row. Let's name the digits, in order, dest, x, y
and op.

We can treat the spreadsheet as an execution trace for a program which
has four registers b, c, d and e, and instructions given in
the A column. The instructions are decoded as:

* take the values in registers x and y as parameters (where 1
encodes b, 2 encodes c, and so on),
* compute the result of operation op with parameters x and y,
where values from 1 to 7 encode addition, subtraction, product,
modulus, bitwise-and, bitwise-or and equality testing,
* take the result modulo 256, and
* write it to the register encoded by dest.

The cell which tests if our parameters are accepted checks for three
conditions. For initial values of the registers, we have b - c == 46
and e - d == 119. Further, the final value of e must be 1. The
first two checks let us compute c and d from b and e, so we
can try all possible values for these two registers (from 0 to 255, so
this doesn't take long) to find one that satisfies the final check and
get the flag.

Original writeup (https://de298.user.srcf.net/writeups/insa/xhell.html).