Tags: misc
Rating:
# Misc: Find Me If You Can
## First look
We are provided an address and port. Running ``netcat`` on these tells us we need to find the number not matching (we have 3 attempts). We are then given a blob of base64 encoded data. Decoding this reveals a header of ``78 9c``, meaning this is zlib compressed data. Decompressing gives us data with a JPEG file header, and viewing this image results in the following:
## Approach
This is clearly a machine learning problem. In fact, the image consists of 20x20 digits, all having a size of 28x28. This is coincidentally the same size as the images in the MNIST dataset. In order to indentify these digits, we train a simple Convolutional Neural Network using the MNIST dataset for training data. We used Keras for its ease of use.
### Note on classification
One noteworthy point is that the output of our CNN is an array of length 10 containing the probabilities of the image being each number. When identifying an image, the highest probability is usually used. When using a CNN with 99% accuracy, this should still lead to 4 or more candidates, where we are only allowed 3. We can get around this by realising that because we know there are 399 correct images and only 1 wrong, we can instead use the geometrical distance from the probability array of each image to the average probability array, to get a better guess of the outlier numbers (the numbers most likely to be different from all others).
## Final Code
```
import keras
from keras.datasets import mnist
from keras.models import Sequential
from keras.layers import Dense, Dropout, Convolution2D, MaxPooling2D, Flatten
from pwn import *
from base64 import b64decode
from zlib import decompress
from io import BytesIO
from PIL import Image
import numpy as np
model = Sequential()
r = remote("34.70.233.147", 7777)
batch_size = 32
num_classes = 10
epochs = 10
def train():
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train = x_train.reshape(x_train.shape[0], 1, 28, 28)
x_test = x_test.reshape(x_test.shape[0], 1, 28, 28)
x_train = x_train.astype('float32')
x_test = x_test.astype('float32')
x_train /= 255
x_test /= 255
print(x_train.shape[0], 'train samples')
print(x_test.shape[0], 'test samples')
y_train = keras.utils.to_categorical(y_train, num_classes)
y_test = keras.utils.to_categorical(y_test, num_classes)
model.add(Convolution2D(32,(3,3),activation='relu',data_format='channels_first',input_shape=(1,28,28)))
model.add(Convolution2D(32, 3, 3, activation='relu'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(256, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(10, activation='softmax'))
model.summary()
model.compile(loss='categorical_crossentropy',
optimizer='adam',
metrics=['accuracy'])
history = model.fit(x_train, y_train,
batch_size=batch_size,
epochs=epochs,
verbose=1,
validation_data=(x_test, y_test))
score = model.evaluate(x_test, y_test, verbose=0)
print('Test loss:', score[0])
print('Test accuracy:', score[1])
def recvwait():
s = b""
temp = r.recv(timeout=3)
while temp:
s += temp
try:
temp = r.recv(timeout=3)
except:
break
return s
def receive():
s = recvwait().decode()
print(s)
s = s.split("b'")[1].strip("'")
d = decompress(b64decode(s.encode()))
s = BytesIO(d)
im = Image.open(s).convert("RGB")
#im.show()
s.close()
out = []
for i in range(20):
for j in range(20):
pix = np.array([[im.getpixel((28*j+dj, 28*i+di))[0] for dj in range(28)] for di in range(28)])
pix = pix.reshape(1,1,28,28)
pix = pix.astype('float32')
pix /= 255
coord = (i, j)
out.append((coord, pix))
return out
train()
while True:
imgs = receive()
out = []
grid = [[0 for j in range(20)] for i in range(20)]
for img in imgs:
res = model.predict(img[1])
val = np.argmax(res, axis=1)[0]
out.append((img[0], list(res[0])))
grid[img[0][0]][img[0][1]] = val
avg = [sum(ent[1][j] for ent in out)/len(out) for j in range(10)]
fin = []
for ent in out:
dists = sum(pow(avg[i]-ent[1][i], 2) for i in range(10))
fin.append((dists, ent[0]))
fin = "("+", ".join(", ".join(str(i) for i in e[1]) for e in sorted(fin)[::-1][:3])+")"
r.sendline(fin)
```
## Results
Running this reaches a classifier accuracy of around 98%. After 100 images, we get the flag, ``inctf{1_D4Y_R0b0tS_will_rul3_th3_w0rld_4_SUR3!!}``
