Rating: 3.0
TL;DW. But if you really want to know, see Writeup from team SwissCyberStorm
This cipher behaves like a stream cipher, the only things affects stream key is biggest byte in the plaintext.
So we enumerate all possible biggest byte with enough length to extract stream key, then try to decrypt flag.