tl;dr after renaming variables to something readable, for every character of the flag we get two random integers (say a, b) written explicitely as red and green (between 1 and 255 inclusively). Blue is just a/256 * b/256 * flag_character * 10. Depending on whether this product is less than 256, we can easily reverse what flag_character is. It doesn't happen each time. As we have the flag encrypted 100 times, for each position of the flag I count how often different characters appear and select the most common one and this gives the flag.