The output is output = sin(flag % pi()).

We can get flag % pi() = arcsin(output) using a binary search. Note that there are 2 possible answers.

So, we want to calculate k such that flag - k * pi() = arcsin(output).

Due to the Decimal, these values are calculated in 300-digits precision. That means,

pi          =           3.1415...4120
k           = xxxx...xxxx
k * pi      = wwww...wwww.wwww...wwww
flag        = yyyy...yyyy
flag - k*pi = zzzz...zzzz.zzzz...zzzz = arcsin(output)

To get correct k, determine from least-significant-digit to most-significant-digit of k so that the lower digits of arcsin(output) + k * pi will all be zero. This can be done in integers.

Considering 2 outputs of arcsin and some noise in arcsin, we get some candidates of k, and get a flag.

Original writeup (https://gist.github.com/n-ari/f3caa913483ed5a7831ab04ff6fcef40).