After having a look in ida,
I deduced that the executable reads the entered password  and calculates  the following:

Some pseudocode:
for each password attempt {
long x=0;
long x=(x*128h) + (each hex value of  the password chars) }
So the password is  roughly  6 or 7 chars in length , because final value from calculation  must equal :CEFF5331D4AAh

finally it checks the value of x;
If the value of the x ==CEFF5331D4AAh (the value of the EDX:EAX registers ,denoting a long value) , we have a match.
I thought the best approach was to try to brute force the password.
So I wrote a java program and used lowercase alpha and digits to try to determine the password .
The password  was "d00m3r" 
flag was : xiomara{MD5(password)}  
flag=xiomara{48c92083dc430eb4e8af78a38f9cc877}

excerpt from java program:

long x=0;
for (int i=0;i<password.length();i++) {             
     x=(x*0x128)+ (byte)(password.charAt(i));          
     if (x==0xCEFF5331D4AA) {                   
             System.out.println(password);                    
             System.exit(0);<span>             
     }
</span>}