The 58-year-old judge was so embarrassed that he appeared to swear after the result was revealed.
Simon had earlier gushed about how great the prize to work with a top producer would be, knowing that his own popular act were in the running.
Host Dermot O’Leary said: “Welcome back to The X Factor. It’s time to find out who has won the prize fight. So let’s welcome back your weekend champions Rak-Su and Kevin.
“Well done gentlemen, congratulations, through to next week. Let’s find out who’s won the prize fight. Okay, it’s time to reveal who you chose for that money can’t buy prize. Good luck to both of you. The winner of the prize fight is Kevin.”
Simon then stood up and flung himself forward over the desk seemingly shouting: “Bo****ks!”
Luckily for Simon, his microphone was not live and he could not be heard.
However, that didn’t stop eagle-eyed viewers at home rushing to social media to berate the music mogul.
One viewer tweeted: “Simon was SO rude to mouth ‘bo****ks’ when Kevin won the prize fight! He deserved it just as much Rak-Su! #XFactor.”
“‘BO****KS!’ My thoughts exactly Simon! #XFactor,” posted another.
A third asked: “Did Simon just shout bo****ks then? #xfactor.”
“Simon Cowell, producer of the show, didn’t realise the camera would be on him when he said ‘bo****ks’ when Kevin won. #xfactor2017 #xfactor,” noted a fourth.
The X Factor continues next Saturday on ITV at 7pm.