Arsenal took the lead at Vicarage Road just before half-time courtesy of a Per Mertesacker header but they fell apart after the restart.
Hector Bellerin was the architect of the Gunners downfall, gifting Watford a way back into the game from the penalty spot following a foul on Richarlison in the 70th minute.
Troy Deeney slotted home the spot-kick and sparked the home side into life who later snatched a winner in added time through Tom Cleverley.
However, Arsene Wenger was not convinced by Watford’s penalty claims and branded referee Neil Swarbrick’s decision to award them a penalty “scandalous”.
“We panicked a little bit in the last seconds,” he told BBC Sport.
“You can talk and talk and talk [about the penalty].
“They scored. It was a scandalous decision and there is little we can do about it.”
The result has seen Watford leapfrog Arsenal into fourth place, with the Gunners slipping to sixth.
However, goalscorer Deeney has admitted the Vicarage Road club are not getting carried away, despite their strong start to the campaign.
“It means nothing, we’re taking it game by game,” the striker said.
“We’re building something really good. We got a last minute winner at Swansea, a last minute equaliser at West Brom and another late winner here.
“It shows hows we have belief. It’s a massive team effort. A great win.”
