It was a sweet victory for Vale boss Micky Adams, a boyhood fan of the Bramall Lane side.
United boss Kevin Blackwell was hampered by a big selection headache in the run up to the re-run of last season's first round clash that saw the home side cruise through to the next round.
David Cotterill, Ched Evans and Kyle Walker were away on international duty, six players were absent injured or suspended and on-loan Blackburn keeper Mark Bunn was barred from becoming cup-tied by Rovers.
Vale were beaten 3-1 by United last season but there was to be no repeat.
The visitors went ahead after 20 minutes when Rob Taylor's cross from the left was headed by Loft and Richards pounced with a close range effort to beat Bennet.
But the home side were level 13 minutes later when Billy Sharp produced a clever turn to beat the keeper with an accurate shot.
Vale keeper Martin and his able defence managed to keep United at bay as they looked certain to score in the latter stages of the first half.
Sharp went close again early in the second half but when Vale broke out from the back the earned a free kick outside the area.
This time Richards dipped the shot over the United wall and keeper Bennett looked to have both hands on the ball. But somehow he fumbled and it dropped over the line to United's horror.
Shellshocked United were on the back foot and Richards came close to grabbing a hat trick in the 76th minute but his header from a corner was held by Bennett.