The Bramall Lane side, beaten twice by Brian Laws' side last season had raced to a 3-0 half-time lead as Wednesday crumbled.
Substitute Etienne Esajas transformed the Hillsborough side, replacing the ineffective Jermaine Johnson and scoring with a spectacular free-kick as Wednesday clawed their way back into the game.
United had started the match in dramatic fashion racing to a two-goal lead with just 15 minutes gone.
Wednesday keeper Lee Grant foiled United new boy Andrew Davies tipping his close-range header over the top, before the home side snatched the lead in the seventh minute.
Winger Keith Treacy curled in a teasing corner, centre-back Chris Morgan headed down and Jamie Ward steamed in to finish with a right-foot shot from close range past the helpless Grant.
Wednesday had hardly caught their breath before United raced further ahead on 15 minutes. Kyle Walker's long throw reached Ched Evans, the £3million striker drilled in a low shot and Darius Henderson, with his back to goal, steered the ball beyond Grant.
Marcus Tudgay went close as he rose to meet Michael Gray's free-kick from the left which was palmed over the top in spectacular fashion by Mark Bunn.
United suffered a blow when Ward was stretchered off in the 33rd minute with a leg injury and David Cotterill came off the bench to take over his role on the right wing.
And it was Cotterill who carved out United's third in the 42nd minute as their more direct approach caused havoc in the Wednesday area.
The former Wigan player sent over a teasing cross, the unfortunate Lewis Buxton stretched to clear under pressure from Henderson and headed into his own goal.
Wednesday came out fighting after the break. Gray saw his effort deflected over the bar by the combined efforts of Morgan and Bunn before Wednesday hit back in the 46th minute.
Dutchman Esajas saw his header blocked and Tudgay raced on to the loose ball to lash an angled drive into the far corner.
Wednesday almost had a second as United reeled, Esajas' stinging free-kick was cleared and the ball fell to the waiting Tudgay, who lashed it over the top from ten yards.
Esajas made it 3-2 in the 66th minute after Cotterill had fouled Richard Wood 20 yards out, the winger hitting his free-kick through the wall into the bottom right-hand corner to set up a nail-biting finish.