Veteran defender David Unsworth, making his debut after signing from Portsmouth, scored from the penalty spot after Sky Blues full-back Richard Duffy had brought down midfielder Alan Quinn.
United, who had to bring in Steve Kabba just before the kick-off after striker Danny Webber was injured in the warm-up, almost came unstuck on six minutes, midfielder Gary McSheffrey producing a precise cross from the left that Craig Short stretched to beat Dele Adebola as the big man lurked at the far post.
Isaac Osbourne tried a long-range strike before Coventry swept ahead in the 24th minute. Willo Flood threaded the ball into James Scowcroft's path as the big striker swept past Chris Morgan to beat Paddy Kenny with a clinical finish from eight yards.
Adebola almost snatched a second two minutes later when he barged past Morgan but stumbled as he shot and the United keeper managed to block the low drive with his body.
The Blades were back on level terms on 27 minutes, Quinn carving out the goal as he picked out Kabba on the right flank and the little marksman cut inside stabbing a ferocious shot into the bottom corner in spectacular fashion from 12 yards.
Kabba headed over the top as he raced in to meet a Paul Ifill cross before Unsworth saw his curling free-kick bend the wrong side of the woodwork.
Coventry, playing their part in an action-packed game, found a gap in the United defence in the 41st minute but Flood miscued from ten yards and dragged his shot wide of the near post.
Full-back Leigh Bromby, in space on the right, produced a stinging shot from 16 yards that was palmed over the top by Stephen Bywater and minutes later the on-loan keeper foiled the lively Kabba as he prepared to strike from close range.
Coventry were reduced to ten men in the 70th minute when Matt Heath was sent off for pulling back Kabba after an earlier booking for a carbon copy foul on Shipperley, but it was the impressive Unsworth who had the last word.