United keeper Paddy Kenny had kept Coventry at bay with a string of saves, before Sharp finally got off the mark in the 69th minute and the fast-moving game ended in a goal flurry as United increased their lead from the penalty spot, before City grabbed a lifeline nine minutes from time.
United escaped in the fifth minute when Scott Dann connected with Kevin Thornton's free-kick, Kenny palmed his angled strike away. The big centre-back had a second attempt, but this time was foiled by Gary Speed as he cleared off the line.
The Sheffield side squandered a chance when James Beattie beat Andy Marshall to the ball, but the striker turned away as it rolled towards the line and Danny Fox raced in to clear.
Coventry went close again, Michael Mifsud producing a stunning volley which almost found the target as it took a cruel deflection off team-mate Elliott Ward, with United keeper Kenny smothering the ball on the line.
City's lone striker Leon Best was stopped by Kenny who dived bravely at his feet and when Mifsud's low strike was deflected by Matthew Kilgallon, the United keeper conjured up another sparkling save to keep the Midlands side at bay.
United went ahead against the run of play, Sharp heading in a David Cotterill cross.
The Sheffield side's second goal came in the 78th minute after Stephen Hughes elbowed Sharp in the area and Speed scored at the second attempt after Marshall blocked his spot-kick.
Ward gave Coventry hope in the 81st minute, smashing his penalty kick into the roof of the net after Chris Morgan had hauled back Mifsud in the box.