Alan Lee sealed the win with his seventh goal of the season when he volleyed in Danny Butterfield's deep cross midway through the first half.
The Blades were poor and never really tested Palace goalkeeper Julian Speroni.
Hart, named as Neil Warnock's successor on Tuesday, was give a rapturous round of applause from the home fans before the game and sprang a surprise by naming Calvin Andrew in the starting eleven alongside fellow striker Lee.
In a dour start to the game there was little action of note until Lee put Palace ahead on 23 minutes.
United failed to clear Butterfield's free kick and Nick Carle laid the ball back for the right-back to supply the cross for Lee to score in a move reminiscent of Neil Warnock's tenure at the club.
Man-of-the-match Carle then put Andrew through two minutes later but the striker lost his cool and fired high and wide.
United threatened for the first time in the 41st minute when Mark Yeates' free-kick got a slight touch off Jamie Ward but Speroni gathered easily under pressure from Richard Cresswell.
Darren Ambrose had a chance to add to Palace's lead just after the break after Shaun Derry played him through but his shot from the angle was weak, enabling Bunn to gather.
Minutes later it was Lee's chance to give Palace a two-goal cushion.
Carle twisted past Toni Kallio twice and crossed to the back post but with Lee leaning backwards, he skied his effort over the bar.
Kevin Blackwell brought on Henri Camara in an effort to liven up United's forward line but they still struggled as Claude Davis and Clint Hill dominated.
Lee was lucky to stay on the pitch after aiming an elbow at Nick Montgomery but escaped without even a word from the referee before Claude Davis, Kyle Bartley and Lee Williamson were all booked for pushing.
United pushed forward in the last five minutes but Camara shot over and then got in the way of Ched Evans' shot.
Evans had another chance but his effort was a foot wide of the post as Palace held on to ease their relegation worries.