Hart has been charged with trying to save Palace from the drop and kicked off his tenure with a priceless victory thanks to Alan Lee's 23rd-minute effort.
He said: "From the minute I came in, the way the players behaved and spoke told me they are a strong bunch and they went out there and proved it.
"I was absolutely delighted for them. I think the performance was excellent. All over the pitch we did what was necessary.
"Sheffield United, whether they are in the Premier League or the Championship, are a very tough team and you always know you have been in a game against them."
Hart clearly likes a challenge. He was at Portsmouth at the start of the season and even had a brief stint at Loftus Road during Flavio Briatore's controversial reign.
Palace are only fighting a relegation battle after being docked 10 points for going into administration, but the 56-year-old insisted: "I'm just grateful to get a job.
"It's just this year it looks as if I like a challenge, but all three clubs I've worked at this year are great clubs with lots of history.
"I have been very lucky to have been asked to manage those clubs.
"I haven't attempted to make massive changes. Everybody wants to keep this club in this division, but it's hard when you are down there."
Palace were well worth the points as they dominated for virtually the whole match and only looked like surrendering their lead in the dying seconds when United eventually showed some attacking intent.
Lee's goal came midway through the first half when the former Republic of Ireland striker stole in ahead of Chris Morgan to fire home from Danny Butterfield's cross.
It was the only notable incident of a dire first half, but the Eagles started the second half strongly.
Darren Ambrose spurned a good chance eight minutes after half-time when he picked up Shaun Derry's pass only to fire straight at Blades goalkeeper Mark Bunn, while Lee missed a great opportunity to grab his eighth goal of the season moments later, nodding Nick Carle's cross over the bar.
Sheffield United have now lost six consecutive away matches in all competitions as their hopes of reaching the play-offs suffered another setback.
Manager Kevin Blackwell said: "I am frustrated, but it's always tough facing a new manager and a new team and I thought they fought like dogs.
"In the first half they battled well and in the second half, although we dominated, we didn't create enough."
Source: Team Talk
Source: Team Talk