With the scores around the country going against them, Southampton were in danger of going down after 65 minutes when Sheffield United's Jonathan Stead made it 2-2.
But Stern John scored his second of the match to put the Saints ahead again after 69 minutes and that is the way it stayed.
Southampton had to hold out under some intense pressure in the last 15 minutes after John had been sent off for a second bookable offence.
Sheffield United started the day with an outside chance of reaching the play-offs, but they were without four key players including ex-Saints striker James Beattie, and they were never at their best.
Even so, they took the lead in the 24th minute when Stephen Quinn was left unmarked to head in a right-wing centre by Billy Sharp.
Southampton had plenty of chances to draw level but Paddy Kenny made a great save from Inigo Idiakez before Southampton drew level three minutes before half-time.
Jhon Viafara's centre from the left was deflected to Marek Saganowski to head in virtually on the goal-line.
John put Southampton ahead in the 53rd minute when he was played clear by Saganowski but in the process of celebrating was booked for taking off his shirt.
The Blades drew level 12 minutes later when Luton Shelton centred from the left for Stead to score at the near post.
John's 19th goal of the season came in the 69th minute when he turned away from his marker to beat Kenny with a low right-foot shot.
But 11 minutes later he was shown another yellow card and then a red by referee Phil Joslin for a late tackle on Derek Geary and Southampton had to survive a tense final 15 minutes before they could be sure of staying in the Championship.
United did all they could to force a draw but could not find a way past Richard Wright in the Southampton goal.