United looked destined for a third point from seven games on their return to the top flight after Yakubu had cancelled out Rob Hulse's 35th-minute opener shortly after the break.
But Jagielka, who scored an own goal in a 3-0 loss at Arsenal last week, emerged the hero with a 30-yard cracker in the 91st minute.
By the interval, the Blades fans could scent victory such was the performance in the opening 45 minutes of their side.
Admittedly, United were fortunate not to have been at least a goal adrift by the time Hulse struck, but they rode their luck to great effect.
There appeared more effort and determination, while their passing and movement was evident, although opportunities were still at a premium.
The errors in defence still need to be eradicated if United are to have any hope of avoiding relegation as a more incisive side than Boro will capitalise on the chances that went begging.
Goalkeeper Paddy Kenny, returning after a three-match absence with a thigh injury, spilled a low, raking 30-yard free-kick from Fabio Rochemback.
That was after Leigh Bromby had received a seventh-minute booking for a foul on Stewart Downing, who was back in the Boro line-up following his own three-match lay-off with a groin problem.
Fortunately for Kenny, striker Yakubu was flagged offside as he attempted to latch onto the loose ball.
Seven minutes later and Michael Tonge, a surprise inclusion as he had missed the last three games with an ankle problem that may yet require surgery in midweek, was caught in possession on his own deadball line.
Lee Cattermole robbed the midfielder, but was twice thwarted in his attempt to cross the ball into the area by superb defensive work from Claude Davis.
Then Boro underlined why they find themselves struggling near the foot of the table as Yakubu and captain Jonathan Woodgate missed what can only be described as 'sitters'.
In the 21st minute, Yakubu was released by Cattermole, beating United's offside trap and leaving him with only Kenny to beat. The Nigerian appeared to do the hard work in rounding the Republic of Ireland international.
But with the goal at his mercy, he struck the right-hand post with an angled shot that went into touch off Jagielka as he attempted to intercept.
Just 60 seconds later, the Blades defence went AWOL, with Woodgate given the freedom of the box as Downing lofted in a left-wing cross, only for the centre back to sidefoot wide from eight yards.
United eventually made Boro pay for their sloppiness in front of goal by capitalising on a breakaway move that culminated in Colin Kazim-Richards setting clear to his right an overlapping Hulse.
After one touch to steady himself and, despite the attentions of a backtracking James Morrison, Hulse dinked a 12-yard shot over Mark Schwarzer for his third goal of the season.
Bramall Lane erupted and for the next 10 minutes United cruised, with Hulse and Kazim-Richards both enjoying further chances before the break.
But the fortune that had favoured United in the first half then disappeared within four minutes of the restart.
A booking for Cattermole for a clumsy challenge on Chris Armstrong gave United possession midway inside the Boro half.
After a short lay-off from Tonge, Davis pumped the ball forward to the edge of the area, only for a booming sidefoot clearance from Emanuel Pogatetz to find Yakubu on the halfway line.
Racing clear, and again with only Kenny to beat, Yakubu made no mistake on this occasion as he fired a right-footed drive from 16 yards beyond Kenny.
It was his second goal of the season, and first since the opening day of the campaign in a 3-2 defeat at Reading.
United, though, were far from demoralised, and instead continued to take the game to their opponents, epitomised by a superb move in the 65th minute.
Tonge and Quinn combined in midfield, with the latter teeing up Keith Gillespie for an angled drive Schwarzer saved low down to his near post before his defence cleared the rebound.
Warnock then went for broke, bringing on strikers Danny Webber, Steve Kabba and Christian Nade in the closing 13 minutes.
But it was centre-back Jagielka who stole the show, lashing a 30-yard half-volley past Schwarzer after taking an Andrew Davies' clearance on his chest.