The Blades looked to be heading for a shock win until centre-back Claude Davis and keeper Paul Gerrard got in a tangle and skipper Gary O'Neil nipped in to steer the ball over the line from close range.
Striker Jon Stead, making his debut after being snapped up from Sunderland for £750,000 just 48 hours earlier, was handed a chance after just 39 seconds. Derek Geary found space on the left to pump over a curling cross but from six yards the striker steered his header straight at David James.
Andy Cole fired over the bar from just inside the box before Stead broke free on the right but his low pass inside was cut out by the sliding Glen Johnson and James tidied up with Rob Hulse waiting to pounce.
The home side almost came unstuck in the ninth minute when Portugal midfielder Pedro Mendes picked out Kanu at the far post with a delicate free kick, but the ex-Arsenal striker steered the ball into the goalmouth and United cleared for a corner.
Defender Linvoy Primus was booked for a foul before the Sheffield side swept ahead after the big defender brought down Chris Lucketti 20 yards out. Midfielder Stephen Quinn stepped up to curl his free-kick beyond the former England keeper.
Former England centre-back Sol Campbell volleyed over the target as he ran on to a Mendes cross and Matthew Taylor had a low drive deflected by Rob Kozluk as Portsmouth threatened to over-run the Sheffield side.
Cole saw his fierce strike pushed round the post by Gerrard before United caught Portsmouth on the break. Stead took on three defenders and pumped in a low cross but James dived in to snatch the ball from under the feet of Hulse.
Portsmouth were back on level terms in the 81st minute. Davis should have cleared as substitute Benjani Mwaruwari charged in and with Gerrard flat-footed, O'Neil stabbed the loose ball over the line from close range.
United almost snatched a winner in injury-time when Lucketti headed back into the goalmouth but Davis saw his close-range strike bounce away to safety off the post.