Jamie Grimes headed the Spireites in front on 16 minutes but that was cancelled out by Dior Angus’ finish on 83 minutes.
And Angus struck again in the 94th minute to send the Blues to their first loss in five.
The Spireites had not really been in danger until the last 10 minutes, but a lacklustre second-half cost them at Moss Lane and, just like last week against Oldham Athletic, they threw away a lead.
Chesterfield started on the front-foot, winning three early corners, and Grimes headed in the last of those, meeting Mandeville’s delivery at the back post on 16 minutes for the sixth set-piece goal of the season already.
The Spireites went in search of a second goal, as they continued to enjoy the majority of possession, but they probably would have wanted to have created more chances for the amount of ball they had.
Darren Oldaker started and ended one slick move, curling over from the edge of the box, but other than that clear-cut opportunities were limited.
Branden Horton was being targeted by speedy winger Justin Amaluzor, but Town’s left-back was also causing problems going the other way, and the Blues could have found him in space earlier on a number of occasions.
On the opposite flank, Jeff King, making his first start of the season, was looking bright and sharp down the right, and he made a timely interception on the edge of his own box.
Joe Quigley, who like King was also making his first start of the campaign, was having a good physical battle with Robins centre-backs James Jones and Sam Roscoe, although the away striker was having his shirt pulled frequently, which went unnoticed.
King, Ash Palmer and Armando Dobra were all booked, harshly in the view of those in the away dugout, and in the press box, and Paul Cook’s frustrations resulted in him also receiving yellow by referee Sam Mulhall.
At the start of the second-half, Amaluzor dragged a shot wide for the hosts, while Tom Naylor hooked one over the bar down the other end after King’s free-kick dropped to him about 10 yards out.
By the hour-mark, the Robins had slightly upped the tempo, but Chesterfield were still pretty comfortable, although they needed to be careful not to become complacent.
With just over 20 minutes remaining, Ryan Colclough and James Berry, both former Altrincham players, came on for the Spireites in a bid to liven things up.
With 10 minutes of normal time to go, Ollie Banks replaced Oldaker, as the second-half continued to be uneventful.
But just like against Oldham, you wondered whether Chesterfield might be punished late on, and they were when substitute Angus burst through and fired across Harry Tyrer into the bottom corner on 83 minutes.
And the home side came close to taking the lead minutes later but Tyrer did well to make a block with his legs from Regan Linney as Chesterfield were looking shaky in the latter stages.
Linney then hit the side-netting during five minutes of stoppage time but then Angus swivelled and blasted in a winner in the 94th minute to send Town to their first defeat of the season.
A wake-up call, perhaps, for Chesterfield.
Thankfully, they have a chance to bounce back quickly on Monday against Hartlepool United.
Chesterfield (4-2-3-1): Tyrer; King, Palmer, Grimes, Horton; Naylor, Oldaker (Banks, 80); Mandeville, Dobra (Berry, 68), Jacobs (Colclough, 68); Quigley.
Unused subs: Williams, Grigg.