Veteran marksman Nigel Jemson scored an eight minute hat-trick to almost rescue a sensational point for Shrewsbury.
The 33-year-old striker, who had a brief spell at Gigg Lane, fired home in the 65th, 72nd and 73rd minutes.
But Bury, who had led 4-0 after 56 minutes, managed to hold on for the three points.
Jemson, now plying his trade at his 14th club, left clutching the match ball.
However, the Shrewsbury star admitted: "To score three goals away from home and not get a point is criminal.
"It's nice to get a hat-trick because I probably haven't scored one since my Forest days, but it would have been better had we come away with a point.
"They were really rocking at 4-3 but credit to them for holding out."
Shakers boss Andy Preece was livid with his team for almost letting slip the three points.
"It was embarrassing and I'm not putting up with it," he blasted. "We should have gone on to win the game 5-0 or 6-0.
"But to only win 4-3 and to be hanging on for the last 15 minutes was shocking. It showed we had a soft centre and I'm not putting up with it."
When George Clegg's early penalty was saved by Ian Dumbavin, Bury probably thought it was going to be one of those days anyway.
However, Jon Newby tapped in a simple opener after Lee Unsworth's initial shot had struck the post.
Terry Dunfield, on loan from Manchester City, doubled Bury's advantage within 43 seconds of the re-start.
And the Lancastrians appeared to be well in control when Pawel Abbott netted his second goal in successive games and Newby rammed in the fourth from Abbott's pass.
Newby could easily have had a hat-trick but for Dunbavin's fine save before Jemson's one-man goal show came within a whisker of stealing a point