The home side ran their First Division opponents close, and Dennis Wyness could have opened the scoring as early as the third minute, but goalkeeper Lee Robinson saved well from his lifted effort.
Derek Holmes spurned Queens' best chance of the first half when he headed wide from a Willie McLaren cross.
On the stroke of half time, Peterhead captain Paul Emslie crashed a 20-yard effort off the crossbar, with Robinson well beaten.
Kenny Brannigan's men eventually opened the scoring after 66 minutes. Ryan Conroy's cross got caught under the feet of Holmes, but broke to Craig Reid who fired past the helpless Jon Bateman.
It was 2-0 for the Doonhamers just a minute later. The excellent McLaren played in Holmes, who found the corner of the net with a calm finish.
Peterhead gave themselves a lifeline when a neat move on the left ended with Daniel Moore crossing for Bavidge to loop a header past Robinson.
McLaren should have finished the contest but delayed his pass to Conroy and the chance was gone. With just a minute remaining the Blue Toon could have gone level, but Robinson dived full length to push a free-kick round the post.
This was a keenly fought contest and great credit must be paid to Peterhead who played out of their skins although at the end of the day perhaps Queens had that little bit extra in class.
Queens Man of the Match - Magic Johnston - He bossed the midfield and took the sting out of the Blue Toon with his calm play.