PITTSBURGH -- The Cincinnati Reds took a four-run lead on the Pittsburgh Pirates only to lose Tuesday as the Pirates scored seven runs in one inning.

The Reds again took a four-run lead, this time in their first at-bat, and made it count by hitting when it counted most. They scored all of their runs with two outs and beat the Pirates 5-2 Wednesday.

"Any time you can add on and extend that lead it's big," manager Bryan Price said. "The two-out hitting has been big, for sure."

Marlon Byrd went 2-for-4 with a home run and three RBIs to lead Cincinnati's 12-hit attack. Right-hander Mike Leake allowed only two runs and eight hits in seven innings.

Cincinnati improved to 6-2 against the Pirates in 2015, the Reds' best win total against any club.

Leake (5-4) walked none and struck out five.

"He did a super job," Price said. "Mike's very familiar with their lineup; that being said, they're familiar with him, too. Mike just kept making pitches."

Right-hander J.J. Hoover worked a scoreless ninth to pick up his first save of the season.

Pirates right-hander Gerrit Cole did not pitch through the fifth inning for the first time this year. Cole (11-3) was charged for a season-high five runs and eight hits in 4 2/3 innings. He struck out five and walked three.

"I didn't bring out a good delivery tonight," Cole said. "Stuff was a little flat. I didn't make pitches when I needed to make pitches in the first."

Two of Cole's three losing decisions on the season have come against the Reds, but that doesn't mean Price looks forward to when his team has to match up against the major-league wins leader.

"There's certain starters in the league where you're going, 'I hope that guy ends up pitching against us because we hit him really, really good,'" Price said. "But Gerrit Cole is not one of them. We beat him typically in low-scoring games, if we beat him."

Cincinnati center fielder Billy Hamilton led off the game off with a walk and stole second and third base. After second baseman Ivan De Jesus Jr. and first baseman Joey Votto struck out, third baseman Todd Frazier doubled to score Hamilton. Right fielder Jay Bruce followed with a double to drive in Frazier.

Byrd then hit the first pitch he saw over the center-field fence for a two-run homer and a 4-0 Cincinnati lead.

"It was big to get on him early because once he starts cruising, he just doesn't let up," Byrd said. "So we needed runs early."

Pittsburgh scored a run in the second when second baseman Neil Walker led off with a double, moved to third on third baseman Josh Harrison's sacrifice and came home on first baseman Pedro Alvarez's groundout.

Byrd drove in another run with a two-out single in the fifth that knocked Cole out of the game.

The Pirates answered in the bottom of the fifth. Alvarez led off with a single and advanced to third on a hit-and-run single by shortstop Jordy Mercer. Catcher Chris Stewart grounded into a fielder's choice that scored Alvarez and cut Cincinnati's lead to 5-2.

The Pirates advanced runners to second and third against Leake with two outs in the seventh, but he struck left fielder Starling Marte out swinging to end the threat.

"(Leake) stays away from the middle of the plate," Pirates manager Clint Hurdle said. "He works the corners, he keeps the ball down, he can spin the ball. He's very crafty, he competes well."

Center fielder Andrew McCutchen walked leading off the eighth but was stranded at second as left-hander Manny Parra struck Alvarez out to end the inning.

Pittsburgh went 0-for-9 with runners in scoring position.

"We just haven't been able to throw that ball in the gap when we need one," Hurdle said.

NOTES: Pittsburgh 1B Corey Hart (shoulder) was placed on the 15-day disabled list before the game. RHP Chris Volstad took his place on the roster. ... Cincinnati 2B Brandon Phillips (thumb) was scratched from the lineup. ... Reds 2B Ivan De Jesus Jr., a former farmhand in Pittsburgh's organization, made his second start of the season in place of Phillips.