PITTSBURGH -- Behind by the third pitch of the game and by two runs twice, the Pittsburgh Pirates had to scramble. It's something they take almost in stride.

"If you want to be an elite team, you've got to come back and win games," Pittsburgh manager Clint Hurdle said Sunday after his club forged two ties, fell behind in the 10th and then won 6-5 on Starling Marte's walk-off, two-run homer in bottom of the 10th for a weekend sweep of the Atlanta Braves at PNC Park.

Atlanta took a 2-0 lead in the first and later held a 4-2 lead.

"We were only down by two. It felt like you might have been down my more, but you weren't," Hurdle said. "That's the thing we kept reminding ourselves."

Freddie Freeman went 4-for-5 with two solo homers, and Dansby Swanson added another solo homer for Atlanta (1-5), which lost its fourth in a row.

"It's a tough one to swallow," Freeman said. "We haven't been playing very good baseball to start the season off."

Felipe Rivero (1-0) pitched the 10th for Pittsburgh (3-2) to pick up the win despite giving up a go-ahead run. Jose Rameriz (0-1) pitched the 10th for Atlanta.

In the 10th, the Braves took a 5-4 lead when Nick Markakis scored on a grounder.

In the bottom of the inning, Adam Frazier led off with a double to deep center field. Marte hit his fourth career walk-off homer toward the same area, just a little deeper.

"I tried to pick a good pitch. I tried to get a good swing, and then that happened," Marte said. "I had a feeling. I knew I got barrel. The wind ... helped me a little bit. I knew I got that one."

Braves starter Julio Teheran gave up two runs -- both unearned -- on six hits, with four strikeouts and one walk over seven innings.

Pittsburgh starter Gerrit Cole allowed runners in each of his six innings. He allowed three runs on eight hits, including two homers, with four strikeouts and two walks.

"Location could have been more crisp today, really throughout," Cole said. "It was compounded by three mistakes in the first inning. But (I) bounced back and got some length, so that was nice."

It was the second day in a row the Pirates erased a deficit to beat Atlanta. Cole attributes it to a club identity.

"This is part of what we pride ourselves on. It's been a few years in the making," he said. "We try to stay consistent with it."

Cole struggled from the outset. He got Ender Inciarte to fly out on his first pitch, but on his third Swanson hit a homer to left for a 1-0 Braves lead. Later in the first, Nick Markakis' two-out RBI single drove in Freeman to make it 2-0.

Teheran, meanwhile, pitched to just one batter over the minimum through three innings. In the fourth, Marte singled and moved to second on Freeman's error at first after Teheran's pickoff attempt caught Marte off the bag. Two outs later, Josh Bell doubled home Marte. Jordy Mercer followed with an RBI single to tie it 2-2.

Freeman hit a one-out, first-pitch homer to left in the fifth to give Atlanta a 3-2 lead.

Cole worked out of another tight spot in the sixth, striking out Inciarte with two out and runners on second and third. The strikeout was completed on Cole's 98th and last pitch.

Freeman hit his second homer, to right, off of Pittsburgh reliever Antonio Bastardo with one out in the seventh for a 4-2 lead.

The Pirates pulled to within 4-3 in the eighth when reliever Arodys Vizcaino hit Josh Harrison with one out, Frazier singled to left, and left fielder Jace Peterson misplayed Marte's single, allowing Harrison to score.

Pittsburgh was not unhappy to see Teheran finish his day.

"Any other option after he was taken out was going to be better for us," Hurdle said. "He was tough hitting today."

Gregory Polanco led off the ninth with a single to left off of reliever Jim Johnson and advanced to second on a passed ball. Josh Bell walked, with Alen Hanson coming in as a pinch runner. After Mercer popped out, pinch-hitter David Freese walked to load the bases for pinch-hitter Francisco Cervelli.

Cervelli's fielder's choice -- broken up at second by Freese -- allowed Polanco to score for a 4-4 tie.

Markakis gave Atlanta a short-lived 5-4 led in the 10th when he singled and later scored from third on Tyler Flowers' grounder.

What happened in the Pirates' half of the 10th had Atlanta manager Brian Snitker feeling for Teheran.

"He was really good, really good," Snitker said. "Had the game right there.

"Just can't give away that many runs, that many extra outs and expect to win. We've got to make better plays."

NOTES: 2B Phil Gosselin made his first start with the Pirates. ... 3B David Freese was out of the Pittsburgh starting lineup for the second day in a row after taking a ground ball to the collarbone on Friday but entered the game in the ninth. ... LF Matt Kemp remained out of the Atlanta lineup after leaving Friday's game because of right hamstring tightness. ... This was the sixth game of a season-opening eight-game, 10-day road trip for the Braves.