BALTIMORE -- Even though Baltimore catcher Matt Wieters had a scheduled day off, he was constantly thinking about ways he could make a difference if his number was unexpectedly called.

That moment came with the game on the line and he delivered in a big way.

Wieters hit a pinch-hit single in the eighth inning off New York closer Aroldis Chapman that scored three runs, lifting the Orioles to a 3-1 victory over the Yankees on Sunday.

Baltimore took the three-game series by also winning the opener Friday.

"We just feel like we're playing nine innings," Wieters said. "We don't look at as if we're playing behind or being ahead. We're playing 27 outs and we're playing nine innings and we want to be ahead when that 27th out is recorded."

New York's CC Sabathia labored through five-plus innings, but managed to keep the Orioles scoreless. He allowed just two hits with five strikeouts and six walks on 111 pitches. He also threw seven shutout innings in Baltimore on May 4 and was looking for his 20th career win against the Orioles.

"It was a grind," Sabathia said. "That put some good at-bats on me. The fouled off some pitches, But I was able to make some pitches when I needed to. It was definitely a battle."

The Orioles finally broke through in the eighth when Mark Trumbo walked and Chris Davis singled off Dellin Betances. After Betances (2-4) struck out Nolan Reimold, the game was delayed because of rain.

When play resumed after 1 hour and 37 minutes, Chapman entered for New York. He struck out Schoop before Francisco Pena lined a single to right to load the bases. Wieters then hit a single up the middle that scored two runs and a throwing error by Jacoby Ellsbury allowed the third runner to score.

T.J. McFarland (1-1) picked up the win with two scoreless innings of relief, while Zach Britton earned his 17th save. Chapman blew his first save of the season. The Yankees had been 25-0 when leading after seven innings.

"I was hoping these guys would get home and have dinner with their families and do whatever people still do while it's still daylight," Orioles manager Buck Showalter said. "Try to act normal. What I'm saying is it will be forgotten pretty quickly. They should take some pride in the finished product today."

Baltimore's Kevin Gausman remains winless in nine starts, despite another effective outing where he got no run support. He allowed one run on seven hits with five strikeouts and two walks in six innings.

"It's awesome," Gausman said about the win. "Ever since I've been with the Orioles that's kind of been our motto. Late in the game, if we are close, it's like we like to make it dramatic. We are really good against those guys at the back end of the bullpen."

The Yankees took a 1-0 lead on an RBI single by Alex Rodriguez that scored Ellsbury, who moved into scoring position on a wild pitch by Gausman. The Yankees threatened again in the fourth, helped by an error on Manny Machado that put runners on first and second. Gausman walked Rob Refsnyder to load the bases before striking out Ellsbury to get out of the jam.

New York ran itself out of a potentially big inning in the fifth when Gardner led off with a double but was thrown out trying to take third on a grounder by Rodriguez. The Orioles loaded the bases in the bottom half of the inning, but Sabathia struck out Trumbo for the third straight time to remain unscathed.

"Both teams had a lot of opportunities," Yankees manager Joe Girardi said. "We had a lot of opportunities but weren't able to get the big hit. They got the big hit at the end and that's why we lost."

NOTES: Yankees C Brian McCann was back in the lineup for the first time in three days. He had hyperextended his left elbow last week against Toronto and was day-to-day heading into the three-game series in Baltimore. ... Orioles SS J.J. Hardy (left foot fracture) was able to play catch from 90 feet for the first time during his rehab. Hardy was placed on the disabled list on May 3, retroactive to the previous day. ... New York C Austin Romine suffered a torn nail on his left thumb in the ninth inning of Saturday's game. However, Romine told manager Joe Girardi that he was available if needed. "It's good. A little sore," Romine said. ... Baltimore C Francisco Pena got his second start since being recalled from Triple-A Norfolk on May 31. It was a special day because Pena's father and former catcher, Tony Pena, is the first base coach for New York.