PITTSBURGH -- Overtime finally was the Pittsburgh Penguins' time. Now, the weary, disappointed Washington Capitals' time might be up unless they can stage a remarkable turnaround.

Patric Hornqvist scored 2:34 into overtime and the Penguins seized a 3-1 series lead over the NHL regular season champion Capitals, winning 3-2 in Game 4 of the Eastern Conference semifinals on Wednesday night.

"It's huge, obviously -- 3-1 is a lot more desirable than 2-2," said Penguins rookie goalie Matt Murray, who made 34 saves. "But we know we've got a lot of work left to do. ... We know nothing's done."

Game 5 will be Saturday in Washington.

The Penguins, winners of 21 of 25 dating the regular season, had lost eight consecutive overtime playoff games until Capitals defenseman Mike Weber -- a Pittsburgh native -- accidentally threw the puck on Hornqvist's stick for a wrist shot that beat goalie Braden Holtby through his legs.

"It was obviously great to see the puck go in, we win the game and we're up 3-1," Hornqvist said. "I tried to shoot it as hard as I could and it went 5-hole."

Weber, back in the lineup for the first time since the opening-round series against the Philadelphia Flyers, was trying to get the puck out when it flew directly to Hornqvist in the right circle for his fifth goal of the playoffs and first playoff overtime winner.

"It's what probably any defenseman would have done in that situation -- the puck's rolling around and you're trying to poke it out of the danger zone," Capitals defenseman Karl Alzner said. "It just so happens the guy (Hornqvist) is flying in there."

Capitals defenseman Taylor Chorney added, "It squirted to somebody who was open. It was unfortunate."

Murray withstood a Capitals goal by center Jay Beagle less than three minutes into the game to win for the 13th time in his last 14 starts -- and the Penguins withstood the loss of star defenseman Kris Letang to a one-game suspension.

"We didn't take advantage of it (Letang's absence)," Capitals coach Barry Trotz said. "Now we've dug ourselves a hole. We'll see if we can dig ourselves out."

Capitals star Alex Ovechkin, who didn't score despite getting seven shots on net, said the Capitals know a comeback from a 3-1 deficit can be achieved -- the New York Rangers accomplished it against them last season.

"We've got to play our game. We got to go period by period and try to turn it around, win the next game," Ovechkin said.

The Penguins, who lost 3-1 series leads to Tampa Bay in 2011 and the Rangers in 2014, have won seven of eight previous playoff series from the Capitals.

But this was supposed to be the best Capitals team ever -- they put up 120 points during an NHL season in which no other team managed even 110. But now, the Capitals must win three straight against a surging opponent to keep their season going.

"It's not the best place to be," Washington defenseman John Carlson said. "But we're proud of our game and, at some point, we've got to dig deeper and work harder and find a way, no matter what."

All four games in the series have been decided by one goal

"They're going to respond (in Game 5)," Penguins defenseman Trevor Daley said. "They're going to be tough and we have to match it."

The Capitals got exactly the start they wanted after losing 3-2 in Game 3 despite outshooting the Penguins by 2-to-1, as Beagle put a bad-angle backhander from along the goal line past Murray only 2:58 into the game.

But instead of building on the momentum created by Beagle's third goal of the postseason, the Capitals allowed the Penguins to wrestle it away.

Daley tied it midway through the first by beating Holtby between the pads on a 40-foot wrist shot. Daley got loose on a Penguins breakout created when they broke Washington's three-man forecheck.

The Penguins' timeless Matt Cullen -- he's six months away from 40 -- made it 2-1 early in the second. Center Oskar Sundqvist beat Ovechkin to the puck off a Penguins faceoff win, and rookie Tom Kuhnhackl deflected Brian Dumoulin's pass off the boards directly to Cullen.

But the absence of Penguins breakout specialist Letang -- he created two goals with stretch passes in Game 3 -- might have hurt on Carlson's fourth goal of the playoffs.

Pittsburgh couldn't clear the puck despite repeated chances, allowing Justin Williams to throw a pass to Carlson for a short wrist shot from the right circle that beat Murray under the crossbar to the glove side at 16:19.

Penguins star Sidney Crosby, who picked up his first point of the series by assisting on the Daley goal, went to the dressing room briefly in the third period after taking an Ovechkin stick to his left wrist -- and Crosby angrily broke his own stick in frustration as he walked down a rampway.

Crosby returned a few minutes later and drew an Alzner high-sticking penalty at 16:22, but the Penguins didn't manage a shot during the Capitals' 23rd consecutive penalty kill. Pittsburgh is 0-for-14 with the man advantage in the series.

But every other advantage in the series belongs to the Penguins.

"I like how the entire team responded -- we got the win and that's all that mattered," Murray said.

NOTES: D Olli Maatta (upper-body injury) sat out a second successive game, leaving the Penguins without their top defensive pair as Kris Letang served his one-game suspension. D Justin Schultz dressed for the first time since Game 1 against the Rangers. ... Capitals LW Marcus Johansson, who absorbed a shoulder-to-head hit from Letang on Monday, played after not practicing Tuesday. ... The Penguins also were without C Eric Fehr (undisclosed injury), who had the game-winning goal in Game 2, but RW Bryan Rust, hurt while blocking multiple shots in the first minute of Game 3, was in the lineup. ... Capitals coach Barry Trotz changed his lineup, dressing Pittsburgh native D Mike Weber for the first time since Game 6 against the Flyers and sitting down D Nate Schmidt. Weber started the game with Taylor Chorney on the Capitals' No. 3 defensive pairing. ... Pittsburgh called up F Tom Sestito and D Steve Oleksy from its AHL farm club, but neither dressed. ... The Capitals scratched Schultz, C Michael Latta and RW Stanislav Galiev. The Penguins held out Maatta, Fehr, Sestito, Oleksy and No. 3 G Jeff Zatkoff.