ATLANTA -- The Buffalo Bills showed the Atlanta Falcons their defense is for real and Stephen Hauschka made sure that the effort paid off with a victory on Sunday.

Hauschka kicked fourth-quarter field goals of 56 and 55 yards and the Bills stopped Atlanta on fourth down at the 10-yard line with 44 seconds left for a 23-17 victory over the previously unbeaten Falcons at Mercedes-Benz Stadium.

"It's the same leg swing whether it's a long kick or a short kick," said Hauschka, who connected from 24 yards early in the final quarter. "But when it's long, the leg swing sometimes is freer and it felt good both times. It's always a nice feeling when you come through."

The Bills got a 55-yard touchdown return of a Matt Ryan fumble by Tre'Davious White and had two interceptions against the Falcons quarterback, who played much of the game without leading receivers Julio Jones and Mohamed Sanu.

"We knew coming into it that they had a very good defense," Ryan said. "They have a lot of talent and we just made too many mistakes out there."

The Bills (3-1) have allowed just 52 points -- the least in the NFL -- and have gone a franchise-record 15 quarters without a turnover, making for an impressive start to former Carolina defensive coordinator Sean McDermott's first year as Buffalo head coach.

"I'm happy for the team. This is a tough place to play," McDermott said. "It's a great win. What more can you say?"

Ryan, who was 24 of 42 for 242 yards throwing mostly to backup receivers, connected with Justin Hardy for a 3-yard touchdown with seven minutes left to cap a 75-yard drive and tie the game.

The Bills weren't to be denied, though.

Hauschka gave the Bills the lead back on his 56-yard kick with 4:44 left and then hit from 55 yards with 3:06 remaining after an interception by Micah Hyde.

White's long return of an upheld Ryan fumble early in the third quarter was the turning point for the Bills against the Falcons (3-1).

Jerry Hughes hit Ryan's arm as the quarterback was about to pass and the ball squirted forward, beyond the line of scrimmage. White picked up the ball and raced down the field as the Falcons stood stunned.

Falcons head coach Dan Quinn challenged the fumble ruling, but the call was upheld.

"It was uncharacteristic sloppiness for us, with a lot of missed opportunities," Quinn said of his team's overall performance.

Hauschka kicked his 24-yard field goal with 12 minutes left in the game to climax a 19-play drive of 82 yards that took more than 11 minutes and put the Bills up 17-10.

The Falcons led 10-7 at halftime as Devonta Freeman scored from 1 yard out with 18 seconds remaining in the second quarter to climax a 78-yard drive. Sanu hurt his hamstring on a wildcat run ahead of Freeman's touchdown.

Bill quarterback Tyrod Taylor, who was 12 of 20 for 182 yards, hit Jordan Matthews for a 9-yard score midway in the second quarter set up by a 44-yard pass to Charles Clay, giving the Bills a lead after Matt Bryant had kicked a 28-yard field goal early in the quarter.

Jones left the game in the second quarter with a hip injury after catching three passes for 30 yards. He led the NFC with 1,409 yards on 83 catches last season and came into the Bills game with 16 catches for 265 yards this year.

Tevin Coleman led the Falcons with 79 yards on nine carries and caught four passes for 65 yards.

"It's disappointing and we take the loss together," Coleman said. "There D-line got a lot of penetration and that a makes it real tough. You have to give the Bills credit."

NOTES: All of the Falcons stood for the national anthem, locking arms. ... Six Bills knelt: DT Cedric Thornton, DE Jerel Worthy, CB Shareece Wright, FB Mike Tolbert, WR Kaelin Clay and RB Taiwan Jones. ... The Falcons lost DT Jack Crawford (biceps) in the second quarter. ... The Falcons had four starters inactive: S Ricardo Allen (concussion) joining DE Vic Beasley Jr. (hamstring), DT Courtney Upshaw (ankle) and RT Ryan Schraeder (concussion) from the previous week. ... The Bills lost S Colt Anderson (arm) in the first quarter. ... DE Shaq Lawson (groin) and LT Cordy Glenn (foot) were inactive for the Bills. Lawson was hurt in practice on Thursday. ... Mercedes-Benz Stadium's complex retractable roof, not yet fully automated, was closed despite perfect fall weather. ... The Falcons wore throwback black jerseys from the 1960s.