Reilly Smith, Zach Whitecloud, Chandler Stephenson and William Carrier also scored for Vegas, which led 5-1 midway through the second period. Logan Thompson finished with 42 saves.
''We just let them off the hook,'' Stone said. ''I think we just kind of thought the game was over. We started trying to make plays, and if you look at the goals we're scoring, we're getting on the forecheck, making them make turnovers, and then we fell into that trap and started doing it to ourselves.
''You definitely want to learn from it. That was not a perfect game from us by any means.''
Claude Giroux and Tim Stutzle each had two goals, and Brady Tkachuk had three assists for Ottawa, which has lost four straight. Anton Forsberg gave up all five goals on 19 shots before being pulled midway through the second. Cam Talbot came on for his season debut and stopped all 13 shots he faced.
''If you play like the way we played in the second and third period you're going to win a lot of games,'' Senators coach D.J. Smith said. ''There's no such thing as moral victories in sports, but you get that effort from the guys that we got tonight and you'll start winning a lot of hockey games.''
Trailing 5-1, Giroux got a cross-ice pass from Jake Sanderson and scored his second of the night with 5 1/2 minutes left in the second.
With less than two minutes remaining in the middle period, Thompson thought he had stopped a shot by Tkachuk, but the puck trickled through his pads and Stutzle pushed it over the goal line to make it a two-goal game.
Ottawa then pulled within one 2:34 into the third when Stutzle took advantage of a Golden Knights turnover and played a give-and-go with Tkachuk before scoring his second of the night and third of the season.
Vegas had an opportunity to pad its lead on a 5-on-3 advantage that lasted 59 seconds, but Talbot made two huge saves for the Senators.
Stone put Vegas on the board first, picking up a loose puck in front and scoring his fourth just 1:22 in.
The Senators tied it less than three minutes later as Giroux, who was looking to go off on a line change, shot the puck from just over center ice and scored as the bounce of the puck fooled Thompson.
''I've probably tried that over 500 times in my career and it never went in,'' Giroux said. ''When it went in, it was overdue. I get pretty excited when I see a goal like that.''
Smith scored a power-play goal at 9:19 to put Vegas ahead, and Whitecloud made it 3-1 with 32 seconds left with his first of the season off a cross-crease pass from Stone.
Stephenson's short-handed goal made it 4-1 for the Golden Knights at 5:15 of the second. Carrier took advantage of a giveaway by Ottawa's Alex DeBrincat, skated in alone on Forsberg and beat him through the legs nearly five minutes to give Vegas a four-goal lead and chase Forsberg.
BETWEEN THE PIPES
Forsberg began the night leading the NHL in shots faced (276) and saves (251).
Golden Knights: At Montreal on Saturday night in the third of a five-game trip.
Senators: Host Philadelphia on Saturday night in the second of a three-game homestand.
Copyright 2023 STATS LLC and Associated Press. Any commercial use or distribution without the express written consent of STATS LLC and Associated Press is strictly prohibited.