Goals in each half from Takuma Asano and Yosuke Ideguchi earned the victory to secure Japan one of the two automatic berths from Group B. The Blue Samurai reached the finals for the sixth time in succession.
Japan won the group with an unassailable four-point cushion over Saudi Arabia, who lost 2-1 away to United Arab Emirates on Tuesday, and Australia with one game remaining.
Vahid Halilhodzic's side play the Saudis away in their final qualifier in Jeddah next Tuesday, while Australia host already eliminated Thailand the same day in Melbourne.
Japan made a bright start and went close to opening the scoring with less than two minutes on the clock, Hiroki Sakai heading Ideguchi's corner just wide of the right-hand post.
Takashi Inui then tried his luck with a curling effort that was deflected out for a corner, and Yuya Osako tested Maty Ryan with a shot that the Australia goalkeeper managed to gather at the second attempt.
Japan got a huge scare when Mathew Leckie hit the post from seven minutes before halftime.
But Halilhodzic's men side took the lead soon after as Asano got on the end of Yuto Nagatomo's cross to steer home a 41st-minute volley and trigger delirious scenes among the home fans.
"It was a great ball from Yuto-san and it was just a case of me getting on the end of it," Asano said of his goal.
"I am always getting myself ready in case I get called on to start and just want to do everything I can on the pitch and try and put a smile on the fans faces."
Japan goalkeeper Eiji Kawashima did well to get down and deny Robbie Kruse as Australia looked to hit back early in the second half.
The Socceroos increased the pressure and a superb block from Sakai thwarted Kruse again but Ideguchi's 82nd-minute strike sealed Japan's first win in eight attempts over Australia in World Cup qualifying.
The Gamba Osaka player, who had earlier had a shot blocked off the line, took a pass from substitute Genki Haraguchi and cut inside to rifle past Ryan for his first Japan goal.
"I was just hoping to get the shot on target and I am glad it went in," said Ideguchi, who was playing in only his third senior international.
"The coach decided to play me on this big stage and so I came into the match determined to deliver results and that led to the goal."
"There is still some way to go until the World Cup so I want to continue to improve so that I keep getting picked."