The Nikkei share average dropped on Tuesday, reversing early gains as hopes for progress in Europe's debt crisis gave way to caution on the domestic front as Japan's corporate earnings season kicked off.
Shares in industrial robot maker Fanuc Ltd's, a widely held stock, skidded after its forecast was in line with expectations, disappointing investors who had been hoping for a positive surprise.
Some investors took profits after Monday's gains for the benchmark on fears that earnings surprises will weigh on the broader market in the days ahead.
There is no reason to buy shares right now, as even investors who are hopeful about earnings want to make sure that results don't disappoint, said Yutaka Shiraki, senior strategist at Mitsubishi UFJ Morgan Stanley Securities.
Indeed after the bell, camera and office equipment maker heavyweight Canon posted higher quarterly profits but cut its full-year outlook below expectations, citing the impact of the Thai floods and a strong yen.
But scandal-hit Olympus rose 8.2 percent, snapping a seven-day share price plunge triggered by the firing of its former chief executive, who has questioned hefty fees paid to financial advisers in past M&A deals.
The Nikkei average fell 0.9 percent to 8,762.31 .N225, while the broader Topix .TOPX shed 1 percent to 747.70.
The weakness in Tokyo came despite big gains on Wall Street on Monday on hopes of a solution to the euro zone debt crisis and upbeat earnings from Caterpillar Inc.
Some Japanese market players were also skeptical about whether European policymakers are doing enough to contain the debt crisis.
If the latest rally was really driven by euro zone hopes, then by rights Italians shares or European banking shares should have lead gains. But that was not the case, said Soichiro Monji, chief strategist at Daiwa SB Investments.
Fanuc slid 3 percent to 12,160 yen, and was the biggest drag on the market, accounting for just under a fifth of the Nikkei's fall.
NGK Insulators Ltd plummeted 17 percent to 926 yen and was the third-heaviest traded share by turnover, on a report that the company is asking customers to refrain from using its high-capacity sodium-sulfur batteries while it investigates the cause of a September 21 fire at the Tsukuba plant of Mitsubishi Materials Corp.
But excavator manufacturers climbed after Caterpillar's results, with Komatsu Ltd gaining 3 percent to 1,833 yen and Hitachi Construction Co climbing 2.7 percent to 1,433 yen.
Shares of Olympus rose in heavy trading, gaining 8.2 percent to 1,189 yen, as short sellers took profits after the stock lost more than 50 percent of its value through Monday.
After the close of trade, a senior Japanese ruling party lawmaker said he has asked regulators to look into Olympus.
Overall trading volume was thin but picked up slightly after very slow trade for a week, with 1.44 billion shares changing hands on the main board, compared to 1.34 billion shares on Monday. More than three shares declined for each one that advanced.