Watson, 52, has worked at the company since 1980, including roles as chief financial officer and head of exploration and production outside North America. He also led Chevron's integration of Texaco after the transaction closed in 2001.
The California native, who currently oversees strategy and development, will take the helm at a time when oil majors face increasing competition from state-run oil companies for access to the largest untapped reserves. Two-thirds of the world's top 20 oil companies are backed by governments.
O'Reilly, a 62-year-old from Dublin, has worked for the San Ramon, California-based company for 41 years and served in the top two roles for the past decade.
While his public profile was lower than that of some other top executives, he made headlines in June by debating the head of the Sierra Club, an environmental group.
A new CEO is just the latest change at the top of Chevron in the past year, which has seen a new chief financial officer take over and the appointment of a new top in-house lawyer.
The CEO change is a mild positive for Chevron's stock, largely because it sets up a smooth transition, said James Halloran, consultant with Financial America Securities in Cleveland, Ohio.
But he said the road ahead looks tough since oil companies largely occupy a second tier after national oil companies.
They can no longer show up at the doorstep of countries and get access to the oil, Halloran said.
In the face of this challenge, Chevron in January dropped its target for 3 percent compound annual production growth from 2005 to 2010, although in July it did bump up its 2009 output growth target to 5 percent from 4 percent.
O'Reilly will depart as a number of sizeable projects come online from Brazil to the Gulf of Mexico to Australia, where the $37 billion Gorgon gas project just got the green light.
He's left him with a well-stocked pond, said Fadel Gheit, analyst at Oppenheimer & Co, who also identified access to new reserves as Watson's biggest challenge.
Watson also faces a potential crisis from a $27 billion claim under a lawsuit in Ecuador, where indigenous people blame Texaco for polluting the jungle and damaging their health.
A ruling in the 16-year-old case had been due in the coming months, but that is now complicated by the judge's replacement amid allegations he was involved in a $3 million bribery plot.
Chevron's board elected George Kirkland, 59, to succeed Watson as vice chairman. Kirkland will retain responsibility for Chevron's global oil and gas exploration and production.
Chevron shares rose 0.1 percent to $70.99 on the New York Stock Exchange. Larger rival Exxon Mobil Corp
(Reporting by Matt Daily in New York and Braden Reddall in San Francisco; editing by Gerald E. McCormick and Andre Grenon)