The German company said late on Sunday it would return to split leadership with Bill McDermott, head of field organization, and Jim Hagemann Snabe, product development chief, both already members of the SAP Executive Board, taking the helm.
The world's largest business-software company said the departure was by mutual consent and did not give a reason for the change at the top.
Apotheker was named as co-chief executive with Henning Kagermann in April 2008 and took over sole responsibility to lead the German company after Kagermann's retirement in July 2009.
Yvonne Genovese, head of software analysis at research firm Gartner, said SAP was reacting to client displeasure and a perceived lack of clarity over the company's long-term vision.
The client base has lost confidence largely because of the maintenance issue fees, she said, adding that SAP's patchwork of messages on cloud computing, software as a service, sustainability among other issues resulted in a lack of clear, cohesive strategy.
SAP raised maintenance fees for its customers by 5 percent when Apotheker took over as sole CEO in July 2009 after keeping rates stable for several years unlike rivals, which raise fees annually.
SAP competes with companies such U.S. software firms Oracle
The company eventually backed down after clients complained and asked SAP for performance indicators by which to measure improvements that would justify an increase in maintenance fees.
Raising fees arbitrarily the way SAP did was seen as a breach of trust because what guarantees would clients have that they would not do it again ... and it is not clear what SAP would use the money for, Genovese said.
In the past, SAP has relied on large, integrated software systems it has sold to many of the world's biggest companies.
But it has yet to adjust to the challenge of securing more smaller deals annually instead of selling one or two big expensive systems that are not in demand in an era of cost-cutting.
Its business by design software targeted at midsized companies has been delayed several times and is now set to rollout at volumes size by mid-year.
SAP sales were hit hard in September 2008 as companies reined in IT spending to combat the effects of the financial crisis.
But SAP staged a recovery on the back of cost cuts and managed to report a strong fourth quarter last month and Apotheker predicted a slow pickup.
Genovese said, Leo had some big shoes to fill -- and he came in at a tough time.
Under Apotheker's reign SAP has had to announce job cuts of 3,000 staff, a first in the company's history.
With the appointment of new co-CEOs, SAP is now trying to right the ship and to support that move it appears that co-founder and powerful chairman Hasso Plattner will have a larger say.
In its statement the company said at the board's request Plattner will advise the new leaders on technology and product development.
In addition, SAP said, Vishal Sikka, Chief Technology Officer, has been appointed to the SAP Executive Board.
One worry is that the two new co-CEOs have never led ... we will see how they work together, Genovese said.
(Additional reporting by James Mackenzie in Paris; Editing by Bernard Orr)