The English Premier League champions crashed out of the competition at the group stage despite a 1-1 draw with Real Madrid at the Etihad Stadium.
But despite the progress the club has made since being taken over by Sheikh Mansour bin Zayed in 2008, the manager warned it could yet be years before they were a match for Europe's top sides.
"Chelsea tried to win Champions League for 10 years," he said.
"In the best moments, when Didier Drogba was young, they didn't win.
"They did it last year when no one thought they could.
"We need to improve our team because there are a lot of teams better than us.
"I am not saying if you get to February you can't win. Anything can happen. But we didn't do that."
City did well to salvage a point given the pummelling they took in the opening stages.
Karim Benzema put the visitors ahead after just 10 minutes following more catastrophic City defending, while Cristiano Ronaldo only failed to double the advantage because he underhit a lob over Joe Hart, allowing Matija Nastasic to clear off the line.
Sami Khedira also failed to profit from two gilt-edged chances provided by Ronaldo, allowing City to scrap their way back into the contest.
Still there was an element of good fortune about their 73rd minute equaliser, with Alvaro Arbeloa harshly judged to have fouled Sergio Aguero, resulting in a red card for the former Liverpool man and a penalty for City, which the Argentina forward clinically dispatched.
"No. I don't fear this," said the Italian. "Why?
"We still have the Premier League and the FA Cup.
"If we think we can win the Champions League after two years we are crazy."
Opposite number Jose Mourinho gave his support to Mancini, admitting that if Real Madrid had exited the competition at this stage he would have faced the exit door himself.
"I admire the fact that clubs support their manager," said Mourinho.
"City is out of Champions League. But Real Madrid and Borussia Dortmund are very good teams and we knew a big team would be out.
"It is good that it was City because Roberto can work without any kind of problem I believe.
"If it was Real Madrid, the press wouldn't let me return to Madrid."
Being Mourinho though, he could not resist a barb at his opponent, who will now be under additional pressure to retain the Premier League.
He said: "It is incredible, when you look at the array of players at their disposal, they have been eliminated from the group stages two years running."
Follow us @SprtNationalUAE