Hoops striker Moussa Dembele gave the home side the lead in the third minute - the first time the Premier League leaders had been behind this season - before Fernandinho equalised in the 12th minute.
Left-back Kieran Tierney’s shot deflected off City winger Raheem Sterling, who then levelled again for the visitors with a well-taken goal.
Dembele made it 3-2 with less than two minutes played of the second half before Nolito equalised again in the 55th minute for a share of the spoils.
“It was a fantastic game for the spectators. We scored three goals away and didn’t win,” Guardiola said. “We gave three goals but created three of four more clear chances.
“It was an amazing effort to recover from that situation in this stadium against a team who had lost three of their last 24 games here (in Champions League group stages).
“It was maybe a good lesson for us, how important it is to start well. It is not easy to recover one goal, imagine two, imagine three.
“It was mentally a shock at the start of each half, to concede a goal after two minutes.
“In the second half, aside from the first five or 10 minutes we played well, we created chances.
“They scored three and but didn’t make many more chances, we created enough chances to win the game.”
City were looking to make it 11 wins in a row since the start of the season, and become the first English team do that since Tottenham in 1960/61.
However, the first setback did not overly concern the former Barcelona boss ahead of the game against Tottenham at the weekend.
He said: “I didn’t speak to my players about that. I am pretty sure the players didn’t think they we going to win every game this season. It is impossible. Not even the best team in the world (could do that).
“We didn’t lose. On balance the players did really well. We had the chances.”