Rovers relinquished a 1-0 lead before going on to lose the match 4-2, much to the disappointment of the large travelling support at Bayview.
It was just the latest in a long line of away matches where John McGlynn’s side have lost from a winning position – a theme carried over from last season.
Supporters are calling for the manager to address the issues, but Spencer believes it’s up to the players to be more organised on the park.
“We’ve got to manage the game better after going 1-0 up, especially conceding so soon after,” he told the Press.
“We were pushing and we felt confident trying to get a second goal, but it’s poor from the players.
“It not the manager or the staff – that result is on the players.
“We’re a team that wants to play on the front foot but we didn’t defend well enough as a team.
“Everybody from back to front, we weren’t good enough on the day.”
Raith were top scorers in League One last season with 75 goals, and they have started the current campaign in similar vein with 13 scoredin the opening five games.
It’s keeping the ball out at the other end that’s proved problematic.
Spencer added: “We’ve had two clean sheets so far, which shows we can do it, but we need to switch on better.
“It was three set pieces again as a team we’ve conceded from, and that’s just not good enough.
“On the day we just weren’t up for it, didn’t battle enough, and they won too many second balls.
“The gaffer has put together a strong team that can go and push for promotion.
“It’s up to us as players when we step across the line to take responsibility as a team, and put things right, and we didn’t do that on Saturday.”
Spencer praised the fans for their backing and apologised for letting them down.
“We don’t want to let the fans down - they’ve been brilliant since day one,” he said.
“You can see it on social media how much they are behind us.
“Come May we want to be top of the table and win promotion for them.
“We apologise for the way we went about things today and for the result.
“But make no mistake we’ll be bounce back in the cup game next week, then back to the league, and hopefully when the next derby comes around we’ll be in a better position.”
While Saturday may not have gone the team’sway, Spencer has enjoyed a fruitful start to his Raith career, catching the eye with a number of killer passes leading to goals.
“I can have as many goals and assists as I want, but if we’re not winning promotion it doesn’t matter for me personally,” he said.
“I can do the dirty side as well, which I don’t think we did well enough on Saturday.”