Prosecutors said Peter Brown, 37, set fire to the complex in Norton, North Yorkshire, after he was refused entry to a party.
The blaze claimed the lives of apprentice jockeys Jamie Kyne, 18, from Kiltrogue, Co Galway, Ireland, and Jan Wilson, 19, of Forfar, Scotland. Neither had been involved in Brown's quarrels.
The court heard how the fire "raged" through the building on 6 September forcing many of the occupants to jump for their lives.
Miss Wilson, who had been staying with her boyfriend Ian Brennan, and Mr Kyne, were asleep in a top-floor flat when the alarm was raised.
Prosecutor Richard Mansell, QC, told the jury that, as the blaze burned, Brown "made what appeared to be a foolhardy and pointless attempt to break into the flat complex".
He added that jurors would have to consider whether this was "an attempt by the man responsible for lighting the fire to reverse what he had done minutes earlier" or whether it was "nothing more than a feigned performance designed to cover his tracks".
The jury heard that Brown – who later described himself to police as a "drunken Scotsman" – had been drinking in several pubs before the fire started.
In the meantime, a party had been going on mainly in flat four of the Buckrose Court complex but those attending also went in and out of flat five – the one where the two jockeys died.
Mr Mansell said Brown returned to the complex where he allegedly used an accelerant, possibly white spirit, to start a fire with rubbish.
Fire alarms sounded and residents woke to find the complex on fire. Many were forced to jump from windows or climb down a drainpipe as the fire took hold.
Miss Wilson was seen screaming at the window as the flat "quickly became filled with smoke", the jury was told.
Mr Brennan managed to jump from a window of flat five with another resident, Dean Pratt.
Mr Mansell said Brown was "keen to point the finger of blame at others", allegedly telling one woman: "This is your fault. This is what happens when you have parties."
Relatives of the two victims and the defendant listened in the public gallery as Mr Mansell said: "We suggest that the defendant, whilst intoxicated and feeling generally low, brooded during the two hours or so which he spent at public houses in town.
"He lit the fire on his return to get back at the occupants of flat four, and possibly one or more of the visitors, who had earlier refused him entry to the party and insulted and humiliated him."
The court was also told that Brown's personal life was "not going particularly well" and he had alcohol problems.
Brown, of School Croft, Brotherton, North Yorkshire, denies two charges of murder, two alternative charges of manslaughter and one charge of arson with intent to endanger life.