In his first interview since taking up the post in June, Mr Yousaf said he had promised the families of two murder victims that he would consider issues around parole and the arrest of offenders who were unlawfully at large.

But he denied his party had been weak at tackling violent crime, accusing his Tory counterpart of “opportunism”.

The Scottish Conservatives said there was a “crisis in confidence” in the justice system that had to be addressed.

It is thought the Scottish Government will move to make parole board decisions more transparent for victims’ families as well as making offenders who break the terms of their early release from prison more of a priority for police.

Last year, father-of-three Craig McClelland, 31, was murdered in Paisley by an offender who had been “unlawfully at large” for five months after breaching the terms of a home release curfew.

Mr Yousaf said he had given an undertaking to Mr McClelland’s family that lessons would be learned from the case to prevent something similar happening in future.

“Where we can strengthen measures for victims and put them even more at the heart of the criminal justice system, I will look to do that,” he said.

“In the next few weeks you should see some very clear signals that we will do what we can to strengthen victims’ rights within the criminal justice system.”

Mr McClelland’s killer, James Wright, had 16 previous convictions, including two for knife crimes. He was jailed earlier this year, with two separate reviews of home detention arrangements now attempting to establish how someone in his position could be at large despite breaching the terms of his release just 11 days after leaving prison.

The case led to criticism of the Scottish Government’s justice agenda, which has promoted home detention and community sentences as an alternative to prison.

Mr Yousaf said: “I gave a promise to the McClelland family and Craig McClelland’s partner that we will find out what went wrong in this case and what lessons we can learn to prevent this from happening in the future.”

Mr Yousaf said he believed confidence in both Police Scotland and the wider justice system was high, but said there were lessons to be learned when things went wrong.

The new justice secretary, who replaced Michael Matheson in a June reshuffle, accused the Tories of “opportunism” over attacks on policies such as the presumption against custodial sentences of less than 12 months. He said: “When it comes to custodial sentences, the evidence speaks for itself. Hence why we are looking at the presumption against short sentences and the UK government are doing the same thing.”

Asked whether the SNP was soft on violent offenders, he said: “The evidence for that doesn’t stack up. We know that people with life sentences are in prison for longer and we’ve increased the maximum sentence for knife crime. So the evidence doesn’t stack up on the claim about soft-touch justice. It’s the Conservative party that has pushed this and it’s starting to backfire.”

Earlier this month, Mr Yousaf met the family of Michelle Stewart, 17, who was stabbed to death in the street in Drongan, Ayrshire, by John Wilson in 2008. Wilson, who was sentenced to 12 years in prison in April 2009, has been deemed eligible for temporary release despite only serving nine years behind bars.

The Stewarts have called for measures to improve the rights of victims and their families. They include the increased use of powers to impose exclusion zones on offenders and changes to the parole system to allow victims and their families to be given reasons for an offender’s release and to make representations in person.

Mr Yousaf said he was looking at ways of making the parole system more transparent and was “open-minded” about looking at examples from elsewhere in the UK.

“My promise to the family of Michelle Stewart is not only that they remain fully informed, but that their views are taken on board as part of any temporary release measures,” he said.

Shadow justice secretary Liam Kerr said: “We need more than talk from Mr Yousaf, we need to see action.”

Humza Yousaf pledges to learn lessons from early release killing