Interview: David Simon - Wire in the blood

THESE days, all the doors are open for David Simon. That's what happens when you make a TV series like The Wire, which changes the way we watch the medium and sets new standards about what we expect from it.

Create a series that takes and breaks the rules about TV drama, that has critics reaching for their highest, rarest adjectives, that snakes through inner-city America, tackling all of its problems with insight and integrity, and this is your reward. Out comes the HBO corporate chequebook and the only tough question is "OK Mr Simon, what do you want to write about next?" Maybe the Iraq war? New Orleans after Katrina?

That's the stage the 49-year-old American writer and producer has reached right now. Generation Kill, his adaptation of Evan Wright's account of the US Marine Corps push into Iraq, will be screened on Channel 4 this autumn, when Simon will start filming Treme, about a blue-collar district in New Orleans. But clearer than any other writer I can think of, you can see where it all came from.

Twenty-one years ago, on the streets of Baltimore, doors would open for David Simon that he'd almost wish wouldn't. Because when they did, the city's homicide detectives whose work he was spending a year shadowing would "put on their kindest faces" and break the hardest news possible to murder victims' families.

Back then, Simon was taking a sabbatical from his job as crime reporter for the Baltimore Sun, researching the non-fiction book that would become Homicide: a Year on the Killing Streets. Fictionalised, we'd catch glimpses of some of the detectives he worked with in this 1990s TV series based on his book, Homicide: Life on the Street, just as we would do again in The Wire. We'd meet one of those bereaved mothers again in Simon's second book, The Corner, just as much of a classic work of reportage about inner-city drug culture as Homicide is about the work of murder detectives. Everything about David Simon's work has deeper roots than you might realise. Simon has an engaging loyalty to the people he met and wrote about in those early days: he keeps in touch, remains friends, finds jobs, helps out. He knows what he owes. There's a similar integrity about his journalism – and his rage against the lingering, corporatised death of newspapers.

Down a hallway at the entrance to the Baltimore Sun offices , there's a quote by its most famous journalist, HL Mencken, that is also used as an epigraph for the final episode of The Wire. Looking back, Mencken realised that of all the things he'd done in his career, being a newspaperman was the most fulfilling: "It really was … the life of kings."

That's how it started out for Simon too, chasing stories, making deadlines, meeting new contacts, chasing more stories: that adrenalin-fuelled cycle of being a crime reporter on a city newspaper – not least one in "Bodymore, Murderland" – was all he ever wanted. It was the tail-end of what he now looks back on as the Golden Age of American journalism – the brief 20-year era that began just after the Tet Offensive (when people realised they weren't being told the truth about Vietnam) and ended when the accountants put the corporate squeeze on editorial budgets. Disillusioned after a strike at the Baltimore Sun, Simon took a year's sabbatical to research Homicide. He was still in his twenties, and although he's still proud of the book, he realises it could have been deeper. As an example, he tells me about witnessing a Baltimore woman, Ella Thompson, being told by detectives that her daughter had been murdered.

"The one thing you're not really called upon to have (as a journalist] is genuine empathy. You need to write as if you have, but you don't really need to have it.

"And the truth is that when I went to Ella's door with the homicide detectives and I watched her break down as they told her about her daughter … well, at that moment, I was completely bleeding for that woman.

"But I was 27. I wasn't a parent myself, and that adds something because you strain everything through your own experience. So what I was really looking at was the detective and how he was dealing with telling her."

Five years on, researching The Corner took him to an inner-city children's club run by the same woman, Ella Thompson. He didn't recognise her. "Tellingly, I didn't remember her from that moment at her door. I didn't even talk to her in court at the trial. It's like, you can be at the door but you know you're going to walk away, you're getting a news story. It's really sad but …"

'But there will be other news stories' is what he means to say. Other news stories that tell themselves simply. Someone killed someone else in town today: here's the who, what, when, how where and why it happened. Here's all the basic information it's your job to tell the average reader. You don't go in deeper.

Compare that to what Simon does in The Corner. By now he's 32 and has a child of his own. Disillusioned by prolonged cost-cutting at his newspaper, three years later he will take redundancy from the only job that, until then, he ever really wanted to do. In the meantime though, his storytelling has grown up. He's almost at the stage he will reach in The Wire when what matters isn't explaining everything simply to the average reader. In fact, he'll ditch the average reader altogether. Instead of simplistically spelling out the basics of a news story, he'll write from within the story. He'll write it so well, so accurately, so objectively, whether you are a homicide detective, a junkie, or a city politician on the make, you'd read it and recognise yourself in it. Get that right, says Simon, and you've got everything right.

And here's the difference. Researching The Corner, he meets Ella Thompson again. He'll spend a couple of years, off and on, talking to her and the street kids who turn up to her club. He'll realise what her daughter meant, how Ella's grief shifted, counterintuitively, into trying to create an oasis of hope in a desert of hopelessness. Already, he's moving beyond journalism's banalities, even before he meets Gary McCullough.

If you stick labels on people, Gary McCullough – the central figure in The Corner – was just a junkie whose drug killed him. To Simon, he was a lot more.

"Gary dying … it was a point of growing up for me. I always knew that, because drugs are drugs, some of the people wouldn't make it. But that was just an intellectual understanding.

"What you don't realise is that there's going go be this person Gary McCullough, that you're going to fall in love with certain things he does, that if you don't see him for a day or two you're going to miss him.

"Because he was a wonderful man in so many ways, being crushed, but nonetheless … And you don't say to yourself, 'But what if Gary dies?' You don't think that the tragedy that you can imagine intellectually will be somebody real, a complete, somebody you've spent a year, longer, getting to know.

"It's things like the time Gary wanted us to get Elie Wiesel's book Night (a classic of Holocaust literature] after he'd seen something about it on television. Or reading Karen Armstrong's A History of God. I'll never forget him reading that book. To see him reading it and sometimes understanding it because he wasn't high, then seeing him read the same pages over and over because he was completely whacked out, these things leave you completely emotionally … you know, that was the first truly grown-up moment, when I felt the psychic cost of what I was doing."

There are echoes of Gary McCullough in Bubbles, the courageously resilient junkie who is one of the main characters in The Wire. "Yeah, I wrote a better ending for Gary than Gary had. Because when he did die, I was devastated, and had all kinds of mixed feelings about being complicit and the exploitative nature of this kind of journalism."

Yet at least "this kind of journalism" – years in the research, whole months at a time just establishing trustworthiness – could cover every aspect of Baltimore life. The newspapers couldn't. They cut their staff, losing expertise and authority, stopped reporting on whole swathes of civic life. By the time Simon wrote The Wire, his old newspaper would have no more known about people like Gary McCullough than they would have known about Baltimore education officials fiddling the attainment statistics or police chiefs doing the same with the crime figures.

So although the final season of The Wire , which focuses on the media, might seem, on the face of it, to be about just one reporter who makes up just one story, it's really about a deluded, denuded newspaper that misses hundreds. A profile of Bubbles aside, not one of the stories that the season tells – of Baltimore's streets, the schools, the police department and city hall – makes it into cold print.

Highlighting all of this, The Wire is, in a sense, an act of deep-buried revenge on the self-blinded media. Reeling from its many individual fictional tragedies, you might not notice this greater, institutional but very real one: that when America – or Britain, come to that – wants to find out what's going wrong in its cities, it now no longer can.

David Simon is at the Edinburgh Book Festival on Saturday, 8pm, and the television festival on Friday. The Wire: The Truth Be Told will be published by Canongate on 1 October, price 20 in hardback, 14.99 in paperback.