Few businesses make a move, or spend so much as a single, devalued Canadian dollar, without the research to back it up. And not just any research – but the kind of down-to-the-molecular-level analysis known only to MBAs and UN weapons inspectors. New products are run past focus groups and checked against buying patterns, education, income, sex and, sometimes, sexual preference. Consumers are polled by phone and cornered in malls by platoons of clipboard-wielding marketers because no CEO, under any circumstances, however daring, wants to lose millions by introducing the next New Coke. Or by passing on the next Seinfeld.

The only exception? Why, Canadian television of course, an industry that, traditionally, has not been big on testing out ideas before they hit the air. Canadian TV rarely if ever produces pilots, preferring instead to progress from treatment to script to table reading and then jump – eyes closed and often over the million-dollar mark – to an episode order. Other research has been sporadic at best.

But with fewer dollars, harsh and fragmented markets, and Telefilm Canada head Richard Stursberg calling for better and better marketed TV, some think audience testing is the way to go. But does polling the viewers actually make a difference? Many insiders say no.

Audience testing didn’t save The Associates, and Anne Marie La Traverse, senior VP of movies and television series at Alliance Atlantis Communications, thinks running polls or focus groups on TV shows is of ‘questionable’ use.

‘We tested The Associates quite extensively in a few cities after the first season. And we were told by audiences…they wanted legal stories, not relationships,’ she says. ‘So we acted very much on that in our second season, and at the end of the day I don’t think anyone else watched.’ CTV cancelled the AAC series in its second season.

The best way to develop or retool a series, within the financial and scheduling constraints of the Canadian system, she says, is for producers to trust their instincts and for broadcasters to let shows ‘evolve’ on air. Her latest series, The Eleventh Hour, was not tested before going to air.

‘[A series] is a very organic thing, as opposed to a movie, you can keep changing it to make it better,’ she says.

Toronto-based media buying firm M2 Universal does all its research in-house, says president Hugh Dow, because he has never seen a show, Canadian or American, that had any reliable research backing it up. His firm sometimes looks at focus groups, or Internet polling or other proprietary testing methods, but they do not carry much weight. Dow says 80% to 90% of his company’s decisions about TV shows are based on instinct and judgment.

‘There’s no foolproof way to determine the success or longevity of a show. If there was then Hollywood, with billions of dollars at stake, would have found it,’ he says. Even in the U.S., where shows are tested from the cradle to the grave, the ‘casualty rate’ is still incredibly high, 60% to 70%. ‘Which leads me to believe that judgment is really the main criteria to evaluate the merits or potential of a TV show.’

Some researchers are rethinking their methods. Christine Wilson, senior director of strategy and planning at CBC, says the network has recently moved away from focus groups or inside panels to more precise, quantitative testing – such as the more than 1,000 people it quizzed about the ‘back-door pilots’ for Rideau Hall and An American in Canada. CBC aired both episodes in 2001 and followed up with a phone-in poll of 150 random viewers. It also, via an outside research firm, sent tapes and questionnaires to another 1,200.

‘We get better regional distribution and a more representative example,’ says Wilson, adding that quantitative research also provides more solid statistical ground by which one can predict larger trends. ‘If a show scored 7.6 out of 10 and that ended up bringing in an audience of x-hundred thousand, I can build models on that that I couldn’t with qualitative research.’

And yet, the whole point of the CBC testing was to determine which of the two shows would be picked up as a series. The results were inconclusive.

‘Neither of them was an overall, overwhelming success,’ says executive director, network programming, Slawko Klymkiw. ‘We decided that both of them deserved a partial chance.’

CBC’s efforts to crunch the numbers did not work out and, in the end, the decision became a judgment call by the execs. The network split its odds and ordered six eps of each show.

‘But [the polling] gave us some hints about who the audience is and where to place it,’ says Klymkiw. ‘It’s a little more helpful than putting your finger in the air.’