Business

How Edward Nortons EDO Is Revolutionizing The Movie Test Screening Biz: No More Pens, No More Papers, No More Waiting

Shutterstock

EXCLUSIVE: The film business has been through various revolutions in the last century: Color, Cinemascope, THX Sound, 3D, premium seating and Large Format have taken the cinema experience to another level.

However, theres one sector of the industry that hasnt evolved and remains stuck in the stone ages: Test screenings.

Lost Horizon

Yes, that post production process by which studios assess the strengths and weaknesses of their films before a recruited audience only to have studio executives and filmmakers pull their hair out. To give you an idea of how long test screenings have been going on, consider the fact that Frank Capra was dealing with walk out audiences and poor scores from the cards around 1936 when he showed off a three-and-a-half hour cut of Lost Horizon in Santa Barbara, CA.

Shutterstock

And do you know what the major revolutions have been in the feature film test screening business since Capras time? Screen test audiences no longer use Golf pencils, they use pens. The surveys went from being on one side to two sides of a piece of paper. While Netflix assesses its audiences via a sophisticated means of algorithms, test screenings results for major studio movies are still being tabulated by hand, riddled with errors.

However, Edward Norton-backed analytics corp EDO is catapulting the screening test process to the 21st century: Theyre taking it digital, and studios such as Warner Bros and other majors are on board.

No more papers, no more pens. If you are a participant at an EDO-run test screening, reach under your chair, and youll find a digital device the size of iPhone in a blue Ziploc bag. The new digital questionnaire means of collecting audience data compiles statistical results within minutes versus the hours it takes to count paper ballots by hand. Not to mention, the results are more accurate and may even make the whole concept of the focus group extinct. Studio research folks can often get nervous about asking too many questions to a participant during a paper test screening for fear that they wouldnt answer everything. As such, theyd confine their questionnaire to two sides of a piece of paper. With the EDO digital test screening, a studio can load as many as five to six pages of deeper questions and the participants will still yield a 100% completion rate.

“We get much more from the audience and the data we get has a much higher fidelity, and we do it at a lightening fast speed. The topline (how the audience rated the movie overall) that used to take 45 minutes to calculate and its now ready within minutes, so there is no waiting around,” says Derek McLay, Partner of EDO.

EDO Test Screening on new digital devices.

“With the old two-sided paper card, the audience member would scan the card and take a buffet approach, they would skip around and leave a lot of questions blank, about 20-40% of the cards just looked unfinished and empty. Since we are only giving the audience member one question at a time, we get single question focus, and we get a 100% completion rate on the closed end (multiple choice) questions. On the open ended (short answer) questions, we have much more data in the responses. People are accustomed to using a digital device such as ours and its not uncommon for us to get three to four times the length of response for the open-ended responses,” says McLay.

“Theres always the question of how do you get the very goods to go excellent, or the goods to very good in the test screening process. With EDOs device, you can ask a specific question to those who rated the movie as good, or those who answered very good and quickly slice that data. Its amazing how instantly you get the score and how complete the answers are. A lot of that has to do with the notion that people are use to typing on a keyboard versus writing by long hand,” says Toby Emmerich, Warner Bros. Pictures Group Chairman who is fan of the product. The EDO digital test screening device also uses “skip logic” in its questionnaire, which creates “nets” to catch data from specific parts of an audience. This assists the filmmaker in gaining greater insight in terms of what parts of the movie may or may not be working. Essentially, depending on the way you answer a question, it can lead to a subset of other questions. “With this technology youre only asking questions to the audience members you want to talk to, and you can treat the whole audience like a focus group. Without the bias you get from the pack mentality of a focus group,” explains McLay.

EDO test screening

For example, earlier this year, EDO was testing a drama romance that had a Sixth Sense type of reveal at the end. The first screening didnt land well with the audience thoroughly confused. The filmmakers addressed the problem and a second screening was held five weeks before the pics wide release. Essentially, the filmmakers added the following question to the questionnaire: “When did you think Character X was dead? A.) The Beginning B.) The Middle C.) When she looks back and sees he is not there.” In regards to the skip logic of the questionnaire, if the audience member answered correctly (answer C) they then just moved on to the next question. However if they answered A or B, EDO gave them a follow-up question that said “What tipped you off that he might be dead?”.

“If we had put that follow-up question on a paper questionnaire, the audience members who had answered the question right would have looked at the follow-up question and would think they missed something and would start changing their answers. The result would be a lot of junk data. With our system, the skip logic allows you to ask questions to only the audience members you want to talk to (in this case the audience members who marked A or B),” says McLay. The 27 audience members who marked A, told EDO that there was a shot of the actor floating dead during the credit sequence at the beginning. The filmmakers were able to alter the shot to make it more ambiguous and not tip off the reveal. Given the sophistication of the EDO screening device, it could conceivably erase the need for a focus group. Thats the tail end of the test screening process when a subset of the audience is brought before the filmmakers to verbally field drilled-down questions about a film. First, theres nothing necessarily statistically viable about the focus group process. Cinema screening pollsters created it to keep the filmmakers busy for the 45 minute-duration when the test screening results are being counted. Focus group opinions can also be erroneous: “People do as they do, not as they say they do,” says McLay about human behavior. Essentially, when asked about their opinion before a group, participants have an inherent nature to sometimes change their responses, as opposed to expressing themselves sincerely and privately on a polling device. Nonetheless, EDO has kept the focus group intact at the clients request because some filmmakers still enjoy it.

Norton and Bruce Willis on set in Motherless Brooklyn Shutterstock

“With the billions of dollars invested in movies, it boggles the mind that when it comes to testing methodologies, the industry is still using paper and pen. The software allows for so much more sophistication and speed, ..not to mention the nuance and depth of the way the people respond to the survey,” says three-time Oscar nominee Edward Norton who is not only an owner of EDO, but a client as well having recently conducted a digital test screening for his upcoming Warner Bros. crime drama directorial Motherless Brooklyn.

Why did it take so long for the testing process to go digital?

“None of the old system companies were willing to invest any money in innovation so we thought there was an opportunity,” says McLay, “When I was at NRG I would look around the theater and watch people filing out cards using their thighs as a desk and it just seemed so inadequate, were using smart phones in every aspect of our lives, why not here?”

The film industry isnt known for embracing new right away. In the research-end of the business, studios executives are often pitched an abundance of products, everything from social media monitoring to release date software programs. Theres sometimes a reluctance to support a new technology because the 80-year old process feels safer, or theyre waiting for their competition to embrace new ways before they safely do. But with streaming a continual threat to theatrical moviegoing, and Silicon Valley companies like Uber and Facebook assessing consumer behavior down to what theyre next moves are, EDO is showing Hollywood how to read moviegoers digital bread crumbs out there in an effort to keep the industry alive.

“This isnt hacking a math problem, were trying to help creative people and studios who finance their work to do better and make interesting content,” says Norton about the improved test audience product “Improved data helps everybody do better, and it flows down a chain with the studios ultimately feeling more comfortable taking risks and making more interesting movies.”

Original Article

[contf]
[contfnew]

Deadline

[contfnewc]
[contfnewc]

Related Articles

Business

Pressed by COVID-19 and low oil prices, Nigeria slips into recession

africanews– Nigeria, Africa’s biggest economy, entered recession for the second time in...

Business

EU Reeling From Yellow Vest Protests. What Happens if There Is a Debt Crisis?

There is a lot of talk about which economic bubble will burst...

Business

EU Reeling From Yellow Vest Protests. What Happens if There Is a Debt Crisis?

There is a lot of talk about which economic bubble will burst...

Business

Till Trump do they part: Top tech firms cut ties with Huawei following US trade blacklisting

Last week, US President Donald Trump signed an executive order aimed at...