Most people who miss Storage Visions (SV) which is held just before CES (Consumer Electronics Show) say storage isn't as sexy as a wearable or connected car ... pity.
SV covered my favorite subjects -- storage and M&E (Media & Entertainment)
Two of the sessions were worlds apart: the professional story tellers and the growing population of independent filmmakers. Both sessions had different views on how our movies, documentaries and shorts are developed, produced, distributed and saved.
In the Pro session, Josef Marc, Archimedia co-founder, spelled out the differences in trends and movements.
Trends:
- More storage
- More bandwidth
- More cloud security (really?)
- Price wars
- Smaller, faster
Movements that could happen:
- Format independence
- Smarter media
- Smarter workflow
- Smarter media collaboration
Good perspective.
Cameras like those from Blackmagic are giving the cinema world the ability to capture more content faster and in a variety of formats including 4K, 8K and beyond.
Then, the industry offers that content in a number of formats because consumers want it anywhere, anytime on the device they're looking at now!
That hunger for content is driving a huge demand for production systems, tools and storage.
Healthy Growth – While most of us are only content consumers, filmmakers/cinematographers are storage consumers. Every segment, from capture to distribution, requires a new layer of storage; and very little media is reused ... content is stored, just in case.
According to Coughlin Assoc and IDC, storage demand will grow by 2.7x through 2019.
By the end of the decade, a high-end production could be hundreds of PBs (Petabytes), even 1 EB (Exabyte).
That increase doesn't even include the challenge of storing good, bad and so-so content for future generations!
In the SV Pro session, Ian Trow, of Harmonic, and Alex Grossman, of Quantum, spelled out what people in the upper atmosphere of production do to give us the entertainment we want, need and ... oh heck, deserve.
Worldwide – Life would be easier for the production team if everything could be done in a single, very large room. To hold down costs today, content owners contract most of the work to specialists and centralized systems track/manage the entire process.
No wonder the budgets soar.
The glamour part is shooting.
After that, folks juggle editing, color correction, special effects, audio sweeting, ingesting and finishing.
Then there are the tough jobs – transcoding, rendering, delivery, archiving, remote operations.
All of that is terribly complex, very expensive!
But as Delacourt explained, "I'm interested in something much larger."
Fortunately, the new systems are equal to the task, especially since the movie process isn't done in a single location anymore. Specialists are global and studios tap the best person for each job, regardless of his/her location.
To handle it all there's the cloud with all of its reliability, scalability, self-healing and in-process content protection.
Of course, it's a different story for Indies, documentary producers and business/informational filmmakers who have a huge layer of challenges, budget restrictions and other obstacles.
Cirina Catania, of The Catania Group, painted a different picture of what Indies contend with every day.
An Indie filmmaker may work alone but there are millions of them telling stories, educating, entertaining, preserving history, inspiring and changing the world
Low-cost 4K cameras, inexpensive, easy-to-use production tools, powerful desktop/notebook systems and fast, reliable, high-capacity storage have made it possible for this segment to emerge and thrive.
Larry O'Connor, of Other World Computing (OWC), said Indies have a much more "hands-on" approach than major studios.
Take, work, manage, store – Indie filmmakers are not only jacks of nearly all trades but they quickly learn they have to master them or at least know how to find someone who can do one of the tasks better than they can. To use specialists, Indies often ship drives here and there. Most trust FedEx or UPS more than the cloud.
O'Connor noted that most Indies use lots of "local" storage.
How?
It's the filmmakers' commitment to best practice techniques:
- Original Media exactly as shot
- Protection Clone (never touched)
- Worker Copy (files renamed, organized)
- Protection Copy of Worker
- Studio copy of Worker
- 3rd Clone of Worker for safety
- Output of project
- Clone and 2nd Protection
Low Budget, High Storage – Since low budget indie filmmakers don't have an IT department, they purchase, manage and control their own production/post production storage. Higher- resolution filming produces higher volumes of content being stored, retained and hopefully preserved.
Whether it's a big budget or no budget film, 30 minutes of content is usually shot for every one minute that is viewed.
Every minute of RAW footage shot resulted in 1.6GB of storage and a 26-minute film can require at least 1.3TB of storage.
Catania noted there is a lot of discussion about working collaboratively in the cloud and storing content but it is still "a challenge" and a concern to most Indie filmmakers.
Issues like remote access latency, reliability of fast uploading/downloading work-in-progress, the gnawing concern over safety and security are major concerns.
The biggest issue for Indies is the ability of the cloud to guarantee the long-term (real long-term) retrievability of content.
Because Indie filmmakers think the cloud is a little foggy, Indies tend to do their work and archive locally, increasingly with Thunderbolt-connected storage systems and software RAID (Redundant Array of Independent Disks).
During the SV session Q&A it was "impartially" suggested that tape was more reliable than HD-based solutions for long-term archiving.
That made me think that most people never heard of – or refuse to address – the problems/challenges that face the content industry when content begins and lives its life in digital formats.
As Max said, "It doesn't end well for the meerkat."
Almost Unwelcome – When Disney decided to repurpose, reissue "Toy Story" management found it wasn't going to be as easy as they thought. Corrupted digital files almost caused the film not to be available to younger generations, but a file print (celluloid) saved the day.
Years ago, Disney decided it was going to mine its archive library and recycle "Toy Story" for a new generation of kids who would beg the folks to buy the DVD and all of the associated toys.
Helluva' idea!
The problem was as much as a fifth of the film's digital files were corrupted.
Fortunately, they had a file print they could use ... good save!
There are millions of archive vaults around the globe that hold billions of dollars of content – entertainment, news, history, you name it – and people are counting on to repurpose and rerelease for new revenue.
But the speed with which we've moved from real film to digital is remarkable. In 2007 it was forecast that 50 percent of movie screens would be digital by 2013. It turned out to be closer to 90 percent.
Christopher Nolan caused uproar when he forced the release of "Interstellar" on film.
Imagine a film on film.
But digital shooting, production, delivery is fast, cheap and easy compared to film and it looks as good every time it's viewed, no rewinding, no scratches, no "noise" - pristine.
Oh yeah, and cheap!
The problem is no matter which medium is used, there is deterioration.
Digital files can be left behind when software is changed or new "next generation" hardware is adopted (which is about every 18 months).
Archive Wink – Filmmakers want their creative work archived/preserved forever but most of the digital storage used needs constant attention or the efforts are lost in 5-10 years. Archivists around the globe are concerned that some of the best (and worst) film efforts could disappear without proper attention.
"Digital preservation is really just an oxymoron," Jan-Christopher Horak, director of the UCLA Film and Television Archive, once said.
After all, preservation isn't a profit center.
Since 80-plus percent of the stuff archived is never accessed, it's tough to justify the cost of saving all that content ... just in case.
Ironically, the only people who have pushed for true archiving (more than 15 years)/preservation has been Facebook. They live/work in the cloud so you would think they'd think just "putting it in the cloud" is good enough?
But at the FMS (Flash Memory Summit) they asked the companies for cheap chips that could be written to once and used infrequently ... the silence has been deafening.
So, they moved to the archiving media that can last 100+ years – optical. That's been an uphill battle for them:
- Discs still only hold 50GB, 100GB maybe this year but Facebook wants discs with 500MB – 1TB.
- The two firms they're pushing/pulling still think knocking out millions of copies of movies on disc is the "their future," so why fiddle with archiving?
It's terribly low tech, but the only thing better than optical for data preservation is paper and film. Yep you can read Dead Sea scrolls that date back to 100 plus BC.
Film (celluloid) isn't perfect but when they rereleased early Chaplin films they were able to piece together one complete copy from several reels.
Try to do that with a cloud/digital file 50-100 years from now.
Digital capture, production, post production and distribution are here to stay and it will only get faster, easier, cheaper.
Big budget folks will have people who take care of archiving painstakingly rotating tape, starting/checking HDs.
Indies aren't wired that way.
They'll keep all of their files close, ready for action and hope that when the time comes, they'll be up to the challenge.
They know.
Dr. Faizel didn't have to tell them, "This isn't Elysium."
