http://www.quantel.com/repository/files/whitepaper_Stereo3DbroadcastingJan09.pdf
January 2009
Two years ago, if someone had said to you ‘Stereo3D TV will be the next big thing’ would you have taken them seriously?
In the next few years, assuming the world economy doesn’t turn up more unpleasant surprises, we may just
be at the very beginning of the introduction of Stereo3D Broadcasting.
Hollywood is already embracing Stereo3D film. Now, around the world, Broadcasters too are starting to
seriously investigate – even plan for – Stereo3D.
Totally new technology introduction always generates lots of questions.
This paper or blog tries to help provide some accurate answers to six common questions about what
services could we expect, how could we get them and when might they happen.
However in the end this is a personal view, based on two years of experience of tests, discussions,
presentations and actual product development. We hope it helps shed some light on the current discussion
and feel free to drop us a line if you agree or disagree or just want to talk.
Mark.horton@quantel.com
Trevor.francis@quantel.com
What is the probability that Stereo3D will happen?
Very high. Here are some pertinent ‘lessons from history’ from the
introduction of sound and colour in film:
‘Talking pictures will never replace the silent drama’
Joseph Shenck, United Artists, C.1928
‘I consider the so-callled “all-talkie”….nothing but rotten trash’
Sergei Eisenstein, 1929
‘The reaction to talking pictures is somewhat problematical. Talking throughout the entire picture has a tendency to retard the action’
Pat Powers, Cinephone, 1929
“The process of Color motion picture photography [has] never been perfected…it would tire and distract the eye, take attention from faces and acting and facial expression, blur and confuse the action….”
Douglas Fairbanks, 1930
And then a few years later:
‘Whether color can make black and white pictures as obsolete as sound made silent pictures, is, as suggested, quite another question. The silent picture was slain overnight by the jawbone of Al Jolson, whose Jazz Singer threw a hitherto sceptical industry bodily into speaking likenesses. But color is not so pronounced a revolution as sound’
Fortune Magazine, 1934
By early 1929 all the important studios in Hollywood had become thoroughly sound conscious. This was a great help to us in introducing color. Prior to that studio executives were loth to permit any change
whatsoever in their established method of photography and production. But with the adoption of sound, many radical changes became necessary. Technicolor was always confronted with objections photographing in color required more light, different costumes, a knowledge of color composition, additional time, and one or the other of these points, plus the added forceful argument that it cost more money, made it difficult for us to get started.
In my opinion, the turning point came when we ourselves produced the series of short subjects. By entering the field as a producer, by keeping very careful records of our time and money schedules, and by openly discussing with studio executives everything that we were doing as we went along, we dissipated most of the prevailing misinformation.
Technicolor Founder H. Kalmus, 1938
A personal favourite book is Alvin Toffler’s influential 1970’s ‘Future Shock’ which predicted a near-future where the speed of change is faster than we can absorb. He predicted this would disorient people and generates many negative reactions. Here we are in that future and he was right. Film and TV are technology driven businesses, yet as in many other industries, many people dislike change.
A very recent case is Digital Intermediate – the process of digital colour correction of movies. Digital
Intermediate took about two and a half more years than actually needed to gain full acceptance, mainly
because of some highly emotional resistance, although the benefits were quite literally there for anyone to see. Now, almost every movie goes through the DI process.
Of course, many much-hyped film innovations prove nothing but fads and there’s nothing wrong with some healthy scepticism. However, as individuals and as an industry, we have a responsibility to rationally think about new technology. The world isn’t going to wait around for us.
Objections fade if and when it’s clear a new technology delivers real benefits – either to the content maker or the audience. The most compelling argument is of course money. If a new technology can either save money or make more money that tends to get attention. Show business is a business.
Digital Stereo3D film making is already being enthusiastically embraced by Hollywood because of current evidence that the numbers work.
Assuming you have a film that already appeals to an audience, Stereo3D presentations get more cash through the door. Last year, Hannah Montana smashed box office records and this year dozens of titles are in production.
There are some who think Stereo3D is just a passing fad. Some say that Stereo3D already failed once in the 1950’s and that what counts is making good content. However, similar arguments were used against sound and colour in film. When film makers of the 30’s made good films that used sound and colour that audiences would have gone to see anyway then sound and colour thrived. What sound and colour did was enhance the audience experience.
Now good movies - that happen to use Stereo3D - are getting good box office. As long as we make good
films there is no reason for Stereo3D films to be a short lived fad. In the analogue 1950’s we simply did not
have the technology to make consistently good stereo films in a reasonable time and at a reasonable cost
and there were many issues with projection. Now with digital capture, digital post and digital projection, the
situation is wholly different.
It follows that the key argument in favour of Stereo3D Broadcasting is the overwhelmingly positive reaction
the general public has to it. As an industry we do well to always consider the general public. They are
everyone’s customers. We aren’t here just to sell to each other.
What exactly do we mean by Stereo3D broadcasting?
One definition of Stereo3D broadcasting might be:
‘The transmission of a high quality left eye and right eye signal to the viewer of a Stereoscopic ready
domestic Television with horizontal parallax information extracted by viewer glasses that helps convey depth’.
This intentionally narrow definition excludes:
• Anaglyph (which is a single signal with colour used to convey depth. Although simple to do, poor
• colour quality makes this an unattractive option)
• Narrowcasting to D-Cinema theatres (although this is an interesting new area)
Just now we would exclude multi-view stereo that does not need glasses. This is a bit complex to broadcast
today, is lacking in resolution and currently has limited ‘sweet spots’. So, right now this is not quite ready for
prime time, especially as some proposed camera systems are being studied that use five rather than two
sensors. However, things are changing very fast and we should keep an open mind on systems that do not
require glasses, as some very smart people are working on this now and this may become very significant in
the future.
Let’s break the production process down into sections – shooting, galleries and trucks, post production suites
and transmission.
Shooting
A pair of matched cameras, typically spaced at roughly adult eye distance, are used to capture the image.
The horizontal offset produces a binocular disparity. That binocular disparity, together with other information
in a scene, including the relative size and height of objects, occlusion, sharpness and detail, linear
perspective, object density, differential brightness, shadows, haze, and relative motion is used by the mind to
create depth perception.
No special cameras are needed, however since the only difference between the eyes should be horizontal
disparity (and not for example color, geometry or focus differences) care is needed to match and
geometrically align the cameras.
Special rigs are used which range from inexpensive and simple (which can produce good stereo but only if
carefully set up and skilfully used) to highly sophisticated rigs, like the 3ality Digital systems shown here, with
advanced mechanics and electronics which need minimal set up times and are easy to use.
3ality Digital rigs have been in action recently in many Broadcast tests including NFL and Sky
Shooting for Stereo3D is a different aesthetic than conventional TV. The audience is far more involved in the
image and wobbly camera work or whip pans are usually avoided, in favour of a more immersive and
smoothly paced style that draws the audience in.
One important topic to understand is ‘where to place the action’. Up to now much stereo has been shot for
large screens, not for domestic television sets. One artistic judgement is whether to place the objects of
interest in a shot on, behind or in front of the TV screen by ‘converging’ – which means ‘toeing’ the cameras
inwards or outwards in a similar way to how we move our eyes when we look at objects close up or further
away.
Positive, Negative and Zero Parallax
Placing objects behind the screen (positive parallax) gives a ‘window’ effect i.e. the viewer is looking through
a frame to a scene behind1 Placing objects in front of the screen (negative parallax) gives the appearance of
action happening in the viewer’s room.
An ‘in your face’ style is great for certain kinds of action but care is needed not to push objects so far that the
viewer can’t ‘fuse’ them and they get an uncomfortable double image.
Also care is needed framing shots where the action comes out of screen, if the objects are cut off by the
screen edges. This gives a so called ‘conflict of cues’ – the image is in front of the screen yet is also
occluded by the screen frame so must therefore be behind the screen.
This is much more of an issue in Television than for example in IMAX where the edges of the screen are
much further out from the point of view of the audience. Converging on an object (i.e. there is no horizontal
difference between the left and right images) places an object on the screen (zero parallax).
So, there are new considerations, as there were when color was introduced, but unlike the changeover to
colour, the cameras are the same, you just need more.
Production galleries and trucks
In some recent tests, existing production switchers (vision mixers if you are from the UK) and existing DVEs
were used. Production switchers can tie crosspoints allowing two cameras to be cut as if they were one and
DVEs can be used to horizontally slide images, simulating convergence control. Graphics inserts can be
made with a twin key and fill, introducing a horizontal offset between the signals. We don’t quite have all the production manufacturers committed to Stereo3D yet but this is changing fast and
expect plenty of news at NAB 09.
One point to mention in passing is that some stereo tests use a mux/demux workflow which combines left
and right eye into a single stream, allowing it to be passed down a single cable (or transmitted as a single
datastream). While this reduces the resolution of the image, one curious property of stereo is that the image
still appears sharp. The way we perceive stereo is still not fully agreed upon but we do know that the brain
creates stereo rather than passively capturing it and, if supplied with two lower resolution signals, many
viewers ‘see’ a high quality result.
There are a number of tests in progress, some using different methods to the ones mentioned but the
common conclusion so far is positive – it can be done.
Post production suites
The ‘fourth dimension’ of stereo is time. As well as removing any stereo or non stereo image errors a critical
role for post production is to create Stereo3D that is comfortable to watch over extended periods of time.
That means handling the z-space information on a shot so that it is technically and artistically correct and
also handling z-space over a sequence of shots, so that the eyes can comfortably adjust.
Shooting fast moving Stereo3D content is quite different from shooting stills (and can be different from
feature film work where there may be more opportunity to set up takes). However, with a little experience any
competent artist can learn how to post produce stereoscopically and there are new measurement and
correcting tools coming on the market to make life easier.
Audio post is an interesting area as 5.1 can be used very effectively to enhance depth.
Today, given the right tools, post production of Stereo3D content is becoming simpler, quicker and cheaper
than ever before.
Stereo images can now be accurately measured and corrected for example using the new 3ality Digital
SIP2100.
Transmission Schemes
Transmitting two full quality Stereo3D HD signals as independent streams is impractical as it uses up
significant bandwidth and risks the two signals picking up unwanted differential artefacts or getting out of
sync. There are several schemes that aim to remove this including Side By Side, Checker Board and
Stereo3D specific compression.
In Side by Side, a single signal is created that ‘squashes’ the left and right eye into a single picture. This is
then expanded out by the viewing device.
‘Side by side’ horizontal squeezing of the original images to one picture. Pictures are then unsqueezed in
final playout.
Because we makes stereo in our brains by some very complex and not fully understood methods, despite
the reduction in resolution, pictures still look very good. If done properly, the actual quality is higher than
many of the excessively compressed HD services being broadcast today.
A single cable can carry the signal around a Broadcast facility prior to transmission and because the signal
travels as a single stream, there is no possibility of two signals getting out of sync. Sensio and Hyundai
currently use this method.
In ‘checkerboard’, each eye is placed in a single frame rather like a chessboard with white spaces as one
eye and black as the other.
Texas Instruments Checker boarding scheme Again there is no possibility of two signals getting out of sync. A number of TV vendors already use this.
Specific Stereo3D compression can use a variety of schemes that take advantage of the similarity between
the eyes to intelligently send only needed data.
Monocular picture streams have much redundant image content between a group of frames which can be
reduced through use of long GOP compression. Stereo3D can take advantage of this too but also remove
redundant data between left and right views.
Compression can be used to reduce payload by removing redundancy between eyes over time. This
example is ‘Worldline’
Using two channels of non Stereo3D compression is very risky to use for Stereo3D because of the possibility
of concatenation or compression artefacts introducing differences between the eyes.
However good Stereo3D compression schemes are likely to find favour and companies like TDV already
have working systems on the market.
Will Stereo3D Broadcast really work?
Yes. However, like sound and colour, not everyone is a believer. Here are some current objections:
The EBU's head of emerging media David Wood said at IBC: "There is a degree of scepticism that we have
enough technology breakthroughs to make 3DTV work at the moment………. the technology is seriously
flawed in two main areas. The furthest point on the screen needs to be fixed at 6.5cm - the same as the
distance between human eyes. This can be achieved by projection in a cinema but there's no way of
knowing what size a viewer's display will be. Secondly there is a conflict between the focus and the
convergence of our eyes when viewing 3D. These psycho-physical limitations can never be solved with a
stereoscopic system….The real long-term future of 3DTV - and we are talking 50 years - is Object Wave
recording, a subset of which is the hologram"
TVB Europe, October 2008
The first point being raised here is that stereoscopic images are created in the mind by presenting two
images of a scene with a horizontal parallax difference that provides binocular disparity. That disparity makes
the viewer perceive depth. The amount of horizontal parallax is important for camera operators (or CGI
artists). Too little or too much and the viewer will not get the view that the Director intended (and in the worst
case too much can lead to headaches and fatigue). It is correct that we need to understand and manage that
depth relationship.
Just as we need to manage the amount of colour or sound in a picture (and we don’t know how loud or what
colour settings viewers are using at home) we need to manage the amount of horizontal parallax. There are
two aspects to this – training and technology. If people know that a project will be shown on TV it is possible
to take that into account during shooting and post to set up an average that will work on any screen. There is
now also technology on the market to help people get it right, or if necessary automatically correct the image
(just as in colour or sound we have metering but also legalisers). The technology can be used during
shooting, post production or delivery.
The second point being raised is that we normally focus and converge on the same point when we look
around but when we look at a Stereo3D TV screen, we focus on the screen plane but converge at a different
point (the implication being that this is unnatural and uncomfortable). However, beyond a certain distance
from a screen the subjective effect is negligible.
Stereoscopic expert Lenny Lipton deals with the topic of accommodation and convergence on his blog
http://lennylipton.wordpress.com/2008/06/11/reality-check/. Lenny wrote ‘Foundations of the Stereoscopic
Cinema’ one of the definitive reference texts on Stereo3D.
However, here is a different kind of objection.
James McQuivey, analyst at Forrester Research, said: "It's a very restricted viewing experience. You need to
sit in a certain part of the room, and wear glasses. You can't turn your head and chat to your friend and more
importantly, even if there were 100 3D films - and there are not - is that really enough to justify $1000 extra
on a television. That is a big premium to pay for just a film every week or so."
The Telegraph January 2009
While it’s true that there will be areas in peoples living rooms which give a better view that others, those
areas will typically correspond to where they sit anyway. It’s true that you need to wear glasses but as we’ll
discuss later there is no definitive evidence yet about consumer acceptance.
The lack of content argument didn’t stop sound and colour.
It is true that we don’t have much content yet. However, one immediately available source of content will
come from Hollywood as the number of new Stereo3D titles grows every month. In our tests, good Stereo3D
film content made for the big screen usually works well on Stereo3D TVs with just a little adjustment. This is
a ‘sunk cost’ as the movies are being made anyway, so with normal rights payments and some remastering
there is an existing pool of film content. There are also many Stereo3D titles from the last century2
– we’ve
been looking at how restoration can work and it seems it makes sense for some of the better ones to be
remastered. This would take a bit of time and money but is perfectly feasible.
If a Broadcaster wants to put a film channel on air, there could be enough new or remastered content for a
film channel to start sometime in 2010 and each year after that the available content will rise.
Of course, as anyone following the tests and real content creation taking place at the moment can tell you,
there’s going to be much more than film to watch – during 2007/2008, music concerts, game shows, wildlife
documentaries, tennis, ice hockey, soccer, air races, winter sports, basketball, martial arts and American
football and much more were all shot in Stereo3D.
Finally the cost of TV sets argument doesn’t hold up in the long term. Stereoscopic TV sets will come down
in price to a small percentage above a monoscopic set. If the demand and the will are there, any short term
issues will go away.
Hollywood is very interested in Stereo3D Blu-Ray right now and that (along with Stereo3D games) will help
drive TV purchases irrespective of Broadcasting.
Here are a few of the remastered titles available. http://www.sensio.tv/en/home_theater/3d_dvd/available/default.3d
How are Broadcasters thinking about Stereo3D now?
As a new choice. Childhood is called ‘the formative years’ for a reason. We grew up, like many, watching just
a few channels of analogue TV. TV was something you watched passively and there wasn’t much choice.
There was plenty of bad TV but some was excellent. Some of the advertising was really good too.
But the world moved on and we need to move on too.
For kids growing up now, things are vastly different:
• Channel proliferation means massive audience dilution for the advertiser.
• Time shifting set top boxes allow viewers to fast-forward through commercials or just miss them altogether.
• Phones are delivering content.
• PC and Mac based home entertainment provided by games, video download sites and Internet social networking sites are yet more alternatives to sitting down at home to watch conventional TV.
Games consoles also appeal to the young and the not-so-young. Games console manufacturers are starting
to look seriously at Stereo3D themselves. That creates new levels of competition for conventional
broadcasters but also presents an opportunity for Stereo3D Broadcasters. There will be an existing
population of Stereo3D TV sets which have been bought for gaming.
Kids are growing up with lots of screens delivering entertainment and information. Conventional broadcast
TV has no inalienable right to be the delivery system of choice in the future. All that said, after a hard day at
work, or at school, it’s still great as many people still do, to sit down and relax in front of a TV – especially if
you established that habit in your formative years. News and sports have also been shown to hold a good
share of the audience, if done well.
All this is well known – the question is ‘what do Broadcasters want to do about it?’ One answer is to simply
concentrate on reducing costs and maximising efficiency. That is a perfectly valid argument for some
broadcasters to take. However the efficiency argument alone can’t bring in new revenue or win new
audiences.
There are many new potential sources of revenue - that have nothing to do with Stereo3D - that
Broadcasters can adopt to offset the decline in the old 30 second commercial. These are all perfectly valid
choices, however amongst hundreds of channels Stereo3D will be a strong differentiator. Stereo3D will be
one of many choices – but a spectacularly different choice.
In fact some Broadcasters considering offering Stereo3D TV are counting on limited competition, at least in
the early days.
What’s the business case for Stereo 3D broadcasting?
Digital Stereo3D Broadcasting is now being seriously studied because it is a potentially lucrative business.
The technology to begin stereo broadcasting exists now and the first pilot channel is already transmitting in
Japan. Show the general public and you get a positive reaction.
A key debate will be around start up costs. There were initial objections to colour TV on the grounds of start
up cost and colour took many years to roll out. Interestingly the technology costs today of moving from
monoscopic broadcast to stereoscopic broadcast are much lower than the inflation adjusted costs in the 50’s,
60’s and 70’s of moving from black and white broadcast to colour.
Compare at a headline level what technology a Broadcaster needs to change:
· Monochrome to Colour Monoscopic to
· Stereoscopic
· Comments
· Cameras New cameras Same cameras Existing HD cameras used in matched pairs are suitable for stereo
· Storage New VTRS Same storage Existing NAS or SANs can be used. Pairs of
· VTRs can be used.
· Some new VTRS can record Stereo signals (e.g. Sony).
· Production galleries
· New production switchers
· Existing production switchers.
· Existing DVEs
· Most current production switchers and twin channel
· DVEs can be repurposed for Stereo
· Post production New equipment Some new, some current
· Transmission New Current
So, while there are costs (and not all of them are covered here), on a technology level they are nowhere near
what some people may imagine. Can we get people to wear glasses to watch TV?
Yes. This is the objection that is the most difficult to refute technically because it is based on the tricky field of
human behaviour. The evidence so far is ‘yes – most but not all people will be prepared to wear glasses’.
About three quarters of the world’s adult population have some level of eyesight issue that glasses or contact
lenses would improve. In those countries currently most interested in Stereo3D broadcasting, the majority of
the adult population own glasses. Of course there is a potential vanity issue, however no one much minds
wearing sunglasses outdoors if it’s too bright and the need to wear earphones didn’t seem to slow down
sales of the Sony Walkman in the eighties.
Why then do we hear assertions like ‘you’ll never get people to wear glasses to watch TV’. Polarised
stereo3D glasses are close cousins of Polarised sunglasses3
. No one much seems to violently object to
sunglasses – it’s a multi-billion dollar business - and if you already wear glasses, it will be possible to have
prescription Polarised versions made of them for Stereo3D TV viewing.
This objection can stem from a number of different factors:
• Negative experience of badly made or badly displayed Stereo3D films.
• Negative experience of viewing content while wearing anaglyph glasses.
• Poor stereo acuity.
• An honest commitment to auto stereoscopic (glasses free systems).
Anyone who has watched poor Stereo3D (including all Anaglyph) is likely to be sceptical that audiences can
be persuaded to watch it over extended periods of time. We wouldn’t want to either.
A percentage of the population have impaired stereo acuity (figures of 6% to 8% are sometimes quoted).
They may see some stereo effect but often wonder what the fuss is all about. It may be worthwhile having
you eyes re-checked if you are planning to work in stereo, as a change in prescription can help someone
with an astigmatisim.
The good news is that watching stereo at home will look much more like the first picture than the second :-)
Glasses free Stereo3D is a great long term objective but for today glasses will be the way forward.
Earphones and headphones used to be seen as ‘geeky’ too. Sony and later Apple made them fashionable.
Maybe we need someone to pick up the challenge of producing fashionable glasses.
iPod is doing quite well with earphones isn’t it?
There are also TV systems using active glasses and these, along with colour frequency based glasses are all possible ways of showing high quality TV pictures. Conclusion
We know how to shoot, produce, post produce and broadcast Stereo3D today. The content will be there in
the next few years if there is a will to create it. TV sets are already on the market, more will be coming.
Stereo3D games are already on the market and DVD will be available.
Who knows what percentage of the population will accept wearing glasses? We don’t and neither does
anyone making definitive sounding negative pronouncements. Maybe they saw bad content. What we’re
reasonably certain of, based on personal experiences of working with many thousands of people here at
Quantel, with partners, or on the road at events, is that if you show good Stereo3D correctly and use good
glasses, many a sceptic is quickly won over. Seeing is believing.
There are lots of smart and dedicated folks in all kinds of companies working hard on making Stereo3D
Broadcasting happen right now. If we want Stereo3D Broadcasting, we can have it.
Since the general public looks like they will want it too, the questions now are increasingly ‘how’ and ‘when’
not ‘if’.
This isn’t the whole story and there is much more to be said. Please drop us a line if you want to talk through
any of these topics in detail.
Mark.horton@quantel.com
Trevor.francis@quantel.com
Posts
