Lately the advances in display resolution, like the Retina display on the latest Iphone or the imminent introduction of 4Kx2K TV sets have raised the question what pixel density is enough? What is the maximum image quality humans can perceive?
The subject and the (sometimes ludicrous ) answers put forward by people triggered a déjà vu.
Some decades ago I have spent years in the development of ultra high quality (digital) printers. On of the most ambitious goals was to be able to proof the final print: the digital print should be an accurate prediction of the final print on paper as it would roll of the (analog) printing presses. This required bridging the sizable gap between the guys trained in bits and the old hands versed in graphics art and analog printing.
One thing we learned is that the human visual perception process is by no means simple or straightforward or linear: some artifacts in a print or display can be extremely tiny when measured but easily perceived by humans, some can be very large but never noticed. (A large part of the know-how we developed was a methodology for image processing that allowed us to shift artifacts to areas where humans are insensitive, improving image quality significantly without increasing the cost of the printing technology.)
Anyway, one of the tests we did was to print a vertical black line with an extremely accurate printer. This printer allowed us to introduce minute variations on the position of the black line, effectively making it a sinus-shaped line instead of a straight line. The amplitude and spatial frequency of the perturbation could be varied so we could test what artifact normal people would see. It showed that most people can pick up variations with an amplitude no larger than tens of microns (micron is 1/1000th of a millimeter) if the spatial frequency is right (or miss perturbations measured in millimeters with very low spatial frequencies). This sensitivity exceeded what was expected from common knowledge (maximum acuity is one black-and-white linepair per arcminute, 20/20 vision is one linepair per 2 arcminutes) . A warning to all who take the acuity of the human perception to lightly….
Getting back to displays and the required resolution, let’ s take what is said about 20/20 vision: people with 20/20 vision can identify black and white line pairs that are 2 arc minutes per pair . (360 degrees in a circle, 60 arc minutes in a degree). One arc minute equals to 0.073 mm when viewed from 250 mm distance (smartphone), or 0.87 mm when viewed from 3000 mm distance (TV set).
So that is the required resolution for a display (expressed in pixels per mm or inch)? Unfortunately not: to be able to visualize a black and white line pair the underlying pixel resolution in a display has to be a factor of 2 to 4 higher.
A display has a fixed immovable raster of pixels that must display lines and colors in any shape or direction. When slightly slanted line pairs have to be displayed (slanted to the pixel raster of the display) at a resolution close to the pixel density it gets hairy: you see sudden shifts from one pixel column to another that are very visible. This is a nasty something which is a combination of aliasing and the Moire effect : when two gratings are superimposed you get visual interference. In image processing the jump-effects of visualizing small curved details with a fixed raster of pixels is known as aliasing. ( There is a trick called anti-aliasing that mitigates the effects by inserting a grey (in case of black and white) pixel to visually smooth out the curve , to push the artifacts to an area of the perception where the eye is less sensitive).
To prevent all this you need a pixel raster that has a much higher resolution than the image you want to display.
The point of all this is that you have to multiply the number of pixels you need by at least a factor of 2 (preferably 4) to be able to show all the details a human with 20/20 vision can see. It requires at least 2 pixels per arcminute, or 700 pixels per inch (preferable 1400 which would give 4 pixels per arc minute) for a smartphone. For a TV set viewed from 3 meters distance you need 58 (preferably 116) pixels per inch.
So what size TV will a 4Kx2K image displayed at this maximum_image_quality_resolution viewed at 3 meters distance give you? Some simple arithmatic tells us: a 76 inch (diagonal) display or if you go for the highest factor a 38 inch screen. Yes , we will see the difference when 4Kx2K is introduced….
The bandwidth a 4Kx2K image needs for its transmission is not only dependent on the level of compression but also on the number of frames per second that are transmitted, and if the frames are transmitted interlaced. Interlaced transmission is sending the even lines of the first frame only, followed by sending the uneven lines of the second frame only, sending the even lines of the third frame and so on. The idea is that if the changes are minimal in the subsequent image frames you will not notice it (which is true) and it halves the bandwidth requirements.
Unfortunately for live sports events with fast moving objects (like tennis) you would like to have a fluid movement of the ball, upping the ante to prevent something called “judder” . It is not yet clear what the right frame rate should be for maximum image quality, especially for 3 D, but it is higher than we have now: rates up to 300 per second are being experimented with.
For now let’s assume some reasonable input numbers to be able to estimate the required bandwidth.
- 4000x 2000 pixel images (4 times the number of pixels of 1920×1080 HD TV)
- No interlacing
- 2 D
- 60 frames per second
How about the required bandwidth? My personal experience is with a 1080i (interlaced) HDTV signal, encoded with MPEG2, at approx 20 Mbps at 24 frames per second. I have been able to compare it to the same transmission encoded at 10 Mbps, and the difference is quite noticeable, so for me the 20 Mbps is a good reference for a 1080i/24fps TV stream. Yes, H.264 or equivalents are better compressions codecs, so lets assume a 40 % reduction by applying a better codec (hey, there is a limit to compression).
The math tells us that a staggering 240 Mbps is needed to support this superduper image quality at 2D.
Will 3D require even more? Yes but less than one might think. The images meant for the left and right eye are different but very similar, making it an ideal subject of compression technology (+20-30% so i am told). Even “3D without glasses” with many viewing angles in one display requires less extra than one might imagine.
One application where this extreme image quality will be preferred is multi-person video conferencing. We humans have an uncanny subconscious ability to read non-verbal reactions from others, which is ever so vital in a meeting. You want to see the reactions of the ones listening to the spoken word of another. Details are of the essence…
So the answer is yes we would love to have 4Kx2K displays with high fps rates as it will definitely add to our quality experience. And this will require sustained throughput far over a 100 Mbps per screen.
