Following some comments on the nature of stereo perception as it relates to stereo video displays, a number of areas of interest are briefly reviewed and accompanied by extensive citations from the patent and technical literature. These include single camera(70 refs.) and dual camera(100 refs) stereoscopy, compatible 3D recording and transmission(57 refs.), head mounted displays(85 refs.), field sequential stereo(285 refs.), and autostereoscopic systems including lenticular(64 refs.), parallax barrier(22 refs.), stereoptiplexer(17 refs.), integral imaging(24 refs.), direction selective mirrors, lenses or screens(26 refs.), volumetric displays(133 refs.), holovision(13 refs.), stereoendoscopy(14 refs.),and stereosculpting(15 refs.). Interfaces for stereo graphics and the interaction of stereo video and stereo graphics are also discussed.
 
 

Stereoscopic television was a goal of the earliest experimenters with this new medium. Electronics pioneers such as Hammond, Logie Baird, Lee DeForest, Zworykin and others described 3DTV devices in their early patents(U.S. 1,725,710, 2,107,464, 2,163,749, British 266564, 292365, 321441, 552582, 557837, 562168, 573008 ). Logie Baird seems to have been the first to actually build working devices. The first commercial device may have been Dumont's dual CRT system that appears to have been sold in the 50's. Experiments with anaglyph(colored eyeglasses) video were numerous and broadcasts were done at least as early as 1953. Anaglyph broadcasts continue to be made sporadically and anaglyph cassettes and videodiscs appear occasionally but this technique, like those employing Pulfrich or prism glasses is hopeless for high quality or comfortable viewing with video, but is better with computer displays. James Butterfield broadcasted side by side stereo images for viewing with prism glasses in Mexico in the 50's. He was one of many to make stereo systems with dual cameras viewed through a binocular stereoscope. He was also one of the first to use polarized glasses to view anaglyphs by placing dichroic polarizers on the face of the CRT, an idea later refined by Benton(U.S. 4,431,265). Polarized glasses for use with dual cross polarized CRTs as well as, interdigitated images on a single display covered with crossed polarizers were proposed many times in the patent literature but the problem of manufacture prevented commercialization until Faris applied lithography to create the micropolarizer arrays(Faris). This may be a viable alternative to field sequential techniques where flat screens or projection are involved, but for CRT's it has the same problem of aligning pixels with the optical elements through a thick glass surface as the lenticular technique. Field sequential devices were described in the patent literature many times without a commercially viable product appearing until the 1980's.

Shmakov, working in St. Petersburg, Russia, devoted much time to this field during the 40's and 50's and wrote the first text on the subject in 1953 but it only appeared in 1958(Shmakov). The proceedings of the SPIE(Merritt and Fisher) and several symposia(Hamasaki-1992) are the best recent sources. The literature on stereoscopic video is large and the patent literature vast. The present review will concentrate on the field sequential technique since it is currently dominant and is likely to remain so well into the 21st century.
 
 

Stereo vision evolved hundreds of millions of years ago in invertebrates as a critical survival mechanism. The first definitive demonstration of stereovision in insects was recently accomplished by a Swiss researcher who glued tiny prisms to the eyes of a praying mantis, which then missed its prey by precisely the calculated amount. Humans have become so genetically degenerate that serious visual problems including loss of stereo perception are common. The vast majority have good depth perception but sophisticated tests show wide variations. The individual variations in stereovision should be of vital concern in the creation and use of stereo systems but are usually completely ignored. As with all other physiological systems, stereovision may improve rapidly with use, both short term and long term. Repeated use of a stereo display can lead to more rapid fusion and greater comfort. Except for a few persons who practice frequently with a wide variety of stereo displays and images, it is not possible to evaluate a stereo display system or image by casual examination. As with any other parameter, a randomly selected individual may be several standard deviations from the mean in either direction including perhaps 10% who have severe problems with stereo under any conditions and 10% who qualify as stereo prodigies due to their rapid, prolonged and comfortable fusion of images which may be unpleasant or impossible for the average person, or to their other abilities such as making very fine depth determinations. Variation with age is to be expected as is a circadian rhythm. Evaluation by a battery of users with known stereovision abilities using the hardware and software exactly as it will be employed by the end user is essential. This should include frequency and duration of use, similar imagery, ambient illumination, viewing distance and exactly the same monitor. The latter is necessary since in the dominant field sequential technique the exact hues and saturations of the images, contrast and brightness and the different persistences of various phosphors are very important. Also, the same hardware and software may yield dramatically different results if the color of figure and background are altered. Long persistence green phosphors are a common problem. Screen size and viewing distance, horizontal and vertical parallax, binocular asymmetries(illumination etc.) and nonstereo depth cues are critical. Most stereo displays and images are created and used with little attention to these factors even when highly skilled personnel are involved. A vital component of a stereo project should be a stereoscopist having extensive experience with many systems and images. This is seldom considered necessary, resulting in defects in hardware, software , viewing conditions and viewers and less than optimal images that are regarded as natural limitations of electronic stereoscopy or of field sequential input or head mounted displays.

It is even said that these are unnatural ways to look at images(as though 2D CRT'S, photos, and books grew on trees). This brings to mind the classic experiments with prism glasses performed three generations ago. When one first puts on glasses which turn the visual world upside down, it is nearly impossible to function. After a few days subjects learn to navigate and the world gradually appears more or less normal. The key phrase in the evolution of most organic systems is "plasticity equals survival". There is even some recent evidence that many strabismic (cross eyed) subjects have some depth perception due to a type of field sequential activation of the optic pathways by the reticular activating system in the brain stem.
 
 

Another common scapegoat for inadequate hardware, software and lack of stereo training is flicker most noticeable in standard frequency(e.g., 60Hz) field sequential systems. Flicker has been the subject of a great deal of research, nearly all of it monoscopic. It varies with many factors, especially screen brightness and screen size. We must distinguish the flicker due to ambient illumination("room flicker") from the flicker due to the display("image flicker"). In addition, the image may flicker due to high luminosity areas or to low rates of update. The image may still flicker even at 120Hz screen refresh if the image is not updated in the proper way(Woods). Decreasing the level of ambient illumination in the room can reduce the room flicker to imperceptible levels while reducing screen luminosity with brightness and contrast controls will reduce image flicker to low or imperceptible levels. This may reduce the contrast excessively and some level of image flicker is usual in 60Hz displays. A white wall will have a noticeable flicker which is exacerbated if a man in a dark shirt is in front of it and even more noticeable if he has a lot of horizontal parallax. The same conditions will tend to give considerable crosstalk with passive glasses or even autostereoscopic systems. However, when the image lacks high luminosity areas flicker and crosstalk may be nearly imperceptible even at fairly high brightness. This condition occurs frequently in natural subjects and can be avoided much of the time if one has the field sequential display in mind.

It occurred to me a decade ago that one could eliminate flicker by color coding the two images in every field and viewing them with field sequential anaglyphic shutters. Field or frame one would contain the right image on the green phosphor and the left on the blue and red, field two the reverse and the viewer would wear dichroic LCD shutters which would filter out the other eyes image. Each eye would get 60 images a second. The coding could also be done with 3 color dichroic shutters and 3 field or frame encoding. A color flicker would then replace the brightness flicker but since the visual system is less sensitive to color flicker this should not be bothersome. British inventor Graham Street patented this approach and actually tested it with rotating color wheels. He maintains that the resulting image was entirely satisfactory and flickerless. The Liquid Crystal Color Shutter invented by Tektronix and now marketed by several companies would probably be suitable for this use.

A related approach would work with three tube video projectors. An LCD polarizing plate is placed over the each of the three tubes. Field one could have the right image on the red tube and the left on the blue and green and field two the reverse. The polarizing plates would switch their polarizing angle by 90 degrees each field so as to images so as to always permit the right eye image to pass the right eye polarizer of the viewer wearing passive standard polarizing glasses. Again, each eye would get 60 images a second. Both these systems would work best with RGB input from video or computer systems.

An entirely different approach has been taken recently by Sadig Faris of VREX Inc. who has used lithographlic techniques to interdigitate orthogonal polarizers. Alternate strips of such a polarizing sheet can be aligned with LCD projection panels, LCD projectors or LCD, electroluminescent, or plasma screens to give flickerless stereo viewable with standard passive polarizing glasses. In some cases, such as the common LCD projection panels, the NTSC to VGA conversion gives perfect stereo from interlaced field sequential stereo with standard equipment retrofitted with a VREX polarizer. Other advantages are low cost, nonintrusiveness and retrofitability.

LCD displays could be engineered from the beginning to give cross polarized stereo pairs but except for a pair of $100,000 custom LCD projectors demonstrated by Sanyo in 1995, no such product has appeared, in spite of its description by various Japanese researchers.

Asymmetrical illumination of all or part of the image in stereo or autostereo systems will exacerbate the flicker even if the difference is only in a small area and even though it may be only a few percent off. Beldie and Kost, studying an autostereo display, found that asymmetries in the range of 3 to 6dB were noticeable and that for moving objects, even a small area of the image with 0.2dB difference was perceived. In a field sequential system Diner found that he had to take special measures to match the camera illuminations and when they were brought to within a few percent, there was a dramatic reduction in flicker. Perhaps as little as 3% difference in transmission of the right and left lenses of LCD glasses may be too much and none of the glasses manufacturers to date seem to have controlled for this. Once again, one is reminded that a great many stereo projects yield modest results which are blamed on hardware, software, or difficulties with stereo perception, but are really due to poor technique. A high degree of stereoscopic literacy is still a rare commodity.

A useful device to have would be an intelligent white gamma reducer which monitored the video pixel by pixel and automatically turned down the brightness of high luminosity areas. Such devices have been discussed in various contexts, but only Stephens(U.S. 4979033) seems to have specifically addressed stereo. I did an experiment with an expensive digital video device called a DA Vinci and found that turning down whites about 10 IRE units and turning up greys and blacks reduced flicker while retaining contrast.

With stereo graphics, it is even easier to avoid serious flicker by avoiding high luminosity areas. A black wireframe figure on a white field at 60Hz will have a serious flicker while the reverse can have no perceptible flicker provided ambient illumination is modest. This does not mean the room has to be movie theater dark, but just lacking in direct outdoor or nearby overhead lights. It is usually easy to turn up the frequency of PC video cards to decrease flicker and 3DTV Corp. was the first to include an automatic FlickerFixer™ in its software. Most television sets can be driven at higher frequencies than the normal 50 Hz (PAL) or 60 Hz (NTSC). One of the many capabilities of the SpaceStation™ (marketed by 3DTV Corp.) is the production of variable frequency field sequential stereoscopic NTSC or VGA out put from 60Hz NTSC input. Of course, as frequency increases, the number of lines per field eventually decreases. At 72Hz on a Sony TV, there was about 1/4 inch of black at the top and bottom of the screen. Experiments show that most TV’s can run at 66 to 70 Hz NTSC and flicker drops off noticeably even at the lower rate (33 Hz/eye). A variable frequency external FlickerFixer box could be built for about $200 as a consumer item, but a custom LSI chip could reduce cost to $20. Broadcast of higher field frequency signal for stereoscopic programs is also a possibility.
 
 

It is frequently stated that stereo images will be of inferior quality to mono images on the same system since each eye is getting half the bandwidth. With graphics, it is often hard to tell since there is usually no clear reference but with standard video camera imagery, the subjective resolution is often strikingly superior. Ordinary consumer NTSC tv's with well done VHS stereoscopic tapes look equal or superior to any HDTV I've seen. One reason is that stereoscopic acuity(resolving ability) is superior to monoscopic acuity. This is due to the fact that that stereo will usually have a greater information content than mono and the highly sophisticated image processing systems in the brain have been evolved to take advantage of this.

In the extreme case, two views having a million pixels each, taken respectively from the right and left sides of the head will present a richer image processing potential than a single two million pixel image taken from directly in front. Of course, there are a wide variety of possibilities and the relative 2D vs 3D vividness, usefulness and information content will depend on precisely how the images are captured, processed, stored, displayed and used.

There are probably neural hardware functions for edge enhancement, shadow detail, perspective, texture, glitter, sparkle, feature extraction etc. which work only (or best) when the stereo systems in the optic cortex are activated. It is to be expected that these will interact in complex ways. The effects of training, fatigue, motivation, drugs and other factors on perception suggest that these functions are programmable to varying degrees(again, this will vary greatly with the individual). This is fertile ground for research, especially with the recent availability of low cost means for field sequential generation and presentation of stereo images. The SpaceStation™ from 3DTV Corp. and the Tiga Stereoscope from Vision Research Graphics are unique devices for such studies.
 
 

There are several approaches for creating stereo images with a single camera. One of the simplest and most frequently used has been to place an optical adapter in front of the existing lens. A lens of this type employing liquid crystal shutters was briefly marketed by Azden Corporation in 1990. These lenses have many limitations such as the need to operate at telephoto, ghosting, and lack of control over interaxial, though a recent design minimizes some of these (JAP-1-147444).

Alternatively, various types of mechanical or electrooptic devices can block the light through parts of the optical path to create field sequential stereo pairs (USSR-138273, 369732, 568220, 1125783, 1109959, US-2508920, 4486076, 4943852, 4281341, 5028994, JAP-57-5490 to 5493, 57-14268 and 14269, 57-25783, 59-225692, 62-98895, 63-227192, 1-22187, 1-55998, 1-41397, 1-41398, 1-47192, 1-47193, 1-132294, 57-72134, 63-237687, 57-14268, 57-14269, 57-75089, 57-62686, 56-158590, 56-83193, 83194, 83195, 83196, EP-269075, GER, 3214021, 2032977). The fact that a small interaxial (stereo base) results from dividing the lens into right and left halves means this technology is only good for close-ups. Stereoendoscopes using internal liquid crystal shutters have recently been created by several companies (SOCS, International Telepresence, OLYMPUS).

An interesting variation is offered by cameras which translate in the Z-axis or have elements which cyclically change their index of refraction to give depth information (JAP-61-80992).Limitations of sensors have led to somewhat complicated line scanning arrangements for single sensor infrared stereocameras (4574197, 4682029), but recent advances in sensors and image intensification may make these obsolete. Alternatively, mechanical, optical or electrooptic barriers can divide up the frame or interdigitate the stereo pair on the image surface every field (USSR-510,812,1107344, JAP-51-958, Masters, US-2317875, GER-3205483). Palmer devised a method for getting an over/under wide aspect ratio stereo pair with one or two cameras in 1951 (US-2786096-cf US-4583117, 5049988). Anamorphic fiber optics which could be useful in this application are now feasible(U.S. 5015065). Many of these approaches using single sensors have had as their object the input for an autostereoscopic display (GB-1401003, EP-335282, 4943860, FRE-1362617, US-4945407, 3932699).

If the subject or camera are moving, stereo pairs can be created by various optical, electrooptic, mechanical or electronic means (JAP-1114293, GB-2180719, US-4231642, 5014126). This approach has been the subject of a great deal of interest in recent research in robotics, stereophotogrammetry and pattern recognition. Light can be scanned over the surface of an object from one or more locations and its spatial location, frequency, time of flight or polarization can be analyzed by the multiple elements of a single sensor to yield positional information (US-4945408, 5018854, 5024529, JAP-56-34289). In some cases this technique can replace the lens and camera with photodiodes. Also, two images can be passed through colored filters, completely overlapped every field and separated at a subsequent stage with colored filters or electronically (USSR-873464, 291376). An inexpensive lens of this type is available from Spondon Film Services in Derby, England. Phillips' method of underscanning the raster on tube cameras could give field sequential stereo suitable for closeup work (US-4740839).

Finally, much effort is being expended in pattern recognition on extracting depth information from a single point of view combined with other information about the scene (US-4754327, Lippert, Alvertos). Any of these imaging techniques with one or more cameras can be combined with a wide variety of display modalities including stereoscopes, polarized, prismatic, anaglyphic, mechanical or electrooptic spectacles, or autostereoscopic (no spectacles) means including lenticular, louvered, or parallax barrier screens as well as large diameter mirrors or lenses or a wide variety of volumetric displays.
 
 

Hundreds of researchers have created mechanisms for controlling various parameters of a stereocamera pair. Though much of the work on stereophotography and stereo motion pictures is relevant, we will limit the discussion to some of the more recent efforts with video. The two cameras need to be kept aligned within close tolerances in all three axes. Most recent work has taken this for granted and Toshiba's patent on it's three axis adjustment means for the two lenses of it's stereocamcorder is one of the few to describe this mechanical setup in detail (JAP-177530, cf. JAP-63-164596, 63-164597, 1-89796). There is a need to control the zoom, focus, interaxial (distance between the cameras) and the convergence point of the two optical axes. Since there are fairly precise relationships between these parameters, much work has been directed at interlocking several functions. The older literature described mechanisms for manual interlock of focus and convergence (USSR-506,953, 506954, 527030, 803128, 902323, 918926, 849547, 720819,506954, 228069, 471689, 1053329, 445175 , JAP-51-142218, 60-216205, 62-100095, 63-228141, 1-11254, FRE-1251830), or of zoom and focus (JAP-57-62687), or for manual adjustment of one parameter at a time for both cameras (JAP-59-192239, 1-225936, 1-212079, 1-11490). Some altered convergence by changing the scanning position on the image pickup surface (JAP-57-109492, cf. US-4740839,5049988, 5063441).

More recent efforts have usually attempted to automate these functions with application specific circuits or with programs written into a dedicated microprocessor or general purpose computer (JAP-56-106490,61-101882, 61-101883, 62-21396, 62-122493,62-266534, 62-266535, 63-228141, 63-164594, 63-153987, 1-212976, 1-93983, 1-93984, 1-251990, GB-2168565, USSR-873458, 552729, 1,148128, 1095454, EP-332403, 146476, US-4819064, 4818858, 4881122, cf. 5020878). Some have relied on digital storage and image processing to compensate for binocular asymmetries from zooming (JAP-1-231590), to reduce excessive horizontal parallax (US-4677468, 4723159 and many others), to effect simultaneous image capture (JAP-1-86692, 1-68192, 1-93977, 1-93978, US 4772944) or to eliminate camera shake (JAP-1-228392. Morishita of NEC has suggested (US-4677468) increasing aperture to blur objects with excessive parallax and automatic locking of the video levels of the two cameras-the latter also described in Japanese patents 63-158993 and 1-177795. Kinoshita also dealt with luminance matching and convergence (JAP-63-7094). A clever Japanese patent shows how to automatically adjust image size during zooms to size of the display to avoid image cutoff and miniaturization (63-296489). We are clearly entering the era of the "smart" stereocamera. Several companies have offered prototypes for sale including Ikegami's system with broadcast cameras and 120Hz scan converter for about $140,000, and one from 3DTV Corp. using for $10,000 which has microprocessor controlled synced zooms. Stereoscopic video is most conveniently and inexpensively created with a pair of genlocked cameras and the Model 100(composite) or Model 200(component) StereoMultiplexer available from 3DTV Corp. These units are battery powered and about the size of a VHS cassette.(Starks, 1990) Stereo video can be genlocked to stereo graphics easily, but one has to be alert to match up the right eye pairs. The same comments on flicker apply as for graphics with the addition that cameras should be very closely matched for luminance(Starks, 1992). The multiplexers give field sequential stereo for recording and for aligning cameras and viewing stereo with any CRT. Hardware for converting 50 or 60Hz stereo to higher frequencies is available from 3DTV Corp.

Demultiplexing of the field sequential image can be done by the Model A StereoDemultiplexer from 3DTV Corp. which separates out the R and L images for dual videoprojection viewed with passive polarized glasses. Flicker is a problem with tube projectors but LCD projectors give little flicker. LCD projectors may require orientation of polarizers different from the movie standard, but this is easily corrected with half wave plates. The Model A takes in composite field sequential video and puts out 30Hz right eye fields alternating with 30Hz black from one BNC and 30 Hz left eye alternating with 30Hz black from the other. The Model B does the same thing with composite or two or three component video. The SpaceStation marketed by 3DTV Corp. in 1994 adds back the missing fields to give the 60Hz right and left fields. To eliminate the trouble of dual VCR record and playback systems, the SpaceStation also permits the two fields to be record on one tape in a side by side or above/below compressed format. This will again give dual 60Hz output on playback.

Timecoding of tapes and playback with dual computer controlled VTR’s permits cheap flickerless high quality stereo. It is also useful to have separate R and L tapes when doing standards conversion since standards converters will destroy field sequential stereo. The R and L tapes can be separately converted and then mixed into stereo in the new standard with the StereoMultiplexers. However this is likely to produce serious artefacts. 3DTV Corp. markets a unique standard converter, that is compatible with field sequential stereo.

Other techniques have been proposed and occasionally marketed, but they involve use of expensive, bulky, nonstandard equipment for recording and display. A sensible approach is to begin with the StereoMultiplexer at 60Hz and move to the dual 60 Hz if desired.

Others have devised new techniques to improve camera performance. Karibe of Sharp Corp. described an automatic camera tilt detector (JAP-62-276987, 62-266533). Many have described camera switching, digital storage and/or processing or novel display techniques to improve the actual or apparent vertical resolution since there is often a decrement in this parameter (JAP-63-164598 and other cited later). Shimada of SONY mixes arbitrary numbers of left and right eye fields (JAP-1-202985). Osawa uses two cameras with electrooptic shutters and a single common optical element to facilitate synchronous zooming (JAP-1-54438). It has occurred to several researchers that one or more high resolution black and white cameras can be combined with a low resolution color camera to give a high resolution stereo image that would be otherwise unobtainable or very expensive (JAP-62-73896, 63-177690, 1-177292). A Mitsubishi patent employs an ultrasonic sensor on the monitor to automatically adjust the camera parallax to a viewers position (JAP-60-236391). Yatagai shows how to transfer charges between two CCD cameras to obtain low light stereo (JAP-1-93982). One of Maezawa's many stereo patents for Sharp describes a simple optical device for matching stereo camera pairs (JAP-63-143524).

Many designs have been directed at robotics, photogrammetry or pattern recognition applications (JAP-60-140113, 60-27085, 60-119191, 60-119192, Schenk and Toth). Hitachi engineers have created sophisticated automatic stereocamera controls for incorporation in a robot used in nuclear facilities (JAP-62-115989, 62-21396, 62-122493). The Harwell nuclear plant has an elegant system (Dumbreck et al., Scheiwiller et al.) which uses computer control to couple focus and convergence but they note that cases arise when the operator should be able to decouple these parameters. This system has also been installed in plants in Korea and elsewhere. Suzuki's stereocamera automatically tracks objects and adjusts the zoom to keep them centered (JAP-60-152193). Multiple fiber optic bundles coupled to sensors have been used as stereo pickups (JAP-60-58789). It is also feasible to use three or more cameras with rapid updating to obtain the best stereo pair or to extract depth information with algorithms that combine all viewpoints (JAP-61-125685, EP-0199269, Cheung, and Brown, Dhond and Aggarwal, Stewart, Wilcox et al.). Copeland suggests using wing mounted cameras as a navigational aid to increase interaxial from the normal 65mm to 65m (U.S.-4805015). Simulator experiments on terrain following with stereo video were carried out in the 1970's (Bruns).
 
 

Much of the early research on color television involved field sequential color systems and many of these workers described means for using their devices in a stereo mode. Baird's efforts (GB-321441) are well known but others were even earlier. Hammond's patent, filed in 1923, described sequential color and stereo (US-1725710). Interestingly, a toy company briefly marketed a field sequential stereo, field sequential color vector graphics system sixty years later. Many subsequent efforts used mechanical shutters for projection and or viewing of stereo slides, motion pictures or television (US-2362030, 2,384259, 2384260, 2408115, 2825,263) and patents on such devices continue to appear (GER-3303739, W0 79/01035) but very few resulted in a commercial product. Knauf's "rotating beer can" (US-3464766) is now obsolete as is the Matsushita viewer for the Sega Subroc 3D arcade game (JAP-56-69985, 56-155917, 56-156079, 57-5490, 57-5491, 57-5492, 57-5493, 57-14269, 57-25783, 59-171392, 60-7291).

Kerr cells and related electroopic polarization rotating devices were employed from the earliest days of television, mostly as a means for obtaining color in field sequential or line sequential schemes and stereo means were often described (US-2002515, 2118160, 2417446, 2616962, 2638816, 2665,335, 3358079, GER-736457, 2055935, 2140944). When the transparent PLZT ceramics became available in the 1970's, they were quickly put into service but were soon supplanted by liquid crystals. The amount of research as evidenced in the technical literature has become staggering. Japanese patent applications on field sequential stereo have exceeded 400 in the last decade alone. A few of the earlier non-Japanese patents to specifically mention LC shutters are those of Varga (Romania-58504), Schieckel (GER-2111067, Hossmann (Swiss-534365), Kratomi (US-3737567), Roese (US-4021846) and Mears (GB-1523436).

The availability of low cost LC shutters greatly stimulated research and means were described to permit video field recognition to ensure the right eye image getting to the right eye (US-4145713, 4387396, JAP-63-164788,1-245693,1-86693), to sync the glasses via a photodiode on the monitor screen (JAP-62-209994, 63-214095, 63-294096, 1-248796,1-68191), via a magnetic pickup on the monitor (JAP-63-248294), or without wires via infrared, radio or ultrasonic transmission (JAP-58-62995, 62-91095, 62-239784, 63-1286, 63-64016, 63-59089, 63-117596, 63-64016, 1-67095, 1-68191, 1-17590, 1-206798, US-2388170, 3621127, 4286286, 4424529, 4562463, 4732457, 4979033, 4967268, FRE-2334255, 2399173, GER 3214021). Many patents contained variations on LC driving circuitry, often with the aim of decreasing the flicker of 60Hz systems (JAP-61-227498, 61-277918, 62-166314, 62-204226, 62-242914, 62-254118, 62-266996, 63-31393, 63-31394, 63-158994, 63-43621, 63-205641, 63-213821, 63-290485, 63-314991, 1-44421, 1-51789, 1-51790, 1-86694, 1-103394, 1-149590, GER-3413211), others were concerned with keeping the shutters transparent when the viewer looked away from the display (JAP-63-212290, 62-231578), when the viewer was looking at the camera of a videoconferencing system (JAP-63-194497), or when viewing a 2D part of the display (JAP-63-215195). One NTT researcher even devised means to remove the glasses entirely by using stored images of the viewers (JAP-1-251989). Some work has been directed at improving performance by novel methods of constructing the shutters (JAP-62-89925, 62-71395, 62-156619, 62-166314, US-4884876). Only a few of these designs ever were marketed. Four types were available from 3DTV Corp. in 1990 for prices ranging from $50 to $200 with a variety of drivers able to accept video or TTL input. These all work well at 60Hz and several perform well at 120Hz, particularly if the background and foreground hue and saturation are adjusted to minimize flicker and crosstalk. By 1992, four different companies had marketed wireless LCD shuttering glasses.

All the above work applied to twisted nematic LC shutters (and in a few cases to PLZT ceramics) incorporating crossed polarizers. Many have suggested using ferroelectric LC shutters (JAP-63-30088, 63-64016, US-4772943) because of their fast switching times. Vision Research Graphics introduced a commercial product in 1992. When used in conjunction with a special amber-green monochrome phosphor, there is virtually no crosstalk(ghosting). Some effort was made to develop cholesteric LC shutters for stereo viewing by scattering without polarizers by various Japanese scientists, and by Milgram in Canada (US-4698668), and Noble and McSherry in California. They seem to offer no advantage since they appear not to decrease flicker and give a milky look to the image, but Noble suggests using a black matrix to reduce scattering. Milgram markets them for use by perceptual psychologists.

Tapes in the field sequential format are compatible with all standard NTSC and monitors except some of the IDTV products and LCD TV's or LCD projectors which mix fields. Some VCR’s by Instant Replay (Miami, Fl. U.S.A.) or the Akai(now Mitsubishi) VSR19EMb, will play NTSC at 60Hz on PAL TV's, but none appear to be 3D compatible. This works with most PAL and SECAM monitors and receivers because they lack vertical countdown circuits and will sync to 60Hz. In some PAL countries(e.g., Sweden) nearly all the TV's accept 443MHz 60Hz NTSC. NTSC and PAL-M (Brazil) VCR's should play 3D NTSC tapes on PAL and SECAM TV's but without color. This trick of driving consumer televisions at higher frequencies should also work for 3D videogame systems and computer graphics and is employed in 3DTV Corp’s FlickerFixer device (SpaceCard).

The 60Hz flicker can be virtually undetectable if the ambient light is low, monitor brightness is adjusted and images avoid large light areas. Acceptance by consumers and professionals has been excellent. Sega sold over 100,000 of their 60Hz home 3D game systems, mostly in the U.S. and Japan and perhaps 40,000 50Hz systems in Europe and elsewhere and Nintendo sold some 80,000 of their 60Hz units in Japan in the late 1980’s. Systems operating at 60Hz have been successfully marketed for the Atari, Amiga, and recently for PC's. Nevertheless, there has been much effort directed at methods of reducing flicker. Some have processed the video to reduce areas of high luminosity (U.S. 4979033). Many workers have suggested eliminating flicker entirely by doubling the field rate to 120Hz. Some have created a four fold interlace by inserting extra vertical sync pulses with standard monitors (US-2696523, 4523226, 4583117, 4517592, cf. US-2389646) while many others used field stores and broad bandwidth monitors to eliminate flicker and perform other image manipulations (JAP-54-30716, 56-168484, 57-87290, 57-119584, 57-138285, 58-139589, 60-100894, 60-203095, 60-223282, 60-263594, 61-113389, 61-273094, 61-293093, 62-86997, 62-132491, 62-133891, 62-136194, 62-145993, 62-150591, 62-265886, 62-278889, 63-30088, 63-31295, 63-46091, 63-88994, 63-95795, 63-116593, 63-131685, 63-131686, 63-133791, 63-164598, 63-181593, 63-219293, 63-224495, 63-231590, 63-232790, 63-245091, 63-258187, 63-266980, 1-27390, 1-39187, 1-47194, 1-47195, 1-47196, 1-54886, 1-61192, 1-61193, 1-69196, 1-93988, 1-93989, 1-93993, 1-93994, 1-212091, 1-252093, US-4,393400, 4672434, 4772944, USSR-1166344) and many others. Siemens, Philips, Sony, Metz and Grundig have marketed limited numbers of tv sets with field doublers, at least some of which can be modified to be stereo compatible(Woods et al.). The 3DTV Spacecard is unique in its ability to give continuously variable frame rates for NTSC or VGA, stereo output from field sequential NTSC 60Hz input. Ikegami Sony-Tek and 3DTV Corp. have introduced units to double the field rate of standard field sequential 3D video. Cahen in his French application of 1948 (US-2665335) and many subsequent researchers (US-3358079, 4772943, JAP-63-46410, 63-116591, 63-116592, 63-245091), noted that one can switch at line rate. Like 120Hz switching, this will eliminate flicker of ambient light, but will not eliminate image flicker unless each eye is given about 45 or more new images each second.

The use of two videoprojectors with crossed polarizers and a front or rear projection polarization preserving screen gives a large screen and allows the use of cheap, standard polarized glasses. In general, it will also give less ghosting than with a single field sequential display whether projected or direct view with active glasses or with passive glasses and screen polarization modulators. This is due to phosphor persistence in the active glasses case and phosphor persistence combined with scattering by LCD modulator in the passive glasses case, since these problems are absent with the cross polarized dual projector system. Two genlocked computers can generate the images or two video players can be run in sync with right eye and left eye time coded tapes and a suitable edit controller. A single field sequential source input to the StereoDemultiplexers. will output two separate signals of about 30Hz(depending on input frequency) alternating with video black. This will flicker most with 50Hz PAL input and CRT projectors and least with 60Hz NTSC input and LCD projectors. The LCD projectors are slower and flicker may be almost undetectable even when each projector is input with 30Hz NTSC. Some LCD projectors(e.g. Eiki, GE models available in 1992) require the use of polarizers at a nonstandard angle(45 degrees to right and left is standard for polarized glasses) but others such as some from Sharp work at the standard angle. A half wave plate will rotate the polarization place if needed.

The following table may be useful to those trying to decide which display option will best suit their needs. It is highly subjective, being based on my own judgment, and image quality will also vary with subject matter, quality of stereo, monitor or projector model, ambient illumination and other factors. Active glasses are LCD shuttering glasses. Passive glasses are circular or linear polarized glasses with polarized images created with an active LCD plate(StereoPlate) on the single monitor or projector or with polarized sheets placed over the lenses of the double projectors. The Model A Demux sold by 3D TV Corp. separates field sequential composite video into separate right and left channels with 30Hz images alternating with 30Hz black fields. The Model B Demux does the same with composite or component input. HighVision™ is the smart line doubler marketed by 3DTV Corp. 

The use of Kerr cells at the CRT with viewers wearing passive polarizing glasses likewise grew out of the early work with sequential color schemes and is mentioned in many of the above citations. Many other references specifically describe screen switching (GB-1448520, JAP-50-75720, 52-110516, 54-101689, 60-153694, 60-203095, 61-9618, 61-203794, 62-71394, 62-81620, 62-299932, 63-85717, 63-182991, 63-203088, 1-128039, EP-0136696, 0223636, 0264927, USSR-544183, 642884, 657673, 1166344, Neth. Appl. 7807206, US-3858001, 4719482, 4719507, 4792850,4870486, 4879603, 4877307). 3DTV Corp. sells a StereoPlate for polarized projection with a single videoprojector.

Much attention has also been directed to adapting existing tape and disc systems for high quality 3D recording and playback. An obvious route is use of a dual head VCR and/or double speed rotation (U.S. 5050010, JAP-62-102679, 62-165488, 62-166669, 62-245784, GER-3234846). David Burder and his colleagues in England have modified an old quad VCR for multichannel 3D. The new JVC digital VCR (1995) is capable of recording two full bandwidth composite signals. Work on 3D discs has included Sanyo's dual system with right and left images on separate discs, Hitachi's machine with the two images on opposite side of a disc (JAP-276393), Pioneer's optical disc recorders (JAP-63-266980), Alps' magnetic disc recording on two adjacent tracks (JAP-1-109892), and numerous others with field sequential or dual track systems usually with 2D compatibility (JAP-55-50638, 55-50639, 61-212192, 61-252778, 62-91095, 62-128294, 62-176394, 62-260496, 62-266995, 62-276989, 62-285595, 62-295594, 62-295595, 63-6992, 63-116590, 63-151293, 63-227296, 63-228895, 63-229994, 63-232789, 63-276393, 63-316981, 1-49396, 1-94794, 1-109989, 1-109990, 1-177294, 1-183993, 1-206798, US-4739418). A field sequential 2D-3D compatible system was offered for a brief period by several Japanese companies in the now defunct VHD system.

Much thought has gone into means of interlacing fields and/or doubling lines (US-3991266, 4736246, 4772943, JAP-61-212191, 61-212192, 61-280193, 62-145993, 62-154894, 62-210797, 62-230292, 63-94794, 63-164598, 1-24693, 1-55999, 1-225295) or otherwise processing video (JAP-61-24393, 1-272286, 63-84292, 63-84393, 63-84394, 63-84395, USSR-303736, 1188910, Woods et al., and many others referenced above) for improving the resolution in field sequential systems. Lowell Noble and Ed Sandberg of SOCS Research in Saratoga, Ca. have developed a stereo compatible board which line doubles and image enhances to give superb 2D or 3D on any VGA or other multisync monitor or projector. Two different groups applied ghost canceling techniques for eliminating crosstalk due to slow phosphor decay (JAP-55-85181, 56-106491). Some workers have described means to compensate for subject motion (JAP-1-165293, 1-171389) while many others have devices for parallax reduction for reduced ghosting and visual fatigue or to manipulate the stereo window (JAP-57-21194, 63-62485, 63-86691, 63-142794, 63-176081, 63-227193, 63-245090, 63-306795, 63-314990, 1-265798, US-4399456).
 
 

Until recently, those who wished to work with stereo graphics have had to spend many thousands of dollars for cards, multisync monitors and LCD glasses and have then had to write their own software systems. Gloves or 6D mice have cost thousands more. Beginning in 1990, 3DTV Corp. began introducing low cost system including universal interfaces, several models of StereoVisors(LCD viewing glasses) and stereoscopic computer sofeware. Several of these interfaces have ports for gloves and related devices. In 1994 and 1995, many other companies began using this technology and complete systems for interactive stereo imaging became available for nearly any computer for less than a tenth the previous cost.

One of the most useful of these devices is the Model 3000 StereoDriver from 3DTV Corp.. It has a cable which replaces or adds on to the end of the VGA cable between the monitor and the PC. It is a high density(15 pin) cable with a D9 size plug. It has an extra wire which takes vertical sync from the PC . This wire terminates in an RCA plug which is attached to either RCA jack of the Model 3000 StereoDriver. The StereoVisors(LCD glasses) plugged into the Driver will now cycle in sync with the right and left eye images. On starting up, there is a 50% chance that the right eye image will go to the left eye. To be certain when this happens, it is advisable to put an R on the right frame and an L on the left frame when creating the software so one can tell immediately which field is being viewed by closing one eye. If the left eye field is being seen by the right eye(pseudoscopic image), flipping the polarity reversal will correct the polarity. It will follow sync to at least 120Hz and can also be used for viewing stereoscopic videotapes, discs, CD ROMS etc. in any format(NTSC, PAL, SECAM, etc.). If the right eye image is always recorded on field one, the field recognition circuit will automatically route the right eye image to the right eye. It has two standard stereo headphone mini jacks, can be used with all commonly available wired LCD glasses, and with use of stereo splitters will drive up to 8 pairs of Visors. Similar cables can be supplied to adapt the StereoDriver to any computer having external access to video sync(usually on the green pin for RGB monitors).

A second stereoscopic interface marketed briefly was the Model RF StereoVisor and Model RF StereoDriver. This driver contains a magnetic pickup which obtains sync for the glasses from the magnetic flux of the monitor. The driver is plugged into AC power and laid on top of the monitor . It will drive the Model RF wireless glasses or most models of wired LCD glasses via two jacks on the rear of the driver. Wired and wireless Visors may be used simultaneously. As with the 3D Cable, there is a 50% chance of a pseudoscopic image on startup. Switching the driver power on and off or moving it a few times will result in a stereoscopic image. Again, the optimal situation will be to have the R and L frames identified in software and with driver will work with monitors, tv's and videoprojectors with the exception of some which are too well shielded. The RF Driver should work will all video and computer systems to at least 80 Hz and perhaps higher and can drive any number of RF Visors and at least 4 pairs of wired Visors. Since it depends only on the monitor flux, the RF system should work with nearly any platform without the need for any connection. At least one other company marketed a magnetic pickup but this method is unreliable and has been discontinued.

A third device is the PCVR. This will interface with parallel or serial ports by flipping a dip switch to the appropriate position. Another dip switch permits line selection when changing from one type of serial port to another and a switch to select polarity of LCD glasses. It will drive the glasses and the Power Glove (marketed as a low cost game device by Mattel in 1990), or other interface devices and has an additional port for the printer which is put on or off-line by turning a knob. Pseudoscopic images are not a problem since the computer now has complete control over the right and left lenses of the LCD glasses, but a switch is provided to reverse the lens polarity. It is probably still a good idea to put R and L indicia in the lower right of the right and left frames respectively, at least when programs are being written. The indicia or any other graphics should not be put at the very top of the screen because hardware and/or software problems producing distortions at the top have arisen in many systems from various sources.

Model O driver uses an optical pickup on the CRT which is triggered by alternate white and black screen indicia created in software.

The Model IR Driver takes sync from VGA like the Model 3000 but transmits the sync via infrared to wireless glasses.

Another device similar to the PCVR is the PGSI, produced as a class project by a group of college students in 1993. This interface plugs into serial ports and drives the glasses and the Power Glove. It contains a microprocessor and has software which allows more sophisticated control of the Power Glove.

The PCS and PCP are the most compact and least expensive of the interfaces, being small enough to fit inside a gender changer and taking power from the serial port or parallel port respectively.

Flicker with these low cost computer systems can be small or even imperceptible if the problem is understood and all parameters are controlled, as discussed above. Most of the LCD glasses that have been marketed have incorporated a layer of black plastic in front of the LCD to reduce room brightness. If the room lighting is reduced and monitor brightness decreased, even field sequential video displayed in the European 50Hz PAL system can be quite acceptable. Most PC graphics cards run at 72 Hz in VGA which much reduces flicker even in bright environments. Any card can have its frequency increased by using its menu. Some 3DTV Corp. software contains an automatic flicker fixer which works with most cards in 320 x 200 mode.
 
 

Starks(91) surveys the available eye and head tracking techniques. Though most of the work to date has been monoscopic, most of the hardware can be used stereoscopically and applications are appearing (G.B.-2,201069, Yamada et al). An obvious use is to couple such a device to a stereocamera for totally automated socs and 3D videotaping (JAP-62-115989). An interesting device from NEC anticipated virtual reality research by using eye movements to alter images in a helmet mounted stereo display (JAP-61-198892). Deering (1992) notes that stereo eyetracking will be necessary for highly accurate interactive stereo.
 
 

Head mounted displays have a long history. McCollum described a field sequential, head mounted system with dual CRT's and wireless transmission in 1943 (U.S. 2388170). Science fiction pioneer Hugo Gernsbach modeled a prototype of unknown origin in the early 1960's. They have been the subject of extensive R&D in many countries, mostly for avionics but more recently for tanks (Brooks, Rallison and Schicker), foot soldiers (Varo Inc.), vision aids(U.S. 5060062), surgery (GER-3532730, U.S.-4737972, 5039198, Pieper et al.), computer workstations (Teitel) and entertainment (U.S.4982278, 5034809). Varo Inc. has a series of intriguing patents describing wireless infrared transmission of video from gunsight to helmet and to other soldiers (U.S.-4884137, 4970589). Another patent uses a head mounted camera for simulator purposes (U.S.-4398799) and Thompson-CSF inputs stereocameras through fiber optics (FRE-2517916). The SPIE volumes series Helmet Mounted Displays, Display System Optics, Large Screen Projection- Avionic and Helmet Mounted Displays, Cockpit Displays and Visual Simulation, etc., the SID Digests of Technical Papers and the NTIS searches on HMD's (PB89-872105/CBY), etc., provide good surveys. Most of these devices have aimed to provide a head up display of flight information or other data with the pilot having his normal view of the cockpit with the data displayed on a semisilvered mirror or holographic optical element (Amitai et al., U.S. 5035474, 5040058). For many other purposes, it is unnecessary or even undesirable to see the real world and the helmet displays all the information. Telepresence and robotics have been mainly concerned with displaying video, while virtual reality research has so far used such systems for computer graphics. One Air Force project developed "hands off binoculars" (U.S.-4465347).

Earlier work used miniature CRT's (U.S.-3614314, 3059519, 2955156) but LCD's and lasers are now frequently used. Most devices have required complicated optical trains to get the CRT image in front of the eyes (U.S.-4859030, 4269476, 4322135, 4761056, 4969724, 4902116, 4468101, 4775217, 4968123, 4961626, 4969714), but recently fiber optics has been employed (Thomas et al., Webster, FRE-2517916, CAE Electronics). Most of the recent systems incorporate head and/or eyetracking (U.S.-4897715, 4028725, East Ger-224691, Arbak). The Eyephone (TM) from VPL Research was marketed in the late 1980's followed closely by the Cyberface(TM) of Pop Optics Labs and the elegantly designed Virtual Research Flight Helmet(TM) in 1991. An ultracompact design by William Johnson of England used his GRIN optics. Some of the more expensive avionics devices such as the Agile Eye from Kaiser Electronics (Arbak) are available to defense contractors and possibly others. Dual LCD systems from three companies entered the personal computer and toy markets in 1995.

There have been many descriptions of dual LCD devices intended to display video for low cost applications (JAP-63-82192, 63-177689,59-117889, 59-219092, 62-272698, 1-61723WO 84/01680, GER-3628458, 3532730, U.S.-4571628, 4982278, 4933755, 4952024, 4805988). The Litton magnetooptic chip has also been used, but it cannot display blue, so full color is not possible (U.S.-4575722).

Another group of lightweight displays intended for helmet or eyeglass mounting has recently appeared (U.S.-4311999, 4902083, 4934773, 4753514, 4867551, Upton and Goodman, Pausch et al.). These involve vibrating optical elements such as mirrors or fiber optics to scan the image onto a mirror. Though these have been monochrome data displays, color and full video would be possible. A device of this kind called The Private Eye has been marketed by Reflection Technology of Waltham, Ma. and Peli has published a careful evaluation of it. British stereographer David Burder has created a stereo HMD using two of these devices which gave a reasonable stereo effect. Nintendo licensed this technology and introduced the Virtual Boy Game System in 1995.

Much of the recent work with HMD's has emphasized wide angle viewing (Howlett, Howlett et al, Fisher, Robinett, Robinett and Rolland, Teitel). Wide angle stereo has a long history in photography and Harvey Ratliff deserves mention as a pioneer in this area and as the father of wide angle stereoscopic video. He built several devices and proposed others in a series of patents in the 1960's (U.S.-3511928, 3504122, 3376381, 3293358,3291204). Ratliff used conventional lenses, while more recent patents on panoramic HMD's have proposed more exotic optical techniques (U.S.-4385803, 4853764, 4874235). It is not clear that wide angle optics give sufficient advantage to justify the trouble and expense nor do there appear to be enough data to tell whether most people will find them comfortable with prolonged or repeated use.

The psychophysics of depth perception in head mounted displays has been the subject of many recent investigations. Uchida and Miyashita were particularly concerned with eyeglass mounted LCD's, Gibson with head up displays and Kruk and Longridge with a fiber optic design. Rebo's thesis is the most extensive published work to date and includes a very detailed analysis of the Polhemus headtracker. The study by Setterholm et al. is also very useful. Many systems have been investigated by the German aerospace company MBB (Bohm et al.). Other workers have been especially concerned with determining the optimal amount of binocular overlap and related parameters (Moffitt, Warren et al., Melzer and Moffitt, Self). Numerous studies have been done in the last three years.

My experiences with a wide variety of stereo displays has been that the greatest problems are usually with inadequate software (Starks). In examining some of these HMD's, it has become obvious that the stereo images need much improvement and with the computer generated images, it is often difficult to tell whether one is seeing stereo or pseudostereo (right and left eye images reversed). This is evident on some of the images presented in the stereoscopic virtual reality tape sold by 3DTV Corp. (Cyberthon in 3D) which are direct video feed from the computer and are not subject to any of the limitations of the HMD. More recently, wider experience with stereo has resulted in much excellent work.
 
 

One of the most sought after goals in 3D video has been a means for recording and /or transmission compatible with 2D reception by ordinary receivers. The 3D receiver would decode the signal to display a stereoscopic picture and in some schemes 2D receivers could be retrofitted with decoders to display 3D. Most of these subtract the two channels to obtain a difference signal which modulates some component of one channel for transmission. The great advances in video bandwidth compression in recent years should render many of these schemes more feasible. Such schemes have been described for many years (Brit- 706182) but it is getting more serious since some recent contributors to this field have been IBM (U.S.-4884131, E.P.-306448), CBS (US-4027333), the BBC (U.S.-4905081, E.P.-267000), NHK (U.S.-4704627, 4743965, WO-86-03924, 86-06914, JAP-59-265798, 60-46402, 60-96285,61-253993, 1-202093), Hitachi (63-256091, 63-100898, 63-217793,63-217794, 63-164593, 62-236294, 62-235896, 62-272697, 63-56089), NTT (Gomi et al.), Sony (52-9317), NEC (1-5291, 1-5292, 1-67094) Seiko (61-251396), Sharp (63-82191, 62-283792, 62-283793), Ricoh (62-150991), Thomson (U.S. 5055927), Telediffusion (U.S. 5043806), Toshiba (63-294090, 1-179593, 63-74292), Matsushita (63-1192),Canon (1-54992), Clarion (63-38386, 61-253992, 61-293094), ATR (1-114294, 1-64489) and others (JAP-51-142212, 59-86383, W0-84-00866, 83-00975, 88-01464, US-4266240,4287528, 4517592, Hudgins, Tamtoui and Labit, Chaissang et al). There has also been considerable work in the USSR by Dzhakoniya and others (USSR-128049). For an introduction to the vast amount of related work on video compression see JAP-63-294087, 52-72117, 62-245784, 62-165488, 62-166669, 63-201878.

Recent advances in hardware and software probably obsolete most of the above work.
 
 

Autostereoscopic displays are those which do not require the user to wear viewing aids. Displays using lenticular screens have been the subject of intensive research for nearly 80 years. The two Ives laid the foundations (US 666424, 725567, 771824, 1262954, 1814701, 1882424, 1883290, 1883291, 1905469, 1905716, 1916320, 1918705, 1937118, 1960011, 1970311, 1987443, 2002090, 2011932, 2012995, 2039648). Hundreds of researchers followed and there are perhaps 2000 patents and several hundred technical papers on the use of lenticular screens and related means for photography, motion pictures and television. It is impossible to cover more than a few of the more prominent or recent which relate most directly to video. Photographic systems have become common with both professional and consumer lenticular cameras. Motion picture applications have been rare with only the Russian lenticular glass screen being publicly shown in the USSR and at the Osaka Expo in 1970. Eight years before his invention of Holography in 1948, Gabor filed three patents on lenticular methods for movie projection(US 2351032,2351033,2351034) and in 1953 he filed what is probably the longest and most detailed patent ever granted on autostereo projection(GB 750911). Remarkably, later researchers seem to have almost completely ignored this work and even Gabor in his 1969 patent on holographic movie projection(US 3479111) fails to reference his last and most complete patent on this topic.

. Lenticular television devices have been prototyped many times but whether the screens were inside the CRT( Wallman, JAP 58-38093) or on the front of the faceplate or projection screen(e.g., Yanagisawa), alignment of pixels with lenslets was a major problem. Makoto Kikuchi of Sony pursued this approach vigorously during the 80's(JAP 53-20347, 56-126235, 56-126236, 56-128085, 56-128086, 56-132752, 56-134895, 56-168326, 57-3487, 57-11592, 57-13886, 57-14270, 57-17546, 57-18189, 57-26983, 57-27544, 57-27545, 57-27546, 57-67393, 57-72250, 57-75090, 57-83990, 57-83991, 57-87291, 57-106291, 57-123787, 58-29283, 58-31692, 58-103285, 58-115737). Tripp(US 3932699) was probably the first to build an adequate system, solving the alignment problem with a 13 inch diagonal fiber optic faceplate with a vertical lenticular screen. The input was a one inch camera tube covered with a specially made lenticular screen having 525 lenticulations per inch. This was made from a metal master hand engraved with the aid of a microscope(the same technique used to engrave dollar bills). This was perhaps the best autostereo CRT based system to date, but it was never duplicated and was soon cannibalized for the expensive fiber optics. Tripp however is a very flexible and ingenious man(one of his early inventions was the escalator) and he claims to have recently invented an extremely high resolution(2000 line pairs/mm) "spatial hologram without lasers" intended for use with his high resolution low dose x-ray system.

The advent of flat panel displays, which do not have the problem of aligning pixels and lenticules through an intervening layer of glass is resulting in renewed interest in this approach(Ichinose). Work is ongoing in France(US4584604), England(Sheat) and Japan(Tetsutani et al.) on a system for a 3D picturephone.

Another problem is that it is desirable to have a large number of these laterally multiplexed stereo pairs to minimize image "flipping" and give a "look around" capability(Schwartz). However, with most of these autostereo techniques, resolution and number of views are inversely related. With a 0.5mm lens size and 50 views, one needs a resolution of 10 microns, near that used for holographic plates and certainly beyond that of any available video display(with the possible exception of some of Tripp's prototypes). When it becomes possible to interpolate many views from a stereo pair, it will stimulate the whole field of autostereoscopy. Scene interpolation has been the subject of much research for robotics and pattern recognition but only a few workers have attempted to apply this directly to autostereoscopic display(Oshima and Okoshi).

Front or rear projection of stereo with lenticular screens has been investigated by many but has rarely resulted in commercial product. Sanyo Corp. has shown large rear projection lenticular systems in 1994 and offered a 50 inch diagonal model for $50,000. Image quality was modest and restriction on head position severe. Joji Hamasaki in Tokyo has been one of the most persistent and successful in this work with multiple video projection and large diameter screens as well as with the Sony beam index CRT(JAP 61-77839, Hamasaki). NHK and other has an active program with multiple LCD rear projection on a Toppan Corp. plastic lenticular screen(Isono). Viewing distance is limited to about 3 meters plus or minus 10cm and careful head positioning is necessary to avoid pseudoscopic zones(problems for all lenticular systems). Hamasaki's efforts and those of NHK are shown on the 3D videotapes "3D TV Technology Vols 1 and 2" marketed by 3D TV Corp. A vigorous program was conducted at the Heinrich Herz Institute in Berlin with front and rear projection on lenticular screens custom made by Philips in Eindhoven(Borner). Philips has extremely high precision computer controlled diamond milling equipment for making lenticular screens for their videoprojectors. A 1500 line screen can be milled in a plastic master in about 2 hours and the poured acrylic screen rapidly cured with uv . Minute corrections in the screen can be reliably programmed, engraved, cured and ready to test in one day for a one time set up fee of about $25,000 and a cost of about $20,000 for a 1M by 1M pair of screens in prototype quantities. This process used to take months and was not very accurate. The final screen is accurate and repeatable to one micron. Dr. Schmitz demonstrated this by making a screen which copied the eye of a bee and proved its extreme accuracy with electron microscope photos. This may result in commercial lenticular systems in the next few years.
 
 

Optically analagous to lenticular screens, parallax barriers consisting of thin vertical opaque strips seem to have been invented in the 17th century by G.A. Bois-Clair. The Frenchman Berthier revived it in 1896 and it has been the subject of hundreds of patents(GB 514426). 3D movies for viewing without glasses were shown commercially in Moscow in the 40's with a conical screen constructed by Ivanov from over 30,000 white enameled wires weighing six tons. The floor was slanted to accommodate an audience of about 250. For a period, the films were also projected with cross polarization so that those not located in the right viewing zones could see the films with polarized glasses. Autostereoscopic projection was apparently discontinued by the early 70's due to customer preference for the glasses.

The simplicity of barrier systems has continued to create interest both for still(US 4927238, Sandin, Myers et al) and moving images(Sexton, Johnson et al.). Eichenlaub has marketed an autostereoscopic workstation employing an LCD in a manner analagous to the barrier(US 5036385, 4717949,4829365, Eichenlaub). In 1995, Sanyo showed several small LCD based systems like this with an 8 inch diagonal model priced at $3000. Image quality was good but head position was critical.
 
 

In this technique, one or more vertical slits are rapidly scanned in the horizontal direction and the appearance of the image points on the screen behind are timed precisely so that a viewer at any position will see a stereo image. Noaillon(US1772782, 2198678) created a device composed of a conical arrangement of slats which was rotated rapidly between the viewers and the screen on which were projected 3D movies. Subsequent improvements were made by Savoye(US 2421393) and others(Jennings and Vanet) and Savoye's version was shown to audiences of 90 persons at Luna Park in Paris after WWII. A smaller system was sold by A. Mattey of Paris for home use. Versions of this "cyclostereoscope" were recently reconstructed by Australian enthusiast R. Blum(Blum) and by French stereo equipment designer Claude Tailleur.

For many years, Homer Tilton has promoted a unique version of this technique using a single mechanically scanned slit called the "Parallactiscope" and has even written a book about it(Tilton). Meacham has built a vibrating multislit device(EP 0114406 A1, US 4740073, Meacham)and Noble has made a version with an LCD slit replacing Tilton's mechanical one. Travis has suggested a laser addressed LCD with a large lens to overcome the low light emission common to most of these approaches(Travis ). Hattori has a system with multiple crt's, and a large diameter lens with an LCD slit scanning inside the lens(Hattori). Kollin's rotating louvers are another approach(Kollin), but any device that uses mechanical moving parts appears unlikely of success. The devices of Tilton, Noble and Meacham are shown and commented on in the 3D videotape "3D TV Technology Vol. 2".
 
 

In the early 60's, Robert Collender invented a dynamic parallax barrier system which had many intriguing features(US 3178720, 3324760), but his most interesting insight was embodied in his next application in 1973(US 3815979). He realized that if one used a screen that was very direction selective horizontally(i.e., diffused normally in the vertical direction but was retroreflective horizontally) he could replace the physical slit between the observer and the screen with a virtual slit. Along with his mechanisms for scanning multiple images on the screen, this made it possible to create a practical system for any size audience with no pseudoscopic or bad viewing zones. In subsequent patents he has extended his ideas considerably to flat screen video displays with few or no(US 4676613)moving parts(US 4089597, 4158487, 4176923, 4231642, 4290083, 4323920, 4349252, 4547050, GB 2076611, GB 2095068, JAP 56-31579, 57-11591, 57-162898). I have seen his simple prototype working with 16mm film and it is exactly as expected-one sees a nice 3D image without glasses from anywhere in the room. Collender thinks it would require about $10 million to build a video prototype of his invention. As an engineer with 30 years experience in high tech design, he is probably not too far off.
 
 

Invented by Gabriel Lippmann in 1908(Lippmann) this autostereo technique is often called "fly's eye lens photography" because of it's use of an array of tiny lenses for taking and displaying the image. As a result it possesses both horizontal and vertical parallax as do most types of holograms. Leslie Dudley was one of the more zealous researchers(US 3613539, 3683773, 3734618, 3675553) followed by Roger de Montebello(US 4732453). De Montebello's recent passing left his work in the hands of panoramic camera inventor Ron Globus of New York City. For video or computer graphics the images may be created by other means and displayed integrally. In spite of substantial problems in fabricating the multiple lens array and in reversing the pseudoscopic image, integral photography has continued to attract attention both for still photography(Shang, McCormick et al., Okoshi(71), JAP 1-154437, WO 89/06818, US 5040871, Ueda and Nakayama, Chutjian and Collier, Burckhardt et al. Burckhardt, McCrickerd) and video(US 3852524, 3878329, 5036385, FR 2014676, WO 88/05554, Igarashi et al,). W. Hickox of Airometric Systems Corp. in Glen Cove, N.Y. and Dave Roberts of Robert Engineering have also made integrams. Recent work with the fabrication of integral lenses holographically may further stimulate research(Hutley). Many other companies including Rank Pneumo, United Technologies, Adapative Optics Associates and Corning have begun fabricating microlens arrays.

The fact that the vertical parallax provided by the integram is usually unnecessary, coupled with the need to reverse the pseudoscopic image and the problems of lens manufacture, make it likely that the integram will see only very limited application in the foreseeable future.
 
 

It is common knowledge that when stereopairs are properly projected on suitably curved mirrors, lenses or screens, an observer in the appropriate viewing zones will see a stereo image. Hundreds of patents and dozens of prototypes attest to the simplicity and popularity of this approach to autostereo(US 3096389, 4062045,4315240, 4315241, 4509835, 4623223). A new type of retroreflector from Precision Lapping and Optical Co. of Valley Stream, N.Y. may make some new designs possible.

Ketchpel has proposed electronic modulation of a large diameter LCD for autostereo projection(Ketchpel, Williams et al.). Though large diameter glass lenses are impractical, the advent of high quality plastic fresnel lenses led to much many attempts to create autostereo systems, usually with rear projection of a stereo pair. A few such systems were created by Northrup for the military about 1980. The images came from a pair of high resolution black and white crt's projected through a custom fresnel about 30cm wide. I saw an excellent stereo image as long as I kept my head within an approximately basketball sized viewing zone. A similar system was built by Martin Marietta Corp. twenty years ago(Tewell et al.). Zehnpfennig has provided a detailed report on the construction of a smaller version (Zehnpfennig et al., US 3711188) and a very small version is commercially available in microscopes marketed by Vision Engineering and other companies.

It has long been recognized that a large diameter curved mirror will project a 3D image of an object suitably placed and illuminated. With appropriate masking of the mirror and object, the viewer sees an image floating in space. George Ploetz invented a clamshell arrangement of two mirrors that has been marketed in the US by Edmund Scientific and others(Ploetz, Coffey). Recently, Steve Welck of Grand Mirage Corp. in California has made large size plastic mirrors(US 4802750) which have begun appearing in advertising displays and even a video game from Sega. Though they could easily be in true 3D, so far all the devices using Welck's mirrors have used a single CRT to project a flat 2D image. Paul Kempf of Metron Optics in California has created a similar but smaller system with input from a stereo pair of cameras(US 4623223, 4840455). An interesting variant was created by Michiel Kassies of Amsterdam who encased millions of tiny mirrored balls between two sheets of plastic.
 
 

A number of investigators have realized that a properly designed holographic screen would be able to fulfill the functions of a lenticular or parallax barrier screen(US 3479111, Umstatter and Trollinger) or even of the camera pickup for an autostereo system(Kopylov). In his US patent(3479111) Dennis Gabor described the design of a holographic screen for the projection of stereo movies by two or more projectors. The book by Hutley has several papers on the fabrication of holographic integral lens arrays. The screen would direct multiple images, projected by conventional means or by laser, to multiple viewing zones. A few experiments have been done for autostereo projection, but the largest such screens are 24 inch by 30 inch created by Komar and his colleagues at NIKFI in Moscow for projection of their holographic movies with four viewing zones for one person each(Komar, Komar and Serov.

Various proposals have been made to use the birefringent properties of liquid crystals to create stereo screens(e.g. Sirat and Charlot).

Okoshi has championed a type of direction device called the curved triple mirror screen(Okoshi et al, Okoshi). Efforts to make such a screen were abandoned due to cost and complexity but with advances in manufacturing since the early 70's, it is undoubtedly worth another look. Many have proposed variants on the standard lenticular screen(Dultz, JAP 59-33449, 60-244943, US 4871233).
 
 

Volumetric displays are those in which the image points originate in a three dimensional space. There have been an amazing variety of volumetric display devices proposed and built(Balasubramonian, Williams) including dynamic ones having rotating or oscillating screens or lenses or mirrors(Withey, Muirhead, Jansson, Harris and Mathisen, Lazik, Yamada et al., Gregory, Szilard, Fajans{cf.Naegele},US 2330225, 2637023, 2967905, 2979561, 3077816, 3079585, 3097261, 3123711 3140415, 3154636, 3177486, 3202985, 3204238 3212084, 3258766, 3300779, 3316803, 3323126, 3335217, 3371155, 3428393, 3462213, 3493290, 3555505, 3604780, 3682553, 3976837, 4160973, 4294523, 4435053, 4315281, JAP 52-11533, 56-69612,56-71387,56-72595,5674219,56-102822,56-104316,56-106215,56-161788,56-161789,57-62020,57-171313,WO80/02218, 82/01259) which may be light emitting(Matsumoto) or upon which the images are projected by CRTs, light valves or lasers(Pressel, Ketchpel, Brinkmann, Matsushita, Tamura and Tanaka, Soltan). A number of these displays make clever use of fiber optics(US 4173391, Kapany, Martin).

Other approaches have used 3D arrays of components which emit or valve light when addressed electronically(Alburger, Nithiyanandam, Hattori, US 2361390, 2543793, 2749480, 2762031, 2806216, 3005196, 3138796, 3501220, 3536921, 3555349, 3605594, 3636551, 3682553,3891305, 3912856, 4134104, 4173391, 4294516, 4333715, 4391499, 4472737, 4629288, GB 1513719, 1601607, JAP 52-68310, 54-32224, 54-143041,56-125720, 56-162714) or by electron or laser beams. In recent years, many have suggested stacking LCD's(Alampiev et al.) and one of the earliest of these remains the most detailed published account(Reiche et al., cf. Cole et al.).

Another common technique addresses a volume of gas, liquid or solid with one or more laser or electron beam to give potentially very high resolution displays(US 1813559, 2604607, 3474248, 3609706,3609707,3829838, 4023158, 4063233, JAP 55-92090, FRE 461600, 733118, GB 1245783). The reports by Hassfurther et al. and Flackbert et al. are the most detailed published studies on a laser addressed display(rare earth doped calcium fluoride crystals).

The only volumetric display that has had any commercial success is the varifocal mirror(Harris, Huggins and Getty, Harris et al., Sher, Fuchs, Mills, Stover, US 3493290, 3632184,3632866). A large diameter mylar mirror is vibrated with a loudspeaker while addressed with a crt or other light source. This display was developed by Bolt, Beroneck and Newman and marketed briefly by Genisco but the dozen or so units sold seem to see little use and given it's size, cost, image distortions, and the need for a high speed computer for processing, this technology is probably a dead end. Nevertheless, a few companies continue to research it as a display for medical images and graphics (US 4,462044, 4607255, 4639081,4674837, 4743748, Sher).
 
 

Okoshi has reviewed much of the work relevant to holographic television(Okoshi) It has been looked into by many since the earliest days of holography but progress has been slow. Until recently, most of the effort was by Kopylov and others in Russia(Shmakov and Kopylov). Progress in electronics, electrooptics and computers has recently renewed interest(Honda, Hashimoto, Katsuma, Sato, Fukushima, Benton, GER 3140418, Boudreaux and Lettieri, Hashimoto and Morokawa, Cheng and Lin, Shyyrap et al.) but a real time full color high resolution system still appears quite remote. The annual volumes on Holography published by the SPIE always have several papers on holomovies.
 
 

There has been sporadic interest in stereoendoscopy, with Olympus Optical Co. most prominent(JAP 1-19319, 63-271493, 1-38811,1-38812,1-38813, US 4924853,4935810, 5039198). Recently, several other groups have developed prototypes with the idea of putting this instrument into clinical use(Jones, McLaurin, F. Oertmann of Aesculap A. G. in Tutlinger, Germany). One group developed techniques for digital image correlation of endoscopic stereopairs(Badique et al.). McKinley Optics of Southampton, Ma. has developed a stereoendoscope which was briefly marketed by American Surgical Technologies in 1993-1994. Lowell Noble of SOCS Research Inc. of Saratoga, California has applied the techniques for getting stereo with a single camera to create such instruments from single standard endoscopes with an internal LCD shutter, the image is superb and this product will be marketed by Smith and Nephew. The Canadian company International Telepresence has done similar work. The recent development of extremely high resolution quartz fiber endoscopes by Ultrafine Technology of North Brentford, England should further stimulate this research.

There has also been a steady trickle of papers on holographic endoscopy(Podbielska and Friesem, Friedman et al., Von Bally).
 
 

Automatic creation of three dimensional objects from stereo information gathered by stereo cameras, lasers etc. or created in computers(stereolithography) has been researched for many years (Kelly Swainson of Omtec Replication in Berkeley, Calif. in the mid 70's) but is only recently coming into practical application(US 4752964,4931817,4935774, JAP 61-88106). A number of companies with devices for automatic acquisition of digital 3D information used for input to CAD-CAM systems now feed this information into computer controlled milling machines for rapid solid modeling(e.g. Cyberware Corp. and Cencit Corp. in the US). A few years ago the large US retail chain Sears began installing such devices in their photo studios for instant modeling of customers heads. Apparently, peoples love of their own face was not sufficient to induce them to part with $40 and the experiment was abandoned.

Some devices are intermediate between stereo sculpture and a stereodisplay since they are too slow to update for a moving image display but neither do they produce a solid sculpture that can be removed. These are usually solids or semisolids which are photochromic or thermochromic(JAP 59-232313,63-157124,US 3399993, 4238840).

This work can be regarded as the successor to the century old art of creating topo maps by machines using stereo pairs as input. Originally drawn by a human operator comparing the stereo pairs thru a stereoscope, in recent years the automatic digital stereoplotters create topomaps without human intervention. One is also reminded of the various ingenious stereo drawing machines which have been in sporadic use for more than a century.

Many devices have been proposed and used to produce solid models by mechanical machining under computer control(Yamamoto). In the last few years, stereolithography has become common with the use of a computer controlled laser to cure the outline of an object in a resin bath(Arita et al., JAP 59-237053,61-116321, 61-116322, US 4961154, 5058988, 5059021, Peterson). As the object rises from the liquid resin, the laser cures more layers until the complete 3D object is done. Several companies including Sony in Japan and DuPont and 3D Systems Inc. of California have developed systems. The stereoscopic art has come full circle with the ability to create 3D objects from stereo images in little more time than is required to create stereo images from 3D objects.