#immersive systems

According to Oxford Reference, “Virtual reality is a synthetic technology combining three-dimensional video, audio, and other sensory components to achieve a sense of immersion.” Merriam-Webster defines virtual reality as “an artificial environment which is experienced through sensory stimuli (such as sights and sounds) provided by a computer and in which one's actions partially determine what happens in the environment, also: the technology used to create or access a virtual reality.”

Both of the above definitions focus on the technological aspect of virtual reality, describing it as a contemporary means. However, virtual reality is more than an artificial environment. It is a simulated experience that can be completely different to our reality. The Online Etymology Dictionary defines that the term “virtual” was being used meaning “influencing by physical virtues or capabilities, effective with respect to inherent natural qualities," as early as the late 1400s. It is also stated that it has been used in the computer sense of "not physically existing but made to appear by software" since 1959. “The term "virtual reality" was first used in a science fiction context in The Judas Mandala, a 1982 novel by Damien Broderick.”

Trying to pinpoint the exact origins of virtual reality has proven to be quite a challenge, considering how difficult it has been to formulate a definition for the concept of an alternative existence. However, in this post, we too will be focusing on the technological milestones that led to today’s applications of virtual reality, making it an innovative –soon to be essential- tool.

In 1838, Sir Charles Wheatstone invents the Stereoscope, a device for viewing a stereoscopic pair of separate images, depicting left-eye and right-eye views of the same scene, as a single three-dimensional image.

A 19th century stereoscope:

In 1935, American science fiction writer Stanley Weinbaum presents a fictional model for VR in his short story Pygmalion's Spectacles. In the story, the main character meets a professor who invented a pair of goggles which enabled "a movie that gives one sight and sound [...] taste, smell, and touch. [...] You are in the story, you speak to the shadows (characters) and they reply [...] the story is all about you, and you are in it." This sound a lot like our idea of a tangible painting, doesn’t it? (image source)

In 1956, cinematographer Morton Heilig creates Sensorama, the first VR machine (patented in 1962). It is a large booth that can fit up to four people at a time. It combines multiple technologies to stimulate all of the senses: there is a combined full colour 3D video, audio, vibrations, smell and atmospheric effects, such as wind. This is done using scent producers, a vibrating chair, stereo speakers and a stereoscopic 3D screen. Heilig thinks that the Sensorama is the "cinema of the future" and he wants to fully immerse people in their films. Six short films are developed for it.

In 1960, Heilig also patents the Telesphere Mask (image source ), which is the first head-mounted display (HMD). This provides stereoscopic 3D images with wide vision and stereo sound. There is no motion tracking in the headset at this point.

In 1965, Ivan Sutherland, a computer scientist, presents his vision of the Ultimate Display. The concept is of a virtual world viewed through an HMD which replicates reality so well that the user won’t be able to differentiate from actual reality. This includes the user being able to interact with objects. This concept features computer hardware to form the virtual world and to keep it functioning in real-time. His paper is seen as the fundamental blueprint for VR. “The ultimate display would, of course, be a room within which the computer can control the existence of matter. A chair displayed in such a room would be good enough to sit in. Handcuffs displayed in such a room would be confining, and a bullet displayed in such a room would be fatal. With appropriate programming such a display could literally be the Wonderland into which Alice walked.”

In 1966, Thomas A. Furness (a.k.a “the grandfather of VR”), a military engineer, creates the first flight simulator for the Air Force. This assists in the progression of VR because the military subsequently provides a lot of funding for producing better flight simulators.

In 1968, Sutherland, with his student Bob Sproull, creates the first virtual reality HMD, named The Sword of Damocles. This head-mount connects to a computer rather than a camera and is quite primitive as it can only show simple virtual wire-frame shapes. These 3D models change perspective when the user moves their head due to the tracking system. It was never developed beyond a lab project because it was too heavy for users to comfortably wear; they had to be strapped in because it was suspended from the ceiling.

In 1969, Myron Krueger, a computer artist, develops a succession of "artificial reality" experiences using computers and video systems. He creates computer-generated environments that respond to the people in it. These projects lead to VIDEOPLACE technology which is mentioned later.

In 1972, General Electric Corporation builts a computerised flight simulator which features a 180-degree field of vision by using three screens surrounding the cockpit.

In 1975, Krueger's VIDEOPLACE, the first interactive VR platform, is displayed at the Milwaukee Art Center. It uses computer graphics, projectors, video cameras, video displays and position-sensing technology and it doesn't use goggles or gloves. VIDEOPLACE consistes of dark rooms with large video screens to surround the user in "VR". The users can see their computer-generated silhouettes imitating their own movements and actions - the users' movements are recorded on camera and transferred onto the silhouette. Also, users in different rooms can interact with other users' silhouettes in the same virtual world. This encourages the idea that people can communicate within a virtual world even if they aren't physically close.

In 1977, the Aspen Movie Map is created by MIT. This program enables users to virtually wander through Aspen city in Colorado, like with Google Street View. There are three modes: summer, winter and polygons. It is created using photographs from a car driving through the city. There are no HMDs but it is the use of first-person interactivity and it suggests that VR can transport people to other places.

In 1977, Sayre gloves are created by Daniel J. Sandin and Thomas A. DeFanti,  at the Electronic Visualization Laboratory, a cross-disciplinary research lab at the University of Illinois at Chicago. These gloves are the first wired gloves. They monitore hand movements by using light emitters and photocells in the gloves' fingers. So when the user moves their fingers the amount of light hitting the photocell varies which then converts the finger movements into electrical signals. This may be the beginning of gesture recognition. Furness creates a working model of a virtual flight simulator, for the military, called the Visually Coupled Airborne Systems Simulator (VCASS). Image of a wired glove:

In 1979, McDonnell-Douglas Corporation integrates VR into its HMD, the VITAL helmet, for military use. A head tracker in the HMD follows the pilot's eye movements to match computer-generated images.

Sources:

Barnard, Dom. "History of VR - Timeline of Events and Tech Development." Virtualspeech. August 06, 2019. https://virtualspeech.com/blog/history-of-vr.

Cakmakci, Ozan, Jannick Rolland. “Head-Worn Displays: A Review.” Journal Of Display Technology 2, no. 3 (September 2006). [accessed June 17, 2020]. https://www.researchgate.net/publication/3453724_Head-Worn_Displays_A_Review.

“Daniel J. Sandin”, in evl|electronic visualization laboratory. https://www.evl.uic.edu/core.php?mod=4&type=5&indi=11.

Hosch, William L. “Ivan Edward Sutherland.” In Encyclopædia Britannica. Encyclopædia Britannica, inc. May 12, 2020 https://www.britannica.com/biography/Ivan-Edward-Sutherland. Access Date: June 17, 2020.

“Thomas A. Furness”, on Industrial & Systems Engineering: University Of Washington. https://ise.washington.edu/facultyfinder/thomas-a-furness.

Jack, Emily. “Artifact of the Month: Holmes Stereoscope.” NC Miscellany. Ocotber 21, 2013. https://blogs.lib.unc.edu/ncm/2013/10/21/artifact-of-the-month-holmes-stereoscope/.

Kelly, Michael. “Virtual Reality.” In Encyclopedia of Aesthetics, 2nd ed. New York: Oxford University Press, 2014. https://www.oxfordreference.com/view/10.1093/acref/9780199747108.001.0001/acref-9780199747108-e-740?fromCrossSearch=true.

Lescop, Laurent. “360° vision, from panoramas to VR.” On ResearchGate.  Fig. 03.: “Morton Heilig, Telesphere Mask, 1960, Source: Wikimedia Commons.” September, 2017. https://www.researchgate.net/publication/319618259_360_vision_from_panoramas_to_VR.

Lowood, Henry E. “Virtual Reality.” In Encyclopædia Britannica. Encyclopædia Britannica, inc. November 11, 2019. Access Date: June 17, 2020. https://www.britannica.com/biography/Morton-Heilig.

Media Art Net. “Ivan Sutherland «Head-Mounted-Display», 1968.” Medien Kunst Netz / Media Art Net. http://www.medienkunstnetz.de/works/head-mounted-display/.

Media Art Net. “Morton Heilig «Sensorama», 1962.” Medien Kunst Netz / Media Art Net. http://www.mediaartnet.org/works/sensorama/.

Media Art Net. “Myron Krueger, «Videoplace», 1974.” Medien Kunst Netz / Media Art Net. http://www.medienkunstnetz.de/works/videoplace/.

Merriam-Webster.com Dictionary, s.v. “virtual reality,” accessed June 17, 2020, https://www.merriam-webster.com/dictionary/virtual%20reality.

Musings Of A Mario Minion. “Pygmalion’s Spectacles: Using Berkeley’s Immaterialism to Understand the Potential for Telepresence in Virtual Reality.” Medium. February 05, 2018. https://medium.com/@musingsofamariominion/pygmalions-spectacles-using-berkeley-s-immaterialism-to-understand-the-potential-for-telepresence-46b9e46eba42.

Naimark, Michael. “Aspen Moviemap 1978-80.” Michael Naimark. http://www.naimark.net/projects/aspen.html.

Online Etymology Dictionary, s.v. “virtual (adk.),” accessed June 17, 2020, https://www.etymonline.com/word/virtual#etymonline_v_7821.

Sandin, Daniel J.  Thomas A. DeFanti, Richard Sayre. “Sayre Glove (first wired data glove).” evl|electronic visualization laboratory. January 1, 1977. https://www.evl.uic.edu/entry.php?id=2162.

Sutherland, Ivan E. “The Ultimate Display”. Information Processing Techniques Office, ARPA, OSD. http://worrydream.com/refs/Sutherland%20-%20The%20Ultimate%20Display.pdf.

Sutherland, Ivan E. “A head-mounted three dimensional display*.” Fall Joint Computer Conference, 1968. Salt Lake City, Utah: The University of Utah. http://cacs.usc.edu/education/cs653/Sutherland-HeadmountedDisplay-AFIPS68.pdf.

The Editors of Encyclopaedia Britannica. “McDonnell Douglas Corporation.” In Encyclopædia Britannica. Encyclopædia Britannica, inc. June 12, 2020. Access Date: June 17, 2020. https://www.britannica.com/topic/McDonnell-Douglas-Corporation.

The Editors of Encyclopaedia Britannica. “Sir Charles Wheatstone.” In Encyclopædia Britannica. Encyclopædia Britannica, inc. February 06, 2020. Access Date: June 17, 2020. https://www.britannica.com/biography/Charles-Wheatstone.

The Editors of Encyclopaedia Britannica. “Stereoscope.” In Encyclopædia Britannica. Encyclopædia Britannica, inc. February 06, 2020. Access Date: June 17, 2020. https://www.tumblr.com/edit/618119398273794048.

Up to this point, during our research, we’ve already laid down a quite detailed time-line of the history of VR. However, as we’ve previously mentioned, VR is not the only means of immersive technology. In this post, we are discussing digital scent technology. “Digital sense technology (or olfactory technology) is the engineering discipline dealing with olfactory representation. It is a technology to sense, transmit and receive scent-enabled digital media (such as web pages, video games, movies and music). This sensing part of this technology works by using olfactometers and electronic noses.” 

Its history begins in the late 1950s when Hans Laube invents the Smell-O-Vision, “a process in which evocative smells were pumped to the cinema audience through pipes leading to individual seats in the auditorium. Bottles of scent were held on a rotating drum and the process was triggered by a signal on the film itself. It made its only appearance in the 1960 film “Scent of Mystery”, produced by Mike Todd Jr. The process injected 30 odors into a movie theater's seats when triggered by the film's soundtrack.”

“In 1906, a Pennsylvania movie theater soaked a wad of cotton wool in rose oil and placed it in front of a fan. When a newsreel about the Rose Bowl played, they turned on the fan and the smell of roses wafted over the theater.”

On the image(source) above we see Mike Todd Jr. (left) with Hans Laube, the inventor of the Smell-O-Vision (right). 

(image source)

John Waters paid homage to Smell-O-Vision with his 1981 film “Polyester”. Waters created the process of Odorama and, rather than pumping in scents, used individual audience "Scratch and Sniff" cards.

“Behind the Great Wall”, a film directed by Italian director Carlo Lizzani was released on December 2, 1959, just three weeks ahead of “Scent of Mystery”. It was accompanied by a process called "AromaRama", to send scents through the air-conditioning system of a theater. The particular technique was invented by Charles Weiss.”

“But in the end all of these methods remained relatively unsuccessful and mainly failed due to the fact that a specific smell cannot be eliminated in time before the next one is used!”

“The Old Grey Hare is a 1944 Merrie Melodies cartoon. The cartoon starts with Elmer Fudd sitting under a tree, crying over his failure to catch Bugs. The "voice of God" tells Elmer to keep trying to catch him. Elmer wonders how long it will it take -and is shown exactly how long by being transported "far into the future" past the years 1950, 1960, 1970, 1980, 1990, until reaching the then-distant year of A.D. 2000, after the sound of the gong. Elmer finds a year 2000 newspaper. One headline says, "Smellevision Replaces Television: Carl Stalling Sez It Will Never Work!" (1:54).”

“In 1999, Joel Lloyd Bellenson and Dexter Smith began to think about how to store and reproduce the smells. They first worked to create a database of smells and then they created the device that would connect to the PC, the iSmell. DigiScents developed a computer peripheral device called iSmell Personal Scent Synthesizer, which was designed to emit a smell when a user visited a web site or opened an email. It was a small device that could be connected to a computer through a Universal serial bus (USB) port and powered using any ordinary electrical outlet. The device contained a cartridge with 128 "primary odors", which could be mixed to replicate natural and man-made odors. The appearance of the device was similar to that of a shark’s fin, with many holes lining the “fin” to release the various scents. Using a cartridge similar to a printer’s, it could synthesize and even create new smells by combining certain combinations of other scents.These newly created odors could be used to closely replicate common natural and manmade odors. The cartridges used also need to be swapped every so often once the scents inside are used up. Once partnered with websites and interactive media, the scents could be activated either automatically once a website was opened or manually. DigiScents had indexed thousands of common odors, which could be coded, digitized, and embedded into web pages or email. However, the product is no longer on the market and never generated substantial sales.”

“In 2013, Haruka Matsukura and a team of colleagues at Tokoyo University of Agriculture and Technology developed a display called the “smelling screen”. The smelling screen combines a digital display with four small fans, one at each corner of the display. Odor stored in tiny gel packets is emitted and blown parallel to the screen. By varying the speed and strength of each fan, an emitted odor is moved to a specific spot on the screen.”

Created in 2013, in Spain, by the Valencian engineer Raúl Porcar OLORAMA TECHNOLOGY seems to be another fascinating and successful venture to a more immersive experience by bringing the sense of smell into the game.

Are you familiar with the 4-D film format? Have you ever been to a movie theatre with a 4-D room? Basically, you watch a 2-D, or 3-D film combinded with various practical effects. Said practical effects could include synchronized motion seats and environmental effects such as water, wind, fog, scent, snow and more, to enhance the action on screen. This technology allows the film to be augmented and it provides  an immersive multi-sensory cinematic experience. 4DX, developed by CJ 4DPLEX is a very successful example of this technology.

Sources:

Agapakis, Christina. “Smell-O-Vision.” Scientific American. November 12, 2012. https://blogs.scientificamerican.com/oscillator/smell-o-vision/.

Brownlee, John. “A Brief History of Smell-O-Vision.” Wired. July 12, 2006. https://www.wired.com/2006/12/a-brief-history-2-2/.

Dirks, Tim. “The History of Film - The 1950s - The Cold War and Post-Classical Era - The Era of Epic Films and the Threat of Television.” amc filmsite. https://www.filmsite.org/50sintro2.html.

Epstein, Adam. “Do you smell that? Hollywood’s fantastic, failed attempts to make audiences smell and “feel” movies, from AromaRama to 4D.” Quartz. April 01, 2016. https://qz.com/649920/hollywoods-fantastic-failed-attempts-to-make-audiences-smell-and-feel-movies-from-aromarama-to-4d/.

Plafke, James. “Japanese smell-o-vision TV releases scents with per-pixel accuracy.” EXTREMETECH. March 29, 2013. https://www.extremetech.com/extreme/152086-japanese-smell-o-vision-tv-releases-scents-with-per-pixel-accuracy.

Platt, Charles. “You’ve Got Smell!” Wired. January 11, 1999. https://www.wired.com/1999/11/digiscent/.

“Willy Wonka and the Chocolate Factory in Smell-O-Vision.” Lincoln Theatre. https://lincolntheatre.org/film/willy-wonka-and-chocolate-factory-smell-o-vision.

Wikipedia contributors, "Smell-O-Vision," Wikipedia, The Free Encyclopedia, https://en.wikipedia.org/w/index.php?title=Smell-O-Vision&oldid=958400019 (accessed June 17, 2020).

This is an introduction to a series of posts that serve as a recording of the history of immersive systems and technologies. We have already explained the idea of a tangible painting. Ultimately, its purpose is to provide us with the greatest immersive experience. However, considering that what we're talking about is still only an idea, we ought to understand it as best as possible, in order to decide whether it could actually work or not. As we’ve already mentioned in our introductory post, the term “immersive technology” mostly refers to modern digital technology. Our understanding is that one is already familiar with VR technology, since it is becoming more and more widespread nowadays. Regardless of whether one’s ever used it or not, they, at the very least, know what we’re talking about. We feel it’s important that we first become familiar with the history of immersive systems/technology. In the following posts of this series, we will be covering various subjects such as the history of VR -from the invention of the stereoscope to today, 4-D cinemas and Smell-O-Vision and last, but certainly not least, haptic technology.