Blue Eyes Technology - Monitoring Human Operator and Intelligence sensing System

Blue Eyes is a technology conducted by the research team of IBM at its Almaden Research Center (ARC) in San Jose, California since 1997. Blue eyes technology makes a computer to understand and sense human feelings and behavior and also enables the computer to react according to the sensed emotional levels. The aim of the blue eyes technology is to give human power or abilities to a computer, so that the machine can naturally interact with human beings as we interact with each other. All human beings have some perceptual capabilities, the ability to understand each other’s emotional level or feelings from their facial expressions. Blue eyes technology aims at creating a computer that have the abilities to understand the perceptual powers of human being by recognizing their facial expressions and react accordingly to them.  

Imagine, a beautiful world, where humans collaborate with computers!! .The computer can talk, listen or screech aloud!! .With the help of speech recognition and facial recognition systems, computers gathers information from the users and starts interacting with them according to their mood variations. Computer recognizes your emotional levels by a simple touch on the mouse and it can interact with us as an intimate partner. The machine feels your presence; verifies your identity and starts interacting with you and even it will dial and call to your home at any urgent situations. This all is happening with this “Blue Eyes” technology.

The main objective of Blue eyes technology is to develop a computational machine having sensory and perceptual ability like those of humans. The Blue Eyes technology system is a combination of a set of hardware and software systems.

The hardware consists of a central system unit (CSU) and data acquisition unit (DAU). Microcontroller- ATMEL 89C52 is the heart of the data acquisition unit. Bluetooth technology is provided for the coordination and communication between the two units. We can adapt this Blue Eyes technology in all working places, where the operator’s attention is continually available. Using the Blue eyes Technology it is able to record and monitor the user’s physiological condition by a technical approach. The aim of this Blue Eyes technology is to provide a machine or system having sensory and perceptual abilities like human beings thus it will support healthy stress free surroundings where the computers and humans can work together as intimate partners.

 Blue eyes technology consist of,

1. Mobile measuring device or Data Acquisition Unit (DAU)
2. Central System Unit (CSU)
3. The Hardware

Mobile measuring device or Data Acquisition Unit(DAU) of Blue Eyes technology:

The DAU used in the Blue Eyes technology is the mobile component of the system. The main function of DAU is to gather the physiological information from sensors and forward to the CSU for processing and verification purposes.

Fig: Over view of Blue Eyes systems

The blue tooth module, which is integrated with the mobile device (DAU), provides a wireless interface between the Central System Unit (CSU) and the user or operator having the sensors. PIN codes and ID cards are assigned to the entire operator’s for authentication purposes. The device uses a five-key keyboard, beeper and LCD display for the interaction with the operators and if, any unwanted situation occurs, the machine uses these devices to inform the operators. The ‘voice’ information from the user is transferred with the help of a headset, which is interfaced with the Data Acquisition Unit using a mini jack plug. DAU incorporates various hardware modules like system-core Bluetooth section, Atmel 89C52 microcontroller, EEPROM, Beeper, LCD display (HD44780), LED indicator, voltage level monitors and 6 AA batteries.

Fig: Data Acquisition Unit (DAU) Components

Central system Unit (CSU) of Blue Eyes technology:

CSU is the next squint of wireless-network connection in the Blue Eyes technology. The CSU mainly contains codec (PCM Codec commonly used for voice information transmission) and a wireless blue tooth module. This CSU section is integrated to a personal computer using USB, parallel and serial cable. The mini-jack socket is used for audio data accessing. The program containing the operators personal ID is amalgamating to the personal computer through the serial and power ports. The microcontroller (Atmel- 89C2051) inside the unit handles the I2C EEPROM- programming and UART transmission.

Fig: Central system Unit (CSU)Components

The Software used in Blue Eyes Technology:

The operator’s physiological condition is continually supervised by this Blue Eyes technology software. The software will respond in real time according to the operator’s physiological condition. This software helps to transfer the data or information from managers to the data analyzers. Then it transfers the processed information from this data analyzers unit to the GUI controls and data analyzers. At last, the data visualization module supports a user supervisor interface section. The visualization module is in the off-line mode and it will continually fetch the information from database and also records the video, audio and physiological parameters. Thus ‘Blue Eyes’ software enables the supervisor to know about the physiological condition of the operators.

The Blue Eyes Technology and Its Basic Structure:

The objective of Blue Eyes technology is to design a computational machine having sensory and perceptual abilities like human beings. Blue Eyes technology uses most modern cameras, microphones and advanced non-obtrusive sensing techniques to interact with humans and understand the emotions of human beings. The machine has the ability to grasp the eye movement of the user, the needs of the user and also can understand the emotional and physical states of a user in front of the machine. The process of making a computer having sensing and emotional capabilities is known as “Affective Computing”.

The steps involved for designing such type of computers are given below.

1. Process of giving sensing capacity.
2. Human Emotion detection or Affect Detection.
3. Respond appropriately and properly.

1.  Process of giving sensing capacity:

Blue Eyes utilizes many sensor mechanisms, which is equivalent for the ears, eyes and other sensory organs that human beings used to express emotions and recognize each other. Blue Eyes uses voice recognition software, cameras and biometric sensors to understand and respond to the emotional levels of humans. The voice recognition software can perceive not only what is being spoken but also the tone how it is said. High resolution cameras are used for tracking the minute facial expressions, hand gestures and eye movements. Biometric sensors are used for measuring and analyzing the muscle tension, body temperature, blood pressure and other physiological gesture correlated with emotions.

2. Detecting human emotions/ Affect Detection:

In Blue Eyes technology, the machines have the ability to identify the minor variations in the moods of human beings. Say a person may strike the keyboard hastily or softly depends on his mood like happy or in angry. The Blue Eyes enables the machines to identify these minor emotional variations of human beings even by a single touch on the mouse or key board and the machines started to react with the users according to this emotional levels. This is done with the guidance of intelligent devices like “Emotion Mouse”. Along with this Emotion Mouse, Simple User Interest Tracker (SUITOR) and Artificial Intelligent Speech Recognition are equipped with the Blue Eyes technology to understand the speech and identify the interest of the peoples at that instance of time.

For implementing the Affective Computing we need Emotion Sensors.

Types of Emotion Sensors used in Blue Eyes Technology:

• For Hand - Emotion Mouse:

The major aim of Brain Computer Interface (BCI) is to develop a smart and adaptive computer system. These types of project must include speech recognition, eye tracking, facial recognition, gesture recognition etc. software and hardware. Similarly in Blue Eyes technologies, we need to build a system have the ability to identify all these perceptual abilities of human beings. In Blue Eyes, the machines have the ability to identify the minor variations in the moods of human beings. Say a person may strike the keyboard hastily or softly depends on his mood like happy or in angry. The Blue Eyes technology enables the machines to identify these minor emotional variations of human beings even by a single touch on the mouse or key board and the machines started to react with the users according to this emotional levels. This is done with the guidance of intelligent devices like “Emotion Mouse”. Actually this Emotion Mouse is an input device to track the emotions of a user by a simple touch on it. The Emotion Mouse is designed to evaluate and identify the user’s emotions such as fear, surprise, anger, sadness, happiness, disgust etc. when he/she is interacting with computer. The main objective of the Emotion Mouse is to gather the user’s physical and physiological information by a simple touch. 

• For Eye - Expression Glass:

Expression Glass is an alternative for the usually available machine vision face or eye recognition methods. By analyzing pattern recognition methods and facial muscle variations, the glass senses and identifies the expressions such as interest or confusion of the user. The prototype used for this glass uses piezoelectric sensors.

MAGIC Pointing:                                        

The Eye gaze tracking methods explores a new way for handling ‘eye gaze’ for man machine interfacing. The gaze tracking has been deliberated as an excellent pointing method for giving input to computers. But many drawbacks exist with this traditional eye gaze tracking methods. To overcome these difficulties an alternative approach – termed as MAGIC - Manual and Gaze Input Cascaded – is projected. In this approach, eye gaze pointing appears to the user as a manual job, utilized for fine selection and manipulation processes. Even so, a large amount of the cursor movement is removed by bending the cursor to the eye gaze portion, which surrounds the target. The selection and pointing of the curser is primarily controlled by manual means but also guided by a gaze tracking mechanism and is commonly known as MAGIC Pointing. The main aim of MAGIC pointing is to use ‘gaze’ to warp the previous position (home) of the curser to the locality of the target, reasonably where the user was looking at, so as to reduce the cursor motion amplitude required for target selection. When the cursor position is identified, only a small movement is needed by the user to click on the target by a manual input device that is to accomplish Manual Acquisition with Gaze Initiated Cursor or Manual and Gaze Input Cascaded (MAGIC) pointing. Two MAGIC Pointing methods – conservative and liberal –in terms of cursor placement and target identification, were outlined, analyzed and executed with an eye tracker unit.

    

           Fig: Liberal MAGIC pointing Technique            Fig:The conservativeM AGIC pointing Technique

 Advantages of MAGIC Pointing Technique:

•  Reduced physical effort compared with the traditional manual pointing techniques.
•  Greater spontaneity than traditional manual pointing.
•  Greater accuracy.
•  Faster speed of operation than manual pointing.

Drawbacks of MAGIC Pointing Technique:

The unconscious jittery movement that an eye continually makes and also the one degree size of the fovea, eye gaze method is not accurately enough to perform UI widgets such as slider handles, scrollbars and hyperlinks on the Graphic User Interfaces. Sometimes the movement of the eye is spontaneously controlled while sometimes it is uncontrollable. In MAGIC pointing many methods such as eye blinking and continuous looking etc. are used for target selection procedures. But sometime it is not working properly because if a user does not look at a particular target continuously for a predetermined threshold value, the target will not be selected. Thus there are more chances for false selections.

The Simple User Interest Tracker (SUITOR):

The Simple User Interest Tracker is revolutionary approaches towards the design of machine having the ability to maintain an intimate relationship between the humans and the computers. The SUTOR continuously analyzes the user that where his eye focus on the personal computer screen. The SUITOR has the ability to determining the topic of interest of the user and also according to this it can able to deliver the appropriate data to a handheld device.

Artificial Intelligent Speech Recognition used in Blue Eyes Technology:

For implementing the Artificial Intelligent Speech Recognition system in Blue Eyes technology, the working environment should be very important. The manner of the user’s speech, grammar, noise type, noise level and the position of the microphone are some important factors that may influence the features of speech recognition system. In Artificial Intelligent Speech Recognition system, an automatic call handling method is implemented without any telephone operator.

Two basic ideas are included in the Artificial intelligence (AI),

• Study the thought of human beings.
• Represents the thought process of human beings through robots, computers etc.

Actually Artificial intelligence (AI) denotes the behavior of a computer or any machines but it is carried out by the humans is called as ‘intelligent’. This AI makes machines more power full, useful, and smarter and also it is less expensive compared to natural intelligence. Natural language processing (NLP) makes artificial intelligence systems to communicate English. The main goal of the Natural language processing (NLP) is to understand the users input and react according to these inputs. The input data or words are continuously scanned and finds matches against inside stored known data or words. And after identifying the key words, the corresponding actions are carried out by the machine. In this way the Blue Eyes technology enables the users to communicate with the machines with their own languages.

Conclusion:

BLUE EYES technological approach assure a convenient technique, that simplifies the life by supporting more elegant and user friendly provision in computing devices. The day is very near, that this Blue Eyes technology will advance its way towards your house hold devices and makes you lazier. In future, even this Blue Eyes will reach as your hand held mobile device…..