Processing Global Hotkey Events

I want to trigger an Event, everytime the ESC Key is pressed.
As shown below, this is npt that hard, when the program is in focus. If it out of focus, the event is not triggered.
Thus, I need global hotkey processing.

I found a page, where this method is explained using a custom class, which utilized Hooks.

More here:
http://www.tutorial-board.de/index.php?page=Thread&threadID=382
http://www.codeproject.com/KB/cs/globalhook.aspx
http://dotnet-snippets.de/dns/globale-hotkeys-tastenkombinationen-SID356.aspx

Trying to understand the smoothing algorithm from EyeWriter

The algorithm looks very simple:
There is one parameter, which controlls the intensity of the smoothin effect.

The formula is structured like this

NewCursorPosition =
       SmoothingFactor * OldCursorPosition
      + (1-SmoothingFactor) * ActualGazePosition

Code from testApp.cpp Line 62
   eyeSmoothed.x = CM.smoothing * eyeSmoothed.x + (1-CM.smoothing) * screenPoint.x;

One example for this:

NewPos = 0,97 * 487 + (1-0,97) * 705
              = 472 + 21
              = 493


The difference between OldPos and New Pos is 493-487 = 5px. This means, that the cursos slowly adopts to the actual gaze position without large jumps..
Increasing the Smoothing Factor leads to a much slower moving of the cursor, decreasing it, results in an abrupt jumping of the cursor, which gives it less stability.

From a first impression, the GT algorithm does not work as good as this simple and lean EyeWriter algorithm, AND uses a much more complicated logic. Therefore, I probably could implement this very simple algorithm instead.

However, I need to understand the GT algorithm first in order to decide.

Built my first Eye Tracker

This is it!! I've built my own eye tracker and I think I can be satisfied with the first try. I was able to remove the IR filter from to PS Eye and add an IR bandpass. I was also able to install the two IR LEDs and power them with two 1,5V batteries. With a twisted aluminum wire, I could mount the tracker to a pair of 1€ sunglasses.

I did testruns with this construction with ITU GazeTracker and the EyeWriter tracking software.

Both showed favourable results.

I could already use my eyes controlling a pointer in both softwares. However it was not that accurate, perhaps caused by my "temporal" eye tracker constuction.

Accuracy was better with ITU tracker, in my opinion.

TODO: Improve accuracy, either with better eye tracker or optimizing the software.

I found out that the EyeWriter software does not do any glint tracking, but only pupil tracking.

The glint tracking on the ITU software seems to require for brighter IR LEDs as sometimes, the tracking was poor.

Another problem, I encountered is that the glasses are black, which may sometimes disturb the tracker from my pupils. 

TODO: Use differently colored glasses or use a colored tape on my current ones

One problem occured because of the human anatomy: there is a huge vein running next to my eyes. Wearing the sunglasses tightly on my skin causes them to bounce, beacuse the vein gives an impulse everytime my heart beats.

TODO: Find sth. I can use for cushoning

All in all, I think these are good results for a first try.

Looking forward wrking on this interesting topic on Tuesday!

PS3 Eye

This Webcam is commonly used to create a DIY multi-touch display.
Thus, I believe it to be a good camera for my project.

Before the cam can be used for tracking, the IR blocking filter needs to be removed.
AFAIK, a filter to block standard light can be attached afterwards so that the IR light is enhanced.
This might result in better tracking result... here is why:

When you illuminate the eye with IR light and observe it through an IR sensitive camera with a visible light filter, the iris of the eye turns completely white and the pupil stands out as a high-contrast black dot. This makes tracking the eye much easier.

from http://www.instructables.com/id/The-EyeWriter/step8/Lite-it-up/

Here are some random links I found during my search:

• Dismounting the PS3 Eye: http://codelaboratories.com/research/view/ps3-eye-disassembly
• How to create an IR Filter (imho: blocking the "normal" light): http://photocritic.org/create-your-own-ir-filter/
• Create a head-mounted eye-tracker with PS3 Eye: http://www.instructables.com/id/The-EyeWriter
• Drivers for using PS3 Eye in Windows: http://codelaboratories.com/downloads

Before the cam can be used for tracking, a lens with a shorter focal length (8mm) needs to be used (why exactly??). ITU uses a standard off-the-shelf webcam without any modifications. Probably, this modification is just extra... However, what to I do now?? Buy a cam, such aus the one the ITU guys use, or use the hacked PS3 Eye??

I think this depends on my final choice... probably.
EyeWriter = PS3 cam and ITU = standard cam

Gaze Tracker by ITU Gaze Group

http://www.gazegroup.org/downloads/23-gazetracker

This seems to be the most promissing project!!

They have an open source lib for eye tracking. In combination with a head-mounted low-cost webcam this might work pretty well.

Works with the Microsoft Webcam from the other post.

TODO:
Find out, if there is an API for their Lib
Try out IR webcam with IR LEDs
Find a good way to create a head-mounted device: maybe basecap or glasses

TrackEye

TrackEye@Codeproject

Could not get this to work... probably try again.

Seems to focus mainly on face and eye detection. The page says, that gaze detection should also be possible.

===== Added on 28.06.2010 =====

Settings to be Done to Perform a Good Tracking

Settings for Face & Eye Detection

Under TrackEye Menu --> Tracker Settings

• Input Source: video
• Click on Select file and select ..\Avis\Sample.avi
• Face Detection Algorithm: Haar Face Detection Algorithm
• Check “Track also Eyes” checkBox
• Eye Detection Algorithm: Adaptive PCA
• Uncheck “Variance Check”
• Number of Database Images: 8
• Number of EigenEyes: 5
• Maximum allowable distance from eyespace: 1200
• Face width/eye template width ratio: 0.3
• ColorSpace type to use during PCA: CV_RGB2GRAY

Settings for Pupil Detection

Check “Track eyes in details” and then check “Detect also eye pupils”. Click “Adjust Parameters” button:

• Enter “120” as the “Threshold Value”
• Click “Save Settings” and then click “Close”

Settings for Snake

Check “Indicate eye boundary using active snakes”. Click “Settings for snake” button:

• Select ColorSpace to use: CV_RGB2GRAY
• Select Simple thresholding and enter 100 as the “Threshold value”
• Click “Save Settings” and then click “Close”

eyewriter.org

http://www.eyewriter.org/

DIY EyeTracking software and hardware.
Enables a paralyzed graffiti artist to draw his pieces again.
They use a Playstation 3 Eye as eyetraker and mount it on sunglasses. Two IR LEDs enable the exact tracking.
Seems to work pretty good.

Open source!

TODO: See if I can use this for my project

Research Seminar 11.05.2010: Multimodal Interaction (Presenter Prof. Wolff)

Slides (02) on DropBox

Notes:

• Interaction becomes more human by using multimodal input devices (detecting mimic, voice or eye state and analyzing it according to human emotions)
• Multimodal input gains popularity in the early 1990sVarious definitions for multimodality
• "... process combined input modes in a coordinated manner..." (interaction manager)
• How can multimodality and multimediality be separated?cf. Nigay u. Coutaz 1993: Multimodality implies information processing on a higher abstraction layer: „... multimodality is the capacity of the system to communicate with a user along different types of communication channels and to extract and convey meaning automatically. We observe that both multimedia and multimodal systems use multiple communication channels. But in addition, a multimodal system is able to automatically model the content of the information at a high level of abstraction. A multimodal system strives for meaning.“
• W3c standard for multimodal applications/browser (in progress) (Candell & Raggett) (EMMA: Extensible MultiModal Annotation Language => I should read about that a bit)
• Larson 2006: Common Sense Recommendations for designing multimodal user interfaces

1. Satisfy Real-world Constraints
2. Communicate Clearly, Concisely, and Consistently with Users
3. Help Users Recover Quickly and Efficiently from Errors
4. Make Users Comfortable

• Do not overload user with too many modalities, e.g. display and read text simultaneuosly; see ~Little's Law: The more WIP the longer the time to process something, i.e. the users receives more information (WIP) and therefore needs longer to make a decision.
• Question: In how far can eye gaze used intentionally? I look unintentionally (natural function of eyes)... but can I use gaze gestures intentionally?

16.04: Resumé

Tried to understand Kumar's saccade detection algorithm, but cannot quite grasp it.
Perhaps reading Identifying Fixations and Saccades in Eye-Tracking Protocols might help.

Found a fancy blog theme, tho' ;-)

Fixation Smoothing and Saccade Detection Algorithm

Kumar, Klingner et al. – Improving the Accuracy of Gaze
http://portal.acm.org/citation.cfm?id=1344488

This article describes an algorithm which determines by the gaze data whether the user is currently starting a saccade or just a microsaccade. If it is a miccrosaccade the current gaze data is not considered. Thus a better stability during a fixation can be achieved.

The algorithm encounters the problem of eye-noise:

fixations are not stable and the eye jitters during fixations due to drift, tremor and involuntary micro-saccades [Yarbus 1967]. This gaze jitter, together with the limited accuracy of eye trackers, results in a noisy gaze signal

As the analysis of the data is done in real time a minimal lag occurs. One data record is processed and afterwards the mouse pointer is set, or not. This results in a one-data-sample lag.

Error rates with gaze pointing and selection are hight than with mouse:

In the paper describing EyePoint [Kumar et al. 2007b], it was reported that while the speed of a gaze-based pointing technique was comparable to the mouse, error rates were significantly higher.

Relevance of this article

In this paper we present three methods for improving the accuracy and user experience of gaze-based pointing: an algorithm for realtime saccade detection and fixation smoothing, an algorithm for improving eye-hand coordination, and the use of focus points. These methods boost the basic performance for using gaze information in interactive applications and in our applications made the difference between prohibitively high error rates and practical usefulness of gaze-based interaction.

Method of the algorithm (TODO: I NEED TO UNDERSTAND THIS):

To smooth the data from the eye tracker in real-time, it is necessary to determine whether the most recent data point is the beginning of a saccade, a continuation of the current fixation or an outlier relative to the current fixation. We use a gaze movement threshold, in which two gaze points separated by a Euclidean distance of more than a given saccade threshold are labeled as a saccade. This is similar to the velocity threshold technique described in [Salvucci and Goldberg 2000], with two modifications to make it more robust to noise. First, we measure the displacement of each eye movement relative to the current estimate of the fixation location rather than to the previous measurement. Second, we look ahead one measurement and reject movements over the saccade threshold which immediately return to the current fixation.