gazegroup.org

Open source eye tracking and gaze interaction

Skip to content

Help in gaze tracking

The core of the Gaze Tracker. Topics concern the algorithms uses for head, eye and gaze tracking (pupil detection, gaze estimation, filtering, eye movement classification etc.) and the image processing libraries used (OpenCV, the EMGU .Net wrapper)

Help in gaze tracking

Postby eduardo » Thu Feb 04, 2010 9:53 pm

Hi forum!
This is my first post. I´m managing to perform an eye tracking tool to move the pc mouse. I´m using the c++ library openCV and common webcam placed at the top of the screen pc (it´s not an IR webcam!).
This is my first attempt so I´m using only one eye.
Till this moment I get the gaze vector for the eye. Let´s call it as "g".
To get the gaze vector I had attention to the vertical rotation and the horizontal rotation of the eye. (I can tell you how have I get it, but I dont´want to bore you with this now...)
Now I´ll have to know the point on the screen that is beeing pointed by the gaze line with the direction of the gaze vector.

I´m considering only one axes system (because unfortunatelly my math skills aren´t the best :? ), with its origin placed at the eye. Parallel to the eye we will have the screen plane. The user will be at a fixed position (say 20 cm from the screen-this is a request of the project). As the screen lies on the xy plane, our screen plane will have the equation : Z=20.
So the line contained a vector will´ve the equation:

(x,y,z)=(x0,y0,z0)+gt
(x,y,z)=(0,0,0)+gt

so:
x=gxt; y=gyt; z=gzt (1)

to know the intersection point of the gaze line with the screen plane z=20 (screen plane) :
20=gzt<=>t=20/gz

This way we get t and we will substitute it in the equation (1) we will get the x y and z coordinates of the intersection.

Obviously that I will have to implement some conditions because the screen is a "limited" plane.

So what are your advices for me? Am I thinking wright? I know that If this step fails I will never achieve my goal...So your advices will be very important !

Thanks in advance for your time,
Eduardo

(PS: sorry for my english :mrgreen: )
eduardo
 
Posts: 7
Joined: Thu Feb 04, 2010 9:22 pm

Re: Help in gaze tracking

Postby eduardo » Mon Feb 08, 2010 6:14 pm

Nobody?
Well I´m facing a lot of troubles getting the gaze vector. I´m wondering if my approach is correct, but unfortunatelly I´m not finding the answers to my doubts...
without solving this i can´t move ahead... :roll:

cheers
eduardo
 
Posts: 7
Joined: Thu Feb 04, 2010 9:22 pm

Re: Help in gaze tracking

Postby javiersanagustin » Wed Feb 10, 2010 11:25 am

Hi,

Sorry for the late reply. To be able to help I need to know what method you're using to estimate the gaze vector. It is not a trivial task, especially if you're using a single, non-IR webcam.

Can you explain your method?
javiersanagustin
 
Posts: 110
Joined: Fri Apr 03, 2009 10:37 am
Location: Copenhagen

Re: Help in gaze tracking

Postby eduardo » Thu Feb 11, 2010 1:56 pm

Javier!
Well, I was placing a coordinate system with the origin in the eye. Imagine that the scrren is distant 40 cm of the eye, so I consider that it lies in the plane Z=40.
Now I considered two angles, one for the horizontal rotation of the eye and the oter for the vertical rotation of the eye.
Considering the two angles I defined two rotational matrices with them (one for rotation about the y axis and another one for the rotation about the x axis).
Multiplying by the matrices by the vector v=[0;0;1] (normal vector to the eye) I will get the gaze vector... I read this in a master thesis...
getting the gaze vector I will parametrize and intersect it with Z=40 and I´ll get the intersection point...
Unfortunatelly I implemented this strategy but it didn´t performed as planned.

So, I emailed Professor Andrew Duchowski (thanks to him!!!) an he advised me to use calibration to build a rotation matrix. After build that Matrix, it should be applied to each eye coordinate get using openCV (this is already working).
I will try this...Hope it works ;)

PS: I´ve also a "programming doubt" . I saw the interface of your group program and I´m wondering if you use WPF to do it becuase it´s really nice... well...I´m using C++ openCV library to get the eye coordinates as I think you also did, but I wish to build a nice interface. So I will did it in WPF. In the past I did a similar task and I used .dll files to exchange data between opencv and C# code... However it´s a simple program... as this is a more complex one i´m afraid about the performance... Could you give me a better idea to do this?
(remember that the main problem is that an opencv image has IplImage structure and c# works with bitmaps, so I had to convert between the to formats which takes a few ms...)


Thanks for your time

Rgds
Eduardo
eduardo
 
Posts: 7
Joined: Thu Feb 04, 2010 9:22 pm

Re: Help in gaze tracking

Postby martintall » Fri Feb 12, 2010 12:31 am

Yes, the everything that is being displayed is based on the Windows Presentation Framework (WPF) available in .Net 3.5 or later. It's a really nice framework, flexible and powerful but it does take a while to learn if your coming from WinForms (worth it!).

So we're using the Emgu C# OpenCV wrapper which wraps almost all functions in OpenCV2.0, there is a small performance penalty but nothing too bad. What is a problem however is the conversion between the image formats as you mention. From OpenCV to GDI+ Bitmaps or WPF BitmapSources. We ended up hosting a WinForm inside the WPF to achieve good performance. Now there is one available in the Emgu framework that offers more functionality, for example you can right-click to see the current (real) frame rate of the video stream.

If you want to keep all of your code .Net check out http://www.emgu.com/ for more information.
User avatar
martintall
 
Posts: 149
Joined: Fri Apr 03, 2009 10:49 am
Location: Menlo Park, CA, USA

Re: Help in gaze tracking

Postby eduardo » Sun Feb 21, 2010 5:47 pm

Hi!
ok!
I will try that wrapper in the next version of my gaze tracker (hope that the 1.0 will work :D ) :P
I tried another wrapper in the past and it gives me a lot of troubles with dll files not found...

In the meanwhile I´ll continue using the dll files to share data between c++ and c#, and if everithing runs perfect i will use the wrapper.

Best rgds

Eduardo
eduardo
 
Posts: 7
Joined: Thu Feb 04, 2010 9:22 pm

Re: Help in gaze tracking

Postby eduardo » Wed Feb 24, 2010 2:35 am

Hi Forum!
I´m facing some problems installing emgu wrapper, to re-write my c++ code in c#. :geek:
Anybody could help me?

Rgds

Eduardo
eduardo
 
Posts: 7
Joined: Thu Feb 04, 2010 9:22 pm

Re: Help in gaze tracking

Postby eduardo » Sun Feb 28, 2010 3:46 pm

Solved ;)
eduardo
 
Posts: 7
Joined: Thu Feb 04, 2010 9:22 pm


Return to Tracking algorithms (head, pupil, glints)

Who is online

Users browsing this forum: No registered users and 1 guest