Direction des Relations Européennes et Internationales (DREI)

INRIA Programme: "Associate Team"

I. DEFINITION

ASSOCIATE TEAM	RTR2A


Selected in	2004

INRIA project: SIAMES/BUNRAKU	Foreign partner
Unité de recherche INRIA : Rennes Thème INRIA : Cognitive	University of Central Florida, US

	French Coordinator	Foreign Coordinator
Last Name, First Name	Kadi Bouatouch	Sumant Pattanaik
Title	Professor	Associate Professor
Institution (précisez le département et/ou le laboratoire)	IRISA, Université de Rennes 1	Computer Graphics Lab . University of Central Florida, School of Computer Science
Adress	Campus de Beaulieu, 35042 Rennes Cedex	School of Computer Sciences, University of Central Florida Orlando, Florida, 32816-2362, USA
URL	www.irisa.fr/siames/Kadi.Bouatouch	http://www.cs.ucf.edu/~sumant/
Tel	02 99 84 72 58	1 407 823 2638
Fax	02 99 84 71 71	1 407 823 5419
E-mail	kadi@irisa.fr	sumant@cs.ucf.edu

Brief description of the proposal

Keywords : High fidelity rendering, real-time, augmented reality, GPU

Topics of the collaboration: The targetted objective is real time rendering and real-time golbal illumination computation for static and dynamic scenes. The first objective is real-time high fidelity rendering of dynamic scenes. Rendering complex scenes using a classical graphics pipeline (Z-buffering, Phong's shading model, multitexturing, etc.) can be efficiently performed with commodity graphics cards. However, real-time rendering of scenes, for which global illumination is computed, still is a challenge for many researchers. Indeed, global illumination followed by a rendering step (that we call high fidelity rendering) is a demanding process in terms of computing and memory resources. The approch proposed in this collaboration has to intensively exploit the performances of the new graphics cards to meet the real-time constraint. The second objective (which has not been tackled yet in this collaboation) is augmented reality. We aim at combining in real time synthetic and real images. To this end, we propose to: calibrate the real images, extract the light sources which lighted the real scenes, insert synthetic objects in the real scene for which we dispose only of their images captured by a camera, and finally light these synthetic objects with the extracted real light sources. The result will help a user, supplied with a head mounted display, to see a real scene augmented with synthetic objects lit homogeneously.

Presentation of the Associate Team

1. Presentation of the foreign coordinator

Sumant Pattanaik is an associate professor at the School of Computer Science, University of Central Florida , Orlando. He has been Research Associate at the Cornell Program of Computer Graphics, Cornell University (1995-2001), got an INRIA postdoctoral position to join the SIAMES project at IRISA/INRIA Rennes (1993-1995) and was Senior Staff Scientist at the Graphics Department of NCST, Bombay (1985-1995).
He got his PhD in the Department of Computer Science and Information Science of BITS, Pilani in November 1993.

2. History of the collaboration

2.1. Between the two teams:

Sumant Pattanaik spent two years with the SIAMES group at IRISA, from september 1993 to July 1995, as an INRIA post-doctoral researcher. Then, I invited him to join us at IRISA as a visiting professor (funding of the university of Rennes 1) from the 15th of June2003 to the 16th of July 2003. Following this last stay at IRISA, we decided to collaborate. This is why Sumant Pattanaik welcomed in his group two of my students: Pascal Gautron and Jaroslav Krivanek, in octobre 2003 for periods of 4 months and 6 mois respectively. I have also been invited as a visiting professor in Orlando from the 4th of January 2004 to the 11th of January 2004. I gave a talk on my research activities in the departement of computer science of the university UCF.

During the presence of Sumant Pattanaik in IRISA for the period 1993-1995, we have published several papers:

- Kadi Bouatouch, S.N. Pattanaik and Eric Zeghers, "Computation of Higher Order Illumination with a Non Deterministic Approach", Eurographics'96, Computer Graphics Forum Conference Issue, Vol. 15(3), pp. C327-C337, 1996.

- S.N. Pattanaik and Kadi Bouatouch, "Interactive Walk-Through Using Particle Tracing", CG International 1995, Leeds, UK, 25-30 June, 1995.

- S.N. Pattanaik and Kadi Bouatouch, "Linear Radiosity with Error Estimation", Eurographics Rendering Workshop'95, Dublin, 12-14 June, 1995.

- Kadi Bouatouch and S.N. Pattanaik, "Discontinuity Meshing and Hierarchical MultiWavelet Radiosity", Graphics Interface'95, May 15-19, 1995, Quebec City.

- S.N. Pattanaik and Kadi Bouatouch, "Haar Wavelet : A Solution to Global Illumination With General Surface Properties", 5th Eurographics Workshop on Rendering, Darmstadt, Germany, June 1994.

- S.N. Pattanaik and Kadi Bouatouch, "Fast Wavelet Radiosity Method", Eurographics'94. Computer Graphics Forum Conference Issue, Vol. 13(3), pp. C407-C420,1994.

- S.N. Pattanaik and Kadi Bouatouch, "Adjoint Equations and Particle Tracing for Global Illumination", Technical Report IRISA-903, 1994.

2.2. Between INRIA other partnars:
Il est recommandé d'insérer autant de liens possibles vers les sites de ces collaborations ou organismes.
Il vous est expressément demandé de placer ici un pointeur vers la page du pays concerné sur le site de la DREI ("zones géographiques") ainsi que vers la page du pays concerné du site de la DRI du CNRS ("fiches pays"). Il est également recommandé de placer un pointeur vers la page du service scientifique de l'ambassade de France au pays concerné.

To my knowledge, there is no other collaborations between INRIA and the University of Central Florida.

3. Impact :

3.1. On the already existing collaboration with your partner

The association with our partner is of high importance since we already published 10 papers (two of them published in the IEEE Trans. CGV journal and one in the IEEE CG&A journal), 4 IRISA/INRIA research reports (see below) and 5 Siggraph sketches. One of my student is doing a PhD jointly supervised by Sumant Pattanaik and myself. He registered as a PhD student at the university of Central Florida and the university of Rennes 1.

3.2. On the collaboration with other INRIA projects

Yes, with the LAGADIC and TEMICS INRIA projects located in IRISA.

3.3. On the collaboration with other teams of the partner

Yes, with Professor Charles Hughes from the UCF . He is involved in virtual and augmented reality. I have invited him for two weeks in our group.

4. Miscellaneous : toute autre information que vous jugerez utile d'ajouter.

II. OUTCOME of 2007 and Balance Sheet

This year has been very rich in terms of publications: 3 journal papers, one has already appeared in the IEEE TVCG journal and another (accepted) will appear in the IEEE CG&A journal before the end of this year. The third paper has just been accepted for publication in the IEEE CG&A journal

Scientific report for 2007

In 2007 we have addressed the following topics : (1) real-time rendering of complex natural scenes with dynamic lighting and shadow casting, (2) Real-time rendering of globally illuminated natural scenes lit by the sun and the sky.

Real-time rendering of natural scenes. Grass is abundant on the Earth’s surface, hence it is an important element in rendered natural 3D scenes. Real-time realistic rendering of grass has always been difficult due to the huge number of grass blades. Overcoming this geometric complexity usually requires many coarse approximations to provide interactive frame rates. However, the performance comes at the cost of poor lighting quality and lack of detail of the grass. In our aproach, we propose a grass rendering technique that allows better lighting and parallax effect while maintaining real-time performance. We use a novel combination of geometry and lit volume slices, composed of Bidirectional Texture Functions (BTFs). BTFs, generated using a fast pre-computation step, provide an accurate, per pixel lighting of the grass. Our implementation allows the rendering of huge arbitrary terrains in real-time, with dynamic lighting, shadows, anti-aliasing and smooth transition between different levels of detail. We get real-time performances with terrains of the size of a football field covered by more than 600 million virtual grass blades. The rendering of arbitrary shaped patches of grass on arbitrary terrains is made possible using our novel density management scheme. This work has been accepted for publication in the IEEE CG&A journal and described in an IRISA/INRIA research report.

Real-time rendering of globally illuminated natural scenes lit by the sky and the sun. Nowadays, computer generated images of natural scenes are getting more and more realistic. Indeed,the targeted goal is to create images that are similar to what the viewer can see in real life withhis/her eyes. The main obstacle in achieving the target is complexity. Nature scenes from real lifecontain a huge number of small details which are hard to model, take a lot of time to render and require a huge amount of memory, unavailable in current computers. This complexity mainly comes from geometry and lighting. The geometric complexity is due to a high number of grass blades or tree leaves for example, a huge number of primitives such as triangles are needed to accurately model them; lighting computation complexity is due to the multiple re°ections of light over the scene objects with complex materials. Overcoming this complexity has been a challenging problem for many years. We address this problem in the context of grass and tree rendering. Algorithms have been developed to render grass and trees in real-time with approximations. Algorithms have also been developed to render them with high quality, for example using raytracing. Even in this latter case, approximations have to be used to render within a reasonable processing time. Our goal is to achieve real-time rendering of nature scenes while providing visually convincing dynamic global illumination. We want to render large amounts of natural elements, such as grass and trees, with approximations that reduce the rendering time while giving the convincing illusion of global illumination in dynamic scenes. A publication will be submitted to the forthcoming Siggraph conference.

Publications in 2007

Pascal Gautron, Kadi Bouatouch, Sumant Pattanaik, Temporal Radiance Caching, IEEE Transaction on Visualization and Computer Graphics Journal., Vol. 12, No. 5, Septembre/Octobre, 2007.
Guillaume François , Sumanta Pattanaik , Kadi Bouatouch , Gaspard Breton, Subsurface Texture Mapping, to appear in 2007 in the IEEE Computer Graphics & Applications journal.

Publications to appear in 2008

Kévin Boulange, Sumant pattanaik, Kadi Bouaotuch, Rendering Grass in Real-Time with Dynamic Lighting, accepted with minor revision (in october 2007) for publication in theIEEE Computer Graphics & Applications journal.

PhD supervision

One PhD student (Kévin Boulanger) is currently jointly supervised by Sumant Pattanaik and myself.

Financial report for 2007

1. EA Expenses (performed by the Associate Team)
	Budget allocated to Associate Team	Spent amount
Welcome	4000 € (including Missions)	1733 €
Missions		2110 €
Total	(a) 4000 €	(b) 3843 €

Used rate of credits allocated to Associate Team (b/a %)

99%

2. External expenses (supported by other institutions)
	Budget allocated to Associate Team	Expense amount
Name of the institution 1 (): INRIA Rennes (*to finance Kévin Boulanger's thesis for 4 months)
Welcome	11600 €	11600 €
Missions
Total	11600 €	11600 €

Name of the institution 2 () :* University of Central Florida, USA (to finance Kévin Boulanger's thesis for 8 months)
Welcome	11000 €	11000 €
Missions
Total	11000 €	11000 €

Total of external funding

allocated: (c) 22600 €

expenses: 22600 €

(*) Ajouter ou supprimer des lignes au tableau ci-dessus de façon à faire figurer tous les organismes ayant contribué au financement de l'équipe associée

Total of Associate Team and external fundings

allocated : (d) 26600 €

expenses: 26443 €

Rate of joint funding (c /d %)

84.86 %

Summary of exchanges performed in 2007

1. Seniors :

Nom	statut (1)	provenance	destination	objet (2)	durée (en semaines)	Coût (EA)	Coût (externe)
Kadi Bouatouch	Professor	IRISA	Orlando	participation in Kevin's candidacy	5 days	2110 €	0 €

Total in weeks

(1) DR / CR / professeur
(2) colloque, thèse, stage, visite....

2. Juniors

Nom	statut (1)	provenance	destination	objet (2)	durée (en mois)	Coût (EA)	Coût (externe)
Kévin Boulanger	PhD student	UCF	Rennes	internship	1.5	1733 €	0 €

Total in months

1.5

(1) post-doc / doctorant / stagiaire
(2) colloque, thèse, stage, visite....

III. EXPECTATION FOR 2008

WORK PLAN

The objective for 2008 is the following:

Real-time rendering of static scenes

Real-time rendering of natural scenes

We have devised fast methods (theory and implementation aspects) for handling indirect global illumination in natural scenes. We have obtained encouraging results. We need to bring some improvements to our methods and make more experiments to validate our choices.

Toward parallel high fidelity rendering

We would like to speed-up the computation of global illumination using parallel machines (clusters, grids) and many GPUs. All the computations will be distributed amongst the computer nodes and GPU nodes. This work has just started. We hope to achieve it in collaboration with another partner, the computer graphics group of the university of UTAH, US.

The mid-term objective is the following:

We sincerely think that there is a long way to go to be capable of performing global illumination computation in a moderate time for complex scenes (such as natural scenes). We think new methods are needed to address the problem of complexity in terms of geometry, photometry and illumination effects. This is what we started to do. Is the kind of method we already proposed is valid for realistic 3D movies and for high quality video games in case of highly complex scenes? Our objective is to take up this challenge.

We want to go further, say we want to perform global illumination in real-time for complex scenes containing natural objects. One solution is parallelism. This problem can be solved with the help of the computer graphcs group of the university of UTAH.

EXTENSION OF THE ASSOCIATE TEAM

My partenar Sumant Pattanaik and myself (Kadi Bouatouch) would wish to include a new partner in our Associate Team RTR2A, the computer graphics group of the university of UTAH, US. The two professors who would be included are Charles Hansen and Steven Parker.

Charles Hansen

School of Computing, University of UTAH

50 S. Central Campus Drive, RM 3190 MEB

Salt Lake City, Utah 84112

Voice: (801)-581-3154

Fax: (801)-581-5843

email: hansen@cs.utah.edu,

web: http://www.cs.utah.edu/~hansen/

          Charles Hansen is a Full Professeur at the university of UTAH (United States). He is working in the following topics : Large Scale Scientific Visualization, Rendering Techniques and Computer Graphics, Parallel Algorithms, Distributed Computation, 3D Shape representation, Computer Vision.
         Including Charles Hansen will allow us to take benefit from his long experience in data visualization (volume rendering, scientific data etc.), parallel algorithms and computer graphics. We would like to extend our global illumination algorithms to volumetric data such as medical images and other semi-transparent objects. Light transport in such objects must account for: multiple scattering, attenuation effects, etc. Computing and rendering these effects amounts to solve the radiative transfer equation which is very time consuming. Realistic and fast approximations are needed. Charles Hansen and his group have a long experience in this field of research. His group has, for example, proposed a new volumetric lighting model which is fast and efficient. Another skill of Charles Hansen's group, that could be beneficial for us, is parallel algorithms for graphics and scientific visualisation.
        Charles Hansen spent 6 weeks at IRISA as a visiting professor from june 1st to july 15th 2007. This visit has allowed to start a collaboration between our two groups.


         Steven G. Parker
        School of Computing, University of UTAH
        50 S. Central Campus Drive, RM 4803 WEB
        Salt Lake City, Utah 84112
        Voice: (801)-585-1504
        Fax: (801)-585-6513
        email: sparker@cs.utah.edu,
        web:  www.cs.utah.edu/~sparker

Steven G. Parker is an Assistant Professor at the School of Computing as well as a member of the Scientific Computing and Imaging (SCI) Institute. He performs research in software architectures for scientific computing, and in interactive ray tracing for large-scale scientific visualization and computer graphics applications. He is the chief software architect for the Center of Accidental Fires and Explosions, where he has helped create a problem solving environment for a complex multi-physics simulation running on thousands of processors. He is also the initial developer of the SCIRun problem-solving environment, the SCIJump distributed component environment, and the Manta interactive ray tracing system. Primary research interests for Steven Parker are scientific visualization, computational steering, high performance computing, interactive ray tracing, computer graphics, problem solving environments, and component architectures.

To sum up, Charles Hansen and Steven PArker, as a new partners in the Associate Team, will allow us to extend our algorithms to volumetric data and to parallelism, enhance our collaboration through new opinions, new skills, new ideas.

REQUEST FOR THE RENEWAL OF THE ASSOCIATE TEAM FOR 3 YEARS

As we are asking for the inclusion of a new partner, we request for a renewal of our Associate Team RTR2A for 3 years. We are asking for a budget similar to that of a newly created Associate Team if possible.

Expected Budget for 2008

1. Joint funding (external funding)

EXPECTED EXTERNAL FUNDING
Institution	Amount
University of Central Florida, USA	Not this year
IRISA (to finance Kévin Boulanger's thesis)	26100 €
University of Utah	Not this year
IRISA, SIAMES/BUNRAKU Project (university contracts managed by Kadi Bouatouch)	6000 €

Total	32100 €

2. Exchanges

EXPENSE ESTIMATION (including the new partner)		Amount
	Number	Welcome	Missions	Total
Senior	3	2 (new partner included)	1	9000 €
Post-doc
PhD students	3	2	1	12000 € (+ 26100 € to finance Kévin Boulanger's thesis from January 2007 to the end of september 2008)
Internship	2	1	1	7000 €
Others
Total	8	5	3	54100 €
		- total of external funding		32100 €
	Grant requested for the"Associate Team"			22000 €