Rapport

• Participants: Annie Ressouche, Jean-Yves Tigli

Activity recognition and monitoring systems based on multi-sensor and multi-device approaches are more and more popular to enhance events production for scenario analysis. Underlying software and hardware infrastructures can be considered as static (no changes during the overall recognition process quasi-static (no changes during two reconfigurations of the process) or really dynamic (depending on dynamic appearance and disappearance of numerous sensors and devices in the scene, communicating with the system, during recognition process). In this last case, we need to partially and reactively adapt the application to the evolution of the environment, while preserving invariants required for the validity of the recognition process.

In order to address such a challenge our researches try to federate the inherent constraints of platform devoted to action recognition, like SUP, with a service oriented middleware approach to deal with dynamic evolutions of the system infrastructure. Recent results, using a Service Lightweight Component Architecture (SLCA) [28] to compose services for device and Aspects of Assembly (AA) to adapt them in a reactive way [27], present interesting prospects to deal with multi-devices and variable systems. They provide a user-friendly separated description for adaptations that shall be applied and composed at runtime as soon as the corresponding required devices are present (in context-sensitive security middleware layer for example [40]). They also underline performances and response times that allow reactive adaptation on appearance and disappearance of devices. However, although composition between these adaptations can verify proved properties, the use of blackbox components in the composition model of SLCA doesn’t allow extracting a model of their behavior. Thus, existing approaches don’t really succeed to ensure that the usage contract of these components is not violated during application adaptation. Only a formal analysis of the component behavior models associated with a well sound modeling of composition operation will allow us to secure the respect of the usage contracts.

In this axis, we propose to rely on a synchronous modeling of component behavior and component assembly to allow the usage of model checking techniques to formally validate services composition. We began to consider this topic in 2008 through a collaborative action (SynComp) between Rainbow team at University of Nice Sophia Antipolis and INRIA Pulsar team. Within the Rainbow team, SLCA/AA experimental platform called WComp is dedicated to the reactive adaptation of applications in the domain of ubiquitous computing. During this collaboration, the management of concurrent access in WComp has been studied as a main source of disturbance for the invariant properties. A modeling of the behavior of components and of their accesses in a synchronous model has been defined in WComp.

This approach allows us to benefit from model checking techniques to ensure that there are no unpredictable states of WComp components on concurrent access. This year, during his training, Vivien Fighiera (already involved in the SynComp action), has completed the theoretical work done in SynComp. He studied how to prove safety properties regarding WComp component models relying on the NuSMV [52] model checker. This year, the collaboration between Rainbow and Pulsar has been strengthened since Jean-Yves Tigli is a full time researcher at Pulsar team since September, in sabbatical year sponsored by INRIA. Now, we plan to modelize the overall assembly of WComp components with a synchronous approach to allow the usage of model checking techniques to formally validate application design. In order to obtain results based on experimental scenarios to evaluate SynComp improvements for adaptive recognition process, we plan to integrate SUP platform as a software services provider.

The SUP platform gathers a set of modules devoted to design applications in the domain of activity recognition. WComp is aimed at assembling services which evolve in a dynamic and heterogeneous environment. Indeed, the services provided by SUP can be seen as complex high-level services whose functionalities depend on the SUP treatments; this latter dealing with the dynamic change of the environment. Thus, considering SUP services as web services for devices for example, the devices associated with SUP services will be discovered dynamically by WComp and used with other heterogeneous devices.

[1] A. AVANZI, F. BRÉM OND, C. TORNIERI, M. THONNAT. Design and Assessment of an Intelligent Activity Monitoring Platform, in “EURASIP Journal on Applied Signal Processing, Special Issue on “Advances in Intelligent Vision Systems: Methods and Applications””, vol. 2005:14, 08 2005, p. 2359-2374.

[2] H. BENHADDA, J. PATINO, E. CORVEE, F. BREMOND, M. THONNAT. Data Mining on Large Video Recordings, in “5eme Colloque Veille Stratégique Scientifique et Technologique VSST 2007, Marrakech, Marrocco”, 21st - 25th October 2007.

[3] B. BOULAY, F. BREM OND, M. THONNAT. Applying 3D Human Model in a Posture Recognition System., in “Pattern Recognition Letter.”, vol. 27, no 15, 2006, p. 1785-1796.

[4] F. BRÉMOND, M. THONNAT. Issues of Representing Context Illustrated by Video-surveillance Applications, in “International Journal of Human-Computer Studies, Special Issue on Context”, vol. 48, 1998, p. 375-391.

[5] Y. C. LIU, M. THONNAT. Understanding of Human Behaviors from Videos in Nursing Care Monitoring Systems, in “Journal of High Speed Networks”, vol. 16, 2007, p. 91-103.

[6] N. CHLEQ, F. BRÉMOND, M. THONNAT. Image Understanding for Prevention of Vandalism in Metro Stations, in “Advanced Video-based Surveillance Systems”, Kluwer A.P. , Hangham, MA, USA, November 1998, p. 108-118.

[7] V. CLÉMENT, M. THONNAT. A Knowledge-Based Approach to Integration of Image Procedures Processing, in “CVGIP: Image Understanding”, vol. 57, no 2, March 1993, p. 166–184.

[8] F. CUPILLARD, F. BRÉMOND, M. THONNAT. Tracking Group of People for Video Surveillance, Video-Based Surveillance Systems, vol. The Kluwer International Series in Computer Vision and Distributed Processing, Kluwer Academic Publishers, 2002, p. 89-100.

34 Activity Report INRIA 2009

[9] B. GEORIS, F. BREM OND, M. THONNAT. Real-Time Control of Video Surveillance Systems with Program Supervision Techniques, in “Machine Vision and Applications Journal”, vol. 18, 2007.

[10] S. LIU, P. SAINT-MARC, M. THONNAT, M. BERTHOD. Feasibility Study of Automatic Identification of Planktonic Foraminifera by Computer Vision, in “Journal of Foramineferal Research”, vol. 26, no 2, April 1996, p. 113–123.

[11] N. MAILLOT, M. THONNAT, A. BOUCHER. Towards Ontology Based Cognitive Vision, in “Machine Vision and Applications (MVA)”, vol. 16, no 1, December 2004, p. 33-40.

[12] S. MOISAN. Une plate-forme pour une programmation par composants de systèmes à base de connaissances, Habilitation à diriger les recherches, Université de Nice-Sophia Antipolis, April 1998.

[13] S. MOISAN, A. RESSOUCHE, J.-P. RIGAULT. Blocks, a Component Framework with Checking Facilities for Knowledge-Based Systems, in “Informatica, Special Issue on Component Based Software Development”, vol. 25, no 4, November 2001, p. 501-507.

[14] J. PATINO, H. BENHADDA, E. CORVEE, F. BREM OND, T. M.. Video-Data Modelling and Discovery, in “4th IET International Conference on Visual Information Engineering VIE 2007, London, UK”, 25th - 27th July 2007.

[15] J. PATINO, E. CORVEE, F. BREMOND, M. THONNAT. Management of Large Video Recordings, in “2nd International Conference on Ambient Intelligence Developments AmI.d 2007, Sophia Antipolis, France”, 17th - 19th September 2007.

[16] M. THONNAT, M. GANDELIN. Un système expert pour la description et le classement automatique de zooplanctons à partir d’images monoculaires, in “Traitement du signal, spécial I.A.”, vol. 9, no 5, November 1992, p. 373–387.

[17] M. THONNAT, S. MOISAN. What Can Program Supervision Do for Software Re-use?, in “IEE Proceedings - Software Special Issue on Knowledge Modelling for Software Components Reuse”, vol. 147, no 5, 2000.

[18] M. THONNAT. Vers une vision cognitive: mise en oeuvre de connaissances et de raisonnements pour l’analyse et l’interprétation d’images., Habilitation à diriger les recherches, Université de Nice-Sophia Antipolis, October 2003.

[19] M. THONNAT. Toward an Automatic Classification of Galaxies, in “The World of Galaxies”, Springer Verlag, 1989, p. 53-74.

[20] A. TOSHEV, F. BRÉMOND, M. THONNAT. An A priori-based Method for Frequent Composite Event Discovery in Videos, in “Proceedings of 2006 IEEE International Conference on Computer Vision Systems, New York USA”, January 2006.

[21] V. T. VU, F. BRÉM OND, M. THONNAT. Temporal Constraints for Video Interpretation, in “Proc of the 15th European Conference on Artificial Intelligence, Lyon, France”, 2002.

Project-Team Pulsar 35

[22] V. T. VU, F. BRÉMOND, M. THONNAT. Automatic Video Interpretation: A Novel Algorithm based for Temporal Scenario Recognition, in “The Eighteenth International Joint Conference on Artificial Intelligence (IJCAI’03)”, 9-15 September 2003.

Year Publications

Doctoral Dissertations and Habilitation Theses

[23] M.-B. KAÂNICHE. Human Gesture Recognition from video sequences, Ph. D. Thesis, Université de Nice Sophia-Antipolis, October 2009, http://tel.archives-ouvertes.fr/tel-00428690/en/.

[24] T.-L. LE. Indexation et Recherche de Vidéo pour la Vidéo Surveillance, Ph. D. Thesis, Université de Nice Sophia-Antipolis, February 2009, http://tel.archives-ouvertes.fr/tel-00393866/en/.

[25] P. VERNET. Interprétation géologique de données sismiques par une méthode supervisée basée sur la vision cognitive, Ph. D. Thesis, Ecole Nationale Supérieure des Mines de Paris, September 2009.

Articles in International Peer-Reviewed Journal

[26] T.-L. LE, M. THONNAT, A. BOUCHER, F. BRÉMOND. Surveillance video indexing and retrieval using objet features and semantic events, in “IJPRAI special issue on Visual Analysis and Understanding for Surveillance Applications”, 2009.

[27] J.-Y. TIGLI, S. LAVIROTTE, G. REY, V. HOURDIN, D. CHEUNG, E. CALLEGARI, M. RIVEILL. WComp middleware for ubiquitous computing: Aspects and composite event-based Web services, in “Annals of Telecommunications”, ISSN 0003-4347 (Print) ISSN 1958-9395 (Online), vol. 64, no 3-4, 2009.

[28] J.-Y. TIGLI, S. LAVIROTTE, G. REY, V. HOURDIN, M. RIVEILL. Lightweight Service Oriented Architecture for Pervasive Computing, in “IJCSI International Journal of Computer Science Issues”, ISSN (Online): 1694- 0784 ISSN (Print): 1694-0814, vol. 4, no 1, 2009.

[29] N. ZOUBA, F. BRÉMOND, M. THONNAT, A. ANFOSSO, E. PASCUAL, P. MALLEA, V. MAILLAND, O. GUERIN. A computer system to monitor older adults at home: Preliminary results, in “The Gerontechnology Journal: Official Journal of the International Society for Gerontechnology”, vol. 8 no. 3, 2009, p. 129 - 139.

International Peer-Reviewed Conference/Proceedings

[30] M. ACHER, P. COLLET, F. FLEUREY, P. LAHIRE, S. MOISAN, J.-P. RIGAULT. Modeling Context and Dynamic Adaptations with Feature Models, in “Models@run.time Workshop, Denver, CO, USA”, October 2009.

[31] M. ACHER, P. LAHIRE, S. MOISAN, J.-P. RIGAULT. Tackling High Variability in Video Surveillance Systems through a Model Transformation Approach, in “ICSE’2009 - MISE Workshop, Vancouver, Canada”, May 2009.

[32] S. BAK, S. SURESH, E. CORVÉE, F. BRÉMOND, M. THONNAT. Fusion of motion segmentation and learning based tracker for visual surveillance, in “The International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISAPP), Lisbon, Portugal”, 5 - 8 February 2009.

36 Activity Report INRIA 2009

[33] P. BILINSKI, F. BRÉM OND, M.-B. KAÂNICHE. Multiple object tracking with occlusions using HOG descriptors and multi resolution images, in “Proceedings of the 3rd International Conference on Imaging for Crime Detection and Prevention (ICDP’09), Kingston University, London, UK”, December 2009.

[34] C. CARINCOTTE, F. BRÉMOND, J.-M. ODOBEZ, L. PATINO, B. RAVERA, X. DESURM ONT. Multimedia Knowledge-Based Content Analysis over Distributed Architecture, in “The Networked and Electronic Media, 2009 NEM Summit - “Towards Future Media Internet”, Saint Malo, France”, September 2009.

[35] G. CHARPIAT. Learning Shape Metrics based on Deformations and Transport, in “Proceedings of ICCV 2009 and its Workshops, Second Workshop on Non-Rigid Shape Analysis and Deformable Image Alignment (NORDIA), Kyoto, Japan”, September 2009.

[36] D.-P. CHAU, F. BRÉM OND, E. CORVÉE, M. THONNAT. Online evaluation of tracking algorithm perfor- mance, in “The 3rd International Conference on Imaging for Crime Detection and Prevention, London, United Kingdom”, 3 December 2009.

[37] D.-P. CHAU, F. BRÉMOND, E. CORVÉE, M. THONNAT. Repairing People Trajectories Based on Point Clustering, in “The International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISAPP), Lisbon, Portugal”, 5 - 8 February 2009.

[38] E. CORVÉE, F. BRÉM OND. Combining face detection and people tracking in video surveillance, in “Proceed- ings of the 3rd International Conference on Imaging for Crime Detection and Prevention, ICDP 09, London UK”, December 2009.

[39] C. GARATE, P. BILINSKI, F. BRÉMOND. Crowd Event Recognition Using HOG Tracker, in “Proceedings of the Twelfth IEEE International Workshop on Performance Evaluation of Tracking and Surveillance, (invited paper), Winter-PETS 2009, Snowbird, Utah, USA”, December 2009.

[40] V. HOURDIN, J.-Y. TIGLI, S. LAVIROTTE, M. RIVEILL. Context-Sensitive Authorization for Asynchronous Communications, in “4th International Conference for Internet Technology and Secured Transactions (IC- ITST), London UK”, November 2009.

[41] M.-B. KAÂNICHE, F. BRÉMOND. Tracking HOG Descriptors for Gesture Recognition, in “The 6th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), Genoa, Italy”, 2 - 4 September 2009.

[42] V. MARTIN, F. BRÉMOND, J.-M. TRAVERE, V. MONCADA, G. DUNAND. Thermal Event Recognition Ap- plied to Tokamak Protection during Plasma Operation, in “IEEE International Instrumentation and Measure- ment Technology Conference”, May 2009, p. 1690-1694.

[43] S. MOISAN, J.-P. RIGAULT. Teaching Object-Oriented Modeling and UML to Various Audiences, in “MOD- ELS’09 Educators’ Symposium, Denver, CO, USA”, best paper award, October 2009.

[44] A.-T. NGHIEM, F. BRÉMOND, M. THONNAT. Controlling Background Subtraction Algorithms for Robust Object Detection, in “The 3rd International Conference on Imaging for Crime Detection and Prevention, London, United Kingdom”, 3 December 2009.

Project-Team Pulsar 37

[45] G. PUSIOL, F. BRÉMOND, M. THONNAT. Trajectory-based Primitive Events for Learning and Recognizing Activity, in “Second IEEE International Workshop on Tracking Humans for the Evaluation of their Motion in Image Sequences (THEMIS2009), Kyoto Japan”, October 2009.

[46] R. ROMDHANE, F. BRÉMOND, M. THONNAT. Handling Uncertainty for Video Event Recognition, in “The 3rd International Conference on Imaging for Crime Detection and Prevention, London, United Kingdom”, 3 December 2009.

[47] M. SIALA, N. KHLIFA, F. BRÉM OND, K. HAMROUNI. People Detection in Complex Scene Using a Cascade of Boosted Classifiers based on Haar-like-features, in “IEEE Intelligent Vehicle Symposium (IEEE IV 2009)”, June 2009.

[48] N. ZOUBA, F. BRÉMOND, M. THONNAT, A. ANFOSSO, E. PASCUAL, P. MALLEA, V. MAILLAND, O. GUERIN. Assessing Computer Systems for the Real Time Monitoring of Elderly People Living at Home, in “The 19th IAGG World Congress of Gerontology and Geriatrics (IAGG), Paris, France”, 5 - 7 July 2009.

[49] N. ZOUBA, F. BRÉMOND, M. THONNAT. Multisensor Fusion for Monitoring Elderly Activities at Home, in “The 6th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), Genoa, Italy”, 2 - 4 September 2009.

[50] M. ZUNIGA, F. BRÉMOND, M. THONNAT. Incremental Video Event Learning, in “Proceedings of the 7th IEEE International Conference on Computer Vision Systems, Liege Belgium”, October 2009.

Scientific Books (or Scientific Book chapters)

[51] G. CHARPIAT, I. BEZRUKOV, Y. ALTUN, M. HOFMANN, B. SCHÖLKOPF. Machine Learning Methods for Automatic Image Colorization, in “Computational Photography: Methods and Applications”, R. LUKAC (editor), CRC Press, 2009.

References in notes

[52] A. CIMATTI, E. CLARKE, E. GIUNCHIGLIA, F. GIUNCHIGLIA, M. PISTORE, M. ROVERI, R. SEBASTIANI, A. TACCHELLA. NuSMV 2: an OpenSource Tool for Symbolic Model Checking, in “Proceeeding CAV, Copenhagen, Danmark”, E. BRINKSMA, K. G. LARSEN (editors), LNCS, no 2404, Springer-Verlag, July 2002, p. 359-364, http://nusmv.irst.itc.it.

[53] N. DALAL, B. TRIGGS. Histograms of Oriented Gradients for Human Detection, in “CVPR, San Diego, CA, USA”, vol. 1, IEEECSP, June 20-25 2005, p. 886–893, http://www.acemedia.org/aceMedia/files/document/ wp7/2005/cvpr05-inria.pdf.

[54] ECVISION. A Research Roadmap of Cognitive Vision, Technical report, no IST-2001-35454, IST Project, 2005.

[55] F. FLEUREY, A. SOLBERG. A Domain Specific Modeling Language Supporting Specification, Simulation and Execution of Dynamic Adaptive Systems, in “MoDELS, Denver, CO, USA”, 2009, p. 606-621.

[56] D. GAFFÉ, A. RESSOUCHE. The Clem Toolkit, in “Proceedings of 23rd IEEE/ACM International Conference on Automated Software Engineering (ASE 2008), L’aquila, Italy”, September 2008.

38 Activity Report INRIA 2009

[57] N. HALBWACHS, F. LAGNIER, P. RAYM OND. Synchronous observers and the verification of reactive systems, in “Third Int. Conf. on Algebraic Methodology and Software Technology, AMAST’93, Twente”, M. NIVAT, C. RATTRAY, T. RUS, G. SCOLLO (editors), Workshops in Computing, Springer Verlag, June 1993.

[58] C. KÄSTNER, S. APEL, S. TRUJILLO, M. KUHLEM ANN, D. BATORY. Guaranteeing Syntactic Correctness for All Product Line Variants: A Language-Independent Approach, in “TOOLS (47)”, 2009, p. 175-194.

[59] A. RESSOUCHE, D. GAFFÉ, V. ROY. Modular Compilation of a Synchronous Language, Research Report, no 6424, INRIA, 01 2008, https://hal.inria.fr/inria-00213472.

[60] A. RESSOUCHE, D. GAFFÉ, V. ROY. Modular Compilation of a Synchronous Language, in “Software En- gineering Research, Management and Applications”, R. LEE (editor), Studies in Computational Intelligence, selected as one of the 17 best papers of SERA’08 conference, vol. 150, Springer, 2008, p. 157-171.

[61] E. ROSTEN, T. DRUM MOND. Machine learning for high-speed corner detection, in “ECCV, Graz, Austria”, vol. 1, Springer, May 7-13 2006, p. 430–443, http://mi.eng.cam.ac.uk/~er258/work/rosten_2006_machine.pdf.

[62] T. STAHL, M. VOELTER. Model-driven Software Development: Technology, Engineering, Management, Wiley, 2006.