Power-efficient deep learning algorithms – Sébastien Loustau

Link for the slides

Next seminar is on October 14th October (14:30) in « Campus Pierre & Marie Curie » of Sorbonne University. It will take place in SCAI seminar room, building « Esclangon », 1st floor

Si vous souhaitez assister en personne à ce séminaire:

Sébastien présentera ses travaux à la salle de séminaire de SCAI (plan d’accès: https://ai4climate.lip6.fr/wp-content/uploads/2020/09/plan_SCAI_extrait.pdf)
Merci de vous inscrire sur ce lien : https://docs.google.com/forms/d/e/1FAIpQLSc4scBTJZnOquz2FZkQbPKAKEvacQ0BC52WKs52CzTD6amCAw/viewform?usp=sf_link
Nous vous conseillons néanmoins d’apporter avec vous votre ordinateur portable afin d’être connecté en même temps sur la salle zoom (voir ci-dessous)


Si vous souhaitez assister à distance: 

Voici le lien zoom: https://us02web.zoom.us/j/81893439500
Vous pourrez également poser des questions sur le chat qui seront retransmises dans la salle.

Sebastien Loustau presentation is entitled:

« Power-efficient deep learning algorithms»

Abstract:
In this talk, I will present both theoretical and practical aspect of how designing power-efficient deep learning algorithms. After a non-exhaustive survey of different contributions about the machine learning perspective (training low bit-width networks), the hardware counterpart (CNNs accelerators) and the relationship with Auto-ML and the NAS procedure, I will present a theoretically based approach to add the power efficiency constraint into the optimization procedure of training deep nets. This work in progress bridges optimal transport and information theory with online learning.

Short bio:
Sébastien is a researcher in mathematical statistics and Machine Learning. He has studied the theoretical aspect of both statistical and online learning. His research interests include online learning, unsupervised learning, adaptive algorithms and minimax theory. He also founded LumenAI 5 years ago.

Inferring causation from time series with perspectives in Earth system sciences – Jakob Runge

The seminar is on December 4th at 10:00 14:00 and will be held remotely.

Link to the zoom session: https://us02web.zoom.us/j/84003686532

Jakob Runge presentation is entitled:

« Inferring causation from time series with perspectives in Earth system sciences »

Abstract:

The heart of the scientific enterprise is a rational effort to understand the causes behind the phenomena we observe. In disciplines dealing with complex dynamical systems, such as the Earth system, replicated real experiments are rarely feasible. However, a rapidly increasing amount of observational and simulated data opens up the use of novel data-driven causal inference methods beyond the commonly adopted correlation techniques. In this talk, I will present a recent Perspective Paper in Nature Communications giving an overview of causal inference methods and identify key tasks and major challenges where causal methods have the potential to advance the state-of-the-art in Earth system sciences. Several methods will be illustrated by `success’ examples where causal inference methods have already led to novel insights and I will close with an outlook of this relatively new and exciting field. I will also present the causal inference benchmark platform www.causeme.net that aims to assess the performance of causal inference methods and to help practitioners choose the right method for a particular problem.

Runge, J., S. Bathiany, E. Bollt, G. Camps-Valls, D. Coumou, E. Deyle, C. Glymour, M. Kretschmer, M. D. Mahecha, J. Muñoz-Marı́, E. H. van Nes, J. Peters, R. Quax, M. Reichstein, M. Scheffer, B. Schölkopf, P. Spirtes, G. Sugihara, J. Sun, K. Zhang, and J. Zscheischler (2019). Inferring causation from time series in earth system sciences. Nature Communications 10 (1), 2553.

Short bio:

Jakob Runge heads the Climate Informatics working group at the German Aerospace Center’s Institute of Data Science since 2017. The group combines innovative data science methods from different fields (graphical models, causal inference, nonlinear dynamics, deep learning) and closely works with experts in the climate sciences. Jakob studied physics at Humboldt University Berlin and obtained his PhD at the Potsdam Institute for Climate Impact Research in 2014. For his studies, he was funded by the German National Foundation (Studienstiftung) and his thesis was awarded the Carl-Ramsauer prize by the Berlin Physical Society. In 2014 he won a $200.000 Fellowship Award in Studying Complex Systems by the James S. McDonnell Foundation and joined the Grantham Institute, Imperial College, from 2016 to 2017. On https://github.com/jakobrunge/tigramite.git he provides Tigramite, a time series analysis python module for causal inference. For more details, see: www.climateinformaticslab.com

A direct approach to detection and attribution of climate change – Eniko Szekely – 24/01/2020

Lien pour les slides

Le prochain séminaire aura lieu le 24 Janvier à 14h30 au campus Pierre et Marie Curie de Sorbonne Université dans la salle 105 du LIP6 couloir 25-26 au 1er étage.

La présentation de Eniko Szekely est intitulée:

« A direct approach to detection and attribution of climate change »

Abstract:

In this talk I will present a novel statistical learning approach for detection and attribution (D&A) of climate change. Traditional optimal D&A studies try to directly model the observations from model simulations, but practically this is challenging due to high-dimensionality. Here, we propose a supervised approach where we predict a given metric or external forcing directly from the high-dimensional spatial pattern of climate variables, and use the predicted metric as a test statistic for D&A. The first part of the talk will focus on daily detection and show that we can now detect climate change from global weather for any single day since spring 2012. The second part of the talk will focus on attribution of climate change. For attribution, we want the prediction of the external forcing, e.g., anthropogenic forcing, to work well even under changes in the distribution of other external forcings, e.g., solar or volcanic forcings. Therefore we formulate the optimization problem from a distributional robustness perspective, and use anchor regression to ensure good predictions even under such distributional changes.

Notice Biographie:

Eniko is a senior data scientist at the Swiss Data Science Center, EPFL & ETH Zurich, working on machine learning for climate science. Previously, she was a postdoctoral researcher at the Courant Institute of Mathematical Sciences, New York University, and she obtained her PhD in Computer Science from the University of Geneva, Switzerland. Broadly she is interested in machine learning for high-dimensional data and nonlinear phenomena arising from dynamical systems. More recently she has been working on using machine learning and statistical learning approaches for climate science, and has been involved in the organization of the Climate Informatics workshop since 2015.

Deep Learning for Satellite Imagery: Semantic Segmentation, Non-Rigid Alignment, and Self-Denoising – Guillaume Charpiat – 4 Decembre 2019

Quand : 4 Decembre 2019 à 10:30

Où : Campus Pierre and Marie Curie (Sorbonne Université) salle 105 du LIP6 couloir 25-26 1er étage.

Résumé
Neural networks have been producing impressive results in computer vision these last years, in image classification or segmentation in particular. To be transferred to remote sensing, this tool needs adaptation to its specifics: large images, many small objects per image, keeping high-resolution output, unreliable ground truth (usually mis-registered). We will review the work done in our group for remote sensing semantic segmentation, explaining the evolution of our neural net architecture design to face these challenges, and finally training a network to register binary cadaster maps to RGB images while detecting new buildings if any, in a multi-scale approach. We will show in particular that it is possible to train on noisy datasets, and to make predictions at an accuracy much better than the variance of the original noise. To explain this phenomenon, we build theoretical tools to express input similarity from the neural network point of view, and use them to quantify data redundancy and associated expected denoising effects.
If time permits, we might also present work on hurricane track forecast from reanalysis data (2-3D coverage of the Earth’s surface with temperature/pressure/etc. fields) using deep learning.

Notice Biographie:

After a PhD thesis at ENS on shape statistics for image segmentation, and a year in Bernhard Schölkopf’s team at MPI Tübingen on kernel methods for medical imaging, Guillaume Charpiat joined INRIA Sophia-Antipolis to work on computer vision, and later INRIA Saclay to work on machine learning. Lately, he has been focusing on deep learning, with in particular remote sensing imagery as an application field.

Affiliation:
Guillaume Charpiat (Équipe TAU, INRIA Saclay / LRI – Université Paris-Sud)

Prévision d’ensemble par apprentissage séquentiel en météorologie, et méta-modélisation en pollution urbaine – Vivien Mallet 20 Septembre 2019

Quand : 20 Septembre 2019 à 14:00

Où : Campus Pierre and Marie Curie (Sorbonne Université) salle 105 du LIP6 couloir 25-26 1er étage.

Résumé
Le séminaire aura pour objectif d’illustrer certains apports de l’apprentissage dans des applications environnementales complexes.
La première partie concernera la prévision d’ensemble. Un objectif est d’agréger un ensemble de prévisions en une prévision unique et meilleure que chaque prévision de l’ensemble. Une approche plus ambitieuse consiste à prévoir une distribution de probabilité afin de conserver une mesure de l’incertitude de prévision. Nous verrons qu’il est possible de prévoir une distribution plus performante que toute distribution empirique formée par une pondération constante des prévisions de l’ensemble. Les travaux seront illustrés par la prévision du rayonnement solaire et de la production photovoltaïque d’EDF.
La seconde partie concernera la substitution d’un modèle environnemental, complexe et numériquement coûteux, par un méta-modèle extrêmement rapide et pourtant suffisamment fidèle au modèle complet. Nous verrons comment il est possible de remplacer un modèle non-linéaire opérant en grande dimension en (1) procédant à une réduction de dimension sur ses entrées et ses sorties, et (2) apprenant le comportement du modèle par un échantillonnage adapté. Il est aussi possible d’y mêler des données d’observation (issues de stations ponctuelles) pour améliorer les prévisions du méta-modèle. L’approche sera illustrée par la simulation de la pollution atmosphérique et de la pollution sonore en milieu urbain, à la résolution de la rue.

Notice Biographie:
Vivien Mallet est chercheur au centre INRIA de Paris. Il travaille sur l’assimilation de données (couplage modélisation/observation) et la quantification des incertitudes pour des problèmes en environnement.

Learning & Dynamical Systems: application to ocean dynamics – Ronan Fablet 15 May 2019

When: 15 May 2019 at 10:30

Where: Campus Pierre and Marie Curie (Sorbonne University) room 105 of LIP6 corridor 25-26 1st floor.

Abstract
Learning techniques and data-driven approaches become relevant alternatives to classical model-driven approaches for a large number of application domains, including for the study of phenomena governed by physical laws. They offer new means to take advantage of the potential of observation and/or simulation big data.
In this talk, we will discuss data-driven strategies for the identification, simulation and reconstruction of dynamical systems with illustrations on ocean monitoring applications (e.g., reconstruction of the sea surface, maritime traffic surveillance). We will specifically address how neural networks can provide novel means for the data-driven identification of representations of dynamical systems, which are imperfectly observed (e.g., noisy data, partial observation, irregular sampling..). We might further discuss the relevance of dynamical system theory for the understanding of state-of-the-art neural networks, especially residual nets.

Background:
Ronan Fablet
got an engineer degree from ISAE-SUPAERO (Institut Supérieur de l’Aéronautique et de l’Espace) Toulouse, France (1997), a MSC. In Applied Math from Univ. Paul sabatier, Toulouse, France (1997) and a the Ph.D. degree in signal processing and telecommunications from the University of Rennes/INRIA Rennes, France (2001). In 2002, he was a INRIA postdoctoral fellow with Brown University, Providence, RI, USA. From 2003 to 2007, he held a full-time research position with IFREMER Brest in the field of signal and image processing applied to fisheries science. In 2008, he joined the Signal and Communications Department, IMT Atlantique Bretagne-Pays de la Loire (formerly Télécom Bretagne), as an Associate Professor, and has been holding a full Professor position since 2012. He was a Visiting Researcher with Institut de Recherche pour le Deéveloppement/Instituto del Mar del Peru, Peru (Peruvian Sea Research Institute) in 2011 and a Visting Professor at IMEDEA (CSIC/UIB, Spain) in 2016. His main research interests are in data science with the main application field in ocean monitoring and surveillance. He has led national and international programs (e.g., EU STREP AFISA, ANR MN EMOCEAN, ANR ASTRID SESAME). He co-authored more than 200 articles and communications in peer-reviewed conferences and journals. Some references: (full paper available on  my researchgate.net )

Artificial Intelligence for Very High Resolution Earth Observation: Environment Monitoring – Mihai Datcu 05 Apr. 2019

When: Friday 5 April 2019 at 14:30
Where: Campus Pierre and Marie Curie (Sorbonne University) room 105 of LIP6 corridor 25-26 1st floor.

This presentation is organised in collaboration with the Chaire Internationale de Recherche Blaise Pascal financed by « Région Ile-de-France », managed by the « Fondation de l’Ecole normale supérieure » and hosted at CEDRIC, Cnam.

Abstract:
The Earth is facing unprecedented climatic, geomorphologic, environmental and anthropogenic changes, which require global scale observation and monitoring. Thus a multitude of new orbital and suborbital Earth Observation (EO) sensors and mission are in operation or will be soon launched. The interest is in a global understanding involving observation of large extended areas, and long periods of time, with a broad variety of EO sensors.  The collected EO data volumes are thus increasing immensely with a rate of many Terabytes of data a day. With the current EO technologies these figure will be soon amplified, the horizons are beyond Zettabytes of data. The challenge is the exploration of these data and the timely delivery of focused information and knowledge in a simple understandable format.Therefore, search engines, and Data Mining are new fields of study that have arisen to seek solutions to automating the extraction of information from EO observations and other related sources that can lead to Knowledge Discovery and the creation of an actionable intelligence. Knowledge Discovery is among the most interesting research trends, however, the real challenge is to combine Artificial Intelligence with the power and potential of human intelligence, this being a primary objective in the field of Human Machine Communication (HMC). The goal is to go beyond the today methods of information retrieval and develop new concepts and methods to support end users of EO data to interactively analyze the information content, extract relevant parameters, associate various sources of information, learn and/or apply knowledge and to visualize the pertinent information without getting overwhelmed. In this context, the synergy of HMC and information retrieval becomes an interdisciplinary approach in automating EO data analysis.

Background:
Mihai Datcu received the M.S. and Ph.D. degrees in Electronics and Telecommunications from the University Politechnica Bucharest UPB, Romania, in 1978 and 1986. In 1999 he received the title Habilitation à diriger des recherches in Computer Science from University Louis Pasteur, Strasbourg, France. Currently he is Senior Scientist and Data Intelligence and Knowledge Discovery research group leader with the Remote Sensing Technology Institute (IMF) of the German Aerospace Center (DLR), Oberpfaffenhofen, and Professor with the Department of Applied Electronics and Information Engineering, Faculty of Electronics, Telecommunications and Information Technology, UPB. From 1992 to 2002 he had a longer Invited Professor assignment with the Swiss Federal Institute of Technology, ETH Zurich. From 2005 to 2013 he has been Professor holder of the DLR-CNES Chair at ParisTech, Paris Institute of Technology, Telecom Paris. His interests are in Data Science, Machine Learning and Artificial Intelligence, and Computational Imaging for space applications. He is involved in Big Data from Space European, ESA, NASA and national research programs and projects. He is a member of the ESA Big Data from Space Working Group. He received in 2006 the Best Paper Award, IEEE Geoscience and Remote Sensing Society Prize, in 2008 the National Order of Merit with the rank of Knight, for outstanding international research results, awarded by the President of Romania, and in 1987 the Romanian Academy Prize Traian Vuia for the development of SAADI image analysis system and activity in image processing. He is IEEE Fellow. He is holder of a 2017 Blaise Pascal Chair at CEDRIC, CNAM.