Machine-learning-nederland November 2022

machine-learning-nederland@list.uva.nl

2 participants
4 discussions

Reminder: Thematic seminar: Umut Simsekli today at 16h00
by Tim van Erven 14 Nov '22

14 Nov '22

Dear all, Heads up: Umut Şimşekli's in person talk at the UvA is today: *Umut Şimşekli* (INRIA/École Normale Supérieure, https://www.di.ens.fr/~simsekli/) *Monday November 14*, 16h00-17h00 In person, at the University of Amsterdam Location: Science Park 904, Room A1.24 *Fractal Structure and Generalization Properties of Stochastic Optimization Algorithms* Understanding generalization in deep learning has been one of the major challenges in statistical learning theory over the last decade. While recent work has illustrated that the dataset and the training algorithm must be taken into account in order to obtain meaningful generalization bounds, it is still theoretically not clear which properties of the data and the algorithm determine the generalization performance. In this talk, I will approach this problem from a dynamical systems theory perspective and represent stochastic optimization algorithms as random iterated function systems (IFS). Well studied in the dynamical systems literature, under mild assumptions, such IFSs can be shown to be ergodic with an invariant measure that is often supported on sets with a fractal structure. We will prove that the generalization error of a stochastic optimization algorithm can be bounded based on the ‘complexity’ of the fractal structure that underlies its invariant measure. Leveraging results from dynamical systems theory, we will show that the generalization error can be explicitly linked to the choice of the algorithm (e.g., stochastic gradient descent – SGD), algorithm hyperparameters (e.g., step-size, batch-size), and the geometry of the problem (e.g., Hessian of the loss). We will further specialize our results to specific problems (e.g., linear/logistic regression, one hidden-layered neural networks) and algorithms (e.g., SGD and preconditioned variants), and obtain analytical estimates for our bound. For modern neural networks, we will develop an efficient algorithm to compute the developed bound and support our theory with various experiments on neural networks. The talk is based on the following publication: Camuto, A., Deligiannidis, G., Erdogdu, M. A., Gurbuzbalaban, M., Simsekli, U., & Zhu, L. (2021). Fractal structure and generalization properties of stochastic optimization algorithms. Advances in Neural Information Processing Systems, 34, 18774-18788. Seminar organizers: Tim van Erven Botond Szabo https://mschauer.github.io/StructuresSeminar/ -- Tim van Erven<tim(a)timvanerven.nl> www.timvanerven.nl

1 0

Reminder: Thematic seminar: Damien Garreau Thu Nov 10, and Umut Simsekli Mon Nov 14
by Tim van Erven 09 Nov '22

09 Nov '22

Dear all, This is to remind you about the two in person talks tomorrow and on Monday in the thematic seminar at the UvA: 1. On Thursday November 10 Damien Garreau from the Université Côte d'Azur will speak about his analysis of the popular LIME method for explainable machine learning. 2. On Monday November 14, Umut Şimşekli from INRIA/École Normale Supérieure will speak about his new generalization bounds for deep neural networks. *1. Damien Garreau *(Université Côte d'Azur, https://sites.google.com/view/damien-garreau/home) *Thursday November 10*, 16h00-17h00 In person, at the University of Amsterdam Location: Science Park 904, Room B0.204 * **What does LIME really see in images? *The performance of modern algorithms on certain computer vision tasks such as object recognition is now close to that of humans. This success was achieved at the price of complicated architectures depending on millions of parameters and it has become quite challenging to understand how particular predictions are made. Interpretability methods propose to give us this understanding. In this talk, I will present a recent result about LIME, perhaps one of the most popular methods. * * *2. Umut Şimşekli* (INRIA/École Normale Supérieure, https://www.di.ens.fr/~simsekli/) *Monday November 14*, 16h00-17h00 In person, at the University of Amsterdam Location: Science Park 904, Room A1.24 *Fractal Structure and Generalization Properties of Stochastic Optimization Algorithms* Understanding generalization in deep learning has been one of the major challenges in statistical learning theory over the last decade. While recent work has illustrated that the dataset and the training algorithm must be taken into account in order to obtain meaningful generalization bounds, it is still theoretically not clear which properties of the data and the algorithm determine the generalization performance. In this talk, I will approach this problem from a dynamical systems theory perspective and represent stochastic optimization algorithms as random iterated function systems (IFS). Well studied in the dynamical systems literature, under mild assumptions, such IFSs can be shown to be ergodic with an invariant measure that is often supported on sets with a fractal structure. We will prove that the generalization error of a stochastic optimization algorithm can be bounded based on the ‘complexity’ of the fractal structure that underlies its invariant measure. Leveraging results from dynamical systems theory, we will show that the generalization error can be explicitly linked to the choice of the algorithm (e.g., stochastic gradient descent – SGD), algorithm hyperparameters (e.g., step-size, batch-size), and the geometry of the problem (e.g., Hessian of the loss). We will further specialize our results to specific problems (e.g., linear/logistic regression, one hidden-layered neural networks) and algorithms (e.g., SGD and preconditioned variants), and obtain analytical estimates for our bound. For modern neural networks, we will develop an efficient algorithm to compute the developed bound and support our theory with various experiments on neural networks. The talk is based on the following publication: Camuto, A., Deligiannidis, G., Erdogdu, M. A., Gurbuzbalaban, M., Simsekli, U., & Zhu, L. (2021). Fractal structure and generalization properties of stochastic optimization algorithms. Advances in Neural Information Processing Systems, 34, 18774-18788. Seminar organizers: Tim van Erven Botond Szabo https://mschauer.github.io/StructuresSeminar/ -- Tim van Erven<tim(a)timvanerven.nl> www.timvanerven.nl

1 0

Tenure track researcher in reinforcement learning at AI Department of the Donders Institute
by Marcel van Gerven 04 Nov '22

04 Nov '22

The AI Department of the Donders Centre for Cognition (DCC), embedded in the Donders Institute for Brain, Cognition and Behaviour, is looking for a tenure track researcher in reinforcement learning with an interest towards applications in neurotechnology. You will be part of the Dutch DBI2 consortium, which is a multi-year/multi-institute project aimed at understanding brain function from a computational and mechanistic perspective (dbi2.nl <http://dbi2.nl/>). You are a researcher in reinforcement learning with an emphasis on safety and robustness. You have a keen interest in applications in neurotechnology and other domains such as robotics, healthcare and/or sustainability. You will perform top-quality research in reinforcement learning, actively contribute to and support the DBI2 consortium, interact and collaborate with other researchers and specialists in academia and/or industry, and be an inspiring member of our staff with excellent communication skills. Furthermore, you will engage with students through teaching and master projects not exceeding 20% of your time. This tenure track position will lead to a tenured assistant or associate professorship after three years (depending on experience and performance). The Donders Institute provides excellent facilities such as computing facilities, a robot lab, a virtual reality lab, behavioural labs, and a technical support group. The AI Department is also a founding member of Radboud AI and the ELLIS Unit Nijmegen (European Excellence Network in Machine Learning). You will join the Artificial Cognitive Systems (ACS) group headed by Prof. van Gerven and interact closely with other machine learning researchers and computational neuroscientists. If you are interested, please check out https://ru.nl/en/working-at/job-opportunities/researcher-in-reinforcement-l… <https://ru.nl/en/working-at/job-opportunities/researcher-in-reinforcement-l…> and feel encouraged to apply (deadline November 23).

1 0

Thematic seminar: Damien Garreau Thu Nov 10, and Umut Simsekli Mon Nov 14
by Tim van Erven 01 Nov '22

01 Nov '22

Dear all, In November we will have two in person talks in the thematic seminar in rapid succession at the UvA. 1. On Thursday November 10 Damien Garreau from the Université Côte d'Azur will speak about his analysis of the popular LIME method for explainable machine learning. 2. And on Monday November 14, Umut Şimşekli from INRIA/École Normale Supérieure will speak about his new generalization bounds for deep neural networks. *1. Damien Garreau *(Université Côte d'Azur, https://sites.google.com/view/damien-garreau/home) *Thursday November 10*, 16h00-17h00 In person, at the University of Amsterdam Location: Science Park 904, Room B0.204 * **What does LIME really see in images? *The performance of modern algorithms on certain computer vision tasks such as object recognition is now close to that of humans. This success was achieved at the price of complicated architectures depending on millions of parameters and it has become quite challenging to understand how particular predictions are made. Interpretability methods propose to give us this understanding. In this talk, I will present a recent result about LIME, perhaps one of the most popular methods. * * *2. Umut Şimşekli* (INRIA/École Normale Supérieure, https://www.di.ens.fr/~simsekli/) *Monday November 14*, 16h00-17h00 In person, at the University of Amsterdam Location: Science Park 904, Room A1.24 *Fractal Structure and Generalization Properties of Stochastic Optimization Algorithms* Understanding generalization in deep learning has been one of the major challenges in statistical learning theory over the last decade. While recent work has illustrated that the dataset and the training algorithm must be taken into account in order to obtain meaningful generalization bounds, it is still theoretically not clear which properties of the data and the algorithm determine the generalization performance. In this talk, I will approach this problem from a dynamical systems theory perspective and represent stochastic optimization algorithms as random iterated function systems (IFS). Well studied in the dynamical systems literature, under mild assumptions, such IFSs can be shown to be ergodic with an invariant measure that is often supported on sets with a fractal structure. We will prove that the generalization error of a stochastic optimization algorithm can be bounded based on the ‘complexity’ of the fractal structure that underlies its invariant measure. Leveraging results from dynamical systems theory, we will show that the generalization error can be explicitly linked to the choice of the algorithm (e.g., stochastic gradient descent – SGD), algorithm hyperparameters (e.g., step-size, batch-size), and the geometry of the problem (e.g., Hessian of the loss). We will further specialize our results to specific problems (e.g., linear/logistic regression, one hidden-layered neural networks) and algorithms (e.g., SGD and preconditioned variants), and obtain analytical estimates for our bound. For modern neural networks, we will develop an efficient algorithm to compute the developed bound and support our theory with various experiments on neural networks. The talk is based on the following publication: Camuto, A., Deligiannidis, G., Erdogdu, M. A., Gurbuzbalaban, M., Simsekli, U., & Zhu, L. (2021). Fractal structure and generalization properties of stochastic optimization algorithms. Advances in Neural Information Processing Systems, 34, 18774-18788. Seminar organizers: Tim van Erven Botond Szabo https://mschauer.github.io/StructuresSeminar/ -- Tim van Erven<tim(a)timvanerven.nl> www.timvanerven.nl

1 0

2024

2023

2022

2021

2020

Machine-learning-nederland November 2022