Featured publications

CVPR 2022

We show that most INR families are analogous to structured signal dictionaries whose atoms are integer harmonics of the set of initial mapping frequencies whose inductive bias is akin to that of signal dictionary formed by the eigenfunctions of the NTK at initialization. We reveal that meta-learning has a reshaping effect of the NTK analogous to dictionary learning, building dictionary atoms as a combination of the examples seen during meta-training.
PDF Code
NeurIPS 2021

We provide strong empirical evidence to determine the practical validity of the linear approximation of neural networks for different learning tasks. Specifically, we discover that, in contrast to what was previously observed, neural networks do not always perform better than their kernel approximations, and reveal that their performance gap heavily depends on architecture, number of samples and training task.
PDF Code
NeurIPS 2020

We analyze the role of the network architecture in shaping the inductive bias of deep classifiers. Specifically, we introduce the neural anisotropy directions (NADs) of a network as the vectors that encapsulate the directional inductive bias of an architecture. These vectors, encode the preference of a network to separate the input data based on some particular features.
PDF Blog Code Video
NeurIPS 2020

Important insights towards the explainability of neural networks reside in the characteristics of their decision boundaries. In this work, we borrow tools from the field of adversarial robustness, and propose a new perspective that relates dataset features to the distance of samples to the decision boundary. Specifically, we rigorously confirm that neural networks exhibit a high invariance to non-discriminative features, and show that very small perturbations of the training samples in certain directions can lead to sudden invariances in the orthogonal ones.
PDF Code

Research trajectory

 
 
 
 
 

Research Intern

Google

Jul 2022 – Oct 2022 Zürich, Switzerland
I am a research intern at Google Zurich hosted by Efi Kokiopoulou and Rodolophe Jenatton.
 
 
 
 
 

Visiting Researcher (ELLIS PhD)

University of Oxford

Jan 2022 – Jun 2022 Oxford, UK
Conducted research on the robustness of neural networks under the supervision of Philip Torr.
 
 
 
 
 

PhD Researcher

EPFL

Nov 2018 – Present Lausanne, Switzerland
Studying the inductive bias of deep learning and how it affects its generalization and robustness properties. My research has already provided insights to improve robustness, out-of-distribution generalization, and to understand the role of architecture in deep learning.
 
 
 
 
 

Research Intern

Philips Healthcare Research

Jul 2017 – Oct 2017 Hamburg, Germany
As a research intern, I was involved in different projects related to computational medical imaging and fetal ultrasound. In particular, I was in charge of the development of different self-supervised deep learning algorithms for representation learning and medical image reconstruction.
 
 
 
 
 

Master Student

Delft University of Technology

Sep 2016 – Aug 2018 Delft, Netherlands
For my master thesis I carried out research on deep learning for multidomain graphs and on sparse sampling for inverse problems with tensors.
 
 
 
 
 

Research Assistant

Microwaves and Radar Group, UPM

Jul 2015 – Jul 2016 Madrid, Spain
During my time at the group, I developed a complete electromagnetic simulator for the mm-wave radar that the group was developing for airport security applications. In particular, we were one of the first groups to propose the use of 3D rendering techniques from computer graphics to accurately model the scattering from the human skin at the submillimeter band.

Awards

Highlighted Reviewer

ICLR 2022 Apr 2022
Recognized as a Highlighted Reviewer for my contributions to the review process of ICLR 2022.

Outstanding Reviewer Award

NeurIPS 2021 Oct 2021
Given to the top 8% of reviewers who were judged to be instrumental to the review process based on Area Chair and author feedback.

National Award for Excellence in Academic Performance (2nd Prize)

Ministry of Education of Spain Nov 2018
Most prestigious academic award for undergraduate students in Spain.

Best Graduate

Delft University of Technology Nov 2018
Top student of the Faculty of Electrical Engineering, Mathematics and Computer Science.

La Caixa Fellowship

La Caixa Foundation Sep 2016 – Aug 2018
One of the most prestigious international postgraduate scholarships in Spain. It provided complete funding of my master studies.

Best ICT Graduate

Universidad Politécnica de Madrid Jan 2016
Best student among all students graduating at UPM in a degree related to Information and Communications Technology (ICT).

Valedictorian

Faculty of Telecommunications. Universidad Politécnica de Madrid Jan 2016
Top student of the Faculty of Telecommunications.

Academic Excellence Scholarship

Autonomous Community of Madrid Sep 2012 – Aug 2015
Yearly allowance to the top undergraduate students in Madrid.

Publications

Ordered by date

Check my Google Scholar for an updated list or quickly discover relevant content by filtering publications.

(2022). Catastrophic overfitting is a bug but also a feature. arXiv (Preprint).
(2022). PRIME: A few primitives can boost robustness to common corruptions. ECCV 2022.
(2022). On the benefits of knowledge distillation for adversarial robustness. Preprint (arXiv).
(2021). A structured dictionary perspective on implicit neural representations. CVPR 2022.
(2021). What can linearized neural networks actually say about generalization?. NeurIPS 2021.
(2021). A neural anisotropic view of underspecification in deep learning. RobustML Workshop (ICLR 2021).
(2020). Neural Anisotropy Directions. NeurIPS 2020.
(2020). Hold me tight! Influence of discriminative features on deep network boundaries. NeurIPS 2020.
(2020). Optimism in the Face of Adversity: Understanding and Improving Deep Learning through Adversarial Robustness. Proceedings of IEEE.
(2020). Redundant features can hurt robustness to distribution shift. UDL Workshop (ICML 2020).
(2020). On the Choice of Graph Neural Network Architectures. IEEE ICASSP 2020.
(2020). Forward-Backward Splitting for Optimal Transport based Problems. IEEE ICASSP 2020.
(2019). CDOT: Continuous Domain Adaptation using Optimal Transport. OTML Workshop (NeurIPS 2019).
(2019). Sparse Sampling For Inverse Problems with Tensors. IEEE Transactions of Signal Processing.
(2018). Sampling and Reconstruction of Signals on Product Graphs. IEEE GlobalSIP 2018.
(2018). Multidomain Graph Signal Processing. MSc Thesis.
(2017). Simulation Framework for a 3-D High-Resolution Imaging Radar at 300 GHz with a Scattering Model Based on Rendering Techniques. IEEE Transactions on Terahertz Science and Technology.

Contact

guillermo.ortizjimenez[at]epfl.ch