I have recently started as a post-doc with Kyunghyun Cho and Krzysztof Geras at New York University. I am also an advisor at Molecule.one . I do my best to contribute to the broad machine learning community. Currently, I am on the Scientific Committee of EEML, and serve as an area chair for NeurIPS. I received my PhD from Jagiellonian University co-supervised by Jacek Tabor and Amos Storkey (University of Edinburgh). During PhD, I spent two summers as a visiting researcher with
Yoshua Bengio, and collaborated with Google Research in Zurich.
My email is staszek.jastrzebski (on gmail).
- (12.2019) Our work on the early phase of the optimization trajectory has been accepted to ICLR 2020 as Spotlight! ArXiv will follow soon.
- (09.2019) Large Scale Structure of Neural Networks' Loss Landscape has been accepted to NeurIPS 2019!
- (06.2019) Received top 5% reviewer award for ICML 2019 :)
- (03.2019) Our paper on parameter efficient training of BERT was accepted to ICML 2019!
- (01.2019) Papers accepted to ICLR 2019, and AISTATS 2019! Also, our preprint on Neural Architecture Search is online.
My main research goal is to understand and improve how deep network generalize. My research interests include:
- Optimization in Deep Learning
- Representation Learning
- Natural Language Processing
- Computer Aided Drug Design
- [MSc] Olivier Astrand (NYU) - Regularization effects of instability in training
- [MSc] Sławomir Mucha (UJ) - Pretraining in deep learning in cheminformatics
- [MSc] Tobiasz Ciepliński (UJ) - Evaluating generative models in chemistry using docking simulators
- [BSc] Michał Zmysłowski (UW) - Gradient structure and generalization
- You? I am always looking for new promising students
- [Defended, MSc] Tomasz Wesołowski - Relevance of enriching word embeddings in modern deep natural language processing
- [Defended, MSc] Andrii Krutsylo - Physics aware representation for drug discovery
- [Defended, BSc] Michał Soboszek - Evaluating word embeddings
- [Defended, MSc] Jakub Chłędowski - Representation learning for textual entailment
- [Defended, MSc] Mikołaj Sacha - Meta learning and sharpness of the minima
For a full list please see my Google Scholar profile.
On the Relation Between the Sharpest Directions of DNN Loss and the SGD Step Length
S. Jastrzębski, Z. Kenton, N. Ballas, A. Fischer, Y. Bengio, A. Storkey
International Conference on Learning Representations 2019
Three factors influencing minima in SGD
S. Jastrzębski*, Z. Kenton*, D. Arpit, N. Ballas, A. Fischer, Y. Bengio, A. Storkey
International Conference on Artificial Neural Networks 2018 (oral), International Conference on Learning Representations 2018 (workshop)
Residual Connections Encourage Iterative Inference
S. Jastrzębski*, D. Arpit*, N. Ballas, V. Verma, T. Che, Y. Bengio
International Conference on Learning Representations 2018
A Closer Look at Memorization in Deep Networks
D. Arpit*, S. Jastrzębski*, N. Ballas*, D. Krueger*, T. Maharaj, E. Bengio, A. Fischer, A. Courville, S. Lacoste-Julien, Y. Bengio
International Conference on Machine Learning 2017
Quo vadis G Protein-Coupled Receptor ligands? A tool for analysis of the emergence of new groups of compounds over time
D. Lesniak, S. Jastrzębski, W. M. Czarnecki, S. Podlewska, A. Bojarski
Bioorganic & Medicinal Chemistry Letters, 2017
Learning to SMILE(S)
S. Jastrzębski, D. Lesniak, W. M. Czarnecki
International Conference on Learning Representations 2016 (workshop track)
Research intern, automatic machine learning
11.2018-, Zurich, Switzerland
Machine Learning intern, fraud detection models and Deep NLP applications
7-9.2016, London, UK
University of Edinburgh
Research intern, Deep Learning for Go, under the supervision of prof. Amos Storkey
9-11.2015, Edinburgh, UK
SDE intern, distributed data systems
2015, Palo Alto, USA
SDE intern, API research and design
2014, Redmond, USA
SDE intern, data processing framework development
2013, London, UK
Word Embeddings Benchmarks
Python package for evaluating word embeddings.
ML algorithms C++ implementations for R, including online clustering, swappable SVM library interface and new clustering algorithm.
Simulator for 2015 and 2016 editions online qualification round.
Understanding How Deep Networks Learn
Invited talk @ PL in ML
2017, Warsaw, Poland