Tobias Christian Nauen

PhD Student

RPTU Kaiserslautern-Landau

German Research Center for Artificial Intelligence (DFKI)

Biography

I’m a researcher of artificial intelligence at DFKI and RPTU Kaiserslautern-Landau. My research interests include efficient deep learning, transformer models, multimodal learning, and computer vision. In my PhD project, my focus lies on the development of efficient transformer models for vision, language, and multimodal tasks.

Interests

Artificial Intelligence
Computer Vision
Efficient Transformer Models

Education

M.Sc. in Mathematics, 2022
Leibniz University Hannover
B.Sc. in Computer Science, 2022
Leibniz University Hannover
B.Sc. in Mathematics, 2019
Leibniz University Hannover

Skills

Technical

Python

Linux Terminal

LaTeX

Hobbies

Electric Guitar

Hiking

Bicycle Traveling

Featured Publications

Tobias Christian Nauen, Brian Moser, Federico Raue, Stanislav Frolov, Andreas Dengel

March, 2025 arXiv

ForAug: Recombining Foregrounds and Backgrounds to Improve Vision Transformer Training with Bias Mitigation

We improve the training of vision transformers by segmenting and recombining objects and backgrounds from datasets. This makes the transformers more accurate, as well as more robust.

Tobias Christian Nauen, Sebastian Palacio, Federico Raue, Andreas Dengel

February, 2025 WACV 2025

Which Transformer to Favor: A Comparative Analysis of Efficiency in Vision Transformers

A comprehensive benchmark and analysis of more than 45 transformer models for image classification to evaluate their efficiency, considering various performance metrics. We find the optimal architectures to use and uncover that model-scaling is more efficient than image scaling.

Recent Publications

Quickly discover relevant content by filtering publications.

Tobias Christian Nauen, Sebastian Palacio, Andreas Dengel (2024). TaylorShift: Shifting the Complexity of Self-Attention from Squared to Linear (and Back) using Taylor-Softmax. ICPR 2024 (oral).

PDF Cite Code Project Project Slides DOI Appendix

Brian Bernhard Moser, Federico Raue, Tobias Christian Nauen, Stanislav Frolov, Andreas Dengel (2024). Distill the Best, Ignore the Rest: Improving Dataset Distillation with Loss-Value-Based Pruning. arXiv.

PDF Cite Project Project Project

Arundhati S Shanbhag, Brian Bernhard Moser, Tobias Christian Nauen, Stanislav Frolov, Federico Raue, Andreas Dengel (2024). Just Leaf It: Accelerating Diffusion Classifiers with Hierarchical Class Pruning. arXiv.

PDF Cite Project Project Project

Brian B. Moser, Stanislav Frolov, Tobias Christian Nauen, Federico Raue, Andreas Dengel (2024). Zoomed In, Diffused Out: Towards Local Degradation-Aware Multi-Diffusion for Extreme Image Super-Resolution. arXiv.

PDF Cite Project Project Project

Sanath Budakegowdanadoddi Nagaraju, Brian Bernhard Moser, Tobias Christian Nauen, Stanislav Frolov, Federico Raue, Andreas Dengel (2024). A Low-Resolution Image is Worth 1x1 Words: Enabling Fine Image Super-Resolution with Transformers and TaylorShift. arXiv.

PDF Cite Project Project Project

See all publications

Projects

Albatross

At its core, Albatross is a research project in the area of continual learning.

Sustainable Embedded AI

Energy- and data-saving methods for environmental perception in embedded AI systems using the case study of smart factory and smart farming applications; funded by the Carl Zeiss Foundation.

SustAInML

SustainML is dedicated to creating a sustainable ML framework for Green AI. By prioritizing energy efficiency, SustainML aims to pave the way for environmentally conscious AI solutions that are both efficient and effective.