Deep learning

Deep Learning for Computer Vision is not just Transformers

Looking back to the 2010s, those years were characterized by the resurgence of Neural Networks and, in particular, Convolutional Neural Networks (ConvNet). Since the introduction of AlexNet, the field has evolved at a very fast pace. ConvNets have been successful due to several built-in inductive biases, the most important of which is translation equivariance.

In parallel, the Natural Language Processing (NLP) field took a very different path, with Transformers becoming the dominant architecture.

These two streams converged in 2020 with the introduction of the Vision Transformer (ViT), which outperformed classic ConvNets when it comes to large datasets. But, the simple “patchify” layer at the beginning, which converts the full image into different patches treated as tokens, made the ViT unsuitable for fine-grained applications, such as semantic segmentation. Swin Transformers filled this gap with their sliding attention windows, which, funnily, made Transformers behave more like ConvNets.

At this point, a natural question would be: if researchers are trying to make Transformers behave like ConvNets, why not stick with the latter? The answer is that transformers have been always considered to have a superior scaling behavior that outperforms classical Convnets in many vision tasks.

Read more

Veille-cyber

Share
Published by
Veille-cyber

Recent Posts

Sécurité des mots de passe : bonnes pratiques pour éviter les failles

Sécurité des mots de passe : bonnes pratiques pour éviter les failles La sécurité des…

4 jours ago

Ransomware : comment prévenir et réagir face à une attaque

Ransomware : comment prévenir et réagir face à une attaque Le ransomware est l’une des…

5 jours ago

Cybersécurité et e-commerce : protéger vos clients et vos ventes

Cybersécurité et e-commerce : protéger vos clients et vos ventes En 2025, les sites e-commerce…

1 semaine ago

Les ransomwares : comprendre et se défendre contre cette menace

Les ransomwares : comprendre et se défendre contre cette menace En 2025, les ransomwares représentent…

1 semaine ago

RGPD et cybersécurité : comment rester conforme en 2025

RGPD et cybersécurité : comment rester conforme en 2025 Depuis sa mise en application en…

1 semaine ago

VPN : un outil indispensable pour protéger vos données

VPN : un outil indispensable pour protéger vos données Le VPN, ou « Virtual Private…

2 semaines ago

This website uses cookies.