• AIPressRoom
  • Posts
  • Fast Zero Shot Object Detection with OpenAI CLIP

Fast Zero Shot Object Detection with OpenAI CLIP

Zero shot object detection is made easy with OpenAI CLIP. A state-of-the-art multi-modal deep learning model. Here we will learn about zero shot object detection (and object localization) and how to implement it in practice with OpenAI’s CLIP.

ILSVRC was a world-changing competition hosted annually from 2010 until 2017. It was the catalyst for the Renaissance of deep learning and was the place to find state-of-the-art image classification, object localization, and object detection.

Researchers fine-tuned better-performance computer vision (CV) models to achieve ever more impressive results year-after-year. But there was an unquestioned assumption causing problems.

We assumed that every new task required model fine-tuning; this required *a lot* of data. and this needed both time and capital.

It wasn’t until very recently that this assumption was questioned and proven wrong.

The astonishing rise of multi-modal models has made the impossible possible across various domains and tasks. One of those is zero-shot object detection and localization.

Zero-shot means applying a model without the need for fine-tuning. Meaning we take a multi-modal model and use it to detect images in one domain, then switch to another entirely different domain *without* the model seeing a single training example from the new domain.

Not needing a single training example means we completely skip the hard part of data annotation and model training. We can focus solely on the application of our models.

In this chapter, we will explore how to apply OpenAI’s CLIP to this task—using CLIP for localization and detection across domains with *zero* fine-tuning.

AI Dev Studio:https://aurelio.ai/

00:00 Early Progress in Computer Vision02:03 Classification vs. Localization and Detection03:55 Zero Shot with OpenAI CLIP05:23 Zero Shot Object Localization with OpenAI CLIP06:40 Localization with Occlusion Algorithm07:44 Zero Shot Object Detection with OpenAI CLIP08:34 Data Preprocessing for CLIP13:55 Initializing OpenAI CLIP in Python17:05 Clipping the Localization Visual18:32 Applying Scores for Visual20:25 Object Localization with New Prompt20:52 Zero Shot Object Detection in Python21:20 Creating Bounding Boxes with Matplotlib25:15 Object Detection Code27:11 Object Detection Results28:29 Trends in Multi-Modal ML

#machinelearning #python #openai