Introduction
The Intention of this blog is to facilitate better communication between Data Scientists and the Data Annotation teams spread across the organization precisely with regards to the labeling task for image classification use case.
As it’s being said, “opportunity never knock twice” but this clear-cut leaflet in the hands of “Image Annotators” will help “Data Scientists” to counter-back loopholes in the training dataset, which were left unattended or ignored during the image cleaning process.
Move to the next section and read through how Data Labelers can act as an alert torch for the team of Data Specialists.
The Six Alarms, Every Data Scientist Should Be Informed About
An image annotator, working on an image classification job’s sole responsibility is not only to perform the given in hand image labeling task.
But also to warn Data Scientists regarding the below-mentioned alerts which may introduce unrealized risks in the dataset if not addressed quickly.
A. Too much “Duplication”
Duplication simply means that there are lots of images in the dataset with respect to class/classes that are repeating/reoccurring throughout the dataset.
It may be due to multiple reasons such as while web scraping the Data Scientist must have scrape the same webpage with images more than once or the same images must have been present in two different webpages.
Or the open dataset which data scientist provided to the labeling team for getting the custom labels was not cleaned appropriately.
Whatever may be the reason, repeating images doesn’t allow the Machine learning model of a Data Scientist to generalize well because it’s learning the same information over and over again.
In short, “more visual data does not always equate to more information“. The next plan of action for the current on-going dataset is to notify the Data Scientist about this image repetitions issue.
B. Blurry Images, unless the whole dataset is blurry.
While dealing with a computer vision use case, considering the Machine Learning model due to lack of visual clarity will not be able to extract prescriptive information or features about the object of interest from the blurry or pixelated images.
Therefore it becomes a necessary next step for labelers to inform the Data Scientist about it and let them take the required action.
But here the catch, if the whole dataset is blurry then maybe the Data Scientist is working on a production use case that demands blurriness of the image, in that case just confirm the same with the Data Scientist.
C. Too many confusing examples
The advantage of any Machine Learning model resides in the quality of inputs given to the model for learning a specific task.
If the Data Scientist provides the Annotation team a dataset that consists of too many confusing examples such as those visible in the below image.
Then the data labelers just have to voice their concern to the Data Scientist and quiz him/her for the next suitable sets of instructions.
D. Biasness towards a particular class in the dataset.
This alert is the one in which the data labelers have to take maximum precaution.
That’s why while labeling image classification dataset or in fact, if data labelers are labeling any other computer vision dataset.
If they observe that one class [such as in the below image more images of females are there] is having too many images as compared to other class/classes.
Then they have to inform the Data Scientists team immediately about it. Otherwise, this dataset will be consumed in making a Machine Learning Model favoring the class that had more images as compared to other class/classes in the dataset.
In short, the Machine Learning Model will be biased towards that particular class. Which directly may lead to loss of revenue or public relation setbacks post-deployment of that AI Model.
Dataset Name: Gender Classification dataset
Class Names: Male, Female
E. The object of interest or class to be labeled is having a blur appearance.
This scenario is generally observed at the class level rather than on the image level. Thus while conducting the image labeling job.
If the data labeler observes that the object of interest or class/classes to be labeled of the dataset is having blurry or unclear appearances in the entire image.
Then they should simply inform the Data Scientist about it and ask for the necessary advice moving forward.
The Data Scientists team might take action such as replacing or removing the images from the ongoing dataset.
F. The object of interest or class intended to be labeled is partially visible.
As it’s being said “Half Knowledge is dangerous” the same is true for any Computer vision Dataset in the world.
For e.g.: Look at the below image, in this image only the paws of either dog or cat are visible.
Thus because of the lack of full-length size image of these animals, the image annotator will think twice before putting them in either the Cat or the Dog Category.
Under such circumstances, the image annotator should alert the Data Scientist. So that she/he can take the required measures towards these kinds of lack of context images present in their Image Classification Dataset.
Dataset Name: Dog vs Cat Classification
Class Name: Dog, Cat
Hence with the above point, it marked the end of this blog.
I hope next time when any Data Scientist will give an Image Classification task to work upon he/she will be transferring the knowledge about these alerts to their data annotation team.
Which eventually will help the Machine Learning teams of various companies in building Machine Learning Datasets that present the true and complete picture of the Objects of interest.
Label Image Classification Datasets through NeuralMarker
Visit NeuralMarker to explore the training data annotation platform or Request a demo to learn how this platform fuelled with AI can assist you in Pre-labeling Dataset for Image Classification Tasks.