Text Classification Without Learning: A Revolution?

A New Approach to Text Classification

Zero-shot text classification stands out for its ability to assign labels to texts without requiring a training phase on a specific dataset. Unlike traditional methods, where a model is trained to recognize predefined categories, this technique relies on a general understanding of language to determine the most appropriate label from a provided list.

This method proves particularly advantageous in contexts where it is crucial to quickly test an idea or work with frequently evolving labels. It also allows for the creation of lightweight prototypes before committing to more costly supervised training. Rather than relying on a fixed match between text and labels, the model uses its understanding of language to evaluate the meaning of each label.

How Zero-Shot Classification Works

The essence of this approach lies in the fact that the model does not view labels as mere categories. Each label is transformed into a natural language statement, and the model checks whether this statement is supported by the input text. This makes the method particularly useful for practical applications such as support ticket routing, article tagging, user intent detection, or organizing internal documents.

For example, if the input text is "The company launched a new AI platform for professional clients," and the candidate labels are "technology," "sports," and "finance," the model reformulates these labels into statements like "This text is about technology." It then compares the text to these statements to determine the most relevant label.

Practical Applications

Instead of limiting itself to general subject labels, a company can choose more specific labels such as "billing issue," "technical support," or "refund request" to classify customer service messages. Similarly, for moderation systems, labels like "spam," "harassment," or "secure" can be used.

Zero-shot classification is not treated as a traditional classification problem. It is rather considered a reasoning problem regarding the relevance of a label description in relation to the text. This explains why it is effective for rapid prototyping, low-resource tasks, and areas where labeled data is scarce.

Practical Implementation Examples

Loading the Classification Pipeline

To get started, it is necessary to install the required libraries:

pip install torch transformers

Next, the pipeline can be loaded:

from transformers import pipeline
classifier = pipeline(
    "zero-shot-classification",
    model="facebook/bart-large-mnli"
)

This pipeline provides a simplified way to use a pre-trained model without having to write complex inference code. The facebook/bart-large-mnli model is often used for this task as it is designed to evaluate whether one text supports another.

Simple Classification Example

Here is a basic example:

text = "This tutorial explains how [transformer](/glossaire/transformer) models are used in [NLP](/glossaire/nlp)."
candidate_labels = ["technology", "health", "sports", "finance"]
result = classifier(text, candidate_labels)
print(f"Best prediction: {result['labels'][0]} ({result['scores'][0]:.2%})")

Output:

• Best prediction: technology (96.52%)

This shows that the model chooses the label that best matches the meaning of the text. Since the sentence discusses transformer models and natural language processing, "technology" is the best fit among the proposed labels.

Multi-Label Classification

Sometimes a text may belong to multiple categories. In this case, the multi_label=True option can be activated:

text = "The company launched a health app and announced strong business growth."
candidate_labels = ["technology", "health", "business", "travel"]
result = classifier(
    text,
    candidate_labels,
    multi_label=True
)
threshold = 0.50
top_labels = [
    (label, score)
    for label, score in zip(result["labels"], result["scores"])
    if score >= threshold
]
print("Top labels:", ", ".join(f"{label} ({score:.2%})" for label, score in top_labels))

Output:

• Top labels: health (99.41%), technology (99.06%), business (98.15%)

This feature is useful when multiple labels may apply to the same text. In this example, the sentence pertains to technology, health, and business, which is reflected in the high scores assigned to these labels.

Customizing the Hypothesis Model

It is also possible to customize the hypothesis model. While the pipeline uses a default formulation, a clearer or more natural model can sometimes improve results:

text = "The user cannot access their account and continues to see a login error."
candidate_labels = ["technical support", "billing issue", "feature request"]
result = classifier(
    text,
    candidate_labels,
    hypothesis_template="This text is about {}."
)
for label, score in zip(result["labels"], result["scores"]):
    print(f"{label}: {score:.4f}")

Output:

• technical support: 0.7349
• feature request: 0.1683
• billing issue: 0.0968

This customization is particularly useful when the labels are specific to a real task. A good hypothesis model provides the model with a clearer statement to evaluate, thus improving the match between the text and the desired label.

Conclusion

Zero-shot text classification offers a powerful solution to eliminate the need for task-specific training in many early-stage workflows. Rather than immediately collecting labeled data, it is often possible to achieve useful results by selecting clear candidate labels and allowing the model to reason about them.

The key idea is that the model does not directly predict labels in the traditional sense. It evaluates whether each label, formulated as a concise hypothesis, is supported by the text. This is why models trained on MNLI, such as facebook/bart-large-mnli, are so effective for this task.

In practice, the quality of the results heavily depends on the clarity of the defined labels. A solid label formulation and a sensible hypothesis model can make a notable difference. While zero-shot classification is an excellent starting point, it works best when the semantics of the categories are carefully considered.