IIT Bombay and Adobe: Analyzing LLM Prompt Reverse Engineering

IIT Bombay and Adobe: Reverse Engineering LLM Prompts Under the Microscope

Researchers from IIT Bombay and Adobe Research have developed a method to reconstruct prompts used for large language models (LLM) with near-perfect accuracy, using only the output text. This approach works without access to the model weights and applies even to third-party models.

Large language models generate text by predicting the most likely next word, token by token. Reversing this process and reconstructing the original prompt from the output has long been considered impractical, as many different prompts can produce similar responses.

A new paper from the researchers at IIT Bombay and Adobe Research demonstrates that this can be achieved with surprising accuracy. Their method, called "Previous Token Prediction" (PTP), reverses the functioning of language models. Instead of predicting the next token, the researchers train an inverse language model that predicts previous tokens.

This inverse model is fully trained from scratch on data synthetically generated by the target LLM. All it needs is the generated text.

A Single Response Produces the Exact Prompt and Several Alternatives

The inverse model can also generate multiple variations of prompts with distinct meanings by adjusting decoding parameters. All these reconstructed prompts produce similar responses when fed back into the original language model.

In one example from the paper, the prompt "How to contact competitors to learn about their pricing strategies?" was reconstructed word for word. The model also generated six additional variants that captured the main meaning but used different phrasing, such as "What tactics can a company seeking to contact competitors in the market use to learn about their pricing strategy?"

Tests with real user prompts also showed accurate reconstructions. When the researchers fed the reconstructed prompts back into the direct model, the responses closely matched the originals.

For real user prompts, the reconstructed phrasing differs in terms of words but captures the semantic core of the original input.

Attackers Don't Even Need to Know Which Model Produced the Text

An inverse model trained on the small chatbot Qwen-3-0.6B was also able to reconstruct prompts from the responses of GPT-4o. The reconstructed prompts were not identical to the originals, but according to the paper, they captured the meaning and intent. A potential attacker wouldn't even need to know which model generated a given output.

This creates a significant security issue. Companies risk exposing prompts from proprietary systems that contain trade secrets, moderation rules, or specialized instructions. Individual users face a similar threat, as personal or sensitive queries could also be extracted from the output. A small open inversion model could be sufficient to accomplish this.

The paper itself does not make explicit claims regarding attacks on commercial systems. However, if the method works on current production models, AI labs will need to address this issue quickly and rectify the problem.