Trendvesting: The Hidden Costs of an AI Project in Production

⚡

Key Takeaways

1Trendvesting, a financial signal analysis platform, was launched on February 14, 2024, with 1,672 commits.

2The main costs of the project come from infrastructure, not model tokens, contradicting initial expectations.

3Cost optimization revealed unnecessary expenses due to poor-quality data and verbose outputs.

💡Why it matters — This article highlights the real costs of AI in production, often underestimated, influencing strategic decisions for tech companies.

An Ambitious Project in Production

Trendvesting is a platform specializing in financial signal intelligence, particularly for stocks and options. The project took off with its first commit on February 14, 2024. Since then, it has experienced significant growth, reaching 1,672 commits. This evolution has been achieved through a multi-mode Go backend, a Next.js application, a React Native client, and a Python FastAPI consensus service, illustrating the complexity and technological diversity of the project.

The Real Costs of Infrastructure

The author of the study emphasizes that, contrary to expectations, the major costs of an AI system in production do not come from model tokens. In reality, it is the fixed infrastructure that weighs most heavily on the budget. Elements such as Kubernetes clusters, backups, and observability are essential to ensure the reliable operation of the system. This revelation challenges cost assumptions based solely on tokens, which are often misleading.

Optimization and Lessons Learned

The analysis of expenses has highlighted how optimization can influence the cost curve. Three main "taxes" were identified during the project:

The cost of poor-quality or free data, particularly crucial for options.
Duplication of work due to a lack of caching or deduplication.
Financial waste caused by verbose outputs, where the absence of constraints on response formats leads to direct costs due to the asymmetry of input and output pricing.

Migration to More Economical Models

In March, a migration to less expensive models was carried out. This transition allowed for measuring the impact of signal loss, demonstrating that smaller models can be effective when paired with rigorous risk management. Emphasis was placed on the importance of stop-losses, which are often more crucial than the sophistication of the model itself.

Cost per Signal and Transparency

The study concludes by calculating the fully loaded cost per signal, estimated between $1.50 and $2.00. This figure highlights that, at a small scale, unit economics are largely dominated by fixed costs. The author encourages others to publish their own detailed ledgers, emphasizing that this transparency is rare but valuable in the field of AI in production.