Roadmap
Our vision for the future of autonomous agent evaluation
This roadmap is continuously updated based on community feedback and emerging research. Want to see a feature? Join the discussion →
🎯Our Vision
Make agent evaluation so seamless and automated that teams can deploy with confidence, iterate rapidly, and trust their AI systems to improve continuously without manual intervention.
Q4 2024
Completed
- â–¸Core evaluation framework with TypeScript code generation
- â–¸Langfuse integration for trace collection
- â–¸Basic meta-prompting for ReAct agents
- â–¸CLI tool for init, generate, and eval commands
- â–¸Open source release under MIT license
Q1 2025
In Progress
- â–¸LangGraph native adapter
- â–¸OpenAI AgentKit integration
- â–¸Web dashboard for eval visualization
- â–¸Deployment gating (block deploys on eval failures)
- â–¸Eval versioning and history tracking
- â–¸Auto-prompt optimizer improvements
Q2 2025
Planned
- â–¸LangSmith and TruLens adapters
- â–¸Multi-language eval generation (Python, Go)
- â–¸A/B testing framework for prompt variations
- â–¸Dataset curation from production traces
- â–¸Collaborative eval editing in dashboard
- â–¸CI/CD integrations (GitHub Actions, GitLab)
Q3 2025
Planned
- â–¸Custom eval templates library
- â–¸Eval marketplace (share/discover evals)
- â–¸Advanced hallucination detection techniques
- â–¸Automated regression testing
- â–¸Cost tracking and optimization insights
- â–¸Enterprise features (SSO, RBAC, audit logs)
Q4 2025
Planned
- â–¸Multi-agent system evaluation
- â–¸Cross-framework benchmarking
- â–¸Reinforcement learning from eval feedback
- â–¸Advanced meta-prompting with constitutional AI
- â–¸Real-time eval streaming
- â–¸Community-contributed eval functions
Help Shape the Future
iofold is open source and community-driven. We want to hear from you:
- ✓Vote on features in GitHub Discussions
- ✓Propose new ideas in GitHub Issues
- ✓Contribute code via pull requests
- ✓Share your use cases in our Discord community