Amazon Bedrock Intelligent Prompt Routing
Overview
Amazon Bedrock Intelligent Prompt Routing routes prompts to different foundational models within a model family, helping you optimize for quality of responses and cost. Intelligent Prompt Routing can reduce costs by up to 30% without compromising on accuracy.
Maximize performance at lower cost
It can be a challenge for developers to understand which queries require more advanced models or could work with a smaller, faster, and cheaper ones. Using advanced prompt matching and model understanding techniques, Intelligent Prompt Routing predicts the performance of each model for each request and dynamically routes each request to the model that it predicts is most likely to give the desired response at the lowest cost. You can choose from two prompt routers in preview that route requests between either Claude Sonnet 3.5 and Claude Haiku, or between Llama 3.1 7B and Llama 3.1 80B.
Reduce your development effort
To achieve the desired performance and cost for your applications, you must often develop complex orchestration workflows, routing each request to the model best suited for that request based on your experience to achieve the desired performance in terms of accuracy. With Intelligent Prompt Routing, you can save months of effort on testing different models and creating complex orchestration workflows.
Easily debug with fully traceable requests
Each request is fully traceable, enabling you to identify which model handles each request and enabling you to easily understand and debug any issues.
Pricing Notes
During preview, customers are charged regular on-demand pricing the models the requests are routed to. See our pricing page for detailed pricing for different model providers.
