原文要点
原文完整内容
Copy Page
Rate limits are restrictions that our API imposes on the number of times a user or client can
access our services within a specified period of time.
Why do we have rate limits?
Rate limits are a common practice for APIs, and they’re put in place for a few different reasons:
They help protect against abuse or misuse of the API.
For example, a malicious actor could flood the API with requests in an attempt to overload it or cause disruptions in service. By setting rate limits, OpenAI can prevent this kind of activity.
Rate limits help ensure that everyone has fair access to the API.
If one person or organization makes an excessive number of requests, it could bog down the API for everyone else. By throttling the number of requests that a single user can make, OpenAI ensures that the most number of people have an opportunity to use the API without experiencing slowdowns.
Rate limits can help OpenAI manage the aggregate load on its infrastructure.
If requests to the API increase dramatically, it could tax the servers and cause performance issues. By setting rate limits, OpenAI can help maintain a smooth and consistent experience for all users.
Please work through this document in its entirety to better understand how
OpenAI’s rate limit system works. We include code examples and possible
solutions to handle common issues. We also include details around how your
rate limits are automatically increased in the usage tiers section below.
How do these rate limits work?
Rate limits use metrics such as
RPM
(requests per minute),
RPD
(requests per day),
TPM
(tokens per minute),
TPD
六、实际策略
- 生产 Key 与实验 Key 分开限额。
- 设置最大 request volume、最大 daily cost、最大 tokens per request 三个护栏。
- 长任务必须有 batch size 和 sleep。
- 使用指数退避，不要固定间隔重试。
- 所有重大任务跑前先做 cost projection。
七、告警模板
警告： - 单项目每日消耗超过 $80 - 单 Key 每分钟请求数超过 300 - 单次输出 tokens 超过 8192 - 失败率连续 5 分钟高于 10% - P95 延迟超过 45 秒