mtmd, server: add "placeholder bitmap" for counting tokens , add */input_tokens API (#23913)
* mtmd: add "placeholder bitmap" for counting tokens w/o preprocessing * fast path skip preproc for placeholder * fix build * correct the api * add server endpoint + tests * add object name * update docs * add proxy handling * fix build * fix audio input path * use is_placeholder in process_mtmd_prompt() * nits * nits (2) * docs: clarify chat/completions/input_tokens is not official * fix merge problem
This commit is contained in:
@@ -1447,6 +1447,36 @@ See [OpenAI Embeddings API documentation](https://platform.openai.com/docs/api-r
|
||||
}'
|
||||
```
|
||||
|
||||
### POST `/v1/responses/input_tokens`: Token Counting
|
||||
|
||||
Similar to [Response input token counts API](https://developers.openai.com/api/reference/python/resources/responses/subresources/input_tokens/methods/count).
|
||||
|
||||
Example response:
|
||||
|
||||
```json
|
||||
{
|
||||
"object": "response.input_tokens",
|
||||
"input_tokens": 11
|
||||
}
|
||||
```
|
||||
|
||||
### POST `/v1/chat/completions/input_tokens`: Token Counting
|
||||
|
||||
Similar to [Response input token counts API](https://developers.openai.com/api/reference/python/resources/responses/subresources/input_tokens/methods/count), but accepts a chat completion body as input.
|
||||
|
||||
Note: This is not an official OAI endpoint, but is added for completeness and convenience.
|
||||
|
||||
Example response:
|
||||
|
||||
```json
|
||||
{
|
||||
"object": "response.input_tokens",
|
||||
"input_tokens": 11
|
||||
}
|
||||
```
|
||||
|
||||
## Anthropic-compatible API Endpoints
|
||||
|
||||
### POST `/v1/messages`: Anthropic-compatible Messages API
|
||||
|
||||
Given a list of `messages`, returns the assistant's response. Streaming is supported via Server-Sent Events. While no strong claims of compatibility with the Anthropic API spec are made, in our experience it suffices to support many apps.
|
||||
|
||||
Reference in New Issue
Block a user