聊天

创建聊天完成

POST /chat/completions

从选定的 LLM 生成聊天消息完成。

Chat

post

Authorizations

Portkey-Key & Virtual-KeyPortkey-Key & Provider-Auth & Provider-NamePortkey-Key & ConfigPortkey-Key & Provider-Auth & Provider-Name & Custom-Host

x-portkey-api-keystringRequired

x-portkey-virtual-keystringRequired

Header parameters

x-portkey-trace-idstringOptional

An ID you can pass to refer to one or more requests later on. If not provided, Portkey generates a trace ID automatically for each request. Docs

x-portkey-span-idstringOptional

An ID you can pass to refer to a span under a trace.

x-portkey-parent-span-idstringOptional

Link a child span to a parent span

x-portkey-span-namestringOptional

Name for the Span ID

x-portkey-metadataobjectOptional

Pass any arbitrary metadata along with your request

x-portkey-cache-namespacestringOptional

Partition your Portkey cache store based on custom strings, ignoring metadata and other headers

x-portkey-cache-force-refreshbooleanOptional

Forces a cache refresh for your request by making a new API call and storing the updated value

Body

application/json

modelany ofRequired

ID of the model to use. See the model endpoint compatibility table for details on which models work with the Chat API.

Example: gpt-4-turbo

stringOptional

or

string · enumOptionalPossible values:

frequency_penaltynumber | nullableOptional

Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim.

See more information about frequency and presence penalties.

Default: 0

logprobsboolean | nullableOptional

Whether to return log probabilities of the output tokens or not. If true, returns the log probabilities of each output token returned in the content of message.

Default: false

top_logprobsinteger | nullableOptional

An integer between 0 and 20 specifying the number of most likely tokens to return at each token position, each with an associated log probability. logprobs must be set to true if this parameter is used.

max_tokensinteger | nullableOptional

The maximum number of tokens that can be generated in the chat completion.

The total length of input tokens and generated tokens is limited by the model's context length. Example Python code for counting tokens.

ninteger | nullableOptional

How many chat completion choices to generate for each input message. Note that you will be charged based on the number of generated tokens across all of the choices. Keep n as 1 to minimize costs.

Default: 1Example: 1

presence_penaltynumber | nullableOptional

Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics.

See more information about frequency and presence penalties.

Default: 0

response_formatone ofOptional

An object specifying the format that the model must output.

Setting to { "type": "json_schema", "json_schema": {...} }enables Structured Outputs which ensures the model will match your supplied JSON schema. Works across all the providers that support this functionality. OpenAI & Azure OpenAI, Gemini & Vertex AI.

Setting to { "type": "json_object" } enables the older JSON mode, which ensures the message the model generates is valid JSON.

Using json_schema is preferred for models that support it.

or

or

seedinteger | nullableOptional

This feature is in Beta. If specified, our system will make a best effort to sample deterministically, such that repeated requests with the same seed and parameters should return the same result. Determinism is not guaranteed, and you should refer to the system_fingerprint response parameter to monitor changes in the backend.

stopone ofOptional

Up to 4 sequences where the API will stop generating further tokens.

Default: null

string | nullableOptional

or

string[] · min: 1 · max: 4Optional

streamboolean | nullableOptional

If set, partial message deltas will be sent, like in ChatGPT. Tokens will be sent as data-only server-sent events as they become available, with the stream terminated by a data: [DONE] message. Example Python code.

Default: false

temperaturenumber | nullableOptional

What sampling temperature to use, between 0 and 2. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic.

We generally recommend altering this or top_p but not both.

Default: 1Example: 1

top_pnumber | nullableOptional

An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered.

We generally recommend altering this or temperature but not both.

Default: 1Example: 1

tool_choiceone ofOptional

Controls which (if any) tool is called by the model. none means the model will not call any tool and instead generates a message. auto means the model can pick between generating a message or calling one or more tools. required means the model must call one or more tools. Specifying a particular tool via {"type": "function", "function": {"name": "my_function"}} forces the model to call that tool.

none is the default when no tools are present. auto is the default if tools are present.

string · enumOptional

none means the model will not call any tool and instead generates a message. auto means the model can pick between generating a message or calling one or more tools. required means the model must call one or more tools.

Possible values:

or

parallel_tool_callsbooleanOptional

Whether to enable parallel function calling during tool use.

Default: true

userstringOptional

A unique identifier representing your end-user, which can help OpenAI to monitor and detect abuse. Learn more.

Example: user-1234

function_callone ofOptionalDeprecated

Deprecated in favor of tool_choice.

Controls which (if any) function is called by the model. none means the model will not call a function and instead generates a message. auto means the model can pick between generating a message or calling a function. Specifying a particular function via {"name": "my_function"} forces the model to call that function.

none is the default when no functions are present. auto is the default if functions are present.

string · enumOptional

none means the model will not call a function and instead generates a message. auto means the model can pick between generating a message or calling a function.

Possible values:

or

Responses

200

OK

application/json

post

/chat/completions

DefaultSelf-HostedDefaultSelf-HostedDefaultSelf-Hosted

curl https://api.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -H "x-portkey-virtual-key: $PORTKEY_PROVIDER_VIRTUAL_KEY" \
  -d '{
    "model": "gpt-4o",
    "messages": [
      {
        "role": "system",
        "content": "You are a helpful assistant."
      },
      {
        "role": "user",
        "content": "Hello!"
      }
    ]
  }'

200

OK

{
  "id": "text",
  "choices": [
    {
      "finish_reason": "stop",
      "index": 1,
      "message": {
        "content": "text",
        "tool_calls": [
          {
            "id": "text",
            "type": "function",
            "function": {
              "name": "text",
              "arguments": "text"
            }
          }
        ],
        "role": "assistant",
        "content_blocks": [
          {
            "type": "text",
            "text": "text"
          }
        ]
      },
      "logprobs": {
        "content": [
          {
            "token": "text",
            "logprob": 1,
            "bytes": [
              1
            ],
            "top_logprobs": [
              {
                "token": "text",
                "logprob": 1,
                "bytes": [
                  1
                ]
              }
            ]
          }
        ]
      }
    }
  ],
  "created": 1,
  "model": "text",
  "system_fingerprint": "text",
  "object": "chat.completion",
  "usage": {
    "completion_tokens": 1,
    "prompt_tokens": 1,
    "total_tokens": 1
  }
}

请求体类似于 OpenAI 的 聊天完成请求，响应将是 聊天完成对象。选择 stream:true 时，响应将是 聊天完成块 对象的流。

Portkey 会自动转换 OpenAI 以外的 LLM 的参数。如果某些参数在其他 LLM 中不存在，则将被删除。

SDK 使用

Portkey SDK 中的 chat.completions.create 方法使您能够使用各种大型语言模型（LLMs）生成聊天补全。此方法旨在与 OpenAI 聊天补全 API 相似，为习惯于 OpenAI 服务的用户提供熟悉的接口。

方法签名

NodeJS

portkey.chat.completions.create(requestParams[, configParams]);

Python

有关 REST API 示例，请滚动 这里。

参数

1. requestParams (Object): 聊天完成请求的参数，详细说明聊天交互。这些参数类似于 OpenAI 请求签名。Portkey 会自动转换适用于除 OpenAI 以外的 LLM 的参数。如果某些参数在其他 LLM 中不存在，它们将被丢弃。Portkey 默认是多模态的，因此与视觉模型相关的参数，如 image_url、base64 data 也受到支持。

2. configParams (Object): 请求的附加配置选项。这是一个可选参数，可以包含此特定请求的自定义配置选项。这些选项将覆盖在 Portkey 客户端中设置的配置。完整的配置参数列表可以在 这里 找到。

示例用法

1. 默认

聊天完成端点接受消息对象的数组，并以聊天消息格式返回完成内容。

NodeJS

import Portkey from 'portkey-ai';

// 初始化 Portkey 客户端
const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY",  // 替换为您的 Portkey API 密钥
    virtualKey: "VIRTUAL_KEY"   // 可选：用于虚拟密钥管理
});

// 生成聊天完成
async function getChatCompletion() {
    const chatCompletion = await portkey.chat.completions.create({
        messages: [{ role: 'user', content: 'Say this is a test' }],
        model: 'gpt-3.5-turbo',
    });

    console.log(chatCompletion);
}
await getChatCompletion();

Python

REST

要向本地或私有托管的模型发送聊天请求，请查看 Ollama 的指南。

2. 图像输入（视觉模型）

聊天完成 API 还支持将图像添加到视觉模型（GPT-4V、Gemini 等）的请求中。

NodeJS

// 生成带有流式传输的聊天完成
async function getChatCompletionVision(){
    const chatCompletion = await portkey.chat.completions.create({
        model: "gpt-4-vision-preview",
        messages: [
          {
            role: "user",
            content: [
              { type: "text", text: "What’s in this image?" },
              {
                type: "image_url",
                image_url:
                  "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg",
              },
            ],
          },
        ],
    });

    console.log(response.choices[0]);
}
await getChatCompletionVision();

Python

REST

3. 流式聊天完成

在请求中将 stream 参数设置为 true，以启用来自聊天完成 API 的流式响应。

聊天完成端点接受消息对象的数组，并以聊天消息格式返回完成内容。

NodeJS

// 生成带有流式传输的聊天完成
async function getChatCompletionStream(){
    const chatCompletion = await portkey.chat.completions.create({
        messages: [{ role: 'user', content: 'Say this is a test' }],
        model: 'gpt-3.5-turbo',
        stream: true
    });

    for await (const chunk of chatCompletion) {
        console.log(chunk.choices[0].delta.content);
    }
}
await getChatCompletionStream();

Python

REST

4. 函数

tools 参数接受可以专门为支持函数调用的模型发送的函数。

NodeJS

// 生成带有流式传输的聊天完成
async function getChatCompletionFunctions(){
  const messages = [{"role": "user", "content": "What's the weather like in Boston today?"}];
  const tools = [
      {
        "type": "function",
        "function": {
          "name": "get_current_weather",
          "description": "获取给定位置的当前天气",
          "parameters": {
            "type": "object",
            "properties": {
              "location": {
                "type": "string",
                "description": "城市和州，例如：旧金山，加州",
              },
              "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
            },
            "required": ["location"],
          },
        }
      }
  ];

    const response = await openai.chat.completions.create({
    model: "gpt-3.5-turbo",
    messages: messages,
    tools: tools,
    tool_choice: "auto",
  });
  
  console.log(response)

}
await getChatCompletionFunctions();

Python

REST

5. 每个请求的自定义配置

可能需要为每个请求覆盖 config 值，或者在请求中发送 trace id 和 metadata 的选项。这可以通过在请求中附加这些参数来实现。

NodeJS

// 使用配置参数生成聊天完成
async function getChatCompletionWithConfig() {
    const chatCompletion = await portkey.chat.completions.create({
        messages: [{ role: 'user', content: '说这是一个测试' }],
        model: 'gpt-3.5-turbo',
    }, {config: "sample-7g5tr4"});

    for await (const chunk of chatCompletion) {
        console.log(chunk.choices[0].delta.content);
    }
}
await getChatCompletionWithConfig();

Python

REST

响应格式

响应将遵循 Portkey API 的 Chat Completions Object 架构，通常包括生成的消息和相关元数据。