Package 'ellmer'

Description

batch_chat() and batch_chat_structured() currently only work with chat_openai(), chat_anthropic(), and chat_google_gemini(). They use the OpenAI, Anthropic, and Google Gemini batch APIs which allow you to submit multiple requests simultaneously. The results can take up to 24 hours to complete, but in return you pay 50% less than usual (but note that ellmer doesn't include this discount in its pricing metadata). If you want to get results back more quickly, or you're working with a different provider, you may want to use parallel_chat() instead.

Since batched requests can take a long time to complete, batch_chat() requires a file path that is used to store information about the batch so you never lose any work. You can either set wait = FALSE or simply interrupt the waiting process, then later, either call batch_chat() to resume where you left off or call batch_chat_completed() to see if the results are ready to retrieve. batch_chat() will store the chat responses in this file, so you can either keep it around to cache the results, or delete it to free up disk space.

This API is marked as experimental since I don't yet know how to handle errors in the most helpful way. Fortunately they don't seem to be common, but if you have ideas, please let me know!

Usage

batch_chat(chat, prompts, path, wait = TRUE, ignore_hash = FALSE)

batch_chat_text(chat, prompts, path, wait = TRUE, ignore_hash = FALSE)

batch_chat_structured(
  chat,
  prompts,
  path,
  type,
  wait = TRUE,
  ignore_hash = FALSE,
  convert = TRUE,
  include_tokens = FALSE,
  include_cost = FALSE
)

batch_chat_completed(chat, prompts, path)

Arguments

Value

For batch_chat(), a list of Chat objects, one for each prompt. For batch_chat_test(), a character vector of text responses. For batch_chat_structured(), a single structured data object with one element for each prompt. Typically, when type is an object, this will will be a data frame with one row for each prompt, and one column for each property.

For any of the aboves, will return NULL if wait = FALSE and the job is not complete.

Examples

chat <- chat_openai(model = "gpt-4.1-nano")

# Chat ----------------------------------------------------------------------

prompts <- interpolate("What do people from {{state.name}} bring to a potluck dinner?")
## Not run: 
chats <- batch_chat(chat, prompts, path = "potluck.json")
chats

## End(Not run)

# Structured data -----------------------------------------------------------
prompts <- list(
  "I go by Alex. 42 years on this planet and counting.",
  "Pleased to meet you! I'm Jamal, age 27.",
  "They call me Li Wei. Nineteen years young.",
  "Fatima here. Just celebrated my 35th birthday last week.",
  "The name's Robert - 51 years old and proud of it.",
  "Kwame here - just hit the big 5-0 this year."
)
type_person <- type_object(name = type_string(), age = type_number())
## Not run: 
data <- batch_chat_structured(
  chat = chat,
  prompts = prompts,
  path = "people-data.json",
  type = type_person
)
data

## End(Not run)

Description

This is a generic interface to all the other chat_ functions that allow to you pick the provider and the model with a simple string.

Usage

chat(
  name,
  ...,
  system_prompt = NULL,
  params = NULL,
  echo = c("none", "output", "all")
)

Arguments

Description

A Chat is a sequence of user and assistant Turns sent to a specific Provider. A Chat is a mutable R6 object that takes care of managing the state associated with the chat; i.e. it records the messages that you send to the server, and the messages that you receive back. If you register a tool (i.e. an R function that the assistant can call on your behalf), it also takes care of the tool loop.

You should generally not create this object yourself, but instead call chat_openai() or friends instead.

Value

A Chat object

Methods

Public methods

• Chat$new()

• Chat$get_turns()

• Chat$set_turns()

• Chat$add_turn()

• Chat$get_system_prompt()

• Chat$get_model()

• Chat$set_model()

• Chat$set_system_prompt()

• Chat$get_tokens()

• Chat$get_cost()

• Chat$last_turn()

• Chat$chat()

• Chat$chat_structured()

• Chat$chat_structured_async()

• Chat$chat_async()

• Chat$stream()

• Chat$stream_async()

• Chat$register_tool()

• Chat$register_tools()

• Chat$get_provider()

• Chat$get_tools()

• Chat$set_tools()

• Chat$on_tool_request()

• Chat$on_tool_result()

• Chat$clone()

Chat$new()

Usage

Chat$new(provider, system_prompt = NULL, echo = "none")

Arguments

Chat$get_turns()

Retrieve the turns that have been sent and received so far (optionally starting with the system prompt, if any).

Usage

Chat$get_turns(include_system_prompt = FALSE)

Arguments

Chat$set_turns()

Replace existing turns with a new list.

Usage

Chat$set_turns(value)

Arguments

Chat$add_turn()

Add a pair of turns to the chat.

Usage

Chat$add_turn(user, assistant, log_tokens = TRUE)

Arguments

Chat$get_system_prompt()

If set, the system prompt, it not, NULL.

Usage

Chat$get_system_prompt()

Chat$get_model()

Retrieve the model name

Usage

Chat$get_model()

Chat$set_model()

Update the model name. Note that unlike some of the ⁠chat_*()⁠ functions, the model name is not validated against available models for the provider.

Usage

Chat$set_model(model)

Arguments

Chat$set_system_prompt()

Update the system prompt

Usage

Chat$set_system_prompt(value)

Arguments

Chat$get_tokens()

A data frame with token usage and cost data. There are four columns: input, output, cached_input, and cost. There is one row for each assistant turn, because token counts and costs are only available when the API returns the assistant's response.

Usage

Chat$get_tokens(include_system_prompt = deprecated())

Arguments

Chat$get_cost()

The cost of this chat

Usage

Chat$get_cost(include = c("all", "last"))

Arguments

Chat$last_turn()

The last turn returned by the assistant.

Usage

Chat$last_turn(role = c("assistant", "user", "system"))

Arguments

Returns

Either a Turn or NULL, if no turns with the specified role have occurred.

Chat$chat()

Submit input to the chatbot, and return the response as a simple string (probably Markdown).

Usage

Chat$chat(..., echo = NULL)

Arguments

Chat$chat_structured()

Extract structured data

Usage

Chat$chat_structured(..., type, echo = "none", convert = TRUE)

Arguments

Chat$chat_structured_async()

Extract structured data, asynchronously. Returns a promise that resolves to an object matching the type specification.

Usage

Chat$chat_structured_async(..., type, echo = "none", convert = TRUE)

Arguments

Chat$chat_async()

Submit input to the chatbot, and receive a promise that resolves with the response all at once. Returns a promise that resolves to a string (probably Markdown).

Usage

Chat$chat_async(..., tool_mode = c("concurrent", "sequential"))

Arguments

Chat$stream()

Submit input to the chatbot, returning streaming results. Returns A coro generator that yields strings. While iterating, the generator will block while waiting for more content from the chatbot.

Usage

Chat$stream(..., stream = c("text", "content"), controller = NULL)

Arguments

Chat$stream_async()

Submit input to the chatbot, returning asynchronously streaming results. Returns a coro async generator that yields string promises.

Usage

Chat$stream_async(
  ...,
  tool_mode = c("concurrent", "sequential"),
  stream = c("text", "content"),
  controller = NULL
)

Arguments

Chat$register_tool()

Register a tool (an R function) that the chatbot can use. Learn more in vignette("tool-calling").

Usage

Chat$register_tool(tool)

Arguments

Chat$register_tools()

Register a list of tools. Learn more in vignette("tool-calling").

Usage

Chat$register_tools(tools)

Arguments

Chat$get_provider()

Get the underlying provider object. For expert use only.

Usage

Chat$get_provider()

Chat$get_tools()

Retrieve the list of registered tools.

Usage

Chat$get_tools()

Chat$set_tools()

Sets the available tools. For expert use only; most users should use register_tool().

Usage

Chat$set_tools(tools)

Arguments

Chat$on_tool_request()

Register a callback for a tool request event.

Usage

Chat$on_tool_request(callback)

Arguments

Returns

A function that can be called to remove the callback.

Chat$on_tool_result()

Register a callback for a tool result event.

Usage

Chat$on_tool_result(callback)

Arguments

Returns

A function that can be called to remove the callback.

Chat$clone()

The objects of this class are cloneable with this method.

Usage

Chat$clone(deep = FALSE)

Arguments

Examples

chat <- chat_openai()
chat$chat("Tell me a funny joke")

Description

Anthropic provides a number of chat based models under the Claude moniker. Note that a Claude Pro membership does not give you the ability to call models via the API; instead, you will need to sign up (and pay for) a developer account.

Usage

chat_anthropic(
  system_prompt = NULL,
  params = NULL,
  model = NULL,
  cache = c("5m", "1h", "none"),
  api_args = list(),
  base_url = "https://api.anthropic.com/v1",
  beta_headers = character(),
  api_key = NULL,
  credentials = NULL,
  api_headers = character(),
  echo = NULL
)

chat_claude(
  system_prompt = NULL,
  params = NULL,
  model = NULL,
  cache = c("5m", "1h", "none"),
  api_args = list(),
  base_url = "https://api.anthropic.com/v1",
  beta_headers = character(),
  api_key = NULL,
  credentials = NULL,
  api_headers = character(),
  echo = NULL
)

models_claude(
  base_url = "https://api.anthropic.com/v1",
  api_key = NULL,
  credentials = NULL
)

models_anthropic(
  base_url = "https://api.anthropic.com/v1",
  api_key = NULL,
  credentials = NULL
)

Arguments

Value

A Chat object.

Caching

Caching with Claude is a bit more complicated than other providers but we believe that on average it will save you both money and time, so we have enabled it by default. With other providers, like OpenAI and Google, you only pay for cache reads, which cost 10% of the normal price. With Claude, you also pay for cache writes, which cost 125% of the normal price for 5 minute caching and 200% of the normal price for 1 hour caching.

How does this affect the total cost of a conversation? Imagine the first turn sends 1000 input tokens and receives 200 output tokens. The second turn must first send both the input and output from the previous turn (1200 tokens). It then sends a further 1000 tokens and receives 200 tokens back.

To compare the prices of these two approaches we can ignore the cost of output tokens, because they are the same for both. How much will the input tokens cost? If we don't use caching, we send 1000 tokens in the first turn and 2200 (1000 + 200 + 1000) tokens in the second turn for a total of 3200 tokens. If we use caching, we'll send (the equivalent of) 1000 * 1.25 = 1250 tokens in the first turn. In the second turn, 1000 of the input tokens will be cached so the total cost is 1000 * 0.1 + (200 + 1000) * 1.25 = 1600 tokens. That makes a total of 2850 tokens, i.e. 11% fewer tokens, decreasing the overall cost.

Obviously, the details will vary from conversation to conversation, but if you have a large system prompt that you re-use many times you should expect to see larger savings. You can see exactly how many input and cache input tokens each turn uses, along with the total cost, with chat$get_tokens(). If you don't see savings for your use case, you can suppress caching with cache = "none".

I know this is already quite complicated, but there's one final wrinkle: Claude will only cache longer prompts, with caching requiring at least 1024-4096 tokens, depending on the model. So don't be surprised it if you don't see any differences with caching if you have a short prompt.

See all the details at https://docs.claude.com/en/docs/build-with-claude/prompt-caching.

See Also

Other chatbots: chat_aws_bedrock(), chat_azure_openai(), chat_cloudflare(), chat_databricks(), chat_deepseek(), chat_github(), chat_google_gemini(), chat_groq(), chat_huggingface(), chat_lmstudio(), chat_mistral(), chat_ollama(), chat_openai(), chat_openai_compatible(), chat_openrouter(), chat_perplexity(), chat_portkey()

Examples

chat <- chat_anthropic()
chat$chat("Tell me three jokes about statisticians")

Description

AWS Bedrock provides a number of language models, including those from Anthropic's Claude, using the Bedrock Converse API.

Authentication

Authentication is handled through {paws.common}, so if authentication does not work for you automatically, you'll need to follow the advice at https://www.paws-r-sdk.com/#credentials. In particular, if your org uses AWS SSO, you'll need to run ⁠aws sso login⁠ at the terminal.

Prompt caching

Bedrock supports prompt caching via cache checkpoints. When caching is enabled, ellmer places cache checkpoints on the system prompt and the last turn, so that the conversation history is cached across turns.

By default (cache = "auto"), caching is enabled for models known to support it (Anthropic Claude and Amazon Nova) and disabled for all other models. You can also set cache to "5m" or "1h" to force a specific TTL, or "none" to disable caching entirely. Note that individual models may have minimum input token thresholds before caching takes effect.

Note that token_usage() does not currently reflect the cost of writing to the cache, which is priced at a premium over regular input tokens. Cache read savings are reported correctly.

Usage

chat_aws_bedrock(
  system_prompt = NULL,
  base_url = NULL,
  model = NULL,
  profile = NULL,
  cache = c("auto", "5m", "1h", "none"),
  params = NULL,
  api_args = list(),
  api_headers = character(),
  echo = NULL
)

models_aws_bedrock(profile = NULL, base_url = NULL)

Arguments

api_args = list(
  additionalModelRequestFields = list(
    thinking = list(type = "enabled", budget_tokens = 4000)
  )
)

Value

A Chat object.

See Also

Other chatbots: chat_anthropic(), chat_azure_openai(), chat_cloudflare(), chat_databricks(), chat_deepseek(), chat_github(), chat_google_gemini(), chat_groq(), chat_huggingface(), chat_lmstudio(), chat_mistral(), chat_ollama(), chat_openai(), chat_openai_compatible(), chat_openrouter(), chat_perplexity(), chat_portkey()

Examples

## Not run: 
# Basic usage
chat <- chat_aws_bedrock()
chat$chat("Tell me three jokes about statisticians")

## End(Not run)

Description

The Azure OpenAI server hosts a number of open source models as well as proprietary models from OpenAI.

Built on top of chat_openai_compatible().

Authentication

chat_azure_openai() supports API keys and the credentials parameter, but it also makes use of:

• Azure service principals (when the AZURE_TENANT_ID, AZURE_CLIENT_ID, and AZURE_CLIENT_SECRET environment variables are set).

• Interactive Entra ID authentication, like the Azure CLI.

• Viewer-based credentials on Posit Connect. Requires the connectcreds package.

Usage

chat_azure_openai(
  endpoint = azure_endpoint(),
  model,
  params = NULL,
  api_version = NULL,
  system_prompt = NULL,
  api_key = NULL,
  credentials = NULL,
  api_args = list(),
  echo = c("none", "output", "all"),
  api_headers = character(),
  deployment_id = deprecated()
)

Arguments

Value

A Chat object.

See Also

Other chatbots: chat_anthropic(), chat_aws_bedrock(), chat_cloudflare(), chat_databricks(), chat_deepseek(), chat_github(), chat_google_gemini(), chat_groq(), chat_huggingface(), chat_lmstudio(), chat_mistral(), chat_ollama(), chat_openai(), chat_openai_compatible(), chat_openrouter(), chat_perplexity(), chat_portkey()

Examples

## Not run: 
chat <- chat_azure_openai(model = "gpt-4o-mini")
chat$chat("Tell me three jokes about statisticians")

## End(Not run)

Description

Cloudflare Workers AI hosts a variety of open-source AI models. To use the Cloudflare API, you must have an Account ID and an Access Token, which you can obtain by following these instructions.

Built on top of chat_openai_compatible().

Known limitations

• Tool calling does not appear to work.

• Images don't appear to work.

Usage

chat_cloudflare(
  account = cloudflare_account(),
  system_prompt = NULL,
  params = NULL,
  api_key = NULL,
  credentials = NULL,
  model = NULL,
  api_args = list(),
  echo = NULL,
  api_headers = character()
)

Arguments

Value

A Chat object.

See Also

Other chatbots: chat_anthropic(), chat_aws_bedrock(), chat_azure_openai(), chat_databricks(), chat_deepseek(), chat_github(), chat_google_gemini(), chat_groq(), chat_huggingface(), chat_lmstudio(), chat_mistral(), chat_ollama(), chat_openai(), chat_openai_compatible(), chat_openrouter(), chat_perplexity(), chat_portkey()

Examples

## Not run: 
chat <- chat_cloudflare()
chat$chat("Tell me three jokes about statisticians")

## End(Not run)

Description

Databricks provides out-of-the-box access to a number of foundation models and can also serve as a gateway for external models hosted by a third party.

Built on top of chat_openai_compatible().

Authentication

chat_databricks() picks up on ambient Databricks credentials for a subset of the Databricks client unified authentication model. Specifically, it supports:

• Personal access tokens

• Service principals via OAuth (OAuth M2M)

• User account via OAuth (OAuth U2M)

• Authentication via the Databricks CLI

• Posit Workbench-managed credentials

• Viewer-based credentials on Posit Connect. Requires the connectcreds package.

Usage

chat_databricks(
  workspace = databricks_workspace(),
  system_prompt = NULL,
  model = NULL,
  token = NULL,
  params = NULL,
  api_args = list(),
  echo = c("none", "output", "all"),
  api_headers = character()
)

Arguments

Value

A Chat object.

See Also

Other chatbots: chat_anthropic(), chat_aws_bedrock(), chat_azure_openai(), chat_cloudflare(), chat_deepseek(), chat_github(), chat_google_gemini(), chat_groq(), chat_huggingface(), chat_lmstudio(), chat_mistral(), chat_ollama(), chat_openai(), chat_openai_compatible(), chat_openrouter(), chat_perplexity(), chat_portkey()

Examples

## Not run: 
chat <- chat_databricks()
chat$chat("Tell me three jokes about statisticians")

## End(Not run)

Description

Sign up at https://platform.deepseek.com.

Built on top of chat_openai_compatible().

Known limitations

• Structured data extraction is not supported.

• Images are not supported.

Usage

chat_deepseek(
  system_prompt = NULL,
  base_url = "https://api.deepseek.com",
  api_key = NULL,
  credentials = NULL,
  model = NULL,
  params = NULL,
  api_args = list(),
  echo = NULL,
  api_headers = character()
)

models_deepseek(
  base_url = "https://api.deepseek.com",
  api_key = NULL,
  credentials = NULL
)

Arguments

Value

A Chat object.

See Also

Other chatbots: chat_anthropic(), chat_aws_bedrock(), chat_azure_openai(), chat_cloudflare(), chat_databricks(), chat_github(), chat_google_gemini(), chat_groq(), chat_huggingface(), chat_lmstudio(), chat_mistral(), chat_ollama(), chat_openai(), chat_openai_compatible(), chat_openrouter(), chat_perplexity(), chat_portkey()

Examples

## Not run: 
chat <- chat_deepseek()
chat$chat("Tell me three jokes about statisticians")

## End(Not run)

Description

GitHub Models hosts a number of open source and OpenAI models. To access the GitHub model marketplace, you will need to apply for and be accepted into the beta access program. See https://github.com/marketplace/models for details.

The defaults are tweaked for the GitHub Models marketplace.

GitHub also supports the Azure AI Inference SDK, which you can use by setting base_url to "https://models.inference.ai.azure.com/". This endpoint was used in ellmer v0.3.0 and earlier.

Usage

chat_github(
  system_prompt = NULL,
  base_url = "https://models.github.ai/inference/",
  api_key = NULL,
  credentials = NULL,
  model = NULL,
  params = NULL,
  api_args = list(),
  echo = NULL,
  api_headers = character()
)

models_github(
  base_url = "https://models.github.ai/",
  api_key = NULL,
  credentials = NULL
)

Arguments

Value

A Chat object.

See Also

Other chatbots: chat_anthropic(), chat_aws_bedrock(), chat_azure_openai(), chat_cloudflare(), chat_databricks(), chat_deepseek(), chat_google_gemini(), chat_groq(), chat_huggingface(), chat_lmstudio(), chat_mistral(), chat_ollama(), chat_openai(), chat_openai_compatible(), chat_openrouter(), chat_perplexity(), chat_portkey()

Examples

## Not run: 
chat <- chat_github()
chat$chat("Tell me three jokes about statisticians")

## End(Not run)

Description

Google's AI offering is broken up into two parts: Gemini and Vertex AI. Most enterprises are likely to use Vertex AI, and individuals are likely to use Gemini.

Use google_upload() to upload files (PDFs, images, video, audio, etc.)

Authentication

These functions try a number of authentication strategies, in this order:

• An API key set in the GOOGLE_API_KEY env var, or, for chat_google_gemini() only, GEMINI_API_KEY.

• Google's default application credentials, if the gargle package is installed.

• Viewer-based credentials on Posit Connect, if the connectcreds package.

• . An browser-based OAuth flow, if you're in an interactive session. This currently uses an unverified OAuth app (so you will get a scary warning); we plan to verify in the near future.

Usage

chat_google_gemini(
  system_prompt = NULL,
  base_url = "https://generativelanguage.googleapis.com/v1beta/",
  api_key = NULL,
  credentials = NULL,
  model = NULL,
  params = NULL,
  api_args = list(),
  api_headers = character(),
  echo = NULL
)

chat_google_vertex(
  location,
  project_id,
  system_prompt = NULL,
  model = NULL,
  params = NULL,
  api_args = list(),
  api_headers = character(),
  echo = NULL
)

models_google_gemini(
  base_url = "https://generativelanguage.googleapis.com/v1beta/",
  api_key = NULL,
  credentials = NULL
)

models_google_vertex(location, project_id, credentials = NULL)

Arguments

Value

A Chat object.

See Also

Other chatbots: chat_anthropic(), chat_aws_bedrock(), chat_azure_openai(), chat_cloudflare(), chat_databricks(), chat_deepseek(), chat_github(), chat_groq(), chat_huggingface(), chat_lmstudio(), chat_mistral(), chat_ollama(), chat_openai(), chat_openai_compatible(), chat_openrouter(), chat_perplexity(), chat_portkey()

Examples

## Not run: 
chat <- chat_google_gemini()
chat$chat("Tell me three jokes about statisticians")

## End(Not run)

Description

Sign up at https://groq.com.

Built on top of chat_openai_compatible().

Usage

chat_groq(
  system_prompt = NULL,
  base_url = "https://api.groq.com/openai/v1",
  api_key = NULL,
  credentials = NULL,
  model = NULL,
  params = NULL,
  api_args = list(),
  echo = NULL,
  api_headers = character()
)

Arguments

Value

A Chat object.

See Also

Other chatbots: chat_anthropic(), chat_aws_bedrock(), chat_azure_openai(), chat_cloudflare(), chat_databricks(), chat_deepseek(), chat_github(), chat_google_gemini(), chat_huggingface(), chat_lmstudio(), chat_mistral(), chat_ollama(), chat_openai(), chat_openai_compatible(), chat_openrouter(), chat_perplexity(), chat_portkey()

Examples

## Not run: 
chat <- chat_groq()
chat$chat("Tell me three jokes about statisticians")

## End(Not run)

Description

Hugging Face hosts a variety of open-source and proprietary AI models available via their Inference API. To use the Hugging Face API, you must have an Access Token, which you can obtain from your Hugging Face account (ensure that at least "Make calls to Inference Providers" and "Make calls to your Inference Endpoints" is checked).

Built on top of chat_openai_compatible().

Known limitations

• Some models do not support the chat interface or parts of it, for example ⁠google/gemma-2-2b-it⁠ does not support a system prompt. You will need to carefully choose the model.

Usage

chat_huggingface(
  system_prompt = NULL,
  params = NULL,
  api_key = NULL,
  credentials = NULL,
  model = NULL,
  api_args = list(),
  echo = NULL,
  api_headers = character()
)

Arguments

Value

A Chat object.

See Also

Other chatbots: chat_anthropic(), chat_aws_bedrock(), chat_azure_openai(), chat_cloudflare(), chat_databricks(), chat_deepseek(), chat_github(), chat_google_gemini(), chat_groq(), chat_lmstudio(), chat_mistral(), chat_ollama(), chat_openai(), chat_openai_compatible(), chat_openrouter(), chat_perplexity(), chat_portkey()

Examples

## Not run: 
chat <- chat_huggingface()
chat$chat("Tell me three jokes about statisticians")

## End(Not run)

Description

To use chat_lmstudio() first download and install LM Studio. Then load a model using the LM Studio GUI and start the local server. To learn more about running LM Studio locally, see https://lmstudio.ai/docs/developer/core/server/.

Built on top of chat_openai_compatible().

Usage

chat_lmstudio(
  system_prompt = NULL,
  base_url = Sys.getenv("LMSTUDIO_BASE_URL", "http://localhost:1234"),
  model,
  params = NULL,
  api_args = list(),
  echo = NULL,
  credentials = NULL,
  api_headers = character()
)

models_lmstudio(base_url = "http://localhost:1234", credentials = NULL)

Arguments

Value

A Chat object.

See Also

Other chatbots: chat_anthropic(), chat_aws_bedrock(), chat_azure_openai(), chat_cloudflare(), chat_databricks(), chat_deepseek(), chat_github(), chat_google_gemini(), chat_groq(), chat_huggingface(), chat_mistral(), chat_ollama(), chat_openai(), chat_openai_compatible(), chat_openrouter(), chat_perplexity(), chat_portkey()

Examples

## Not run: 
# https://lmstudio.ai/models/zai-org/glm-4.7-flash
chat <- chat_lmstudio(model = "zai-org/glm-4.7-flash")
chat$chat("Tell me three jokes about statisticians")

## End(Not run)

Description

Get your API key from https://console.mistral.ai/api-keys.

Built on top of chat_openai_compatible().

Known limitations

• Tool calling is unstable.

• Images require a model that supports images.

Usage

chat_mistral(
  system_prompt = NULL,
  params = NULL,
  api_key = NULL,
  credentials = NULL,
  model = NULL,
  api_args = list(),
  echo = NULL,
  api_headers = character()
)

models_mistral(api_key = mistral_key())

Arguments

Value

A Chat object.

See Also

Other chatbots: chat_anthropic(), chat_aws_bedrock(), chat_azure_openai(), chat_cloudflare(), chat_databricks(), chat_deepseek(), chat_github(), chat_google_gemini(), chat_groq(), chat_huggingface(), chat_lmstudio(), chat_ollama(), chat_openai(), chat_openai_compatible(), chat_openrouter(), chat_perplexity(), chat_portkey()

Examples

## Not run: 
chat <- chat_mistral()
chat$chat("Tell me three jokes about statisticians")

## End(Not run)

Description

To use chat_ollama() first download and install Ollama. Then install some models either from the command line (e.g. with ⁠ollama pull llama3.1⁠) or within R using ollamar (e.g. ollamar::pull("llama3.1")).

Built on top of chat_openai_compatible().

Thinking models

Some Ollama models (e.g. qwen3) support extended reasoning or "thinking". When using these models, thinking content is automatically captured in the turn. You can control thinking with reasoning_effort:

chat <- chat_ollama(
  model = "qwen3:4b",
  params = params(reasoning_effort = "none")
)

Which values are supported depends on the model. For example, qwen3 only supports "none" (off) vs the default (on), while gpt-oss supports "low", "medium", and "high" but ignores "none". See https://docs.ollama.com/capabilities/thinking for details.

Known limitations

• Tool calling is not supported with streaming (i.e. when echo is "text" or "all")

• Models can only use 2048 input tokens, and there's no way to get them to use more, except by creating a custom model with a different default.

• Tool calling generally seems quite weak, at least with the models I have tried it with.

Usage

chat_ollama(
  system_prompt = NULL,
  base_url = Sys.getenv("OLLAMA_BASE_URL", "http://localhost:11434"),
  model,
  params = NULL,
  api_args = list(),
  echo = NULL,
  api_key = NULL,
  credentials = NULL,
  api_headers = character()
)

models_ollama(base_url = "http://localhost:11434", credentials = NULL)

Arguments

Value

A Chat object.

See Also

Other chatbots: chat_anthropic(), chat_aws_bedrock(), chat_azure_openai(), chat_cloudflare(), chat_databricks(), chat_deepseek(), chat_github(), chat_google_gemini(), chat_groq(), chat_huggingface(), chat_lmstudio(), chat_mistral(), chat_openai(), chat_openai_compatible(), chat_openrouter(), chat_perplexity(), chat_portkey()

Examples

## Not run: 
chat <- chat_ollama(model = "llama3.2")
chat$chat("Tell me three jokes about statisticians")

## End(Not run)

Description

This is the main interface to OpenAI's models, using the responses API. You can use this to access OpenAI's latest models and features like image generation and web search. If you need to use an OpenAI-compatible API from another provider, or the chat completions API with OpenAI,use chat_openai_compatible() instead.

Note that a ChatGPT Plus membership does not grant access to the API. You will need to sign up for a developer account (and pay for it) at the developer platform.

Usage

chat_openai(
  system_prompt = NULL,
  base_url = "https://api.openai.com/v1",
  api_key = NULL,
  credentials = NULL,
  model = NULL,
  params = NULL,
  api_args = list(),
  api_headers = character(),
  service_tier = c("auto", "default", "flex", "priority"),
  echo = c("none", "output", "all")
)

models_openai(
  base_url = "https://api.openai.com/v1",
  api_key = NULL,
  credentials = NULL
)

Arguments

Value

A Chat object.

See Also

Other chatbots: chat_anthropic(), chat_aws_bedrock(), chat_azure_openai(), chat_cloudflare(), chat_databricks(), chat_deepseek(), chat_github(), chat_google_gemini(), chat_groq(), chat_huggingface(), chat_lmstudio(), chat_mistral(), chat_ollama(), chat_openai_compatible(), chat_openrouter(), chat_perplexity(), chat_portkey()

Examples

chat <- chat_openai()
chat$chat("
  What is the difference between a tibble and a data frame?
  Answer with a bulleted list
")

chat$chat("Tell me three funny jokes about statisticians")

Description

This function is for use with OpenAI-compatible APIs, also known as the chat completions API. If you want to use OpenAI itself, we recommend chat_openai(), which uses the newer responses API.

Many providers offer OpenAI-compatible APIs, including:

• Ollama for local models

• vLLM for self-hosted models

• Various cloud providers with OpenAI-compatible endpoints

Usage

chat_openai_compatible(
  base_url,
  name = "OpenAI-compatible",
  system_prompt = NULL,
  api_key = NULL,
  credentials = NULL,
  model = NULL,
  params = NULL,
  api_args = list(),
  api_headers = character(),
  preserve_thinking = FALSE,
  echo = c("none", "output", "all")
)

Arguments

Value

A Chat object.

See Also

Other chatbots: chat_anthropic(), chat_aws_bedrock(), chat_azure_openai(), chat_cloudflare(), chat_databricks(), chat_deepseek(), chat_github(), chat_google_gemini(), chat_groq(), chat_huggingface(), chat_lmstudio(), chat_mistral(), chat_ollama(), chat_openai(), chat_openrouter(), chat_perplexity(), chat_portkey()

Examples

## Not run: 
# Example with Ollama (requires Ollama running locally)
chat <- chat_openai_compatible(
  base_url = "http://localhost:11434/v1",
  model = "llama2"
)
chat$chat("What is the difference between a tibble and a data frame?")

## End(Not run)

Description

Sign up at https://openrouter.ai.

Support for features depends on the underlying model that you use; see https://openrouter.ai/models for details.

Usage

chat_openrouter(
  system_prompt = NULL,
  api_key = NULL,
  credentials = NULL,
  model = NULL,
  params = NULL,
  api_args = list(),
  echo = c("none", "output", "all"),
  api_headers = character()
)

Arguments

Value

A Chat object.

See Also

Other chatbots: chat_anthropic(), chat_aws_bedrock(), chat_azure_openai(), chat_cloudflare(), chat_databricks(), chat_deepseek(), chat_github(), chat_google_gemini(), chat_groq(), chat_huggingface(), chat_lmstudio(), chat_mistral(), chat_ollama(), chat_openai(), chat_openai_compatible(), chat_perplexity(), chat_portkey()

Examples

## Not run: 
chat <- chat_openrouter()
chat$chat("Tell me three jokes about statisticians")

## End(Not run)

Description

Sign up at https://www.perplexity.ai.

Perplexity AI is a platform for running LLMs that are capable of searching the web in real-time to help them answer questions with information that may not have been available when the model was trained.

This function is a Uses OpenAI compatible API via chat_openai_compatible() with the defaults tweaked for Perplexity AI.

Usage

chat_perplexity(
  system_prompt = NULL,
  base_url = "https://api.perplexity.ai/",
  api_key = NULL,
  credentials = NULL,
  model = NULL,
  params = NULL,
  api_args = list(),
  echo = NULL,
  api_headers = character()
)

Arguments

Value

A Chat object.

See Also

Other chatbots: chat_anthropic(), chat_aws_bedrock(), chat_azure_openai(), chat_cloudflare(), chat_databricks(), chat_deepseek(), chat_github(), chat_google_gemini(), chat_groq(), chat_huggingface(), chat_lmstudio(), chat_mistral(), chat_ollama(), chat_openai(), chat_openai_compatible(), chat_openrouter(), chat_portkey()

Examples

## Not run: 
chat <- chat_perplexity()
chat$chat("Tell me three jokes about statisticians")

## End(Not run)

Description

PortkeyAI provides an interface (AI Gateway) to connect through its Universal API to a variety of LLMs providers via a single endpoint.

Usage

chat_portkey(
  model,
  system_prompt = NULL,
  base_url = "https://api.portkey.ai/v1",
  api_key = NULL,
  credentials = NULL,
  virtual_key = deprecated(),
  params = NULL,
  api_args = list(),
  echo = NULL,
  api_headers = character()
)

models_portkey(base_url = "https://api.portkey.ai/v1", api_key = portkey_key())

Arguments

Value

A Chat object.

See Also

Other chatbots: chat_anthropic(), chat_aws_bedrock(), chat_azure_openai(), chat_cloudflare(), chat_databricks(), chat_deepseek(), chat_github(), chat_google_gemini(), chat_groq(), chat_huggingface(), chat_lmstudio(), chat_mistral(), chat_ollama(), chat_openai(), chat_openai_compatible(), chat_openrouter(), chat_perplexity()

Examples

## Not run: 
chat <- chat_portkey()
chat$chat("Tell me three jokes about statisticians")

## End(Not run)

Description

The Snowflake provider allows you to interact with LLM models available through the Cortex LLM REST API.

Authentication

chat_snowflake() picks up the following ambient Snowflake credentials:

• A static OAuth token defined via the SNOWFLAKE_TOKEN environment variable.

• Key-pair authentication credentials defined via the SNOWFLAKE_USER and SNOWFLAKE_PRIVATE_KEY (which can be a PEM-encoded private key or a path to one) environment variables.

• Posit Workbench-managed Snowflake credentials for the corresponding account.

• Viewer-based credentials on Posit Connect. Requires the connectcreds package.

Known limitations

Note that Snowflake-hosted models do not support images.

Usage

chat_snowflake(
  system_prompt = NULL,
  account = snowflake_account(),
  credentials = NULL,
  model = NULL,
  params = NULL,
  api_args = list(),
  echo = c("none", "output", "all"),
  api_headers = character()
)

Arguments

Value

A Chat object.

Examples

chat <- chat_snowflake()
chat$chat("Tell me a joke in the form of a SQL query.")

Description

vLLM is an open source library that provides an efficient and convenient LLMs model server. You can use chat_vllm() to connect to endpoints powered by vLLM.

Uses OpenAI compatible API via chat_openai_compatible().

Usage

chat_vllm(
  base_url,
  system_prompt = NULL,
  model,
  params = NULL,
  api_args = list(),
  api_key = NULL,
  credentials = NULL,
  echo = NULL,
  api_headers = character()
)

models_vllm(base_url, api_key = NULL, credentials = NULL)

Arguments

Value

A Chat object.

Examples

## Not run: 
chat <- chat_vllm("http://my-vllm.com")
chat$chat("Tell me three jokes about statisticians")

## End(Not run)

Description

Use the beta Files API to upload files to and manage files in Claude. This is currently experimental because the API is in beta and may change. Note that you need beta-headers = "files-api-2025-04-14" to use the API.

Claude offers 100GB of file storage per organization, with each file having a maximum size of 500MB. For more details see https://docs.claude.com/en/docs/build-with-claude/files

• claude_file_upload() uploads a file and returns an object that you can use in chat.

• claude_file_list() lists all uploaded files.

• claude_file_get() returns an object for an previously uploaded file.

• claude_file_download() downloads the file with the given ID. Note that you can only download files created by skills or the code execution tool.

• claude_file_delete() deletes the file with the given ID.

Usage

claude_file_upload(
  path,
  base_url = "https://api.anthropic.com/v1/",
  beta_headers = "files-api-2025-04-14",
  credentials = NULL
)

claude_file_list(
  base_url = "https://api.anthropic.com/v1/",
  credentials = NULL,
  beta_headers = "files-api-2025-04-14"
)

claude_file_get(
  file_id,
  base_url = "https://api.anthropic.com/v1/",
  credentials = NULL,
  beta_headers = "files-api-2025-04-14"
)

claude_file_download(
  file_id,
  path,
  base_url = "https://api.anthropic.com/v1/",
  credentials = NULL,
  beta_headers = "files-api-2025-04-14"
)

claude_file_delete(
  file_id,
  base_url = "https://api.anthropic.com/v1/",
  credentials = NULL,
  beta_headers = "files-api-2025-04-14"
)

Arguments

Examples

## Not run: 
file <- claude_file_upload("path/to/file.pdf")
chat <- chat_anthropic(beta_headers = "files-api-2025-04-14")
chat$chat("Please summarize the document.", file)

## End(Not run)

Description

Enables Claude to fetch and analyze content from web URLs. Claude can only fetch URLs that appear in the conversation context (user messages or previous tool results). For security reasons, Claude cannot dynamically construct URLs to fetch.

Requires the web-fetch-2025-09-10 beta header. Learn more in https://docs.claude.com/en/docs/agents-and-tools/tool-use/web-fetch-tool.

Usage

claude_tool_web_fetch(
  max_uses = NULL,
  allowed_domains = NULL,
  blocked_domains = NULL,
  citations = FALSE,
  max_content_tokens = NULL
)

Arguments

See Also

Other built-in tools: claude_tool_web_search(), google_tool_web_fetch(), google_tool_web_search(), openai_tool_web_search()

Examples

## Not run: 
chat <- chat_claude(beta_headers = "web-fetch-2025-09-10")
chat$register_tool(claude_tool_web_fetch())
chat$chat("What are the latest package releases on https://tidyverse.org/blog")

## End(Not run)

Description

Enables Claude to search the web for up-to-date information. Your organization administrator must enable web search in the Anthropic Console before using this tool, as it costs extra ($10 per 1,000 tokens at time of writing).

Learn more in https://docs.claude.com/en/docs/agents-and-tools/tool-use/web-search-tool.

Usage

claude_tool_web_search(
  max_uses = NULL,
  allowed_domains = NULL,
  blocked_domains = NULL,
  user_location = NULL
)

Arguments

See Also

Other built-in tools: claude_tool_web_fetch(), google_tool_web_fetch(), google_tool_web_search(), openai_tool_web_search()

Examples

## Not run: 
chat <- chat_claude()
chat$register_tool(claude_tool_web_search())
chat$chat("What was in the news today?")
chat$chat("What's the biggest news in the economy?")

## End(Not run)

Description

Use these functions if you're writing a package that extends ellmer and need to customise methods for various types of content. For normal use, see content_image_url() and friends.

ellmer abstracts away differences in the way that different Providers represent various types of content, allowing you to more easily write code that works with any chatbot. This set of classes represents types of content that can be either sent to and received from a provider:

• ContentText: simple text (often in markdown format). This is the only type of content that can be streamed live as it's received.

• ContentImageRemote and ContentImageInline: images, either as a pointer to a remote URL or included inline in the object. See content_image_file() and friends for convenient ways to construct these objects.

• ContentToolRequest: a request to perform a tool call (sent by the assistant).

• ContentToolResult: the result of calling the tool (sent by the user). This object is automatically created from the value returned by calling the tool() function. Alternatively, expert users can return a ContentToolResult from a tool() function to include additional data or to customize the display of the result.

Usage

Content()

ContentText(text = stop("Required"))

ContentImage()

ContentImageRemote(url = stop("Required"), detail = "")

ContentImageInline(type = stop("Required"), data = NULL)

ContentToolRequest(
  id = stop("Required"),
  name = stop("Required"),
  arguments = list(),
  tool = NULL,
  extra = list()
)

ContentToolResult(value = NULL, error = NULL, extra = list(), request = NULL)

ContentThinking(thinking = stop("Required"), extra = list())

ContentPDF(
  type = stop("Required"),
  data = stop("Required"),
  filename = stop("Required")
)

Arguments

Value

S7 objects that all inherit from Content

Examples

Content()
ContentText("Tell me a joke")
ContentImageRemote("https://www.r-project.org/Rlogo.png")
ContentToolRequest(id = "abc", name = "mean", arguments = list(x = 1:5))

Description

These functions are used to prepare image URLs and files for input to the chatbot. The content_image_url() function is used to provide a URL to an image, while content_image_file() is used to provide the image data itself.

Usage

content_image_url(url, detail = c("auto", "low", "high"))

content_image_file(path, content_type = "auto", resize = "low")

content_image_plot(width = 768, height = 768)

Arguments

Value

An input object suitable for including in the ... parameter of the chat(), stream(), chat_async(), or stream_async() methods.

Examples

## Not run: 
chat <- chat_openai()
chat$chat(
  "What do you see in these images?",
  content_image_url("https://www.r-project.org/Rlogo.png"),
  content_image_file(system.file("httr2.png", package = "ellmer"))
)

plot(waiting ~ eruptions, data = faithful)
chat <- chat_openai()
chat$chat(
  "Describe this plot in one paragraph, as suitable for inclusion in
   alt-text. You should briefly describe the plot type, the axes, and
   2-5 major visual patterns.",
   content_image_plot()
)

## End(Not run)

Description

These functions are used to prepare PDFs as input to the chatbot. The content_pdf_url() function is used to provide a URL to an PDF file, while content_pdf_file() is used to for local PDF files.

Not all providers support PDF input, so check the documentation for the provider you are using.

Usage

content_pdf_file(path)

content_pdf_url(url)

Arguments

Value

A ContentPDF object

Description

These generic functions can be use to convert Turn contents or Content objects into textual representations.

• contents_text() is the most minimal and only includes ContentText objects in the output.

• contents_markdown() returns the text content (which it assumes to be markdown and does not convert it) plus markdown representations of images and other content types.

• contents_html() returns the text content, converted from markdown to HTML with commonmark::markdown_html(), plus HTML representations of images and other content types.

These content types will continue to grow and change as ellmer evolves to support more providers and as providers add more content types.

Usage

contents_text(content, ...)

contents_html(content, ...)

contents_markdown(content, ...)

Arguments

Value

A string of text, markdown or HTML.

Examples

turns <- list(
  UserTurn(list(
    ContentText("What's this image?"),
    content_image_url("https://placehold.co/200x200")
  )),
  AssistantTurn("It's a placeholder image.")
)

lapply(turns, contents_text)
lapply(turns, contents_markdown)
if (rlang::is_installed("commonmark")) {
  contents_html(turns[[1]])
}

Description

In order to use a function as a tool in a chat, you need to craft the right call to tool(). This function helps you do that for documented functions by extracting the function's R documentation and using an LLM to generate the tool() call. It's meant to be used interactively while writing your code, not as part of your final code.

If the function has package documentation, that will be used. Otherwise, if the source code of the function can be automatically detected, then the comments immediately preceding the function are used (especially helpful if those are roxygen2 comments). If neither are available, then just the function signature is used.

Note that this function is inherently imperfect. It can't handle all possible R functions, because not all parameters are suitable for use in a tool call (for example, because they're not serializable to simple JSON objects). The documentation might not specify the expected shape of arguments to the level of detail that would allow an exact JSON schema to be generated. Please be sure to review the generated code before using it!

Usage

create_tool_def(topic, chat = NULL, echo = interactive(), verbose = FALSE)

Arguments

Value

A register_tool call that you can copy and paste into your code. Returned invisibly if echo is TRUE.

Examples

## Not run: 
  # These are all equivalent
  create_tool_def(rnorm)
  create_tool_def(stats::rnorm)
  create_tool_def("rnorm")
  create_tool_def("rnorm", chat = chat_azure_openai())

## End(Not run)

Description

df_schema() gives a column-by-column description of a data frame. For each column, it gives the name, type, label (if present), and number of missing values. For numeric and date/time columns, it also gives the range. For character and factor columns, it also gives the number of unique values, and if there's only a few (<= 10), their values.

The goal is to give the LLM a sense of the structure of the data, so that it can generate useful code, and the output attempts to balance between conciseness and accuracy.

Usage

df_schema(df, max_cols = 50)

Arguments

Examples

df_schema(mtcars)
df_schema(iris)

Description

When this tool is enabled, you can include URLs directly in your prompts and Gemini will fetch and analyze the content.

Learn more in https://ai.google.dev/gemini-api/docs/url-context.

Usage

google_tool_web_fetch()

See Also

Other built-in tools: claude_tool_web_fetch(), claude_tool_web_search(), google_tool_web_search(), openai_tool_web_search()

Examples

## Not run: 
chat <- chat_google_gemini()
chat$register_tool(google_tool_web_fetch())
chat$chat("What are the latest package releases on https://tidyverse.org/blog?")

## End(Not run)

Description

Enables Gemini models to search the web for up-to-date information and ground responses with citations to sources. The model automatically decides when (and how) to search the web based on your prompt. Search results are incorporated into the response with grounding metadata including source URLs and titles.

Learn more in https://ai.google.dev/gemini-api/docs/google-search.

Usage

google_tool_web_search()

See Also

Other built-in tools: claude_tool_web_fetch(), claude_tool_web_search(), google_tool_web_fetch(), openai_tool_web_search()

Examples

## Not run: 
chat <- chat_google_gemini()
chat$register_tool(google_tool_web_search())
chat$chat("What was in the news today?")
chat$chat("What's the biggest news in the economy?")

## End(Not run)

Description

This function uploads a file then waits for Gemini to finish processing it so that you can immediately use it in a prompt. It's experimental because it's currently Gemini specific, and we expect other providers to evolve similar feature in the future.

Uploaded files are automatically deleted after 2 days. Each file must be less than 2 GB and you can upload a total of 20 GB. ellmer doesn't currently provide a way to delete files early; please file an issue if this would be useful for you.

Usage

google_upload(
  path,
  base_url = "https://generativelanguage.googleapis.com/",
  api_key = NULL,
  credentials = NULL,
  mime_type = NULL
)

Arguments

Value

A ⁠<ContentUploaded>⁠ object that can be passed to ⁠$chat()⁠.

Examples

## Not run: 
file <- google_upload("path/to/file.pdf")

chat <- chat_google_gemini()
chat$chat(file, "Give me a three paragraph summary of this PDF")

## End(Not run)

Description

These functions are lightweight wrappers around glue that make it easier to interpolate dynamic data into a static prompt:

• interpolate() works with a string.

• interpolate_file() works with a file.

• interpolate_package() works with a file in the inst/prompts directory of a package.

Compared to glue, dynamic values should be wrapped in {{ }}, making it easier to include R code and JSON in your prompt.

Usage

interpolate(prompt, ..., .envir = parent.frame())

interpolate_file(path, ..., .envir = parent.frame())

interpolate_package(package, path, ..., .envir = parent.frame())

Arguments

Value

A {glue} string.

Examples

joke <- "You're a cool dude who loves to make jokes. Tell me a joke about {{topic}}."

# You can supply valuese directly:
interpolate(joke, topic = "bananas")

# Or allow interpolate to find them in the current environment:
topic <- "applies"
interpolate(joke)

Description

• live_console() lets you chat interactively in the console.

• live_browser() lets you chat interactively in a browser.

Note that these functions will mutate the input chat object as you chat because your turns will be appended to the history.

Usage

live_console(chat, quiet = FALSE)

live_browser(chat, quiet = FALSE)

Arguments

Value

(Invisibly) The input chat.

Examples

## Not run: 
chat <- chat_anthropic()
live_console(chat)
live_browser(chat)

## End(Not run)

Description

Enables OpenAI models to search the web for up-to-date information. The search behavior varies by model: non-reasoning models perform simple searches, while reasoning models can perform agentic, iterative searches.

Learn more at https://platform.openai.com/docs/guides/tools-web-search

Usage

openai_tool_web_search(
  allowed_domains = NULL,
  user_location = NULL,
  external_web_access = TRUE
)

Arguments

See Also

Other built-in tools: claude_tool_web_fetch(), claude_tool_web_search(), google_tool_web_fetch(), google_tool_web_search()

Examples

## Not run: 
chat <- chat_openai()
chat$register_tool(openai_tool_web_search())
chat$chat("Very briefly summarise the top 3 news stories of the day")
chat$chat("Of those stories, which one do you think was the most interesting?")

## End(Not run)

Description

If you have multiple prompts, you can submit them in parallel. This is typically considerably faster than submitting them in sequence, especially with Gemini and OpenAI.

If you're using chat_openai() or chat_anthropic() and you're willing to wait longer, you might want to use batch_chat() instead, as it comes with a 50% discount in return for taking up to 24 hours.

Usage

parallel_chat(
  chat,
  prompts,
  max_active = 10,
  rpm = 500,
  on_error = c("return", "continue", "stop")
)

parallel_chat_text(
  chat,
  prompts,
  max_active = 10,
  rpm = 500,
  on_error = c("return", "continue", "stop")
)

parallel_chat_structured(
  chat,
  prompts,
  type,
  convert = TRUE,
  include_tokens = FALSE,
  include_cost = FALSE,
  max_active = 10,
  rpm = 500,
  on_error = c("return", "continue", "stop")
)

Arguments

Value

For parallel_chat(), a list with one element for each prompt. Each element is either a Chat object (if successful), a NULL (if the request wasn't performed) or an error object (if it failed).

For parallel_chat_text(), a character vector with one element for each prompt. Requests that weren't succesful get an NA.

For parallel_chat_structured(), a single structured data object with one element for each prompt. Typically, when type is an object, this will be a tibble with one row for each prompt, and one column for each property. If the output is a data frame, and some requests error, an .error column will be added with the error objects.

Examples

chat <- chat_openai()

# Chat ----------------------------------------------------------------------
country <- c("Canada", "New Zealand", "Jamaica", "United States")
prompts <- interpolate("What's the capital of {{country}}?")
parallel_chat(chat, prompts)

# Structured data -----------------------------------------------------------
prompts <- list(
  "I go by Alex. 42 years on this planet and counting.",
  "Pleased to meet you! I'm Jamal, age 27.",
  "They call me Li Wei. Nineteen years young.",
  "Fatima here. Just celebrated my 35th birthday last week.",
  "The name's Robert - 51 years old and proud of it.",
  "Kwame here - just hit the big 5-0 this year."
)
type_person <- type_object(name = type_string(), age = type_number())
parallel_chat_structured(chat, prompts, type_person)

Description

This helper function makes it easier to create a list of parameters used across many models. The parameter names are automatically standardised and included in the correctly place in the API call.

Note that parameters that are not supported by a given provider will generate a warning, not an error. This allows you to use the same set of parameters across multiple providers.

Usage

params(
  temperature = NULL,
  top_p = NULL,
  top_k = NULL,
  frequency_penalty = NULL,
  presence_penalty = NULL,
  seed = NULL,
  max_tokens = NULL,
  log_probs = NULL,
  stop_sequences = NULL,
  reasoning_effort = NULL,
  reasoning_tokens = NULL,
  ...
)

Arguments

Description

A Provider captures the details of one chatbot service/API. This captures how the API works, not the details of the underlying large language model. Different providers might offer the same (open source) model behind a different API.

Usage

Provider(
  name = stop("Required"),
  model = stop("Required"),
  base_url = stop("Required"),
  params = list(),
  extra_args = list(),
  extra_headers = character(0),
  credentials = function() NULL
)

Arguments

Details

To add support for a new backend, you will need to subclass Provider (adding any additional fields that your provider needs) and then implement the various generics that control the behavior of each provider.

Value

An S7 Provider object.

Examples

Provider(
  name = "CoolModels",
  model = "my_model",
  base_url = "https://cool-models.com"
)

Description

Creates a controller that can cancel an in-progress stream. Pass it to Chat's ⁠$stream()⁠ or ⁠$stream_async()⁠ via the controller argument, then call ⁠$cancel()⁠ from anywhere (e.g. a Shiny observer) to stop the stream after the next chunk arrives.

The same controller can be reused across multiple streams. Call ⁠$reset()⁠ to clear the cancelled state, or pass it directly to a new ⁠$stream()⁠ call — it will be reset automatically.

Usage

stream_controller()

Value

An ellmer_stream_controller object with the following elements:

• ⁠$cancel(reason = "cancelled")⁠: Cancel the stream. The reason string is stored on the controller and used as the AssistantPartialTurn's reason property.

• ⁠$reset()⁠: Clear the cancelled state and reason.

• ⁠$cancelled⁠: A logical flag indicating whether the controller has been cancelled.

• ⁠$reason⁠: The cancellation reason string, or NULL if not cancelled.

Async cancellation in Shiny

In a Shiny app, use an ExtendedTask for non-blocking chat and a stream_controller() to wire up a cancel button:

controller <- stream_controller()

chat_task <- ExtendedTask$new(function(user_query, controller = NULL) {
  chat <- chat_openai(model = "gpt-4.1-nano")
  stream <- chat$stream_async(user_query, controller = controller)
  shinychat::markdown_stream("response", stream)
})

observeEvent(input$ask, {
  controller <<- stream_controller()
  chat_task$invoke(input$query, controller = controller)
})

observeEvent(input$cancel, {
  controller$cancel()
})

Examples

chat <- chat_openai(model = "gpt-5.4-nano")

ctrl <- stream_controller()
stream <- chat$stream("Write a short story.", controller = ctrl)

i <- 0
coro::loop(for (chunk in stream) {
  i <- i + 1
  if (i > 10) ctrl$cancel()
})

chat

Description

Call this function to find out the cumulative number of tokens that you have sent and recieved in the current session. The price will be shown if known.

Usage

token_usage()

Value

A data frame

Examples

token_usage()

Description

Annotate a function for use in tool calls, by providing a name, description, and type definition for the arguments.

Learn more in vignette("tool-calling").

Usage

tool(
  fun,
  description,
  ...,
  arguments = list(),
  name = NULL,
  convert = TRUE,
  annotations = list(),
  .name = deprecated(),
  .description = deprecated(),
  .convert = deprecated(),
  .annotations = deprecated()
)

Arguments

Value

An S7 ToolDef object.

ellmer 0.3.0

In ellmer 0.3.0, the definition of the tool() function changed quite a bit. To make it easier to update old versions, you can use an LLM with the following system prompt

Help the user convert an ellmer 0.2.0 and earlier tool definition into a
ellmer 0.3.0 tool definition. Here's what changed:

* All arguments, apart from the first, should be named, and the argument
  names no longer use `.` prefixes. The argument order should be function,
  name (as a string), description, then arguments, then anything

* Previously `arguments` was passed as `...`, so all type specifications
  should now be moved into a named list and passed to the `arguments`
  argument. It can be omitted if the function has no arguments.

```R
# old
tool(
  add,
  "Add two numbers together"
  x = type_number(),
  y = type_number()
)

# new
tool(
  add,
  name = "add",
  description = "Add two numbers together",
  arguments = list(
    x = type_number(),
    y = type_number()
  )
)
```

Don't respond; just let the user provide function calls to convert.

See Also

Other tool calling helpers: tool_annotations(), tool_reject()

Examples

# First define the metadata that the model uses to figure out when to
# call the tool
tool_rnorm <- tool(
  rnorm,
  description = "Draw numbers from a random normal distribution",
  arguments = list(
    n = type_integer("The number of observations. Must be a positive integer."),
    mean = type_number("The mean value of the distribution."),
    sd = type_number("The standard deviation of the distribution. Must be a non-negative number.")
  )
)
tool_rnorm(n = 5, mean = 0, sd = 1)

chat <- chat_openai()
# Then register it
chat$register_tool(tool_rnorm)

# Then ask a question that needs it.
chat$chat("Give me five numbers from a random normal distribution.")

# Look at the chat history to see how tool calling works:
chat
# Assistant sends a tool request which is evaluated locally and
# results are sent back in a tool result.

Description

Tool annotations are additional properties that, when passed to the .annotations argument of tool(), provide additional information about the tool and its behavior. This information can be used for display to users, for example in a Shiny app or another user interface.

The annotations in tool_annotations() are drawn from the Model Context Protocol and are considered hints. Tool authors should use these annotations to communicate tool properties, but users should note that these annotations are not guaranteed.

Usage

tool_annotations(
  title = NULL,
  read_only_hint = NULL,
  open_world_hint = NULL,
  idempotent_hint = NULL,
  destructive_hint = NULL,
  ...
)

Arguments

Value

A list of tool annotations.

See Also

Other tool calling helpers: tool(), tool_reject()

Examples

# See ?tool() for a full example using this function.
# We're creating a tool around R's `rnorm()` function to allow the chatbot to
# generate random numbers from a normal distribution.
tool_rnorm <- tool(
  rnorm,
  # Describe the tool function to the LLM
  .description = "Drawn numbers from a random normal distribution",
  # Describe the parameters used by the tool function
  n = type_integer("The number of observations. Must be a positive integer."),
  mean = type_number("The mean value of the distribution."),
  sd = type_number("The standard deviation of the distribution. Must be a non-negative number."),
  # Tool annotations optionally provide additional context to the LLM
  .annotations = tool_annotations(
    title = "Draw Random Normal Numbers",
    read_only_hint = TRUE, # the tool does not modify any state
    open_world_hint = FALSE # the tool does not interact with the outside world
  )
)

Description

Throws an error to reject a tool call. tool_reject() can be used within the tool function to indicate that the tool call should not be processed. tool_reject() can also be called in an Chat$on_tool_request() callback. When used in the callback, the tool call is rejected before the tool function is invoked.

Here's an example where utils::askYesNo() is used to ask the user for permission before accessing their current working directory. This happens directly in the tool function and is appropriate when you write the tool definition and know exactly how it will be called.

chat <- chat_openai(model = "gpt-4.1-nano")

list_files <- function() {
  allow_read <- utils::askYesNo(
    "Would you like to allow access to your current directory?"
  )
  if (isTRUE(allow_read)) {
    dir(pattern = "[.](r|R|csv)$")
  } else {
    tool_reject()
  }
}

chat$register_tool(tool(
  list_files,
  "List files in the user's current directory"
))

chat$chat("What files are available in my current directory?")
#> [tool call] list_files()
#> Would you like to allow access to your current directory? (Yes/no/cancel) no
#> #> Error: Tool call rejected. The user has chosen to disallow the tool #' call.
#> It seems I am unable to access the files in your current directory right now.
#> If you can tell me what specific files you're looking for or if you can #' provide
#> the list, I can assist you further.

chat$chat("Try again.")
#> [tool call] list_files()
#> Would you like to allow access to your current directory? (Yes/no/cancel) yes
#> #> app.R
#> #> data.csv
#> The files available in your current directory are "app.R" and "data.csv".

You can achieve a similar experience with tools written by others by using a tool_request callback. In the next example, imagine the tool is provided by a third-party package. This example implements a simple menu to ask the user for consent before running any tool.

packaged_list_files_tool <- tool(
  function() dir(pattern = "[.](r|R|csv)$"),
  "List files in the user's current directory"
)

chat <- chat_openai(model = "gpt-4.1-nano")
chat$register_tool(packaged_list_files_tool)

always_allowed <- c()

# ContentToolRequest
chat$on_tool_request(function(request) {
  if (request@name %in% always_allowed) return()

  answer <- utils::menu(
    title = sprintf("Allow tool `%s()` to run?", request@name),
    choices = c("Always", "Once", "No"),
    graphics = FALSE
  )

  if (answer == 1) {
    always_allowed <<- append(always_allowed, request@name)
  } else if (answer %in% c(0, 3)) {
    tool_reject()
  }
})

# Try choosing different answers to the menu each time
chat$chat("What files are available in my current directory?")
chat$chat("How about now?")
chat$chat("And again now?")

Usage

tool_reject(reason = "The user has chosen to disallow the tool call.")

Arguments

Value

Throws an error of class ellmer_tool_reject with the provided reason.

See Also

Other tool calling helpers: tool(), tool_annotations()

Description

Every conversation with a chatbot consists of pairs of user and assistant turns, corresponding to an HTTP request and response. These turns are represented by the Turn object, which contains a list of Contents representing the individual messages within the turn. These might be text, images, tool requests (assistant only), or tool responses (user only).

UserTurn, AssistantTurn, and SystemTurn are specialized subclasses of Turn for different types of conversation turns. AssistantTurn includes additional metadata about the API response.

Note that a call to ⁠$chat()⁠ and related functions may result in multiple user-assistant turn cycles. For example, if you have registered tools, ellmer will automatically handle the tool calling loop, which may result in any number of additional cycles. Learn more about tool calling in vignette("tool-calling").

Usage

Turn(role = NULL, contents = list(), tokens = NULL)

UserTurn(contents = list())

SystemTurn(contents = list())

AssistantTurn(
  contents = list(),
  json = list(),
  tokens = c(NA_real_, NA_real_, NA_real_),
  cost = NA_real_,
  duration = NA_real_
)

AssistantPartialTurn(
  contents = list(),
  json = list(),
  tokens = c(NA_real_, NA_real_, NA_real_),
  cost = NA_real_,
  duration = NA_real_,
  reason = "interrupted"
)

Arguments

Value

An S7 Turn object

An S7 AssistantTurn object

An S7 AssistantPartialTurn object

Examples

UserTurn(list(ContentText("Hello, world!")))

Description

These S7 classes are provided for use by package devlopers who are extending ellmer. In every day use, use type_boolean() and friends.

Usage

TypeBasic(description = NULL, required = TRUE, type = stop("Required"))

TypeEnum(description = NULL, required = TRUE, values = character(0))

TypeArray(description = NULL, required = TRUE, items = Type())

TypeJsonSchema(description = NULL, required = TRUE, json = list())

TypeIgnore(description = NULL, required = TRUE)

TypeObject(
  description = NULL,
  required = TRUE,
  properties = list(),
  additional_properties = FALSE
)

Arguments

Value

S7 objects inheriting from Type

Examples

TypeBasic(type = "boolean")
TypeArray(items = TypeBasic(type = "boolean"))

Description

These functions specify object types in a way that chatbots understand and are used for tool calling and structured data extraction. Their names are based on the JSON schema, which is what the APIs expect behind the scenes. The translation from R concepts to these types is fairly straightforward.

• type_boolean(), type_integer(), type_number(), and type_string() each represent scalars. These are equivalent to length-1 logical, integer, double, and character vectors (respectively).

• type_enum() is equivalent to a length-1 factor; it is a string that can only take the specified values.

• type_array() is equivalent to a vector in R. You can use it to represent an atomic vector: e.g. type_array(type_boolean()) is equivalent to a logical vector and type_array(type_string()) is equivalent to a character vector). You can also use it to represent a list of more complicated types where every element is the same type (R has no base equivalent to this), e.g. type_array(type_array(type_string())) represents a list of character vectors.

• type_object() is equivalent to a named list in R, but where every element must have the specified type. For example, type_object(a = type_string(), b = type_array(type_integer())) is equivalent to a list with an element called a that is a string and an element called b that is an integer vector.

• type_ignore() is used in tool calling to indicate that an argument should not be provided by the LLM. This is useful when the R function has a default value for the argument and you don't want the LLM to supply it.

• type_from_schema() allows you to specify the full schema that you want to get back from the LLM as a JSON schema. This is useful if you have a pre-defined schema that you want to use directly without manually creating the type using the ⁠type_*()⁠ functions. You can point to a file with the path argument or provide a JSON string with text. The schema must be a valid JSON schema object.

Usage

type_boolean(description = NULL, required = TRUE)

type_integer(description = NULL, required = TRUE)

type_number(description = NULL, required = TRUE)

type_string(description = NULL, required = TRUE)

type_enum(values, description = NULL, required = TRUE)

type_array(items, description = NULL, required = TRUE)

type_object(
  .description = NULL,
  ...,
  .required = TRUE,
  .additional_properties = deprecated()
)

type_from_schema(text, path)

type_ignore()

Arguments

Examples

# An integer vector
type_array(type_integer())

# The closest equivalent to a data frame is an array of objects
type_array(type_object(
   x = type_boolean(),
   y = type_string(),
   z = type_number()
))

# There's no specific type for dates, but you use a string with the
# requested format in the description (it's not guaranteed that you'll
# get this format back, but you should most of the time)
type_string("The creation date, in YYYY-MM-DD format.")
type_string("The update date, in dd/mm/yyyy format.")