ChatGroq integration

Integrate with the ChatGroq chat model using LangChain JavaScript.

Groq offers fast AI inference powered by LPU™ AI inference technology.

For a list of available models, see the Groq model documentation.

This page helps you get started with Groq chat models. For detailed documentation of all ChatGroq features and configurations, see the API reference.

Overview

Integration details

Class	Package	Serializable	PY support	Downloads	Version
ChatGroq	@langchain/groq	❌	✅

Model features

See the links in the table headers below for guides on how to use specific features.

Tool calling	Structured output	Image input	Audio input	Video input	Token-level streaming	Token usage	Logprobs
✅	✅	❌	❌	❌	✅	✅	✅

Setup

To access ChatGroq models you’ll need to create a Groq account, get an API key, and install the @langchain/groq integration package.

Credentials

To use the Groq API, create an API key in the Groq console. Then, you can set the API key as an environment variable in your terminal:

export GROQ_API_KEY="your-api-key"

If you want to get automated tracing of your model calls you can also set your LangSmith API key by uncommenting below:

# export LANGSMITH_TRACING="true"
# export LANGSMITH_API_KEY="your-api-key"

Installation

The LangChain ChatGroq integration lives in the @langchain/groq package:

npm

npm install @langchain/groq @langchain/core

Instantiation

Now we can instantiate our model object and generate chat completions:

import { ChatGroq } from "@langchain/groq"

const llm = new ChatGroq({
    model: "llama-3.3-70b-versatile",
    temperature: 0,
    maxTokens: undefined,
    maxRetries: 2,
    // other params...
})

Invocation

const aiMsg = await llm.invoke([
    {
      role: "system",
      content: "You are a helpful assistant that translates English to French. Translate the user sentence.",
    },
    { role: "user", content: "I love programming." },
])
aiMsg

AIMessage {
  "content": "I enjoy programming. (The French translation is: \"J'aime programmer.\")\n\nNote: I chose to translate \"I love programming\" as \"J'aime programmer\" instead of \"Je suis amoureux de programmer\" because the latter has a romantic connotation that is not present in the original English sentence.",
  "additional_kwargs": {},
  "response_metadata": {
    "tokenUsage": {
      "completionTokens": 73,
      "promptTokens": 31,
      "totalTokens": 104
    },
    "finish_reason": "stop"
  },
  "tool_calls": [],
  "invalid_tool_calls": []
}

console.log(aiMsg.content)

I enjoy programming. (The French translation is: "J'aime programmer.")

Note: I chose to translate "I love programming" as "J'aime programmer" instead of "Je suis amoureux de programmer" because the latter has a romantic connotation that is not present in the original English sentence.

Invoke with JSON output

const messages = [
  {
    role: "system",
    content: "You are a math tutor that handles math exercises and makes output in json in format { result: number }.",
  },
  { role: "user",  content: "2 + 2 * 2" },
];

const aiInvokeMsg = await llm.invoke(messages, { response_format: { type: "json_object" } });

// if you want not to pass response_format in every invoke, you can bind it to the instance
const llmWithResponseFormat = llm.bind({ response_format: { type: "json_object" } });
const aiBindMsg = await llmWithResponseFormat.invoke(messages);

// they are the same
console.log({ aiInvokeMsgContent: aiInvokeMsg.content, aiBindMsg: aiBindMsg.content });

{
  aiInvokeMsgContent: '{\n"result": 6\n}',
  aiBindMsg: '{\n"result": 6\n}'
}

---

API reference

For detailed documentation of all ChatGroq features and configurations head to the API reference.

---

Connect these docs to Claude, VSCode, and more via MCP for real-time answers.

Edit this page on GitHub or file an issue.