Groq: Llama 3.1 8B Instant

Name: Llama 3.1 8B Instant API
Brand: Groq

stable

Llama 3.1 8B Instant is available through Everstack's OpenAI-compatible gateway with routing, fallback, retries, and observability.

llama-3.1-8b-instantllamareleased Jul 23, 2024

Provider Quick start

Context

131.1K

tokens

Max output

131.1K

tokens

Input

$0.050

per 1M

Output

$0.080

per 1M

Activity

Daily period usage

Aggregated managed Everstack traffic. Buckets below 5 tenants or 100 requests are withheld.

Providers

ProviderContextInput /MOutput /M

Groq

131.1K$0.050$0.080

Pricing

Item	Price	Unit
Input	$0.050	per 1M tokens
Output	$0.080	per 1M tokens

Performance

Reliability

Compare

Compare usage and performance

Place this model beside up to three other catalog models.

Capabilities

Modalities

Input

Output

Details

Model ID: llama-3.1-8b-instant
Family: llama
Status: stable
Knowledge: -
Released: Jul 23, 2024

Gateway Config

API: 2024-01-01
Auth: GROQ_API_KEY
Discovery: static
Provider type: meta

Quick Start

index.ts

import Everstack from "@everstack/node";const client = new Everstack({  apiKey: process.env.EVERSTACK_API_KEY!,  baseUrl: "https://{instance}.{region}.everstack.ai",});const completion = await client.chat.completions.create({  model: "@groq/llama-3.1-8b-instant",  messages: [{ role: "user", content: "Explain vector cache routing." }],});console.log(completion.choices[0]?.message?.content);

Endpoint

Base URL

https://api.groq.com/openai/v1

Chat completions

https://api.groq.com/openai/v1/chat/completions

Model

llama-3.1-8b-instant

Auth env

GROQ_API_KEY

Llama 3.1 8B Instant FAQ

How much does the Llama 3.1 8B Instant API cost?

Through Groq, Llama 3.1 8B Instant costs $0.050 per 1M input tokens and $0.080 per 1M output tokens.

What is the context window of Llama 3.1 8B Instant?

Llama 3.1 8B Instant accepts up to 131.1K tokens of context and returns up to 131.1K output tokens on Groq.

What can Llama 3.1 8B Instant do?

Llama 3.1 8B Instant supports chat and function calling. It accepts text input.

How do I call Llama 3.1 8B Instant through Everstack?

Point any OpenAI-compatible SDK at your Everstack gateway and set the model to @groq/llama-3.1-8b-instant. Everstack adds retries, provider fallback, caching, per-tenant budgets and OpenTelemetry traces for every request.