LLMStep¶

LLMStep is the general LLM step. For each record it fills a prompt template, sends it to a served model, and writes the answer back as one or more new columns.

The specialized LLM steps write the prompt for you. LLMStep is the one where you write it yourself.

At a glance¶

Everything after model is keyword-only.

The prompt¶

The prompt template uses {column} placeholders. Each column you name in it must be listed in input_columns, and each column in input_columns is looked up on the record and injected.

from datafast import LLMStep, openai

step = LLMStep(
    prompt="Write one short question about {topic}",
    input_columns=["topic"],
    output_column="question",
    model=openai(),
)

A placeholder that is not in input_columns raises KeyError when the step runs. A literal brace in the prompt must be doubled — {{ and }} — or it is read as a placeholder.

prompt also accepts a file path, as a str or a Path. The file is read on first use and cached.

from pathlib import Path

from datafast import LLMStep, openai

step = LLMStep(
    prompt=Path("prompts/summarize.txt"),
    input_columns=["text"],
    output_column="summary",
    model=openai(),
)

A missing file raises FileNotFoundError at the first record, before any call is made. That covers every Path, and any string that reads as a path to a prompt file — no whitespace, and one of the suffixes .txt, .md, .jinja, .jinja2, .j2, .prompt, .tmpl. Any other string is the prompt itself.

The output¶

parse_mode decides how one raw answer becomes columns.

json and xml require output_columns; without it the constructor raises ValueError. Both modes also append format instructions to the end of your prompt, naming the columns they expect, so you do not have to describe the format yourself.

from datafast import LLMStep, openai

step = LLMStep(
    prompt="Write a question and its answer about {topic}",
    input_columns=["topic"],
    output_columns=["question", "answer"],
    parse_mode="json",
    model=openai(),
)

In json mode a missing key gives an empty string in that column, with a warning. In xml mode a missing tag does the same; tag matching ignores case. Answers wrapped in markdown code fences are unwrapped before the JSON is read.

output_column and output_columns are not alternatives to each other: if you pass output_columns in text mode, the answer goes into its first entry and the rest are never written, while output_column is ignored.

How many LLM calls¶

A step fans out. For every input record it makes one call per combination of:

prompts × served models × languages × num_outputs

Each call produces one output record, so the step does not preserve record counts — it multiplies them.

No language counts as one, not zero. Multiply before you run: this is the number you pay for.

from datafast import LLMStep, openai

step = LLMStep(
    prompt=["Summarize: {text}", "List the key points of: {text}"],
    input_columns=["text"],
    output_column="result",
    model=[openai(), openai("gpt-6.1-sol")],
    num_outputs=2,
)

Eight calls per record: 2 prompts × 2 served models × 1 language × 2 outputs.

Languages¶

language adds a language axis to that fan-out and gives the template two extra placeholders: {language} for the code and {language_name} for the name.

from datafast import LLMStep, openai

step = LLMStep(
    prompt="Write one sentence about {topic} in {language_name}",
    input_columns=["topic"],
    output_column="sentence",
    model=openai(),
    language={"en": "English", "fr": "French"},
)

Use the dict form when the prompt should name the language, since a model reads French better than fr.

Which columns come through¶

By default every column of the input record is copied to the output record, then the generated columns are added on top.

from datafast import LLMStep, openai

step = LLMStep(
    prompt="Summarize: {text}",
    input_columns=["text"],
    output_column="summary",
    model=openai(),
    forward_columns=["doc_id"],
)

forward_columns wins if both are given; exclude_columns is then ignored. A generated column with the same name as an input column overwrites it.

Skipping records and system prompts¶

from datafast import LLMStep, openai

step = LLMStep(
    prompt="Summarize: {text}",
    input_columns=["text"],
    output_column="summary",
    model=openai(),
    skip_if=lambda record: len(record["text"]) < 100,
    system_prompt="You are a concise technical writer.",
)

A skipped record is dropped, not passed through. If you want to keep it, split the pipeline with Filter and Concat instead.

Metadata columns¶

Every output record carries the step's own columns, all starting with _.

With a single prompt and no language, the only added column is _model. All three names are reserved: compile() treats them as present downstream, so do not use them for your own data.

Temperature and max tokens¶

The step takes neither. It calls ServedModel.generate(), which takes no such arguments, so the served model's own settings decide temperature and length. Set them where the served model is built — see Served models.

Errors¶

on_parse_error is "skip" or "raise"; anything else raises ValueError at construction.

It applies the same way under Pipeline.run() as under the step's own process(), and it covers any failed call — a rate limit or a timeout as much as an unparseable answer. With "skip", compare the record count with the number you expected; with "raise", the run stops on the first failure.

Things worth knowing¶

• Count the calls before you run. Records × prompts × served models × languages × num_outputs. Three small choices multiply into a large bill.
• A mistyped prompt path raises. FileNotFoundError, before the first call.
• json and xml add format instructions to the end of your prompt. Do not write your own on top of them.
• A missing key or tag is an empty string, not a failure. The record is still yielded.
• output_columns in text mode writes to its first entry only.
• skip_if drops records rather than letting them pass.
• _prompt_index is conditional. With one prompt it never appears; with a Sample of prompts it appears on some records and not others. Pass a list for a stable schema.

Where to go next¶

• Specialized LLM steps — Classify, Score, Compare, Rewrite, Extract.
• Pipelines & execution — the steps in context.
• Served models — building the models this step calls.
• Source and Seed — where the records come from.
• Structured output — parse modes and provider-side schemas.
• Full Glossary — record, column, parse mode, served model.
• API reference — the generated signature.