Keep a conversation
A single question and answer is rarely the whole job. You ask something, read the answer, and then ask a follow-up that depends on it, the way you would with a person who remembers what they just told you. A model doesn’t remember. Each request starts from nothing, and the model knows only what that request contains. So in LM15, as with every language-model API, the conversation lives in your program: a list of messages that you keep, add to, and send with every request.
On this page, you’ll learn how to keep that list. We’ll start with a single question, and look inside the message that comes back. Then we’ll turn it into a loop that holds a longer conversation, and see what each turn costs. Next, we’ll save a conversation to a file and pick it up again the next day. We’ll finish with what to do when a conversation grows long.
To run the examples, you’ll need LM15 installed and an API key; see Make your first request, which also shows why a follow-up question needs the conversation before it.
One question, kept
Section titled “One question, kept”The examples in these guides follow one small program: an assistant for a wildlife research station. Here is its first question again, this time with the messages kept in a list of their own:
router = LMRouter()
messages = [Message.user(
"What might be eating the acorns under our oak trees at night?"
)]
response = router.complete(Request(
model="provider:model",
system=(
"You are the field assistant for a wildlife research "
"station. Answer in two sentences."
),
messages=messages,
))
messages.append(response.message)
print(response.text)Likely nighttime acorn-eaters include deer, raccoons, opossums, mice, rats, and—where present—wild boar; squirrels and jays may also cache or remove acorns around dawn and dusk. Check for tracks, droppings, rooting, or gnaw marks, and set a motion-triggered trail camera to identify the visitor.
The list starts with your question. When the answer comes back, the model’s message goes on the end, so the list now holds the whole exchange: a question and its answer. The next request will send both. The instructions aren’t in the list. They belong to each request, alongside the messages, and you send them every time.
What’s in the model’s message
Section titled “What’s in the model’s message”A message is made of parts. Print the kind of each part in the model’s message:
for part in response.message.parts:
print(part.type)thinking text
The second part is the text you’ve just read. The first is the model’s reasoning, the thinking it did before it answered. This provider keeps its reasoning to itself, so the part holds no text at all. What it holds instead is a sealed copy, which only the provider can read, and which goes back to the provider if you send the message back.
That’s why you keep the model’s whole message, rather than just its text. It isn’t free. We sent the second question twice: once after the whole message, and once after a new message holding only the answer’s text.
| Kept before the second question | Input tokens |
|---|---|
| The model’s whole message | 164 |
| Only its text | 124 |
The difference is the sealed reasoning, which the provider received and counted. Both answers were fine. So why keep it? Because the model’s message is more than its text, and what the other parts are for is up to the provider. OpenAI and Anthropic both ask, in their documentation, for the reasoning to come back while the model is in the middle of using a tool, and Anthropic calls it required. (In our own test, Anthropic answered without it. Don’t count on that.) A message can also hold tool calls and citations, and keeping only the text throws those away too. Keeping the whole message is right for every provider, and LM15 sends each part back in the form that provider expects.
A good rule of thumb: keep the text for people to read, and the whole message for the model to continue.
Keep going
Section titled “Keep going”A real conversation has more than two turns. Keeping a list makes that a loop: add the question, send everything, add the answer, and go round again. Here are three of the station’s questions in a row:
router = LMRouter()
model = "provider:model"
instructions = (
"You are the field assistant for a wildlife research station. "
"Answer in two sentences."
)
messages = []
for question in [
"What might be eating the acorns under our oak trees at night?",
"Would the same animals eat hazelnuts?",
"How could we find out which one it is?",
]:
messages.append(Message.user(question))
response = router.complete(Request(
model=model, system=instructions, messages=messages,
))
messages.append(response.message)
print(">", question)
print(response.text)
print(response.usage.input_tokens, "tokens in")> What might be eating the acorns under our oak trees at night? Likely nighttime acorn eaters include deer, raccoons, opossums, mice, rats, squirrels, chipmunks, and wild pigs, depending on your region. Look for tracks, droppings, rooting, or gnawed shells, or set up a motion-activated trail camera to identify the visitor. 40 tokens in > Would the same animals eat hazelnuts? Yes—many of the same animals eat hazelnuts, especially squirrels, chipmunks, mice, rats, raccoons, deer, and wild pigs; opossums may eat kernels they can access. Rodents usually leave neatly gnawed shells, while deer and pigs may crush or swallow nuts and disturb the ground. 150 tokens in > How could we find out which one it is? Set an infrared motion-activated trail camera 30–60 cm (1–2 ft) above the ground, aimed at the nut fall area, and check footage over several nights. Also smooth a patch of damp soil or sand to capture tracks and note scat, rooting, and shell damage—paired chisel-like gnaw marks suggest rodents, while crushed nuts and churned soil suggest deer or wild pigs. 283 tokens in
Each answer builds on the ones before it. “The same animals” in the second question means the animals from the first answer, and the model knew that. The third question, “which one it is”, only makes sense after the first two.
The input tokens tell a second story: 40, then 150, then 283. Every request sends the whole conversation again, so each turn costs more than the one before, and you pay for the early turns again every time you ask something new. That’s worth remembering before you let a conversation run for a hundred turns.
Pick up where you left off
Section titled “Pick up where you left off”A conversation doesn’t have to end when your program does. Suppose the station runs the assistant in the evening, and the observers want to carry on the next morning. Save the conversation, with its model and instructions, to a file:
saved = Request(model=model, system=instructions, messages=messages)
with open("conversation.json", "w") as f:
json.dump(request_to_dict(saved), f)LM15 writes the conversation in its own JSON form, which is the same in all six languages, so a file saved by one reads in any of the others. Nothing is left out: the file holds each answer’s sealed reasoning along with its text. It also holds everything anyone said, so keep it as private as the conversation.
The next day, a new program reads the file and asks one more question:
with open("conversation.json") as f:
saved = request_from_dict(json.load(f))
followup = Request(
model=saved.model,
system=saved.system,
messages=[*saved.messages, Message.user(
"Which of those should we look for first?"
)],
)
router = LMRouter()
print(router.complete(followup).text)Look first for small rodents—especially mice or rats—because they commonly collect hazelnuts at night and leave small tracks plus neatly gnawed shells; squirrels and chipmunks are mostly daytime visitors. If there are crushed shells, larger tracks, scat, or churned soil, next check for raccoons, deer, or wild pigs.
The question says “those”, and the model knew which animals were meant, because the conversation it received was yesterday’s, in full. It even kept track of the hazelnuts, from the second question.
When a conversation gets long
Section titled “When a conversation gets long”Because each turn sends everything again, a long conversation gets slower and more expensive, and eventually it no longer fits: every model has a limit on how much one request can hold, called its context window. LM15 doesn’t shorten a conversation for you, because what to keep depends on your application. Here are three common choices:
- Start a new conversation when the topic changes. It’s the cheapest, and often the clearest, option.
- Drop the oldest turns, and keep the instructions, which aren’t in the list anyway. Remove each question together with its answer, and keep a tool call together with its result.
- Summarize. Ask the model to summarize the early part of the conversation, then start a new list with that summary.
If the start of a conversation stays the same from one request to the next, prompt caching can make sending it again cheaper.
Try it yourself
Section titled “Try it yourself”Predict what will happen before you run each one.
- In the loop, add each question to the list, but not the model’s message. What will the second answer say?
- Print the input tokens of the next day’s request. Will they be more or fewer than 283?
- Before asking “which of those” the next day, remove the first question and its answer from the saved conversation. What can the model still work out?
What to expect
- The model receives two questions in a row, and no answers. It doesn’t know which animals “the same animals” are, so it may answer both questions, or ask what you mean.
- More. The saved conversation holds everything the third request did, plus the third answer, and then the new question.
- The second question and its answer still name the animals, so “those” still means something. What’s lost is why you asked: nothing now says the acorns were disappearing from under the oak trees at night.
When answers get long, you may want to show them as they’re written; see Stream a response. The same list of messages also carries tool calls and their results: Call your own functions builds its loop that way.