Anatomy of a conversation

Getting Started with LLM APIs in R

R/Pharma 2025

2025-09-16

Anatomy of a Conversation

first, let’s talk about how talking to llms works

you’ve probably talked with chatgpt or claude or gemini and seen a conversation that looks something like this:

You ask a question and ChatGPT responds. You can keep talking to ChatGPT and it keeps responding, and it feels kind of like you’re having a conversation with a person.

how is this happening? The entire conversation happens via HTTP requests and responses

What’s an HTTP request?

HTTP requests power the internet.

When you visit a website, your browser sends an HTTP request to the server hosting that website. The server responds to the request by sending back the HTML of the webpage. And then your browser makes a bunch more requests to get the images, stylesheets, and scripts that make up the page, etc.

What’s an HTTP request?

HTTP requests power the internet.

When you visit a website, your browser sends an HTTP request to the server hosting that website. The server responds to the request by sending back the HTML of the webpage. And then your browser makes a bunch more requests to get the images, stylesheets, and scripts that make up the page, etc.

What’s an HTTP request?

HTTP requests power the internet.

When you visit a website, your browser sends an HTTP request to the server hosting that website. The server responds to the request by sending back the HTML of the webpage. And then your browser makes a bunch more requests to get the images, stylesheets, and scripts that make up the page, etc.

Talking with ChatGPT happens via HTTP

Talking to an LLM like ChatGPT also happens via HTTP requests. When you send a message to ChatGPT, you’re sending an HTTP request to OpenAI’s servers, this time a POST request.

OpenAI processes the request, runs the model, gets the answer and sends it back to you as the response to that POST request.

Talking with ChatGPT happens via HTTP

Talking to an LLM like ChatGPT also happens via HTTP requests. When you send a message to ChatGPT, you’re sending an HTTP request to OpenAI’s servers, this time a POST request.

OpenAI processes the request, runs the model, gets the answer and sends it back to you as the response to that POST request.

Talking with ChatGPT happens via HTTP

Talking to an LLM like ChatGPT also happens via HTTP requests. When you send a message to ChatGPT, you’re sending an HTTP request to OpenAI’s servers, this time a POST request.

OpenAI processes the request, runs the model, gets the answer and sends it back to you as the response to that POST request.

Messages have roles

Now let’s talk about how these messages are structured.

When you send a message to an LLM, you’re not just sending text - you’re sending text with a specific ROLE attached to it.

There are three main roles that messages can have, and understanding these roles is key to working with LLMs programmatically.

• Then USER role - marks what the person using your app types
• And ASSISTANT role - marks what the LLM responds with

This might seem like a small detail, but these roles are fundamental to how you control and guide LLM behavior.

Messages have roles

Now let’s talk about how these messages are structured.

When you send a message to an LLM, you’re not just sending text - you’re sending text with a specific ROLE attached to it.

There are three main roles that messages can have, and understanding these roles is key to working with LLMs programmatically.

[As the diagrams appear] - USER messages - what the person using your app types - And ASSISTANT messages - what the LLM responds with

This might seem like a small detail, but these roles are fundamental to how you control and guide LLM behavior.

there’s also the SYSTEM PROMPT - This is your instruction manual for the LLM. It’s where YOU as the developer tell the model how to behave, what personality to have, what constraints to follow. The user typically never sees this. think about how you would never really come across the chatgpt system prompt. The system prompt is particularly powerful because it persists across the entire conversation and shapes how the assistant responds to every user message.

Messages have roles

Now let’s talk about how these messages are structured.

When you send a message to an LLM, you’re not just sending text - you’re sending text with a specific ROLE attached to it.

There are three main roles that messages can have, and understanding these roles is key to working with LLMs programmatically.

[As the diagrams appear] - USER messages - what the person using your app types - And ASSISTANT messages - what the LLM responds with

This might seem like a small detail, but these roles are fundamental to how you control and guide LLM behavior.

there’s also the SYSTEM PROMPT - This is your instruction manual for the LLM. It’s where YOU as the developer tell the model how to behave, what personality to have, what constraints to follow. The user typically never sees this. think about how you would never really come across the chatgpt system prompt. The system prompt is particularly powerful because it persists across the entire conversation and shapes how the assistant responds to every user message.

Messages have roles

Now let’s talk about how these messages are structured.

When you send a message to an LLM, you’re not just sending text - you’re sending text with a specific ROLE attached to it.

There are three main roles that messages can have, and understanding these roles is key to working with LLMs programmatically.

[As the diagrams appear] - USER messages - what the person using your app types - And ASSISTANT messages - what the LLM responds with

This might seem like a small detail, but these roles are fundamental to how you control and guide LLM behavior.

there’s also the SYSTEM PROMPT - This is your instruction manual for the LLM. It’s where YOU as the developer tell the model how to behave, what personality to have, what constraints to follow. The user typically never sees this. think about how you would never really come across the chatgpt system prompt. The system prompt is particularly powerful because it persists across the entire conversation and shapes how the assistant responds to every user message.

Message roles

to recap

Role	Description
system_prompt	Instructions from the developer (that’s you!) to set the behavior of the assistant
user	Messages from the person interacting with the assistant
assistant	The AI model’s responses to the user

So now that we understand how messages work and what roles they play, let’s see how to actually DO this in R.

ellmer is an open source R package from Posit that makes it easy to work with LLMs. It handles all the HTTP requests and API details for you, so you can focus on building with LLMs rather than wrestling with API documentation.

Let’s see what the code looks like.

First step - load the ellmer package, just like any other R package.

library(ellmer)

Now we create a chat object. This sets up a connection to OpenAI’s API. The chat object is what we’ll use to send messages and get responses.

library(ellmer)

chat <- chat_openai()

Now we can use the chat object to send a message using the chat method

and pass in our message as a string.

library(ellmer)

chat <- chat_openai()

chat$chat("Tell me a quick fact about sheep.")

And we get a response back!

think back to what we just learned about roles. What are the user and assistant roles in this example?

The user role is our question - “Tell me a quick fact about sheep” The assistant role is the response we got back.

library(ellmer)

chat <- chat_openai()

chat$chat("Tell me a quick fact about sheep.")
#> Sheep have excellent memories—they can remember up
#> to 50 individual sheep faces and even recognize
#> human faces for years!

❓ What are the user and assistant roles in this example?

One of the nice things about ellmer is that you can easily inspect the chat object to see the full conversation history.

If we just call ‘chat’, we can see all the messages and their roles. You can see the user message and the assistant response, as well as how many tokens were used

but what about the third role – the system prompt? where is that? we didn’t set up one, so the model just uses whatever behavior it defaults to

chat

<Chat OpenAI/gpt-4.1 turns=2 tokens=15/23 $0.00>
── user [15] ───────────────────────────────────────────────────────────────────────────────────
tell me a quick fact about sheep
── assistant [23] ──────────────────────────────────────────────────────────────────────────────
Sheep have excellent memories—they can remember up to 50 individual sheep faces and even recognize human faces for years!

❓ What about the system prompt?

the chat_openai() function includes a system_prompt argument where you can set a system prompt Let’s try adding one

This is a silly example - we’re telling the model to always answer in haikus. But it demonstrates how system prompts work and can control model behavior

library(ellmer)

chat <- chat_openai(
  system_prompt = "Always answer in haikus."
)

chat$chat("What is chirality?")

It actually answered in a haiku.

The system prompt shapes EVERY response the model gives. this is really powerful because you can exert quite a bit of control over how the model behaves

library(ellmer)

chat <- chat_openai(
  system_prompt = "Always answer in haikus."
)

chat$chat("What is chirality?")

Left and right differ—
mirrored, yet not the same thing.
Chirality's twist.

Now when we inspect the chat object, we can see all three roles - system, user, and assistant. The system prompt is there shaping the conversation.

ellmer

chat

<Chat OpenAI/gpt-4.1 turns=3 tokens=22/22 $0.00>
── system [0] ───────────────────────────────────────────────────────────────────────────────────────────
Always answer in haikus.
── user [22] ────────────────────────────────────────────────────────────────────────────────────────────
What is chirality?
── assistant [22] ───────────────────────────────────────────────────────────────────────────────────────
Left and right differ—  
mirrored, yet not the same thing.  
Chirality's twist.

Your Turn 02_word-game

1. Set up a chat with a system prompt:

   You are playing a word guessing game. At each turn, guess the word and tell us what it is.

2. Ask: In British English, guess the word for the person who lives next door.

3. Ask: What helps a car move smoothly down the road?

4. Create a new, empty chat and ask the second question again.

5. How do the answers to 3 and 4 differ? Why?

05:00

1. Chat objects maintain conversation history - the original chat “remembers” the British English context
2. New chat objects start with no history - creating a new chat means starting fresh

Demo: clearbot

Demo showing what’s really happening behind the scenes - the full conversation history being sent with each request. This reveals that LLMs are stateless.

👩‍💻 _demos/03_clearbot/app.py

System prompt:

You are playing a word guessing game. At each turn, guess the word and tell us what it is.

First question:

In British English, guess the word for the person who lives next door.

Second question:

What helps a car move smoothly down the road?

You just saw in clearbot what’s really happening when you have a conversation with an LLM.

Those conversations look something like this - you ask, it responds, back and forth. But as clearbot showed, something interesting is happening behind the scenes.

Is this actually a conversation?

It IS a conversation, but it works differently than you might think.

LLMs are stateless

• The LLM doesn’t remember anything between requests

• You have to send the entire conversation history with every message

• The LLM reconstructs the “conversation” from what you send

The LLM has amnesia - it forgets everything after each response. Each API request is completely independent.

Just like if you were talking to someone with amnesia, you have to recap the entire conversation every time you want to continue. That’s what clearbot showed you - the full history being sent with each request.

This isn’t a limitation of transformers specifically - it’s how the HTTP API is designed. Though it aligns with how transformers work (each forward pass is independent).

How LLMs work (briefly)

Now that you understand how conversations work with LLMs - the statelessness, sending full history - let’s talk briefly about how LLMs themselves actually work. How do they understand what you’re asking? And how do they construct their responses?

How do LLMs understand?

This is how LLMs understand the conversation. They’re trained on massive amounts of text data - essentially everything written on the internet, in books, etc. By learning patterns from all this text, they develop the ability to understand context, meaning, and how language works.

If you read everything
ever written…

• Books and stories

• Websites and articles

• Poems and jokes

• Questions and answers

…then you could…

• Answer questions
• Write stories
• Tell jokes
• Explain things
• Translate into any language

How do LLMs respond?

Now let’s talk about how LLMs construct their responses. They don’t generate whole sentences at once - they generate responses token by token, choosing the most likely next piece based on everything they’ve seen so far.

LLMs think in tokens

LLMs don’t think in words - they think in tokens. This is the fundamental unit they use to construct responses, one token at a time.

• Fundamental units of information for LLMs
• Words, parts of words, or individual characters
  • “hello” → 1 token
  • “unconventional” → 3 tokens: un|con|ventional
• Important for:
  • Model input/output limits
  • API pricing is usually by token
• Not just words, but images can be tokenized too

Demo:
token-possibilities

This demo shows how LLMs construct responses in real-time. You’ll see the model generating tokens one at a time, and you can see the probabilities for what token might come next. This is how all LLM responses are built - token by token, choosing the most likely next piece.

The breakthrough insight is the attention mechanism in the Transformer architecture. Instead of processing text sequentially (word by word), GPT models can look at all words in a sequence simultaneously and understand how they relate to each other.

The attention mechanism allows the model to:

• Weigh the importance of every word in relation to every other word
• Capture long-range dependencies that traditional models missed
• Process sequences in parallel rather than sequentially

👩‍💻 _demos/04_token-possibilities/app.R

How to think about LLMs

Some motivational words by Joe urging everyone to approach LLMs with curiosity and experimentation rather than preconceived notions about limitations, while building understanding from the ground up through practical experience.

Think Empirically, Not Theoretically

• It’s okay to (mostly) treat LLMs as black boxes.

• Just try it! When wondering if an LLM can do something,
  experiment rather than theorize

• You might think they could not possibly do things
  that they clearly can do today

• And you might think surely they can do something
  that it turns out they’re terrible at

• Understanding the technical details can lead to bad intuition about capabilities
• You might think “they could not possibly do things that they clearly can do today”
• Empirical testing reveals actual capabilities vs. theoretical limitations

LLMs are jagged

Don’t expect smooth, predictable performance across tasks. LLMs can surprise you - excelling at some complex tasks while struggling with seemingly simple ones.

LLMs are jagged

Embrace the Experimental Process

Embrace the Experimental Process

• Explore!
  Focus on learning and engaging with the technology, not outcomes

• Failure is valuable!
  those are some of the most interesting conversations that we have

• It doesn’t have to be a success.
  Attempts that don’t work still provide insights

• Use the best models available for experimentation
• Don’t constrain yourself to “practical” applications initially
• Think of it as forming your own independent conclusions about usefulness

Start Simple, Build Understanding

Start Simple, Build Understanding

• We’re going to focus on the core building blocks.

• All the incredible things you see AI do
  decompose to just a few key ingredients.

• Our goal is to have fun and build intuition
  through hands-on experience.

• Think very generally about what tools can do - they’re as powerful as any software you can write
• Remember: “Everything decomposes” to the basic components once you understand the underlying APIs
• Build intuition through hands-on experience rather than theoretical study
• We want to have fun in this course! It’s worth saying up front that our examples are not necessarily practical, but they’re full of practical lessons.

What if I want to keep chatting back-and-forth?

Transition to showing how to build interactive chat interfaces with shinychat.

ellmer can do that, too!

	Console	Browser
	live_console(chat)	live_browser(chat)

As we’ll see, everything we’re going to do today can be done in either R or Python and ellmer and chatlas are designed to be similar.

But they’re in different languages and those languages have different conventions and idioms, so they’re not completely identical. If they were, one or the other would start to feel unnatural to use.

Demo:
live

This demo shows live_console() and live_browser() - easy ways to chat with an LLM interactively from R.

👩‍💻 _demos/05_live/05_live.R

shinychat

shinychat and ellmer work together - ellmer handles the LLM API calls, shinychat provides the chat interface for Shiny apps.

shinychat in R

We’ll build up a shinychat app step by step. Start with a basic Shiny app structure.

Start with a basic shinyapp

library(shiny)
library(bslib)

ui <- page_fillable(

)

server <- function(input, output, session) {

}

shinyApp(ui, server)

shinychat in R

Add the two key packages - shinychat for the UI and ellmer for LLM interactions.

Load {shinychat} and {ellmer}

library(shiny)
library(bslib)
library(shinychat)
library(ellmer)

ui <- page_fillable(

)

server <- function(input, output, session) {

}

shinyApp(ui, server)

shinychat in R

Use the shinychat module pattern - chat_mod_ui in the UI and chat_mod_server in the server. The “chat” ID connects them.

Use the shinychat chat module

library(shiny)
library(bslib)
library(shinychat)
library(ellmer)

ui <- page_fillable(
  chat_mod_ui("chat")
)

server <- function(input, output, session) {
  chat_mod_server("chat")
}

shinyApp(ui, server)

shinychat in R

Finally, create the chat client and pass it to chat_mod_server. Now you have a working chatbot in your Shiny app!

Create and hook up a chat client to use in the app

library(shiny)
library(bslib)
library(shinychat)
library(ellmer)


ui <- page_fillable(
  chat_mod_ui("chat")
)

server <- function(input, output, session) {
  client <- chat_openai()
  chat_mod_server("chat", client)
}

shinyApp(ui, server)

Your Turn 03_word-games

Now participants will build their own shinychat app. The exercise file has the setup - they just need to add the shinychat and ellmer code to make it work.

1. I’ve set up the basic Shiny app snippet and a system prompt.

2. Your job: create a chatbot that plays the word guessing game with you.

3. The twist: this time, you’re guessing the word.

07:00