Introduction to LLMs

Lecture 17

Dr. Benjamin Soltoff

Cornell University
INFO 4940/5940 - Fall 2025

October 28, 2025

Announcements

Announcements

• Homework 05 due tomorrow
  • Submit model card via Gradescope
  • Publish model API using http://appliedml.infosci.cornell.edu
• Project 01 draft due Thursday

Learning objectives

• Define a conversation with a large language model (LLM)
• Identify the key roles in an LLM conversation
• Use {ellmer} in R or chatlas in Python to converse with an LLM
• Define a generative pre-trained transformer
• Create basic chat apps using Shiny

Application exercise

ae-16

Instructions

• Go to the course GitHub org and find your ae-16 (repo name will be suffixed with your GitHub name).
• Clone the repo in Positron, install required packages using renv::restore() (R) or uv sync (Python), open the Quarto document in the repo, and follow along and complete the exercises.
• Render, commit, and push your edits by the AE deadline – end of the day

⌨️ 01_hello-llm

Instructions

Test that you can connect to OpenAI’s API using {ellmer} in R or chatlas in Python by running the provided code.

Copy your .env file from your user directory to the ae-16 folder.

How to think about LLMs

Some motivational words by Joe urging everyone to approach LLMs with curiosity and experimentation rather than preconceived notions about limitations, while building understanding from the ground up through practical experience.

Think empirically, Not theoretically

• It’s okay to treat LLMs as black boxes. We’re not going to focus on how they work internally

• Just try it! When wondering if an LLM can do something, experiment rather than theorize

• You might think they could not possibly do things that they clearly can do today

• Understanding the technical details can lead to bad intuition about capabilities
• You might think “they could not possibly do things that they clearly can do today”
• Empirical testing reveals actual capabilities vs. theoretical limitations

Embrace the experimental process

• Don’t worry about ROI during exploration. Focus on learning and engaging with the technology

• Failure is valuable! Those are some of the most interesting conversations that we have

• It doesn’t have to be a success. Attempts that don’t work still provide insights

• Use the best models available for experimentation
• Don’t constrain yourself to “practical” applications initially
• Think of it as forming your own independent conclusions about usefulness

Start simple, build understanding

• We’re going to focus on the core building blocks.

• All the incredible things you see AI do decompose to just a few key ingredients.

• Our goal is to build intuition through hands-on experience.

• Think very generally about what tools can do - they’re as powerful as any software you can write
• Remember: “Everything decomposes” to the basic components once you understand the underlying APIs
• Build intuition through hands-on experience rather than theoretical study
• We want to have fun in this course! It’s worth saying up front that our examples are not necessarily practical, but they’re full of practical lessons.

Anatomy of a conversation

Those conversations often look something like this:

You ask a question and ChatGPT responds. You can keep talking to ChatGPT and it keeps responding, almost like you’re having a conversation with a person.

The entire conversation happens via HTTP requests and responses

What’s an HTTP request?

HTTP requests power the internet. When you visit a website, your browser sends an HTTP request to the server hosting that website. The server responds to the request by sending back the HTML of the webpage. And then your browser makes a bunch more requests to get the images, stylesheets, and scripts that make up the page, etc.

Talking with ChatGPT happens via HTTP

Talking to an LLM like ChatGPT also happens via HTTP requests. When you send a message to ChatGPT, you’re sending an HTTP request to OpenAI’s servers, this time a POST request.

OpenAI processes the request, runs the model, gets the answer and sends it back to you as the response to that POST request.

Messages have roles

Message roles

Role	Description
system_prompt	Instructions from the developer (that’s you!) to set the behavior of the assistant
user	Messages from the person interacting with the assistant
assistant	The AI model’s responses to the user

Hello, {ellmer} and chatlas!

R

Python

Hello, {ellmer} and chatlas!

R

library(ellmer)

Python

import chatlas

Hello, {ellmer} and chatlas!

R

library(ellmer)

chat <- chat_openai()

Python

import chatlas

chat = chatlas.ChatOpenAI()

Hello, {ellmer} and chatlas!

R

library(ellmer)

chat <- chat_openai()

chat$chat("Tell me a joke about R.")

Python

import chatlas

chat = chatlas.ChatOpenAI()

chat.chat("Tell me a joke about Python.")

Hello, {ellmer} and chatlas!

R

library(ellmer)

chat <- chat_openai()

chat$chat("Tell me a joke about R.")
#> Why did the R programmer go broke?
#> Because he kept using `sample()` and lost all his data!

Python

import chatlas

chat = chatlas.ChatOpenAI()

chat.chat("Tell me a joke about Python.")
#> Why do Python programmers prefer using snakes as pets?
#> Because they don't mind the indentation!

❓ What are the user and assistant roles in this example?

Hello, {ellmer} and chatlas!

R

chat

<Chat OpenAI/gpt-4.1 turns=2 tokens=14/29 $0.00>
── user [14] ────────────────────────────────────────
Tell me a joke about R.
── assistant [29] ───────────────────────────────────
Why did the R programmer go broke?

Because he kept using `sample()` and lost all his data!

Python

print(chat)

Hello, {ellmer} and chatlas!

R

chat

Python

print(chat)

## 👤 User turn:

Tell me a joke about Python.

## 🤖 Assistant turn:

Why do Python programmers prefer using snakes as pets?

Because they don't mind the indentation!

❓ What about the system prompt?

Hello, {ellmer} and chatlas!

R

library(ellmer)

chat <- chat_openai(
  system_prompt = "You are a dad joke machine."
)

chat$chat("Tell me a joke about R.")

Python

import chatlas

chat = chatlas.ChatOpenAI(
  system_prompt="You are a dad joke machine."
)

chat.chat("Tell me a joke about Python.")

Hello, {ellmer} and chatlas!

R

library(ellmer)

chat <- chat_openai(
  system_prompt = "You are a dad joke machine."
)

chat$chat("Tell me a joke about R.")

Why did the letter R get invited to all the pirate parties?

Because it always knows how to *arr-r-ive* in style!

Hello, {ellmer} and chatlas!

R

chat

<Chat OpenAI/gpt-4.1 turns=3 tokens=25/28 $0.00>
── system [0] ───────────────────────────────────────
You are a dad joke machine.
── user [25] ────────────────────────────────────────
Tell me a joke about R.
── assistant [28] ───────────────────────────────────
Why did the letter R get invited to all the pirate parties?

Because it always knows how to *arr-r-ive* in style!

⌨️ 02_word-game

Instructions

1. Set up a chat with a system prompt:

   You are playing a word guessing game. At each turn, guess the word and tell us what it is.

2. Ask: In British English, guess the word for the person who lives next door.

3. Ask: What helps a car move smoothly down the road?

4. Create a new, empty chat and ask the second question again.

5. How do the answers to 3 and 4 differ? Why?

Demo: clearbot

👨‍💻 _demos/03_clearbot/app.py

System prompt:

You are playing a word guessing game. At each turn, guess the word and tell us what it is.

First question:

In British English, guess the word for the person who lives next door.

Second question:

What helps a car move smoothly down the road?

• Setup basic chat with no system prompt

• Add the system prompt, reset chat, and ask questions

• Take a look at the trace and see how conversation builds

• Have students play around with different models and adjust the temperature

  What changes about the responses?

How to talk to robots

Those conversations often look something like this:

You ask a question and ChatGPT responds. You can keep talking to ChatGPT and it keeps responding, almost like you’re having a conversation with a person.

But this is not a conversation.

It certainly feels like a conversation, but something else is happening behind the scenes.

When you talk to ChatGPT

1. You write some words

2. The ChatGPT continues writing words

3. You think you’re having a conversation

ChatGPT

Chatting with a Generative Pre-trained Transformer

LLM → Large Language Model

GPT is a type of LLM, but not all LLMs are GPTs.

LLM: Deep learning on huge amounts of text data to understand and generate human language. GPT: Specific architectural design of LLMs.

From now on, we’ll use LLM to refer to any large language model.

How to make an LLM

If you read everything
ever written…

• Books and stories

• Websites and articles

• Poems and jokes

• Questions and answers

…then you could…

• Answer questions
• Write stories
• Tell jokes
• Explain things
• Translate into any language

The cat sat in the ____

• 🎩
• 🛌
• 📦

• 🪟
• 🛒
• 👠

The breakthrough insight is the attention mechanism in the Transformer architecture. Instead of processing text sequentially (word by word), GPT models can look at all words in a sequence simultaneously and understand how they relate to each other.

The attention mechanism allows the model to:

• Weigh the importance of every word in relation to every other word
• Capture long-range dependencies that traditional models missed
• Process sequences in parallel rather than sequentially

Actually: tokens, not words

• Fundamental units of information for LLMs
• Words, parts of words, or individual characters
  • “hello” → 1 token
  • “unconventional” → 3 tokens: un|con|ventional
• Important for:
  • Model input/output limits
  • API pricing is usually by token
• Not just words, but images can be tokenized too

Demo: token-possibilities

👨‍💻 _demos/04_token-possibilities/app.R

Programming is fun, but I kind of like ChatGPT…

{ellmer} and chatlas can do that, too!

	Console	Browser
	live_console(chat)	live_browser(chat)
	chat.console()	chat.app()

As we’ll see, everything we’re going to do today can be done in either R or Python and ellmer and chatlas are designed to be similar.

But they’re in different languages and those languages have different conventions and idioms, so they’re not completely identical. If they were, one or the other would start to feel unnatural to use.

⌨️ 05_live

Instructions

1. Your job: write a groan-worthy roast of students at Cornell University

2. Bonus points for puns, rhymes, and one-liners

3. Don’t be mean

04:00

shinychat

shinychat in R

Start with the shinyapp snippet

library(shiny)
library(bslib)

ui <- page_fillable(

)

server <- function(input, output, session) {

}

shinyApp(ui, server)

shinychat in R

Load {shinychat} and {ellmer}

library(shiny)
library(bslib)
library(shinychat)
library(ellmer)

ui <- page_fillable(

)

server <- function(input, output, session) {

}

shinyApp(ui, server)

shinychat in R

Use the shinychat chat module

library(shiny)
library(bslib)
library(shinychat)
library(ellmer)

ui <- page_fillable(
  chat_mod_ui("chat")
)

server <- function(input, output, session) {
  chat_mod_server("chat")
}

shinyApp(ui, server)

shinychat in R

Create and hook up a chat client to use in the app

library(shiny)
library(bslib)
library(shinychat)
library(ellmer)


ui <- page_fillable(
  chat_mod_ui("chat")
)

server <- function(input, output, session) {
  client <- chat_openai()
  chat_mod_server("chat", client)
}

shinyApp(ui, server)

shinychat in Python

Start with the shinyapp snippet

from shiny import App, reactive, render, req, ui

app_ui = ui.page_fillable(
  ui.input_slider("n", "N", 0, 100, 20),
  ui.output_text_verbatim("txt"),
)


def server(input, output, session):
  @render.text
  def txt():
    return f"n*2 is {input.n() * 2}"


app = App(app_ui, server)

shinychat in Python

Remove the parts we don’t need

from shiny import App, ui

app_ui = ui.page_fillable(

)

def server(input, output, session):
    pass

app = App(app_ui, server)

shinychat in Python

Create a chatlas chat client

import chatlas
from shiny import App, ui

app_ui = ui.page_fillable(

)

def server(input, output, session):
    client = chatlas.ChatOpenAI()

app = App(app_ui, server)

shinychat in Python

Add the Chat UI and server logic (client and chat aren’t connected yet!)

import chatlas
from shiny import App, ui

app_ui = ui.page_fillable(
    ui.chat_ui("chat")
)

def server(input, output, session):
    client = chatlas.ChatOpenAI()
    chat = ui.Chat("chat")

app = App(app_ui, server)

shinychat in Python

When the user submits a message…

import chatlas
from shiny import App, ui

app_ui = ui.page_fillable(
    ui.chat_ui("chat")
)

def server(input, output, session):
    client = chatlas.ChatOpenAI()
    chat = ui.Chat("chat")

    @chat.on_user_submit
    async def _(user_input: str):
        # Send input to LLM
        # Send response back to UI

app = App(app_ui, server)

shinychat in Python

we’ll send the input to the LLM…

import chatlas
from shiny import App, ui

app_ui = ui.page_fillable(
    ui.chat_ui("chat")
)

def server(input, output, session):
    client = chatlas.ChatOpenAI()
    chat = ui.Chat("chat", client)

    @chat.on_user_submit
    async def _(user_input: str):
        response = await client.stream_async(user_input)
        # Send response back to UI

app = App(app_ui, server)

shinychat in Python

…and then stream the response back to the UI.

import chatlas
from shiny import App, ui

app_ui = ui.page_fillable(
    ui.chat_ui("chat")
)

def server(input, output, session):
    client = chatlas.ChatOpenAI()
    chat = ui.Chat("chat", client)

    @chat.on_user_submit
    async def _(user_input: str):
        response = await client.stream_async(user_input)
        await chat.append_message_stream(response)

app = App(app_ui, server)

⌨️ 06_word-games

Instructions

1. I’ve set up the basic Shiny app snippet and a system prompt.

2. Your job: create a chatbot that plays the word guessing game with you.

3. The twist: this time, you’re guessing the word.

07:00

Interpolation in R

library(ellmer)

words <- c("elephant", "bicycle", "sandwich")

interpolate(
  "The secret word is {{ sample(words, 1) }}."
)

[1] │ The secret word is elephant.

Interpolation in R

library(ellmer)

words <- c("elephant", "bicycle", "sandwich")

interpolate(
  "The secret word is {{ words }}."
)

[1] │ The secret word is elephant.
[2] │ The secret word is bicycle.
[3] │ The secret word is sandwich.

Interpolation in Python

import random

words = ["elephant", "bicycle", "sandwich"]

f"The secret word is {random.choice(words)}."

'The secret word is bicycle.'

Wrap-up

Recap

• LLM conversations happen via HTTP requests with messages that have roles
• {ellmer} and chatlas make it easy to converse with LLMs in R and Python
• Generative pre-trained transformers use the Transformer architecture and attention mechanism to generate text
• You can build chat apps using Shiny

Acknowledgments

• Materials derived in part from Programming with LLMs and licensed under a Creative Commons Attribution 4.0 International (CC BY) License.