Document models

Lecture 14

Dr. Benjamin Soltoff

Cornell University
INFO 4940/5940 - Fall 2025

October 16, 2025

Announcements

Announcements

• No homework this week
• Continue working on your project 01 EDA

General feedback on proposals

• Gradescope team submissions

• This is a machine learning course - an ML model must be at the core of your project

• Most teams will deploy their model via API for their deliverable - that’s fine

Learning objectives

• Review the MLOps lifecycle
• Identify audiences for model documentation
• Define a model card
• Describe the major sections of a model card
• Utilize a model card for a foundational LLM

You’ve built a model - now what?

Deploy & monitor

Document

• Your model and its performance
• The data used to train the model
• The model’s assumptions and limitations

Deploy

• Deploy the model to a production environment
• Monitor the model’s performance in production
• Update the model as needed

Documentation

Who is documentation for?

• Yourself
• Peers
• Manager/supervisor
• Clients
• Jane Q Public

What to document

• Data collection/provenence
• Exploratory analysis
• Feature selection
• Model tuning/selection
• Model evaluation

How to document

File types

• Literate programming (aka Quarto notebooks)
• Scripts with code comments
• README files

Version control

• Git and GitHub
• Commit history
• Branches

Deployment

Deployment is the process of integrating a trained ML model into a production environment so it can be used to make predictions or decisions.

• Batch processing
• Web app
• iOS/Android app
• API

Deployment is often in collaboration with software engineers, but you should be able to informatively participate or demo the product.

Common deployment methods

• Batch processing locally
• Web application with Shiny
• API with {plumber}/fastapi and {vetiver}
• Export model to implement in different programming language (e.g. SQL, Java, C, C++, Python, Ruby)

Documenting models

Model cards

Major sections

• Model details
• Intended use
• Factors
• Metrics
• Evaluation data
• Training data
• Quantitative analyses
• Ethical considerations
• Caveats and recommendations

Model details

Basic metadata such as

• Person or organization developing the model
• Model date
• Model version
• Model type
• More information
• Citation
• License
• Contact info

Intended use

What should the model be used for? What should it not be used for?

• Primary intended uses
• Primary intended users
• Out-of-scope uses

Factors

Summary of model performance across relevant factors

• Groups
  • Race/sex/ethnicity
  • Marginalized minority groups
• Instrumentation
  • Camera type
  • Backgrounds
  • Microphone
• Environment
  • Lighting conditions
  • Humidity

Metrics

Not the metrics themselves, but how the model was evaluated

• What metrics did you use and why?
• Decision thresholds
• Approaches to uncertainty and variability

Evaluation data

Details about how model was evaluated

• Datasets
• Motivations
• Preprocessing

Training data

Same info if possible. If not, summary statistics are good.

Quantitative analyses

Report metrics of performance

• Overall model performance
• Disaggregated based on chosen factors

Ethical considerations

Document ethical analysis performed during model evaluation

• Sensitive data
• Human life
• Mitigation strategies
• Risks and harms
• Use cases

Caveats and recommendations

• Is further testing necessary?
• Were relevant groups excluded from the evaluation?
• Additional recommendations for model use?

Application exercise

📝 Documenting published LLMs

Instructions

Use the Llama 3.2 model card on Hugging Face to answer the questions on the handout.

20:00

Wrap-up

Recap

• You are not finished once you’ve built a model
• Model cards are a way to document models for various audiences
• They contain information such as model details, intended use, and evaluation criteria