Document models

Lecture 14

Dr. Benjamin Soltoff

Cornell University
INFO 4940/5940 - Fall 2025

October 16, 2025

Announcements

Announcements

TODO

Learning objectives

  • Review the MLOps lifecycle
  • Identify audiences for model documentation
  • Define a model card
  • Describe the major sections of a model card
  • Utilize a model card for a foundational LLM

You’ve built a model - now what?

A flowchart diagramming the machine learning operations lifecycle, including collecting data, understanding and cleaning data, training and evaluating models, deploying model, and monitoring model.

Documentation and deployment

Documentation

  • Document your model and its performance
  • Document the data used to train the model
  • Document the model’s assumptions and limitations

Deployment

  • Deploy the model to a production environment
  • Monitor the model’s performance in production
  • Update the model as needed

Documentation

Who is documentation for?

  • Yourself
  • Peers
  • Manager/supervisor
  • Clients
  • Jane Q Public

What to document

  • Data collection/provenence
  • Exploratory analysis
  • Feature selection
  • Model tuning/selection
  • Model evaluation

How to document

File types

  • Literate programming (aka Quarto notebooks)
  • Scripts with code comments
  • README files

Version control

  • Git and GitHub
  • Commit history
  • Branches

Deployment

Deployment is the process of integrating a trained ML model into a production environment so it can be used to make predictions or decisions.

  • Batch processing
  • Web app
  • iOS/Android app
  • API

Deployment is often in collaboration with software engineers, but you should be able to informatively participate or demo the product.

Definition: In-depth Guide to Machine Learning (ML) Model Deployment

Common deployment methods

  • Batch processing locally
  • Web application with Shiny
  • API with {plumber}/fastapi and {vetiver}
  • Export model to implement in different programming language (e.g. SQL, Java, C, C++, Python, Ruby)

Documenting models

Why document models?

  • ML practitioners
  • Model developers
  • Software developers
  • Policymakers
  • Organizations
  • ML-knowledgable individuals
  • Impacted individuals

Model cards

A screenshot of a model card called 'Smiling Detection in Images'.

Source: “Model Cards for Model Reporting”

Major sections

  • Model details
  • Intended use
  • Factors
  • Metrics
  • Evaluation data
  • Training data
  • Quantitative analyses
  • Ethical considerations
  • Caveats and recommendations

Model details

  • Person or organization developing the model
  • Model date
  • Model version
  • Model type
  • More information
  • Citation
  • License
  • Contact info

Intended use

  • Primary intended uses
  • Primary intended users
  • Out-of-scope uses

Factors

  • Groups
  • Instrumentation
  • Environment
  • Relevant factors
  • Evaluation factors

Metrics

  • Model performance measures
  • Decision thresholds
  • Approaches to uncertainty and variability

Evaluation data

  • Datasets
  • Motivations
  • Preprocessing

Training data

Same info if possible. If not, summary statistics are good.

Quantitative analyses

  • Disaggregated based on chosen factors
  • Model performance
  • Unitary/intersectional results

Ethical considerations

  • Sensitive data
  • Human life
  • Mitigation strategies
  • Risks and harms
  • Use cases

Caveats and recommendations

  • Is further testing necessary?
  • Were relevant groups excluded from the evaluation?
  • Additional recommendations for model use?

Application exercise

📝 Documenting published LLMs

Instructions

Use the Llama 3.2 model card on Hugging Face to answer the questions on the handout.

20:00

Wrap-up

Recap

  • You are not finished once you’ve built a model
  • Model cards are a way to document models for various audiences
  • They contain information such as model details, intended use, and evaluation criteria