Deploying models to the cloud

Lecture 16

Dr. Benjamin Soltoff

Cornell University
INFO 4940/5940 - Fall 2025

October 23, 2025

Announcements

Project draft

Learning objectives

Introduce a container model for deploying models as API
Define key terms for working with containers and Docker
Implement Docker containers for model APIs
Authenticate with cloud storage platforms for model versioning
Expand {vetiver} model APIs through additional metadata and endpoints

Where does {vetiver} work?

Enterprise software platforms such as Posit Connect and AWS SageMaker
A public or private cloud, using Docker

Docker

Containerized environments for your code

Virtual machine

Container

Benefits to containers

Lightweight
Portable
Consistent
Scalable
Secure

Why Docker?

Open source
Reproducible
Bring your own container philosophy

Create Docker artifacts

Start with a trained and versioned model

Dockerfile
Model dependencies, typically requirements.txt or renv.lock
File to serve API, typically app.py or plumber.R

Create Docker artifacts

Start with a trained and versioned model

R

vetiver_prepare_docker(
  board,
  "tompkins-housing",
  docker_args = list(port = 8080)
)

Python

vetiver.prepare_docker(
    board, 
    "tompkins-housing",
    port = 8080
)

Dockerfiles for {vetiver}

R

# Generated by the vetiver package; edit with care

FROM rocker/r-ver:4.4.0
ENV RENV_CONFIG_REPOS_OVERRIDE https://packagemanager.rstudio.com/cran/latest

RUN apt-get update -qq && apt-get install -y --no-install-recommends \
  libcurl4-openssl-dev \
  libicu-dev \
  libsodium-dev \
  libssl-dev \
  make \
  zlib1g-dev \
  && apt-get clean

COPY vetiver_renv.lock renv.lock
RUN Rscript -e "install.packages('renv')"
RUN Rscript -e "renv::restore()"
COPY plumber.R /opt/ml/plumber.R
EXPOSE 8080
ENTRYPOINT ["R", "-e", "pr <- plumber::plumb('/opt/ml/plumber.R'); pr$run(host = '0.0.0.0', port = 8080)"]

Dockerfiles for {vetiver}

Python

# # Generated by the vetiver package; edit with care
# start with python base image
FROM python:3.13

# create directory in container for vetiver files
WORKDIR /vetiver

# copy and install requirements
COPY vetiver_requirements.txt /vetiver/requirements.txt

#
RUN pip install --no-cache-dir --upgrade -r /vetiver/requirements.txt

# copy app file
COPY app.py /vetiver/app/app.py

# expose port
EXPOSE 8080

# run vetiver API
CMD ["uvicorn", "app.app:api", "--host", "0.0.0.0", "--port", "8080"]

Build your container

docker build -t housing .

If you have an Apple Silicon Mac

Add the --platform linux/amd64 to install R packages from compiled binaries rather than source.

Run your container

docker run -p 8080:8080 housing

Run your container

docker run --env-file .env -p 8080:8080 housing

Make predictions

R

endpoint <- vetiver_endpoint("http://0.0.0.0:8080/predict")
predict(endpoint, housing_test)

Python

endpoint = vetiver.vetiver_endpoint("http://0.0.0.0:8080/predict")
vetiver.predict(endpoint=endpoint, data=X_test)

Application exercise

`ae-15`

Instructions

Go to the course GitHub org and find your ae-15 (repo name will be suffixed with your GitHub name).
Clone the repo in Positron, run renv::restore() to install the required packages, open the Quarto document in the repo, and follow along and complete the exercises.
Render, commit, and push your edits by the AE deadline – end of the day

⏱️ Your turn

Activity

TODO check that we get a persistant local board on Posit Workbench

TODO make sure students use random ports for the output

Create a Docker container for your model using a local board board_local().

Build the Docker container and run it locally. Make predictions using the API.

07:00

pins 📌

The pins package publishes data, models, and other R and Python objects, making it easy to share them across projects and with your colleagues.

You can pin objects to a variety of pin boards, including:

a local folder (like a network drive or even a temporary directory)
Amazon S3
Azure Storage
Google Cloud

Use `board_gcs()` to connect to Google Cloud Storage

R

library(googleCloudStorageR)

board <- board_gcs(
  bucket = "info-4940-models",
  prefix = "<NETID>/"
)

Python

from pins import board_gcs

board = board_gcs("info-4940-models/<NETID>/", allow_pickle_read=True)

Authenticating with Google Cloud Storage

service-auth.json

R

.Renviron

GCS_AUTH_FILE="service-auth.json"

Python

.env

GOOGLE_APPLICATION_CREDENTIALS="service-auth.json"

Ensure Python loads .env

from dotenv import load_dotenv
import os

# Load environment variables from .env file
load_dotenv()

⏱️ Your turn

Activity

TODO test this

Create a Docker container for your model using a Google Cloud Storage board board_gcs().

Build the Docker container and run it locally. Make predictions using the API.

07:00

Building {vetiver} Docker artifacts

TODO is this the same for Python? I think not.

vetiver_prepare_docker() decomposes into two major functions:

vetiver_write_plumber() to create a Plumber file
vetiver_write_docker() to create a Dockerfile and {renv} lockfile

Requires additional tinkering with plumber.R and Dockerfile to work successfully

⏱️ Your turn

Activity

TODO see what needs to be done for Python

Create a Docker container for your model using a Google Cloud Storage board board_gcs().

Ensure the Docker container is correctly configured to use {googleCloudStorageR}.

Build the Docker container and run it locally. Make predictions using the API.

07:00

Docker resources

Model metrics as metadata 🎯

library(tidyverse)
library(tidymodels)

housing <- read_csv("data/tompkins-home-sales.csv")

set.seed(123)
housing_split <- housing |>
  mutate(price = log10(price)) |>
  initial_split(prop = 0.8)
housing_train <- training(housing_split)
housing_test <- testing(housing_split)

rf_rec <- recipe(
  price ~ beds + baths + area + year_built,
  data = housing_train
) |>
  step_impute_mean(all_numeric_predictors()) |>
  step_impute_mode(all_nominal_predictors())

housing_fit <- workflow() |>
  add_recipe(rf_rec) |>
  add_model(rand_forest(trees = 200, mode = "regression")) |>
  fit(data = housing_train)

import pandas as pd
import numpy as np
from sklearn import model_selection, ensemble

housing = pd.read_csv('data/tompkins-home-sales.csv')

np.random.seed(123)
X, y = housing[["beds", "baths", "area", "year_built"]], np.log10(housing["price"])
X_train, X_test, y_train, y_test = model_selection.train_test_split(
    X, y,
    test_size = 0.2
)

housing_fit = ensemble.RandomForestRegressor(n_estimators=200).fit(X_train, y_train)

Model metrics as metadata

R
Python

housing_metrics <- augment(housing_fit, new_data = housing_test) |>
  metrics(truth = price, estimate = .pred)

housing_metrics

# A tibble: 3 × 3
  .metric .estimator .estimate
  <chr>   <chr>          <dbl>
1 rmse    standard       0.195
2 rsq     standard       0.457
3 mae     standard       0.142

from sklearn import metrics

metric_set = [metrics.root_mean_squared_error, metrics.r2_score, metrics.mean_absolute_error]
y_predictions = pd.Series(housing_fit.predict(X_test))

housing_metrics = pd.DataFrame()

for metric in metric_set:
    metric_name = str(metric.__name__)
    metric_output = metric(y_test, y_predictions)
    housing_metrics = pd.concat(
        (
            housing_metrics,
            pd.DataFrame({"name": [metric_name], "score": [metric_output]}),
        ),
        axis=0,
    )

housing_metrics.reset_index(inplace=True, drop=True)
housing_metrics

                      name     score
0  root_mean_squared_error  0.195166
1                 r2_score  0.493361
2      mean_absolute_error  0.142366

Model metrics as metadata

R
Python

library(vetiver)
v <- vetiver_model(
  housing_fit,
  "tompkins-housing",
  metadata = list(metrics = housing_metrics)
)
v


── tompkins-housing ─ <bundled_workflow> model for deployment 
A ranger regression modeling workflow using 4 features

from vetiver import VetiverModel
v = VetiverModel(
    housing_fit, 
    "tompkins-housing", 
    prototype_data = X_train,
    metadata = housing_metrics.to_dict()
)
v.description

'A scikit-learn RandomForestRegressor model'

Model metrics as metadata

We pin our vetiver model to a board to version it
The metadata, including our metrics, are versioned along with the model

R
Python

library(pins)
board <- board_gcs(
  bucket = "info-4940-models",
  prefix = "<NETID>/"
)
board |> vetiver_pin_write(v)

from pins import board_connect
from vetiver import vetiver_pin_write
from dotenv import load_dotenv
load_dotenv()

board = board_gcs("info-4940-models/<NETID>/", allow_pickle_read=True)
vetiver_pin_write(board, v)

⏱️ Your turn

Activity

Compute metrics for your model using the testing data.

Store these metrics as metadata in a vetiver model object.

Write this new vetiver model object as a new version of your pin.

05:00

Model metrics as metadata

How do we extract our metrics out to use them?

R
Python

extracted_metrics <- board |>
  pin_meta("tompkins-housing") |>
  pluck("user", "metrics") |>
  as_tibble()

extracted_metrics

# A tibble: 3 × 3
  .metric .estimator .estimate
  <chr>   <chr>          <dbl>
1 rmse    standard       0.195
2 rsq     standard       0.457
3 mae     standard       0.142

metadata = board.pin_meta("tompkins-housing")
extracted_metrics = pd.DataFrame(metadata.user.get("user"))
extracted_metrics

                      name     score
0  root_mean_squared_error  0.195166
1                 r2_score  0.493361
2      mean_absolute_error  0.142366

⏱️ Your turn

Activity

Obtain the metrics metadata for your versioned model.

What else might you want to store as model metadata?

How or when might you use model metadata?

07:00

Extensions

Wrap-up

Recap

Docker containers are a lightweight, portable, and consistent way to deploy code
Vetiver can help you create Docker artifacts for your models
Model metrics can be stored as metadata in {vetiver} models

Acknowledgments

Materials derived in part from Intro to MLOps with {vetiver} and licensed under a Creative Commons Attribution 4.0 International (CC BY) License.

Deploying models to the cloud

Announcements

Announcements

Learning objectives

Where does {vetiver} work?

Docker

Virtual machine

Container

Benefits to containers

Why Docker?

Create Docker artifacts

Create Docker artifacts

R

Python

Dockerfiles for {vetiver}

R

Dockerfiles for {vetiver}

Python

Build your container

Run your container

Run your container

Make predictions

R

Python

Application exercise

ae-15

⏱️ Your turn

pins 📌

Use board_gcs() to connect to Google Cloud Storage

R

Python

Authenticating with Google Cloud Storage

R

Python

⏱️ Your turn

Building {vetiver} Docker artifacts

⏱️ Your turn

Docker resources

Model metrics as metadata 🎯

Model metrics as metadata

Model metrics as metadata

Model metrics as metadata

Model metrics as metadata

⏱️ Your turn

Model metrics as metadata

⏱️ Your turn

Extensions

Wrap-up

Recap

Acknowledgments

`ae-15`

Use `board_gcs()` to connect to Google Cloud Storage