Model card

The model card is a document that accompanies a trained machine learning model. It provides a summary of the model’s performance and limitations, as well as the context in which it was trained and used. The model card should be written in the model-card.qmd file in your project repo using the template we learned in class and practiced for homework.

Evaluation criteria

Category	Less developed projects	Typical projects	More developed projects
Model details	Lacks basic information regarding the model version, type, and other relevant details.	Answers basic questions regarding the model version, type, and other relevant details.	All expectations of typical projects + questions are answered thoroughly.
Intended use	Unclear why the model was created and/or how it should be used.	Explains why the model was created and how it should and should not be used.	All expectations of typical projects + questions are answered thoroughly.
Factors	Does not identify relevant factors for contextualizing the model. Identified factors are artificial or superfluous.	Identifies relevant aspects or factors that influence how users contextualize the model.	All expectations of typical projects + demonstrates serious thought and consideration to relevant factors.
Metrics and data sets	Metrics are unstated or inappropriate for the model. Unclear what data was used to train and/or evaluate the model.	Clearly states the metrics used to evaluate the model. Identifies the data used to train the model. Identifies the data used to evaluate the model.	All expectations of typical projects + questions are answered thoroughly.
Quantitative analyses	Doesn’t report overall or disaggregated metrics. Metrics are reported in a shoddy or unclear manner.	Reports overall metrics of model performance using the evaluation data. Metrics are disaggregated based on relevant factors previously identified.	All expectations of typical projects + effectively uses tables/figures to communicate results of analyses. Figures and tables are properly labeled. Figures are aesthetically pleasing and clearly interpretable.
Ethical considerations	Fails to seriously address ethical considerations.	Identifies ethical considerations that went into model development.	All expectations of typical projects + demonstrates serious thought and consideration to relevant considerations.
Caveats & recommendations	Caveats are superficial or meaningless. Recommendations for how the model should be used to make decisions are missing or unclear.	Describes appropriate caveats or remaining concerns to using the model. Articulates recommendations for how the model should be used to make decisions.	All expectations of typical projects + demonstrates serious thought and consideration to relevant caveats and recommendations.