Report

Your written report must be completed in the report.qmd.

Before you finalize your write up, make sure the printing of code chunks is off with the option echo: false in the YAML.

The mandatory components of the report are below. You are free to add additional sections as necessary. The report, including visualizations, should be no more than 10 pages long. There is no minimum page requirement; however, you should comprehensively address all of the analysis in your report.

Be selective in what you include in your final write-up. The goal is to write a cohesive narrative that demonstrates a thorough and comprehensive analysis rather than explain every step of the analysis.

You are welcome to include an appendix with additional work at the end of the written report document; however, grading will largely be based on the content in the main body of the report. You should assume the reader will not see the material in the appendix unless prompted to view it in the main body of the report. The appendix should be neatly formatted and easy for the reader to navigate. It is not included in the 10-page limit.

Report components

Introduction

Identify the project motivation, data, and objectives. What is the context of the work? What problem are you trying to solve?

Justification of approach

Describe the product(s). What did your team create? Who is the intended audience? How will the product(s) meet their needs?

Design process

Summarize your design process for the product(s). Explain the key design challenges you encountered in creating the main product(s). What were the most important considerations your team faced in designing and constructing the final product?

Limitations

Assess the limitations of your work. What hurdles did you fail to overcome? If you had the opportunity to do this again, how would you improve on your product(s)?

Generative AI reflection

As stated in the syllabus, include a written reflection of how you used GAI tools (e.g. what tools you used, how you used them to assist you), what skills you believe you acquired, and how you believe you demonstrated mastery of the learning objectives. This should be a thorough relfection since you are using LLMs extensively for the project, not only (potentially) as a tool to help write code but also as a core component of your project. We want a thoughtful reflection of your experience using GAI tools in the context for this project.

Organization + formatting

While not a separate written section, you will be assessed on the overall presentation and formatting of the written report. A non-exhaustive list of criteria include:

• The report neatly written and organized with clear section headers and appropriately sized figures with informative labels.
• Numerical results are displayed with a reasonable number of digits, and all visualizations are neatly formatted.
• All citations and links are properly formatted.
• If there is an appendix, it is reasonably organized and easy for the reader to find relevant information.
• All code, warnings, and messages are suppressed.
• The main body of the written report (not including the appendix) is no longer than 10 pages.

Evaluation criteria

Category	Less developed projects	Typical projects	More developed projects
Introduction	Less focused and organized. May jump to technical details without explaining why the project is important. Research questions or project objectives are not clearly stated.	Provides background information and context. Introduces key terms and data sources. Outlines research question(s) or project objectives clearly.	All expectations of typical projects + clearly describes why the problem is important and what is at stake. Even if the reader doesn’t know much about the subject, they understand why they should care about the project and its outcomes. Compelling motivation for using LLMs to address this problem.
Justification of approach	Simple description of deliverables with little justification. Unclear who the intended audience is or how the product meets their needs. Minimal explanation of why LLMs are appropriate for this problem.	Clearly defines the deliverable(s) and intended audience. The chosen approach is explained and justified. Explains how LLMs are used and why they are appropriate for the problem.	All expectations of typical projects + demonstrates deep understanding of alternative approaches and clearly articulates why the chosen approach is superior. Thoughtful consideration of how product design meets specific user needs. Strong justification for LLM selection and usage patterns.
Design process	Design process is not clearly summarized or is missing key details. Hard to understand what design choices were made. Little discussion of prompt engineering or LLM integration challenges.	Summarizes the design process including key decisions made. Addresses major design challenges encountered. Discusses prompt engineering approach and LLM integration.	All expectations of typical projects + comprehensive discussion of design process including decision points and alternative paths not taken. Clear explanation of how the team iterated on prompts and LLM integration. Demonstrates thoughtful consideration of trade-offs and design constraints.
Data description	Data sources are poorly documented or not mentioned. Little consideration for data quality, ethics, or provenance. No discussion of preprocessing or data preparation.	Describes data sources in sufficient detail including origin and collection methods. Acknowledges data quality considerations and ethical concerns. Explains preprocessing steps taken.	All expectations of typical projects + credits and thoroughly documents data sources. Critically examines data quality, biases, and limitations. Thoughtful discussion of ethical considerations in data collection and use. Clear explanation of how data preparation decisions impact LLM performance.
Limitations	Limitations are not explained in depth or are missing. No mention of how limitations affect results or product quality. Little awareness of potential harms or biases.	Identifies reasonable limitations to the scope of work. Addresses potential biases in data or LLM outputs. Acknowledges common LLM limitations (hallucination, bias, etc.). Proposes potential remedies in future iterations.	All expectations of typical projects + creatively identifies potential harms, data gaps, and model limitations. Describes how these could affect the meaning of results and impact on users. Design process clearly shows how the team considered and worked to minimize limitations. Thoughtful discussion of responsible AI considerations.
Generative AI reflection	Minimal or superficial reflection on GAI tool usage. Does not address skills acquired or mastery of learning objectives. Treats GAI as just a coding assistant.	Provides adequate reflection on GAI tools used and how they were applied. Discusses skills acquired and connection to learning objectives. Acknowledges both benefits and challenges of using LLMs.	All expectations of typical projects + thoughtful, comprehensive reflection demonstrating deep learning about LLM capabilities and limitations. Insightful analysis of how GAI usage evolved throughout the project. Clear articulation of how experience relates to broader AI/LLM landscape and responsible AI practices. Strong connection to course learning objectives.
Organization & formatting	Report is poorly organized or difficult to follow. Figures are poorly formatted or lack labels. Code, warnings, or messages are visible. Exceeds 10-page limit or lacks professional presentation.	Report is neatly organized with clear section headers. Figures are appropriately sized with informative labels. Numerical results use reasonable precision. All code, warnings, and messages are suppressed. Within 10-page limit.	All expectations of typical projects + exceptional presentation quality with polished formatting throughout. Effective use of visual hierarchy and white space. Professional-grade figures and tables. Appendix (if present) is well-organized and easy to navigate. Citations and links are properly formatted. Document reads as a cohesive narrative.