It is increasingly important to train novice innovators in the skills they need to lead the design and research of solutions to today’s most pressing problems. A fundamental skill that innovators need is a way to evaluate the underlying rationale of their proposed solutions, a practice that experts use to assess and improve their designs. By identifying key issues in design rationale, innovators can focus efforts on addressing these issues, thereby improving their designs through iterative planning.

However, prior literature and our own needfinding demonstrate that novices leading creative work like design struggle to evaluate their design rationale and identify critical issues on which to focus their plans. Consequently, novices often focus on surface- level issues when independently evaluating their own designs, or they must rely heavily on limited mentoring resources to identify critical issues. While scaffolding and mentoring interventions can support novices as they evaluate and improve their designs, existing scaffolds do not focus on improving the design argumentation that is the foundation of a design’s rationale. Further, we’ve observed that mentors often focus their coaching efforts on scaffolding students to evaluate design arguments and identify critical issues, leaving few mentoring cycles to train novices on how to address issues and improve designs.

To overcome these challenges, we contribute Polaris, an assessment scaffold that supports student design researchers in constructing and evaluating design arguments for critical issues in their design rationale (see figure). Polaris embeds expert guidance for constructing and identifying issues within design rationale via a design argument template and reflection prompts derived from expert process and practice. The design argument template represents four components of a design argument: outcomes, obstacles, characteristics, and arguments, whose relationships are emphasized by conceptual links. The reflection prompts for each component of the design argument help students evaluate for potential issues that novices might miss but mentors often catch. Together, Polaris’ template and prompts help students evaluate their design arguments by making it easier for them to visually and procedurally identify core structural issues within and across the components of their design argument. We hypothesize that this approach can help students independently identify critical issues in their design arguments, and help mentors devise strategies for helping students address them.

A study of Polaris with 13 undergraduates leading 10 independent research projects and 3 research mentors demonstrates that students using Polaris were able to identify structural issues in their design rationale that mentors considered critical and actionable for coaching discussions. 95.6% of the 91 issues students identified with Polaris were focused on core design argument components, and 96.7% of identified issues noted gaps, misalignment, or a lack of depth in their design rationale. Our findings also demonstrate that students identified issues that their mentors considered critical, and that the issues they identified were generally similar to what their mentor identified. Further, mentors found issues students identified useful for making sense of a student’s understanding, and devising strategies to help students plan to address issues, or overcome misunderstandings of issues prior to planning.