The CS Concepts Inventory is intended to measure students’ understanding of the four core concepts of CS—variables, conditionals, loops and algorithms— taught at the middle school level. Additionally, we incorporated the concepts of debugging, comprehension and development into the assessment. The assessment was guided by a conceptual framework informed by a Focal Knowledge, Skills and Abilities—FKSAs framework developed by Grover and Basu (2017), the K-12 CS Framework (K–12 Computer Science Framework, 2016) and the Computer Science Teachers Association (CSTA) Standards (CSTA, 2017). The assessment utilizes elements from a block-based programming environment as the context for every question, based on findings that suggest learners, especially novice ones, experience less conceptual and cognitive difficulties using these tools (e.g., Grover, Pea & Cooper, 2015; Robins, Rountree, & Rountree, 2003).