Teacher Beliefs and Attitudes Toward STEM (T-STEM) Survey: Tips for Using Your Data
What the T-STEM Survey Measures
The T-STEM survey invites teachers to give information about: their self-efficacy for teaching; their belief that teachers affect student learning; how often students use technology; how often they use certain STEM instructional practices; their attitudes toward 21st century learning; their attitudes toward teacher leadership; and their awareness of STEM careers.
The Research/Evaluation/Continuous Improvement Process
These 5 steps are a useful process by which individuals/teams can organize efforts to learn about the impact of their program on participating students. (1) Develop Research Questions: determine important questions you want to know about program activities and impacts. (2) Identify Data: data that can help answer those questions, anything from a set of informal conversations with participants to student test scores. (3) Collect Data: administer instruments and gather data. (4) Analyze Data: Figure out the themes or patterns, or results, in the data. (5) Interpret Results: Decide what those themes or patterns mean about the program.
Asking Important Questions about Your Program
If you do not already have question(s) that are important to you, or if you would like to brainstorm other possible questions, consider reflecting on your program’s mission statement, logic model, or questions that other groups have (parents, school boards, funders, etc.). What do you want to know about your program?
Keep in mind that it can be much more informative and critical to uncover the outcomes of your program, instead of the outputs. Outputs are easily measurable products of a program’s activities, often expressed in units such as hours of training provided, numbers of people participating, numbers of completed actions, etc. Outcomes are the impacts of a program’s activities, often expressed in terms of changes in behaviors, beliefs, or practices. Outcomes are usually measures of the broader, deeper goals of a program. For example, the number of science teachers enrolled in a summer professional development program is an output, the change in those teachers’ instructional practices is an outcome.
Results from the T-STEM Survey give information on teacher attitudes toward STEM from only one angle: a survey. Consider collecting other information to see if the results are similar – if they point to a common conclusion. Interviews or observations, for example, are other angles from which you can try to understand teacher attitudes toward STEM.
NOTE: If you are receiving T-STEM Survey data from the Friday Institute, it will not contain any identification of students. This is because the Friday Institute is an organization outside of the school system and cannot identify any individuals who responded to any questions; it can only report grouped/aggregated results.
Don’t forget to allow time for this. If you have raw data, especially survey data, test score data, or other sets of numbers, you will most likely have to make sure all of your numbers are correct and in the right place in your spreadsheet before you analyze them. NOTE: save copies of all original datasets and spreadsheets before doing any cleaning.
Common data cleaning steps include: fixing formatting when downloading data from one program into another (e.g., from Survey Monkey into Microsoft Excel); deleting empty rows or columns for people who may not have usable information (e.g., if they started but did not complete the survey); removing any personally identifying information or combinations of information, like name or school ID number; removing any data you won’t use in your analysis.
Looking for Themes/Patterns in the Data, or “Analysis”
The T-STEM Survey has 7 sections (see companion document “Teacher Beliefs and Attitudes Toward STEM Survey: Development and Psychometric Properties”). To get the most thorough measure of teacher attitudes, the questions in all 7 sections of the survey should be summarized together, by section. For example, to get the most thorough understanding of a single teacher’s, say, “Teaching Outcome Expectancy Beliefs” their responses to all 9 questions from the section “Teaching Outcome Expectancy Beliefs” should be averaged. Assign the value of “1” every time a teacher responds “strongly disagree;” “2” for “disagree;” “3” for “neither agree nor disagree;” “4” for “agree;” and “5” for “strongly agree.” Average those numbers together for all of the teacher’s responses in the section and you’ll have a “score” for that teacher for their “Teaching Outcome Expectancy Beliefs” – the higher their score, the more they believe student learning can be impacted by teacher actions. You can average all teachers’ scores together to get a score for, say, a whole school (as depicted below).
|Survey Section||Science Teachers
|Personal (STEM) Teaching Efficacy and Beliefs||4.0||4.1||4.0|
|(STEM) Teaching Outcome Expectancy||3.4||3.4||3.4|
You can also compare answers to individual survey items that you find interesting, however strong conclusions about a teacher’s attitude should not be made from their responses to just a single question. The surveys have been validated at the construct-level, not at the item-level; therefore it is most recommended that comparisons be made at the construct-level.
NOTE: Most survey questions are positively worded, like “I know what to do to increase students’ interest in math.” A few, however, are negatively worded, like “I wonder if I have the necessary skills to teach math.” The negatively worded questions must be assigned values in the reverse order of all the other questions (“5” for strongly disagree, “4 for disagree,” etc.), since agreement to those questions represents an attitude opposite of the attitude for agreement with the other questions. Refer to the companion document mentioned above for a list of negatively worded questions.
You can also combine response options and report percentages of teachers who were generally positive (e.g., responded either “Agree” or “Strongly Agree”), for example.
|Career Area||Proportion “Agree/Strongly Agree”|
|I am continually finding better ways to teach (STEM subject)||95.6%||96.8%||96.3%|
|I know the steps necessary to teach (STEM subject) effectively.||89.1%||95.8%||92.1%|
More Analysis Tips
- Consider common ways to summarize data points – mean, range, frequency counts, percentages, median, and mode.
- Report the number of responses related to summarized numbers (“n” for subsample; “N” for the entire population).
- All analyses discussed in this guide can be done in the common software application Microsoft Excel.
- Create levels of units that reflect what you want to know – do you want student-, classroom-, or school-level results?
- Focus on comparisons – by sub-groups of students (gender, race/ethnicity) or time (beginning vs. end of year).
Reporting data is a balance between showing every single point of data and combining the points of data (averages, combined response options like agree/strongly agree, etc.). You have to decide what is best. It is important to give the audience a clear picture and not to be misleading, while also summarizing the data so the main themes can be easily understood and the presentation is not overwhelming. Do not be afraid to point-out limitations in the data.
Pie chart: Show smaller groups in one larger group (e.g. the number of males and females in a program).
Bar chart: Show different findings, organized by categories, within a group or across groups.
Line graph: Show how results change over time, within a group or across groups.
Using and Sharing Data
Results – the patterns and themes found – can be used to decide to take new action in your school or program; to assess the degree to which goals are being met; to inform others about impacts of the program, including program participants, program staff, parents, school boards, etc.; to complete grant proposals; etc. Importantly, remember to share results with those who provided the information (who responded to surveys, etc.) – this helps promote buy-in and future collaboration.