It is common to make one of two mistakes when considering the role of analysis in the data use cycle. The first is to assume that good data analysis requires high level statistical skills. The second is to assume that no analytical skills are necessary for good data use. A better paradigm is to leverage what skills and resources are available while prioritizing the importance of organizational learning. This better paradigm highlights the importance of data literacy. 

The EDC Oceans of Data Institute’s Building Global Interest in Data Literacy: A Dialogue Workshop Report (2016) defines data literacy as follows, “The data-literate individual understands, explains, and documents the utility and limitations of data by becoming a critical consumer of data, controlling his/her personal data trail, finding meaning in data, and taking action based on data. The data-literate individual can identify, collect, evaluate, analyze, interpret, present, and protect data.”

Developing data literacy skills while leveraging what resources a program already has will allow for continuous growth for both the program and the analysis of program data. This metacognition of learning to learn also creates an openness to learning and continued growth. 

It is recommended that programs invest in faculty and staff professional development around data literacy and analytics, as well as providing dedicated funding that will support the hiring of individuals that are comfortable with the skills necessary for advanced statistical analyses. However, even with a few basic skills, individuals that are responsible for reporting data should be able to confidently provide an adequate analysis of program and candidate-level data. There are a few key concepts that each program should build into their analysis framework from the beginning:

• Conduct a Root Cause Analysis (RCA) to support in discovering the root causes of outcomes with the goal of identifying appropriate solutions. The National School Reform Faculty Harmony Education Center (2017) provides this tool to support "The 5 Whys for Inquiry." We recommend this protocol to assist your team in conducting an RCA.
• The importance of validity and reliability of data (see the introduction to this toolkit) and the limitations of interpretation when they exist.
• Determining when qualitative or quantitative methodologies are more appropriate.
• Understanding when summative versus formative data is most appropriate.
• Ensuring interpretation aligns to the degree to which data is direct versus indirect.