Here’s an interesting fact: Children spend 80% of their waking hours beyond school. At Boston After School & Beyond, they’re making those hours count through a citywide, year-round learning system for Boston youth.
What is Boston After School & Beyond?
Boston After School & Beyond is a nonprofit intermediary founded in 2005 that connects school, after-school, and summer learning so that young people can develop to their full potential. Specifically, they help catalyze partnerships among the City of Boston, Boston Public Schools (BPS), funders, researchers,and our network of 359 programs serving 27,934 K-12 students.
Data Drives Improvement
Boston Beyond has a national reputation for being particularly data oriented. Every after school and summer program in their network participates in measurement, or data collection, to facilitate improvement in program practice and student skill development.
“All data collection focuses on answering three key questions – whom did we serve? Are we providing high quality experiences for students? Did students experience skill growth?” —Lisa Gomi Hui, Director of Measurement & Improvement
Boston Beyond’s four-person Measurement & Improvement team is responsible for collecting data from each of their 359 programs, cleaning the data, and presenting it so that programs, Boston Public Schools, the City of Boston, and researchers can get answers to these three key questions.
The Measurement & Improvement team’s challenge was to scale up their data collection, cleaning, and presenting for their rapidly growing network, going from 183 programs in 2017 to 359 programs in 2020 while maintaining their small team size. Hui explains:
“First, we needed get data back to our program partners more quickly. We wanted partners to be able to review their data and apply it to program improvements right away, but it was taking us up to four months to get the data report back to them. Our small team was busiest in the seasons when data was needed most (end of school year / beginning of summer, end of summer / beginning of school year).”
“The Excel-based PDF data report we were using was outsourced to a contractor that was increasingly taking longer and longer to produce accurately. We hired a Data Architect dedicated to our data infrastructure and built a data dashboard using Microsoft Power BI, but we still had an enormous amount of data to clean and return within a few weeks. (Data included student enrollment, attendance, four different youth surveys, two different observation ratings, and a staff survey. As an example, one of the observation rating tools used at nearly all participating programs use has 394 data fields per observation – and we coordinate two observations during the school year and one during the summer.)”
“Second, staff time was overwhelmingly being spent on coordinating measurement activities like survey administration and data collection, but not on improvement activities like reviewing data with partners.”
“Given how onerous the data cleaning load was (especially with only Microsoft Excel and IBM SPSS at our disposal), we never had time to organize data review sessions with partners before the next school year or summer required our attention for measurement logistics.”
“Lastly, we were struggling to meet the data reporting needs of all our different stakeholders. The Boston Public Schools, funders, and researchers all wanted data about the programs in our network but often in very specific, different arrangements. Some needed this data annually; others were more ad hoc. All of our program and student information is housed in Salesforce, which helped with producing basic reports. However, our stakeholders often wanted data presented in ways that required significant outside calculating or editing outside of Salesforce, which we were doing in Excel or SPSS. This work kept our staff away from the core work of measuring and improving programs.”
Four Ways Alteryx Has Helped Solve Boston Beyond’s Challenges
“In June 2019, our data architect told us about Alteryx. He had seen it used at his previous job by an analytics team and thought it could be a helpful tool to solve our problems. He was right,” said Hui.
Fast
When cleaning data with Microsoft Excel or IBM SPSS, we had to rely on point-and-click or SPSS syntax, and the limited data manipulation tools (e.g. VLOOKUP, Filters, Pivot Tables etc.) Cleaning data sources this way often took days of multiple staff people’s time. With Alteryx, once you built a workflow, the workflow itself took seconds.
Forgiving
Partly why data cleaning took so long was because mistakes in Microsoft Excel or IBM SPSS were often not undoable. When we made mistakes in data cleaning in the middle of the process that we couldn’t control-Z our way out of, we had to close the file and start over. We’d even resorted to saving the file every so often at intermediate stages, leading to cluttered and confusing folders. With Alteryx, we could edit the workflow and run the workflow as many times as we needed until we got our final desired output.
Intuitive
Although each member of the M&I team is comfortable with interpreting and sharing data, only our data architect knew programming languages to quickly edit data and maintain databases. Alteryx was like giving programming ability to those who did not know programming languages. The tool palette and the workspace made intuitive sense, and once we understood the tools, we could easily edit data as needed.
Documented
As our team grew and our program partner network grew, we needed to document processes better. With Excel or SPSS, we would have to write documentation in a separate file and the person reviewing would need to look at the file side-by-side with the documentation. With Alteryx, the entire process was laid out in the workflow – and with the Comment tool, you could clarify any parts of the process.
Measures are Only as Good as the Change They Inspire
“In the first two months of Alteryx, our data architect figured out how to solve our first challenge of cleaning data in an accurate and timely way for program partners. For every data source, he built an extensive workflow that led to a final, cleaned output – all field names consistent with past files, correct organization and program names, ID numbers, affiliated initiatives or research projects. Instead of taking days to clean the file (and have to do it over and over at the end of each semester), now we could take a given file and clean it in a matter of minutes and have it write directly into our SQL database for our data dashboard,” said Hui.
“We went from needing four months to four weeks to return an accurate, complete data dashboard to all 359 of our program partners.”
—Lisa Gomi Hui, Director of Measurement & Improvement
“Doing this had the added benefit of freeing up other staff time. Previously, the entire team took on cleaning data, checking it for accuracy, before I would conduct analysis. By automating the process through these workflows, coordinators could now spend time on what we call “data coaching” calls, or one-on-one opportunities to review the data dashboard and plan improvements. Once we had Alteryx in place and didn’t need to use all staff for data cleaning, we were able to pilot data coaching in August 2019. Two coordinators ended up doing 25 review calls; in August 2020 this almost doubled to 42 review calls,” said Hui.
“Within three months of acquiring Alteryx, I started using it to build workflows for the custom data reports requested by our research partners, the Boston Public Schools, and funders. Once built, I could re-run it for stakeholders in seconds and provide daily or weekly updates as needed. This became especially important in 2020 with the COVID-19 pandemic.”
“With almost all Boston Public School students learning from home, stakeholders wanted to know the extent that students were engaged in online or in-person after-school or summer programs. Funders wanted to know which programs could provide learning hubs (groups of students safely doing remote school and enrichment together) and wanted us to help them figure out who should get funding,” said Hui.
“With Alteryx, we were able to provide regular reports to the Boston Public Schools from spring 2020, summer 2020, and the current 2020-2021 school year in an effort to help the district understand student engagement. And, by connecting it to our Salesforce instance using Alteryx’s Salesforce Input tool, I created data reports for funders to ensure $2.4 million reached after-school programs at a time when students needed it most,” said Hui.
“Learning shouldn’t stop when the school day does. I am proud that Boston continues to lead the way in providing more opportunities for young people to learn and grow.”
—Mayor Martin J. Walsh, City of Boston
Apply Now.
Nonprofits: Interested in learning more? Apply here.
Connect with Boston Beyond:
Twitter: @BostonBeyond
LinkedIn: Boston After School & Beyond