August 4, 2020

Stirring the imagination

As the UW seeks to support data management and analysis more broadly across disciplines, the answer may just lie with Jupyter Notebooks.

 

By Ignacio Lobos

When David Shean started searching for a more effective and interactive tool to teach students how to analyze complex geospatial data sets in his mixed graduate and undergraduate class, he zeroed in on Jupyter Notebooks.

The Jupyter Notebook is not a laptop — it’s a powerful open-source web application that offers students the opportunity to work with 100 different coding languages. They can combine live code with text and narrative to support their data analysis work, and graphs, equations, videos, animations, maps and other visualizations to explain their results more fully — whether they’re working in the humanities or STEM fields.

“The notebooks are an amazing teaching and learning tool,” said Shean, an assistant professor in the UW Department of Civil & Environmental Engineering who carefully built his most recent geospatial data analysis classes around their use.

“One of the things that makes the notebooks great is that students write code, and immediately see results,” Shean said. “They can interactively tweak, interpret, and repeat until they are satisfied, which enables data-driven discovery.”

Using their notebooks, students in Shean’s class learned how to analyze geospatial data to answer questions such as, “How much of Whidbey Island would be flooded due to sea level rise if the Antarctic and Greenland ice sheets melted?” “What are the most hazardous roads in Washington state?” “How much snow was on Mt. Rainier yesterday?” “Is the Arctic warming faster than the rest of the planet?”

In his class, students worked with sophisticated open-source software packages to visualize a given problem with a geographic component. “The notebooks can be run so that all of the plots are interactive, such as zooming or panning around high-resolution satellite images or clusters of data points on a chart, which is really powerful,” he said.

Shean partners with UW-IT to launch Jupyter Notebooks project at UW

Shean has used the notebooks in two pilot projects for UW courses in the past year, most recently in a partnership with UW Information Technology’s Academic Experience Design & Delivery (AXDD) unit.

“With David’s class and other classes that participated in the pilot, we’re carefully creating an enterprise service to support Jupyter notebooks for teaching and learning,” said Tom Lewis, AXDD director. “We are collaborating with the eScience Institute, UC Berkeley and others for knowledge transfer and system design. The goal is to provide easy-to-use research tools for use in courses.”

Why Jupyter Notebooks are so powerful

At the UW, where the Office of the Provost is working to support data management and data analysis more broadly across disciplines, the notebooks fit the bill. Among their advantages:

• Students in multiple areas of study can use the notebooks to explore and work on a multitude of real-world applications, such as helping a city relieve traffic gridlock, a health department track a viral infection, or a utility reduce operating costs.
• The notebooks are ubiquitous worldwide, a tool used by professionals to collaborate across multiple fields.
• Students access their work and share it from anywhere — which is proving valuable during the COVID-19 pandemic, with classes being held remotely. All students need is a web browser and an internet connection, and they’re ready to work.
• The notebooks lower the entry barrier, bringing powerful tools within the reach of more students.
• Because students use their own laptops and other devices, the notebooks decrease or eliminate the need for expensive computer labs with limited hours of operation.
• Faculty can provide a state-of-the-art computing environment to their students with none of the hassles of maintaining it.
• The notebooks are hosted in the cloud, with powerful servers that can handle the computational work and vast amounts of data.
• With easier and cheaper access to cloud computing, the notebooks are becoming a more viable and powerful tool in pedagogy.

And, most importantly, students love them.

“It is the future,” said Friedrich Knuth, a Ph.D. candidate who participated in Shean’s winter 2019 class.

“The software is free and open-source, so as a student and young professional, I can take the methods that I learned to develop with me anywhere I go,” Knuth said. “I don’t need to own a powerful machine myself, as all computation is done on the servers hosting the notebook. This lowers the barrier of entry for scientific data analysis and exploration.”

How Shean built a class using Jupyter Notebooks and other tools

When Shean made the decision to use the notebooks, he worked with AXDD to carefully build a computing environment to enhance the teaching and learning experience in his classroom. After lengthy discussions about Shean’s needs, AXDD set up the system, maintained it and covered the costs of operations, including buying cloud server time, during the pilot.

The notebooks are one of several key components that make up Project Jupyter, a non-profit, open-source organization that supports interactive data science and scientific computing with robust online tools.

“We’re using a JupyterHub for cloud infrastructure, a JupyterLab environment for interactive computing, and the Jupyter Notebooks for teaching and development,” Shean said.

“The UW-IT support freed me up to focus on the course content, and to develop the notebooks each week,” Shean said. “I developed and tested all course material on the JupyterHub, and the students did all of their lab exercises, homework, and final projects using the same resources. There was never a question about compatibility.”

Other faculty, he pointed out, “might want to use notebooks for a few labs, or encourage students to use the notebooks as a substitute for MATLAB or Excel. Or, as with my course, the notebooks can provide the foundation.”

“I perform interactive tutorial/demos with notebooks. All of the lab/homework instructions and questions are embedded in the notebooks. And I designed the notebooks to build in complexity, starting with simple examples to illustrate basic concepts, then adding new datasets and challenges with fewer instructions, so students apply what they did earlier to solve more complex problems with real-world implications,” he said.

“The Jupyter resources transformed the way I taught the class,” said Shean, a young faculty member who has been in front of the classroom for only a couple of years.

“Students are very comfortable in the modern tech environment. They get it, and more and more, they’re demanding tools that will allow them to explore and solve large data problems.”

How to get started with Jupyter Notebook

In the past year, AXDD has worked with 15 faculty members from different areas of study to understand their teaching needs. It also built and tested a set of JupyterHub/Notebook environments that can be quickly deployed by instructors. The pilot was so successful that UW-IT will be offering the service to faculty starting this fall. Currently, there’s no cost for this service. Visit IT Connect to learn how you can bring the notebooks into your classroom.

“Students love the notebooks,” Shean said. “They see value in them, and are learning from them. It’s really rewarding to hear.”

Resources:

David Shean and Rob Fatland, UW-IT Research Computing Director, will host a session on Jupyter Notebooks at the UW TechConnect Conference, Sept. 2-3.

Want to use Jupyter Notebook in your class? Visit IT Connect to learn how to acquire it and learn from others

What is the Jupyter environment? A quick overview
Shean’s Geospatial Data Analysis course
Shean’s guide for faculty who want to use Jupyter Notebooks in their course
Try it out right now
Read about JupyterHub
Check out GitHub