Emily is a product manager at Seesaw, building software that empowers students to be their best and helps teachers and families support their child’s learning. When Emily is not making Seesaw better, she loves to paint, cook and surf.
Heidi Waterhouse: So we've come back from lunch and now we get to hear a session that's going to be super cool from a first-time speaker. Emily works for Seesaw and they are an educational software company working to empower and enable both students and parent to understand better what's going on. I sort of think of it as instrumentation for homework. So Emily's going to talk to us about five ways they actually use feature flags at Seesaw and how that's been working out for them. So I'm super excited to hear this. Please give her a very warm welcome. Emily Voigtlander: Hi everyone. All right, so let's get this going. I'm here to talk today about five ways that we used feature flags last year in our first year of using feature flags at our scaling startup. First, I'm Emily. I'm a product manager at Seesaw. Our mission at Seesaw is to create an environment where students are inspired to be their best. Seesaw is used by millions of students in more than half of the schools in the United States, and over 150 countries around the world. And students really use Seesaw to share their learning with the people who care most about their success. This means we have a bunch of creative tools that students can use to show their learning, and that learning gets shared with their teachers and their families so that families can be involved in the learning process and support their student every day instead of once or twice a year at a parent-teacher conference. And teachers really have a good understanding of how each student is doing in their class and how they can better support them as they're on this sort of educational journey together. As a PM, my job really is to ship a great product to our students, teachers, families, and school administrators. And for anybody who's not a PM, another really big part of my job is bringing the rest of the company along for the ride. It's not just about building great stuff; it's about making sure that everybody internally and also externally understands why we're building what we're building and why it matters for our customers. And so when I think back to where we stared, I joined Seesaw in 2014. We were a company of three people, and over time we grew to five people and that felt huge, right? It was like not just me and two co-founders talking in a single room at a single desk. We actually didn't have a desk; it was a couch. And then over the past few years the team has grown a lot. We've had the amazing opportunity to grow the team because we actually built a product that schools wanted to buy and give us money for, which meant we could grow our team. When we were really small, it was super easy for us to stay on the same page because we sat in one room, we could only work on one thing at a time, and we had lunch together every day. Now it's harder. We have 50 people on our team working across product, engineering, sales, marketing, customer success, operations. We have departments now which is crazy, but that means that it's actually a job to keep everybody in the loop and make sure not only are we building the right product for our customers, but that everybody understands why we're moving in that direction and is able to communicate that product vision, and really have an impact on our customers, and using sort of their role and responsibilities to sort of channel that. Another exciting piece of how the company sort of has changed as our customer base has scaled is now millions of students need Seesaw to just work for them. When they open the Seesaw app or go to the Seesaw website, Seesaw needs to be there and it needs to be working. When we had a few thousand classrooms using Seesaw, if an issue or a bug affected some of those customers the sphere of impact was also pretty small and so it didn't really matter as much. Now Seesaw is a tool in classrooms like GitHub or Slack is a tool for all of you. So if you opened your computer on Wednesday, and GitHub or Slack wasn't there and wasn't working or looked completely different than it did on Tuesday, you're going to have a problem with that and it's going to really impact your day. Imagine you're a teacher, you're teaching 30 students and those 30 kindergartners open Seesaw and Seesaw isn't working or looks different. Now you have 30 people. Imagine your entire team panicking about not being able to do their work. Also now we have a product that people are really using and loving. Hooray. But we need to make sure that we can continue to grow and grow that product, change it, evolve it in the right ways, but also not totally disrupt our customers in the middle of the key sort of workflow for them. So we had to sort of reflect on that growth in the company and the growth in the customer base and really ask ourselves like, "How do we build a great product and engage the right people in that process?" And now sort of also we have more people on the team. We have more engineers. We have more capacity, the thing that we've been wanting right from the very beginning. Instead of just being five people, we now have 20 engineers. What does this mean? It means we can do more than one thing at a time, but that also means we can also break more than one thing at a time and so we want to make sure to work on more than one thing at a time and also have that not be a total disaster. So this is where feature flags enter the picture. Hooray. In early 2018, at the strong pleading of one very vocal engineer, we added LaunchDarkly to our team's toolkit. And I have to say it has been a total game changer to the way we develop product at Seesaw. We do this thing where, at the end of the year, everyone has one slide where they can propose some change that they want to see in the company. This particular engineer was like, "Feature flags." And honestly, I had not really ... I didn't even really know what he meant when he said feature flags, and I think it's really changed the way that we do work at Seesaw. And so we're going to talk a little bit about what that looked like, specifically for the product that we're building. So the first thing that we learned is don't forget you have feature flags. Adding something new to your team's workflow, even if you're a small team that is excited about changing, is still hard. You have to remember that you have this new tool in your toolkit. So one of the things that we did was just add a small reminder into our product and technical spec template. It's really just like a one-page document that's like, "Are we making sure we're doing the right stuff?" We have other things that I've forgotten when writing various product specs or other people have forgotten when writing their technical specs in here like, "Oh, we should probably figure out what we're going to do with notifications if this feature impacts that." We added feature flags into here and have made feature flags sort of part of the conversation early in the product development process so that kind of everybody on the product and engineering side is sort of on the same page about how we're thinking about this feature and how we're thinking about flagging it. So the actual ways we used feature flags. So I'm going to talk about the ways we used feature flags across a few areas of our business: product, marketing, sales, legal, and user research. And each of these teams have different goals. We use feature flags in different ways to support them and really make sure that they're along for the ride. So our first ... on the product side, our goal was to launch a new feature on three platforms. Seesaw supports iOS, Android, and the web. We need to make sure that we launch the same features on all of these platforms at about the same time. At Seesaw specifically, our goal last year was to launch a lesson library and assignment flow that integrates with a bunch of our existing product but also had a bunch of new features. So we have got this tricky thing of saying, "We have some new features that we want to add in while also integrating with existing product features." So our teachers are going to find a lesson in the library, they're going to share that lesson to their students, students will respond using all of the stuff that we already have, and then teachers are going to review those responses before they're shared with families. So we're merging sort of new features, existing features. How's this going to work? So we used flags a variety of ways on this project. Sort of the idea for the lesson library was originally inspired by behavior we saw in our community. Teachers were creating these slide decks of all of these lesson templates and sharing them on Twitter, and Facebook, and stuff like that. We're like, "Great. We can take that. We can make something better." And so in order to get this feedback on this feature early, we were able to reach out to those content creators and demo the features or have them test those features in sort of early beta of those features and get feedback that we incorporated into the product development and engineering process. We also use flags to verify backwards compatibility and integration with our existing product features. Before feature flags, basically we would just kind of hope for the best here with integrating with the existing product features. We would test as much as we could in beta, and then we'd basically just like push everything to production, and then quickly test in production and make sure it wasn't terrible. Now with feature flags we can just turn the feature off, test, make sure everything is working, turn the feature on, test, make sure everything is working. Way, way simpler. In the same environment because there's always differences between production and beta, all this stuff. And as a result, we've had far fewer pretty bad bugs get out to our customers which is really important when you've got those 30 kindergartners who are expecting Seesaw to work for them. Flags were also super helpful for us with synchronizing the rollout of all of our features on iOS, Android, and the web at the same time. We want to make sure ... you know, we have a native iOS app, a native Android app, and a web app, and a shared backend. We want to make sure that the features are available for everybody, but as many of you probably know, you need to submit your iOS app and get that one approved by Apple before it can go live to your customers. Android, you can kind of submit any time. And the web, we can push whenever we want. Before feature flags we would submit iOS, and wait for the app to get approved, and then push web to production, and then quickly release iOS, and test to make sure it was working, and then if that was okay we'd submit the Android app. Now we can submit all of the apps and push to production on whatever schedule we want, whatever works for our development team. And then product can work with marketing to turn the flags on for a feature at a specific time. So basically instead of having a release day that was basically like a ton of chaos, we can just release features when we want to turn features on for the customers we want to turn them on for. It's amazing. So I think feature flags is also one of the reasons this feature has been like the most quickly and widest adopted feature of any feature that we've launched since our initial release. We have launched many features since 2015, but this lesson library and assignment feature was the quickest adopted. Typically features will take almost a school year cycle for teachers to really incorporate them into their routine, and this one was adopted super, super quickly, and as I think one of our sort of popular features other than the first feature which was like the portfolio for student work. So really successful for us, and I think really do a lot to ... you know, the fact that the feature actually solved a real customer need and also worked in the classroom. And all of that was really possible because of these sort of different checkpoints along the way that were enabled by feature flags. Cool. Okay. Next one. Marketing. Why does marketing want feature flags? Okay. So our goal for this project or sort of a generalizable goal that maybe any of you might have is to get user-generated content before a public release. So for us at Seesaw, this meant building out a library of thousands of teacher-created lesson ideas that are ready to use in Seesaw classrooms but before the feature is released. So before we had five Seesaw-made lessons. We controlled the content and there was very little of it. So what we want to say is we're going to go from this, five activities per grade level created internally to tens of thousands of teacher-made activities in the activity library that teachers can use. So how did flags help us with this? First, we gave specific brand ambassadors and power-user teachers early access to this new feature flow and then also gave them the ability to contribute their lesson ideas to our library. Our goal really was that we would have some amount of high quality user-generated content in the lesson library before it was released publicly to all of our teachers in the community. We didn't want them to get there and basically have like ... it's like, "We launched a lesson library feature and there's five things in it," right? We wanted to say like, "We launched a lesson library and there's thousands of lessons that you can use in your classroom tomorrow." So because of feature flags, we were able to get I would say I think like 3,000 teacher-created lessons, which I honestly I'm not really sure how we would've done before feature flags. And now, post-release, this feature is public. It's available to all of our teachers. We now sort of have on the marketing side a different goal. We've got a steady stream of content coming in. We have contributing authors sharing new ideas every week. But the problem is now how can we make sure that the lessons that we have in the library are high quality and meet the expectations that we have for sort of the curriculum that we're putting out into classrooms? And so now, we sort of shifted gears and now use feature flags to allow contributions from teachers who meet specific criteria to maintain the quality of the submissions. So not every teacher who's using our app can submit, only teachers who meet specific criteria that the marketing team ... and the marketing team has flexibility to decide what that criteria is, which is also great because it means we don't need an engineer to add a user to a specific list to give them access to this special feature. The marketing team can decide like, "We want to open up contributions to this grade level or this type of teacher customer," and they have the power to do that which is super cool. Sales. Okay. So the goal for this feature was basically to preview a top-requested feature to a high-value customer. At Seesaw specifically, we wanted to preview some new student data privacy controls to these high-value customers who had requested these features. We also wanted to get a little bit of in-the-wild testing in a contained sort of environment before we rolled out a control to all of our paying customers, right? We want to keep our best customers happy, and really involve them in the product development process, and also not break things for everybody else. So for this particular feature, it's very technical and also impacts student privacy. So making sure that it works the way that school administrators expect is really important for the success of the feature. And to be honest, we weren't exactly sure we got a lot ... we work with a lot of different schools.