When I was an Engineering Manager at Vignette, I'd run beta programs for our new releases. The beta programs had a dual purpose. First, we wanted to get feedback on the stability and validity of our features. But the beta also fed marketing with happy reference customers for our launch announcement. Customers liked being part of a beta because it gave them early access to features they had been waiting for, as well as an opportunity to influence product direction.
What had changed? The word beta has been overloaded to mean “we're not entirely ready for prime time, so please be patient”. Gmail was in beta for five years! At TripIt, we had a beta tag for multiple years.
Canary release - exposing features to some subset of users (whether it be opt-in, random rollout, or specific segments) is now used to describe what was once a beta.
- Microsoft: In development of Windows 10, Microsoft used “canary” releases to test with internal users within Microsoft. Gabe Aul, who leads the Data & Fundamentals Team in the Operating Systems Group (OSG), said "our Canary ring probably sees 2X-3X as many builds as OSG because we catch problems in Canary and don't push to OSG."
- Instagram: “Using 'canary' releases, updates go out to a subset of users at first, limiting the ability of buggy software to do damage." Mike Krieger, Instagram co-founder and CTO, said he uses canary releases because "If stuff blows up it affects a very small percentage of people”.
- Google: For Chrome, Google offers Chrome Canary, which it labels with “Get on the bleeding edge of the web, Google ChromeCanary has the newest of the new Chrome features. Be forewarned: it's designed for developers and early adopters, and can sometimes break down completely.”
So yes, canary is the new beta.
.webp)
