What Is Platform Engineering?

Platform engineering isn’t brand new, but it has been getting a lot of airtime recently. It made the Gartner® Hype Cycle™ for Software Engineering in 2022 and has been suggested as the evolution of DevOps but, much like DevOps, the industry has yet to converge on a definition of platform engineering. The lack of consensus can make it tricky for organizations to develop their own platform engineering practice. Where do you start when the playbook is still being written?

In this post we’ll take a look at what platform engineering is, the benefits it can bring, and how it differs from DevOps and site reliability engineering (despite these terms often being used interchangeably). We’ll also explore how to decide if your organization should invest in an internal developer platform, and if so, how to go about building your platform engineering team. 

What is platform engineering?

The common theme of platform engineering is around integrating the tools and designing and creating the workflows that enable self-service capabilities for software developers, often focusing on cloud computing services (Kubernetes) and Infrastructure as Code tools such as Terraform. While your development teams work on building and improving your product, to the platform engineering team, your internal developer platform is the product.

"A platform is the sum of all tech and tools that a platform engineering team binds together to pave a golden path or paths. Developers leverage this path to self-serve with low cognitive load."— Kaspar von Grünberg, CEO of Humanitec

Low cognitive load is a key concept here: one of the common gripes with DevOps is that developers don’t want to do operations work in addition to their existing jobs. They’re reluctant to learn the intricacies of infrastructure provisioning, continuous integration pipelines, and container orchestration. 

In small companies, team members are often expected (and expect) to straddle a number of roles, but as an organization grows this isn’t scalable. It’s just not efficient for one team member to be writing code and business logic, as well as provisioning servers and architecting deployment strategies. A platform engineering team can set up and maintain the elements that make up your build and software delivery processes so that developers don’t have to become experts in these to self-serve them. 

With a robust internal developer platform, your development teams can focus on application development, thanks to the automation of repetitive tasks and streamlining of common workflows. 

Note: internal developer platforms should not be confused with Platform as a Service (PaaS) products. These are provided by third-party vendors and therefore aren’t built as a custom internal developer platform for your organization’s specific needs. 

How is platform engineering different from DevOps?

DevOps emerged over a decade ago and so is more established than platform engineering. DevOps is a set of practices that aim to eliminate the twin silos of development and operations in software delivery by improving cross-functional collaboration and automating manual tasks.

Many see platform engineering as an extension or natural evolution of DevOps. The outcomes of successful DevOps implementation and platform engineering are largely the same—less time spent on toil and a smoother path to software delivery—but the methods to get there look slightly different. While DevOps can improve coordination between development and operations, an effective internal developer platform reduces the need for coordination in the first place.

How is platform engineering different from site reliability engineering? 

These two concepts are also related: both are about creating and maintaining systems, but while platform engineers are responsible for building and evolving the internal developer platform, site reliability engineers (SREs) are focused on the systems that run applications, automatically and reliably. You could say that platform engineering serves developers, while SRE serves operations teams.  

Platform engineering examples

Let's take a look at some real-world examples of platform engineering in action:

1. Spotify's Backstage

Spotify created Backstage, an open-source platform for building developer portals. It provides a centralized interface for managing microservices, documentation, and infrastructure.

Key features include:

• Service catalog: A unified view of all services, APIs, and resources
• Software templates: Standardized templates for creating new projects
• Tech docs: Centralized technical documentation
• Plugins ecosystem: Extensibility for integrating various tools and services

2. Netflix's Spinnaker

Netflix developed Spinnaker, an open-source, multi-cloud continuous delivery platform. It allows engineers to:

• Manage and view cloud resources across multiple providers
• Implement complex deployment strategies like canary releases
• Automate deployments with built-in safety mechanisms

3. Airbnb's Kube-gen

Airbnb created Kube-gen to simplify Kubernetes manifest management. This tool:

• Generates Kubernetes manifests from simplified configuration files
• Supports multi-environment deployments
• Reduces boilerplate and improves consistency across services

Kube-gen has helped Airbnb standardize their Kubernetes workflows and reduce complexity for developers.

4. Uber's Michelangelo

Uber developed Michelangelo, a machine learning platform that:

• Enables data scientists and engineers to build, deploy, and operate machine learning models at scale
• Provides end-to-end ML workflow management
• Offers standardized processes for feature engineering, model training, and deployment

Michelangelo has accelerated Uber's ability to leverage machine learning across their products.

5. Google's Borg

While not strictly an internal developer platform, Google's Borg system is a precursor to many modern platform engineering concepts. Borg:

• Manages the deployment and scaling of Google's internal applications
• Provides efficient resource utilization across large clusters
• Inspired the development of Kubernetes, which has become a cornerstone of many platform engineering efforts

The benefits of platform engineering

With an internal developer platform, software engineers can take more ownership of their applications beyond just writing the code, without having to learn everything about continuous integration or container orchestration—these operations are automated via the platform. And these self-service capabilities have a number of effects:

Faster, more frequent releases

The platform paves the way for continuous delivery, which automates the entire software delivery process. Developers can move more quickly since they’re able to focus on writing code. They’re not spending time learning how to do operations tasks or relying on a dedicated operations team to enable them. 

Standardized workflows

Platform engineering can help to bring consistency to your organization’s toolchain and workflows. For example, Spotify shares how they reduce fragmentation in their software ecosystem though “golden paths,” their recommended (although not mandated), supported methods for achieving something, while still leaving room for edge cases. 

There can also be drawbacks to a prescriptive, internal developer platform. Platform engineering efforts can backfire if you try to force everyone to use the platform. 

“Golden paths should be the easiest path, the best one, the one that covers most of your cases. But it's almost impossible to avoid exceptions. And if developers don't have those exit routes to do some exceptions, they will be stuck on your golden path. And it's no longer a golden path; it's a golden cage or a golden tunnel.”— Mathieu Frenette, Director of DevOps, Nesto 

Those exceptions and the exit routes are useful signals: you may be able to build and integrate the solutions for them into the overall platform.

Better developer experience

We touched on the idea of “cognitive load” above, and Puppet’s 2021 State of DevOps report also notes that “highly evolved teams tend to do a good job of limiting extraneous cognitive load on delivery teams (through good practices, automation, and support from other teams).” Reducing painful, manual tasks and the resulting mental costs is critical for developers to spend more time in flow state, resulting in happier team members and better retention.

Does your org need a platform engineering team?

The State of DevOps report cites platform engineering as “key to success at scale.” Not every engineering organization needs or will benefit from a platform engineering team … yet. If yours is an early-stage startup or small-scale company, you may not need a dedicated internal developer platform. With no legacy systems to maintain or complex migrations to manage, your engineering time is probably better spent on building and shipping new features and functionality for your customers. 

Your lean team may already be comfortable with cloud services and other new technology, so they’re able to self-serve without the support of a platform. With a small number of engineers working closely together, you probably don’t have the enterprise-scale challenge of people duplicating efforts to solve common problems over and over. 

As you scale though, the pressure to deliver new features increases and technical debt accumulates, because your team members can’t afford to spend time on the underlying platform. At the point where you have multiple teams working concurrently—building processes, services, and solutions that may be useful to other teams—it’s time to consider building your internal developer platform. 

“If you can take the service and give it to another product team, or even give it to another company, and they can use it right away, then that belongs in the platform.”—George Hantzaras, Director, Cloud Platform Engineering at Citrix Systems, Platform Engineering: DevOps evolution or a fancy rename?

By standardizing the solutions to complex, recurring problems, you can free up your developers to move faster and enable new hires to onboard and start contributing more quickly. 

Best practices for building a high performing platform engineering team

Building a high-performing platform engineering team requires a combination of technical expertise, soft skills, and strategic planning. These best practices will help you create a team that delivers value, improves developer productivity, and drives innovation across your organization.

1. Treat your platform as a product: Approach your internal developer platform with a product mindset. Connect the platform's goals to overall business objectives and demonstrate its value proposition to stakeholders.
2. Create a customer-centric mindset: Remember, your primary customers are other engineering teams. Regularly gather feedback, prioritize high-impact requests, and balance the needs of engineering teams with those of end-users.
3. Build a diverse team with complementary skills: Hire engineers with a mix of expertise in cloud technologies, CI/CD, and infrastructure as code.
4. Promote cross-functional collaboration: Work closely with development, operations, and security teams. Establish clear communication channels and organize regular knowledge-sharing sessions to align on goals and challenges.
5. Implement agile methodologies: Use agile practices to manage platform development and improvements. Break down large projects into smaller tasks and conduct regular sprint planning and review sessions.
6. Focus on automation and self-service: Automate repetitive tasks to increase efficiency and reduce errors. Develop self-service tools that empower developers to work independently and create comprehensive documentation for your platform.
7. Prioritize scalability and flexibility: Design your platform to accommodate future growth and changing needs. Use modular architecture to allow for easy updates and integrations, and regularly evaluate your tech stack.
8. Establish clear governance and best practices: Define and document standards for using the platform. Create "golden paths" for common workflows while allowing for exceptions (when necessary).

What skills and experience does a platform engineer need?

Qualities

• Empathy: Platform engineers must develop a deep understanding of the needs of developers and the points of friction they experience in getting their code deployed successfully. Engineers who only want to write code aren’t a good fit for a platform role.
• Strong communication and collaboration skills: These help platform engineers understand the problems that the platform needs to solve for other teams. 
• Willingness to teach: Helping others to adopt the platform and golden paths is a core part of the role.

Experience and expertise

“Platform engineers need to have an interest and knowledge of the interface between the software and the machine,” says Camille Fournier, Managing Director at Two Sigma. Below are some of the common technologies platform engineers need exposure to and experience in.

• Kubernetes and container orchestration
• Cloud native technologies
• Microservices
• API management
• Continuous integration and pipeline management, continuous delivery
• Infrastructure as Code (Terraform, AWS Cloudformation)
• Observability, monitoring, and alerting 
• Experience on an SRE or DevOps team (advantageous but not essential)

The role of the platform engineer will vary depending on your company, product, and stack, and will likely continue to evolve as the practice climbs towards Gartner’s Peak of Inflated Expectations. Maintaining a laser focus on serving your development teams’ and customers’ needs will help keep you on track.