Test Environments 101: Definition, Types, and Tests

Test environment definition

A test environment is a designated space where software undergoes experimental use to identify bugs, assess performance, and guarantee functionality before deployment to end-users. It includes the hardware, software, and network configuration necessary to simulate real-world conditions.

Types of test environments

An application’s server location is itself an environment. However, it's standard to segment and name subsections based on the state of the application.

We typically have four environments along any software's lifecycle.

1. Development environment: Where developers write and initially test code.
2. Testing environment: Dedicated space for running various tests on the software.
3. Staging environment: Mimics the production environment for final testing.
4. Production environment: Live environment where the application is accessible to end-users.

Development environment

The development environment is the location of the main branch of a software application. This is where developers spend time writing the first lines of code. From here, an application transforms from concept (on paper) into the MVP (minimum viable product).

Testing environment

When you set aside an entire server (could be a container) to run tests uninterrupted, you’ll have created a test environment. Typically, a test environment meets the minimum requirements for your application to function. 

Staging environment

Staging is when you create an instance of an application that you're confident enough to show the immediate owner (sponsor) but not users. It's meant to simulate production as much as possible. The staging environment is often restricted to a few select people. Whitelisted emails, IPs, and your developer team are the only groups that can use the application in staging. 

Production environment

As an end-user, when you use a web or mobile application, the application is running on a production server. It has been produced. We mentioned how testing was necessary for every application, however, not every environment is a prerequisite. 

Thanks to feature flags, many development teams are doing away with some staging environments altogether. Tests can be conducted during production with new features being released safely at the same time. Feature flags make it possible to show a few specific users a future version of an application, while the rest enjoy its current version.

Testing environment tests you can run

While the staging environment typically gives test teams the chance to take an app for "a ride," this kind of testing is far from sufficient for an app to satisfy quality assurance. Beforehand, the app will have undergone test executions looking to break it in a safe environment. System testing gives developers the chance to fix it without the pressure of active users breathing down their necks.

Here's a list of (some of) the software testing routines an application goes through before development. 

1. Performance tests
2. System integration testing
3. Alpha tests
4. Beta tests
5. Unit tests
6. Accessibility testing
7. User acceptance testing (UAT)
8. Usability testing

We'll discuss the first four as we explore an application under testing and its developers practicing DevOps methods through different environments.

1. Performance test example

Software testing of this kind measures how fast an app responds to an interaction. You'll typically get page load speeds and input processing speed as reports here. You can run performance tests before and after deployment. 

Here's a typical test that developers at Facebook might run, for example:

This particular report rates the performance of a specified application out of 100. It also provides a granular outline of what the score is, making it possible for developers to improve it if desired. Different tools can be used when performance testing, all giving different test results in the end.

2. System integration testing

It's common practice for teams of developers to assign sections of an app to each developer. In fact, modules made by these team members can continue to run isolated from each other. Inevitably, there comes a time that they absolutely have to integrate. 

This is the stage when integration tests are run. The resulting system testing makes sure all modules can communicate and share the same database. Depending on test results, it may take longer for the system to then come together.

3. Alpha tests

This is the first end-to-end run conducted on the application when all modules have been integrated successfully. Its main objective is to make sure the application does what the client is expecting.

For example, a payment gateway would undergo payment transactions in sandbox mode as its alpha test. Quality assurance managers typically tick a checklist until the application satisfies the list of functionalities. It's more of a quantitative testing process as it cares more about what’s possible than how it happens.

4. Beta tests

You may have seen or even used some software applications in beta mode. It's during this phase that real-world users put an application under stress. This load testing serves to unearth and report any bugs the developers may have overlooked. Some bugs actually only appear during production. This means a staging environment with open access is necessary: hence, the existence of beta versions of apps. 

5. Unit tests

Unit testing is the practice of testing individual components or functions of the software in isolation. Developers write these tests to verify that small, specific pieces of code work as expected. Unit tests are typically automated and run frequently during the development process.

6. Accessibility testing

Accessibility testing checks that the software can be used by people with various disabilities, including visual, auditory, motor, or cognitive impairments. This type of testing is essential for creating inclusive products and often necessary for legal compliance.

7. User acceptance testing (UAT)

User acceptance testing is the final phase of testing before the software is released to all end-users. In UAT, actual users or client representatives test the software to guarantee it meets their business requirements and functions as expected in real-world scenarios.

8. Usability testing

Usability testing focuses on how user-friendly and intuitive the software is. It involves observing real users as they interact with the application to complete specific tasks and noting any difficulties or confusions they encounter.

The art of testing in production

While it may seem scary at first, testing in production is more efficient and effective than soaking your code in staging (note: we're not necessarily encouraging teams to shed their staging environments; we’re just highlighting the benefits of testing in production comparatively).

And LaunchDarkly feature flags remove much of the risk typically associated with such a testing approach.

With feature flags, you can release a new feature to a small user segment (e.g., a canary group) while concealing it from the rest of the user base. Thus, if the feature throws a bug in production, it will only impact a small portion of users. Moreover, with feature flags, you can disable the buggy feature instantly without having to change a config file and restart your application.

Get started with LaunchDarkly and embrace the philosophy of testing in production to build and deliver better software, faster. 

***

Request a demo of LaunchDarkly’s feature management platform.