Different test validations for environments

Different test validations for environments


Most probably, your staging and production APIs respond to most requests with 200 OK. Having a shared set of validations between multiple environments is handy as you can set 200 OK once and have it validated against all of environments. What about response time? Does your staging API respond to login requests as fast as your production API does? Probably not and that’s OK because for test environment, the response time is not that important. How would you set the “Response Time” validation for a scenario like this? Up until today, there wasn’t an easy way to set these kind of validations but that’s not the case anymore.

Apart from shared validations, Testfully also allows you to define a set of validations specific to an environment. Moreover, you can mix and match both features together and have a set of validations (for example, Response Code) shared between all environments and have Response Time per environment.

How It Works

The Validations tab in Test Editor continues to be the place to set validations. We have updated this section so that it accepts a set of validations for all environments and a set per environment that is added to your workspace. Click on the environment names to expand and collapse the validations form for that environment.

Set validations per environment in Testfully

We’re very excited to see how this feature improves your development workflow and would love to hear what you think about this feature. Also, we have put together some FAQs for you and highly recommend to go through them, it won’t take much of your time.


What’s going to happen to my existing test cases?
Your existing test cases will continue to work as you expect. The validations appear under the “All Environments” section.

Do I need to set validations for all environments?
No! You can choose to set validation for one environment and leave the rest empty.

What would happen if I set a common Response Time validation of 200 milliseconds and 250 milliseconds for my staging API?
When running your test agsinst staging API, Testfully takes the 250 milliseconds as that’s what you have set explicitly for the staging API. For any other environment you might have (for example, Production), Testfully will use 200 milliseconds as the expected Response Time.

What would happen if I set shared Response Body and a Response Body for my production API?
When it’s time to validate Response Body of your test against the production API, Testfully will merge the expected “Response Body” for Production API into the shared Response Body and uses the result as the expected response body. This is also same for Response Headers.

comments powered by Disqus

Recent Articles

7 HTTP methods and how to use them

HTTP protocol works by clients sending requests to the servers and servers responding to the requests. We do CRUD operations (Create, Read, Update, Delete) by sending HTTP requests with different HTTP methods, sometimes called HTTP verbs.

Introduction to API Blueprint

API blueprint is a powerful high-level API design language for web APIs. In this article, we want to dive deeper into it and learn more about how it works, the differences between API blueprint and Swagger, and what makes it unique that leads to its extensive use. But before we dig into API Blueprint, we must ensure a solid base of information about the “API first approach” concepts.

False positive & false negative in software testing

Exports in automated software testing have borrowed false positive and false negative terms from the medical examination field. In the medical field, the purpose of a test is to determine whether the patient has a particular medical condition or not. As far as software testing is concerned, a false positive indicates a bug when there is none. Conversely, a false negative indicates no bug when there is one.