OMG! Cloud Testing is so different..!

Unless you took a refugee or voluntary retirement or moved to Himalayas, you must be aware of the ‘Virtualization’ and ‘Cloud Computing’ concept. Cloud computing has passed through the stage of hype to become a reality of modern enterprise infrastructure. Every other Tech giant has its cloud offerings, be it Amazon, Google, HPE, IBM, Microsoft or Salesforce. And how did it impact the Testing landscape? Yeah! Every big innovation in the technology space has its parallel influence on the Test processes & techniques. In no time testing evangelists started exploring the Clouds and came up with new Test approaches suited for the Cloud. Cloud Testing started trending in the town and I too jumped into this bandwagon. Yes, recently I have been working on a Salesforce Marketing Cloud project and OMG! Cloud Testing is so different..!


Stealing the definition from Wikipedia,

“In computing, virtualization refers to the act of creating a virtual (rather than actual) version of something, including virtual computer hardware platforms, storage devices, and computer network resources.”

Do you use RSA Token to connect to client network? Or access a virtual machine from your computer to work on secured client applications? Yeah! That’s the simplest practical usage of virtualization, i.e. accessing a virtual machine that looks like a computer with say Linux OS from your actual computer that is running Microsoft Windows. The obvious benefits – sharing computing resources to reduce capital costs, increase scalability and enable easy administration of the IT infrastructure.

Cloud Computing

The most common analogy – having a meal at restaurant vs. home. At home, everything from ingredients, recipe, utensils and cleanliness has to be managed by the homemaker. But a restaurant is ‘catering as a service’ where homemaker just visits, orders the meal, enjoy the appetite and leaves after paying the bill 🙂 on demand & Pay-as-per-use service!

Quoting it from the Wikipedia,

“Cloud computing is a new form of Internet-based computing that provides shared computer processing resources and data to computers and other devices on demand. It is a model for enabling ubiquitous, on-demand access to a shared pool of configurable computing resources (e.g., computer networks, servers, storage, applications and services), which can be rapidly provisioned and released with minimal management effort. Cloud computing relies on sharing of resources to achieve coherence and economy of scale, similar to a utility (like the electricity grid) over an electricity network.”

  • Infrastructure as a Service (IaaS): Physical resources such as Storage, Network devices, compute servers, etc. are available on demand, where a user pays as per the usage.
  • Platform as a Service (PaaS): Access the run-time environment on demand, e.g. a development or test environment. Example – VM usage that contains the complete environment like OS, required middle-ware, etc. when you need it.
  • Software as a Service (SaaS): The applications are available on demand via the internet. Instead of acquiring per-user license, this tends to be the most cost effective way of ensuring that the license is always in use. Example – Gmail, Google docs, Photoshop, etc.

Types of Cloud

  • Public Clouds: Services are available to everyone by dynamically provisioning the resources as per request.
  • Private Clouds: Restricted to an organization & its employees/users by a well-managed firewall rules.
  • Hybrid Clouds: As the name suggests, it’s the fusion of Public & Private Clouds. Keep some services public and rest private.

OMG! Cloud Testing is so different..!

I was genuinely excited to enter the world of Salesforce Cloud testing. Why? For obvious reasons,

  • It’s Cloud Testing, doesn’t the name itself sound so ‘Technical’. Only a few chosen ones get to work on the latest trending technologies.
  • Cloud Testing is so different (I knew that already) & in demand. Will be challenging & quite a learning.
  • Salesforce is a booming technology and like Cloud Testing – in demand.
  • I can also boast around ‘Yeah! I have practical hands-on experience with Cloud Testing. It’s so different & challenging you know’.
  • Will get to add Cloud Testing and Salesforce Testing in my so-called resume.

Salesforce Marketing Cloud includes integrated solutions for customer journey management, email, mobile, social, web personalization, advertising, content creation and management, and data analysis. Marketing Cloud delivers data and application functionality through the web, rather than having them reside on a user’s desktop/laptop/mobile device, hardware drive, or in what is otherwise known as on-premise solutions. Users with web access simply log on to an app to access, share, and collaborate on relevant data and business processes immediately. In short – It’s in the Cloud somewhere!

As we started with the Test activities, the first step was to gather requirements. I was waiting to see some cloud-specific requirements but you know what, it all looked like web functional testing requirements. Okay, it will be different in Design & Execution, I said to myself. We drafted the Test cases based on the functional requirements, i.e. design was also just like another website testing. Okay, execution will be challenging I guess, since it’s in the cloud dude. Now that we have completed the execution as well, I am wondering what-did-I-know and what-I-have-learnt-today. Damn! I am done with the prestigious Cloud Testing but what was so different in it? Actually nothing! This got me searching online…

Cloud Testing is a vague term

With applications moving from dedicated client-server model to a more scalable cloud environment, what do you think Cloud Testing actually mean? Testing the Cloud setup OR Testing the application in the already setup Cloud? Yeah! Both are drastically different. Let me explain…

Cloud is not already available. Enterprises have to avail Cloud partner services in order to host their applications in the cloud (which can be either public, private or hybrid). Say XYZ organization approaches Google to setup a private cloud for its service offerings. Google provisions a private cloud for XYZ and then applications are hosted in this Cloud environment. It’s a two-step process.

  • Testing the Cloud: When you are testing the Cloud setup by Google for XYZ organization.
  • Testing the application in the already setup Cloud: When you are verifying (& validating) the ‘functionality’ of a particular application.
  • Test Environment in the Cloud: When your Test environment itself is setup in the cloud, such as Mobile Test lab by setting up emulators (Device & OS combinations) in the cloud environment OR setting up UAT & Performance Test environments in the Cloud.

If you are into the second Test team, according to me it doesn’t make a difference in the Testing approach. At the end, we have to verify that the application is functionally & non-functionally behaving as per the requirements. It doesn’t matter to me as a tester where the application is actually hosted. Same Test tools can also be used to drive Testing in the cloud-based applications. Dedicated vs. Cloud is more of a concern to designers, developers and architects. It’s like asking a Business analyst – What will be the difference in the Gmail Sign-in functionality if it is hosted on a dedicated server vs. in the Cloud 😛 Absolutely nothing!

But yes, if you are part of the first or last Test team – it’s different. Basically both the approaches are dealing with the environment setup (infrastructure), i.e. if you are testing the environment itself which will obviously be different from the application testing. As a Tester, we need to make sure that the environment is secure & scalable after the application is deployed. Security & Performance are two major factors here!

Security & Performance are the only key

I understand that security & performance of an application varies depending upon the underlying infrastructure. This is the only area I feel which is impacted by moving from dedicated infra to the clouds. Since Cloud is not under the application-owner-user purview, Security & Performance testing has to be thorough. Data privacy, Exception mechanisms, Disaster recovery, Reliability, Backups, Security vulnerabilities, Dynamic scalability, etc. have to be explicitly tested to ensure smooth operations.

Benefits of Cloud Testing (for Organizations)

  • Organizations can save money by setting up Test environments in the cloud instead of investing in dedicated hardware/software infrastructure.
  • No need for maintaining a separate Performance test environment simulating the actual production. Instead, use Cloud’s scalable property for Performance testing in the same Test environment.
  • Instead of maintaining physical infrastructure to cover device-OS-browser combinations, all this can be managed in a virtual Test Lab in the Cloud.

Keep it simple, silly!

Cloud Testing has today become one of those “big bangs” in the industry. Most organizations are now leaning to adopting the cloud because of its flexibility, scalability and reduced costs. But remember, using the cloud for Test environments is altogether different from Cloud Testing. Leveraging Cloud capabilities for QA infrastructure, Test environment setup, Browser-OS virtualization, Mobile Test Lab setup is altogether different from Testing an application that is hosted in the Cloud. Organizations are leveraging Cloud to minimize the QA infrastructure costs – but it doesn’t impact you as a Tester. Applications are now hosted in the Cloud – but it doesn’t impact you as a functional Tester. But if you are a Security researcher or a Performance engineer – Yes, these both might impact the way you approach Testing!

I hope this article cleared some of your confusion around Cloud Testing, Cloud-based Test environment and Cloud-based application testing. Cloud Testing is a vague term with no official definition. As always with testing concepts, QA evangelists across the world have produced their own versions of what they call ‘Cloud Testing’ – exaggerating its technical challenges 😛 I believe in “Keep it Simple, silly”. But still if you have anything to add/comment/contradict please don’t hesitate to put it in words in the comments section below…



Leave a Reply

Your email address will not be published.