OpKey

Soon after the worldwide release of OpKey Personal edition 1.0 we had a Sprint Review Meeting discussing how hectic the sprint was, what all assumptions went wrong, and what good decisions made the release possible. We decided that for the next few months, we will only be focusing on Client Issues and no further Feature Development will be entertained until widely required.

We were happy for the release, and thought that handling the client issues will not be much of a problem, and we will get enough time to rejuvenate from the recent hectic deadlines & work timings. A pleasure much awaited by all the IT Professionals.

Just before the Happy Ending of the review meeting, a young lady from the pre-sales team (probably a new joinee and a fresher) raised a rather rare concern, “OpKey was build for manual testers to do automation testing with quick authoring and speedy executions. But, I feel, that manual testers must be well aware of the keywords to author the TestCases, and authoring takes much more time than the current Selenium IDE. So Can we make TestCase authoring a little bit faster and seamless?”

We ignored the concern (as we had no idea how to do it then), but our Business Analyst was so moved by the concern, that he asked me, “Anshuman, what it takes to build a web recorder for OpKey?” And I was like, “Mmmm, a couple of years, may be”.

“I want the OpKey Recorder in the coming release, which is planned next month…”

So, here was a little concern, raised by a fresher that became a nightmare for the Development Team. But, the release was planned, and the requirement was crystal clear, “OpKey Recorder in one month”

I started with the .Net based WebBrowserControl (Embedded IE) but soon realized, that many-a-sites show up differently on older versions of IE and many-a-users still use IE 6. Then I started making an Extension for Google Chrome which will tell me about the page events, but was stuck handling issues with Google’s Security measures and container object handlings.

Then there came the life savior, Selenium IDE itself. The thought of extending Selenium IDE came from our Business Analyst himself with these backing up his point:

Selenium IDE is OpenSource, so you can hook-in your code-piece in it.

Six years of rich development, bug fixes and new version support.

Huge User base, who are comfortable with the UI &Usability offered by Selenium IDE.

 Support of ancient as well as newest HTML standards & controls.

Open Forums, multiple object identifiers etc etc.

So, twenty days of hardwork, and now we are proud to announce that OpKey supports Recording via Selenium IDE. In addition to Selenium IDE, the Selenium IDE for OpKey supports the following:

It fetches more object properties and element attributes.

Create separate entities for TestScripts & ObjectRepositories, to manage them better.

Intelligently adds keywords as per scenario, like SyncBrowser, VerifyPopup and then HandlePopup.

Promotes usage of more than one property to uniquely identify objects on a webpage.

Redirects runs to the default browser(By using the GlobalVariable feature of OpKey)

Creates more user readable scripts, like Click on a button and on a link is actually recorded as ClickOnButton & ClickOnLink respectively.

OpKey has now become much more usable and user friendly by having the Recorder feature in the automation tool itself. And now I too feel that record-n-Play is one of the coolest features of OpKey.

  Not only this, we charted out an experimental study with a closed group of users which showed us that we have reduced the test case creation time by more than 60% and that was a great achievement in itself. We also figured out generic queries which we believe our end user community will have, we know testers are inquisitive. So here’s a small faq list-

Q1- Selenium IDE for OpKey, where can I find it?

Download your copy from <link>SourgeForge</link> and drag-drop the xpi file onto your Mozilla Firefox. It will prompt you to install and then restart it.

Please Note, that the SeleniumIDE for OpKey is OpenSource and the source code is available on request

Q2- Which version of Selenium IDE is being used?

Currently we are supporting SeleniumIDE version 2.3. We are planning to release the versions 2.2 & 2.4 soon.

Q3- I have a different version of selenium IDE, so will the xpi file work?

Yes, the SeleniumIDE for OpKey must work as it is all together a different Plugin. Although, I would suggest that you create different Firefox Profiles and install different plugins.

Launchfirefox.exe –pfrom Run Window

Or follow this link: https://support.mozilla.org/en-US/kb/profile-manager-create-and-remove-firefox-profiles

Q4- I am not sure I understand the port number which I have to fill, what is it for?

A local machine port is needed for the communication of OpKey & SeleniumIDE for OpKey. You can give any random number ranging from 1 to 65535. The default port(8282) must just work fine.

Just make sure your firewall or antivirus is not blocking it.

Q5- I added the xpi file, and have followed all steps. But my actions are not getting recorded by OpKey what to do?

Can you check the following?

You have entered the same port number in OpKey Recorder & SeleniumIDE-> Tools-> OpKey Settings

You have clicked on Start Recording in OpKey Recorder

You have opened SeleniumIDE for OpKey in firefox and not the SeleniumIDE

  Q6- Will this work only for Firefox?

SeleniumIDE only works with Firefox, so, Yes. Although, we plan to integrate our solution with other recorders as well to support multiple browsers.

You can also ask firefox to behave as InternetExplorer or GoogleChrome or an AndroidBrowser. Check out this link: http://www.howtogeek.com/113439/

Q7- So you are saying for each selenium action; there is an OpKey keyword, where can I find that mapping?

Document is in progress. We shall update it on the website soon.

Q8- Can I add the objects of different test case recording in one Object Repository?

Select the same ObjectRepository for different TestCases while recording.

Q9- Where is the recorder located, I can’t find it?

Soon after the worldwide release of OpKey Personal edition 1.0 we had a Sprint Review Meetingdiscussing how hectic the sprint was, what all assumptions went wrong, and what good decisions made the release possible. We decided that for the next few months,…

As we see the current trend we understand that business is accelerating, and it is accelerating at the Speed of Thought! Well exciting times for business has always meant a huge load on its supporting functions Testing being the major chunk of it. Challenges most commonly faced by our sector are-

We as always have a Limited Test Budget

To add to it a strictly Time bound Schedule

A Large number of test cases to be executed on many environments with different tools

All this Leading to high costs incurred per tests

Geographical distribution of users, add to the challenges.

  Let us take a real time scenario – Automating an E Commerce Application Using Selenium WebDriver

Now Regression Testing on one Browser OS combination takes close to 5 hours. This has to be tested on 16 different combinations of browser and OS and Two parallel test environments with each release. This sounds like one mammoth effort! One way to achieve this is – “Create your Own Test Lab”.

      You can get 30 odd machines or VM’s having different operating systems and browsers. Ask your IT team to ensure that its health is always up to date. Timely updates, handling crashes and providing 24/7 support for the ever changing business requirements. So much effort to be put in a solution which is going to be occasionally used and mostly to report the Failures! Well seems like we should look for an alternative solution.

Another alternate solution is to move your functional test automation to cloud, where we basically execute our created test scripts to identify defects in application on cloud instead of our local infrastructure set up.

This approach provides many benefits, in terms of -

On Demand Infrastructure-

The infrastructure is always available On Demand and is highly scalable and flexible to our business test requirements. We can use as many browsers and OS combinations as we like. Moreover their maintenance overhead is not incurred by us.

Test Run Time

The Test run time is greatly reduced as the Cloud allows Parallel Test Runs on different browser and Os combinations.  And once you have scheduled your runs, your machine is free to work upon for other activities. Thus helping saving test time in total.

Cost

We are no longer bound by any expensive seat licensing solutions as we Pay Per Use only. In cloud test automation, you pay only for those minutes which you need to run your test suite. This causes a significant reduction in the cost incurred for test automation.

There are many big players who are trying to make foray into this area. CresTech is one of the first to launch a cloud enabled solution. CresTech most sought after Tool Agnostic Test Automation Platform Solution OpKey is now – Cloud Enabled! So, create your Automation Test without any scripting with any test tool and run it on cloud.

  With the OpKey Cloud, you author your test scripts on your local machine. And for execution you can chose from the 100+ configurations available of operating systems and browsers to run your test suite on. These tests will be executed in parallel on the cloud environment.

With OpKey, you do not have to worry about knowing any programming language. With its more than 400 keyword available, you can author test scenarios with ease. It also provides in built object and test data management interface.  And with OpKey recorder, an extension of Selenium IDE, create your test case with ease as you navigate steps while recording. Your first test case to run on cloud is ready within moments of installing and setting OpKey Personal edition. Moreover you are provided 100 FREE Automation minutes on Cloud.

Functional Test Automation – Let’s do it the Cloud Way

As we see the current trend we understand that business is accelerating, and it is accelerating at…

OpKey Integration with ALM Tool QC

Application Lifecycle Management

Abbreviated as ALM, Application Lifecycle Management refers to the…

OpKey can be easily integrated with Continuous Integration tools to automate the execution process. The automated, continuous execution process increases productivity.

Let us consider an example of integrating OpKey with a commonly used tool Hudson.

About Hudson

Hudson monitors the execution of repeated jobs, such as building a software project. Hudson focuses on the following two jobs:

1)      Building/testing software projects continuously, Hudson provides an easy-to-use so-called continuous integration system, making it easier for developers to integrate changes to the project, and making it easier for users to obtain a fresh build. The automated, continuous build increases productivity.

2)      Monitoring executions of externally-run jobs, Hudson keeps track of outputs and makes it easy for you to notice when something is wrong.

Integrating OpKey with Hudson

A Job can be created in Hudson for passing arguments to OpKey. The example of Jobs in Hudson can be seen in the screenshot.

OpKey reads from an input file which is in a human readable form.

This file is located in the current user local directory.

It provides an advantage for multiple users using same desktop to maintain their data individually.

Another advantage of using this location is that this location accessible to current user only.

This architecture is designed to support the users using Citrix environment.

The input file contains the information for executing a suite in OpKey like

Project Path

Mode

Plugin

Suite Name

Session Name

Build Name

Step Time Out

Take Snapshot

Snapshot Quality

Result Report

This information can be easily updated in the file as per the requirements of the user for executing a Suite.

OpKey has a great feature to execute the suite using command line inputs, using this feature user can pass various combinations for input arguments to execute a suite and achieve the results as per the requirements. After updating the input file with the required information you can easily pass a command using the Continuous Integration tool to initiate execution. OpKey provide various set of features and flexibility for execution. Some of its features are:

Use Automatic build and session naming.

Strict check for input information correctness.

Create Dynamic Suite from the existing flows.

Achieve outputs in multiple file formats like CSV and Excel.

Skip non essential information from input file.

OpKey Integration with CI Tool Hudson

OpKeycan be easily integrated with Continuous Integration tools to automate the execution process.…

World is moving towards Open Source adoption. Currently 1.5 million Users are registered on www.opensourcetesting.org .There are multiple tools available in open source world to automate application available on different platforms.

As of now, there exists no tool, open source or otherwise, to facilitate automation of test cases which run across platforms. For e.g. selenium is great to handle across browser testing; but has no support when a need to do image based testing; web service testing or mobile application testing, arises. Currently there is no framework available where one can make use of more than one automation tool in the same test case.

While Automating applications for different clients we feel some challenges that are mentioned below:-

What are the challenges?

One Automation tool often doesn’t support all Platforms

Selenium\Sahi is great for web, but how about mobile, flash, flex or web service.

No Single Solution available in market

People need one stop solution for their application that is available on multiple platforms.

People cannot migrate their Script from one tool to other one

World is moving toward open source so people want to migrate their Scrip from a paid Automation toll to open source tool. But they are unable to do so.

High Skilled professional are required

Organizations have to hire automation experts who have extensive knowledge of Scripting languages and for different tool they have to learn different languages.

 Let’s talk about some typical Business Scenarios

Business Case 1:

A mid-sized product development company is developing a rich media app, which includes online videos and rich media advertising and marketing. The application is required to have a great user/UI experience. Thus it was no surprise that Flex was dominant here.

The Challenge:

To test the application frequently in an agile development environment, the QA team decided to go for test automation using an open source test automation tool called Selenium. Since Flex was dominant, the QA team to it surprise found out that selenium had problem recognizing components developed using Flex, thus leading to incomplete or unsuccessful test automation.

The Analysis:

A thorough analysis of the situation demanded using a set of tools that can be used to automate application components developed in Flex. After a detailed study the QA team narrowed down on Selenium, Sikuli and FlexMonkey. This required a platform which could support multiple tools.

 Business Case 2:

An US base media company has decided to develop an application that is available on multiple platforms-Mobile, Desktop, Web and Cloud for transfer of real time information from users and broadcasting centers across the world.

The Challenge:

The prevailing business environment in the social media space required the client (US based startup) to have the solution implemented as soon as possible, putting extensive pressure on the project timelines. This and given the budget constraints, the vendor had to opt for a test automation strategy that had to utilize open source test automation tool.

The Analysis:

The QA team based on their analysis of the situation at hand and the open source tools available came to the conclusion which had serious implication for their over ambitious QA goals. The solution demanded extensive interface testing which required the test to be run on different platforms (Web, Desktop, Mobile and Cloud).

A typical Business process Test/Integration test is depicted in the figure below: 

  For this the QA team had to use combination of automation tools (Robotium, Selenium, AutoIT and SOAP-UI) to automate the end to end business processes, as no one tool had the capability to run test across all the available platforms i.e. Web, Desktop, Mobile and Cloud. This required a platform which could support multiple tools.

 What should be the Solution?

Technology independent Platform

The Solution provided should be independent of technology used in developing and testing Applications.                                                                   

End User Abstraction

The Solution provided should be intended to target large segment of Audience. The largest segment in an application test life cycle is owned by Manual Testers. The solution provided had to have such an ease of usage that any tester; who is not aware of automation; or any script or any language should be able to get started with it

Seamless integration with Automation tools

The solution provided should be the solution designed has to allow easy integration with any tool whether commercial; open or home made.

 How OpKey tackle this challenge?

Seamless integration with Multiple Automation Tools

OpKey works as a one stop portal for multiple Automation tools like Selenium, QTP, RFT, Robotium, Sikuli, SOAPUI, and AUTOIT.

One Stop Authoring Platform-

Common Object Repository-

OpKey facilitates user a single Object repository that can be used by any of above mentioned tools to create test   case.

Common Keyword Repository-

OpKey provides single set of prebuilt keywords that are tool independent and user can execute same test case by using separate tools to multiple tools in single Go.

A Single test case can contain keywords from different tools. Let’s take an Example.

Common Data Repository –

OpKey provides common data repository that can be used throughout application and on all available tools.A single data repository that helps business users to manage data effortlessly.

 One Stop Execution Platform-

OpKey facilitates user to choose his/her flow with any of the Automation tool. It means a Test case that is created during Authoring can be executed with multiple Automation tools. If user wants to migrate from a paid tool to Open Source then he does not add any migration cost in his estimation list.

It is basically providing HAPPY MIGRATION!!!!

Let’s take Business Case 1:-

To Automate the above mentioned Scenario of Media Application we have used three different Plugins

Web Components:- Selenium

Image Components- Sikuli.

Flash Components- Flex Monkey.

So a Single Data repository and test case for all three Plugins has resolve the  complexity of above scenario.

Let’s take Business Case 2 :-

Which All Tools are used to Automate the above give Scenario:-

We have used Above mentioned Tools to Automate Business Case 2 and We didn’t need to create separate Object Repository,Test case or data repository to Automate it.

OpKey provides a lot of feature that makes testers life easy  and Business As well.

Multi-Tool Test Automation Platform

World is moving towards Open Source adoption. Currently 1.5 million Users are registered on www.op…

What is Data Driven Testing ?

Using variables and input data that are stored in external sources like Excel,database to test an AUT (Application under Test)  is a common practice that let the tester to see how the application respond with various inputs. It increases test coverage and also helps to reduce the necessitaties of creating more test cases with different variables.We  generally term it as Parameterization.

Why do we need to test same Test Case using different Datasets?

Parameterization allows user to check same operation with multiple set of data.In some scenario user has to provide multiple set of data for a single test case for example if you want to check the Login credential so user has to provide multiple values for the Login. To perform this operation user has 2 options, one is user can provide these value one by one manually after completing the each flow or second one is user can use multiple set of data value  in the test case so they can easily execute the flow without changing value manually. This is also called the Parameterization when you can test AUT (Application under test) with multiple set of data values.

Advantages of data driven testing (Parameterization):-

You can test your application with multiple set of data values during Regression testing.

You can test your application with positive and negative scenarios.

Different ways to parameterize your test case in OpKey:-

You can do the parameterization in OpKey with the help of Data Repository. We can parameterize test data by two ways:-

Entering Data directly into the Data Repository.

Importing Test Data from External files.

Entering Data Directly into Data Repository:-

Add Data Repository -> Add columns and rows according to your need -> save this Data Repository

Importing Test Data from External files:-

Add Data Repository -> Import any Excel and CSV file -> save this Data Repository

(You can see in detail how to import test data from External files http://opkey.crestechglobal.com/help/pages.php?subid=57&accordid=3#Data-Repository-File-Features)

Consider the simple test case of application login. You want to test the application response to right and wrong pairs of username-password. You might also be interested in block-after-three-attempts scenario. We start with a simple test script as OpenBrowser(“internet Explorer”, “http://www.yoursite.com/loginpage.php”) TypeTextOnEditBox(“txt-box-1″, “your-username”) TypeTextOnEditBox(“txt-box-2″, “password”) ClickButtonAndWait(“bttn1″) VerifyObjectExists(“logout-link”)

The idea is, if the logout-link exists then we say that the page accepted the given username and password. Depending on the validity of username/password pair we provided, the test either passes or fails. A problem with this approach is that if we need to test with a different username/password pair, we need to alter the script and then rerun it. That will not be feasible if there are say, tens or hundreds of data pairs. We need to parameterize the username and password fields (variables) and feed the data from a table. This is where data repositories come in handy.

A data repository is a simple table which holds the data for one test. There could be many-to-many relationships between test cases and data repositories but for now let there be one data repository per test case. A data repository is a table in row/column fashion. Each column represents a variable/parameterized-data from the test case. In our case we would have two columns USERNAME and PASSWORD. Each row represents one iteration of test case with values taken from the cells in that row. Let us create a data repository as such.

Go to test case module. Add step and provide keyword in this step. Select the provide data field under Provide Data value for Data Input Argument panel. Instead of providing static value, go to Data-Repository section. Expand the newly created item. Select the username column. Do same with text keyword. Select the password field of second TypeTextOnEditBox step. Change the static value to Data-Repository. Expand the data-repository from the drop down. Select the password column. Test case looks like below image.

Now execute the test and watch. The whole test will run three times, one with each pair of username password. We say that the test has iterated three times, which was the maximum depth of any column in all the repository columns used in current test case. You can increase iterations by increasing the number of rows. For much complex tests which are heavily parameterized, the number of columns will increase.

So we have seen how we can easily parameterize OpKey test scripts and iterate a test over a large set of test data. In next post we will discuss scenarios when data in one column repeats for data in other columns.

Advanced Test Parametrization

In part 1 of this article we discussed on how we can parameterize OpKey test scripts and how we can iterate the test over a large set of data. Now we will be discussing some advanced aspects of it. In our previous discussion we tested the login and credential validation process of an application. Now we want to test the product-search feature of the same application. We have a text box where we enter search keywords. We hit enter and we are presented with a page full of relevant products. But before we do it we need to login first. So we create a test script as:

//this part does the login

OpenBrowser(“internet Explorer”, “http://www.yoursite.com/loginpage.php”) TypeTextOnEditBox(“txt-box-1″, “your-username”) TypeTextOnEditBox(“txt-box-2″, “password”) ClickAndWait(“bttn1″) //here we search TypeTextOnEditBox (“txt-box-3″, “search-keyword”) VerifyObjectExists(“item-count-link”)

When executed, this test script opens internet explorer, logs in with provided credentials and searches for a product. We now want to parameterize this. There are a total of three variables – username, password and keyword. We could go with a data repository of three columns but then we will have a slight problem: we have just one (or limited) set of login credentials but fairly large set of keywords. A basic data repository will look like:

To try this feature, go to the data repository and delete the redundant cells. You may select the cells and clear them. If you execute the test right now you will see that in fourth iteration OpKey tries to login with blank username and password. This is the default behavior. In the fourth row the username and passwords and empty hence the TypeTextOnEditBox is provided empty. This behavior can be overridden.

Go back to the test case. Locate the username parameter it would be bound to data repository and the icon next to it will tell you so. We need to redo the binding process. Expand the data repository but do not select the column now, not yet. Rather look to the top of the popup. There you will find two radio buttons CYCLIC and CONTINOUS with continuous being selected by default. Select the cyclic button and then select the USERNAME column. What we did is that we bound the username column in cyclic fashion. So once the data in the column finishes (after three) iterations, OpKey will automatically cycle to the first element in the set. Do same with password field. Save and run.

Data Driven Testing with OpKey

What is Data Driven Testing ?

Using variables and input data that are stored in external sources…

The Past

Software testing is a necessary process and everyone seems to agree on that. Where people differ is the timing and depth of the initiative. The industry has long moved from a stage when testing was considered a post-development, maintenance…

While automating a web application, quite often you might have experienced failure of your test cases during their execution due to changed properties of dynamic objects fetched in your Object Repository. These dynamic objects often change their properties such as name, id, value etc.

To handle these dynamic objects, you can use ‘Is Regex’ feature

The following illustration will explain about dynamic objects and the usage of ‘Regex’. Here I’m using a website in which the name property of ‘Name’ edit box usually changes whenever I launch that page again, which means my test cases for this page would fail due to those changes. So, I will be using ‘IsRegex’ which will handle the changes in the name property so that my test cases don’t get affected.

In the following screen shot, while fetching the objects of a website, I got the following properties

The above ‘Screen shot 1’ shows the window of ‘Import from Application’ where I’ve fetched the ‘Name’ text field.

Properties are shown in the following screen shot

In above screen shot you can see the Object Properties, in the ‘name’ field value is as ‘ctl00$ContentPane$ctl00$Name55’.

Now if I again fetch the properties of ‘Name’ edit box of that webpage, I will get the following properties

In above screen shot you can see the Object Properties, in the ‘name’ field value is as ‘ctl00$ContentPane$ctl00$Name87’.

Similarly if I fetch again the properties of ‘Name’ edit box of that webpage, ‘name’ field value is ‘ctl00$ContentPane$ctl00$Name90’.

This clearly implies that during execution of my test cases, the ‘name’ field value will change again.

Now, to handle these changes I will use ‘Is Regex’ which will handle the above changes.

Now, after fetching the object, now observe the following screenshot

                                                         Screenshot  3

                                                        Screenshot 4

The above screen shot 3 shows the view of Object Repository window and screen shot 4shows the view of ‘Properties’. In screen shot 4 you can also see ‘Is Regex’. Now, in my case the ‘name’ field value are as ‘ctl00$ContentPane$ctl00$Name55’ ‘ctl00$ContentPane$ctl00$Name87’, ctl00$ContentPane$ctl00$Name90’. On observing these field values you will observe only last two values are changing else everything is same.

So, I will rename the value of name in ‘Properties Window’ as ‘ctl00$ContentPane$ctl00$Name’ and I will check the checkbox corresponding to the name property. Following screen shot will make it clearer to you

In the above screen shot 6, I have changed the value of ‘name’ to ‘ctl00$ContentPane$ctl00$Name’ and I’ve checked the checkbox under ‘Is Regex’ corresponding to the name property.

The Past

Software testing is a necessary process and everyone seems to agree on that. Where people differ is the timing and depth of the initiative. The industry has long moved from a stage when testing was considered a post-development, maintenance activity. We gradually interleaved the development and testing process. Both the development and the testing process tend to merge, bringing both new benefits and new challenges. The obvious benefit is fast feedback. We need to check an error as quick as possible. The not-so-obvious challenge is maintenance and overhead. This will come to haunt us later.

Long since the early days, the industry adopted Continuous Integration. Whenever a developer is ready to push some code, he first has to pass it through the Unit Tests. If all is green then only can he actually commit the code. Even per-commit jobs are gaining popularity. People are delegating the responsibility of code validation to the build server itself. The idea is to integrate the Code – Build – Test – Deploy process in a single pipelined workflow. Every stage in the pipeline provides immediate feedback to the preceding stage.

On the developers’ front automated Unit Tests have established themselves as an invaluable checkpoint. What about the managers and the business users? Everyone wants things to be well tested. But Unit Tests and their reports are not much meaningful for them. The unit tests are also not reliable in integration testing. Components having passed their unit tests may still fail to integrate. Here comes the need of functional testing. Organizations have dedicated departments of manual-testers. They validate the product at whole build level. Detailed reports are generated, modules’ health is monitored and bug tracking is integrated to this process.

The Present

Bulk of functional testing has always been manual. A big reason was lack of tools which can automate applications, detect and report the failures in doing so. But the last decade saw an exponential growth the automation sector. The authority of pioneering products was challenged by emerging tools. A majority of them were open source, promoting accessibility to even small companies. So when we can automate the functional testing process, why not integrate it to the CI server? Why not unit test and functional test every code batch? Here comes the hurdle which we knew but overlooked at the initial stages – maintainability.

Automation scripts are traditionally codes. They are codes which test and validate other code; and they being code, need to be designed, written, maintained and reviewed. But they have a big advantage – they can execute just as any other code. That’s why JUnit and similar frameworks are so popular. Problem is they don’t scale well with functional testing. For one thing, functional tests are very broad, covering many workflows, scenarios, test cases and unique steps. Such big a code is difficult to maintain and soon outgrows the product itself. Another thing is being code; they don’t truly lie in tester’s domain. They need seasoned developers. You can have programmers write the test scripts but then a load mismatch remains.

Welcome the era of Keyword Driven Automated Functional Testing. Encapsulating the programming process in test scripts gives many benefits. First and foremost, we no longer need hard programmers for testing task. Any programming-literate can do the task. The Test scripts are more maintainable and language agnostic. Now we can actually have both unit tests and functional tests which can be triggered on each code check in. While unit tests validate code conformity, functional tests validate product’s behavioral consistency. And both work unattended. Developers can get feedback meaningful to them from their unit tests. Business users can get module health status meaningful to them from their functional tests, all in near real time.

The Future

There is no limit to innovation. Imagine the development of a web application. Let’s say the application needs to be supported on all major browsers, IE, FireFox, Chrome, Safari, and Opera. Across all Platforms: Windows/Linux/Mac Desktops/Laptops. Let us push the limits, ask for support on all major mobile devices: Android, iPhone, Blackberry, Windows; talk about orientations, resolutions. Now lets us ask the test engineers to automate the whole thing. Run all regression flows on all Platforms/browsers on all check-ins. After all we want a strict quality control.

Difficult to accept but this simply cannot be done! Not without an automation framework which can do the heavy lifting. None of tools or frameworks in the market can cope with the requirements that we will be facing in coming years. What the industry needs is a test automation framework which:

Can integrate with the existing CI solutions.

Perform functional tests as seamlessly as Unit Tests. Run a test every night, every check in or on every Run-Test request.

Scripting process is better to be keyword driven than language based. We want one simple ClickOnButton; not a bunch of find, hooks, performs, again recheck hook and so on.

Provides high maintainability of Scripts. Should not need programmers to modify that validation.

Can gracefully orchestrate multiple tools to perform activities which otherwise don’t come under ambit of any one single tool. If Selenium cannot click on an image, get it done with Sikuli.

Can launch multiple configurations of a same test in parallel. We want to take the same test on all five major browsers.

Gives a uniform interface over heterogeneous tools. Tools may vary in language support even on platform support but we need it. We need to test our application on both Linux and Windows.

Generates business oriented reports. We want reports in terms of Business flows not individual test cases. We want health check of modules. We want basic metrics. We want to visualize defects over last five builds.

Drill down to step level failures, so that both testers and developers can get workable information regarding the failure.

Integrate with major Defect management tools like Bugzilla, Jira etc. Should be capable of automatically logging confirmed bugs.

Integrate with major Application Lifecycle Management software like HP ALM.

Now that is a whooping wish list and no product in the market claims to do it all. But this wish is soon going to be our requirement. Success and failure of products will depend on timely validation on all the target environments. Even if we somehow manage the sheer volume of test scripts and all the plumbing, we are stuck at the test infrastructure. We need to test on five browsers, each on latest version and three backward versions. We need to test those browsers on Windows, Linux and Mac. Now that makes about a hundred parallel executions on a hundred infrastructure. How are you going to manage that infrastructure? What happens when you need a bunch of Mobile devices? Are you going to purchase it all? Rent it all? Of course not.

The future is of cloud computing. We will rent browsers on cloud. We will rent devices on cloud. We will rent the execution systems on cloud. Volumes test data will be stored on cloud, analyzed on cloud systems and reported from cloud. All we need on premises is just one CI tool and one test-automation-tool. The cloud will provide all the rest. We will rent resources on pay-per-use basic. Pay for the tests we run. No need to inventory the hardware resources, browsers, devices and all.

And our steps to it

That future is not so distant. You can get the machines from IaaS provides like Amazon or Rackspace. SauceLabs can rent you browsers; Xamarin can run your tests on rented mobile devices. These solutions are already available. But are these true solutions to your pains? No. you need an end to end automation solution and these isolated initiatives don’t help much. A service is required which can bring all these together. A lot of tools have targeted the challenges and most have succeeded too. But until now it had been mostly isolated, unorganized attempts. Notable efforts are from QTP – which along with its Open Test Architecture is the big brother of the current generation of Test Automation solutions and the other is OpKey – the next generation of Test Automation Framework.

OpKey is determined to solve the problems which the future automation engineers and managers are about to face. It is designed for scalability and maintainability. The future is of multi platform deployment and so the challenge lies in multi-platform simultaneous quality control. OpKey is the only player which can tackle this challenge. Being scalable means anyone, even small scale developers can start with a basic OpKey setup and then scale up as the application grows. A time will come when a use case will span Web, Database, Web-service and Mobile interactions. At that moment you will realize that only OpKey can orchestrate that many tools, platforms and resources.

How to handle Dynamic Objects with OpKey ?

While automating a web application, quite often you might have experienced failure of your test…

Why is it necessary to Automate Testing on Mobile

There has been an explosion in Mobile Technology over the last two decades. We moved on from simple voice (1G) to voice-text (2G) to voice-text-gprs-edge (2.5G) to voice-text-data-external app access (3G). We will soon be headed to 4G, which claims to support 10 times higher bandwidths.

We have also seen the rise of the Smart-Phone and other mobile devices that support email clients, calendars, full web browsing, social networking apps, camera’s with easy sharing integration, video players such as youtube, maps, office apps, games etc.

·         With the Smart-Phone revolution upon us and the multitude of applications being released on a daily basis there is an increasing need for tools and automation to test all of these mobile applications.

·         Newer versions of apps are released every other day. How to cover their test cycles in maximum devices/platforms manually?

·         Any changes in your app, needs testing of the whole app again. One cannot do it manually.

·         How to ensure that your app will sustain in such enormous App Stores/Markets.

  Currently available Solutions and their feature comparison

  *I couldn’t do an effective comparison for PerfectoMobile as I couldn’t get my hands on the software.

-No

 Points to ponder upon before choosing an Automation Tool

•       There is some difference between emulators & actual devices, so we may need to test on both.

•       What should be the Object Recognition technique: Property Based Approach or Image Comparison Based Approach?

•       A feature may be working on a device but may fail on another, so the more devices we test upon, the better is our stability.

•       No single tool can support all functionalities. We may need one functionality of a tool and another functionality of another.

•       What parts of the app can be tested through automation?

•       Platform agnostic tools may be of much help if the behavior of your app is same over multiple platforms.

•       Can we use multiple tools for handling complex scenarios?

  What should be the optimum solution?

•       An IDE which supports both PropertyBasedApproach as well as ImageBasedApproach.

•       Keyword Based approach: To avoid scripting learning curve.

•       Extensibility: Multi tool plugin support.

•       Avoid need of Source Code: Because in the end you will be publishing the App not the Agentified-App (App + TestAgent)

•       On demand Mobile Cloud Support: You need not buy each & every device to test a single app.

•       Multi platform support: A TestScenario may be spread across Web, Mobile & Desktop App. A sample Scenario you would like to test is: Create a user through Web Interface, Login through the Mobile app, do some testing and use the desktop utility for expiring all open sessions of that particular user.

•       Component Reusability, Data-Sheet support, Multi lingual Object support.

•       Meaning full Reports: Must contain the execution environment details, screenshots.

Maintainability is the ease with which a product can be maintained in order to:

isolate defects or their cause

correct defects or their cause

meet new requirements

make future maintenance easier, or

cope with a changed environment

In the world of Automation testing, Engineer’s face several challenges in creating and maintaining the automated test scripts. OpKey works like a charm in meeting those challenges.

Example: Changing properties of objects

The challenge mostly faced in automation testing is the changing properties of objects. Let’s have an example for OpKey dealing with such situations. Suppose I have a test script which clicks on an object with name “Sign In”. So I have added an object in ObjectRepository with “name” property set to “Sign In” as shown below:

Now I have used this object in my “TestCase” and executed it. It works perfectly fine but when next day I executed the same script it fails saying “Object with specified properties not found”. To resolve the nightmare when I checked the object properties I found that the site developer has changed the “name” property of that object. The new “name” is “Sign_In”. So instead of modifying my whole script I simply modified the updated property in my “ObjectRepository” as shown below:

Again executing the same script with this minute change in object properties worked perfectly.

      Example: Dealing with frequently changing data

  Another challenge we normally face is the frequently changing data. Let’s have another example to see the ease with which OpKey handle such situations. Suppose I have created a test script which takes “username” and “password” as input for login. So I have created a DataRepository in OpKey with two columns “User Name” and “Password” as shown below:

Now I have used this DataRepository in my testcase. It works perfectly till these user’s are eligible to login but what if these users are deleted by the administrator or they are no more eligible to login? Do I need to re-write my test scripts? The answer is NO; OpKey ease with data management provides flexibility to modify the existing data without modifying the scripts. I can simply provide the updated credentials in my DataRepository as shown below:

Again executing the same script worked absolutely fine.

Example: Dealing with changing objects behavior

Another challenging situation arises due to the modifications in behavior of an object. Let’s have another example for this situation. Suppose I have a “TestCase” skipping the text in a non-mandatory field. So I have added steps in my “TestCase” for filling the mandatory fields and clicking the submit button as shown below:

It was executing fine till I faced a situation where one non-mandatory field becomes mandatory. But thanks to OpKey for meeting the new requirements and adapting the changing environment very easily. I have simply modified my “TestCase” by adding a new step for the new mandatory field as shown below

Just with this simple modification my existing script again started working absolutely fine.