Subscribe to Advertising & Marketing Review!|
Contact Ken Custer at 303-277-9840.
Mobile Device Testing Services
by Glen Emerson Morris
Copyright © 1994 - 2010 by Glen Emerson Morris
All Rights Reserved
keywords: Internet advertising, Internet marketing, business, advertising, Internet, marketing.
For more advertising and marketing help, news, resources and information visit our Home Page.
In the previous column we took a look at open source solutions to the mobile device browser/platform testing problem. The open source solution uses development tools like Joomla to automatically handle page reformatting and interface issues for different browse/platform combinations, like the iPad and Android devices. While this will solve a number of development and testing problems, there are several areas it doesn't cover. These include functionality, performance, scalability and usability testing. Whether done in-house or outsourced, this kind of testing requires a significant investment of talent, tools, and most of all, a budget.
Is it worth the effort to push your company's Website on to mobile devices like the iPad and Android? You bet, but keep in mind you have to make sure your Website looks and behaves completely native to the mobile device. According to the mobile device testing service uTest, Consumer conversion (from surf to purchase) on iPhone or Android native apps is 3X - 10X times that of mobile web sites. Why? Because users are more comfortable with the device. In most cases, the usability of the native app, including page load speed, is much greater than the mobile web.
The question isn't whether to go native on mobile devices, the question is, How can you possibly afford to? As we said in our previous column, the cost of acquiring a single mobile device can run between $1,000 and $2000, counting the cost of the device and the usual mandatory two year contract required. When you consider that there are over 2000 combinations of device, browser, operating system and carriers, this adds up. Just supporting five different devices could cost between five and ten thousand dollars, and that's not counting the labor costs for running the tests.
Fortunately, over the last few years a number of companies have developed services to help meet mobile device testing demands. These include:
Getting some other company to do the work can be the easiest way to handle testing, but it can also be the most expensive. In order to make this approach work, adequate documentation needs to be available. The QA team provided will know nothing about the product, and if adequate testing documentation isn't provided to the QA team, part of the budget will go just for them to figure out how to test the product.
Pros: Provides a complete testing solution. You don't have to add more people to your payroll.
Cons: Usually the most expensive way to go. Requires test documentation to be cost efficient.
Online manual testing of online mobile devices
Several companies have developed Web based interfaces that allow full access to a variety of mobile devices. You sign up and pay by the hour for access to a particular device. Costs can range anywhere from $8 to over $16 an hour, on top of a setup charge that can cost $0 to $200+. This approach when no test documentation exists, but it's always better to have it.
Pros: Testing can be performed by non-engineers, as no special training is required other than the operation of the application itself.
Cons: Slower than automation.
Online automated testing of mobile devices
A few of the online mobile device testing services have developed automation frameworks that allow you to record, or program, a series of actions then play them back automatically. The more sophisticated versions handle all of the device specific issues for you, so an automation script recorded on one device can be played back on any other device no matter what browser/OS it uses.
Pros: Faster testing. Testing is precise and highly repeatable.
Cons: Requires engineering skills
The following is a list of some of the major companies providing mobile device testing services.
Provides online access and testing services using their QA staff for Android, Linux, RIM, Symbian and Windows-based devices. Their automation tool TestQuest Pro can simultaneously run different tests on multiple - and different - target devices. Their development tool CountDown integrates test design, management and execution while enabling collaboration among distributed development teams.
Contact info: www.bsquare.com Phone 1.888.820.4500
This company offers one of the largest selections of device/browser/platform combinations available. Their own automation system will allow tests recorded on one mobile device to be played back on nearly any of their other devices. They go even further though. Their automation framework will interface with most of the major commercial and free automation systems, including WinRunner, Silk, Rational products and Selenium.
Contact info: www.deviceanywhere.com Phone: 1.650.655.6400
This Indian company offers both mobile application development and testing services for the iPhone, Android, Blackberry and Windows based devices.
Contact info: www.payoda.com Phone: 1.212.400.7541
This Israel based company uses a proprietary automation language called ScriptOnce, capable of executing the same tests on different devices. Very reasonable pricing. Their Full Cloud package of 600 devices - all platforms, all carriers, all countries, ranges from $16 an hour for pay as you go, to $12 per hour in blocks of 50 hours. The single platform cloud (one device) ranges from $14 hour pay as you go to $10 an hour in blocks of 10 hours, and $8 an hour in blocks of 50.
Contact info: www.perfectomobile.com Phone: +972-3-9260100 (Israel)
Headquartered near Boston, uTest functions as a marketplace for software testing services and is about as close to cloud QA as you can find. The company provides real-world QA services through its community of 30,000+ professional testers from over 165 countries around the world.
Contact info: www.utest.com Phone: 1.800.445.3914
Most of these companies offer a few hours of free trials. It could well be worth your effort to check them out and see what your company's Internet presence looks like on an iPad or Android. Given the number of so called iPad killers scheduled for release over the few months, the number of devices you're going to have to support is only going to grow. It's not too early to start planning how you're going to manage that.
Glen Emerson Morris was recently a senior QA Consultant for SAP working on a new product to help automate compliance with the Sarbanes-Oxley law, an attempt to make large corporations at least somewhat accountable to stockholders and the law.
He has worked as a technology consultant for Yahoo!, Ariba, WebMD, Inktomi, Adobe, Apple and Radius.
Back to top