Is there any Tool Available in the market to test the localized mobile apps under different locales.
Like, the Typical Tests performed on localized mobile apps will be ..
1)Application functionality - For basic navigation
2)String Truncation Validation
3)Validate the translated Symbols and Images must be appropriate to the Target Language
and so on ..
Please provide me your valuable information on this ..
Really thankful to you..
I have some experience using Robotium to automate the localization testing of our mobile applications.
The main thing you want to look for in any automation solution for localized applications on any platform, is the ability to uniquely identify each object you will need to interact with (buttons etc.), using an identifier that will not change on different locales. That way, your automation code will run on any locale (we would not want to have a different code base per locale).
In my experience, it's much quicker to develop a Robotium (Java based) solution for automated testing of Android applications (other solutions such as SeeTest exist, but I have not found it great).
QTP I've found to be very useful for desktop applications, but not mobile. Selenium is good for automating web applications, and you can use Selenium Mobile to interact with the browser on a mobile device if you need to.
I blogged recently detailing how we use Robotium to test our Android application, and my experiences using it, you can find this at the link below.
Automate your Android App Testing Using Robotium
When testing your application on the devices and networks that it’s intended for, carrying out user acceptance testing in the target market could highlight any market-specific concerns and return some very useful market specific feedback. Consider a beta (test stage) program, customer early-adopter program or work with a company like ICanLocalize who use only translators that translate into their native tongue.
If, for example, your global mobile app implements its own pop-up touchscreen keyboard, you should consider the needs of your international users to ensure that they can enter text in their native script. While mobile devices and smart phones are being shipped with increasingly higher screen resolutions, usability and aesthetics need to be considered too. Translated text can often expand by as much as 25 percent, affecting the layout and visual appeal of your app. Some scripts may be very difficult to read if the displayed font size is not sufficiently large.
There are more things to consider when localizing an application and I’m not talking about plain old translation of code here. Any company hoping to win over an international audience can make this job significantly easier by planning ahead for the translation process:
Think carefully about words used to describe menus.
Get a native-language linguist to test your application
Avoid colloquialisms
Consider navigation.
http://www.icanlocalize.com/site/2012/10/planning-ahead-for-translation-when-developing-a-mobile-app/
Related
I am working on implementing accessibility (for visually impaired individuals) for one of our web application. It need to be ARIA compliant. Right now we are testing our changes with screen reader manually.
For example we have Tree control in our application. I open NVDA screen reader and then navigate through my Tree Nodes. NVDA screen reader speaks out
Node XYZ expanded, (When I expand XYZ node with right arrow key)
Node XYZ collapsed, (When I collapse XYZ node with left arrow key)
Along with the voice it also write down this text.
But all this is manual. Now we want to setup automated test cases for the same so that any regression bugs can be caught by are test cases. Do there exist any such tool which we can use to automate our test cases. Any direction will be helpful.
PS: Just for a sake of comparison. We have nunit to write test cases for c# application. After writing test cases we integrate them into our build process. Any breaking change is caught when we run the build. I am looking for something similar to test out our aria compliance and screen reader's behavior with our web application.
I don't know of any existing tools for testing screen readers, however, there are accessibility APIs that test websites and web applications.
axe-core from Deque Systems is widely used and well-supported.
I wrote a python package to run automated web accessibility tests that uses axe-core and selenium.
While it isn't quite what you are looking for, it does cover about 60% of accessibility guidelines, including aria roles and attributes. It should help with determining screen reader usability.
You could integrate axe into C#, similar to my python package and the Java package, also created by Deque.
I hope this helps!
It sounds like you're already performing some pretty good manual accessibility testing against your web application, which no automated testing tool is going to be able to replicate completely. That said, if you're looking to take care of any low-hanging fruit with an automated solution, like Kimberly suggested, there are several automated accessibility testing tools out there that you can relatively easily integrate into your existing web application's testing framework that might help you.
One such tool is Continuum, which doesn't have a C#-based library offering at the moment, but could be used in a separate testing framework to be run against your web application after it has already been built. This may be preferable depending on your use case, as code linters for accessibility aren't perfect and are highly language-dependent, whereas testing the HTML of your web application more closely matches the screen reader use case you say you're trying to test for. You could even integrate Continuum into your existing CI/CD process to make sure your application is tested during development as opposed to afterwards, to reduce your manual accessibility testing load.
Continuum has a few sample projects to get you started, depending on your technologies of choice. Free versions are available at webaccessibility.com if you're interested. Most of them are Java- or JavaScript-based at the moment.
Appreciate this is quite an old question, but having explored this area a lot recently thought was worth updating with the state as of 2023 as there is now some progress in this space.
Current tooling available at time of writing (that I’m aware of, may not be exhaustive):
guidepup - NodeJS automation for VoiceOver and NVDA supporting all keyboard commands and getters for spoken phrases (disclaimer: I’m the author).
auto-vo - CLI for navigating sequentially through a page with VoiceOver and reporting the spoken phrases, also exports a separate Node module for some interactions with VoiceOver.
screen-reader-reader - NodeJS automation for VoiceOver and NVDA for starting, stopping, and getting spoken phrases.
web-test-runner-voiceover - NodeJS plugins for #web/test-runner to automate VoiceOver testing.
nvda-testing-driver - .NET automation for NVDA supporting all keyboard commands and getters for spoken phrases.
assistive-webdriver - NodeJS implementation of a Webdriver server that allows remote testing of screen readers (e.g. NVDA, JAWS) running in a VM.
As stated in other answers, there are also a number of static analysis tools such as axe, as well numerous browser extensions offering similar static analysis, and companies such as Assistiv Labs offering remote environment services to interact with screen readers manually (similar to SauceLabs/BrowserStack/etc. but for screen readers, magnification etc. - no affiliation and haven’t used services so can’t vouch, simply an observation).
Worth calling out that none of these cover the full range of a11y requirements - there is more to a11y than just screen readers. A combined/layer approach including automation, manual testing, and user testing likely preferred.
As far as I know TideSDK is what used to be Titanium Desktop, right?
I'd like to ask if somebody has experience in reusing (mostly the business logic and data access) components of Titanium mobile in TideSDK?
Ive developed for both and in my experience Titanium for mobile and TideSDK are fairly incompatible at this point.
This is one of the reasons Appcelerator open sourced the project and stopped working on it. It was diverging so much from their company goal of a robust mobile API that it did not make sense for them to work on it. The API's at this point are very different, as well as the technology and workflow, across the dev stack from UI to business logic, to data access, to backend.
The only parts I was able to reuse were pure JavaScript utility files, and the Titanium local DB access scripts.
You can use most of your javascript code within TideSDK without much modifications. There is fundamental difference between various UI components on Desktop platforms and Mobile platforms. Taking example .. in desktop your application have window..and u can have separate UI Items like system tray, taskbar etc. whereas in mobile you would generally control the entire screen and various UI components are very different than desktop.
This is why you need to implement the native interfaces in desktop and mobile separate way but most of the business logic would remain unchanged.
Just new on software testing...
Regarding testing, I think GUI applications are pretty difficult to automate. Some testing involves with interacting particular GUI objects in particular sequence (e.g., clicking buttons). The interface often changes from one window to another. And the timing and sync sometimes also pose an issue (e.g., recording mouse clicks and replaying may screw up).
Is there any solution for testing such applications with less human labour? Thank you for sharing your experience.
Yes, GUI apps are indeed tough to automate. Regardless of the app's technology (Swing, web, WPF, iOS), you first have to focus on automating high-value tests. Moreover, test automation shouldn't be at just the GUI level, it should be a mix across unit, integration, and functional (GUI) tests too.
Are you working on a web app? If so, have a look at great open source tools like Watir or WebDriver. (I'll also pitch Telerik's Test Studio to you; however, for full disclosure I'm their evangelist for that tool.)
Desktop applications (or mobile) bring a lot of challenges to automation, and it's totally dependent on what platform you're working with. Test Studio supports WPF, but you can also look to other commercial and a few free tools. I don't know of any tools for Swing apps, but that lack of knowledge is due to me having been out of that domain for many years. (And maybe I'm so out of it that Swing's not even the normal Java GUI toolset...)
iOS and Android are tough ones to find reliable automation tools for. I know the Frank framework/API will work on iOS (Test Studio has a free recorder in the App Store), but I don't know of any other tools that reliably support the extraordinary matrix of Droid hardware and OS versions.
Regardless of your platform and toolset, you need to learn the basic approaches for dealing with GUI testing: focus on high value tests, learn to avoid duplication through approaches like Page Object Pattern, learn how to deal with synchronization/timing issues in your specific application.
It's a long haul, but if you work carefully it's totally worth it.
(And fun, too, IMO.)
I've been looking over Sproutcore, Ember and Blossom and other competitive framework efforts (e.g. Sencha) to select for a HTML5 client side application project. The state, information, and documentation from these projects is a bit fragmented and in need of clarity, so I am presenting this to the community.
My project is to be a native-like HTML5 application with desktop level complexity in need of a complete application framework, that will work well on desktops and run with good speed on mobile devices with touch awareness. The widgets should be native-like (not web-like), but customizable so to be unique to the application.
Questions/framework Requirement:
Native vs. Web style Applications. Framework should make it easy to
build native-like user experiences with the ability to make a custom
native feel (not just wholly imitating mac/win/iOS). Some of the text
surrounding Ember indicates it is really meant for web-style apps - which given no
UI layer maybe goes without saying. Frameworks like Sencha, can it easily accommodate custom widgets?
Mobile Appropreatness. Framework should be appropreate for mobile devices and have facilities for touch input and
gestures.Several notes I've seen in my research indicate that Sproutcore and
Blossom aren't very appropreate for mobile, and that Ember is better
geared towards mobile (size?). It isn't clear whether the
touch/mobile libraries are very developed in Sproutcore/Blossom and if they will be supported it the
current state going forward. (and blossom compile to native is not acceptable). On the otherhand, Frameworks like Sencha, do they have the facility to work well on desktop as well as mobile?
Framework Completeness. The framework should be a fairly complete application framework, with desktop-like OO expectations and management for automatically and efficiently syncing, managing, and serializing the data model with the server. Not sure if there is much difference between Ember and Sproutcore, how do other efforts like Sencha stack up?
Your question covers a lot of ground. I will pick some quotes and answer them directly.
My project is to be a native-like HTML5 application with desktop level
complexity in need of a complete application framework
Ember.js specifically bills itself as a "web-style" framework, not a an RIA framework. That said, you can build anything you want, but you would be trailblazing.
Sproutcore bills itself as an RIA framework. You have complete control over the DOM, so if you can do it in the browser, you can do it in Sproutcore.
Ext-Js is also a good application framework for desktops (Sencha Touch is for Mobile). If you like the way its examples look, then its a good choice. You can of course customize the dom and write your own widgets.
Blossom is basically Sproutcore with a canvas based view layer. It just went into beta, so you would definitely be trailblazing if you went with it.
So, you can basically use any of the frameworks you mentioned for the RIA part of your enterprise. I would eliminate Ember.js simply because the framework itself purposes itself for web-style (e.g. twitter) as opposed to RIA (e.g. iCloud) apps, which is not what you want.
The widgets should be native-like (not web-like), but customizable so
to be unique to the application.
All three of your remaining options can do this. If you like Senchas widgets, its a good choice. I don't know if they are native enough for you. That said, with any of the remaining frameworks you can customize the DOM to your heart's content.
Mobile Appropreatness. Framework should be appropreate for mobile devices
This is a tough one. Sencha Touch (which is separate but similar to Ext-Js) is very popular and gets the job done. It is performant too; a non-trivial app ran fine on my original Droid (which surprised me).
Sproutcore is very heavy weight. It has mobile support (i.e. for touch events) but you need to very careful about the dom you create, so as not to overwhelm the browser. I wouldn't choose Sproutcore for mobile, although you could if you are very careful.
and blossom compile to native is not acceptable
That does not seem reasonable to me. To be clear, NONE of these frameworks run natively on mobile devices; they ALL run in the browser. Blossom comes closes as the canvas API is mapped directly to the native API, giving you a truly native app. The only way you could get closer would be to use objective-c/java for iOs and Android.
So basically, at this point your left with Sencha(Ext-Js) and Blossom. Blossom is still in Beta, you would be trailblazing if you tried it. Sencha is established, has great support (Blossom support is good on irc), and a large developer base.
So Sencha is the choice, unless you really want to be cutting edge, and take a little risk.
Troy. Indeed, ember can run with another view layer framework such as jQuery Mobile which can provide a "app-like" look and feel.There is a github project: https://github.com/LuisSala/emberjs-jqm. In my view, if you need very cool animation you can use blossom.If you want to build a app, SC or ember should be OK. I'll choose ember because it 's loosely coupled.
I'm looking for a code coverage tool that I can use with a BlackBerry application. I'm using J2ME-Unit for Unit Testing and I want to see how much of my code is being covered by my tests.
I've tried using Cobertura for J2ME but after days of wrestling with it I failed to get any results from it. (I believe that the instrumentation is un-done by the RAPC compilation). And despite this message, the project seems to be dead.
I've looked at JInjector but the project seems very incomplete. There is little (if any) documentation and although it claims to be able to work with BlackBerry projects, I haven't seen any places where it has been used for that purpose. I've played with the project quite a bit but to no avail.
I've also tried the "Coverage" view in the BlackBerry JDE, even though I use Eclipse for development. The view stays permanently blank, regardless of clicking "Refresh" and running the application from the JDE.
I've looked at most of the tools on this SO thread, but they won't work with J2ME/BlackBerry projects.
Has anyone had any success with any code coverage tools on the BlackBerry? If so, what tools have you used? How have you used them?
If anyone has managed to get JInjector or Cobertura for J2ME to work with a BlackBerry project, what did you have to do to get it working?
I can't speak for Coberatura or JInjector, because I don't know how they collect test coverage probe data.
What is
critical is how this data is captured (does it need Java runtime support only available in standard Java VMs?) and how it is exported to the test coverage display/report generation tools.
Our SD Java Test Coverage tool instruments your source code; at runtime this produces an array of native Java booleans representing the coverage data, without need for any special VM support. Normally, this array is exported directly to a file, used by the test coverage display mechanism, by a TCVDump method provided with the test coverage tool, as your application exits.
Java (and other programming langauges used) in embedded systems often requires custom methods to extract the test coverage data. You might need to code a special dump procedure (in Java) to write out that boolean array to an accessible place. Our experience with building such custom dump procedures is that they are generally pretty simple (a few dozen lines); the real trick is deciding how/where to put the data, so that it can be easily moved to the target file. Mostly this is just a peculiar pair of copies, the first of which copies the boolean array to some staging location, and the second which writes the staged data into the destination file. (The standard TCVdump method is provided in source form to enable this kind of customization).
While I haven't specifically looked at BlackBerry, if you can write the data anywhere, you can pretty much be assured you can achieve this. We've had success with other embedded hand-set systems, such as Symbian, doing this.
If you want a complete overview of how to generally instrument code for test coverage following this strategy, see this paper: Branch Coverage for Arbitrary Languages Made Easy
I was actively involved with JInjector while working at Google. We were able to use it to successfully obtain code coverage for Blackberry applications. The application lifecycle for Balckberry apps is less predictable than J2ME and we found we had to tweak the application code to ensure the coverage data was gathered. I didn't personally work on the blackberry apps, several other engineers did. I'd hoped we'd create an example blackberry application and make it available on the jinjector site, but events and life got in the way.
If you would be willing to provide a sample blackberry apps with some unit tests, I'd be willing to spend a few hours trying to help you get the code coverage working. I'm not actively working with either J2ME or Blackberry (I'm currently working on Android apps when I have time to experiment with mobile) so I'm quite rusty. I have a day job that doesn't involve much mobile test automation, however I continue to work on ways to improve the test automation for mobile apps e.g. http://code.google.com/p/mwta/downloads/list for Android Test Automation.
I'm julianharty at gmail.com