Doing L10N Testing, Asking If It's I18N Ready?

Still, some vendors are not aware of the localization industry, or maybe some vendors' employees are not actually familiar with that, but doing the tasks.

I saw some project team members including the PM and the testers are not aware what they are testing. They thought they are doing software L10N testing, but are they? And are they sure the software is I18N ready?

If you are doing L10N testing, please ask yourself first, do you think the software is I18N ready? Are you really very sure what you are asked to perform the testing is part of L10N testing?

Are you spending too much time on checking the functionalities are working fine or not in localized version? And do you discover many functional defects which are reproducible in more than one localized version or even the English version?

Testing all the possibilities?

4/26/2006

Have you ever planned to try to test all the possibilities and finally you surrendered?

When you are in high school, there is a lesson in mathematics class to teach you how to count the probability.
The lesson started with something like you need to pick up a red ball inside a black box.
The first thing you need to know is how many balls there, then you need to know how many red balls.
If there are 10 balls and only one red ball, then you will get the result that the probability to pick up a red ball is 1/10.

The lesson will then teach you how to count something more complex.
A similar example is to count the probability that you pick up a red ball and a blue ball from the black box.
If there is totally 10 balls inside the box, only 1 red ball and 1 blue ball. Then the probability is 1/10 * 1/9, which is 1/90.

In the real world, the reason it's so hard to list all the possibilities is due to we may not be able to find all the variables. In mathematics, you can, but in the real world, you can not.

A Sample Test Plan Structure

Below is a sample Test Plan structure which I created before for some project.

1. Background
2. Purpose of this Document
3. Reference
4. Features to be Tested
5. Features not to be Tested
6. Test Approach
7. Test Item Configurations
7.1 Rack Configuration
7.2 Platform Configurations for Each Testing Item
7.3 Environment Locale to be Tested
8. Resources
8.1 Human Resources
8.2 Hardware Resources
8.3 Software Resources
9. Test Cycle and Schedule
10. Risks
11. Assumptions
12. Test Entrance/Exit Criteria
12.1 Entrance Criteria
12.2 Exit Criteria
13. Defects Reporting
14. Contacts

What kinds of testing should be considered?

Type of Testing	Description
Black box testing	Not based on any knowledge of internal design or code. Tests are based on requirements and functionality.
White box testing	Based on knowledge of the internal logic of an application's code. Tests are based on coverage of code statements, branches, paths, conditions.
Unit testing	The most 'micro' scale of testing; to test particular functions or code modules. Typically done by the programmer and not by testers, as it requires detailed knowledge of the internal program design and code. Not always easily done unless the application has a well-designed architecture with tight code; may require developing test driver modules or test harnesses.
Incremental integration testing	Continuous testing of an application as new functionality is added; requires that various aspects of an application's functionality be independent enough to work separately before all parts of the program are completed, or that test drivers be developed as needed; done by programmers or by testers.
Integration testing	Testing of combined parts of an application to determine if they function together correctly. The 'parts' can be code modules, individual applications, client and server applications on a network, etc. This type of testing is especially relevant to client/server and distributed systems.
Functional testing	Black-box type testing geared to functional requirements of an application; this type of testing should be done by testers. This doesn't mean that the programmers shouldn't check that their code works before releasing it (which of course applies to any stage of testing.)
System testing	Black-box type testing that is based on overall requirements specifications; covers all combined parts of a system.
End-to-end testing	similar to system testing; the 'macro' end of the test scale; involves testing of a complete application environment in a situation that mimics real-world use, such as interacting with a database, using network communications, or interacting with other hardware, applications, or systems if appropriate.
Sanity testing or smoke testing	Typically an initial testing effort to determine if a new software version is performing well enough to accept it for a major testing effort. For example, if the new software is crashing systems every 5 minutes, bogging down systems to a crawl, or corrupting databases, the software may not be in a 'sane' enough condition to warrant further testing in its current state. Smoke tests get their name from the electronics industry. The circuits are laid out on a bread board and power is applied. If anything starts smoking, there is a problem. In the software industry, smoke testing is a shallow and wide approach to the application. You test all areas of the application without getting too deep. This is also known as a Build Verification test or BVT.
Regression testing	Re-testing after fixes or modifications of the software or its environment. It can be difficult to determine how much re-testing is needed, especially near the end of the development cycle. Automated testing tools can be especially useful for this type of testing.
Acceptance testing	Final testing based on specifications of the end-user or customer, or based on use by end-users/customers over some limited period of time.
Load testing	Testing an application under heavy loads, such as testing of a web site under a range of loads to determine at what point the system's response time degrades or fails.
Stress testing	Term often used interchangeably with 'load' and 'performance' testing. Also used to describe such tests as system functional testing while under unusually heavy loads, heavy repetition of certain actions or inputs, input of large numerical values, large complex queries to a database system, etc.
Performance testing	Term often used interchangeably with 'stress' and 'load' testing. Ideally 'performance' testing (and any other 'type' of testing) is defined in requirements documentation or QA or Test Plans.
Usability testing	Testing for 'user-friendliness'. Clearly this is subjective, and will depend on the targeted end-user or customer. User interviews, surveys, video recording of user sessions, and other techniques can be used. Programmers and testers are usually not appropriate as usability testers.
Install/uninstall testing	Testing of full, partial, or upgrade install/uninstall processes.
Recovery testing	Testing how well a system recovers from crashes, hardware failures, or other catastrophic problems.
Failover testing	typically used interchangeably with 'recovery testing'
Security testing	Testing how well the system protects against unauthorized internal or external access, willful damage, etc; may require sophisticated testing techniques.
Compatibility testing	Testing how well software performs in a particular hardware/software/operating system/network/etc. environment.
Exploratory testing	Often taken to mean a creative, informal software test that is not based on formal test plans or test cases; testers may be learning the software as they test it.
Ad-hoc testing	Similar to exploratory testing, but often taken to mean that the testers have significant understanding of the software before testing it.
Context-driven testing	Testing driven by an understanding of the environment, culture, and intended use of software. For example, the testing approach for life-critical medical equipment software would be completely different than that for a low-cost computer game.
User acceptance testing	Determining if software is satisfactory to an end-user or customer.
Comparison testing	Comparing software weaknesses and strengths to competing products.
Alpha testing	Testing of an application when development is nearing completion; minor design changes may still be made as a result of such testing. Typically done by end-users or others, not by programmers or testers.
Beta testing	Testing when development and testing are essentially completed and final bugs and problems need to be found before final release. Typically done by end-users or others, not by programmers or testers.
Mutation testing	a method for determining if a set of test data or test cases is useful, by deliberately introducing various code changes ('bugs') and retesting with the original test data/cases to determine if the 'bugs' are detected. Proper implementation requires large computational resources.