#multilingweb - #i18N and #l10n testing framework

A subject for the conference about the Multilingual Web is what gaps exist in supporting a multilingual web. Obviously people who are living the multilingual web like my colleagues in the WMF localisation team suffer these gaps. Our team has been asked what to do next and one idea of Amir I love to put forward.

A testing framework for localization
I searched and I couldn't find any testing framework that is focused on localization. Many localization-specific issues must be tested, for example, grammatical correctness of generated messages, text readability in different scripts, support for encodings and fonts, etc. You can find a fuller list in Wikipedia (i wrote most of that section myself). It is possible to test all these things using the current frameworks, but much of it would be manual.

For example, i'm not familiar with any tool that would automatically or semi-automatically create screenshots of all the possible translated strings with their complete context. This would be useful for the translator, to see how to translate a message; for the developer, to see whether any message runs out of the screen or hides a button that must remain visible; and for the tester who speaks the language and wants to see whether all the generated messages are grammatically correct. Currently, a developer must do this manually;
it is time-consuming, inefficient, hard to plan and to maintain, and the fact is that the developers are hardly ever doing this.

Such a testing framework would be really great for so many organisations. At translatewiki.net we support many organisations with their internationalisation and localisation. A framework will make it easy to repeat the testing often. This will improve consistency and quality and makes for a great multi lingual experience.
Thanks,
     GerardM