Tuesday, November 28, 2006

A new language on the Internet ?

I received a mail that was forwarded to me by Martin Benjamin (the Kamusi project). The mail was by a gentleman who wants to promote his mother tongue, the Bangubangu language. This is the first time that written material in this language has been created. The first project is a book in English, French and Swahili called “Teach yourself the Kibangubangou”.

I am thrilled with initiatives like this. The book is there now what to do next. Do you print it with an organisation like Lulu? Do you advise to make it a Wikibook? To what extend are concepts like copyright and licenses relevant and understood ..

It would be great to make this project succeed and have kids learn their mother tongue. There are stumbling blocks. Google does support less languages then the Wikimedia Foundation; Google does some hundred and The WMF some two hundred and fifty. One of the problems is that there is not much material in any but the bigger languages and Google does good by already doing this much.

To make an impact, I think it is crucial for material in particularly the endangered languages, to be tagged correctly. This gives Google and the other search engines a fighting chance to function on little material. For the Bangubangu language, there is no proper tag yet. The question is if the IETF will consider to do something about it. They are religious in their belief that the ISO-639-3 is not approved yet and that the bnx code therefore is not to be used. At issue is that there is a need that presents itself for this language.

Thanks,
GerardM

Sunday, November 26, 2006

Kanab Ambersnail

According to a heading on the English Wikipedia mainpage, the article about the Kanab Ambersnail was the 1.500.000th article. I think this is very spectacular. Personally I find it funny that I can write about it before I have read about it on slashdot :)

Congratulations to all who make Wikipedia so special.

Thanks,
GerardM

The I&I conference

Last Wednesday and Thursday I was at the annual I&I conference in Lunteren. This conference brings together many of the ICT (information & computer technology) coordinators of Dutch and also some Flemish schools. I have had the privilege to be for the second time. This year we had beautiful new Wikipedia folders, Marc Bergsma demonstrated a OLPC motherboard; it got a lot of interest particular when it was learned that a full system can be available for Dutch schools for the school year 2008/2009.

One great project I learned about is called TEEM. TEEM is a project where teachers evaluate educational websites and CD/DVD based resources for their use in classrooms. The way it is organised is such that I can understand why British teachers trust it as a resource. The way that it is funded however is one that creates a sad systemic bias. Reviews are paid for by publishers the consequence is that open content course ware will not be evaluated. This funding model also keeps out those publishers that do not pay for a review. This governmental political choice was to encourage innovation and competition. It is not unreasonable to suggest that it effectively costs the British schools more money as they do not learn about what is available for free.

Thanks,
GerardM

Friday, November 24, 2006

Whose language is it anyway ?

Much of Microsoft's software has been translated in the Mapuzugun or is it the Mapundugun language. They did this in consultation with the Chilean government.

The Mapuche people have gone to court because they disagree that Microsoft or the Chilean government had the right to do this. The language it is claimed is theirs, and the translation was done without consulting the Mapuche people.

Many people do not understand or know what is behind this. Why would people be opposed against becoming part of the digital world? To me, the key thing to appreciate in the reporting is the accusation of "violating their cultural and collective heritage". There are competing orthographies for this language and one is strongly favoured by the government.

So if I remember things well, this has everything to do about the involvement of the people that use the language. Mapundugun is spoken in both Argentina and Chile and here the government of a nation and the most powerful company of the world seem to be taken on because they do not represent the Mapuche people and are denied the right to decide for them.

In that light it makes perfect sense to go to court and insist that this is very much not wanted.

Thanks,
GerardM

Friday, November 17, 2006

To the winner go all the spoils

When changing you name makes you money, would you do it? Would you change your name for a pig or a goat ? Accepting such an offer could make me "Pig Meijssen", it would go well with my mascot. The name that the people changed their name to, "Hornsleth", is perfectly honourable. This "project" by the Danish artist Kristian Von Hornsleth is very much intended to demonstrate that international aid fails people. It fails people because it assumes that "the way of the donor" is best.

If you want to be helped you have to do this, that whatever. For me the great thing about Wikipedia is that it helps. It brings information to people. Its intention have always been to bring people information in their language. This means that culture, people and language are respected and that have people are enabled to help themselves.

For the western languages there is an abundance of great information. For many other languages, Wikipedia still has to take root. As the world becomes more wired, I expect Wikipedia will take root and become relevant for as a resource for both the culture, the people and the language.

When it is considered acceptable to bring information only in English, French, Arab, Chinese or whatever is considered a big language, the notion of Neutral Point of View that the Wikimedia Foundation offers is deminished. A NPOV exists because all points can be brought in the diversity that are the languages and cultures that are reflected in the 250 Wikipedias that currently exist.

It may be efficient to concentrate on the "important" languages.. but I do not want to consider the loss.

Thanks,
GerardM

Sunday, November 12, 2006

The worth of MediaWiki

According to the dark art of economics, everything can be valued. Everything can be given a price tag. People may object to this, and do on principle, but sorry, they have done just this for MediaWiki.

MediaWiki has a worth of $3.810.127,- when you assume that a developer costs $55.000,- a year and some other stuff. It was "valued" per the first of January 2006 and as the year is almost gone, it will be worth a lot more.

Another economic truism is that money makes money. Because of the success of MediaWiki more people will start developing MediaWiki. I am not in a position to deny this. WiktionaryZ extends MediaWiki. With new functionality much more becomes possible.

There is one thing missing in this argument. What is wrong is that MediaWiki is a tool. A tool that produces something that is far more valuable. Wikipedia is not the only project that MediaWiki enabled. I am sure some people who understand this dark art of valuation will be able to come up with a better number.

Given that money makes money, it is possible to leverage the MediaWiki generated content. Much proprietary content is not really relevant because it does not get exposure. By making content available it can get exposure. Material was often created to get exposure. By keeping it proprietary, thereby hidden from view, it does not do all that it could do. By making it available under a free license new opportunities arise.

MediaWiki enabled among others Wikipedia. WiktionaryZ has potential. My hunch is that like MediaWiki it will enable the creation of content in a different way. I hope and expect that it will help us to negotiate the release of much content under a Free/Open license and allow us to collaborate with many organisations and people.

Thanks,
GerardM

Friday, November 10, 2006

Running the interwiki bot for Wiktionary

I run the interwiki functionality of pywikipedia bot on all the Wiktionaries. It is a thing that I started and it is the kind of public service that needs doing. It links all the words that are spelled the same by adding "interwiki" links. These are the things that you see at the left hand side where it is indicated that there is also information in another language.

I have done this now for over a year and what I just noticed is the amount of words that I do not understand is growing rapidly. On the one hand it is to be expected as it is in line with the rapid growth of projects like the Vietnamese wiktionary. What now starts to happen more is that multiple wiktionaries have words together. That is what I see when I watch the bot.

In a way it would be fun to have WiktionaryZ in there. Currently we have 159.004 Expressions and we have 10.557 DefinedMeanings. Based on the expressions we would be the fourth project in size, it would be more reasonable to use the DefinedMeaning for the comparison and this would have us as the 26th in size.

Comparing Wiktionary with WiktionaryZ is like apples and oranges. Where Wiktionary has each word only once, WiktionaryZ counts them as existing in a language. Where there can be many red-linked articles on a Wiktionary page, the WiktionaryZ expressions are implicitly there.

It makes better sense to appreciate what the implications are of the numbers. In lexicology size counts. Only when people have a good chance of finding the information they are looking for will they find a resource useful. It is one reason why it makes sense to concentrate on certain topics or domains. WiktionaryZ is rich in ecological terminology due to the information that we got by including the GEMET thesaurus. By working on the OLPC children's dictionary we get a lot of the basic stuff that is the bread and butter of dictionaries.

Tuesday, November 07, 2006

Some thoughts on Alexa

Alexa is a website that provides an indication of the popularity of websites on the Internet. What I do does not matter, as Alexa only measures the use of Internet Explorer which is statistically becoming a less brilliant idea. Given the amount of people using Firefox on WiktionaryZ, I am sure that they do not know where many of the alpha crowd hangs out.

We have had our downtime this week and, I have been looking at how this affects our standing at Alexa. Sure enough, after two days of downtime, we hit the 900.000th place. Now that we are back up, we rebound nicely and today we are already back at number 478,316 for the weekly average.

WiktionaryZ has its own statistics, here you will find that our daily average hits did take a pounding. Given no more downtime and given that the trend of continued interest continues, the numbers will improve but the average will be depressed. At this moment all this is not crucial. When we get people to rely on WiktionaryZ, our service level needs to be much improved.

In a conversation with a developer I said once, when you respect our users, you have to treat them as if they cost us $150,- an hour. The point is that with the realisation how valuable contributors are, you are more likely to give them with the respect that they are due. With professional people using wikis, there actually is money paid for the time spend editing wikis this makes it more plain but it does not make a difference. Editors are to be respected and it is important to make the most out of what they do for us.

Thanks,
GerardM

Wednesday, November 01, 2006

The semantic web to the rescue ?

The reporting on the Internet Governance Forum in Athens is mighty interesting. Yet another nice article on the BBC website with some thought provoking ideas.

I find it really interesting that spoken languages are considered. However, practically at this stage the Internet is very much oriented towards written languages and, given the amount of stumbling blocks that exist to integrate languages other than the ones using a Latin script, I am afraid this is just a red herring.

I was also amused to see that the semantic web was brought to the fore as one solution to the problem of linguistic diversity. Yes, it is intended to be understood by computers. Computers are used by people and the semantic web is decidedly English. This raises the question how this computer that apparently understands English communicates to its user who does not.

There are however some great things to be said about the semantic web; first of all the terms used should be unambiguous. This in turn means that it should be possible to translate it to other languages than English. This is a challenge that we face in WiktionaryZ. We are able to have semantic relations and our semantic relations do translate to the language of the User Interface. So when the terms have been translated, in WiktionaryZ relations can be understood not only but also by computers.

This is a good moment for a disclaimer; WiktionaryZ is pre-alpha software. Many of the issues that have been tackled in the development of the semantic web we have not considered let alone touched. We hope / expect that we will be allowed to stand on "the shoulders of giants".

Thanks,
GerardM

Is this a "good" word

MediaWiki is the software that drives Wikipedia but also WiktionaryZ. It is probably one of the best pieces of software when it comes to internationalisation and localisation. This is demonstrated by the many localisations that have been done already for the software. Singing the praises of MediaWiki from me can be expected; why else develop WiktionaryZ on top of MediaWiki?

This does however not mean that all is well. I have written before about the problems with the Neapolitan language and there issue with the '' combination. Today I learned that a language called Hai||om uses the "pipe" character and consequently, I cannot make it work properly in a MediaWiki installation.

There are ways around such a problem; I can use one of the alternate names; San or Saan. I can expect that there will be no Wikipedia created in this language (only 16.000 speakers). But the point is, that even a system that does really well is only as good as the next language that proves that it has an issue with it's presumptions.

Thanks,
GerardM

Tuesday, October 31, 2006

The importance of good standards

At the Internet Governance Forum Mr Vint Cerf has said that changing the way the Internet works to accommodate a multi-lingual Internet raises concerns. The question raised in the BBC article is whether it is a technical issue or not.

Interoperability on the Internet is possible when the standards used are such that interoperability is possible. The current URL system is based on the Latin script. This was a sensible choice in the days when computing was developed in America. It made sense in a world when the only script supported by all computers was the Latin script. In these days computers all support UTF-8. All modern computers can support any script out of the box. This means that all computers are inherently able to display all characters. This does however not mean that all computers are able to display all scripts; even my computer does not support all scripts and I have spend considerable time adding all kinds of scripts to my operating system.

The next issue is for content on the Internet to be properly indicated as to what language they are. Here there is a big technical issue. The issue is that the standards only acknowledge the existence of a subset of languages. The result of this is that it is not possible to indicate using the existing standards what language any text is in.

Yes, the net will fragment in parts what will be "seen" by some and not "seen" by others. This is however not necessarily because of technical restrictions but much more because the people involved and the services involved do not support what is in the other script, the other language. When I for instance ask Google to find лошадь or paard, I get completely different results even though I am asking both times for information about the Equus caballus. In essence this split of the Internet already exists. The question seems to me to be much more about how to make a system that is interoperable.

The Internet is interoperable because of the standards that underlay it. With the emancipation of the Internet users outside of it's original area, these standards have to become usable for both the users of the Latin, Cyrillic, Arabic, Han and other scripts. It seems to me that at the core of this technical problem is the fact that the current standards are completely Latin oriented and also truly focused on what used to be good. At this moment the codes that are used are considered to be human readable. I would argue that this is increasingly not the case as many of these codes are only there for computers to use. When this becomes accepted fact, it will be less relevant what these codes look like because their relevance will be in them being unambiguous.

For those who have read this blog before, it will be no surprise that the current lack of support of ISO-639-3 for language names is one of my hobby horses. As I have covered this subject before I will not do this again. What I do want to point out that insisting on "backwards compatibility" is more likely to break the current mould of what is the Internet than preserve it.

Thanks,
GerardM

Sunday, October 29, 2006

New functionality of Firefox and blogging

I use Firefox as my browser. I have upgraded it to the latest version and now, my English will be spell checked for me in a real time fashion. The thing I probably will like best is, that a word like "localising" is now seen as correctly spelled. This is because Firefox allows me my British English. Blogger, although nice expects people to use American English spelling. This is useless when I were to blog in any other language.

Another thing that is nice is, that it allows me to accept words that are correct in the texts that I write; WiktionaryZ is such a word .. and so are MediaWiki, Wikimedia and Wikipedia.

By having spell checking done client side, the server functionality becomes cheaper for the service provider as well. The quality goes up.. all in all a good reason to upgrade to the latest Firefox.

Thanks,
GerardM

Monday, October 23, 2006

The NPOV of language names

Yesterday Sannab pointed me to this posting on the linguistlist. The gist was that there had not been full consultation with the academic community about the adoption of the Ethnologue database for the ISO-639-3 codes of languages. A secondary argument was that Ethnologue is primarily a religious organization and the question was raised if it could be ethically to have such an organization be the guardian of what is to be considered a language.

This e-mail is a reaction to what Dr. Hein van der Voort wrote in the SSILA-Bulletin number 242 of August 22 of 2006.

The problem I see with the stance taken is not so much in the realization that some of the Ethnologue information needs to be curated, it is also not in the fact that some would consider Ethnologue to be the wrong organization to play this part, the problem is that no viable solution is offered. The need for the ISO-639-3 list is not only to identify what languages there from a linguistic point of view, it very much addresses the urgent need to identify text on the Internet as being in a specific language.

At WiktionaryZ we are creating language portals. These language portals are linked into country portals, both countries and languages have ISO-codes. When I had questions about language names, Ethnologue was really interested in learning what I had to say about what are to me obscure languages. The point here is, Ethnologue wants to cooperate. Some people do not want to cooperate for ideological reasons and at the same time do not provide a viable alternative. This is from my point of view really horrible. The need for codes that are more or less usable is expanding with Internet time and not with the glacial time that is the time of academics.

When WiktionaryZ proves itself and becomes a relevant resource for an increasing number of languages, all kinds of services will be build on the basis of it using standardized identification for content. The ISO-639-2 code is inadequate. It is not realistic to expect ISO to review it's decision at this stage and not include Ethnologue. It is not realistic to expect such a review without providing an alternative that is clearly superior to what is ISO-639-3. It is clearly better to improve together on what is arguably in need of improvement than not to provide the tools to work with in the first place.

PLEASE COLLABORATE ..

Thanks,
GerardM

Wednesday, October 18, 2006

Learning a language .. because you must

When you want to go live in another country, to live there permanently, it is best to know the country, the language. When you want to emigrate to the Netherlands, it is often required that you first pass a test that shows that you have some basic ability speaking Dutch. This test is to be passed while still abroad. This test can be taken at a Dutch embassy; both for the embassy and for the people who have to take this test, it is a logistical challenge.

The test is to find if the "A1-" level of comprehension exists. People need to be able to listen to someone who speaks Dutch SLLOOWWWLY and uses a limited range of words. This list is finite. It is likely that many of these words are the same words that WiktionaryZ needs for it's OLPC project.

Many of the techniques that you would use in a school are the same as the ones needed to prepare for this exam. You need soundfiles, you may want illustrations; both pictures and clips, you want definitions in the many languages that people understand.

Given a list of words expected to be known for the "A1-" exam, it would be easy to get the communities of the would be emigrant to add translations for both the word and the definition. As there is a group of people that do not read or write for both, a soundfile needs to be produced as well.

The next thing is making this content available on the Internet and serve it as a public service. Maybe there would even be public money to make this a public service. Then again, even if there is no money available for it. It is a nice challenge and, when you can do this.. There are other countries that people emigrate to, even people from the Netherlands :)

Thanks,
GerardM

Monday, October 16, 2006

How to integrate wordlists in WiktionaryZ

There are many GREAT resources on the Internet. One I (re)discovered the other day is http://dicts.info. It provides information for some 79 languages and the information they provide is Freely licensed; the data can be downloaded for personal use as it can change rapidly.

So how do we integrate such information in WiktionaryZ ? WiktionaryZ insists on the concept of the DefinedMeaning. As this is central to how WiktionaryZ works, it is crucial that we have the concept defined. The dicts.info is split into two parts; a from part and a to part. The translations include synonyms and alternate spellings.

An application that is to include these translations could work like this: When an Expression is found in the to language and the translation is not there already, a user is shown the WiktionaryZ content with the suggestion to add the translation. This way the new information is integrated into WiktionaryZ.

The one part of this "how to" needed is possibly some discussion on the finer details, but certainly someone who will take up this challenge and develop this for us.

Thanks,
GerardM

Saturday, October 14, 2006

Eating your own dogfood

WiktionaryZ is about lexicology, terminology and ontology. Consequently you would expect that these concepts are in there .. Now they are.

For me not understanding a word and looking up what it means is a moral obligation to include it in WiktionaryZ .. the latest word I did not know was divot. It was used on the BBC news website and yes, you would appreciate what it would be like. The word in Dutch still escapes me :)

Thanks,
GerardM

Wednesday, October 11, 2006

Medical terminology

Yesterday I read an article on the BBC news website claiming that the term schizophrenia is invalid. The article points out that schizophrenia is not a single but a multitude of syndromes. The problem with the word is that given that it is understood to be a single syndrome, many patients are treated in a "one cure fits all" fashion. This is tragic as schizophrenia is seen as something that cannot be treated which is not necessarily true.

As we do not have much medical data yet, I have added the word to WiktionaryZ. WiktionaryZ will soon include an important resource of medical data, the UMLS. This will certainly include words like schizophrenia. I have added the resource that I created the definition on. With such a large body of medical terminology, it can be expected that many people will find their way to WiktionaryZ to learn what this medical terminology is. These people in turn will be interested in creating definitions that are consistent with what the scientific field considers something to be.

The word schizophrenia is well entrenched, it has a specific meaning to the laymen and my understanding from the BBC article is, that many professionals also need to learn about the ambiguity of the term. The question is, when is there enough support to depreciate the old meaning from a well known word like schizophrenia.. How will peer review work out in a wiki and how will the "public" react to these definitions..

Thanks,
GerardM

Tuesday, October 10, 2006

Localization

Yesterday I met some people at the Universiteit of Amsterdam the topic was there use of a content management tool called Sakai. This tool is to be used for a collaboration and learning environment for education. It is a tool that is used very much in universities.

At the UvA they want to use it as a shared environment for Dutch and Iranian students. This is in and of itself a splendid idea. The software is open source so much can be done with the software. I recommended that they should localize the software into Persian in order to provide a friendly environment. For me one of the bigger challenges was that Sakai provides a wicks environment but with a high level of authorization of what people can and cannot see. The challenge is how to make such an environment get to its tipping points where its community takes off and becomes autonomous in its action. Carving it up makes it much more problematic I expect.

The people who wrote the software however were braindead when it came to the use of their software in other languages. I learned from the University of Bamberg that they will not use it because localization is done by changing texts that can be found in the source code. A university in Spain I was told decided not to upgrade the software because it was not feasible to localize the software again..

It is sad when a tool with such promise is dead in the water because it was not considered that in order to be useful you have to allow for proper localization.

Thanks,
GerardM

Monday, October 09, 2006

Bi-lingual content

We are working hard to help the OLPC with content. WiktionaryZ seems to be really user oriented. WiktionaryZ wants to work together with organizations, with professionals. Quite often they have rich resources that are extremely useful, but need integration. Often this information comes in the format of a spreadsheet. When these are two column affairs with words in one language and words in another, it is not known what meaning these words have and, yes they can be imported but only people who know both languages can integrate it. At this moment it is manual work.

Manual work is sometimes necessary but it is time consuming and as it is, we do not have the tools to make it less time consuming. When such lists are imported with a "flag", it would be possible to locate them easily and compare them to existing Expressions for that language. This would help integration. When the Expression does not exist for both linked Expressions, it does not follow that the DefinedMeaning does not exist.

I am sure that we will have to deal often with these situations. We are at the stage where it makes sense to think about this. We are approaching the point where we have to deal with this.

Thanks,
GerardM

Friday, October 06, 2006

Using Microsoft as an initial standard

I do confess that I use Microsoft operating system and software. I will not actively buy new Microsoft software and I will not buy any new MS software if I can help it. Having said all that, it is clear that Microsoft's monopoly does not allow me to buy a laptop without paying the Microsoft tax.

I was using Word on my computer and I wanted to change the language as I learned not to write American but British English. Changing the language for a text I found a rich resource of languages that Microsoft supports. Of particular interest for me was how it splits English in many versions. This is something that I can easily emulate in WiktionaryZ. The question is; I can but should I. Would it be better to wait until someone wants to include something that is for instance Jamaican English or is it better to be proactive ?

Thanks,
GerardM