Showing posts with label off line. Show all posts
Showing posts with label off line. Show all posts

Monday, July 02, 2012

#Kiwix - the interview

#Kiwix is the tool that allows you to read the content of a Wiki offline. It has been developed with Wikipedia in mind but is equally usable for Wikisource or Wikibooks. I am really happy to have interviewed Emmanuel who knows all the ins and outs of this wonderful piece of software.
Thanks,
     GerardM


What is Kiwix and what is it used for
Kiwix is a software that wants to enable people to read Web contents without internet connection. It's a reader which works with ZIM files containing all the content. It's used to access Wikipedia offline, by reading pre-packaged Wikipedia ZIM files. It's mainly used by people who want to have an encyclopedia, but are too poor to have access to the internet. It's also used,for example, by travelling people (plane, ship, train), prisoners and students at school. 

Can you tell us something about its popularity
We have users all over the world and the audience is increasing quickly: we have had 25.000 downloads of Kiwix in May. 

In how many languages is Kiwix supported 
Thank to the Translatewiki Web site andits community, Kiwix is localised in more than 80 languages. We also provide content (ZIM files), mainly Wikipedia, in around 25 languages. But we want to do more: thanks to a grant of WMCH soon we will offer ZIM files of Wikipedia in all languages. 

How do you support languages written in scripts like Malayalam, or Tibetan
Contents are Web contents and Kiwix itself is a sort of browser getting the Web pages from the ZIM file instead of the Web. So, we do not have special handling in Kiwix itself to render the contents. Everything should be well organised in the ZIM file, for example by using Web fonts. But, from the Kiwix fulltext search engine point of view, this is challenging. Natural languages have a lot of particularities. Kiwix uses the Xapian search engine and tries to integrate CLucene. We do our best with them to offer the smoothest user experience possible. 

Do you provide fonts with Kiwix for the languages that use these scripts
The Wikipedia ZIM files we are preparing still do not provide the Web fonts. Already for a few months, the integration of Web fonts has been a part of the Wikimedia projects, so we have to fix that ASAP, this is not a big challenge.

For some languages like Chinese and Serbian, we show the content in two scripts ... Can Kiwix do this as well ?
Kiwix does not provide any transliteration tool for now, but all the technology is already there in the soft. We use a powerful unicode library called ICU (http://site.icu-project.org/) which can do that. We
want to use it to allow users to do custom transliterations. C++ developer wanted there!

Kiwix uses the OpenZIM format ... can you tell us more about this format
The format is called ZIM. There is a volunteer driven project called openZIM  created a few years ago to specify the format and develop a standard library. The ZIM format allows to put millions of contents together, to compress a part of them, and add Metadata. In the end, you get only one file, which is, at the same time, extremely compressed and allows a constant and quick random access.

Nowadays, many publications are in the EPUB standard ... can Kiwix handle this as well
Kiwix is not able to deal with EPUB, but in the future it will. We think EPUB & ZIM format are complementary and we want Kiwix being able to perfectly deal with EPUB. Our plan is to integrate "Monocle" to do that. Also there, developers are wanted.

How do people find content available for Kiwix
Kiwix has its own content manager, so you can download content from Kiwix itself. But you may also download the ZIM file from the Kiwix Web site (http://www.kiwix.org) or using the Mediawiki Collection extension.


In the future, we want to have a platform (something like Itunes) to offer  really easy to find and download contents (both ZIM and EPUB files). We have started a project in that perspective. We need your support! 

What is your biggest challenge at this moment in time with Kiwix
Building Kiwix-mobile for Android. We will have a first release in autumn. But we have many other projects running at the same time and others for which we still need volunteers.

Thank you
Emmanuel

Tuesday, April 03, 2012

#Chennai hackathon IV

Several of the Chennai hackathon projects were interesting because of the different approach they took to known problems. A hackathon is a perfect place to experiment; there are many people around and they often have their own ideas about how things should be done.

Wikiquotes via SMS
You send the name of a person and in reply you get quotes from that person. The idea makes sense but the problem is that the quotes in Wikiquote are not available in a structured way. When quotes are individually tagged, an application to send SMS messages becomes possible from the source. The approach of this project was to copy quotes into a database and use this.

When the Wikiquote communities start using templates for each quote, it will become possible to use Wikiquote itself.

Translation of Gadgets/UserScripts to tawiki
There is no doubt, the translation or localisation of software makes a hell of a difference to the usability of software. The standard approach is to localise at translatewiki.net but gadgets and user scripts are not yet supported in this way.
It is wonderful to learn that the ProvIt and the TwoColumn gadget are now available in Tamil. The challenge left is how to make them available in other languages and how to maintain them when the code needs changing.

Lightweight offline Wiki reader
Even though there are several off line readers like OkaWix and Kiwix, it was the considered opinion that there is room for another one, a reader that can be used when there is only a little room available. The Qvido project was revived at this hackathon and is now available with "build" instructions.

Open source projects exist as long as there is an interest in developing a project and as long as there are people who benefit from it. Being able to host MediaWiki content off line as widely as possible is certainly reason enough for this project to get attention.

Program to help record pronunciations for words in tawikt
When you learn a foreign language it is really hard to get the pronunciation of words right. A program that allows the recording of some 500 words in half an hour is really useful. However, getting it uploaded to Commons is equally important because this can be an enormous time-sink and this is one part of the puzzle that still needs solving.


WikiPronouncer
Another approach to recording pronunciations was to do this on an Android phone. Developing an app in a day is too much of a challenge.. What these two projects make clear is how much pronunciations is a feature that is very much in demand.

As soundfiles are used extensively on other wiktionaries, it does make sense to learn the best practices elsewhere. The least they need is a Tamil makeover eh, a localisation.
Thanks,
       Gerard

Thursday, September 15, 2011

The roads of #India

One of the projects on the English #Wikipedia describes the roads of India. Given the size of India, the end result is something that can easily be called a book. As such it is not that strange when the book becomes a book that is available from PediaPress.


A book like this is not without its issues. How do you for instance make a NPOV map of India when the borders of  India are in dispute or what to do when the generation of the book creates errors.

Making maps of India that are acceptable to all is problematic. India has border disputes with several of its neighbours and as a consequence Indian maps show where India expects its borders to be. The same is true for its neighbours and consequently the maps Google presents are different from where in the world you view them.

The law is often used to ensure the "right" presentation of borders and therefore a book like this will never be commercially available in India.

Getting attention from the right people is (relatively) easy to do; you create a bug in bugzilla and / or you talk to people involved in the off line functionality on IRC and / or you scratch your own itch and dive into the code.
Thanks,
      GerardM


Monday, August 22, 2011

There are few messages to localise for #Kiwix

Kiwix can be localised at #translatewiki.net. and it started with 14 languages already localised. Two days later, 18 languages are completely localised and for 6 languages the localisation is underway.

With only 79 messages to localise, there is not much effort needed to localise Kiwix. Given that it is the client that does support the openZIM format, it helps support the Wikimedia Foundation in its effort to have a credible off-line strategy.

Check out the status, and help us to create the best off-line experience for your language.
Thanks,
       GerardM

Saturday, June 18, 2011

#Kiwix is preparing for #translatewiki.net

At the Berlin hackathon,we talked with the Kiwix developer and he indicated that making Kiwix ready for localisation at translatewiki.net is one of his priorities.

As it one of those jobs that just does not go away, it is at some stage easier to give it a whirl. The good news is that a first iteration has found its way to translatewiki.

It is now a matter of making sure that new messages and localisations can find its way to and from Kiwix before the translatewiki community can work its magic.

More news when we have it :)
Thanks,
     GerardM

Wednesday, April 06, 2011

#Okawix makes it a two horse race

Going from an announcement to committed code makes Okawix the second off line program that supports openZIM. At this moment it is only the Linux client that has its initial code, but Okawix has the benefit of localisations in many languages.

When the software is stable for all the platforms, it will be interesting to see if the Wikimedia Foundation will acknowledge Okawix and give it the same status of that other open source product that supports openZIM, Kiwix.


In the end, the pudding is in the eating and when all the other things are equal, the difference will be in the quality and number of the localisations. Okawix has the advantage that is localised at translatewiki.net while Kiwix is localised using a text editor. Given that for many languages the localisation is not done from English and that there are no utilities that make the life of a localiser easy, Okawix has the best cards.

To be fair, we want a two horse race and handicaps are at the moment evenly divided. The race would be more interesting without handicaps; the Kiwix localisations deserve proper tooling and Okawix deserves support from the WMF.
Thanks,
      GerardM

Wednesday, March 23, 2011

A #PDF library with potential

#Wikipedia content can be combined and exported in the PDF or Portable Document Format. This ability is extremely powerful as it allows people to produce speciality content as readers straight from a Wikipedia and share it either printed, by mail or by sneaker net.

While awesome, the current software is deeply flawed because it does not support all the scripts and languages of our projects. The excuse has been that the library that "everybody" is using is external to MediaWiki and the problem is in this library.

Has been, but not for much longer. Development is progressing nicely on another library. A library that will support other scripts. Vasudev Kamath's first package entered debian and it is the python-pypdflib. It is experimental and it may not yet support right to left scripts properly but with its support for the scripts from India and probably Cyrillic it forecasts a brighter day.

The project page can be found here and, here is your opportunity to test the existing functionality. 
Thanks,
       GerardM

Friday, March 11, 2011

Which languages offer the most potential for your website?

When resources are limited, this question is relevant. Then again, what resources are limited. When commercial translators wonder which languages offer the most potential, the resources are people translating.

Affluence is a factor as well as the size of the Internet population. When Inuktitut is rated over Bengali however, it makes you wonder what value such numbers hold. Surely the people in Bangladesh who have Internet make more then the average wage.

For Wikipedia its resources are not limited by the number of editors as they are by volunteers. With over 270 Wikipedias even another language is hardly noticed.

The limiting factor for the potential of many of our projects that is solvable is technology. In order to invest in such technology, the potential of the languages involved has to be considered. The direction of the answer for the Wikimedia Foundation is quite unique; the question is not what language but what technology to invest in. Many enabling technologies have applications for any script, any country even any language.

Technical issues do have a direct impact on the generated traffic; Malayalam Wikipedia provides a typical example. As we support the latest Unicode technology its traffic suffered. There are two solutions that will generate more traffic
  • web fonts so that everyone can read the text
  • providing search engines text expressed in different variations for a script
The best part of many language technologies is that they are generic. The technology is often applicable for multiple languages, scripts and even platforms and applications. As we invest in such things as off line readers, even Internet penetration is not that important when our off line support hits its mark.
Thanks,
      GerardM

Thursday, March 10, 2011

Test #offline #Wikipedia on your #Android or #iPhone

#Okawix is completing its functionality for the best selling smart phones. Now they need people to test it for them so that they can pick off all the bugs.

Testing and localising the Okawix software are the perfect preparation for the next phase of its development, the support for openZIM. The realisation of this next phase will surely make the Wikimedia Foundation really happy because it will finally provide them with an application that supports Wikipedia content in the openZIM format with a user interface in the same language.
Thanks,
      GerardM

Monday, February 28, 2011

#Okawix will support #openZIM

Okawix will support openZIM. #Wikipedia off line reading is done best when a localised interface is important to you. It is the only application that can be localised that will support openZIM.

With the support of openZIM, Okawix supports two formats for texts; their original one  Zeno and openZIM which is more efficient in compressing text. The installed base of Okawix includes smart phones like Android and iPhone. As Android does not yet support lzma2 compression, Okawix will move to the openZIM format when this support is available.


Okawix can be localised at translatewiki.net and with zeno it is a tool that not only displays the content of your wiki but also can also have a user interface in your language.
Thanks,
       GerardM

Wednesday, January 26, 2011

#openZIM is not supported here

The off line reader for #Wikipedia will be based on openZIM. Now openZIM is one of several formats that are actively maintained for use in off line readers. One of them has been specifically developed for use with Indic languages and others like Okawix and WikiReader are internationalised and are localised in many languages.

A choice has been made for a tool that has no obvious qualifications to support the over 270 languages supported by Wikipedia.

The good news is that at translatewiki.net we are quite happy to support competing applications. As we reuse existing localisations for new applications, openZIM may even benefit from the work that has been done on off line readers.

So Wikimedia Foundation, it is not obvious why you support openZIM. OpenZIM is not supported here but that can be remedied when you care for supporting all the languages of Wikipedia.
Thanks,
     GerardM

Thursday, July 15, 2010

The #Malayalam#Wikipedia success story continues

At #Wikimania in Gdansk, the news of a CD with some of the best content of the Malayalam Wikipedia was well received. It was not only an important exercise that taught valuable lessons, it was also distributed to 7.000 schools by the government of Kerala.

At Wikimania the developer of the software and Siebrand had a look at the code; together they discussed the optimisation and the internationalisation and localisation of the software. It would be the first software for translatewiki.net where the original software is not written for the English language..

In a new development it shows again the importance of open source and open content. As the CD project is both, its content has been included in a DVD with many Free Software tools and applications. This DVD is sent to 60.000 teachers in Kerala.

If you are interested, there is an online version of the CD and the ISO image can also be downloaded.
Thanks,
     GerardM

Wednesday, April 28, 2010

CD dowloaded 4390 times in 10 days

Talk about pent up demand, 500 #Wikipedia articles have been put on a CD and within 10 days, there have already been 4390 downloads. Guess what would happen if all of the Malayalam Wikipedia would be available in this way..

That would become possible when the Malayalam content would be available for the Wikireader. The Wikireader people are testing with content in other languages. This does include support for the Greek and the Cyrillic script.

It would be really great if the Wikireader is also able to support the Indic languages, it is a big market and Internet access is not everywhere a given.
Thanks,
      GerardM

Sunday, April 18, 2010

The best of the #Malayalam #Wikipedia

Promoting Wikipedia on the Indian subcontinent is important. All the Wikipedias can do with more readers, more editors and more attention.

The best 500 articles of the Malayalam Wikipedia have now been published on a CD. This project was presented at the Malayalam Wikipedia Meetup 2010 and it was announced on the blog of Santhosh Thottingal.

When you read the announcement, you will learn about the technical issues they faced including issues with Unicode. It is really nice that the software written in Python is available for use with other languages, my question is, is the software in English or does it need localisation..
Thanks,
      GerardM

Friday, March 12, 2010

WikiReader update

WikiReader, the gizmo that allows you to read Wikipedia off line, is being localised at translatewiki.net. It is doing really well with 17 languages completely localised and 8 more localisations under way.

On the WikiReader website, there is nothing yet about the product being available for other languages, for other Wikipedias. This had people of the translatewiki community asking the question: "What is the point.."

It turns out that as WikiReader is a physical product, it needs proper preparation and testing. It turns out that existing customers are using beta software for French, Spanish, German. They are working hard to launch multi-language support and they hope to roll it out by the end of the month.

At translatewiki.net we are ready with 17 languages. The question is how many languages will be ready at the end of the month ...
Thanks,
     GerardM

Friday, March 05, 2010

#WikiReader supported at #translatewiki.net

WikiReader is the latest open source application that finds its language support at translatewiki.net.

The blurp on the project page at translatewiki says:

The WikiReader (homepage) is a device that comes specifically designed to facilitate your focus in a world with constant interruptions. With three simple buttons and three million topics, WikiReader brings the iconic Wikipedia to all generations. The WikiReader software is available under the GPL.

There are 32 messages to translate, so if you think of buying a WikiReader, you can not only check out if there is a localisation, it will take you relatively little effort to help yourself.
Thanks,
GerardM