Missing award recipients in both Wikidata and the Wikipedias

Professor Fei-Fei Li is one of the recipients of the 2025 Queen Elizabeth Prize for Engineering. It says so on the English Wikipedia and it is confirmed on the website of the prize.

There are nine Wikipedias with an article for the award and there is Wikidata. When the 2025 awardees are known on a Wikipedia, "2025" should be available in the text of the article. Otherwise the article is likely out of date. The recipients should be known on Wikidata AND there should be an "award received" for the award with a date of 2025.

When you check Wikidata for this award using "Reasonator", you will find that Wikidata is in need of an update. It is by accident that I learned of this award. Updates are an hit or miss affair, this would be improved when a bot produces a list of all the awards that are in need of updates. When a bot produces this list for every Wikipedia for all the known awards, it enables people to do this maintenance work. 

Obviously 2025 is this year and it will have the most mutations. A similar job can be run for other years but it is less likely to bring many additions, more likely these list will become reduced in size over time.

Thanks,

      GerardM

#Wikidata user stories - the sum of all #knowledge

Map showing all places English Wikipedia covers

Map showing all places GeoNames covers

They say "a picture paints a thousand words". There is no argument; English Wikipedia covers only so much. With such a lack of coverage it is impossible to understand what is missing and its relevance particularly to people who do not read English.

LSJbot has created lots of articles for the places GeoNames knows about in several Wikipedias. As a consequence through the backdoor much of the missing information enters Wikidata. There have been some rumblings among Wikidatans that the GeoNames data is not perfect.. But hey, let's make "Be bold", a Wikipedia quality a Wikidata quality as well.

For many Wikipedians, the notion of bot generated articles is an anathema. For others the fact that there is so much that we do not cover is as problematic. The good news is that more information in Wikidata will enable us to predict what is lacking in content. We only need to acknowledge that Wikipedia is not the sum of all knowledge.. yet.
Thanks,
      GerardM

#Wikidata - #statistics

This chart is really interesting; it was made from the last dump (2015-03-30). Item numbers are subdivided into intervals of 100,000. So for instance (X,Y) = (192,73720) represents the interval [Q19200000, Q19299999] having 73,720 out of 100,000 items without a single claim.

It is interesting because it indicates to me that items are imported in batches. As the items in those batches are similar, it follows that even though many statements are added all the time, these items do not get similar attention.

Reasonator has a "Random item" feature and I used it to get a feeling what items do not have any statements. It seems to me that they are mostly items with a sitelink to "another" language and often they are about subjects that are in a relatively small class.

The items with a sitelink to "another" language are probably the most problematic. It is realistic to expect that many of them could be linked to another item when it was clear what it is the item is about. For that you need people who know the language and/or you need a way to figure things out using tools like Kian.
Thanks,
         GerardM

#Kian - Adding humans known to #German #Wikipedia

The first feat of Kian was to recognise humans based on articles from the German Wikipedia. Now that is the holy grail because once you know that an item is human, it is trivially easy to make all kinds of assumptions.

It was done by learning from what is already there. Given that for 90% of all Wikidata items it is already known what they are, there is plenty of learning material for Kian.

Once everything was said and done, 3600 articles are now known to be about humans. Given that much of the learning is based on Wikidata data itself, it follows that languages like Malayalam or Chinese can be tackled in the future. The future is bright and we happily welcome the first 3600 humans Kian recognised.
Thanks,
       GerardM

#Kian - the future is bright

Send someone bright to a course on artificial intelligence, someone with a pedigree of great bot-work and you get "Kian". In Farsi it means "glory" and,yes I have seen the light.

Amir introduced Kian in an e-mail: "Kian is a three-layered neural network with flexible number of inputs and outputs. So if we can parametrize a job, we can teach him easily and get the job done." If I understand properly what this means, I will stop doing most of the work I have been doing on Wikidata with AutoList. I will not reach three million edits and I will be ever so happy.

Amir's example for a first job: Add P31:5 (human) to items of Wikidata based on categories of articles in Wikipedia. The only thing we need to is get list of items with P31:5 and list of items of not-humans (P31 exists but not 5 in it). then get list of category links in any wiki we want[2] and at last we feed these files to Kian and let him learn. Afterwards if we give Kian other articles and their categories, he classifies them as human, not human, or failed to determine. As test I gave him categories of ckb wiki (a small wiki) and worked pretty well and now I'm creating the training set from German Wikipedia and the next step will be English Wikipedia. Number of P31:5 will drastically increase this week.

I am ever so happy because once we know what an item is, it becomes easy to make all kinds of inferences.
Thanks,
      GerardM

The #Italian job; a perspective on #bias

The Italian #Wikipedia is really into registering death and place of death. This is why an initial bot-run by Amir resulted in more deaths in Italy then elsewhere. This bias was visualised by Vizidata, I blogged about it in the past.

Today Amir finished another run of his bot on the Italian Wikipedia and he increased the bias with over 100,000 edits.

The point is very much that Wikidata has a bias and it is based on the quality of the Wikipedia data depended on. Running a bot repeatedly will increase quality and by inference bias. People die in Italy, apparently not so much in the Netherlands.

Now who says this Italian bias is bad?
Thanks,
      GerardM

#Wikipedia - To bot or not to bot II

In a mail it was announced that the Lsjbot finished adding plants to the Swedish Wikipedia. Several classes of subjects have so far been added and it had quite a surprising effect on the vitality of the Swedish Wikipedia:

"We are also gladdened by the hard numbers. Reader accesses show a healthy increase even from our already high number. And a trend of a slight decrease of editors has now turned into an increase. We can not say for certain why and it could be temporary but we believe the botgenerated articles has a part of this positive development".

These hard numbers referred to fly in the face of all the pundits who claim the opposite. Evidently, Wikipedia works best when it does what it is meant to do; share in the sum of all knowledge. There is no sharing when no information at all is provided for "esthetical" or whatever reasons.

We can argue about the best way of providing additional information and, it is good that a door is kept open for Wikidata to play a role. In the end both for all the Wikipedias and for Wikidata it is about priorities and I agree with Anders that the quality of the data is key in this. In addition the priority of Wikidata should be much more centred on what we do it all for; making information available to people not so much machines.
Thanks,
       GerardM

#Wikipedia - Municipio de #Prescott (#Dakota del Norte)

Prescott is a township in Renville county, North Dakota. It has 28 inhabitants and such is its notability that only the Spanish and the Vietnamese Wikipedia have an article. Pictures can be found thanks to Google maps..

What is really funny is that all the localities of Renville county can be found in the Spanish category. It is not hard at all to add all of them to Wikidata. It is funny to see all those Spanish names when Reasonator presents information about Renville county.

A message to the researchers of Wikipedia; Rambot did not add all the places of the Unites States. Wikidata is actually more inclusive than the English Wikipedia. Who knew ..
Thanks,
       GerardM

#Pywikibot - wants you to peek under the hood

For most #Wikimedia projects, Pywikibot has proven itself to be a trusted hard working tool. Literally millions and millions of edits were only possible because of the people operating the bot.

Before Wikidata, the interlanguage links were maintained by what was called "pywikipedia" bot. Now Wikipedia has been harvested for information with this bot to enrich Wikidata.

As time goes by, the architecture of MediaWiki has changed a lot. Consequently Pywikibot had to evolve as well. Its architecture changed as a result; it aims to use the latest API and other MediaWiki functionality.

From July 24th, 2014 and ending on Sunday, July 27th there will be a big online event to learn what more needs to be done for Pywikibot. What bugs need an urgent fix, what features are missing or incomplete. Obviously, it is a also time to look at the code and look for "bit rot".

It would be awesome when more people who care for the Wikimedia projects and know their Python get involved and ensure that Pywikibot will remain the meanest and most powerful bot platform around.
Thanks,
    GerardM

#Wikipedia - an alternative to #bots

What bots do is generate text from the data that is provided to them using an algorithm. When you have a lot of data, it will create a lot of text or a lot of articles. When data is added at a later date, the text of a bot generated article will not change. Typically an article is served from a cache.

As bot generated articles are a bad idea when they are in a fixed format, why should we have them? That is, why not generate them when they are requested and serve them from a cache, just like any other article.

Typically, the data of a bot generated article can be provided by Wikidata and Commons. Scripts that generate articles already exist. What is needed is to import the data in Wikidata and generate the text when requested. The number of articles will go down and this is actually a good thing. It means that the articles left are the ones that humans have edited, are involved in.

So what we need are some people with language skills to create the scripts for all the languages we support. We need the WMF engineers to consider articles on demand and think through how such an article gets into the caches and statistics. We need to inform our communities about what people are reading and where a human touch can bring such articles to the next level of quality.
Thanks,
      GerardM

#Wikidata - Heroes of the Soviet Union

On the Russian Wikipedia, there are 10898 entries in the category for heroes of the Soviet Union. Only 9740 of them have a Wikidata item. With the Creator tool it is easy to add the missing 1158 items. It gently adds them one at a time.

Adding statements for over 10.000 heroes is a bit too much for the AutoList tool. There are several edits to make. First, all the people are a human and then they have to receive the recognition in Wikidata for the hero they are.

It is much better to use a bot for this. What clinges it is that many people on the Russian Wikipedia have a template with much more information than just this one award. Things like dates of birth and death, places of birth and death. Other awards they have received..

The Russian Wikipedia is a really rich resource and it will be wonderful when more of its information is reflected in Wikidata.
Thanks,
       GerardM

#Wikidata - killing off zombies

I was brave, I killed off many of the zombies of the class of 2014. It felt good but then, new zombies showed up.

I killed off hundreds of them and, more are coming. There are the ones of last year and, the year before last. There are the many, many more from before 1900.

It is overwhelming.. How do I get away from it all? Do I need a bicycle because it runs quiet, a motorcycle because it is fast, a zeppelin...

Actually, I may need an army of drones that mindlessly looks up the data of the dead and expires them all.. Tirelessly, one at a time..

When such drones have worked their way through them all, we have a fighting chance. A chance to know the dead stay properly dead. It will give us the quality time to remember them for who they once were.

When we expire at Wikidata the ones who are dead, we do inform the projects who care about these people. If they are on a watch-list, they will show up as dead. That is one way of keeping on top of the zombie menace and, it is the best way of improving quality all around.
Thanks,
      GerardM

#Wikidata - #bots help place objects on the #map

The Sanandaj airport "is in the administrative-territorial entity" of the "Howmeh Rural District". Information like this is harvested by a bot from a template in Wikipedia. It checks that the lowest level of such an entity is registered. When for instance a "province" is registered in stead of the "county", the province will be replaced. In this way we make good use of the available information and the information provided gains in relevance.

For some countries it is the first time this functionality runs. It will make an impact on Nepal and Iran for instance. On other countries it will be at least a second run. The USA for instance will see changes to many if not all counties, townships and cities.

When the complete administrative or territorial structure for a country is available, the Reasonator will bring the best out of the available information. This will show even more clearly when categories and lists include the statements that run the relevant queries. The result can be compared with the content of the lists and categories. This makes for a tool that helps people complete the information provided in their Wikipedia.
Thanks,
     GerardM

#Wikidata - bots at work

When you roam through Wikidata using the Reasonator, you will find a lot of qanats. They are water works of a different kinds and they are equally important. These water works prevent starvation.

For many of them there are articles in the Wikipedia in Persian and, the only thing known was that they were qanats. I asked a friend and he told me that it was known in what "administrative-territorial entity" they are. As you can see in the screen print, that one is in the Kahak District of Iran.

The only problem was, Reasonator did not show it. Adding the fact that the Kahak District is part of Qom County proved enough to show 221 items, all qanats. My friend will add the other (missing) counties to districts. It will not only enrich the information about qanats but as relevantly also about Iran itself.

What I would like to see is to have all the villages, cities and what not be as well covered for Iran as it is for China. China has at this moment the best coverage of its administrative system.
Thanks,
      GerardM

#Reasonator - Johann Sebastian Bach in so many words

When you look at Wikidata, it Is very much like the proverbial glass. It is not even half full but It is filling rapidly. Unlike a liquid, every new bit of data becomes part of a web of data. At that it is more like a telephone network where every new endpoint makes the network more valuable to its users.

Like Wikipedia, Wikidata aspires to help people gain access to the sum of all knowledge. It already knows about more subjects than any Wikipedia but what it knows is incomplete, sometimes even wrong and often not accessible to everyone. Many of the Chinese villages and towns only have Chinese labels. Most of the locations in the USA do not refer to the lowest level of administrative unit they are in. For the towns and cities of so many other countries we do not have any data at all. Sometimes we may know they exist.

Like Wikipedia, Wikidata is a work in progress. Many articles are stubs and for many items there are no statements. Bots operate using list to generate more Wikipedia stubs.  Some clever programming uses all kinds of substitution to create a readable text and often the data in the list is the basis for an info-box as well. That is fine, when you can do that, more power to you.

Creating such texts is something that can be done once or multiple times. Magnus wants a better narrative the Reasonator produces. The beauty of this approach is that more text even better text will be generated when more information becomes available. Better texts will also become available when the routines that generate the texts become more clever.

For now the improved narrative is in English only. Improving the software is an iterative process. The first iteration is just to have something that works, the second iteration is where language specific code is separated so that code for a different language can be used when it becomes available.

It will be interesting to see how popular this functionality will be. For language technology students or professionals it will be a fun project to have a stab at this kind of stub language. What will be interesting is what resources they will ask for in order to make their text really expressive.

Thanks,

        GerardM

#Reasonator - Albert's creeping zinnia

Compare the Reasonator page to the #Wikipedia article for Albert's creeping zinnia. There are a few extra features on each side. Obviously the Wikipedia article has more text. Reasonator features:

• a picture
• a hover box for all the items it refers to
• they may have their own photo
• the concept cloud
• a QRcode 

It is the picture that warrants special attention. Many of the species on Wikidata do not have a picture. Many of them do not have a picture on their Wikipedia article either.

To remedy this Magnus is running a bot looking for freely licensed pictures on Flickr. Such targeted actions do have results... lots of results. The only question is who is going to verify that they are indeed of the missing species.

Thanks,

        GerardM

#Bots to the rescue

In #Wikidata, any item that does not have a label is of questionable value. According to the statistics, at the beginning of the year there were some 114,000 items without a label.

It is not a real problem as the name of an article can be used for a label. The most recent Wikidata statistics prove what bots can do. Currently there are some 9,000 items left without any label.

Some items have multiple links to Wikipedias and some of these links are without a label. These have also been updated. They are the real winners because these items are more likely to have statements. This is where all this effort truly adds value.
Thanks,
      GerardM

#Wikidata: Operation "Fat Head" is a GO

New functionality for #Pywikipedia has been writen. When you use search in Wikidata, there were many missing links because there was no label for them.

Operation "Fat head" is a job executed by Dexbot it uses the names of Wikipedia articles for the new labels. They are used for searching Wikipedia so they will work equally well for Wikidata.

I expect that the most requested items will benefit the most and, as a result Wikidata will gain relevance in many languages.

Thanks,

      GerardM

The #Vietnamese #Wikivoyage is joining #Wikidata

The Vietnamese Wikivoyage has been live for some time now and, many of its articles were linked in the classical way to articles in the English Wikivoyage. Thanks to Dexot, all the articles are getting the royal treatment. The old links are deleted and they are linked into Wikidata.
Thanks,
      GerardM

A #multilingual #Wikipedia supported by #Wikidata

A proposal has been made for a multilingual Wikipedia. To make it work it will use available information in Wikidata.

Above you find genealogical information of Nintoku, a Japanese emperor. As you can read in the screen shot 1990 people are included and you may notice many of these people do not have a label in English.

As there are labels in at least one language, it is possible to transliterate it into other scripts and or other languages. There are standards for transliterations and consequently it is technically possible to create the missing labels.
Thanks,
        GerardM