#Wikidata - all notable #Psychiatrists - a query for big data

When you look at all the psychiatrists known to Wikidata, there are currently some 2992 psychiatrists known. When you think about psychiatry, the relevance is in the number of people who have to deal with it.

The tool used is one that does not get that much attention. It takes its time to complete but it gets whatever it is the query says. The tool does its job admirably for several years now. The one redeeming advantage is that it does what 'official query' does not offer. It can be used never mind the size of the results.

Some say that it is unrealistic to ask for this quality of service from 'official query' because "it has not been designed with this in mind". The friendliest thing to say is that this is a mistake. Official Query was supposed to replace WDQ and when it cannot by design, the design if wrong. A better argument would be that Wikidata is one of the biggest public facing resources on the Internet and people are actually using it; at this time it cannot cope. It takes money and lots of money to serve the whole world. Possibly. This approach however is an acceptable argument. It allows for seeking one or more solutions.

One existing solution is the "Toolkit", when you can have your own datastore, you can throw as much hardware at it to get results. You can give the WMF targeted money to have more hardware for you and I or implement existing software that may do the trick. We could explore if federated technology as it exists for Wikipedia could make a difference. What cannot be done is hiding behind an arbitrary choice that was insufficient from the start because official query is to replace what we already have and not take away from it.
Thanks,
      GerardM

#Wikidata - my #bias and two articles about #diversity

As a volunteer, I spend a great amount of time making Wikidata more informative. With currently 1,896,739 edits, it is obvious that I use tools.

What I am looking for in tools is that I can use them. They do not have to be scientific, they just have to be functional. It means that I can use it at home or wherever I happen to be on any computer.

At this time there are two tools for querying Wikidata. One provides us with near real time data and the other has huge prerequisites. It is however the preferred option by people with a scientific bend.

Both approaches have been used to write about diversity. Their outcome is similar. However, I am biased towards the tool that is available to me. If I wanted to, I could run the same queries and will have have similar results. Results that will be different because of the time that has passed.

The other tool requires huge investments of me and it will only provide me with static data. Maybe the results are the same and very scientific but it will not help me improve Wikidata, it is therefore of no use to me. It reflects on data from the past. It does not compare data from the present with data I have elsewhere.

On this blog I did mention gender ratios like the two publications do. My issue with all that information is that it misses on one thing; how Wikidata is becoming more informative about diversity. As it is becoming more informative, it becomes also more useful as a tool to look at diversity in Wikipedia in the past.
Thanks,
       GerardM

#Wikidata - WDQ with load balancing

The #Wikipedia app on the #mobile is to give you everything that is near you. The question is, should it be based on Wikipedia or on Wikidata data. In order for software to find geo references, "magic words" need to be employed in Wikipedia. These same magic words can be used to harvest the information for Wikidata.

So what are the benefits of using Wikidata over Wikipedia with magic words .. Most importantly, there is only one Wikidata and there are 280+ Wikipedias. Everyone seeking information about subjects nearby is as entitled to great information as anyone else.

Wikidata does not have official query functionality. But it does have WDQ. Magnus and Yuvi are finishing the implementation of load balancing for WDQ. So the question is not can we serve geo coordinates from Wikidata but can we afford to let this opportunity slip us by.
Thanks,
      GerardM

PS There has been no evaluation of WDQ yet by WMF engineers.. Why not?

#Wikidata - #Cambridge revisited II

When you are interested in #maps, projecting the results of a Wikidata query on a map will become increasingly exciting. It seems obvious but only those items that have a geo-coordinate will find their place on a map. When I blogged about Cambridge revisited, the message was very much that we can find all items within a certain radius and, that we can process them with AutoList and WD-Fist.

Magnus is now providing us with something new. The results of a WDQ query are projected on a map. When you run a query, it could result in every item with geo-coordinates in a municipality, a county. It could be all the castles of the United Kingdom.. Just give your imagination some room.

As always, this is the first iteration of functionality. It makes use of components that we have been using for a long time. What will be interesting for us is to learn if the map functionality is able to cope. We learn by trying things out and at that, this is the perfect environment for you.
Thanks,
       GerardM

#Wikidata - #Cambridge revisited

When you are interested in #maps, it helps that Wikidata includes more and more geo-coordinates. So yes, you can put Cambridge on a map. This is old news, but somewhere hidden in the documentation of WDQ, it says that you can query for all items that are within a specific radius (in km) from a given coordinate.

When you apply this to Cambridge, the API result can be found here. The query definition can be used to produce an AutoList but you may also be interested in adding a missing image.

For me these tools and Wikidata is where we are "getting a job done". The job is being able to share the sum of all knowledge with our public. Wikipedia is great as far as it goes. Wikidata has the potential to inform in any language about any subject whether there is an article or not. It is becoming less of a potential as more data becomes available because Wikidata is increasingly potent.
Thanks,
      GerardM

#Reasonator - bugs and a different kind of awesome

Unlike Pallas Athena, Reasonator did not rise fully armed out of the head of Zeus. It has developed incrementally and every now and then it has its moments where bug manifest itself. It is software and, Magnus does a splendid job at terminating bugs.

The typical workflow is as follows:

• build a feature
• test in test
• promote to live environment
• bug Magnus with a bug (reiterate process)

This works well enough. Typically the environment Reasonator operates in is stable and bugs are obvious. Sometimes there are issues. Issues like a major re-write of functionality and an upgrade of the Labs environment. This is when the unexpected happens and bugs have an impact on functionality.

At this moment the update process kills Reasonator. But as the show must go on, it is better to do away with updates for some time until there is a bug fix. At this moment the daily backups are not available in the new Labs environment. The problems with updates cannot be limited to only one day.. BUMMER

The good news is that one bug has been found and fixed. It was late so it was time to sleep. Today it will be time to test. When things turn out to be fine, the update process will run again and both Reasonator and WDQ will provide near real time results from Wikidata.

It is when all the small things that happen every day that amount to so much start to show up again.

Thanks,

      GerardM

#Wikidata project - Actors from #India

#Wikidata becomes useful when there are many items that are connected. Lists and categories connect items because they imply that statements on all items are shared.

Lists and categories can be defined with properties that can be used to run a query. Important is that the inverse is true as well. A Wikipedia list or category implies that particular statements are true for the items that exist on a list or in a category.

A category like "Actors in Hindi cinema" indicates:

• "instance of" "human"
• "profession" "actor / actress"
• "language" "Hindi"

It does not imply that these actors have the "nationality" of "India". This is implied by the category "Indian film actors".

Thanks to WDQ, it is possible to know which actors who play in Hindi movies do not have the Indian nationality. This query makes it possible for people who care about Bollywood to consider the nationality of the 224 actors who fit this query.

This describes actionable information of a project. Your project can be done in the same way. You add data using AutoList or WD-Fist and Reasonator will display the information in an attractive way. These are tools that provide you grip on your projects. Wikidata provides you with information that may exist in any or none of the Wikipedias.

Thanks,

       GerardM

#Wikidata - What to do with "list articles"

When you trawl through Wikidata using the "random item" in Reasonator, the lists, categories, disambiguation pages are a pest. As it is, there is nothing for them to show. Given that people want them to be part of Wikidata, you can either ignore them, the current practice, or you can seek relevant information for them to show.

When you have a "List of sieges", it is obvious what they intend to show; on a Wikipedia, such lists include articles that are about sieges. Wikidata is becoming rich in information and, at this time there are 825 sieges known. That is more than on any of the existing lists.

When you analyse the information on this "List of sieges", it does include everything to build a query "is a list of" and "siege". It becomes more interesting when you want to identify sieges in a specific time frame or sieges with a specific participant. There are lists for that as well. To find them in Wikidata, the items we search for have to be rich enough in data for a query to find them.

The first thing though is to provide enough information on the item for the list to build a query, the second is to have enough information on the items it is to find.
Thanks,
        GerardM

An update to #WDQ, the tool that queries #Wikidata

WDQ is the tool where you define and run queries on the Wikidata data. It is not on official tool, it runs on replicated data in Labs and, it works really well.

While you prepare your query, you will see the number of results and the first 500 results. They are the maximum number of results that Wikidata provides when you request for information through its API.

When the query definition is satisfactory, you could create a "permalink" for future reference. What is new is that you can move on to the "AutoList:". This is where you can page through all the results and select items for special attention.
Thanks,
       GerardM

Women's History Month, #India-2014

On March 8th several edit-a-thons will be held in India to celebrate "International Woman's Day". In preparation a list of women whose articles need to be created or expanded has been created.

When you look at it from a Wikidata perspective, the first thing is do they have an item. When you start from an English perspective, you can search for them using the Reasonator; all of the women mentioned now have an item.

WDQ is a tool that really helps; it allows you to query the number of Indian women in Wikidata. It provides a nice interface where you can construct queries. A constructed claim like (claim[27:668]_AND_claim[21:6581072]) can be used in yet another tool, the AutoList.

The AutoList provides much more informative results and, it has the added advantage that you can work with all the items on the list even when there are more than 500.

Both Reasonator and AutoList are great tools to coordinate multi lingual projects.
Thanks,
      GerardM

#WDQ - the Wikidata #query is pumping

#Wikidata supports quantities. Consequently #Reasonator has to show available quantities. The obvious next step is: being able to query for quantities.

One of the obvious quantities people like to boast about is the number of cylinders in their motor. Obviously both the motors and their numbers of cylinders have to exist in Wikidata. At this moment there are only two engines with 14 cylinders known.

It is great to be able to find them. This is one kind of miracle. The other miracle is the Wikidata community; they will start pumping huge quantities of quantities in Wikidata. This will really have a big impact on Wikidata because the number of statements per item will mushroom. We live in interesting times.
Thanks,
       GerardM

#Wikidata - the day Pete Seeger was born

Recently Pete Seeger died. The day he was born is random enough to show you the latest improvement in #Reasonator. All kinds of persons and events are on display. Some do not have a label in Russian while others do not have a label in English. You may notice that the Paris Peace conference was under way..

You may even notice events, the death or birth of people that are missing. For these events, just add them in Wikidata and given the vagaries of replication, you may notice them a moment later. Technically, five queries are happening simultaneously. They may take some time when you experiment with longer periods. To do that you click on the calendar in the sidebar. To make the waiting time a bit more dynamic, a waiting bar has been added.

By showing you events that are deemed notable in any and all of our projects, Wikidata truly shares in the sum of all knowledge. Reasonator is happy to visualise the sum of all knowledge as we have it in Wikidata.
Thanks,
       GerardM

#Reasonator - the need for speed

If Reasonator is to work well and has to continue to work well, it has to scale and be as efficient as possible. Magnus has done a lot of work recently to improve its responsiveness. As it is, it does get its data from elsewhere and it is all about efficiently handling requests for information.

These are three results of the recent improvement in performance. 

Johann Sebastian Bach (Q1339)Wikidata : 18 sec (no images, no related items)
Old Reasonator : 12 sec (14 with images)
New Reasonator : 6 sec (9 sec with images)
Mammillaria (Q311120)Wikidata : 5 sec (no images, no taxon tree, no related items)
Old Reasonator : 8 sec (11 with images)
New Reasonator : 7 sec (10 sec with many more images)
Cambridge (Q350)Wikidata : 7 sec (no images, no location tree, no maps, no related items)
Old Reasonator : 8 sec (9 with images)
New Reasonator : 7 sec (10 sec with many more images)

The data is retrieved from Wikidata using the available APIs. There are two ways to look at that; you either accept what you are given OR you know that WDQ is able to provide the same data but faster. WDQ is as you read on Magnus's blog all of Wikidata but running completely in memory. At start time it is only 868MB RAM in size.

Help is welcome to make Reasonator and WDQ scale and to improve its user experience. Its page view statistics indicate a bight future and therefore an urgent need to consider all of this. Things that are considered are:

• using puppet for better management and multiplexing the services provided
• fixing memory leaks
• packaging WDQ and Reasonator

We expect that Wikidata will be able to cope with us in bringing the best information available to us.

Thanks,

      GerardM