Showing posts with label performance. Show all posts
Showing posts with label performance. Show all posts

Saturday, February 08, 2020

The performance of Wikidata - Denis Karuhize Byarugaba

Professor Denis Karuhize Byarugaba is one of the Fellows of the Uganda National Academy of Sciences. At Wikidata we know about papers that he wrote, we know this because of the author strings that point to him.

One of the Scholia tools allows for the disambiguation of these papers by linking to the Wikidata item. It is important that we do because in this way we build on the existence of African scientists on Wikidata.

That is the theory, the practice is that it is increasingly cumbersome to even try to add papers because Wikidata more often than not informs about Too many requests. When this happens occasionally it is fine but when only 10 percent of the requests is honoured, the tool is effectively dead.

Wikidata is the most promissing tool of the Wikimedia Foundation and there is as far as I know no path forward. Obviously it affects people in what they do it affects the projects that are not progressing as fast as they could or should. Even when there is a notion of improved performance it is easily missed because of the pent up demand for much more power. Power to query and power to edit the data. We are not sharing in the sum of all knowledge when it is this hard to make it available.
Thanks,
      GerardM

Tuesday, October 28, 2014

#Wikimedia - Men at work; preparing a #presentation III - #WCN2014

The bane of every live demonstration is when the software just does not work. My intention is to show #Wikidata in action. Demonstrate the Reasonator and AutoList2. When the experience of the last few weeks is anything to go by, I have a 50% chance of a reasonable result on the day.

There are many factors that can play up. Time outs at Wikidata are no exception at the moment and when Wikidata does not play ball, everything downstream from it suffers as a consequence. It means that I may not have a recent list of recent deaths because ToolScript does not function.

AutoList2, relies on WIDaR. It relies on being able to contact Wikidata reliably. Without this, AutoList2 does not run.

The subject of my presentation is firmly solution oriented. I can always fall back on screenshots. That feels like cheating.
Thanks,
      GerardM

Tuesday, February 04, 2014

#WDQ - the Wikidata #query is pumping

#Wikidata supports quantities. Consequently #Reasonator has to show available quantities. The obvious next step is: being able to query for quantities.

One of the obvious quantities people like to boast about is the number of cylinders in their motor. Obviously both the motors and their numbers of cylinders have to exist in Wikidata. At this moment there are only two engines with 14 cylinders known.


It is great to be able to find them. This is one kind of miracle. The other miracle is the Wikidata community; they will start pumping huge quantities of quantities in Wikidata. This will really have a big impact on Wikidata because the number of statements per item will mushroom. We live in interesting times.
Thanks,
       GerardM

Monday, January 27, 2014

#Reasonator - the need for speed

If Reasonator is to work well and has to continue to work well, it has to scale and be as efficient as possible. Magnus has done a lot of work recently to improve its responsiveness. As it is, it does get its data from elsewhere and it is all about efficiently handling requests for information.

These are three results of the recent improvement in performance. 
Johann Sebastian Bach (Q1339)Wikidata : 18 sec (no images, no related items)
Old Reasonator : 12 sec (14 with images)
New Reasonator : 6 sec (9 sec with images)
Mammillaria (Q311120)Wikidata : 5 sec (no images, no taxon tree, no related items)
Old Reasonator : 8 sec (11 with images)
New Reasonator : 7 sec (10 sec with many more images)
Cambridge (Q350)Wikidata : 7 sec (no images, no location tree, no maps, no related items)
Old Reasonator : 8 sec (9 with images)
New Reasonator : 7 sec (10 sec with many more images)
The data is retrieved from Wikidata using the available APIs. There are two ways to look at that; you either accept what you are given OR you know that WDQ is able to provide the same data but faster. WDQ is as you read on Magnus's blog all of Wikidata but running completely in memory. At start time it is only 868MB RAM in size.

Help is welcome to make Reasonator and WDQ scale and to improve its user experience. Its page view statistics indicate a bight future and therefore an urgent need to consider all of this. Things that are considered are:

  • using puppet for better management and multiplexing the services provided
  • fixing memory leaks
  • packaging WDQ and Reasonator
We expect that Wikidata will be able to cope with us in bringing the best information available to us.
Thanks,
      GerardM