Showing posts with label statistics. Show all posts
Showing posts with label statistics. Show all posts

Friday, February 16, 2024

Food for thought; statistics and Wikidata - DONT BE A KARELIAN

The lumberjacks in Karelia Finland got all the physical activity you can expect for lumberjacks, they looked the part and they died in droves before their fifties. This was as well known in the world of health scientists as well as the fact that in Japan people had the least problems with heart failure. Epidemiologists started one of the most famous studies, the "Seven Countries Study" to learn about these phenomenon. The Karelians ate a lot of meat and butter, this caused arthrosclerosis and it was identified as the cause of all these early demises. 

The Finish government wanted this to change, the lumberjacks loved their meat but their wives loved their hubbies more and they started them on a different diet. The government did a double blind research project and the fine Karelian gentlemen started to outperform their fellow Fins... As a consequence the Finnish government promoted healthy food to all Fins.

In Wikidata we have MANY scientific publications with "Seven Countries Study" in the name of the publications. With more than 100 such publications tagged, many authors, publications and subjects have become apparent. This can be seen in the Scholia for the Seven Countries Study. Statistically it is likely that when another 100 publications are added, the patterns found may slightly differ. Additional authors may be represented but the relative weight of existing authors is likely to remain the same. 

Ancel Keys is the architect of the Seven Countries Study, he authored both papers and books with many publications and publishers and he collaborated with many of the most prominent scientists in his time. The results of all these published studies are profound and not only for the Karelian lumberjacks. Not everybody is happy with the results. Influencers have us believe that Mr Keys misrepresented the facts of the study. However, when you look at the co-author graph, Mr Keys is not really central to all the collaborations. It is also obvious that there were many different publishers involved. 

The meat of the matter is obvious. Don't be a Karelian of centuries past, be smart, be there for your nearest and dearest and understand that a traditional Japanese diet or the Mediterranean diet gives you more mileage. The Seven Countries Study had a run for over fifty years, it knows about what people ate and the mortality that is the consequence of their diet. You can ignore this at your own peril :)

Thanks, GerardM

Saturday, June 27, 2020

Hey @Wikimedia lets move the needle

The Wikimedia projects are biased. They favour only one language, the English language. When you look at Wikipedia traffic English Wikipedia is something like 50% and it does not represent 50% of our intended public. 

The objective is to improve the usefulness of the other projects and thereby increase their traffic. That is, more articles and books are read, more pictures are seen and downloaded.

Lets pick one language, Yoruba, as an example. There are currently 32,624 pages in its Wikipedia. There are some 40 million people speaking the language. So what can we do for Yoruba editors and readers. How can we track what makes a difference and also what makes a difference and what can the WMF do to achieve this.

* We can improve list support. 
Currently the best support for supporting lists in a Wikipedia is "Listeria". It is supported by Magnus.. Listeria lists have been shown to be more up to date then manual lists on English Wikipedia, for less resourced projects this will be even more true. When existing lists can be easily included in an article, it will expand available information hugely.. Here an example of Listeria lists on the Yoruba Wikipedia. Content of these lists show in Yoruba. Lists are better supported and adopted when it is WMF supported functionality.

* Choosing pictures for illustration
When people look for a picture, they have to goto Commons or they visit Wikipedia articles on the same subject and use these same pictures. When the Special:MediaSearch is available as a tool from every Wikipedia article, a much richer palette of pictures becomes available to choose from. (The search is for "Agbègbè Ìjọba Ìbílẹ̀ Mushin")..

The cool thing is, when this tool is available when writing an article, it is easy to more pro-actively add labels to Wikidata. This will improve the performance for the Special:MediaSearch even more.

What would truly support Special:MediaSearch is disambiguation. It is unreasonable to expect that we get descriptions in all the 300+ languages we support. What Reasonator supports are automated descriptions. It makes it easy and obvious to choose the right item in any language.

For the Wikimedia Foundation to support other languages, for it to move the needle on any and all languages, we need to measure what is meaningful. The number of searches by Special:MediaSearch and what language was used. The number of pictures used in each Wikipedia. The effect lists have on the writing of new articles.

When we did not measure such numbers so far, it is what we should do to move the needle. One needle is the total number of reads quite another is the number of reads for each project. Same for the use of Wikidata and Commons.
Thanks,
     GerardM

Saturday, August 10, 2019

#Statistics or how many researchers are a #physicist

At @Wikidata most "researchers" are given this "occupation" out of convenience. We do not know how to label them properly, there are too many, so as all scholars must be researchers we make them so.

Nothing inherently wrong here; it is better to know them for what they also are then to know nothing about them at all. One issue though; we do not know the physicists from the chemists, from the behaviorists or any other specialism in science. We can query for physicists anyway but we will not catch them all.

Queries that show the numbers for a profession are easy enough to make. The value of such one time wonders is minimal, the results are fleeting, any moment now another scientists like Walter Hofstetter may become known to be a physicist and the numbers are no longer true. They are useful when we run queries like these regularly, save the results and present them like Magnus does for Wikidata itself.

What it takes is a mechanism that mimics Magnus's approach. We gain an insight in how Wikidata is performing over time and it provides motivation for people who care, for instance about physicists.
Thanks,
       GerardM

Tuesday, August 06, 2019

#Statistics for National Young Academies

The Global Young Academy is linked to many national young academies. It and they represent many relevant scientists. They represent all of science and, they are interested in representing science to the public. The question: how can we make them visible.

First you add the orgs to Wikidata and then the scientists to the orgs. When you then add the same Listeria lists to Wikipedias, we will see a picture when we have one and we may notice who has a local Wikipedia article.

There are many interesting statistics possible:
  • the gender ratio
  • the different professions in the mix
  • awards received 
  • the known number of publications per person
  • the organisations they are employed at
However, first things first. It is my intention to include all the current members and the alumni of the Young Academy of Sweden before Wikimania.. Second, these scholars are bright :) once they put their mind to it, they will help themselves to nice statistics based on the info we accumulate in Wikidata. They can be linked to on the Wikipedia pages.
Thanks,
     GerardM

Friday, March 10, 2017

#Wikidata - dating awards


Many awards are dated using "point in time". With a query you can count them, certainly when you are WikidataFacts. Looking at a graph like this, you will see that many awards for 2016 and 2015 are still missing. Many of these awards were imported in the past and have not been updated yet.

It would be cool if we knew what Wikipedias have an article for the awards and would be "pinged" when new values are known.
Thanks,
       GerardM

Saturday, October 22, 2016

#Wikidata - statements are doing fine

In September there are more Wikidata items with 10 or more statements than items with no statements. Wikidata is growing up.
Thanks,
     GerardM

Wednesday, March 02, 2016

#Wikidata - Statements per item

One of the most exciting statistics for Wikidata is the one where you see the number of statements per item. As Wikidata matures, more statements are added. The scale is from 0 to 9 and finally10 or more statements. The latest available dump has 10 or more at over 10%. The number that has been reported in the past is about the number of items that make up 50%. Today they have 0, 1, 2 or 3 statements.

Slowly but surely we are getting to the point where ten or more statements is too big much as it no longer satisfies our need to know. It is a good thing that wiser heads will determine when that will be :)

PS At Wikidata we are doing great :)
Thanks,
      GerardM

Sunday, September 27, 2015

#Wikidata - primary sources tool statistics

It is a good thing that there are statistics for the primary sources tool. It is a dashboard that shows the current state only.

Given the discussion on the usefulness of this tool, this is not really helpful. It does not help any argument because everyone will be given different numbers at a different time.

Compare this with useful statistics for Wikidata. Here values are available that show trends over time. Consequently action can be undertaken based on the numbers. It would be really welcome that as part of the creation of these statistics, current numbers for the primary sources tool would be included.

Either way, success or failure, statistics help when people agree that numbers are relevant.
Thanks,
     GerardM

Saturday, September 26, 2015

#Wikidata - #Freebase atrophies in the Primary sources status

It is a good thing that there are statistics for the primary sources status. It demonstrates clearly how dysfunctional it is. Only 18K statements have been approved. After all the time that the tool exists, it is not even one percent.

For a "serious" power user it is quite possible to do add this number of statements in a day to Wikidata any day. The sad thing is there is every reason to believe that the quality of a power user is just as good as anything that is in this dump in the first place.

Mathematics show that it is easy to check and verify the data that is in Wikidata with other sources. When such a process is well designed, it is iterative and consequently adding data that is deemed useful for inclusion in Wikidata will be processed in every iteration.

These sad statistics demonstrate one thing and one thing only; the failure that is in this approach. It would be wise to abandon it and concentrate on workflows instead that leverage the value that is in the huge community that may serve fixing issues.
Thanks,
       GerardM

Monday, July 13, 2015

#Wikidata - more great statistics

When you want to know what Wikidata data is about, you find it in the statements "instance of" and "subclass of". The illustration is part of a set of new statistics by Zolo. It concentrates on these instances and statistics and a big surprise is that there are still so many more items without one (32.6%). The statistics by Magnus show that only 19.07% of all items have no statement.

The great thing of the new statistics page is that it shows pie charts for many Wikipedias. English Wikipedia for instance for instance does not know about 1.5 million people Wikidata knows about. When you analyse this, you get into the domain of set theory. These people can already be subdivided in the following way:
  • people only en.wp knows about
  • people en.wp knows about as well as others
  • people en.wp does not know about but another wp does
  • people no Wikipedia knows about
  • items Wikidata does not recognise as people
People and unidentified items are half the pie for Wikidata. In all cases they are pretty well connected to articles in whatever project and provide a perfect opportunity to provide information. I blogged about it before, wikilinks and red links are obvious targets for links to other information not only for people but also for people.

Many people have #Babel information on their user page. They thereby indicate their proficiency in a language. When a Wikipedia does not have an article, why not provide an article in a known language? When there is no article why not provide at least the information that is available? It will help an editor write a new article, it will provide a reader with at least some information.

The point is that we should share in the sum of all available knowledge because we can.
Thanks,
     GerardM



Sunday, April 05, 2015

#Wikidata - #statistics


This chart is really interesting; it was made from the last dump (2015-03-30). Item numbers are subdivided into intervals of 100,000. So for instance (X,Y) = (192,73720) represents the interval [Q19200000, Q19299999] having 73,720 out of 100,000 items without a single claim.

It is interesting because it indicates to me that items are imported in batches. As the items in those batches are similar, it follows that even though many statements are added all the time, these items do not get similar attention.

Reasonator has a "Random item" feature and I used it to get a feeling what items do not have any statements. It seems to me that they are mostly items with a sitelink to "another" language and often they are about subjects that are in a relatively small class.

The items with a sitelink to "another" language are probably the most problematic. It is realistic to expect that many of them could be linked to another item when it was clear what it is the item is about. For that you need people who know the language and/or you need a way to figure things out using tools like Kian.
Thanks,
         GerardM

Thursday, January 22, 2015

#Wikidata - a year in numbers


The Wikidata dumps are finally available, Magnus did run his job so we have new statistics for Wikidata.

As we have the statistics for all of 2015, we know how much Wikidata has changed. The two most relevant indicators are labels and statements.

It is great to notice how many items have more than 5 labels. It is great to notice that we have few items with no labels; it is a constant struggle to keep it that way. Amir does a great job.

Statements are doing fine. Many more have been made. The number of items with no statement fell by 15.92%.The number of items with more than 10 statements more than doubled and the high end of this table has increased considerably as well.

It is obvious that in the last year, Wikidata became considerably more relevant.When you have used a tool like Reasonator for a long time, this is obvious. It will be interesting to see what 2015 will bring us in new functionality and data. Wikidata is like Wikipedia in its early days; interesting and showing promise. It is very much in need for collaboration to expose the parts where a Wikipedia does not reach.
Thanks,
      GerardM

Thursday, December 18, 2014

#Google - Would you use your #trickery for us?

After the shock and awe of yesterday's announcement, There has been time to think. Google showed a real interest in Wikidata. It created a new tool to help improve the quality of its data. But the real expertise of Google is in determining the probability of facts. It is part and parcel of its ranking algorithms.

It would be as awesome when Google would indicate those statements it deems to have a less than even chance of being true. The combination of such a list and the new tool would make the efforts of the people seeking sources all the more relevant. When statements are debunked, it has a potential quality effect to all the associated Wikimedia projects. Given that it is probable that most statements are fine, it makes for more concentrated effort and consequently its effects will be noticed.

While we are on this line of thought, given the data of Freebase, Google could indicate based on its algorithms how probable its sets of data are. Everything that is highly likely should be a candidate for import in Wikidata. The other reason for importing data into Wikidata anyway is that it is an invitation to all the Freebasers to join our ranks, increase our expertise and together be awesome.
Thanks,
      GerardM

Monday, December 08, 2014

#Wikipedia- The numbers are what II ?

Numbers, statistics have a purpose. Their typical use is to have manager types consider how things are moving. Even though they have infinite wisdom there is not much that they can do when all numbers do is show trends.

The funny thing is that only the numbers that are collected are reflected in these trends. Wikidata for instance attracts no readers and consequently it may not be considered as a source of attention. It is a fallacy and it does not motivate people interested in Wikidata.

There are so many lists that could motivate people. Articles that need writing, differences in data between Wikipedias. The wonderful thing is that they all bring a sense of purpose and are an inspiration to improve both quality and quantity.

My favourite list is the list of zombies for 2014. Currently there are 400+ zombies that need to be killed of. It motivates because I know what to do and it is a convenient way to find categories of information that can easily be imported in Wikidata.

Statistics, numbers can motivate people to be more effective. That is how you influence the numbers these manager types look at.
Thanks,
       GerardM

Friday, December 05, 2014

#Wikipedia - #Russia and the #USA

Yesterday I wondered when Russia will become the champion users of Wikipedia. Today I noticed a presentation with the numbers.. Traffic to Wikipedia from the USA is down by -8.6% while the annual growth of Wikipedia traffic from Russia is +10.3%. A difference of almost 20%.
Thanks,
     GerardM


Thursday, December 04, 2014

#Wikipedia - #Russia #rules OK


English Wikipedia is still the biggest. The Russian Wikipedia however is growing much faster. It replaced the Germans, the Spanish and the Japanese to become the second biggest Wikipedia. When it continues to grow like this it will overtake English Wikipedia.

There is one question in the back of my head... When you consider only the traffic from the USA for the English Wikipedia, and Russia for the Russian Wikipedia, how will they compare ? How long will it take for Russia to overtake the USA as the champions of Wikipedia.
Thanks,
      GerardM

Sunday, November 30, 2014

#Wikimedia - #Wikidata; a recurring subject


The last Foundation Metrics meeting is kinda interesting, particularly when you are interested in #Wikidata. In his part of the meeting Erik considers the implications of Wikidata and wonders what it takes to help it lift off.

Erik wants us to change the world. Now that is a big statement. It can be done. It takes big thinking, maybe even bigger thinking or maybe no thinking at all. In the vision Erik presented, it is all about data and leveraging the data for instance in info-boxes.

Have a closer look at Reasonator, to its statistics and to these specific statistics (it takes a long time to load). What you find is probably the most easy and effective thing to do. It is allowing people to add labels in their own language. Labels leverage all Wikidata statements for a language. With those labels you can disambiguate effectively in a search. Try it in Reasonator. Now change the language and notice that effectively the automated descriptions are still there. Now do the same with search in Wikidata.. See ?

At Wikimania there was a heated discussion about the need for descriptions . The only half baked argument to keep the current descriptions was that people outside expect them. Lets not be strangers.
Thanks,
        GerardM

Sunday, November 23, 2014

#Wikimedia #chapters, the data

Guess what, Wikimedia chapters are linked to many other organisations. These organisations are known in Wikidata and now the chapters are known as well.

For many GLAM partners we have all kinds of statistics. We could link the partners to the chapters that they are connected to.. It is the basis for information on the usefulness of the chapters.
Thanks,
      GerardM

#Wikimedia - the point of #collecting #data?

If #Wikidata is one thing, it is useful. It was useful from the start by including all the Wikipedia articles who are linked to articles in other languages. In the next phase statements were added and more and more articles were added that did not link to other articles. They were needed because they were a part in the expression of a statement. Then for all articles Wikidata items were created and still more items were created because they were needed in the statement of expressions.

There is a point to linking the articles. It enables people to read about the same subject in other languages. There is a point to adding statements to items; it enables articles to be linked to whatever. This combination enables us to report on Wikipedia in ways not yet done.

If you want to know about the gender division; currently these are the men, the women in all our projects. Since June 2014 90,850 more items became known to be women and 445,240 as men. Interesting but this information is not in a format that is "academic" or useful.  Having this information in a bar chart with regular intervals gives more insight in what we have. Using old dumps for this is one solution. Breaking the information up per Wikipedia provides even more granular information.

Providing statistics in this way is good for several reasons:
  • it is public and verifiable information
  • it stimulates people to add statements about gender
  • it stimulates people to write about men and women
  • it makes it obvious that it is Wikidata where we know these things
Thanks,
      GerardM

Wednesday, October 29, 2014

#Wikimedia - Men at work; preparing a #presentation IV - #WCN2014

The Dutch community has one question to answer: what to do with available information in Dutch? How will we make it available. Currently there are 3,054,955 items [1] with labels and there are 1,890,905 items [1] that link to the Dutch Wikipedia. It follows that 62% of the in Wikidata known items do not have an article in Dutch.

This is a substantial amount of information that can be presented in Dutch. Similar numbers can be presented for any language; for English it is 39% and for German 121%..

Arguably, these items fulfill notability requirements somewhere. Arguably the Swedes have demonstrated that having more information available revitalised their community. Arguably, allowing for search results from Wikidata is an easy first step towards opening up all our available knowledge.
Thanks,
     GerardM

[1] these links take a few minutes to load; they provide real time information