Showing posts with label Search. Show all posts
Showing posts with label Search. Show all posts

Friday, June 19, 2026

Retracted and outdated sources from a Wikimedia perspective


A recent article related to the quality of Wikipedia references indicates that when a paper is retracted, the median time for a correction is 3.68 years. There is a bot for that, the RetractionBot, updated in 2024, the problem is that people have to use it AND "expecting Wikipedia editors to continuously monitor every citation for new retractions is unrealistic"...

HOWEVER

Retracted papers are hardly our only problem. Information is often superseded in later publications. This does not mean that earlier works are retracted it means that the information our articles are based on is stale. There is no bot for that AND expecting Wikipedia editors to continuously monitoring for new information is unrealistic...

ALSO

As our existing content needs maintenance, our public is diminishing and all our communities of volunteer contributors have their own objectives we have a problem; what to do?

Why not flip the script, why not provide a search engine that includes all our references, our articles and items. We enrich it with information from Retraction Watch, Crossref, ORCiD and obviously the Internet Archive. As the new "WikiFind" community adds new information items, it links them to articles and items and builds a field with potential new references. 

The objective of the "WikiFind" search engine is to be informative and provide a structure that brings our projects together AND present all of this to a new public. The implementation should be similar to how we started Commons; at the time Erik Möller started a new Wiki and only later did it serve images to Wikipedia articles... Maybe a challenge this time.
Thanks,
      GerardM

Thursday, June 18, 2026

Ziyad Al-Aly, clinical epidemiologist from a Wikimedia perspective

 Ars Technica featured an article "Heart protection from COVID shots remains amid updates, study finds".  The article refers to multiple studies and mentions Ziyad Al-Aly, one of the authors of several of the studies. 

When you consider Dr Al-Aly as a Wikimedia consumer, then his Scholia produces the most entries into the Wikimedia rabbit hole like publications, co-authors and topics. Drilling down, a
Wikipedia article informs about his education and awards.

As a Wikimedia contributor, there is so much that can be done: attributing articles, adding missing articles thanks to the ORCiD-scraper, adding co-authors to papers.. And then there is the deep dive, adding for instance the awards known to the Wikipedia for instance.

Now for the Wikimedia perspective; it is losing its audience. Without an audience what motivates people to contribute? Without contributors, who maintains our content? Its main perspective for the future should be: how will we reach a public for the sum of all the knowledge we have on offer.

The essence of all that knowledge is in known references. The Ars Technica article could obviously serve as a reference for Wikimedia content. When the Wikimedia Foundation were to offer references and sources in a search engine, traditional Wikimedia content is available for a deep dive. When these sources are associated with topics that have a Wikipedia article, it also serves as a tool to update these articles.

A WMF search engine based on sources and references is not commercial, would provide a distinct service and will entice renewed interest in its projects.
Thanks,
       GerardM


Tuesday, May 26, 2020

@WikiCommons - Meanwhile in a school in India, Japan, Russia

These students in India have to do a project. The subject is Botswana. Their teacher wants them to find many pictures so he searched Wikimedia Commons among others for pictures of  Mokgweetsi Masisi, the president of Botswana. He marked the pictures that depicts Mr Masisi and now his pupils will find more pictures of him when they look for मोकेगसेसी मासी.

At the same time in Japan students have to do a project about Botswana. Their teacher is pleasantly surprised when he find so many pictures for モクウィツィ・マシシ...
Thanks,
       GerardM

Monday, May 25, 2020

@WikiCommons - meanwhile in a different universe

And again there was a discussion that it should not be this hard to find pictures in Commons. The big difference this time is that there is now a wealth of images that have been tagged for what they "depict". They are linked to Wikidata items and they have a wealth of labels in many, many languages. In essence it has always been an objective of Wikidata to share its content in any and all of the 300+ languages supported by a Wikipedia.

The ideas that floated around soon made it into a "proof of concept" and as so often it actually worked after a fashion. The first iteration was in true Wikimedia tradition English only. The proof of concept got its second language in Dutch, Hay Kranen the developer is Dutch. Now there are nine languages and we are waiting for French to be the tenth.

So what does it do. You can look for pictures in Commons, it has 61 million media files, and when you are looking for available pictures in your language, you will find it as long as Wikidata has a label in your language.  This is for instance a result in Japanese and this is the result in German.

What can you do to make it better? Add labels in your language for the things you want to find and find media files that depicts what you are looking for. When nobody translated the software in your language, you can even do that.

Why is this so relevant? Have you ever wondered how many pictures you find in one of the smaller languages using Google or Bing? Let me tell you, it is disappointing to be polite. Commons is the repository of the mediafiles that illustrate all the Wikipedias so yes, it covers "almost anything".

The Wikimedia Foundation has this big strategy for its movement to be inclusive. This is a wonderful opportunity to show how agile it is, that it understands and supports a need that has been expressed for many many years. The beauty is the the way forward has been expressed in something that already works.

ABSOLUTELY, there will be challenges in integrating this functionality where it fulfills a need.

Luckily it is not necessary for it all to be done in one go. The first step can be as little as to take the "proof of concept" an rewrite it in the preferred language of the WMF, internationalise and localise it and keep it stand alone for now. The people who know about it will use it and they will be the first to point out what more they want to be done. A priority will be to retain its KISSable nature.

The objective is to open up Commons. Open it up in any and all languages. For me it is obvious. I will gladly give it my attention in the expectation that both Wikidata and Commons actually find a public, have a purpose that is more than what we do for ourselves.
Thanks,
      GerardM

Sunday, March 24, 2019

#Sharing in the Sum of all #Knowledge from a @Wikimedia perspective II

When we are to share in the "sum of all knowledge" we share what we know about subjects; articles, pictures, data. We may share what knowledge we have, what others have and that is what it takes  for us to share in the sum of all knowledge. The question is why should we share all this, how to go about it and finally how will it benefit our public and how will it help us share the sum of all knowledge.

At the moment we do not really know what people are looking for. One reason is that search engines like the ones by Google, Microsoft and DuckDuckGo recommend Wikipedia articles and as a consequence the search process is hidden from us. We do not know what people really are looking for. However, some people prefer the "Wikipedia search engine" in their browser. We can do better and present more interesting search results. From a statistical point of view, we do not need big numbers to gain significant results.

When we check what the "competition" does we find their results in many tabs; "the web" and "images" are the first two. The first is text based and offers whatever there is on the web. What we will bring is whatever we and organisations we partner with, have to offer. It will be centered on subjects and its associated factoids presented in any language.

One template to consider is how Scholia presents. It differs. It depends on whether it is a publication, a university, a scholar, a paper. Large numbers make specific presentations feasible and thanks to Wikidata we know what kind of presentation fits a particular subject. A similar approach is possible for sports, politics. It takes experimentation and that is what makes it a Wiki approach.

Thanks to this subject based approach, language plays a different role. Vital is that for finding the subjects potentially differing labels are available or become available. One important difference with the Google, Microsoft or DuckDuckGo approach is that as a Wiki, we can ask people to add labels and missing statements. This will make our subject based data better understood in the languages people support. Yes, we can ask people to have a Wikimedia profile and yes, we may ask people to support us where we think people looking for information have to overcome hurdles.
Thanks,
       GerardM

Saturday, March 16, 2019

#Sharing in the Sum of all #Knowledge from a @Wikimedia perspective I

Sharing the sum of all knowledge is what we have always aimed for in our movement. In Commons we have realised a project that illustrates all Wikimedia projects and in Wikidata we have realised a project that links all Wikimedia projects and more.

When we tell the world about the most popular articles in Wikipedia, it is important to realise that we do not inform what the most popular subjects are. We could, but so far we don't. The most popular subjects is the sum of all traffic of all Wikipedia articles on the same subject. Providing this data is feasible; it is a "big data" question.

We do have accumulated data for the traffic of articles on all Wikipedias, we can link the articles to the Wikidata items. What follows is simple arithmetic. Powerful because it will show that English Wikipedia is less than fifty percent of all traffic. That will help make the existing bias for English Wikipedia and its subjects visible particularly because it will be possible to answer a question like: "What are the most popular subjects that do not have an article in English?" and compare those to popular diversity articles.

In Wikidata we know about the subjects of all Wikipedias but it too is very much a project based on English. That is a pity when Wikidata is to be the tool that helps us find what subjects people are looking for that are missing in a Wikipedia. For some there is an extension to the search functionality that helps finding information. It uses Wikidata and it supports automated descriptions.

Now consider that this tool is available on every Wikipedia. We would share more information.With some tinkering, we would know what is missing where. There are other opportunities; we could ask logged in users to help by adding labels for their language to improve Wikidata. When Wikidata does not include the missing information, we could ask them to add a Wikidata item and additional statements, a description to improve our search results.

This data approach is based on the result of a process; the negative results of our own Search and it is based on active cooperation of our users. At the same time, we accumulate negative results of search where there has been no interaction, link it to Wikidata labels and gain an understanding of the relevance of these missing articles. This fits in nicely with the marketing approach to "what it is that people want to read in a Wikipedia".
Thanks,
      GerardM

Saturday, March 09, 2019

A #marketing approach to "what it is that people want to read in a @Wikipedia"

All the time people want to read articles in a Wikipedia, articles that are not there. For some Wikipedias that is obvious because there is so little and, based on what people read in other Wikipedias, recommendations have been made suggesting what would generate new readers.This has been the approach so far; a quite reasonable approach.

This approach does not consider cultural differences, it does not consider what is topical in a given "market". To find an answer to the question: what do people want to read, there are several strategies. One is what researchers do: they ask panels, write papers and once it is done there is a position to act upon. There are drawbacks; 
  • you can only research so many Wikipedias
  • for all the other Wikipedias there is no attention
  • the composition of the panels is problematic particularly when they are self selecting
  • there are no results while the research is being done
The objective of a marketing approach is centered around two questions: 
  • what is it that people are looking for now (and cannot find) 
  • what can be done to fulfill that demand now
The data needed for this approach; negative search results. People search for subjects all the time and there are all kinds of reasons why they do not find what they are looking for.. Spelling, disambiguation and nothing to find are all perfectly fine reasons for a no show. 

The "nothing to find" scenario is obvious; when it is sought often, we want an article. Exposing a list of missing articles is one motivator for people to write. Once they have written, we do have the data of how often an article was read. When the most popular new articles of the last month are shown, it is vindication for authors to have written popular articles. It is easy, obvious and it should be part of the data Wikimedia Foundation already collects.. In this way the data is put to use. It is also quite FAIR to make this data available. 

For the "disambiguation" issue, Wikidata may come to the rescue. It knows what is there and, it is easy enough to add items with the same name for disambiguation purposes. Combine this with automated descriptions and all that is requires is a user interface to guide people to what they are looking for. When there is "only" a Wikidata item, it follows that its results feature in the "no article" category.

The "spelling" issue is just a variation on a theme. Wikidata does allow for multiple labels. The search results may use of them as well. Common spelling errors are also a big part of the problem. With a bit of ingenuity it is not much of a problem either.

Marketing this marketing approach should not be hard. It just requires people to accept what is staring them in the face. It is easy to implement, it works for all the 280+ language and it is likely to give a boost to all the other Wikipedias but also to Wikidata.
Thanks,
        GerardM

Saturday, June 24, 2017

#Wikipedia - Sister projects in search results

The Wikipedia Signpost informs that the discovery team extended the results for search on Wikipedia. New is that English Wikipedia now includes results from
Wikisource, Wiktionary, Wikiquote and  Wikivoyage and that is indeed welcome news.

There is one puzzling part in the information; "Wikidata and Wikispecies are not within the scope of this feature." It is puzzling because including Wikidata search results is where search has been augmented for years in many Wikipedias including the English Wikipedia by the people who added this little bit of magic Magnus provided.

As you can see in the screenshot of the search for Wilbur R. Leopold, an award was conferred on him and the origin of this factoid is the article on the award. Thanks to Wikidata, information is available for Mr Leopold. There are so many references in Wikidata that have no article in a Wikipedia or any other project that from a search perspective it is probably the next frontier.

When wiki links, red links and even black links can be associated with Wikidata items, it becomes even easier to add precision to the search results. Adding these links is the low hanging fruit to improved quality in Wikimedia projects anyway. 
Thanks,
     GerardM


Tuesday, July 26, 2016

Have #Wikipedia share the sum of available #knowledge

If Wikipedia is to succeed in sharing the sum of all knowledge, it has to first share the sum of available knowledge. To do this Wikipedians have to become more inclusive. They have to realise that Wikipedia is not about them but about its readers.

Typically the question "What do readers want" is answered by what readers find. This answer has one flaw. It assumes that Wikipedia includes what people seek and it forgets what people seek and do not find. This is a lost opportunity on many levels. To start with, Wikipedia is not singular and a subject may exist in another language. As we do not know what is missed, we do not know what to write to satisfy an existing demand. Finally more and more available information does not even have a Wikipedia article but its information is available in other projects.

A partial solution to these issues was around for a long time. It extends search by adding results from Wikidata. It allows you to find data in any script from any project. If there was no article, it shows information using the Reasonator. It is relatively easy to revive this and it will make even more sense when it results are included as positive results.

Once Wikipedians consider Wikidata as a tool, they will find that both red links and wiki links may link to Wikidata items. Typically they are the same links for the same subject in any language. This is relevant to editors because it is one way to clarify what links exist to an article and, it is only one step away to annotate them as statements in Wikidata and thereby document such links. They will find a lot of erroneous links and it will improve overall quality.

The good news, the links between wiki links and Wikidata items already exist. What is lacking is a verification process that these wiki links are good. Adding links to statements for red links is technically not that hard. It will add some turmoil at the Wikidata end; many items will be added and will have to be merged eventually. One benefit of this approach is that it is not necessary for everyone to collaborate but it will benefit the people that matter most; all the readers of all the Wikipedias.
Thanks,
      GerardM


Thursday, March 10, 2016

#Wikimedia - Learning from #Google #search

When the Wikimedia Foundation is to learn one thing from Google to improve its search, you can find the basics in the above image. When you consider it carefully, it is all about compartmentalising. What Google offers are distinct categories of results and, these categories exist in our projects as well.

  • The first tab would provide our current search results. No investment needed.
  • The second tab is images.. A massive amount of work is needed to make it useful but hey, we have it on Commons. So why not offer it. As people start to use it, we can tweak it, improve it 
  • The third tab is videos.. 
  • Maps is something we could do. We should do it in cooperation with OpenStreetMap and go from there
  • News.. why not, it will be a shot in the arm for Wikinews
  • We could have results for Wikidata.. It will be usable certainly when it is done in a Reaonator kind of way..
We do not need much funding to provide search in this way. We will do a better job at fulfilling our mission. What do you think?
Thanks,
       GerardM

Thursday, January 07, 2016

#Wikipedia - #Search - #Wikidata


Getting oodles of money to improve search is wonderful. A lot of good work will likely be done. It is also a great moment to consider existing questions that have not been resolved. It is all about having no result and how it might be improved giving the unique opportunities of the Wikimedia projects.

The first thing to consider is a favourite story of the first chairperson of the Dutch chapter. He volunteered at a library and people complained that they could not find what they were seeking for. He tracked what people could not find and realised that there is a people problem; they cannot spell. He aggregated the numbers and included a redirect for the really problematic words. People were happy. Librarians were happy because they now had a way of divining what books to buy on what topic; they knew what people were looking for and the library could not provide.

This is also possible for a Wikimedia search. When we know what people are looking for and cannot find, we can provide information anyway. The information on Mrs Boevink has been fleshed out but it only exists on Wikidata. When it has a Wikipedia article, it may not be in your language. It can already be provided as a search result but that means that "other" projects are taken seriously.

When a subject is not found at all, it means that people have an interest. It means that an article on such a subject will actually be read. This is a metric that people would enjoy; how well read are the articles that I started.  What impact do I have. It is only a thought but it will motivate people to write Wikipedia articles. This is particularly important to the small Wikipedias. When they write what people want to read, it will prove a process that positively affects the community.

So oodles of money have come in. Now lets be serious of finding Waldo.
Thanks,
        GerardM

Sunday, September 06, 2015

#Wikimedia - improving #search

One "key performance indicator" for search is the number of people who get zero results for a search. The objective is to make the number of people who do as small as possible. [1]

In the early days of the Dutch Wikipedia, a librarian was always happy to explain how he improved search results at his library.

His first observation was that he needed to know what people could not find. The observations were aggregated in timeslots. In this way he knew what people were looking for. His favourite observation was "People are stupid; they do not know how to spell". Allowing for the most prevalent spelling errors improved the results a lot. The other part was that people were looking for things the library did not provide.

The message for the Wikimedia search team.. Consider publishing the known errors aggregated over time. Have the community mark the spelling errors as such and use that to serve content anyway. The other part where we do not have data, consider that Wikidata has more information than any Wikipedia, when results do not exist as articles, publish what people seek and there might be a community out there adding missing articles.
Thanks,
       GerardM

Friday, April 24, 2015

#Wikidata -#UN Prize in the Field of Human Rights

The United Nations recognises organisations and people who have made a difference. The United Nations Prize in the Field of Human Rights shows many people and organisations that were awarded in this way.

At this time this list is a "work in progress" at Wikidata. The people known to the English Wikipedia are on it. It is now a matter of filling in the "red links" and removing a few erroneous entries. The only way to see this list is by using the Reasonator.

What is really interesting is that most of the missing people already exist on Wikidata. They just do not exist on the English Wikipedia, Adam Lopatka is one example. Completing a list like this is therefore an exercise of patiently checking every entry for existing items. For others, like the Vicaría de la Solidaridad the link has not been made.

The beauty of working on a special interest like human rights in Wikidata is, that you can highlight the people that matter. This does not mean that they are easily found. Amir is working on a bot that will do transliteration. That will be a boon particularly to find the people who are only known in a different script. Mehranguiz Manoutchehrian and Petr Emelyanovich Nedbailo are two examples that may have an article after all. The first is from Iran, the second from the USSR.

It is wonderful to see how the pieces are coming together for Wikidata.
Thanks,
     GerardM

Sunday, December 07, 2014

#Wikipedia- The numbers are what ?

So the numbers are flat.. You want initiatives that help provide us with relevance.. Ok, how about this scenario:

A reader queries Wikipedia for a subject and does not find it. Many more people query for this subject and it becomes the most wanted subject without an article. An editor writes the article and it proves popular. It becomes the most read new article in the next month.

In this way:
  • we give our search statistics a purpose
  • we indicate what subjects our readers want articles about
  • we celebrate the most read new articles and their editors
  • we advertise that we ask our editors to write articles people are looking for
  • we can do this for every Wikipedia in every language
Yes, we can provide search results from Wikidata as well. When people make use of this, it counts as a not found instance. In the mean time we did provide information that is available to us.
Thanks,
      GerardM

Sunday, November 30, 2014

#Wikimedia - #Wikidata; a recurring subject


The last Foundation Metrics meeting is kinda interesting, particularly when you are interested in #Wikidata. In his part of the meeting Erik considers the implications of Wikidata and wonders what it takes to help it lift off.

Erik wants us to change the world. Now that is a big statement. It can be done. It takes big thinking, maybe even bigger thinking or maybe no thinking at all. In the vision Erik presented, it is all about data and leveraging the data for instance in info-boxes.

Have a closer look at Reasonator, to its statistics and to these specific statistics (it takes a long time to load). What you find is probably the most easy and effective thing to do. It is allowing people to add labels in their own language. Labels leverage all Wikidata statements for a language. With those labels you can disambiguate effectively in a search. Try it in Reasonator. Now change the language and notice that effectively the automated descriptions are still there. Now do the same with search in Wikidata.. See ?

At Wikimania there was a heated discussion about the need for descriptions . The only half baked argument to keep the current descriptions was that people outside expect them. Lets not be strangers.
Thanks,
        GerardM

Friday, August 01, 2014

#Wikidata - #priorities

Wikidata is relevant because it is used. The primary use case that makes Wikidata relevant are the interwiki links. Wikidata gained relevance as more Wikimedia projects used Wikidata for its interwiki links.

There are ambitions to add more relevance to Wikidata. This is done by giving an application to the data that is included in Wikidata.

  • external sources are shown based on Wikidata information
  • automatically generated text for "humans" in English
  • automated descriptions
  • presentation of information in Reasonator
  • query in WDQ, Autolist and Autolist2.
  • automated descriptions are used in "Wikidata search" on many Wikipedias
Particularly for humans, this works rather well. This house of cards relies very much on one premise; that we know what a Wikidata item is about; is it a human, a house, a settlement or a record label?

For most of our items Given that we do not know what an item is about. For me it is a priority that as many items get identified for what they are as soon as possible. This makes it possible to make a statement that a person is a journalist, an engineer or whatever. 

There are issues with this approach. There is this potential for false positives and at the same time it provides us with possibilities as well. Once false positives have been recognised as such, they can be used to identify other items for what they are. Once the right identifiers have been set as well, it becomes easier to remove the wrong identifiers.

The biggest issue however is with our community. Not everyone shares the same priorities and insights and this is made worse because many people Wikidata only know the standard functionality. For them "Widar" is playing a game. They are not aware that the bulk of the edits are done with Widar or by bot and that the interface they use is inferior. These people have good intentions but they have no clue.

Add to this that communication channels do not really function and what we have is quite messy. 

It will not really improve until we share more of our priorities and listen to each others arguments and work together towards solutions. As it is there are too many perspectives and, conflicting priorities aplenty. Consequently there is plenty frustration shared by all.
Thanks,
      GerardM

Sunday, May 04, 2014

#Wikidata - May 7th #Elections in South Africa

South Africa will elect members to its National Assembly. Some of its members have a Wikipedia article but most do not. They are however "known" because of red links in a list and as red links in a template.

Adding them to Wikidata is easy and, by adding a distinct statement: "position held" "member of the National Assembly" everything is in place to select them using AutoList. Obvious statements to make are "is a" "human", "nationality" "South Africa" and "occupation" "politician".

The information shown for members of the National Assemblee looks like this in the Reasonator. As these members of the National Assemblee do not have an article, WD-Search enables access to this information. For all these new items there are only four statements requiring eight labels.

When the results of the elections are in, many of the incumbents will be re-elected. When the list of winners includes that fact, it will be easy to complement the list of members of the National Assembly with the latest crop of representatives of the South African people.
Thanks,
       GerardM

Tuesday, April 22, 2014

#Wikipedia - the search for ఈలా గాంధీ in Telugu

Mrs Ela Gandhi or ఈలా గాంధీ in Telugu does not have an article yet in the Telugu Wikipedia. This does not mean that she is not of interest to the people who seek information about her.

Now that the Telugu Wikipedia has added this one line in its common.js, it is at the bottom, people will be able to find her when they search for her. At the bottom you find the search results from Wikidata.

I think Mrs Gandhi already looks smashing in the Reasonator in Telugu :) What is missing is a label for one of political parties of South Africa.
Thanks,
     GerardM

Tuesday, April 01, 2014

#Wikipedia - searching for ദേവദാസ് ഗാന്ധി


This screen shot needs some explaining. Netha added the label in Malayalam for Devdas Gandhi. As a consequence everybody can find Mr Gandhi written in Malayalam on all the wikis that participate in the "extended Wikidata search". The screen shot demonstrates this for the Tamil Wikipedia.
Thanks,
      GerardM

Wednesday, March 12, 2014

#Wikidata search and automated descriptions

Reasonator uses #Wikidata wherever possible. There is no point in improving what is already good enough. Wikidata search is good enough so the results are used in Reasonator. It is used and the results are presented with automated descriptions.

To the right is a picture of the gate of Plötzensee prison. It was quite notorious during the Nazi regime; many people were executed by beheading or otherwise. There is a Plötzensee monument, a Plötzensee neighbourhood, a Plötzensee lake and, it helps when you can find the correct Plötzensee item when you are searching for it.

When you look at the lake, you will find that someone died there. Actually he died at the prison. When you look at the neighbourhood, you will find 185 people who probably all died at the prison. For the prison itself only 116 people are known to have died there. Several of these 116 have died at two locations.


The results of a search for Plötzensee need to be different in every language if the results are to be understood. It is not obvious to everyone that a "See" is a lake and, a "Denkmal" is a monument in English. With a location it is indicated that we know "it" is somewhere but it is not known to Wikidata what "it" is.

Thanks to automated descriptions it is possible to provide clarity in what the search results are. Obviously, when rich information is available in Wikidata, the results improve.
Thanks,
       GerardM