#Wikidata - Harriet Martineau and some social opportunities

When you do not already know about Mrs Martineau, do read one of the many Wikipedia articles, she is considered to be the first female sociologist and introduced many subjects into sociology that were up to that time not considered.

The picture is a crop of a painting at the National Portrait Gallery by Richard Evans. The picture is known at Wikidata, at Commons the Creator template is missing.

At the Biodiversity Heritage Library Mrs Martineau was know for her book a complete guide to the English lakes. It was the only book known for her at Open Library.  Given the relevance of Mrs Martineau this was strange and sure enough she was known as "Martineau, Harriet" and changing the link to the book was easily done.

At Wikidata meanwhile, there was a hidden link to Mrs Martineau to Open Library thanks to all the good work of the Freebase volunteers. Approving the change was obvious.

At Wikidata there is now a link to both VIAF, to the BHL, to OL for Mrs Martineau and to over 20 more sources. The BHL has links to both Open Library and VIAF. When the links differ, it becomes obvious where work needs to be done.

The result is a better service for all the people who make use of any or all of these resources. We truly should collaborate and strengthen our partners, the partners we share data with.
Thanks,
      GerardM

#Wikidata - #Freebase atrophies in the Primary sources status

It is a good thing that there are statistics for the primary sources status. It demonstrates clearly how dysfunctional it is. Only 18K statements have been approved. After all the time that the tool exists, it is not even one percent.

For a "serious" power user it is quite possible to do add this number of statements in a day to Wikidata any day. The sad thing is there is every reason to believe that the quality of a power user is just as good as anything that is in this dump in the first place.

Mathematics show that it is easy to check and verify the data that is in Wikidata with other sources. When such a process is well designed, it is iterative and consequently adding data that is deemed useful for inclusion in Wikidata will be processed in every iteration.

These sad statistics demonstrate one thing and one thing only; the failure that is in this approach. It would be wise to abandon it and concentrate on workflows instead that leverage the value that is in the huge community that may serve fixing issues.
Thanks,
       GerardM

#Wikidata's embarrassment of riches

Wikidata is improving its content constantly. Proof may be found in people pointing to issues and the follow up it generates. They add data, change data and remove data; Wikidata is better for it.

With the official Wikidata Query being live, it is even easier for people who understand SPARQL to query, compare and comment on Wikidata's content. As mentioned before, it is in the comparison of data that it is easiest to improve both quality and quantity.

For this reason it is an embarrassment how a rich resource that is Freebase is treated; it might as well not exist. It lingers in the "primary sources tool" a lot of well intentioned work is done. In Q3/2015 there may even be a workflow to include even more data in there.

Probably, this tool is only relevant for static data and, that is not necessarily the best. Actively maintained data is much to be preferred.  When I understand things well, people may tinker with it in this data dungeon and it is then for the "community" to decide upon inclusion in Wikidata. It is not obvious what its arguments could be. It is not even obvious how any data will compare to the quality of Wikidata itself. Its quality is not quantified for quality either.

Once data is included, there are many ways to curate the data. It is done by comparing it against other sources. It is obviously a wiki way because it invites people to collaborate.
Thanks,
      GerardM

#Wikidata - collaborating on data #sources II

#Freebase had a vested interest in the MacRobert Award. Its recent laureate is Artemis Intelligent Power,  As Wikidata continues to update its information, Freebase will become increasingly incomplete and as a consequence of less relevance to the people who use its data at :BaseKB.

When you do not know about the MacRobert Award, read about it. When this is the first time you hear about Artemis Intelligent Power, read the BBC article about it.. News like this gives us hope for our future.

We can support :BaseKB with our data, we can support any other source who is interested in what Wikidata has to say. It all starts with acknowledging them as sources and it all starts that quality has everything to do with it.

It is all in being open to quality. It is all to being open to reevaluating quality.
Thanks,
      GerardM

#Wikidata - collaborating on data #sources

x

#Freebase's data found a home at :BaseKB and that is a great opener to an alternate approach to the Freebase data for Wikidata. This proposal has many parts that makes for great cooperation between multiple sources.

• items always have a link to its source(s)
• statements always state its source
• an indicator for the status of a statement is added

Linking to external sources is something Wikidata does a lot. It allows us to have a look at the source itself. When it is the origin of the data, it follows that the statements from that origin have to be exactly the same. When they are not, it should be indicated with a status.

When a statement differs from a source, we have identified something that needs work. In essence. such statuses have a function in a workflow as well. Because this is where intervention makes a difference. 

• we set the flag for people to investigate the difference
• we can flag the source itself that we found an issue 
• we can find alternate sources to find what is likely correct
• we can find corroboration in literature
• we change the statement when needed and set a flag to indicate the status

In this way, we spend our human capital wisely. We do not blindly spend time on "approving" any and all statements. We do it only where we know some research is needed. 

As far as I am aware, you can not copyright facts. We compare our facts against facts known elsewhere. We include our missing data where we may and we always investigate differences. We signal the work we have done and, in this way we not only improve our quality, we also provide a path to the sources we work with for them to work on their quality.

In this way we spend the time and effort of our community wisely and, we optimise the amount of information available in Wikidata. In this way everyone is a winner.

Thanks,

       GerardM

#Wikidata - the failure of the "primary sources tool"

I add data all the time. My tools do not see data that is waiting to be curated.  So I add data.

I find I am actually wasting my time.
Thanks,
      GerardM

#Wikidata - failing the #Freebase community

#Google in its infinite wisdom assumed that as Wikidata has a bigger and growing community, it will go beyond what Freebase could do. It had the power to end Freebase and it did in the expectation that the two communities would work together. An assumption that is quite reasonable.

When you work together, you share resources and you have respect for each others achievements. Wikidata may have the bigger community but Freebase has the most data and for whatever reasons the data from Freebase was not accepted at Wikidata.

The consequence is that the people who rely on this data now lost their source. It was lost because the service at Freebase was discontinued. It is therefore great news that the last Freebase dump has a new home at :BaseKB.

Some of the Freebase data is waiting for "approval" hidden in Wikidata. It is part of the Primary sources tool experiment and if you care to "curate" this data, you have to first enable a gadget. When this is not done, it is likely that the same data will be added because there is no other way to learn that the data is waiting to be approved. To add insult to injury, when data is added in this way, it does not report Freebase as the source..

This would not have happened when Wikidata was true  to the Wikimedia mantra of "sharing in the sum of all knowledge". I urge the powers that be at Wikidata to consider the following remedial steps:

• add Freebase as the source of data imported from Freebase
• report on the process of "curating" Freebase data
• when a link between Wikidata and Freebase is known, add missing statements
• add data from Freebase where there is none at Wikidata, data can be merged later when need be
• consider tools that allow for the selection of data based on its source for further curation and conversion
• invest in tools for people working on the inclusion of data from the "Primary sources tool" and from other sources
• compare data from sources and report on where there are differences. This is where time of our community is well spend
• consider a tool where the statements of Wikidata are easily available to Wikipedia editors. It may help them with additional information and it will help us in them curating this data

Most important, Wikidata is of particular importance because it links articles from Wikimedia projects. This is where it shines. The quality of its statements is debatable but improving. By including the lovingly maintained data from Freebase, we may expand our community and we do expand both the quantity and quality of the statements. 

Finally Wikipedians love their sources and they are important. When Wikipedians can easily compare the information that is in statements and in the text, we will find them more involved when there is more to curate. Given that Wikipedians get involved in this way, the sources are implicitly known through Wikipedia. This is yet another argument to make haste by including missing data, available from Freebase and other sources.

Thanks,

      GerardM

#Wikidata - appeasing the #Wikipedia crowd

To understand #quality, you have to make assumptions. For Wikipedians, quality is in the source. This source is a statement somewhere that a given fact is true. This makes sense in the text based Wikipedia.

One problem Wikidata faces is that many Wikipedians are sceptical about the quality of the information at Wikidata. "There are no sources" is the oft repeated mantra. Who checked the fact, who approved... Why should we trust the information? This resulted in a culture at Wikidata where "because of the Wikipedians" things that should be obvious are no longer obvious. When the Freebase data is imported, it is not imported it is kept in purgatory, waiting for someone to say OK.

The consequence is an epic fail when you use tools to add information. Data that is in purgatory is not seen by the tools and consequently it is added again, without the sources and without the additional qualifiers known in purgatory.

Google and the Freebase community spend an inordinate amount of time and effort on making this information right. It was used by Google in its products and WHY do we need to doubt its quality? Why should we think ourselves superior just because of the uneasy sentiments of Wikipedians?

The results are an epic fail because;

• the newly added information is not seen in tools
• there is no way to automatically compare it with other sources and thereby verify it
• it demonstrates mistrust where it is not needed.

When we want to import data from external sources, it makes sense to verify its quality. However, it is best done at the gate, before information is added to Wikidata. When a source is of such a quality that we want it, we should import it. Just like we did for all the settlements in China, Just like we are doing all the time from incomplete data from many Wikipedias.

We should do better and when we forget about Wikipedians for a moment, we know better.

Thanks,

      GerardM

#Google - Would you use your #trickery for us?

After the shock and awe of yesterday's announcement, There has been time to think. Google showed a real interest in Wikidata. It created a new tool to help improve the quality of its data. But the real expertise of Google is in determining the probability of facts. It is part and parcel of its ranking algorithms.

It would be as awesome when Google would indicate those statements it deems to have a less than even chance of being true. The combination of such a list and the new tool would make the efforts of the people seeking sources all the more relevant. When statements are debunked, it has a potential quality effect to all the associated Wikimedia projects. Given that it is probable that most statements are fine, it makes for more concentrated effort and consequently its effects will be noticed.

While we are on this line of thought, given the data of Freebase, Google could indicate based on its algorithms how probable its sets of data are. Everything that is highly likely should be a candidate for import in Wikidata. The other reason for importing data into Wikidata anyway is that it is an invitation to all the Freebasers to join our ranks, increase our expertise and together be awesome.
Thanks,
      GerardM

#Google - What it does with #Freebase is beyond awesome

In an e-mail Denny Vrandečić announces an astounding bit of news. It effectively says that Wikidata can have all its data if it wants it.

It then goes on saying that it is not expected for Wikidata to accept all this data and follows with an announcement for a tool that is to source data. This news is best read as it was announced..
Thank you Google!
        GerardM

Freebase was launched to be a “Wikipedia for structured data”, because in 2007 there was no such project. But now we do have Wikidata, and Wikidata and its community is developing very fast. Today, the goals of Freebase might be better served by supporting Wikidata [1]. 

Freebase has seen a huge amount of effort go into it since it went public in 2007. It makes a lot of sense to make the results of this work available to Wikidata. But knowing Wikidata and its community a bit, it is obvious that we can not and should not simply upload Freebase data to Wikidata: Wikidata would prefer the data to be referenced to external, primary sources. 

In order to do so, Google will soon start to work on an Open Source tool which will run on Wikimedia labs and which will allow Wikidata contributors to find references for a statement and then upload the statement and the reference to Wikidata. We will release several sets of Freebase data ready for consumption by this tool under a CC0 license. This tool should also work for statements already in Wikidata without sufficient references, or for other datasets, like DBpedia and other machine extraction efforts, etc. To make sure we get it right, we invite you to participate in the design and development of this tool here: 

 https://www.wikidata.org/wiki/Wikidata:Primary sources tool 

I hope you are as excited as I am about this project, and I hope that you will join me in making this a reality. I am looking forward to your contributions!  

[1] https://plus.sandbox.google.com/109936836907132434202/posts/bu3z2wVqcQc

Denny Vrandečić via lists.wikimedia.org

Linking to #external #sources

Recently references to #Freebase have been added to #Wikidata. When you look at the #statistics, a significant number of references have been added. It the last period 595,778 statements were added that refer to other sources.

When you consider that there are over 14 million items in Wikidata, many more references are to be expected from Freebase. Then again, making the link to other resources is not trivial. As Wikidata gains more statements for its items it will become possible to make those links.

Many of the items that are linked to Freebase, are linked to other sources as well. What I wonder is to what extend these other sources have their own links to other sources. Combining all these links in Wikidata will theoretically make it possible to find information that is currently missing in Wikidata.

Many of these sources have a known error rate. One area in Wikidata were a realistic error rate can be determined is in the interwiki links. The rest is incomplete and immature. We do know that a lot of work needs to be done on the interwiki links; many issues exist that have not been resolved.

Given the lack of quality and quantiy of the existing Wikidata data, importing data from external sources makes sense. Comparing the data with all the external sources makes sense. With more data it becomes easier to infer knowledge from Wikidata, it will become easier to compare with external sources but more important, Wikidata will be better able to provide information to the projects that rely on it.
Thanks,
      GerardM

#Wikidata & #Freebase - an #interview with Denny Vrandečić

When Denny signalled the availability of data that links Wikidata to Freebase, there was a lot of follow up and several questions were left unanswered.  So I asked my questions and, I am happy with the answers I received. So enjoy this interview about the Wikidata and Freebase connection. 
Thanks,
       GerardM

By moving to California you embrace yet another culture, language.What does it do to you, your family

My wife is from Uzbekistan, I am from Croatia. We met in Israel. She has lived the last few years in Estonia, I in Germany. We have moved to California, and we are very excited about this move. Finally to live in a country where we both speak the language! And also, we have high hopes with regards to the weather.
The US in general, and San Francisco in particular, is a real melting pot - or rather fruit salad, as they say these days - of people from all over the world. This seems to be a good match to our own background, so we are looking forward to see what this will mean for us.

What is your job description and title at Google (and what is it you actually do)

My job title is "Ontologist". I am working on Google's Knowledge Graph. The Knowledge Graph for Google is basically what Wikidata is for the Wikimedia projects: a repository of structured knowledge about the world, that is used in many different ways in various different applications. I am working, together with a great team, on the schema and ontology of the Knowledge Graph: its data model, schema, and the way it captures and represents knowledge.

What is Freebase and how does it compare with Wikidata

Freebase is the publicly available and editable part of the Knowledge Graph. Freebase and Wikidata are very similar. At the same time, there are quite a few differences: the user communities, the incentive architecture, the license, the way sources and knowledge diversity are handled, to name the big differences. There are also minor differences in the data model, the way classes, properties, types interact with each other, the UI, the workflows, the prominence of internationalization, currently also the size and scope of the knowledge bases, etc. Wikidata currently has 22 Million statements, Freebase has 2 Billion facts. Now one should always be careful with such simple metrics, and especially these numbers are not comparable, but it hints at some differences in the knowledge base.

Do you work on FB or is this only an outward facing project

Freebase is a part of the Knowledge Graph, and I work on the Knowledge Graph, so yeah, I also work on Freebase.

With a link between FB and WD it will be possible data can flow from WD because of its license. How is it for a flow from the other direction

I am not a lawyer, but unfortunately it seems that the licenses used by Freebase and Wikidata would not allow for that direction. We are aware that this is not good, and we are working on ways to fix this.
But even without actually letting data flow from Freebase to Wikidata, such an alignment can be valuable for the Wikidata editors: bots could compare the data, and flag inconsistencies. Bots could do simple comparisons, like, go through all the countries, compare the capitals, and report on the differences. And this can be done not only with Freebase, but with any other structured or semi-structured base that Wikidata connects to. One example is the work that Maximilian Klein did, comparing the sex/gender of authors connected through the VIAF ID.

The data for the FB<->WD connection is based on a WD dump, How will it be maintained in the future 

This is not decided yet. In principle, we could create this dump regularly, but I am unsure if this will be needed. We will have to see how things develop before deciding on what to do next.

When info exist in both WD and FB and it differs, what would you like to see done

The difference to be fixed, obviously! Both knowledge bases may and do contain errors, and the ability to compare them can lead to an increased quality level in both. It is obvious that the respective communities should take care of such errors in their knowledge bases. By providing links between the knowledge bases, this should get easier to automate.

Would that be a model for collaboration with other sources like VIAF

Yes, in many cases that should work. It is clear that some sources and their communities are more open to corrections than others. Wikidata can become a central hub for identity on the Web, collecting and reconciling IDs from many different sources. What makes Wikidata so interesting in itself is its commitment to let everyone share in the sum of all knowledge - both in participating in creating it as well as in accessing it. The barriers are so much lower than almost anywhere else. In which other knowledge base can you easily fix an error?

Does FB have sources for its statements

Yes, but this is quite a different notion from what Wikidata does with sources. When a dataset is being uploaded to Freebase, the source for this upload is usually recorded. It is closer to what is usually called "provenance", whereas Wikidata's notion of a source is closer to what is called a "reference", an external authority that makes the given claim. A strength of Wikidata is the diversity of the sources, and that they can be refined later by the community. A statement in Wikidata can have several sources (which makes sense if you think of them as references supporting the statement), but in Freebase every statement only has one source (which again makes sense if you think about it being the provenance of that statement, from where this statement came from).

spot the differences

What do you prefer, no diffs or sourced diffs

(I don't understand the question, and rephrase it to: do you prefer an unsourced statement over no statement?)
In most cases, yes, I'd rather have an unsourced statement than no statement at all. In a perfect world, all statements in Wikidata would have great, authoritative sources. But for now, I really think that Wikidata should be lenient with regards to unsourced statements in most cases. There are obvious cases where this is not true: data about living persons has to be more carefully sourced, especially when it has the opportunity to hurt the person. Also, we have to remember, that a Wikipedia community can decide to use a statement from Wikidata only if it is sourced, and to drop it otherwise. Once Wikidata has matured a bit, I expect it to move towards a more stricter policy, and to see tools develop that help with getting there. That will be a quite exciting time.

What FB feature would you LOVE to have in Wikidata

The expressive query features of Freebase. The Wikidata team is hard on working towards enabling this, and I am very much looking forward to see it happen: this will open a whole new world of possibilities for everyone using Wikidata's data, and it will lead to much more visibility of the data and help to refine the Wikidata properties.

Battle of the sexes, the #Wikidata story III

#Freebase is a rich resource. It boasts to have 39,296,842 topics and much of it is linked to #Wikipedia. Freebase has been around for a considerable amount of time and, there are great applications that make use of the data.

One of these applications informs you if a name is typically male or female. I would not expect the name Roxy to be used by men but it is.

The information for many of the people involved is linked to Wikipedia. Consequently it is relatively easy about the sex of someone like Roxy Walters and update Wikidata. At this time no statements have been made for Roxy Walters.

The only potential problem is that Freebase is using the CC-by-sa license and that makes it incompatible with Wikidata. Given that you cannot license facts, Google may decide either to change the license for Freebase or ignore when Freebase is used to update Wikidata.
Thanks,
       GerardM