III: Small: Collaborative Research: Towards End-to-End Computer-Assisted Fact-Checking

Duke University, Durham NC

Investigators

Abstract

This project will develop ClaimBuster, an end-to-end system for computer-assisted fact-checking. This system will monitor live discourses, social media, and news to catch factual claims, detect matches with a curated repository of fact-checks from professionals, and deliver the matches instantly to readers and viewers. For various types of new claims not checked before, ClaimBuster will automatically check them against knowledge databases and report if they are truthful. For novel claims where humans must be brought into the loop, the system will provide algorithmic and computational tools to assist laypersons and professionals in understanding and vetting the claims. ClaimBuster, upon completion of the proposed work, is positioned to become the first-ever automated fact-checking system for use on a broad spectrum of factual claims. Its use will be expanded to verify claims in various types of narratives, discourses and documents such as sports news, legal documents, and financial reports. It can benefit a large base of potential users including consumers, publishers, corporate competitors, and legal professionals, among others. It directly benefits consumers by improving information accuracy and transparency. It helps news organizations speed their fact-checking process and also ensure the accuracy of their own news stories. Businesses can use ClaimBuster to identify falsehoods in their competitors' and their own reports and press releases. It also assists professionals such as lawyers in verifying documents. ClaimBuster will use database query, data mining, and natural language processing techniques to aid fact-checking. The detailed research tasks in this project will be as follows. (1) Investigate how to model factual claims and produce their internal representations. For this, the team will create taxonomies of claim templates in different domains, categorize claims based on the taxonomies, and generate internal representations through semantic parsing of the claims' textual forms. Such domain-specific modeling and internal representation of claims will enable novel methods and systematic, coherent solutions for other components of the system. (2) For algorithmic fact-checking, they will devise novel methods for translating claims into structured queries, keyword queries and natural language questions. Results of these queries over general and domain-specific databases and knowledge graphs will be compared with the answers embedded in the claims themselves, to verify if the claims check out. (3) By viewing claims as parameterized queries, they will develop methods based on perturbation analysis to find counter-arguments to claims and to find "interesting" factlets from datasets. These results will help ClaimBuster in identifying "cherry-picking" claims -- claims that are correct but misleading.