Release of the LiT.RL News Verification Browser Source Code: Detecting Clickbait, Satire, and Falsified News

On 6 December 2018 the Language & Information Technology Research Lab (LiT.RL) from Western University (Canada) released the source code for the LiT.RL News Verification (NV) Browser via LiT.RL GitHub Repository. The LiT.RL NV Browser is a recently developed proof of concept software that identifies clickbait, satirical news, or falsified news. The source code is available under the GPLv3 license for anyone to download (as a self-installing executable file), use and experiment with, or for developers to improve upon We do not collect any data from any users. The Browser is each user's own stand-alone version.

News Verification Browser Frequently Asked Questions (FAQ)

What is NV Browser? What does it do and who is it for?

The LiT.RL NV Browser is a research tool for news readers, journalists, editors or information professionals. The tool analyzes the language used in digital news web pages to determine if they are clickbait, satirical news, or falsified news, and visualizes the results by highlighting content in color-coded categories.

The Browser is under the GPLv3 license for anyone in the research and development community to use or improve on. The public can download the browser for experimentation on their own computers with “no strings attached”. Libraries are welcome to use it as part of their programming (such as media and information literacy).

Why do we need NV Browser-like tools?

Widespread adoption of internet technologies has changed the way that news is created and consumed. The current online news environment is one that incentivizes speed and spectacle in reporting at the cost of fact-checking and verification, encouraging the proliferation of misinformation and disinformation. The LiT.RL NV Browser is a system that offers a first step counter-measure by automatically detecting and highlighting fake news headlines and articles. The system is presented to users as a set of assistive technologies built into a stand-alone browser tuned to identify different varieties of fakes . Our algorithms look for patterns of subtle lexico-syntactic features in text. Images, audio and video formats are unsupported. The core of the functionality is in the natural language processing (NLP) of textual data and automated classification of results with machine learning (ML) using support vector machines (SVM).

What does NV Browser look like?

See Figure 1 below.

Figure 1.Screenshot of LiT.RL News Verification Browser clickbait detection on the CNN homepage (December 12, 2018)

What does Figure 1 mean or how can it be interpreted?

Figure 1 shows that on 12 December 2018 the CNN's home page was moderately clickbaiting, with 44% of the 147 headlines links exhibiting clickbait properties. Green bars on the right show a progression from not-clickbait to slightly, moderately and heavily clickbating headline totals.

What does NV Browser look like when it is running? Can you explain it?

See a 3-minute demo for an explanation of what you will see when you run the NV Browser Clickbait Detector functionality.

How do I install, update or delete the software?

As stated in our README.MD (text) file on https://github.com/litrl/litrl_code, here is how to install your version of the LiT.RL NV Browser executable, if you are willing to proceed.

Prerequisites to install:

  1. VC++ 2017 redistributable: https://aka.ms/vs/15/release/vc_redist.x64.exe
  2. .NET 4.6.2 (minimum) runtime: https://dotnet.microsoft.com/download/thank-you/net462

Then run the installer for the LiT.RL Browser. The installer package for modern Windows systems (7, 8.1, 10) is available under "Releases." on https://github.com/litrl/litrl_code. The installer takes a few minutes - please be patient! A lot of work is currently done with a batch script - the command prompt may be open for a few minutes - do not close it. The default install location is your desktop, where one folder and one shortcut will be created (other locations are untested).

To uninstall the software, simply delete the LiT.RL Browser shortcut and LITRL folder from your desktop. That's it!

What should you be cautious about with this software package?

It is NOT designed as a replacement for your day-to-day web browser and should not be used where security is critical.

The LiT.RL Browser should be used with caution as it is still highly experimental, may contain bugs and security issues, and was intended to be used as a tool for further research into deception on the internet and the effectiveness of deception detectors. The Falsifications Detector is still in very early alpha stages and is still being written - although it performs acceptably on test data, in practice it is not always effective. Use this feature with added caution.

Where can I find out more about the NV Browser?

You can read additional information about the Project on our Lab's webpage.
Please feel free to look through our publications.

The most recent NV Browser description is in:

Rubin, Victoria L. (2018). "La détection des nouvelles trompeuses et fallacieuses dans les contenus en ligne" [en en français] [or "Detecting Deceptive and Misleading News in Online Content," an English pre-print will be made available upon request]. In Sauvageau, Florian, Thibault, Simon et Trudel, Pierre (eds.), Les fausses nouvelles, nouveaux visages, nouveaux défis. Comment déterminer la valeur de l’information dans les sociétés démocratiques ? Québec : Presses de l’Université Laval.

A broader overview of the technologies related to deception detection is in:
Rubin, V. L. (2017) Deception Detection and Rumor Debunking for Social Media. [Abstract available; pre-print upon request.] Chapter 21. In The SAGE Handbook of Social Media Research Methods, Chapter: Deception Detection and Rumor Debunking for Social Media , Publisher: Sage, Editors: Luke Sloan, Anabel Quan-Haase, pp.342-364 Library of Congress Control Number: 2016940832, ISBN 978-1-4739-1632-6

Are there any 'fake news' training datasets available (for NLP/ML developers interested in improvements)?

Yes, please see our Data-To-Go section of the Project's website.

How good is NV Browser's functionality? How well does it perform?

The Browser is an experimental tool. It identifies clickbait, satire, falsified news, with varying success rates (accuracies in 90s%, mid-80s%, and mid-70s%, respectfully). It has certain limitations that everyone should be aware of.

This browser is meant to augment our human discernment, rather than replace it, by highlighting potentially false information which may require further scrutiny. Digital literacy is key for everyone to effectively evaluate potential misinformation online, and the LiT.RL Browser is NOT a replacement for that. News readers’ critical thinking remains key to navigating the increasingly fraught online information landscape. The News Verification Browser is not a replacement for a day-to-day web browser and should not be used where security is critical.

Why was News Verification packaged as a browser, not a plugin?

Various plugins have already been attempted in this area. We decided to develop a separate research tool for deception with a simple user interface that allowed for easier real-world testing of our group's previous work.

Who funded this R&D effort?

This research has been funded by the Government of Canada Social Sciences and Humanities Research Council (SSHRC) Insight Grant (#435-2015-0065) awarded to Dr. Victoria L. Rubin for the 2015-2018 project entitled “Digital Deception Detection: Identifying Deliberate Misinformation in Online News”.

What is NV Browser's commercial status? Is it for sale?

There are no plans to commercialize this product. It is developed in the academic community. It is meant for the public.

Who worked on the NV Browser?

A dedicated team from LiT.RL has worked to accomplish this proof of concept implementation. Please see names listed in the citation below.

How should the NV Browser code be cited?

Suggested citation for the source code:

Rubin, Victoria L.; Brogly, Chris; Conroy, Nadia; Chen, Yimin; Cornwell, Sarah E.; Asubiaro, Toluwase V. (2018). litrl/litrl_code: LiT.RL Browser Experimental 0.12.0.0 Public (Version exp-0.12.0.0). Zenodo. 10.5281/zenodo.2016627.

The code is currently under peer-review in the The Journal of Open Source Software (JOSS) under the title "A News Verification Browser for the Detection of Clickbait, Satire, and Falsified News."

Where do I write, if I have comments or suggestions?

For feedback on the NV Browser functionality, please contact us via: litrlbrowser@gmail.com

What is in the future for NV Browser?

In 2019 we will be looking for usability testing participants to help us improve the program's ease of use and performance. Stay tuned. We will also be experimenting with its effectiveness in media/information literacy programming in educational settings.

 

 

 

 

1 Comment on “Release of the LiT.RL News Verification Browser Source Code: Detecting Clickbait, Satire, and Falsified News

Leave a Reply

Your email address will not be published. Required fields are marked *