Dec 18, 2016
Reimagining Our World at Planetary Scale: The Big Data Future of Our Libraries
What happens when massive computing power brings together an ever-growing cross-section of the world’s information in realtime, from news media to social media, books to academic literature, the world’s libraries to the web itself, machine translates all of that material as it arrives, and applies a vast array of algorithms to identify the events and emotions, actors and narratives and their myriad connections that define the planet to create a living silicon replica of global society? The GDELT Project (gdeltproject.org/), supported by Google Zigsaw, is the largest open data initiative in the world focusing on cataloging and modeling global human society, offering a first glimpse at what this emerging “big data” understanding of society looks like. Operating the world’s largest open deployments of streaming machine translation, sentiment analysis, geocoding, image analysis and event identification, coupled with perhaps the world’s largest program to catalog local media, the GDELT Project monitors worldwide news media, emphasizing small local outlets, live machine translating all coverage it monitors in 65 languages, flagging mentions of people and organizations, cataloging relevant imagery, video, and social posts, converting textual mentions of location to mappable geographic coordinates, identifying millions of themes and thousands of emotions, extracting over 300 categories of physical events, collaborating with the Internet Archive to preserve online news and making all of this available in a free open data firehose of human society. This is coupled with a massive socio-cultural contextualization dataset codified from more than 21 billion words of academic literature spanning most unclassified US Government publications, the open web, and more than 2,200 journals representing the majority of humanities and social sciences research on Africa and the Middle East over the last half century. The world’s largest open deep learning image cataloging initiative, totaling more than 150 million images, inventories the world’s news imagery in realtime, identifying the objects, activities, locations, words and emotions defining the world’s myriad visual narratives and allowing them for the first time to be explored alongside traditional textual narratives. Used by governments, NGOs, scholars, journalists, and ordinary citizens across the world to identify breaking situations, map evolving conflicts, model the undercurrents of unrest, explore the flow of ideas and narratives across borders, and even forecast future unrest, the GDELT Project constructs a realtime global catalog of behavior and beliefs across every country, connecting the world’s information into a single massive ever-evolving realtime network capturing what's happening around the world, what its context is and who's involved, and how the world is feeling about it, every single day. Here’s what it looks like to conduct data analytics at a truly planetary scale and the incredible new insights we gain about the daily heartbeat of our global world and what we can learn about the role of libraries in our big data future.
One of Foreign Policy Magazine's Top 100 Global Thinkers of 2013, Kalev is a Senior Fellow at the George Washington University Center for Cyber & Homeland Security and a member of its Counterterrorism and Intelligence Task Force, as well as being a 2015-2016 Google Developer Expert for Google Cloud Platform. From 2013-2014 he was the Yahoo! Fellow in Residence of International Values, Communications Technology & the Global Internet at Georgetown University's Edmund A. Walsh School of Foreign Service, where he was also an Adjunct Assistant Professor, as well as a Council Member of the World Economic Forum's Global Agenda Council on the Future of Government. His work has been profiled in Nature, the New York Times, The Economist, BBC, Discovery Channel and the presses of more than 100 nations, while he has been an invited speaker throughout the globe, from the United Nations to the Library of Congress, Harvard to Stanford, Sydney to Singapore. In 2011 The Economist selected his Culturomics 2.0 study as one of just five science discoveries deemed the most significant developments of 2011. Kalev’s work focuses on how innovative applications of the world's largest datasets, computing platforms, algorithms and mind-sets can reimagine the way we understand and interact with our global world. More on his latest projects can be found on his website at http://www.kalevleetaru.com/ or http://blog.gdeltproject.org