The Benevolent Side of Big Data

• Author: Christopher W. Surdak and Sara Agarwal
• Position: the author of Data Crush: How the Information Tidal Wave Is Driving New Business Opportunities and works in the field of electronic discovery and analytics, and is the Director for International Finance Organizations at Hewlett-Packard Company.

A farmer uses a simple mobile phone to obtain crop prices and subsequently finds a buyer who will pay the most: this is the most common example of how technology can spur social and economic advancement.Â

But today’s technology allows us to take that example up a notch. With a smartphone and simple apps, buyers can track how many farmers they see in a day, and farmers can keep tabs on how much they make in a year. With a year’s worth of collected information an app can let farmers know the best day and location to sell their goods. Applications link market demand with weather patterns so farmers know the best day to plant and harvest. Such analytic tools also help farmers understand precipitation and soil conditions and how these will affect their crops. They may also alert farmers to the spread of crop diseases and how to prevent them—with the click of a button. Through the use of big data, small-scale farmers can make better decisions that lead to even greater efficiency and profitability.Â

Big data has often been defined as the collection, analysis, and use of vast quantities of data. But existing tools already do that; what makes big data different is the ability to discover previously undetected relationships through large and sometimes unrelated sources. We can now ask new questions and get new answers. And we can do it via shared infrastructure (such as cloud computing), so almost anyone can use it. We can learn more about human behavior than ever before, and behavior is strongly linked to problem solving.Â

As with all technologies, big data does have a dark side. The collection of vast quantities of data can enhance people’s lives, but it can also be used to control, manipulate, and exploit. Most organizations use information with good intentions. Some do not. There is growing concern over how people are being influenced by data, and many governments are waking up to the need for regulation.Â

Realizing the promise of big data in developing economies requires a revolution both in the use of technology and its application. Only significant change on both fronts will allow big data to fully aid development. Governments, along with nongovernmental organizations (NGOs) and the donors and foundations that support them, have a critical role to play.Â

Big data is rapidly becoming an exercise in asking new and better questions. Each time you use your phone or post to a social media platform, you generate data that accumulate in a broader...