Data-driven approaches have led to powerful prediction, optimization and automation techniques. Powered by large-scale, networked computer systems and machine learning algorithms, these have been very impactful to-date and hold great promise in many disciplines, even in the humanities and social sciences. However, no new technology arrives without complications, and we have recently seen the press and various political circles illustrating real, potential, and fictional implications of Big Data.
This presentation aims to balance the opportunities provided by Big Data and its associated artificial intelligence techniques with a discussion of the various challenges that have ensued. I review eleven types of challenges, including those which are technical (resilience and complexity), societal (difficulties in setting objective functions or understanding causation), and humanist (issues relating to free will or privacy). I provide example problems and suggest ways to address some of the unanticipated consequences of Big Data.
Dr. Alfred Spector is Chief Technology Officer at Two Sigma, a firm dedicated to using information to undertake many forms of economic optimization. His career has led him from innovation in large scale, networked computing systems (as a professor at CMU and founder of his company, Transarc) to broad research leadership: five years leading IBM Software Research and eight years leading Google Research. Recently, Spector has lectured widely on the growing importance of computer science across all disciplines (CS+X) and on the Societal Implications of Data Science. He received an AB in Applied Mathematics from Harvard and a Ph.D. in Computer Science from Stanford, where he was a Hertz Fellow. He is a Fellow of the ACM and IEEE, and a member of the National Academy of Engineering and the American Academy of Arts and Sciences. Dr. Spector won the 2001 IEEE Kanai Award for Distributed Computing, was co-awarded the 2016 ACM Software Systems Award, and was a 2018-19 Phi Beta Kappa Scholar.