乐播传媒 College has received a National Science Foundation grant of $3.97 million to create a groundbreaking Visual Experience Database to support research in fields that rely on the analysis and recognition of images, such as neuroscience, cognitive science, and artificial intelligence.
Assistant Professor of Neuroscience Michelle Greene is the principal investigator for the project to create a Visual Experience Database (Theophil Syslo/乐播传媒 College)
The largest-ever federal grant awarded to 乐播传媒, the four-year award will fuel the creation of a vast gallery of videos that depict what, and how, people see as they go about daily activities. 乐播传媒 developed the grant proposal collaboratively with researchers at North Dakota State University and the University of Nevada, Reno.
Michelle R. Greene, an assistant professor of neuroscience at 乐播传媒 who studies how the brain makes sense of what we see, is the principal investigator for the project.
鈥淚鈥檓 delighted and overwhelmed,鈥� said Greene, 鈥渁nd intensely excited. I have a terrific team of co-principal investigators, so fostering and furthering those connections will make these next four years really fun.鈥�
The co-principal investigators are Benjamin Balas, a neuroscientist and associate professor of psychology at North Dakota, and Paul MacNeilage and Mark Lescroart, neuroscientists and assistant professors of psychology at Nevada.
鈥淲e are honored to be the lead partner in this multi-state collaboration,鈥� said 乐播传媒 President Clayton Spencer. 鈥淭his grant is important for Maine, Nevada, and North Dakota, and it also has the potential for significant impact on the future of vision research, neuroscience, and artificial intelligence.鈥�
鈥淚t鈥檚 meaningful that the National Science Foundation has chosen a national liberal arts college like 乐播传媒 to take the lead on this project.”
鈥淭his grant brings deserved recognition to Professor Greene, an exemplary member of the 乐播传媒 faculty who has contributed significantly to the body of published work on visual perception and who engages students extensively in her research,鈥� said Malcolm Hill, vice president for academic affairs and dean of the faculty.
鈥淚t鈥檚 meaningful that the National Science Foundation has chosen a national liberal arts college like 乐播传媒 to take the lead on this project,” Hill added. “Indeed, 乐播传媒 is well-positioned to collaborate on the work to create a Visual Experience Database to support researchers around the world as they grapple with, and bring greater understanding to, the social and ethical consequences of computer vision and artificial intelligence.鈥�
Student researcher Hanna De Bruyn 鈥�18 (left) works with Assistant Professor of Neuroscience Michelle Greene to prepare an EEG test in March 2018. (Phyllis Graber Jensen/乐播传媒 College)
鈥淢aine鈥檚 colleges and universities are consistently at the forefront of groundbreaking research that improves people鈥檚 lives and enhances our understanding of the world around us,鈥� said U.S. Sens. Susan Collins and Angus King of Maine in a joint statement.
They added, 鈥淭hrough this funding, 乐播传媒 College will partner with two other universities to build a database to study human behavior and development through first-person experiences. We applaud the NSF鈥檚 investment in 乐播传媒鈥� project, which will help advance the field of vision science.鈥�
The VED will comprise more than 240 hours of video created specifically for this project and findable through a publicly accessible database. Wearing cameras that simulate human vision, as well as devices to track head and eye movements, observers will undertake routine activities such as walking, shopping, or touring a museum.
Because they were not intended for research purposes, existing still and moving images are compromised by the many biases their creators bring to them.
By enlisting diverse observers local to each of the three participating institutions, the project will record how changes in environment, age, and task affect the act of looking.
Much of the data used in such fields as visual neuroscience, psychology, computer vision (a branch of artificial intelligence), and computational sociology consists of vast collections of still and moving images. These are curated largely from public online resources such as YouTube and Google 鈥� but because they were not intended for research purposes, they are compromised by the many biases their creators bring to them.
Artificial intelligence systems have biases because they’re not being fed enough solid data, says Michelle Greene.
The reasons someone may choose a particular photo subject, frame an image a certain way, or upload one image and not another are all biases that diminish the material鈥檚 value as data. Such 鈥渂iases exist at every level,鈥� said Greene, 鈥渁nd all of the databases that we鈥檝e been using for years are subject to them.鈥� The VED assets, in contrast, will be created specifically to represent ordinary scenes and will be subject to experimental controls.
Undergraduates at all three schools, including 28 at 乐播传媒 over the four-year grant period, will take part in the research. Among other roles, students will serve as videographers, creating assets for the VED, Greene said.
鈥淚 can imagine that next summer there鈥檚 going to be a small army of folks going out into various parts of the world, seeing what the world looks like when we’re hiking, when we’re at the beach, when we’re grocery shopping, and all kinds of more mundane things.鈥�
The project to create a Visual Experience Database will benefit 乐播传媒 students “at all levels,” says Michelle Greene, an assistant professor of neuroscience at 乐播传媒 who is the principal investigator for the project.
Students will benefit from the many research questions that the VED will engender. 鈥淎 database like this essentially means there will be thesis projects for decades to come,鈥� said Greene. 鈥淭here are many basic questions that we haven鈥檛 been able to answer because we haven’t had the data.鈥�
She said that 乐播传媒’ being a liberal arts college will enrich the VED project in distinctive ways. 鈥淥ne is our focus on equity and inclusion. We鈥檙e trying to get a diverse set of visual experiences to catalog in the VED, and I think that holding that in the forefront is something a liberal arts college can do that might be a somewhat harder sell鈥� at other types of institutions.
Innovation in artificial intelligence, in particular, stands to benefit from the VED.
乐播传媒鈥� intimate scale, coupled with the liberal arts approach to education, 鈥渁llows us to engage in some multidisciplinary and interdisciplinary thinking,鈥� she added. 鈥淎nd one of the things I particularly love about 乐播传媒 is that I engage in conversations with faculty members across the college 鈥� if I were at a larger institution, we probably wouldn’t touch paths and learn from one another.
鈥淪o I’m particularly excited about the ways in which this type of data can be used across disciplines.鈥�
Innovation in artificial intelligence, in particular, stands to benefit from the VED. Model systems in computer vision 鈥渁re very data-hungry,鈥� said Greene. 鈥淭hey tend to require tens of millions of images, and have been downloading these tens of millions of images from the internet. We will now give them tens of millions of images that are more representative of daily-life experience.鈥�
The VED will be a public resource. 鈥淭his is taxpayer-supported,鈥� Greene said, which means that it should be publicly accessible, both for the sake of transparency and simply because it should be a public good.
She added, 鈥淲e see throughout digital life that when a resource is available, people appropriate it in really interesting ways. I’m hoping that there may be artists, digital historians, computational sociologists that might want to use this database. And, as such, it should be available to everybody.鈥�
Michelle Greene explains why it is important and valuable for society that the Visual Experience Database will be publicly available.
The VED grant was made through the NSF鈥檚 Research Infrastructure Improvement Program, part of the Experimental Program to Stimulate Competitive Research. These initiatives are designed to build research capabilities in underserved regions of the country and thereby make those regions more competitive in seeking other federal R&D funding.
The project team will release a suite of software tools for using the database. The team will also establish a program of 鈥淏ig Data Skills Summer Workshops鈥� to give students basic programming and computational literacy skills that will not only support their contributions to this project, but help prepare them for a variety of STEM occupations.
鈥淚f we can take the next generation of students and get them the best skills, the kind of experience I wish I鈥檇 had as a student early on, that is a key part of the workforce development component of the grant,鈥� Greene said.
