George Orwell’s iconic dystopian novel 1984 famously featured cameras capable of discerning a person’s state of mind – their contentedness, truthfulness or trustfulness – simply by looking at their face. The year 1984 came and went without such a technology emerging, but as demonstrated by Diane Paverman ’13 and Eric Murray’s ’13 summer research on the functional near-infrared spectrometer (fNIRS), scientists are getting closer to achieving Orwellian-like surveillance capabilities.
The fNIRS uses 52 sensors to record data about a subject’s blood flow oxygenation levels and then interprets that data to determine a subject’s state of mind. In the past, the fNIRS has successfully been used to determine a subject’s state of mind regarding truthfulness and suspicion, but Paverman and Murray hope to expand the capabilities of the fNIRS to allow it to determine a subject’s state of mind regarding trustfulness and distrustfulness.
Paverman and Murray are working under Stephen Harper Kirner Professor of Computer Science Stuart Hirshfield to utilize eBay’s rating system in order to construct controlled situations of trust and distrust. Participants were told that they would be purchasing a car on eBay and then given access to review profiles for three sellers. After reviewing the seller profiles, participants were shown a video in a controlled environment that asked questions regarding the trustworthiness and untrustworthiness of the eBay sellers. Fifteen participants in the study were monitored by the fNIRS system while another 15 participants – the control group – completed the same task without fNIRS monitoring. As the participants answered questions, the fNIRS recorded their blood flow oxygenation levels and looked for patterns.
Then, using a program known as a machine learning algorithm, Paverman and Murray taught the fNIRS to associate certain blood flow oxygenation levels with states of trustfulness or distrustfulness. If the pattern of blood flow oxygenation levels corresponds strongly enough to the controlled scenarios of trust and distrust, then the fNIRS should be able to anticipate a participant’s state of trustfulness or distrustfulness in real time.
According to Paverman, “this experiment is really only among the first steps in a much greater project… our goal is to apply a machine learning algorithm to our data in a way that can accurately predict a user's state of trust solely based on their brain data.” Eventually, scientists will be able to connect brain data like that collected by the fNIRS with more easily observed parameters, such as a person’s mouse movements or keyboard activity while using a computer. This ability would be especially useful to government agencies conducting online surveillance or infiltrating crime or terrorist organizations via the internet, as it would allow government agents to detect when computer users start to become suspicious of surveillance or infiltration. Paverman and Murray emphasized, though, that such capabilities will not emerge without significant further research.
Paverman is a computer science concentrator with a minor in philosophy while Murray is a computer science and math dual concentrator. Paverman plans to continue studying human-computer interaction and hopes to become a researcher in the field. She is a member of the Hamilton College varsity diving team. Murray also plans to go into research.
Diane Paverman is a graduate of Byram Hills High School (N.Y.) and Eric Murray is a graduate of Chittenango High School (N.Y.)
