Machine learning algorithms had been used before to study brain scans and generate visualisations of what a person is thinking when referring to binary images like black and white letters or simple shapes.
However, the team from Kyoto have developed a new technique built on top of the scientific platform – BioRxiv – that decodes thoughts using deep neural networks. This new approach allows scientists to decode far more complex images.
Kamitani, one of the lead scientists at Kyoto University explains that their ‘previous method was to assume that an image consists of pixels or simple shapes. But it’s known that our brain processes visual information hierarchically extracting different levels of features or components of different complexities’.
10 Month Study
The study involved showing three subjects a series of natural images, alphabetical images and geometric shapes for varying lengths of time over the course of 10 months. In some instances, the brain activity was measured whilst the subjects were looking at one of the 25 images, in other cases brain activity was monitored afterwards, when the participants were asked to think of the image they had seen earlier in the day.
Once the brain activity was monitored, a computer decoded the information to generate the visualisation of the subjects’ thoughts. The flowchart breaks down the science of how brain activity is decoded.
The image below shows the results of the reconstructed image for participants where their brain activity was monitored whilst they were looking at the images and letters:
There scientists were also able to reconstruct visual imagery, just by asking the subject to think of the images they had seen earlier in the day. As expected, as time passes, it becomes harder for the AI system to decode the brain signals into images (See below). This is because it’s harder to recall the image exactly as it was seen and as Kamitani explains, ‘the brain is less activated’. However, the team are now working on refining the algorithms to improve clarity.
Healthcare and Policing Applications
The potential for this type of technology spans everything from healthcare to policing. Witnesses to a crime could simply think back to what happened and the visualisation technology could map out the scene – no more dodgy e-fits! Mental healthcare professionals could also use the system to understand how patients think.
While the idea of a computer tapping into your brain and reading what you’re thinking may sound like something from the future, the Japanese researchers aren’t the only ones with some skin in the game.
Entrepreneur, Bryan Johnson, is working to build computer chips that you can implant in the brain to improve neurological functions, and ex-Google employee – Mary Lou Jepsen – is working on developing a hat that will make telepathy possible within the next 10 years! The future is certainly going to be an interesting one.