“dissonant imaginary” — Daito Manabe + Kamitani Lab, Kyoto University and ATR
Daito Manabe
November 17, 2018
This work focuses on the relationship of visuals and sound, using the cross modal attention phenomenon to create the audio-visual installation.
[about Research]
The relationship between brain activity, sensibility, sensation, and input image and sound has been an inexhaustible interest with a long history.
The detection and analysis of electroencephalogram or EEG has been attempted in the past for its inexpensive and manageable nature, but as it can only acquire the sum total of the various brain activity happening at a single moment, vague descriptions such as “whether the subject is awake or not” could only be used. Therefore, the use of brain activity analysis alongside art has been conceptualized without any large result for a longtime.
Daito Manabe, who has been creating works through the relation of mathematics and bodily expression, has been previously experimenting with the usage of deep neural networks (DNN) alongside advanced science research such as neural activity.
On the other hand, Kamitani Yukiyasu laboratory at Kyoto University / ATR has been developing technology ahead of the world that analyzes the state of mind through technology called “brain decoding” which utilizes human brain activity pattern measured by functional magnetic resonance imaging (fMRI) alongside pattern recognition by machine learning. In recent years, the usage of large scale databases alongside DNN and “brain decoding” to analyze arbitrary objects perceived from the brain activity pattern has attracted much attention around the world.
Daito Manabe has focused on the research of “brain decoding” early on in 2014 contacting Kamitani initially, to then create a collaborative project based on the cross modal attention phenomenon, which focuses on the relationship of the brain activity, visuals and sound.
This work focuses on the mutual relationship of visual senses and auditory senses, the relationship between sound and image, and the visualization of reconstructed imagery based on the activity of the visual cortex.
Have you ever experienced vivid emotional imagery associated to a soundtrack of a movie or nostalgic music that you heard long ago?
How does music affect the way we experience video and vice versa, how does video affect the way we experience music?
In the future, what kind of form will music and visuals be realized as technology that enables music to be automatically generated from brain activity simply by watching a movie or visuals being generated by listening to music is realized.
In this work, we speculate on the future of video and sound through the usage of brain decoding.
Year of production 2018
Duration 16 minutes (1 minute x 16 pieces)
Material: Audio Visual
---
History
[talk]
2018
Nov.17 2018
Daito Manabe × Yukiyasu Kamitani（Professor. Kyoto University）Talk Session
at Kirishima Open-Air Museum, Art Hall
https://open-air-museum.org/news/news-36519
2019
Jul.19 2019
Reading thoughts with Yukiyasu Kamitani and Daito Manabe
at Sónar+D in Barcelona SPAIN
Dec.9
MUTEK.JP Presents 「MUTEK.JP2019 × SUPER DOMMUNE」
DJ：Kode 9（Hyperdub｜from UK）、Daito Manabe（Rhizomatiks Research）
LIVE：Saskiatokyo
SPECIAL TALK：「dissonant imaginary」Moderator : Arina Tsukada
Daito Manabe + Yukiyasu Kamitani（Kamitani Lab / Professor Department of information Science and Technology, Graduate School of Informatics, Kyoto University）
Moderator : Arina Tsukada
https://www.dommune.com/streamings/00017/
Department of information Science and Technology, Graduate School of Informatics, Kyoto University
[performance]
2018
Nov.16 2018-Jan.14 2019
"Daito Manabe Rhizomatiks Research Exhibition"
Kagoshima, Kirishima Open-Air Museum Art Hall (Kagoshima, JAPAN)
https://open-air-museum.org/en/event/event-36404
2019
Jul.18 2019
Daito Manabe + Kamitani Lab: “dissonant imaginary”
at Sónar Barcelona (SPAIN)
Aug.3-Sep.29 2019
Reborn Art Festival2019 (Ajishima, Miyagi, JAPAN)
https://2019.reborn-art-fes.jp/artist/daitomanabe-2
Oct.6 2019
The art lecture "present of artificial intelligence and picture expression"
Aichi University of the Arts University Art Museum practice room (Aichi, JAPAN)
https://www.aichi-fam-u.ac.jp/news/topics_017501.html
MUTEK.MX (Mexico)
Nov.23 2019
https://twitter.com/rhizomatiks/status/1196729781145853952
MUTEK.JP (Tokyo)
Dec.11 2019
https://tokyo.mutek.org/jp/past-editions/edition-2019
2020
Feb26 - April20 2020
"Neurons: Simulated Intelligence" at Centre Pompidou, (Paris, FANCE)
https://www.centrepompidou.fr/en/program/calendar/event/cbEr6Ko
2021
Mar.20-22.Jun 2021
"rhizomatiks_multiplex" at Museum of Contemporary Art Tokyo (JAPAN)
https://www.mot-art-museum.jp/en/exhibitions/rhizomatiks/
Nov.11 2021
Daito Manabe + Kamitani Lab "dissonant imaginary 2021"
PHASE相 in Shanghai.
https://www.instagram.com/p/CWLjZ0gv3BH/
[about Research]
The relationship between brain activity, sensibility, sensation, and input image and sound has been an inexhaustible interest with a long history.
The detection and analysis of electroencephalogram or EEG has been attempted in the past for its inexpensive and manageable nature, but as it can only acquire the sum total of the various brain activity happening at a single moment, vague descriptions such as “whether the subject is awake or not” could only be used. Therefore, the use of brain activity analysis alongside art has been conceptualized without any large result for a longtime.
Daito Manabe, who has been creating works through the relation of mathematics and bodily expression, has been previously experimenting with the usage of deep neural networks (DNN) alongside advanced science research such as neural activity.
On the other hand, Kamitani Yukiyasu laboratory at Kyoto University / ATR has been developing technology ahead of the world that analyzes the state of mind through technology called “brain decoding” which utilizes human brain activity pattern measured by functional magnetic resonance imaging (fMRI) alongside pattern recognition by machine learning. In recent years, the usage of large scale databases alongside DNN and “brain decoding” to analyze arbitrary objects perceived from the brain activity pattern has attracted much attention around the world.
Daito Manabe has focused on the research of “brain decoding” early on in 2014 contacting Kamitani initially, to then create a collaborative project based on the cross modal attention phenomenon, which focuses on the relationship of the brain activity, visuals and sound.
This work focuses on the mutual relationship of visual senses and auditory senses, the relationship between sound and image, and the visualization of reconstructed imagery based on the activity of the visual cortex.
Have you ever experienced vivid emotional imagery associated to a soundtrack of a movie or nostalgic music that you heard long ago?
How does music affect the way we experience video and vice versa, how does video affect the way we experience music?
In the future, what kind of form will music and visuals be realized as technology that enables music to be automatically generated from brain activity simply by watching a movie or visuals being generated by listening to music is realized.
In this work, we speculate on the future of video and sound through the usage of brain decoding.
Year of production 2018
Duration 16 minutes (1 minute x 16 pieces)
Material: Audio Visual
---
History
[talk]
2018
Nov.17 2018
Daito Manabe × Yukiyasu Kamitani（Professor. Kyoto University）Talk Session
at Kirishima Open-Air Museum, Art Hall
https://open-air-museum.org/news/news-36519
2019
Jul.19 2019
Reading thoughts with Yukiyasu Kamitani and Daito Manabe
at Sónar+D in Barcelona SPAIN
Dec.9
MUTEK.JP Presents 「MUTEK.JP2019 × SUPER DOMMUNE」
DJ：Kode 9（Hyperdub｜from UK）、Daito Manabe（Rhizomatiks Research）
LIVE：Saskiatokyo
SPECIAL TALK：「dissonant imaginary」Moderator : Arina Tsukada
Daito Manabe + Yukiyasu Kamitani（Kamitani Lab / Professor Department of information Science and Technology, Graduate School of Informatics, Kyoto University）
Moderator : Arina Tsukada
https://www.dommune.com/streamings/00017/
Department of information Science and Technology, Graduate School of Informatics, Kyoto University
[performance]
2018
Nov.16 2018-Jan.14 2019
"Daito Manabe Rhizomatiks Research Exhibition"
Kagoshima, Kirishima Open-Air Museum Art Hall (Kagoshima, JAPAN)
https://open-air-museum.org/en/event/event-36404
2019
Jul.18 2019
Daito Manabe + Kamitani Lab: “dissonant imaginary”
at Sónar Barcelona (SPAIN)
Aug.3-Sep.29 2019
Reborn Art Festival2019 (Ajishima, Miyagi, JAPAN)
https://2019.reborn-art-fes.jp/artist/daitomanabe-2
Oct.6 2019
The art lecture "present of artificial intelligence and picture expression"
Aichi University of the Arts University Art Museum practice room (Aichi, JAPAN)
https://www.aichi-fam-u.ac.jp/news/topics_017501.html
MUTEK.MX (Mexico)
Nov.23 2019
https://twitter.com/rhizomatiks/status/1196729781145853952
MUTEK.JP (Tokyo)
Dec.11 2019
https://tokyo.mutek.org/jp/past-editions/edition-2019
2020
Feb26 - April20 2020
"Neurons: Simulated Intelligence" at Centre Pompidou, (Paris, FANCE)
https://www.centrepompidou.fr/en/program/calendar/event/cbEr6Ko
2021
Mar.20-22.Jun 2021
"rhizomatiks_multiplex" at Museum of Contemporary Art Tokyo (JAPAN)
https://www.mot-art-museum.jp/en/exhibitions/rhizomatiks/
Nov.11 2021
Daito Manabe + Kamitani Lab "dissonant imaginary 2021"
PHASE相 in Shanghai.
https://www.instagram.com/p/CWLjZ0gv3BH/
Credit
Daito Manabe + Kamitani Lab, Kyoto University and ATR