The purpose of the “hackathon” is to explore and extend what is currently possible. We tap into the participants’ creativity and create future solutions and scenarios, now. In 2016 I took part as an artist in #htb2016.
It started with an introduction followed by people standing up to share their ideas. I stood up to announce my idea: to design an app that warns you before you go to sleep. I thought it would be perfect and simple. After everyone pitched their ideas people had to join the ideas they liked.
Unfortunately for me none of the people there thought this great idea was worth joining and I was left as the only person without a team. However, the mentors were prepared for this and now people were told to share their what they had and what they still needed.
Polle van Duuren mentioned that his team was rather large and that if I were to join them that they could split in two separate groups. This left us without a idea, so we decided to follow the instructions given to us by #htb2016 and make a list of key terms. From this small list we created a fun interactive and portable device to measure group and individual engagement.
Divided into groups
Ehsan: daily use/ transfer to action/ nice design/ professional /Perfectionist/ everyone understands what is going on.
Carlos: user friendly/ real time, or close, provides feedback/ improve mental health
Polle: fun weekend , clear functionality, practical no spacekadets/ learning from eachother/ interesting for masses
Lydia: Fun/ Easy to use/ Friendly/ Useful – nut/ portable
From this list of values we came up with the following list of values for the group.
1: social fun
2: real time feedback
3: compacts simple friendly
4: improve daily health
Working title: “The group harmonizer”
After a few brain storm sessions we quickly agreed that we wanted to make a physical object that would light up to show the brain frequency of the individual. If all members of the group would peak in a similar brain frequency then the a tree will light up. Each group member would have an individual light and be connected to central light.
With light you can have an artistic way of showing to yourself and others where you are at. This might be fun for meditation, games or anything involving a group. The fun would be to see if you can test how you can influence your brain frequencies actively via direct neuro feedback.
In a speed reading course they teach how one can trick the brain to peak in Beta or Alpha frequencies and through classical music. For the most hard core speed readers they recommended bio sensors EEG and speed reader software to allow direct feedback so that you can optimize your reading and memorization. Instead of using this has a training tool, how can this be turned into a toy or something interactive and fun?
Inspired by Mariko Mori and her art installation called “Oneness” that used sensors to measure the brain frequencies of a group of three. The whole group had to be inside what almost looks like an alien pod. Inside you plug the electrodes into a seat that allows you to look up at a grey screen. With your brain frequencies different colors would show up on a screen. This basically allowed each participant to make art with their brain frequencies. Mine stayed gray for a long time as the lower frequencies were (according to the museum in Groningen) not programed.
The second day arrived and time pressure made us realize we had to split into two, since we only had one weekend to create and make our device we had restrictions to lock our creativity and allowed us to focus on what was possible. What materials could we get access to, and what materials would be fun? How would we be able to conduct enough research to create something new and interactive.
GROUP ONE: Real time feedback
One group will built the Arduino and make the lights for two pod’s and one main light.
I was part of group one. I never had as much fun soldering all the lights. But then came the challenge. As the staff of the FabLab at the Waag was kind enough to help with the lights I was wondering how to make something that is FUN, BEAUTIFUL and COMPACT SIMPLE FRIENDLY.
GROUP TWO: Data collection and coding
Other group will look into the brain frequencies and group engagement. The coding and the emotive.
Initially we thought of a tree or some central item that could show the group brain light up. Somehow as a group we decided that this was not novel enough of a metaphor. We wanted a symbol that could stand for connectivity and would be as mysterious as the brain itself. Something more elusive, fragile, and mysterious than the everyday tree.
The Rafflesia is an organism that you can find in Malaysia and Indonesia. If you are lucky you can venture into the jungle and see one in bloom. The rafflesia has no leaves no stem and no roots. With all the technology and knowledge we have we cannot grow it in a greenhouse. It can grow to be 100 cm in diameter making it the largest flowering organism.
Just like the Rafflesia, much still is unknown about our brain. We keep learning more and more about how it functions and connects to the outer regions of our body. But the structure of the rafflesia and the individual growing pods would be perfect to turn into small lights with a larger light symbolizing the group brain in the shape of a flowering rafflesia.
The delicate design was sustainable and made out of thin paper, using the concept of a Chinese lantern. This way the blue print can be used and replicated anywhere in the world. The making of the device would also add to the fun. You can do it with your friends, coworkers or kids in a couple of hours.
Inside these balls we place the lights that would be inserted into an arduino. The little lights we could program to have a specific color.
As GROUP TWO was collecting data and programming. We decided to change our focus away from our initial idea and to measure the engagement of the individual and the engagement of the group. Although measuring engagement of individuals in this way is not something new, using this to measure group engagement is something we have not come across. Making the idea novel and scientifically interesting.
What implications this would have on the group as an entity? We were interested in using BCI’s to measure a group of people in a social context. Through an exploration of possibilities therein we pinned ourselves down on measuring Engagement using [beta/(aplha+theta)] as described in a number of scientific papers. What was left was the hardware and code necessary to make our idea come to life.
Thanks to the help of Harry who was part of another team we were able to have the code we needed to make the lights to what we wanted them to do. We have worked with Muse and wrote (with a lot of help) a code in Python to extract the values of all individuals, as well as the group. We then used Arduino to activate a number of RGB LEDs that change based on the different engagement scores.
Scientific basis of engagement calculation
- Berka et al., EEG Correlates of Task Engagement and Mental Workload in Vigilance, Learning, and Memory Tasks (2007)
- McMahan et al., Evaluating Electroencephalography Engagement Indices During Video Game Play (2015)
- Pope et al., Biocybernetic system evaluates indices of operator engagement in automated task (1995)
Now that we finished the code, the lights, the paper and the glue we had to come up with a way to present our idea to the jury. The presentation consisted of two parts one where the research that was used was presented, as well as the concept and design. The second part consisted of three of the team members playing a computer game where each individual was being scanned and when engaged the light of the individual would light up.
Just before the we had to go and present our idea the software crashed and Harry came to our rescue again. Another team was kind enough to take our spot to allow us to reboot the system. The software would crash if one of the scanners would be too far away from the computer. We learned our lesson, thanked all the other teams for helping us and we went on to present our interactive engagement measuring device to the rest.
It was interesting to learn that the group engagement could light up when the individuals were not engaged. Or when the individual was engaged that was not a measure for the group to be engaged. This showed that the unengaged individuals can be engaged as a group, and the engaged individual as a group can be unengaged. Due to the time pressure we were not able to research this further, but it was interesting to find the engagement also responding to emotional states as team members would die in the game and get frustrated or upset. Some team members only showed engagement when they died within the game, while others would continuously be engaged.
All the groups had amazing devices and ideas. One of the groups ended up designing the app I had pitched at the onset of #htb2016 and they wanted to sell their app to Tesla and other self driving cars. So hopefully we can soon all get this app and prevent people from falling asleep behind the wheel! The first place went to an amazing project. That team led by Diego Maranan had arrived with a prototype they wanted to continue to work on and it is absolutely amazing. With a fashion designer coders and scientist they truly made something magnificent. If you think a massage chair is a great invention, contact Diego!
As I was amazed at what people created in a few days I was not paying attention to the awards being called out. So my teammates had to drag me out of the audience when I was not responding after a few days of focused tinkering. Turned out that the jury liked our idea and wanted to see it applied in a larger setting or art installation.