Neckband Detects User Thoughts And Translates to Speech [Neural Interface]
I recently came across news of a device that geeked me out. Its a neckband that can detect and analyze neural firings when we think about saying something, and translate them into audible words via speech synthesizer. Beyond the obvious use of bettering the lives of people who’ve lost their ability to speak, it could enable us to make phonecalls without having to actually talk (as is demonstrated in a video in this article). The creators of the device mention that they’ll have a product by the end of the year for people with ALS (a.k.a. Lou Gehrig’s Disease).
In my aforementioned geek-out craze I told my girlfriend about the device, called the Audeo, who immediately identified the problem of the device saying a thought you don’t actually want the other person to hear. You’re on the phone with your boss when you suddenly hear the device blurt out “Are you never going to shut up about those damn TPS Reports!?“.
Good point. But the creators say the device can differentiate between things that you’re thinking, and things that you actually want to say. You have to think about using your voice for the device to pick up on it.
I’m sure that this ability is a beneficial byproduct of making the device a “collar” around your neck monitoring the nerves that control muscles of the larynx.
Our Head’s Too Messy, Go for the Neck
The device is not a brain interface worn on the head, so it stands to reason that (a) they are monitoring neural activity to the muscles that control speech (larynx/voicebox), and (b) by doing so it’s easier to detect things that you actually want to say, as opposed to what you’re casually thinking.
The larynx is innervated by branches of the vagus nerve on each side. Sensory innervation to the glottis and supraglottis is by the internal branch of the superior laryngeal nerve. The external branch of the superior laryngeal nerve innervates the cricothyroid muscle. Motor innervation to all other muscles of the larynx and sensory innervation to the subglottis is by the recurrent laryngeal nerve.
However, I’m sure we’ve all been in situations where we are on the verge of saying something, perhaps in an emotionally colored debate, but think twice and eventually say something less aggressive. In such a situation I’m sure the device could accidentally be triggered. So the user must make sure to be perfectly balanced, one with himself and the universe before using it for important conversations. At least for now.
Writing this I get the idea that this problem could be overcome with AI; natural language processing could detect potentially insulting sentences or harsh language. The user could then be prompted to verify whether he meant to say a particular sentence (whether this would introduce too much lag is another question).
Voiceless Phonecalls
The device, currently able to recognize 150 words, is under development by Ambient Corporation, co-founded by Micahel Callahan who demonstrates the device in the following video at the TI Developer Conference’08 by placing a “voiceless phonecall”.
For the past few decades, humans have increasingly been extending their intellectual capacity with the use of machines. An example is using mobile devices to retrieve knowledge on the fly — making each device-wielding human more intellectually capable than one 20 years ago. But this a matter of perspective, and many only see future invasive devices as “extensions of intelligence” (e.g. neural-interfaced memory storage device) and everything else as tools.
Modern technology is starting to blur this line between intellectual extensions and tools. The “Smartest Person in the Room” project is one of these: Using the Audeo, a person thinks of a question — the question is consequently sent to a web knowledge-application, the answer found and tunneled back out through the speakers. Question never audibly asked, yet answered. Quite brilliant.
Looking forward to monitoring the developments of this project, feeding my interest in machine interfaces right along Emotiv’s Epoc and Neurosky’s non-invasive neural interfaces.
Links & References
Other Similar Posts
- Layar's Augmented Reality Web for mobiles: finally layered information
- Useful augmented reality tool shipped by US Postal Service
- The AlloSphere: Visualizing data inside a gigantic metallic sphere
- Augmented reality games, but what's reality doing there?
- Wikitude AR - Augmented reality on Google Android
6 Comments, Comment or Ping
polypus
“Writing this I get the idea that this problem could be overcome with AI; natural language processing could detect potentially insulting sentences or harsh language.”
why is it a ‘problem’? wouldn’t it be better if we did actually say what we were thinking? is this political correctness gone berserk?
another point to ponder, is that if AI could be used to warn the user, then it could just as easily be used to completely silence them.
Apr 9th, 2008
frank burns
Something is fishy, because the various phonemes of speech are not produced in the neck/vocal folds. These muscles in the larynx are only responsible for voicing and intonation. The articulation of all vowels and consonants takes place at the lips, tongue and velum. Monitoring only the neck would not provide enough information for identifying phonemes, no way.
Nov 27th, 2008
Hrafn Thorisson
Clever observation Frank! I haven’t done more research than for what’s in the article above, but I believe we can safely assert that the device works in a manner similar to many neural interfaces, like Emotiv’s Epoc, where unique neural signatures are paired with full words (as opposed to parts of words like phonemes).
Nov 27th, 2008
BRIG
I do not understand this!
How can you find a unique neural signature for a specific word ( I am guessing they are using sub-vocalisation for this…a process where you actively think the message in language and image to speak it…this alone will filter out most of your ruder thoughts!), using …electrodes that sit on the skin??? This means that every time you place them they will not exactly fit on the same spot. How do you filter out all the surrounding neural activity in the surrounding muscles? Where do you place the electrodes to catch those neural signatures? Also: In conditions such as ALS, MS and MND the signals from the brain to the muscles are disrupted and become progressively more so, so sub-vocalisation will not work as well either: Even if they could pick up signal ‘templates’ for specific words and messages: These will become distorted.
It is true that most speech sounds, (except vocalisation) are made further up in the vocal tract…however: I understand that base-of tongue and sublingual movement can indicate a lot of those sounds.
Still I really do not understand how this can work. I am seriously intrigued…and skeptical. You would think that with a device like this one should be able to measure swallowing activity very accurately…however no device is around which can do this.
I will definitely try and find out more. Thank you for posting, but for now I would have to agree with Frank. Fishy.
Feb 20th, 2009
BRIG
Oh and: I do not think you can compare the emotive epoc with this device, really: the epoc is measuring neural activity in the brain - or so it seems.. this means they are picking up on increased activity on certain brain centres…which would explain why they can ( likely with some inaccuracy) kind of tell you about the emotion stuff! They never even hinted at being able to pick up specific linguistic thoughts!
The electrodes in the neckband can realistically only pick up neural activity from nerve to muscle and really: if only the activity in the larynx is measured and they get speech interpretation from that, I need to go and study again as I am apparently not up to date.
The only way I have to explain this is if the guy on stage has actually learned to work the muscles in his neck to an extend that allows the neck collar to pick up relatively clear signals and translate them into words. Even with this ( that would be a feat! And forget being able to use that easily with people with progressive conditions: It would mean a LOT of practice and drill!) I would have expected a message that is less smooth than the one we got over the phone. That sounded to me like a pre-recorded message! So maybe he learned to produce a certain muscle pattern in his neck and programmed the machine to produce this message when it picks up this particular neuro-signal for long enough
( note time lapse of answers).
I would love to proven wrong, working , as I do, with people with progressive illnesses.
Where can I find more about this?
Feb 20th, 2009
Reply to “Neckband Detects User Thoughts And Translates to Speech [Neural Interface]”