Victor's problem isn't really the vowels or pacing. The final consonants are soft or not really audible. I am not hearing the /ŋ/ of "long" as the most marked example. It sounds closer to "law". In his "improved" recording he hasn't fixed this.
I sometimes see content on social media encouraging people to sound more native or improve their accent. But IMO it's perfectly ok to have an accent, as long as the speech meets some baseline of intelligibility. (So Victor needs to work on "long" but not "days".) I've even come across people who are trying to mimick a native accent but lose intelligibility, where they'd sound better with their foreign accent. (An example I've seen is a native Spanish speaker trying to imitate the American accent's intervocalic T and D, and I don't understand them. A Spanish /t/ or /d/ would be different from most English language accents, but be way more understandable.)
A very important part of people trusting you is them being able to understand what you say without making extra efforts compared to a native speaker.
An easy way to improve intonation and fluency is to imitate a native speaker. Copying things like the intervocalic T and D is a consequence of that. It would be easier for a native Spanish speaker to say the Spanish /t/ and /d/ but intonation and fluency would be impaired.
The sounds don't "flow" as they should.
> An easy way to improve intonation and fluency is to imitate a native speaker.
There are lots of variations in English pronunciation. Singaporean, Australian or Scottish native speakers do sound very differently. I don't know to what extent they benefit from adjusting their accent if working in a different English speaking country to match the local dialect.
Also, as a non-native speaker I wonder if it's worth practicing my accent considering that everybody has a different accent anyway. Rather than trying to mimic a north american accent (which I'll never be able to do anyway), I'd be more interested to identify and fix the major issues in my prononciation.
The specific problem is that American intervocalic /t/ and /d/ is very similar to Spanish /ɾ/. But if they don't get it right it's not perceived as the right phoneme. The Spanish /t/ is more dental and the undervocalic /d/ is more of a [ð] but they will sound correct in English.
People have all sorts of motivations for learning languages and accents. Right now, I'm using this tech to work on my accent in Spanish. Honestly I would rather mumble almost unintelligibly with an decent Mexican accent than speak Spanish slowly and clearly with an American accent. There is a difficult but necessary period of learning an accent where intelligibility drops. For a while, I made a strange [ð]-like noise when learning the alveolar trill (rolled R), and it would have been more intelligible to use something like alveolar tap. But, I built up the muscle memory, and can now make the correct sound. Hearing a version of myself (rather than a different speaker) gives me a more useful target to mimic, and the distance metric gives me a useful measure of whether I'm closer or further from the target.
"If Victor wanted to move beyond this point, the sound-by-sound phonetic analysis available in the BoldVoice app would allow him to understand the patterns in pronunciation and stress that contribute to Eliza’s accent and teach him how to apply them in his own speech."
Indeed Victor would likely receive a personalized lesson and practice on the NG sound on the app.
Thank you for pinpoints my confusion/disconnect on what lack of improvement that I was sensing. There was an improvement on pacing, and cadence, yes, but that was not the main challenge with Victors accent. Visually I'd say victor improved by at most 5% and not 50% as indicated by the visualization. In some regards it was even harder to understand than the original due to speed and cadence without improvement in core pronunciation.
Yeah, as long as it’s intelligible an accent is perfectly fine
It’s also perfectly fine to want to sound like a native speaker - whether it be because they are self conscious, think it will benefit them in some way, or simply want to feel like they are speaking “correctly”
Sorry to pick on you, it’s just amazing to me how sensitive we are to “inclusivity” to the point where we almost discourage people wanting to fit in
I believe I have some kind of auditory processing issue because with some accents, it can be really hard for me to understand what someone is saying, when other people can understand them fine. It's gotten to the point where I avoid going to some shops in my town because I know the staff have quite strong accents and I feel embarrassed having to ask them to repeat themselves all the time
My wife has this problem. She got chided because she asked for a translator for the doctor at the children’s hospital and they just assumed she was being racist. He was Thai and his English was very difficult to understand.
Yeah, that's my concern too. I went to a beauty salon where the woman who saw me had a strong Eastern European accent, and the only fabric shop in town is staffed by people who have a strong... Indian accent? it feels like somewhere in that region. I have no problem with the people themselves, but I just struggle to properly understand their accent sometimes
That kind of implies that there's a "correct" accent for English, even though many countries and regions natively speak it. Someone from Glasgow is just as much of a native speaker as someone from Los Angeles even though the accents are wildly different.
Hence in quotes, man
I've literally heard a story of a kid arriving to the US from Scotland and being sent to speech therapy to remove a Scottish accent.
And I've heard other such stories of American schools flagging kids for speech therapy when what they have is an accent. I feel like Americans are actually some of the worst about that.
A strong enough accent can make someone impossible to understand.
Huh? I’m talking about my own post where I said maybe they want to feel like the are speaking “correctly” - I added the quotes because obviously what’s correct is debatable
Besides it’s not like there isn’t correct either - if you’re out in the Midwest what’s correct is just what everyone is speaking.
Its obvious that a kid from Ohio who speaks perfect isn’t going to go to Scotland and speak it “correctly”
Like it’s such low hanging fruit to always be that guy to point out the lowest level, most obvious exception
Being legible also means to cater to your audience. I work in an English-speaking company in a country where English isn't the native language, with loads of non-native speakers from around the world. Sometimes the native/best English speakers are the ones being misunderstood, because they use idioms or advanced words. None of us are bad at English, and I don't mean that I need to "dumb it down" (if anything, verbally I'm one of the worser ones), but I don't feel like I'm missing out on speaking simple with an accent.
Generalizing from my own experience, it’s easier for me to understand a non-native Spanish speaker than a native Spanish speaker and I would guess that the same applies with ESL speakers. One thing I found really fascinating is that even though I’d never studied French¹, I actually had an easier time understanding a conversation between my ex-wife and her aunt in French than when they spoke Spanish in which I was functional (my skill in the language has gone up a great deal since then so that I now read fluently, and speak and listen reasonably well, albeit less well than I would like).
⸻
1. Thanks to my kids studying French on Duolingo and my joining them, I can no longer say that I’ve never studied it.
>Generalizing from my own experience, it’s easier for me to understand a non-native Spanish speaker than a native Spanish speaker and I would guess that the same applies with ESL speakers.
You guessed right -- it's /usually/ easier to understand other non-native speakers, both because of accent and less idioms. That is unless the accent is really heavy and doesn't match your own.
I don't think it's really to do with accent, but rather that non-native speakers tend to talk slower and use simpler grammar and a more limited vocabulary (including fewer idioms).
That does help, I’m sure. There’s also a tendency to incorrectly apply foreign grammatical structures to the language in a way that might be strange or incomprehensible to a native speaker while making perfect sense to a fellow second-language speaker.
Intelligibility heavily depends on what you expect to hear, and that depends on your native language or even locality. Even a tiny amount of French accent in English makes it sound like gibberish to me (but not others, and I don't have this issue with other thick accents). I'm sure my native accent is also incompatible with someone else's ears. That's the reason people pay accent coaches.
Yes, should go without saying that intelligibly is perfectly provided it’s intelligible in whatever context you’re in
It's fascinating that "long" is the biggest tip-off you have, given that Mandarin Chinese (based on the mention of "a noticeably strong Chinese accent") does have words that have the same IPA pronunciation (if you set aside tone) [0] as an American whose speech follows the cot-caught merger [1].
Meanwhile the thing that stood out to me in the initial recording were the vowel sounds: for instance, "young" sounded almost like it rhymed with "long" before training. (That makes sense, since Mandarin similarly has a word with that sound, as can be found in the common last name Yang [2].)
Incidentally, Mandarin has words that sound like "lung" (e.g. the word for "cold" [3]), but if you replace the "l" sound at the front with a "y" sound, depending on which of two transformations you use, it turns the vowel sound into a long o [4] (near rhyme with "lone"). (There is another transformation that you can use that results in a leading "y" in pinyin, but in that specific case, the vowel turns into a long e, and the "y" is largely silent (e.g. the word for "solid" [5]).)
In the last recording, Victor is clearly rushing through the sentence, and you can tell that where he previously had a clear "s" ending for the word "days", it's now slurred into a "th" sound. Agreed that that's actually a net negative for intelligibility.
The wiktionary links below have clips of pronunciation. I will note that not all native speakers have a Standard Chinese accent [6,7], so there are assuredly some differences in pronunciation to be expected depending on exactly which region said individual hails from.
[0] https://en.wiktionary.org/wiki/%E6%B5%AA
[1] https://en.wiktionary.org/wiki/long
[2] https://en.wiktionary.org/wiki/y%C3%A1ng#Mandarin
[3] https://en.wiktionary.org/wiki/%E5%86%B7
[4] https://en.wiktionary.org/wiki/%E7%94%A8
[5] https://en.wiktionary.org/wiki/%E7%A1%AC
What the vector-space data gets right, and what the human commentary tends not to, is the idea that accents are a complex statistical distribution. You should be careful about the concept of a "default" or "neutral" accent. Telecommunications has spent the 20th century flattening accents together, as has accent discrimination. There's always the tendency for people to say "my accent is the neutral standard against which all others should be measured".
For sure, and I don't think we ever use the term default or neutral. The "the American English accent of our expert accent coach Eliza" is just that -- it's one accent.
As a learning platform that provides instruction to our users, we do need to set some kind of direction in our pedagogy, but we 100% recognize that there isn't just 1 American English accent, and there's lots of variance.
> There's always the tendency for people to say "my accent is the neutral standard against which all others should be measured".
You can measure this by mutual intelligibility with other accent groupings.
Well, no. That would also measure what accents someone has the most exposure to, which doesn't necessarily reflect its "absolute intelligibility", but rather its popularity. Popularity and optimality are not the same thing. You would first need to measure if an accent's popularity is a result of its optimality to make the claim that your measure is accurate.
> Popularity and optimality are not the same thing.
Yes they are. If we lived in a world where Australia was a world superpower we might be having this conversation upside down and with r’s where there shouldn’t be any, but we don’t. Every student wants to learn to speak with an American accent because it has the highest level of intelligibility owing to exposure via cinema, music, expatriate communities, etc.
This is so cool. Real-time accent feedback is something language learners have never had throughout all of human history, until now.
Along similar lines, it would be useful to map a speaker's vowels in vowel-space (and likewise for consonants?) to compare native to non-native speakers.
I can't wait until something like this is available for Japanese.
The approach in the article is roughly equivalent to having someone listen to you speak and then repeating back in their own voice so you can attempt to copy their accent. Certainly nice to have available on demand without needing to coordinate schedules with another human.
A good accent coach would be able to do much better by identifying exactly how you're pronouncing things differently, telling you what you should be doing in your mouth to change that, and giving you targeted exercises to practice.
Presumably a model that predicts the position of various articulators at every timestamp in a recording could be useful for something similar.
> something language learners have never had throughout all of human history
.. unless they had access to a native speaker and/or vocal coach? While an automated Henry Higgins is nifty, it's not something humans haven't been able to do themselves.
Native speakers are less helpful at this than you might think. Speech coaches are absolutely the way to go, but they're outside the price range for most people ($200+/hr for a good coach). BoldVoice gives coach-level feedback and instruction at a price point that everyone can access, on demand.
Do you have another blog post showing your product giving targeted feedback about individual speech sounds? That's what I would expect from a coach.
Not yet - this was our first technical blog post. You can check out the BoldVoice app and test out the sound-level feedback yourself. Or watch this app walkthrough video - https://www.youtube.com/watch?v=3Sv5K4Z9P4c
You can take a language class rather than have a personal instructor. Although accents are a sensitive topic so I don't remember mine going into it much.
As someone who took English classes for years growing up, I wish that were the case. In fact, most teachers don't really know how to teach pronunciation. Also, in a typical group class setting, it's challenging to give each student one-on-one feedback. On BoldVoice, we solve that with 1) unlimited instant feedback from sound-level AI - your most patient coach. 2) in-depth video lessons from the best coaches in the world (Hollywood accent coaches). I'm a cofounder of BoldVoice, by the way. :)
Language class and accent coaching are very different things.
Try learning a language where they won't understand you with a foreign accent. I assume tonal languages are like this but haven't tried learning any.
Japanese is sort of like this - you have to say foreign words the Japanese way very forcibly, to the point that Americans will think you're being racist if they hear you do it.
> Real-time accent feedback is something language learners have never had throughout all of human history, until now.
Do you have a source for this? It doesn't seem plausible to me, but I'm not an expert.
That's a fascinating idea! Definitely something to try out for our team. We actively and continuously do all sorts of experiments with our machine learning models to be able to extract the most useful insights. We will definitely share if we find something useful here.
Like others recently, I've been extremely impressed by LLM's ability to play GeoGuessr, or, more generally, to geo-locate random snapshots that you give them, with what seem (to me) to be almost no context clues. (I gave ChatGPT loads of holiday snapshots, screenshotted to remove metadata, and it did amazingly.)
I assume that, with enough training, we could get similarly accurate guesses of a person's linguistic history from their voice data.
Obviously it would be extremely tricky for lots of people. For instance, many people think I sound English or Irish. I grew up in France to American parents who both went to Oxford and spent 15 years in England. I wouldn't be surprised, though, if a well-trained model could do much better on my accent than "you sound kinda Irish."
We actually did something like this for non-native English speakers a few months back. Check out https://accentoracle.com (most mind-blowing if you're a non native English speaker)
Interestingly enough, it thought I was Russian, even though my native language is French. It was tied at 32% with French though.
Edit: Tried it a few times and also got English as an accent. Pretty fun application!
Fun. I have a strongly modulated North American midwestern accent so unsurprisingly it had me read several paragraphs before only being able to say with any certainty that my accent was 83% English with the rest being Spanish/Russian. It couldn't detect the country of origin.
Agreed, pretty meh. Tried my usual accent (the one where natives mostly can’t tell where I'm from) — got 78%. Then went full cartoon russian ‘bad neighborhood’ mode — somehow scored 68%.
Well, it says I'm Finish. But now I have a new game, where I put on my best Italian or Russian or Greek or Australian accent and try to see how close I am.
I'm terrible, according to the program. My Italian is Russian or Hungarian or Swedish, my Australian is English.
New party game unlocked.
Amazing! If you can make it go viral again too, I will love you!
I've been building that exact game
accentgame.xyz
I would love to be able to explore combinations of X spoken language with Y accent, like for example I've always been curious how French sounds spoken with an Indian accent.
Swiss-German accent doesn't seem to be on the list, so it guessed mostly Swedish.
I'm 42% Arabic apparently! And 20% Russian. Got an 81% American accent level. I guess it is tuned to non-native-English speaker accents.
Was that right? Or what is the correct native language it should have predicted? Note the %s in the accent breakdown section are prediction probabilities
Wow that was actually accurate
Yes, although I believe this is a speaker embedding model here, so not LLM related.
This kind of speech clustering has been possible for years - the exciting point with their model here is how it's highly focused on accents alone. Here's a video of mine from 2020 that demonstrated this kind of voice clustering in the Mozilla TTS repo (sadly the code got broken + dropped after a refactoring). Bokeh made it possible to directly click on points in a cluster and have them play
https://youtu.be/KW3oO7JVa7Q?si=1w-4pU5488WxYL3l
note: take care when listening as the audio level varies a bit (sorry!)
Correct, not LLM
I bet you are right.
I had a forensic linguistics TA during college who was able to identify the island in southeast Asia one of the students grew up on, and where they moved to in the UK as a teenager before coming to the US (if I am remembering this story right).
From what I gather, there are a lot of clues in how we speak that most brains edit out when parsing language.
Or the classic scene in Mrs Doubtfire where Pierce Brosnan attempts to locate the origin of Robin Williams’s fake English accent.
I’ve seen some online quizzes that based on regional variations in accent (does root rhyme with foot or boot?) and vocabulary (what do you call a sweet fizzy beverage) that did a great job of locating where my Facebook friends back in the day grew up. It got me a bit off largely because while I grew up in Chicago, I had spent most of my adult life in Los Angeles so I tend to prefer “freeway” to “expressway” (changing that answer moved me from Rockford to Chicago).
Unrelated with th article itself, but I strongly disagree with the usage of the verb "hears" used in the title.
This verb is used for living organisms, and AFAIK, AI is not living, nor an organism.
This kind of subtle yet nasty personification is what encourages people to believe more in the fact that an AI "thinks" by itself or by extension that "AGI" is really close, or even worse, tricks more easily people into believe AI outputs.
"AI detects" would have been way more suitable.
What a great AI use-case! At first, I felt excited ...
But then I read their privacy policy. They want permission to save all of my audio interactions for all eternity. It's so sad that I will never try out their (admittedly super cool) AI tech.
I wonder if voice will be treated like facial data in terms of privacy. At least the EU has strong provisions against PII.
You can reach out and request your data to be deleted at any time.
"if you wish to opt out of future collection of voice samples, you may do so by disabling voice-related features in the BoldVoice app. Please note that this may limit the functionality of certain services."
Yeah, I can opt out. By not using any voice-related feature in their voice training app.
If you're still actively using the app, the voice will be retained and processed so that you can receive instant feedback, and also so that you receive additional personalized practice items and video lessons based on your speech needs. If you don't want the samples saved "in perpetuity", you can request them to be deleted once you decide that you're done with the application. Hope this helps!
This is fascinating work. Love seeing how you’re combining machine learning with practical coaching to support real accent improvement. The concept of an “accent fingerprint” is especially clever, and the visualization of progress in latent space really brings it to life. Excited to see where you take this next!
This is cool and one of the applications of LLMs that I'm actually looking forward to: accent training when acquiring a new language, particularly hearing what you would sound like without an accent!
That said, I found the recording of Victor's speech after practicing with the recording of his own unaccented voice to be far less intelligible than his original recording.
Looking forward to seeing the developments in this particular application.
Fair point! When Victor tried to speed up to speak as fast as Coach Eliza, while it sounded somewhat less accented, a few parts of the phrase did get less intelligible. 10 minutes of practice is only a start after all.
Interesting to note that we're also developing a separate measure of intelligibility that will give a separate sense of how intelligible versus accented something is.
This is really cool.
Just had an employee at our company start expensing BoldVoice. Being able to be understood more easily is a big deal for global remote employees.
(Note - I am a small investor in BoldVoice)
This is super cool.
A suggestion and some surprise: I’m surprised by your assertion that there’s no clustering. I see the representation shows no clustering, and believe you that there is therefore no broad high-dimensional clustering. I also agree that the demo where Victor’s voice moves closer to Eliza’s sounds more native.
But, how can it be that you can show directionality toward “native” without clustering? I would read this as a problem with my embedding, not a feature. Perhaps there are some smaller-dimensional sub-axes that do encode what sort of accent someone has?
Suggestion for the BoldVoice team: if you’d like to go viral, I suggest you dig into American idiolects — two that are hard not to talk about / opine on / retweet are AAVE and Gay male speech (not sure if there’s a more formal name for this, it’s what Wikipedia uses).
I’m in a mixed race family, and we spent a lot of time playing with ChatGPT’s AAVE abilities which have, I think sadly, been completely nerfed over the releases. Chat seems to have no sense of shame when it says speaking like one of my kids is harmful; I imagine the well intentioned OpenAI folks were sort of thinking the opposite when they cut it out. It seems to have a list of “okay” and “bad” idiolects baked in - for instance, it will give you a thick Irish accent, a Boston accent, a NY/Bronx accent, but no Asian/SE Asian accents.
I like the idea of an idiolect-manager, something that could help me move my speech more or less toward a given idiolect. Similarly England is a rich minefield of idiolects, from scouse to highly posh.
I’m guessing you guys are aimed at the call center market based on your demo, but there could be a lot more applications! Voice coaches in Hollywood (the good ones) charge hundreds of dollar per hour, so there’s a valuable if small market out there for much of this. Thanks for the demo and write up. Very cool.
> It seems to have a list of “okay” and “bad” idiolects baked in
We're back to "AI safety actually means brand safety": inept pushback against being made into an automated racism factory with their name on it.
100%
(Minor nitpick, but I think "dialect" is a more appropriate word than "idiolect" here—at least according to Wikipedia, "idiolect" refers to a single person's way of speaking, whereas AAVE et al. are shared and are therefore considered dialects.)
OK, good read for me here. Based on your feedback and some research, I think I should have use ‘sociolect’ for both in that I was less complaining about ChatGPT’s unwillingness to use, say, finna, in a sentence, and more complaining about the vocalized accents. Anyway good catch, thanks!
Sociolect is the right term for a dialect used by a particular social group. A related idea is "register" when multiple related and mutually understandable standards exist, and are used in different contexts.
Super cool work, congrats BoldVoice team! I've always thought that one of the non-obvious applications of voice cloning/matching is the ability to show a language learner what they would sound like with a more native accent.
This and more exciting features are coming to the BoldVoice app soon!
this^
The hear my own voice without an accent thing is a really cool party trick.
I’d consider making this feature available free with super low friction, maybe no signup required, to get some viral traction.
What if it was already available? Try it out at https://accentfilter.com!
Hmmm. Initially impressive but upon retries and reflection ... not that great. It doesn't even maintain timing ... unless that's part of the transform.
Indeed yeah that’s one of the key weaknesses of the approach that we’re using. It overrides the speakers cadence and accent while keeping their voice profile / timbre in place. Different techniques may not do this but also may not copy over the accent to the resulting clip as effectively. So far we’re using this to support pedagogical (and lead-gen) use cases where we think it works sufficiently enough.
Let's put it a different way. I grew up in the UK till 24. I've lived in the USA for 36 years. The UK/US accent conversions dramatically altered my voice/accent; the AU one left it mostly unchanged.
This is offensive :))
How's the "accent conversion model" work? Is it all embedding based?
If so—and if you want to transfer-learn new downstream models from embeddings—then seems to me you are onto a very effective way of doing data augmentation. It's expensive to do data augmentation on raw waveforms since you always need to run the STFT again; but if you've pre-computed & cached embeddings and can do data augmentation there, it would be super fast.
I (an American from suburban Connecticut) was recently in London for an event and someone misheard me. Another Londoner said “It’s because of your accent!”, which of course was nonsense to me. What accent?
I’d be really interested to play with this tool and see what it thinks of my accent. Can it tell where I grew up? Can it tell what my parents’ native languages are (not English!)
A free tool like this would be great marketing for this company.
We did built two free tools, which are geared towards non-native English speakers. You can find them at https://accentoracle.com and https://accentfilter.com. They're less effective for English native speakers, but could still be fun.
What I find interesting is that it seems that folks from the UK tend to focus on consonants in distinguishing accents while in the US we distinguish more on vowels.
Scammers around here calling me saying my tax number is blocked will be happy to mask their accent with the one they are supposed to have. They will probably buy a premium subscription for this too.
Best kind of AI in the best kind of a free market.
I didn't find international english, would have been interesting.
Also, the USA writing convention falls short, like "who put the dot inside the string."
crazy. Rationals "put the dot after the string". No spelling corrector should change that.
I'm always very entertained when I'm talking with someone and pick up on some very slight deviation from the "norm" in their accent. I think it shows two things: that its near impossible to totally wipe that fingerprint of a past tongue, and that our ears are incredibly adept pieces of tooling
This is cool. I hadn’t come across an objective measurement of accents before.
Glad to see BoldVoice here.
I’ve been using it for a few months, and I can confirm it’s working.
Happy to see a happy BoldVoice user. Please don't hesitate to reach out to our team with feedback or thoughts on how we can continue to improve your learning journey. Helping you succeed is our #1 priority!
This is some insanely cool work. It's going to help so many people.
Thanks, we're doing our best!
This could be cool to apply as an acting aid, for actors trying to get in the role of a particular dialect-speaker.
Very interestng! Have you tested for other factors like speaking speed, emotional tone, or microphone quality to see what else is (or isn’t) influencing model perception?
For sure we did! The training data we used for this was purposely highly varied to account for these various factors so they don't cause too much bias in the model. But there's also an error rate regardless of how good you make it. We keep improving!
I'm torn.
On the one hand, the tech is impressive, and the demo is nicely done.
On the other, I think the demo completely misses the point. There's a disconnect between what learners need to learn and what this model optimises for, and it's probably largely explainable by how difficult (maybe even impossible) getting training datasets is. That, and marketing.
I believe most learners optimise for two things: being understood [1] and not being grating to the ear [2]. Both goals hinge on acquiring the right set of phonemes and phonetic "tools", because the sets of meaningfully distinct sounds (phonemes) and "tools" rarely match between languages.
For example, most (all?) Slavic languages have way fewer meaningfully distinct vowels than English. Meaningfully distinct is the crucial part. Russian word "молоко" as it's most often pronounced has three different vowels, at least two of which would be distinct to an English speaker, but Russian speakers hear that as one-ish vowel. And I mean "hear it": it's not a conscious phenomenon! Phoneme recognition is completely subconscious, so unless specifically trained, people often don't hear the difference between sounds that are obviously different to people who speak that language natively [3].
Same goes for phonetic "tools". English speakers shorten vowels when followed by non-voiced consonants, which makes "heart" and "hard" distinguishable even when t/d are transformed into the same sound (glottal stop or a tap). This "tool" is not available in many languages, so people use it incorrectly and it sounds confusing.
So, how would ML models learn this mapping between sounds and phonemes, especially when it's non-local (like with the preceding vowel length)? It's relatively straightforward to find large sets of speech samples labelled with their speakers' backgrounds, but that's just sounds, not phonemes! There is very little signal showing which sound structures matter for humans listening to the sound and which don't. [4]
There's also a set of moral issues connected to the "target accent" approach. Teaching learners to acquire an accent that superficially sounds like whatever they chose as a "target" devalues all other accents, that are just as valid and are just as English, because they have the same phonetic system (phonemes + "tools"). It can also make people sound a bit cringe, which I saw first hand.
Ideally learners should learn phonetic systems, not superficial accents. That's what makes speech intelligible and natural, even if it's has an exotic flavour [5][6]. Systems like the one the company is building do the opposite. I guess they are easier to build and easier to sell.
[1]: On that path lies a nice surprise: being understood and understanding are two sides of the same medal, so learning how to be understood a language learner inevitably starts to understand better. Being able to hear the full set of phonemes is the key to both.
[2]: There's a vast, VAST difference between people not paying attention to how someone speaks and them not being able to tell that something's off when prompted.
[3]: Nice demonstration for non-Hindi speakers: https://www.youtube.com/watch?v=-I7iUUp-cX8 When isolated and spoken slowly, the consonants might sound different, but in normal speech they sound practically indistinguishable to English speakers with no prior exposure. Native speakers would hear the difference as clear as you would in cap/cup!
[4]: Take their viral accent recognition demo. Anecdotally, among three non-native speakers with different backgrounds I talked to, the demo was guessing the mother tongue much better than native speakers, and it errors were different. This is a sign of the model learning to recognise the wrong things.
[5]: Ever noticed how films almost always cast native English speakers imitating non-English accents rather than people for whom that's their first language? That's why, English phonetic system with sprinkles of phonetic flavour is much more understandable.
[6]: By the way, Arnold Schwarzenegger understands this very well.
I don't believe that phonemes are real. They are a product of centuries long literacy, and the stereotypically "foreign" accent results from people being taught to construct the words from phonemes.
Not all languages can be neatly split into a nice set of phonemes - Danish phonology in particular seems mostly imaginary, and the "insane grammar" of Old Irish appears to result from the fact that word/morpheme boundaries can occur within the "phonemes".
wow always wanted to know an objective measure of my Russian accent in French. I ve been living here for a long, long time and some people tell me it's impossible to recognise where i come from. i d like to put that to test
Did you publish that accent dataset somewhere?
No, the dataset isn't published beyond what you see on the 2D visualization. Sorry.
you may be interested in Mozillas CommonVoice dataset
Does CommonVoice have any kind of tagging related to the accent/skill of the speaker?
Damn, this is really cool.
thanks!
Cool stuff
Is it just me, or did the sound files get hugged-to-death?
Oh pssh. There's no such thing as accent strength. There's only accent distance. Accent strength is just an artefact of distance from the accent of a socially dominant group.
The article defines accent strength in precisely this way, as the difference "relative to native speakers of English".
That group has a vast range of accents, but it's believable that that range occupies an identifiable part of the multi-dimensional accent space, and has very little overlap with, for example, beginner ESL students from China.
Even between native speakers, I bet you could come up with some measure of centrality and measure accent strength as a distance from that. And if language families exist upon a continuum - there must be some point on that continuum where you are no longer speaking English, but say Scots or Friesian or Nigerian Creole instead. Accents close to those points are objectively stronger.
But there is a lot of freedom in how you measure centrality - if you weight by number of speakers, you might expect to get some mid-American or mid-Atlantic accent, but wind up with the dialect of semi-literate Hyderabad call centre workers.
> relative to native speakers of English
> Even between native speakers, I bet you could come up with some measure of centrality and measure accent strength as a distance from that
Is that what BoldVoice is actually doing? At least from the article is saying, it is measuring the strength of the user's American English accent (maybe GenAm?), and there is no discussion of any user choice of native accent to target.
> Is that what BoldVoice is actually doing?
No, I don't think it is doing that, I'm just taking issue with cccpurcell, who seems to believe that any definition of accent strength is chauvinistic.
Indeed, although the inference output of the model is based on the ratings input that we trained it on. And that rating input was done by American English native speakers, so this iteration of the model is centered towards those accents more than e.g. UK or Australian or other accents of English from outside the US.
> Accent strength is just ... distance from the accent of a socially dominant group.
Yes, that is a good definition of accent strength.
> There's no such thing as accent strength.
??! You literally just defined it.
Which "socially dominant group" would you pick in the United States? Or the UK?
Native white middle class, from the South East. BBC accent is fairly neutral these days. If you're American, someone like John Oliver has a very neutral English accent.
I'm not American so I don't want to comment on that.
Sure, that's fair. We apply labels that have a connotation of strength based on the distance, but the underlying calculation is indeed based on distance.
What a silly nitpick. You’re just using different words to say the same thing.