It's not 100% clear to me, from reading TFA, what the actual vuln is.
Suppose DankStartup folds and I, being a morally-dubious sort of fellow, purchase dankstartup.net which I then use to sign into DankStartup's O365, or DankStartup's ChatGPT as a DankStartup employee.
Isn't that a failure on DankStartup's part, to not shut down their business accounts? And isn't it also a failure on e.g. Microsoft or OpenAI's parts, since they're providing service to a defunct business entity who can't pay its bills?
To describe this as a vuln in oauth doesn't really make sense to me.
Well, think about it this way:
1. I create DankStartup and my company uses Google workspaces and Google auth for a bunch of stuff, like payroll.
2. DankStartup goes under and we close our Google accounts/let our domain lapse.
3. Someone else buys DankStartup.com, sets up a Google workspace, and attempts Google auth to log into stuff, and it works.
The problem is that the original DankStartup has a Google account that they create in #1, and Google goes around telling other sites (via Auth) "this is user X from company Y".
Then, the impostors in step #3 create a different google account with the same domain, and Google says "yeah, these are definitely the same guys as before", even though Google is fully capable of discerning that that is not the case; these are different people with a different workspace account, different names, different payment information, and so on, but Google is saying that if you're holding the domain you are therefore the same people as far as they're concerned and is asserting that to other companies. They are (or were) refusing to provide any indication to those other companies that these are not, in fact, the same people, so those other companies aren't even capable of doing their due diligence of extra validation if they want to.
It's similar to looking at a driver's license and just matching the name rather than the actual ID number; it's possible for someone else to have the same name as you, and identity documents have unique identifiers for specifically that reason.
I don't see the problem here.
We've already settled on DNS being the auhoritative authentication key for everything. If you bought the domain then you've bought the corporate identity. It's working as designed.
>Then, the impostors in step #3 create a different google account with the same domain, and Google says "yeah, these are definitely the same guys as before", even though Google is fully capable of discerning that that is not the case; these are different people with a different workspace account, different names, different payment information, and so on, but Google is saying that if you're holding the domain you are therefore the same people as far as they're concerned and is asserting that to other companies.
Yep I understand the mechanism by which this gets abused; I think we just disagree on the implications. I don't work for Google but it seems from the outside that they're treating the OIDC subject claim as referencing the domain attached to the workspace account, or something similar. I've seen implementations where the `sub` claim is more granular, so to me that indicates the field is underspecified.
Given all that, I suppose TFAuthor's proposed solution is a good way forward.
I still think classifying this as an OAuth vulnerability isn't correct.
Traditionally, SAML / OIDC trust is established using public/private keypairs. Each IdP/SP pair gets a unique combination. In this case, a domain changing hands would not allow the new owner to gain access to the old owner's accounts.
In the case of Google OAuth, it's possible to forego this in order to allow any Google user from any Google workspace to login to your application. See the distinction between "public and internal applications" here: https://support.google.com/cloud/answer/6158849?hl=en#zippy=...
Some applications (e.g. Tailscale) take advantage of the public Google OAuth API to provide private internal corporate accounts. A common misconfiguration here is to use the domain portion of the "email" attribute - this can be spoofed by Google Workspace admins. That's not what's happening here.
Instead, Google instructs you to look at the "hd" parameter, specific to Google, to determine the Google Workspace a given user belongs to for security purposes. This field cannot be overridden by Google Workspace admins. The trust breaks down when the domain changes hands, a new Google Workspace account is opened, but the old "hd" value is reused.
You can read more about "hd" here: https://developers.google.com/identity/openid-connect/openid... (find the table and read the descriptions for both "email" and "hd".)
You can avoid this issue by using a custom Google OIDC IdP configured for internal access only in your applications, rather than using a pre-configured public Google OIDC IdP (be very careful you mark it internal!) A new domain owner would not be able to retrieve the secret key you previously generated.
To clarify, any multi-tenant app that has a Google login button that "just works" without you having to set up your own OAuth client creds is using the "public" app option. Effectively the OAuth client side doesn't care whether you use a public Gmail account or enterprise Workspaces account to login.
However, on the Workspace Admin side you can set policies as to whether the org accounts can login to arbitrary public apps by default, or set up an app allowlist, etc. It's definitely a best practice not to let your users login to arbitrary apps, even if it's only for the profile data and not other API scopes.
If you do login to a public app with an org account, how that app decides to group/authorize users from (what it perceived as) the same org/domain could be a sensitive procedure with opportunity for exploit, and it seems like maybe Google should offer a more reliable unique org id claim.
If you are making your own internal app then client is going to be internal only by default. Note this is managed in a GCP project. If you're using Workspace and GCP then you may want to use GCP services with built-in Google auth like Cloud Functions, App Engine, Identity Aware Proxy, etc.
Yea "hd" doesn't work here.
Users can make google accounts with corporate domains but without gsuite hosting attached. The same concern applies to just @gmail.com addys. Google has gotten big on nuking inactive accounts now, what stops similar openings where a person's inactive account gets recreated by an attacker and waltz his way into SSO systems? There's some corporate systems (usually benefits) that allow personal emails attached to the same account as work emails for example.
I imagine the fact the author is writing that Google reopened the bug and is issuing a bounty means there are Google engineers more than aware of the issue rather than trying to argue what Google does and does not intend.
It seems there are two possible problems.
The first is whether taking over a lapsed domain allows you to takeover an existing Google Workspace (or Cloud Identity) organization. This it what houses the corporate email accounts and OAuth client registrations. If Google allows this scenario then the linked account takeover is simply one symptom / side effect among many. TFA is not clear on whether this step actually happened... I assume not, since if it were the case we'd be talking about direct access to the Google account data rather than only linked SP accounts.
The second is when an SP doesn't properly use the `sub` claim as a unique identifier. It sounds like some products don't understand this requirement and why it "seems to change 0.04%" of the time. I do agree that a unique identifier for the org itself would be a good addition to the token.
That said, I'm still not clear how the second problem manifests if the old OAuth client creds (housed in the old Workspace org id) are invalid. Presumably attacker can login to the SP admin account using just email based password recovery, then reconfigure the OAuth integration with new secrets. In that case the SP is failing to do MFA on the email login.
Would love to hear if I'm missing something.
> That said, I'm still not clear how the second problem manifests if the old OAuth client creds (housed in the old Workspace org id) are invalid.
As far as I understand, this is not a necessary step. The SP is configured to trust Google's public OAuth IdP, not a specific Google Workspace account. So there are no special secrets shared between the old Google Workspace account and, say, Slack. The Slack org trusts any user that Google's public OAuth IdP says is a valid user in the example.com domain. Slack doesn't have to do any MFA for these accounts, they trust Google did that already.
Now, you may not be able to access the Slack org admin account in this way, say to add/remove users or delete the org. But you can access all of the other information that any random employee in the org could access back when it was setup, including a list of all other users in the org.
Ah, right, Slack can have their own public oath client which is used for the code grant.
So what happens is: 1. New Workspace org created with same (old) domain name 2. Same domain name is sent in `hd` property, existing email address sent in the `email` property, new uuid in the `sub` property.
If the app is only matching on email instead of sub, then it will grant access to previous user data. Additionally, even if it makes a new user based on the new sub, it may still grant access to other SP resources associated with the existing domain based on the email address or hd value.
Instead there needs to be something like `hd` but uniquely identifying the Workspace org entity itself, not just the domain.
>>>The problem is that the original DankStartup has a Google account that they create in #1, and Google goes around telling other sites (via Auth) "this is user X from company Y".
Google is telling other sites that it's bob@DankStartup.com - isn't that true? Isn't this on DankStartup to close down operations cleanly?
Login credentials are more than just the email used, and should not be conflated with identity. To be honest I'm surprised google isn't using a uuid tied to the account for the identity given to others, especially since internally google knows and treats it as a separate account that just happens to have the same e-mail. The e-mail should only be used as metadata for contact info.
It's a different bob@DankStartup.com, and in fact a completely different DankStartup.com. Google shouldn't conflate the two.
There are exactly 0 situations where the current behavior is useful. There is no reason whatsoever to have the exact same auth info for two Google accounts that happen to have the same domain.
>>>It's a different bob@DankStartup.com, and in fact a completely different DankStartup.com. Google shouldn't conflate the two.
Does Google have any reliable way of knowing that?
>>>There are exactly 0 situations where the current behavior is useful. There is no reason whatsoever to have the exact same auth info for two Google accounts that happen to have the same domain.
I have a personal google workspace account with a few domains. At some point I might want to spin one off to be its own (maybe I start a company). But I'd still expect pat@mydomain.com to keep working throughout. So that's 1 situation.
Whoever has access to the inbox and account for bob@DankStartup.com is bob@DankStartup.com. If Google was being asked if the SSN for bob is 123-45-6789 and they were saying yes, then I would agree that's an issue, but all Google is saying is "this person can authenticate to our services as bob@DankStartup.com" and that is true.
Part of my company dissolution process is to renew the domain name for 10 years to prevent exactly this
Is the hope that this sort of attack is just less useful in 10 years? What happens after 10 years?
Presumably after 10 years of failing to collect on their invoices, Microsoft would have killed your O365 account, so there's one fewer SaaS account left to log into.
Yes
Yes, this. I also use a service to capture all emails (catch all) so that I can detect any loose ends that might have been overlooked. Services like ForwardMX or Cloudflare can do this for you at relatively low (or no) cost.
Sure, but DankStartup failed and doesn’t exist anymore. If I am just a lowly employee, I can’t force the failed startup owners to properly shutdown, and now my payroll information is available to hackers.
What is my remedy?
Probably sue the failed startup owners for negligence?
If DankStartup left all your HR files lying around in their office and the guy who bought it five years later found them, well, DankStartup should have shredded them. It'd sure be nice if the cleaners shredded them for you, but I wouldn't count on it (nor make them liable).
In a rational world the remedy would be to sue Google for exposing your payroll information.
> They are (or were) refusing to provide any indication to those other companies that these are not, in fact, the same people
That is not quite true, the sub field will be different.
This is not necessarily useful. The sub field only indicates that this is a different user, which maybe protects the private info of the old user. However, a big part of OIDC integration is to automatically allow any valid user registered with the IdP to automatically have access to the corporate account, and to any company-wide resources, which can still include very sensitive information.
But authors does imply that sub will also change in place for users in step #1, without the workspace beeing recreated. And as such the sub is not usable as a general identifier for the user resource differentiation.
The sub property appearing to change for the same email address is a valid scenario. SPs failing to respect that scenario because they don't understand it, or because it's not what some of their users want, is not a valid excuse.
https://support.google.com/a/answer/33314?hl=en&co=DASHER._F...
To me it is reasonable that orgs may want to eventually reuse an email address on a different user account. That's a feature decision made by the IdP so SPs need to respect it. I believe other IdPs like Okta and Entra have equivalent features too.
> To me it is reasonable that orgs may want to eventually reuse an email address on a different user account. That's a feature decision made by the IdP so SPs need to respect it
I think everyone, including the authors hopefully agrees with that logic and sentiment. And that would be the literal point of the sub claim after all!
But the implication in the article is still, as i read it, that it changes in place in practice, and not in the case of re-creating the user under the same workspace. But i obviously do not have the background to clarify!
If by "same people" you mean recognizing a specific user, yes the sub field changes.
If by "same people" you mean being able to tell whether a new user is part of the same organization, the sub field is useless and no other field has this information either.
Any domain takeover allows email takeover which allows you to send password reset emails for former employees. Does not matter if it is with oauth or not
And many vendors will send "restart your service today for $x" for months afterwards so data not deleted
Some SaaS ecommerce platforms and email marketing services will likely give a restarted domain entire customer databases ...
There should be some service for permanently destroying a domain with a single prepaid cost. Somehow preventing bankruptcy laws from getting the domain.
Pay X$ up front, then Y$ per month to keep active. Once you cannot pay, it gets blocked forever (paid for by the up front cost). Owned by your service provider so not part of your firesale
What happens when your service provider that owns your domain collapses?
Those password resets should have some sort of MFA step.
Sue lost hee phone, she used KeePass for everything and didn't back it up. She get some support from IT, what do they do to solve Sue's problem?
They reset her password and 2FA and have her redo them. She probably gets a lecture about backups or she spurns a brand new company policy that "Everyone should now use LastPass and nothing else is supported".
If they as administrators cannot do that, Sue has now lost significant business data, there will be a dexent amount of work stopped to get Sue onboarded again and this is a significant issue.
An auditable log that X reset Sue's password ane 2FA codes at x time while at x location with biometric authentication is pretty secure. If X also ca nnot touch those logs the next strawman falls apart.
What does this have to do with the scenario above? Of course you can reset the corporate SSO account as many times as you want. The point is that federated apps with a password reset flow should have some sort of MFA.
It defeats the purpose of the first S in "SSO": it's not a Single Sign-On, and for Sue in GP's example, after losing access to her MFA, she can't access federated service data, nor can the domain operators restore her access.
Yes but he's saying the data on 3rd-party websites should be deleted by the failed startup when they shutdown not just left to sit there.
He's right, but having failed a number of times, after you've put all your savings into the business, fired all your team, notified and disappointed all of your investors and customers, helped your team find new gigs, filed all the dissolution paperwork, handled all the taxes, disposed responsibly of all of the assets and you now find yourself out of work and often out of cash, occasionally you'll forget to jump through all the hoops to close down all of the SaaS accounts before you stop paying the bill personally to host the email accounts.
Of course perfect world you shut down earlier and in a more orderly fashion, but there are so many cases of companies almost failing and then not, it's hard to shutter a company when there is a chance you may go out of business - especially when you feel you're getting close to another raise or becoming default alive :(
There’s an interesting question on service providers responsiblity who use Oauth for primary auth.
Should they reset all access to a company account if a domain transfers or becomes publicly available?
There are some 3rd party accounts that can be accessed via your SSO or via your personal credentials once you leave. The main ones I can think of is your brokerage account containing your 401K and vested RSUs and your payroll provider like ADP and Paylocity. You still need to have access to past paystubs and end of year tax documents.
SSO should stop working when the IdP org is disabled/deleted. IdPs should not allow the org to be resurrected based solely on domain ownership alone. And if a new org is created with the same domain, the SP will need to be reconfigured with new OAuth client creds, and should be relying only on the `sub` claim anyway.
Any accounts you need after leaving a company should be tied to your personal email.
My brokerage account could be accessed by both. I agree that is how it should work. But my brokerage account provider is never told to disable access via my IdP. It’s up to my former IdP to not do something stupid like giving someone else my old email address.
It's not IdP's responsibility to "fix" the internet.
Just like someone can buy a home and will get mail targeting a previous owner, the same happens with domain names and emails.
Domains are, however, much cheaper and more abundant, so it happens more often.
I agree, I don't think this is a problem with Google's Oauth implementation, it's a problem with the service providers who authenticate users via the mere existence of an email address ending in @company.com without checking if the email address actually belongs to an active employee.
If, when you logged into Slack via Google Oauth with the email address user@company.com, Slack checked with company.com whether user@company.com was a valid user that should be allowed to login, then this problem would be avoided entirely because the defunct company would no longer report any valid users.
This would also avoid further problems with attackers being able to login with unattended email addresses like support@company.com, as was discussed here: https://news.ycombinator.com/item?id=41818459 (There's a lot of discussion below about whether the "sub" claim is stable or not but it's a red herring because of this IMO, also the proposed fix in the article wouldn't address it either.)
If, when you logged into Slack via Google Oauth with the email address user@company.com, Slack checked with company.com
By looking the account up with Google's People API - https://developers.google.com/people
They would have to verify the account is active, AND the id hasn't changed
By looking the account up with Google's People API - https://developers.google.com/people
They would have to verify the account is active
AND the id hasn't changed
If, when you logged into Slack via Google Oauth with the email address user@company.com, Slack checked with company.com whether user@company.com was a valid user that should be allowed to login, then this problem would be avoided entirely because the defunct company would no longer report any valid users.Right, this additional check should not be necessary in a typical OAuth or OIDC flow. This workaround is only necessary in this case because the API Google offers to services has a hole in it.
Either slack or some other third party provider could have a whitelist maintained by IT?
Just spitballing.
It is certainly an option to pre-provision accounts in your application (e.g. Slack) and then have any users authenticating from the SSO product compared against the list of authorized users. And for some products which sell licenses by seat, this is exactly what they do.
But for many products which are meant to be available to an entire organization, this is a big part of what SSO was supposed to solve in the first place: IT no longer has to provision (and de-provision) user accounts in every single application. Maintaining an allowlist in each application makes this pointless.
This is a misunderstanding of the problem. See my comment downthread: https://news.ycombinator.com/item?id=42701912
"hd" is Google's solution to this problem, and "hd" is also the source of this vulnerability.
I don't think so, but please go ahead and clarify if that is the case.
Maybe you can clarify what part of the linked comment didn't make sense? It's a bit hard to make out where the confusion is if the above comment didn't help clarify.
First of all, the whole point of SSO is that the only and final source on who are the valid active users is the IdP, in this case Google's public OAuth instance. If the IdP says that the current request is coming from the real user1@example.com, you give them access. And Google's public OAuth instance will confirm this even if the current Google Workspace associated with example.com is a different one than a few months ago (though the "sub" field of the attestation will be different than before, which the service provider is supposed to check).
Second of all, even if it was recognized that this is a different user1@example.com, they'd still have access to all sorts of company internal resources, that may still contain sensitive data, especially for small companies which inherently trust all employees.
If the attacker is in control of company.com, checking against this domain would not help.
I'm not talking about checking against the domain, but checking against a directory of active users.
Where does this “directory of active users” exist? If it is controlled by slack, then you are relying on a failed startup to properly notify ALL the 3rd parties when they shut down. Failed startups don’t always shut down cleanly like that.
> Failed startups don’t always shut down cleanly like that.
Agreed, and with the number of services and the "ease" of oauth it's likely impossible to even track. You could make a list of the major ones, but there could be hundreds per user, ultimately thousands of unique services used depending on the breadth of the startup's activities.
That's what an identity provider (e.g. AD, OneLogin, Okta, Duo SSO, Google OAuth, etc.) is supposed to be, ostensibly.
Yes. If you've set up your Slack so each login checks against the identity provider to ensure an active user is logging in, that would resolve the issue, no?
Even if you take over company.com's domain you can't reconfigure company.com's Slack to point to a new identity provider?
I think you may be a bit confused about the players here. When you use Google OAuth to login, it _is_ your identity provider, and it is reporting to Slack that the user exists. Google is reporting the user exists because it exists in the Google Workspace directory. You use this as your source of truth for provisioning users, and they automatically get access to all of your company's apps.
The problem is that even though the user has the same email (joe@example.com), and the same Google Workspace domain ("hd": example.com), this is actually a _new_ Google Workspace account. But nothing Google provides to Slack allows them to detect this.
Slack, et al can fix this by _not_ using the public Google OAuth integration, and forcing every use to configure an individual internal Google OAuth integration. But they use the public one because Google has said it is a safe and secure way to operate their service.
What I'm suggesting is if you were able to pre-configure Slack to only allow logins for valid users from Google Workspace X, then even if someone creates a new workspace Y with the same domain, Slack would still be checking against workspace X. (And similar for non-Google based identity providers.)
And people are telling you that this is not possible with the Google public OAuth API. When Slack asks Google's public OAuth instance if user abc@example.com is valid, Google checks with the Google Workspace associated to example.com, and returns to Slack a response saying "yes, that user is valid, here is more information from example.com". This can be the same Workspace or another one, Google isn't really telling Slack apparently.
Now, there is another field called "sub", that should be a unique ID for the Google Workspace or the specific account, but it seems that this is not always stable, per the article, so people integrating with Google OAuth don't trust it.
Where would this directory be stored?
Yes, why the Slack or HR or interview etc. data would still exist while being inaccessible to the original owner is very strange. The article seems to take it as a foregone conclusion that those accounts should all be expected to still exist with all the business data, and I don't know if this is based on the author's experience, or it's just a way to make the vuln sound far more serious than it really is.
On the other hand, I don't think people normally expect that OAuth depends strictly on domain ownership like this. I think most would expect that it depends on some kind of secret being stored on the IdP account side, uniquely identifying the IdP account holder beyond the email addresses presented. With regular password-based authentication, with MFA, you would at least get an MFA prompt if someone had gotten access to your email address and was being sent a password reset code. But with this type of SSO, any MFA verification is done on the IdP side, so if the IdP recognizes anyone who controls example.com as the rightful owner of any claims, then there is nothing you can do as the former owner of example.com.
>why the Slack or HR or interview etc. data would still exist while being inaccessible to the original owner is very strange
Ever been in a company that's collapsed before? Nobody hangs around to shut down the WorkDay account when they've been told they're not getting paid for the last 29 days work.
Yes, it's a failure on DankStartup's part.
Not really much different than a user buying dankstartup.net, setting up a catch-all email, observing what comes in, and performing password resets for those accounts, allowing for account takeovers.
Calling it a vuln in oauth may be a bit hyperbolic, but Google could help prevent it.
I have catchall email accounts in every domain name I own (mostly so I can do differentiated emails for every service to track/combat leakage), and you would not believe the amount of emails I get that are intended for previous domain owners (and typos too). I haven’t actually done any reset password flows, but there are a bunch of social media and SaaS accounts I could easily take over if I wanted. I used to try to track down whoever the emails were intended to go to and forward it to them and let them know to change it, but that got to be too tedious so nowadays I just ignore them.
Still, I wouldn’t call this a vulnerability on the service provider’s part, it’s just user negligence.
> Suppose DankStartup folds … Isn't that a failure on DankStartup's part, to not shut down their business accounts?
Expecting an entity that has already failed* to not fail again isn’t an effective security control, unfortunately.
* - not every startup that folds has “failed”, but the point still stands
>Expecting an entity that has already failed* to not fail again isn’t an effective security control, unfortunately.
Sure, I'm sympathetic to that, but again I don't see how that's within the scope of oauth.
I register a Google Workspace and add CorpDomain.com to it. I then use that to OAuth to other companies (e.g. Slack, payroll companies, etc.). Then my company goes under or closes up and the domain lapses.
Someone else comes along, registers a completely different Google Workspace but attaches that same domain to it. The e-mail address is the same, but it's obviously a new Google Workspace with new people, new payment info, new users, etc.
Google knows that these are two different workspaces and that there is effectively no connection between the two other than the domain, but they are not presenting that information through OAuth (which is possible) so other companies are not able to do any sort of diligence in ensuring that the correct people are accessing that data.
OAuth provides the capability to make this distinction, but Google is (or was?) refusing to provide data to other companies to allow them to make that distinction.
This sounds hyperbolic, but Google is effectively lying to these other services that that someone else is in fact the original person that service expects, even though Google knows full well (or is capable of knowing) that that is almost definitely not the case.
I'm not entirely clear on what you expect in this case?
You're registering with those 3rd parties using a property (the email address under corpdomain.com) that is now owned by the new party.
This feels a lot like complaining that you hired a lawn service and told them to mow at your address, and then didn't update the address or cancel service after you moved.
You've sold the domain. Assets associated with the domain are under the control of a new party. For all Google knows, you did this entirely above board and in a coordinated fashion.
That new party controls the property. Email resets will also dump right into their hands (They control the MX records for corpdomain.com now...).
Legally speaking, it's not even clear you're right - the new person might well be the person actually entitled and expressly supposed to be accessing that service as that account (if the domain was sold as part of an acquisition or sale).
> Legally speaking, it's not even clear you're right - the new person might well be the person actually entitled and expressly supposed to be accessing that service as that account (if the domain was sold as part of an acquisition or sale).
No, this is absolutely not the case. If you were selling the identity, you would transfer access to the Google Workspace account. There is no reason whatsoever that a new Google Workspace account should have access to the same services as a completely different Google Workspace account just because they happen to use the same DNS domain.
> I'm not entirely clear on what you expect in this case?
Not parent-poster, but from what I can piece together it sounds like the distinction between Google attesting that:
1. This is the person who currently controls the same e-mail address.
2. The is the person who currently controls a sub-account of the same Google Workspace which was in the past identified by this e-mail address.
> You've sold the domain. Assets associated with the domain are under the control of a new party.
True, but that "control" does not (and should not) automatically mean legal ownership of every kind of internet account that was ever made with that e-mail address.
While the prior-controller has a degree of responsibility, so does Google when it knows that ownership of those other facets were not transferred.
> This feels a lot like complaining that you hired a lawn service and told them to mow at your address, and then didn't update the address or cancel service after you moved.
I feel the analogy need at least three kinds of party: The old/new home owners, the lawn-cutting service, and a marketplace or middleman which is failing to pass along the change in ownership.
The other reply to this comment kind of nailed it, but in short:
Right now Google, via OAuth, attests that someone's identity is "user@domain" as a unique global key, even if they know that that user@domain is controlled by a new Google workspace (i.e. it is not the same people/organization/legal entity).
All Google has to do is attest that someone's identity is ("user@domain", "unique workspace ID") as a unique global key and all of this will go away.
> This feels a lot like complaining that you hired a lawn service and told them to mow at your address, and then didn't update the address or cancel service after you moved.
I think it's more like hiring a lawn service through a third party, and then, after you move, a second person moves into your house and calls the lawn service asking for a copy of all of the credit card receipts from your payments and they give it to the person because the third party assures them it's the same person, even though it's not, just because the address is the same.
> Email resets will also dump right into their hands (They control the MX records for corpdomain.com now...).
Well yes, and people shouldn't use e-mail/password authentication for that and many other reasons (unless 2FA is involved).
The difference is that, in this case, we are outsourcing authentication to Google under the assumption that Google will handle authentication better than just a username/password combination; I'm relying on Google to say "yes, this is who that person claims to be and we have verified that". The problem is that Google knows that they is not who they claim to be, that they just have the same e-mail address but a different workspace, but they're still telling the other site that yep, it's him for sure, absolutely, and the other site has no way of validating that any of that is true so they have to go with it.
> Legally speaking, it's not even clear you're right - the new person might well be the person actually entitled and expressly supposed to be accessing that service as that account (if the domain was sold as part of an acquisition or sale).
Potentially yes, but if that were the case then they would have transferred ownership/control of the workspace itself, including all of the accounts, email, google drive data, and so on, and not deleted the entire workspace and created a new one from scratch. In a case where they did want to do that (e.g. to migrate off of Google workspaces or consolidate) that seems like something they should do via customer support to ensure continuity.
Seems like relying on domain alone is a design flaw of OAuth?
OAuth allows you to provide information above and beyond just the domain, and the article describes examples of how this could be done. This is just a flaw in how Google presents an identity to OAuth clients.
In other words, right now it's basically "user@domain", but it could be ("user@domain", "unique workspace ID") which would prevent these sorts of attacks as well as make logical sense.
If a security mechanism doesn’t account for failure cases, it’s a failure of the security mechanism.
It’s a hard problem to solve and I don’t have a solution, but it’s a core goal of every security tool to account for edge cases and failure cases like this. If you tell me that OAuth is completely insecure due to a security issue, it’s not going to make me feel any better if you say “but it’s totally not OAuths fault” - I don’t care who’s fault or scope it is, the end result of a security issue is the same, and to avoid it I’m just not going to use OAuth.
So you use email/pass and the reset password email dumps right to the new party as well, because they control the MX records for the domain?
That's why allowing account recovery using (exclusively) email is indeed a security problem.
It's similar to Gmail or Hotmail or Yahoo Mail hypothetically allowing a new user to adopt an abandoned email address. Most email services don't allow this for this very reason.
The scenario I was envisioning was a simple as a developer that worked at that startup added DankStartup email to their Github account, committed a bunch of code. Company shut down, but dev doesn't think to disassociate that domain/email. Malicious actor uses this technique to then sign into the Github account, gaining access to that developer's account, not just the DankStartup repos which were probably disabled after non-payment.
Now, obviously some fault in that scenario lay with the person who 1) used the same account, and 2) didn't remove the old email once the startup failed. But I'm just using that as a kind of example - there may be other accounts as others have said that need to be accessible years down the line, like financial records and the like, regardless of whether the company is still around.
> DankStartup's O365,
If you wanted to try to re-take over a pre-registered domain in Azure/M365, you would need to involve Microsoft Support who would require you to prove via government documentation that you were the same entity for the now-abandoned Azure/M365 account.
Creating a new tenant using the same public domain won't get you access to the Azure/M365 tenant of a previous customer, you would have a different internal domain name (dankstartup2.onmicrosoft.com).
At best you could register your domain post-expiration of the previous tenant and impersonate them >90 days out.
Is this basically equivalent to registering a domain, setting up email, and getting services to send you password reset emails to recycled addresses?
Besides the github account with a legitimate name, but impostor ownership, I can imagine another case which is problematic. HR SaaS companies might be required by law to maintain the account for a period of time - let's say until the end of the next fiscal year. The account still existing is not a failure on either Google, the SaaS company or DankStartup. The problem, as other posters mentioned is the fact the new account has the same ID in OIDC as the previous one, which is what the author of TFA proposes.
Third parties is google as single sign on. If such an org/domain is abandoned, a malicious person could 'resurrect' the accounts on those third party sites. It's kind of like a similar situation where a user deletes an email account and then someone else creates an new one in its place, and now they have access to all the third party password reset links. Except in the case linked above, the user never had the ability to delete their accounts before off boarding, and the abstraction isn't just an email, but the account itself.
In emails case all the larger providers I'm aware of retire addresses to prevent this sort of abuse, probably learned the hard way. Could Google OAuth do something similar here?
It now occurs to me that large businesses might have the same issue with emails and former employees (easy to imagine John Smith departing and at a later date John Smith being hired) so I wonder how they typically handle this.
Companies generally don’t reuse emails. If you come back, you can sometimes get your old email and access to your old enabled services.
I know when interns were at Amazon and left, their accounts were disabled. But when they came back, they got their old accounts back and the related Slack users
What if DankStartup doesn't fold and just forgot to renew their domain, or lost control of their domain for some reason. OAuth shouldn't let you Auth someone else's account just because you can hijack their domain.
I don't think it would be 100% on the failed startup considering most of, if not all, of the applications mentioned likely do not actually delete accounts even when you say to delete them. There probably could be some allowance made for the situation but also you're repurposing the domain of a failed startup.
I don't think the problem was necessarily with Google though as I believe they create a unique ID for the domain. The problem with be with the other services like ChatGPT, Slack, etc that associate with the domain and create accounts for each user in the domain.
My father changed careers and dissolved his business over a decade ago, at which point he let his old domain lapse.
A couple years ago, I began to wonder if anyone was still trying to reach him, so I bought the domain (domain squatters weren't even interested in it anymore) and setup an inbox for his old email address. In less than a week, the unread count was in the triple digits. It was basically all marketing and spam so I closed the inbox within the month, but multiple years with a 100% bounce rate is apparently not enough to deter LexisNexis & friends from trying to win him back.
This is a corporate planning/governance problem, not a vulnerability in Oauth, and I bet the problem is even bigger than TFA describes since any previous domains of a company that were almost-but-not-completely migrated away from could easily be taken advantage of, possibly to even greater effect since the company likely has a bunch of systems still up which have the domains whitelisted. (Which makes me wonder if there are hacking groups who use domain squatting as a front or side-gig while they take advantage of lapsed corporate domains.)
This is very much a critical flaw in systems which rely entirely on email or domain names as the sole security factor. OAuth itself is flawed in that it allows this terrible practice, though to be fair it does discourage it. Google's specific OAuth solution encourages this compromised practice even more, because the only identifier it provides for a user group is the domain name associated with the account (in the "hd" field). For individual users they at least provide a "sub" field with a unique user ID (though per the article, it seems this is possibly flaky, or at least misunderstood) - this could be used to make sure that a new user@example.com doesn't get access to the old user@example.com account. But to check if a new user should have access to the organization account, there is nothing that Google provides that can be used securely: all they tell the service provider is that this is a new valid user for the example.com org, not that this is a completely different example.com org.
That's a really good point. Perhaps the protocol should be expanded with an explicit identifier unique to an extant legal entity (or string of identifiers, to account for ownership changes like acquisitions), so it would be easy for SSO-enabled applications to keep previous tenants' data safe from someone who just buys the domain later on.
Of course, this would still rely on the provider: it might be a great solution for large providers like Google which can implement ironclad formal verification procedures, but if you and the previous company self-hosted SSO, you control the response and can impersonate them completely.
And not only that, almost every form of auth has the same vulnerability not just log in with Google. If you own the domain, you own the email IDs as well and you can very likely reset password.
Yeah, this is the part I'm struggling with. This is absolutely not unique to google oauth, it genuinely seems like a misunderstanding of how the web manages trust.
If you own the domain, you own all the property associated with the domain, including all the old email addresses. Magic links and password resets are all going to give the new owner access.
Your best bet as a solution is to be using strict 2fa (ex - a yubikey might help here) but even that is likely just "a conversation with support" away from being circumvented.
This is why winding down a company is supposed to have specific stages and policies associated with the dissolution. You don't just abandon the offices and leave all the filing cabinets behind either, for similar reasons...
I don't think the claim is that Google has introduced a totally novel vuln, just that they are a stable, trusted middleman who is failing to mitigate a common vuln where they can.
You don't abandon filing cabinets, but given that some percentage of startup failures are sudden and surprising and the people who could do something are unmotivated / unable to, it's not a best practice for a commercial landlord to put abandoned files on the street with a sign "free", files and all. Maybe they have a legal right to, but it's not how I would operate in that situation.
It seems to be at the registry level. But registries do show changes in owner [implicitly at least].
Maybe there needs to be a way to do some signalling/lookup when domains changes hands.
Sounds like a job for a blockchain, lol.
> Magic links and password resets are all going to give the new owner access.
Which is why you should never exclusively rely on either for sensitive services.
It's better than SMS based password recovery or "call support to reset your password". What do you suggest is the best approach.
The contention isn't that you can impersonate DankStartup, that's obviously not a vulnerability since you are indeed the domain owner. It's that former entities with DankStartup accounts might have used OAuth to create relationships of their own. And when the startup folds, they don't magically disappear.
Basically if Sally, the CTO of DankStartup, signed up for Taskrabbit or whatnot, it's possible for you as the owner of the domain to impersonate Sally in the context of that relationship.
Obviously the root cause here is that someone misused an account to do something not related to the business. And the actual impact is probably low since high value services tend strongly not to take tiny email domains as identity roots (i.e. sally@dankstartup.com clearly doesn't have a Vanguard account to steal).
So... like most security announcements it's oversold and spun. But it's real enough as I read it.
Why do you assume sally@dankstartup.com doesn’t have a vanguard account? I’ve absolutely had similar retirement account logins that became difficult to access once I left that employer. Had to contact HR and get them to help me log into my account. If the company had folded during that timeframe I would’ve been screwed. Of course for financial institutions you can probably recover your account through some identity proving process, and generally money transfers require a second factor sms auth, but a domain takeover would probably have been sufficient to at least get someone logged in and able to see my account balance.
> I’ve absolutely had similar retirement account logins that became difficult to access once I left that employer.
Yes, but your employer's email domain wasn't repurposed. It's still an operating business, and the account is still there, and still yours. And for accounting reasons they still remember who you are and can recover your important information for creditors, etc... The fact that it's difficult for you to access that account is a feature and not a bug, because it means it's difficult for others too. But you can, and people do all the time.
The case here is failed businesses who have abandoned their domains and employee account history. Almost by definition that's a violation of financial regulation already. And this, btw, is the reason why DankStartup employees couldn't use their work emails to open Vanguard accounts.
> Basically if Sally, the CTO of DankStartup, signed up for Taskrabbit or whatnot, it's possible for you as the owner of the domain to impersonate Sally in the context of that relationship.
I don't think this is the issue, unless someone went to some pretty extreme lengths. Configuring OAuth such that the company Google Workspaces account is recognized by Taskrabbit as a valid SSO option is not as simple as signing up to Taskrabbit with your company email instead of personal email.
Even then (in my experience) it's pretty common to setup an email based account and it will auto link to an oauth one that already exists. Even if Google revoked oauth, many platforms let you use the email directly to login
Yes, but that's not a bug with Google OAuth. If Taskrabbit (in the example) decides to trust dankstartup.com emails as a root of trust for identity even though the business failed, that's on them[1], and certainly nothing Google can have prevented.
The contention in the article is that dankstartup.com's new owners can leverage their control of the domain to get access to existing OAuth-based sessions that will look to the service providers as if Google has authenticated the account holder.
[1] And just to repeat: that risk is precisely why serious businesses (Vanguard in the example) don't allow obscure email domains as authentication anymore.
That sounds like a TaskRabbit vulnerability (in your example), not a Google vuln? It is also a vulnerability in any email based sign in, which relies on email alone without a password to demonstrate account ownership. (Including password resets that rely on email).
> a failure on DankStartup's part, to not shut down their business accounts
They seized operations, why should they be responsible for any other assets?
I mean ethically yes of course. But there's no law/obligations to make sure domains are closed.
it's vital to have extremely skilled and nuanced people managing bug and security report triage because it seems many can overlook the complexities being given.
whats is TFA ?
"The F-ing article" (derived from old internet slang "RTFM", for "Read the F-ing manual", a thing that new users were prone not to do before getting online and asking questions easily answered by RTFM.)
thanks
"The f___ing article"
In polite terms it can also mean “the featured article”. And IMO it makes more sense to read it that way in a top-level comment.
Whereas reading it as the f’ing article makes more sense when the word “TFA” is used in a response to another comment like “well, you say that but TFA said such and such”.
Agreed. I've also seen "The Fine Article"
This is the old school “polite” version, as in, “read the fine manual”
thanks
I really don't understand here, the proper way to use Google's OpenID implementation to authenticate someone is to use the `sub` claim. Don't use the email, don't verify it yourself, use the `sub` claim. It's a known fact and is properly documented[1].
If the `sub` changes, it's because it's not necessarily the same person so have a flow ready for that. It could be an employee left and came back, a domain change, an IT error that lead to a reprovisioning of the user, etc.
I also fail to see how the proposed solution of having a 'A unique user ID that doesn’t change over time' is different from the `sub` claim. However, the new ID associated to the domain could make sense to enforce a strong 'Everyone from the @domain.com has access' statement.
[1] https://developers.google.com/identity/gsi/web/reference/js-...
There's a reason this happen though. Whether you like it or not and regardless what the standards say, people want a technology that matches with the identifiers they want to use (eg email) and not whatever opaque value happens to be in the sub claim.
I don't like blaming users for wanting something sensible.
It all depends on how true this statement is:
> “The sub claim changes in about 0.04% of logins from Log in with Google. For us, that's hundreds of users last week”.
I've been working with an app that uses Google to login for the past 10 years, and I've had problems with sub changing when these situations happened : - Domain change - Company being bought by another one and being integrated in their Google Workspace - Employee leaving and coming back
To us, it's very very far from the quoted 0.04% which is to me very high. I had to deal with it 5-6 times in the past 10 years but of course that number will vary depending on the usage of your app and I'm not gonna venture and put a percentage on it.
In my opponion, all of those cases very well justify a manual check, or some sort of extended identification before the user is let in.
It indicates a deeper cultural issue of "convenience/profit over security" if those are sufficient reasons to not check the sub parameter.
> all of those cases very well justify a manual check, or some sort of extended identification before the user is let in.
Just curious, what would that check look like that's not open to the same vuln?
"Your account seems to have changed hands and is locked for your security.
The person paying for your subscription must contact us to verify your account is still legit."
Right, and how would you further verify “the person paying for your subscription”?
Payment info
Exactly. How is there an entire alarmist article and 165 comments on this thread. This comment, and it's legitimacy/factualness, is the only thing worth discussing.
`sub` _IS_ the immutable reliable identifier. If it's not, (1) I want to see actual proof, not an anonymous rando (sorry, but this thread re-inforces how little I trust context-less comments like that), and (2) I'd want to hear a convincing argument that ... `sub2` would actually be less mutable.
Threads like this make me really question other peoples' general comprehension skills.
every trufflehog post I've seen on hn has been alarmist clickbait. could've been an opportunity to discuss security tradeoffs of `sub` vs `email` and how to handle `sub` changes, but nope their take is "sub doesn't fix the problem we found"
0.04% is a few times higher than I'd expect it to be, but if it were actually that bad I'd expect some form of previous report about it to be findable on the internet. I did some searching but couldn't find anything. It would be a significant bug on Google's part, but stranger things have happened.
What's astonishing to me is that apparently all these big service providers did notice, and then they decided to disregard the one identifier Google tells them to use? Fundamentally that's the security bug being reported here, it's just being reported to Google instead of those service providers.
A stable alternative for the hd claim would of course be a good idea. It would provide a more complete way to deal with the inherent security issue of allowing domain-based signup without further authorization steps. But given the above I'm not convinced these service providers wouldn't start ignoring it after the first time somebody re-registers a domain with Google Workspace.
I am pretty confident that the statement is false.
The `sub` claim value is equivalent to the user ID that's exposed in the Directory API, it's derived from the underlying user account's unique user ID, and it won't change unless the user account is recreated.
I've implemented OAuth flow glue logic a bunch in my career and I agree that the logic doesn't seem to follow from what the documentation says. What's not clear to me though while reading this is, if I delete/lose a domain, and someone re-makes the domain and account with the same email address, does the sub value then change? The docs would seem to suggest yes, but then I am not clear what google actually ended up "fixing" in its bug bounty. Was that maybe the bug, and this is just not clearly explained in this post?
The reason this is an issue is that the sub value changes often enough that intergrations ignore it rather than bother users with having to re-OAuth.
As far as what Google is doing in the bug bounty, that's a good question -- we don't know. The author is proposing two new values, for the domain and user, that wouldn't change in the cases that sub changes now, but would change in this case.
Yeah, I'm not really clear on the vuln. It sounds like clients don't validate sub claims because they are unstable? Google's docs say they should.
When I first read the post, I thought that maybe Google was reusing sub claims when they shouldn't, but on a closer read, maybe it's the opposite problem - that the sub claim is changing when it should not change?
From the article -
“The sub claim changes in about 0.04% of logins from Log in with Google. For us, that's hundreds of users last week ... Because the sub claim is inconsistent, it cannot be used to uniquely identify users - leaving services reliant on the email and hd claims."
So maybe clients aren't using the sub claim because validating "email matches && sub matches" breaks real users who are not actually changing accounts? AKA, Google is changing Google Account IDs unpredictably? That would be a bug on Google's part IMO.
If those account IDs in the sub claim are changing, at the very least that's a bug in the doc you linked IMO.
I'd have to read the full OIDC spec [0] to see if the sub must remain constant for the same end user, and if clients are required to validate the sub claim before processing a login. If so, and it's changing without the end user's identity changing, that is a bug in Google's implementation and client implementations for anyone using Google as an IDP.
If it's allowed to change, though, and validating sub claim is not required in the spec, maybe clients are correct in not relying on it.
[0] https://openid.net/specs/openid-connect-core-1_0.html#rfc.se...
Edit: from reading the sign in with google overview, it's unclear if they claim to be OIDC compliant: https://developers.google.com/identity/gsi/web/guides/overvi...
it says it's "based on OAuth" rather than "implements OAuth+OIDC" so I guess not being fully spec compliant wouldn't necessarily be a "bug."
I ran into this when implementing OIDC at work. One of my teammates wasn't able to login even though he just linked his google account. Turns out based on which browser session he was using, the same google login was giving a different sub. Super weird behaviour.
very skeptical about this - I believe they are two accounts.
It is possible to create two google account linked with same email address.
I think I need more proof they are actually same account. For example, changing the profile pic in one session is reflected in another session.
Genuine question, what would that flow look like?
This is more of a fundamental issue with the way we rely on DNS to secure *gestures wildly* all the things. The fact that domains can expire and be registered by someone new allows the new owner to do almost anything the old owner could have done when it comes to anything trusting email addresses, or anything else relying on DNS (ACME certs) for authentication.
It's great for "do they own this right now" validation and that's what we use it for, but beyond that links will be saved, email addresses will be added to databases and address books alike, and that's more or less a reality of most systems. For example, my snail mail (and occasionally packages) is still mistakenly delivered to my former addresses from time to time just because it's hard to track who has that address, and update it everywhere. The same goes for internet infrastructure.
I was able to take over an Instagram account because I received a password reset email to a domain catch all address that I'd set up years ago. Turns out it was a whole brand, but they refuse to change the email address and don't understand that I'm going to shut it all down. It totally makes my site look bad (it was an adultish brand and I was working on a community project) and I now have the power (and ability to do whatever with the account and any other that I happen upon with an email at that domain.
At least it counts as a new category on the "It's DNS fault" list.
I disagree. DNS stores enough information in WHOIS to see if ownership has changed, it's not DNS' fault that nobody looks.
Probably the least-wrong thing to do with current DNS is to have authentication servers keep track of the WHOIS UpdatedDate of email domains. If a WHOIS UpdatedDate is newer than the corresponding user's linked email address verification, that user's email address is no longer trusted. Next time they log in ask them to update or re-confirm their email address, and if they try to password reset they can't use an unconfirmed email address.
Yes that's more tricky work. Authentication is hard. Nobody should be DIYing authentication anymore in this day and age, it's just too much.
What does this protect against? If the WHOIS changes because someone new buys the domain then they could simply receive this reset emails, no?
Yes, but the user had to go through the process of "wait do I still have that email address? Did I receive it?"
Like, let's say I have an email address pxtl@example.net, and I used that to register an account on service.com, and example.net goes under.
In theory I know that this event has occurred, I no longer have access to my email address at pxtl@example.net.
So I log into my service.com account and get told "hey your email was pxtl@example.net - example.net has changed ownership. Is that still your email?" and I'll say "no" and put in a new email.
Or maybe I don't realize that example.net is gone. So I try to verify the account, find that I'm not receiving the email, and realize my mistake and set up a new email account, and click the button that says "I did not receive the email". The authentication server can prevent this window of time being an attack vector by forcing a delay between email validation and password reset, and by de-validating the email address (and treating it as a red flag on the whole domain) if the user clicks "I did not receive the email" a few minutes after the email address has been verified.
And if I forget my password and try to reset password on service.com using my unverified pxtl@example.net? "example.net had an ownership change since this email address was registered, please use another means to reset your password like SMS". Which is the main benefit of this process. Which I know doesn't require full verification.
Now, obviously the WHOIS updateddate is a noisy signal. Ideally the DNS system would expose a more granular ownership-change date - for example, gmail.com lists a WHOIS updateddate of July 11th 2024. UpdatedDate isn't supposed to change with every renewal but lots of things aren't supposed to happen.
Following up on this: Apparently my knowledge is out-of-date.
WHOIS has been superceded by RDAP, and RDAP provides event data for registration and re-registration. So even better!
edit: it doesn't seem like registrars actually do re-registration, and many cctlds don't even use RDAP yet.
You can put whatever you want in WHOIS, including just replicating the information that was there previously. What if the WHOIS email is an email on the domain in question?
Maybe registrars could set a unique ID per registrant, and if a domain expires and is purchased by a different entity/account than the previous one the registrant GUID is refreshed. That could then be a signal that all previous reliance on the DNS of the domain name should be null and void
This led me to go do a deeper dive.
1) WHOIS has been partially replaced by RDAP although many ccTLDs don't support it yet (notably .au and .us for example). Spec for RDAP query results:
https://datatracker.ietf.org/doc/rfc8977/
2) RDAP does specify that the registration date should be of the last time registered - if a domain has lapsed and picked up by somebody else it's supposed to use the verb "reregistered". But of course, you're depending on the registrar to do that. It does look like "registered" is properly followed - I looked into some known cases of poached lapsed domains and checked their RDAPS and the registration date corresponds to the date the domain drop-caught but no past expiry or re-registration is listed (example[1]).
3) Either way, don't use the content of the WHOIS/RDAP, just the dates.
It's crazy just how little effort it takes to get a "Google = bad" article to the top of HN.
There is no vulnerability in Google OAuth. This is exactly how every OAuth server is supposed to work. If you take over a domain, you automatically own every email address in that domain, and thus whatever external account relies on that email for login. Heck the result would be the same even if that service didn't use Google OAuth, or any OAuth at all.
Nothing in that write-up makes sense.
Is this really just google=bad, though? I work at a startup and this seems like a legit security risk that I'm happy to learn about.
It seems like the only mitigation would be to let your HR SAAS know when your company shutters and ask them to delete the records. Or just squat the domain yourself as an ex-employee.
Yes it is, otherwise the title would be "don't use your email address to log in to any application" and it wouldn't be ragebait enough. The whole issue has nothing to do with OAuth and nothing to do with Google.
The staying power of a Google = bad article on HN top page is insane.
And the opposite: The half-life of an Apple = bad article on HN is insane in the other direction.
It's crazy that they do that only because "Google search bad now" or "Youtube bad now".
I think part of the issue is that this is where the abstraction that we call "account ownership" starts to leak.
You may correctly have access to an account through this scenario, but that does not make it your account. This becomes obviously when we consider an account at a bank, for example.
Yes, this about sums it up. If you take over a domain, you control its registrar records for what its authoritative nameservers are, so of course you can set it to your own custom nameservers and then define whatever MX you want to receive incoming mail flow. You don't even have to go to any effort of configuring working outbound mail. You just need to put up a very minimal zonefile with the MX defined, basic postfix email server with a catchall configuration and then receive any incoming emails for *@domain.com.
Nearly a decade ago, there was a website called thehunt.com that basically ran out of money and some employees were acqui-hired by pinterest.
All of the assets were left to rot and at the time the company was a good match for another startup of mine. So I reached out to the investors I found on crunchbase and asked if I could acquire everything. We worked out a deal and I did. The issue was the complete lack of people from the old company willing to assist and the complete lack of data for alot of things. There was 1 or 2 people who we could ping from the old company to ping who were super helpful, but the big thing was many things were just lost to time- passwords, history, code repos, etc..
Simply creating a new google apps account allowed us to get full access back to everything - We could even read old slack messages (even DMs!) by resetting each accounts password. The whole thing was shocking to say the least, but with that access we got back into literally every service they used and managed to get it up and running again within a week, which was a good thing because nearly every service it was using was threatening to shut it down every day for lack of payment.
I think the solution here is actually way simpler than most make it out to be and could easily be a startup for someone:
Create a startup that lets customers simply enter in domains. If the domain EVER goes into the "pendingDelete" status, inform the customer. The customer would be random SAAS's that want to protect against this type of attack and could simply choose to disallow access to any account that has had their domain go into that status.
Huh - it's almost like the mere existence of a pendingDelete in the history of a domain should make every email existing on that domain at that time eligible for https://haveibeenpwned.com/ .
> To resolve this issue, Google could implement two immutable identifiers within > its OpenID Connect (OIDC) claims: > 1. A unique user ID that doesn’t change over time. > 2. A unique workspace ID tied to the domain.
1. is the OIDC `sub` claim! I strongly suspect that in those 0.04% of accounts where the anonymous quoted engineer reports that the `sub` claim changed, what actually happened was some provisioning/onboarding/offboarding system resulted in the account being deleted and recreated.
2. is sensible, and is just a versioned version of the `hd` claim.
1. Yep, your hypothesis seems likely. Consumer gmail addresses can't be used again after they are deleted, but it seems in Workspace orgs they can be reused/reassigned after 20 days: https://support.google.com/a/answer/33314?hl=en&co=DASHER._F...
If services are not respecting the `sub` claim in this case, then they are giving the new Google account access to the old account's data. Companies probably wouldn't complain about this because they think it is the expected/reasonable behavior. Also it's likely that in many scenarios it is the same human behind the different accounts, e.g. if they leave a company then return.
Reinforcing the face-palm at the heart of this, which is that anyone deciding to you know, just use email instead of asking why an immutable ID changed... just probably enabled information leakage. Seriously, I'm so thankful that my colleagues would be principaled about this and ask questions instead of just doing something to make it "work". Where "work" means some GSuite user probably logged into some other defunct GSuite user's RP-account.
As others have noted, this feels like an issue in the relying parties not relying on the `sub` field to validate the user. It feels the exact same as this issue here: https://bhavukjain.com/blog/2020/05/30/zeroday-signin-with-a....
In both, the details around the `sub` field, the field that should actually be used to identify the user, is poorly described. I would say that both of these feel a bit sensationalist, but then again, if relying parties are NOT using the sub field to validate users, they should be called out.
It seems to be that a good way to make some money is find every such situation where RPs are not using sub, and submit vuln bugs.
> According to a staff engineer at a major tech company:
> “The sub claim changes in about 0.04% of logins from Log in with Google.
I've had a staff SWE also claim to me that they generated colliding UUIDv4s, and a separate staff SWE who worked in GIS claim that circles only exist in map projections and they're always distorted, and that you cannot have a circle IRL, nor project it onto a projection.
Neither's attention to quality, or rather, lack thereof, could cash those outlandish claims.
There is a simple explanation here: Your staff SWE is wrong. `sub` is the claim you're looking for.
Extraordinary claims require at least a trifling of proof. Or, what can be asserted without evidence can also be dismissed without evidence.
And this alleges to be a security blog. As a writer with that target audience, you should know this is going to be the objection to the thesis that your reader will have, and you attempt to just handwave it away…?
> I've had a staff SWE also claim to me that they generated colliding UUIDv4s, and a separate staff SWE who worked in GIS claim that circles only exist in map projections and they're always distorted, and that you cannot have a circle IRL, nor project it onto a projection.
The claim about circles I don't know. It depends on what exist means, and I don't know what it means to have a circle in real life, and likely don't care. I can only draw a rough approximation of a circle, and that's been fine for me.
Generating a colliding UUIDv4 seems pretty simple though; if you have a broken enough random generator setup and manage to run it without seeding, especially if it was in the times where it was pretty easy to run a virtual machine with totally broken random (virtio-random was developed for a reason), and spawned a bunch of virtual machines in very similar conditions. You can no true scotsman your way out of this by declaring that a broken system, but from inspection, I don't know how you can determine if a given UUIDv4 was generated with proper or broken random techniques. See also Debian Security Advisory 1571-1 [1], and similar issues where random values that were intended to be secure turn out to be predictable. It's a plausible claim. But that doesn't mean a claim by a 'staff engineer' is default plausible. It's just an appeal to authority of a title that doesn't mean a lot.
[1] https://lists.debian.org/debian-security-announce/2008/msg00...
> I've had a staff SWE also claim to me that they generated colliding UUIDv4s,
UUIDv4 is random. Could be a bad PRNG, or just very very very bad luck.
> and a separate staff SWE who worked in GIS claim that circles only exist in map projections and they're always distorted, and that you cannot have a circle IRL, nor project it onto a projection.
It is insanely difficult to project circles. Most of time we just pretend it never exist.
Is this really an issue? Back when I implemented Google OAuth I used the "sub" that Google gave me as well as the email. I'd wager that Google does not re-use a sub for the same E-Mail so if the account is deleted and then somebody grabs the domain and re-created the account they'd have the same E-Mail but a different sub which would make the login fail. Seems to not be an issue in Google's OAuth Flow but rather the implementation on services like ChatGPT and so on?!
EDIT: Just read that they claim that “The sub claim changes in about 0.04% of logins from Log in with Google” - never had this happen in the past years but if this is true then I guess thats a bad thing.
Are your users primarily using gmail accounts or are they using accounts from custom domains? TFA does't say exactly, but I wonder if this stat only applies to users with custom domains rather than @gmail accounts.
The "millions of accounts" is highly speculative, is it not? It relies on the idea that failed startups keep their SAAS accounts enabled, rather than offboarding correctly.
You would think that because you're a reasonable person. However at my previous startup, which no longer exists, the new CEO failed to save the company and parachuted out so fast that the company who later acquired the startup couldn't get the domain name transferred to them because the CEO and CTO were both gone. They had to bring back the predecessor CEO because he was still able to log into the registrar to transfer the domain.
In the case of a fire, some less than conscientious people will fend for themselves first and don't give a second thought to anyone else.
I agree there will some number of companies doing this, between 1 and $number_of_companies. I'm just saying it's pretty unlikely it's all candidate companies, which the article seems to be saying.
>> is highly speculative, is it not?
I'd say NOT.
Shutdown of a small company running out of funds is rarely fully orderly. Critical things may get taken care of, but everything is unlikely, as people have already left. Plus, with things like already-paid-in accounts, there's even an incentive to keep them open until the term expires, e.g., why not keep the prepaid ChatGPT API account live for execs and techs to use between jobs?
Just like wiping hard drives on EOL computers — everyone should do it every time, but people are always finding full HDDs and SSDs on eBay and Craigslist...
also, there are lots of non-technical people running small businesses that use G services because that's all they know. They can barely operate them let alone be cognizant of things like this "exploit".
Not just this, but execs and employees like to keep websites live for a while while they job-hunt (otherwise their resume has dead links). And ideally keep their name@startup.com email working. But to keep those running requires a long tail of "live" DNS and finance stuff you can't quite close out.
Wouldn't it be cheaper just to pay for the domain registration for an extended time like 10 years vs the 1 or 2 year options?
This actually happened to me a few years ago when I closed my consulting company. My bad, I know. My domain was taken over by someone in Vietnam. Back then, he was actually able to see my old emails via the Google Apps. Grandfathered free Apps accounts was a big part of the value proposition. He demanded $500 to delete my emails and docs. I offered $50 for his trouble since I recognized that I was at least partially at fault but he refused. To his bad luck, one of my main clients was Google itself. I reported him via my internal connections and the next day got the following email: "Oh my G-d, all forty of the domains I bought got Google Apps cancelled".
> “The sub claim changes in about 0.04% of logins from Log in with Google. For us, that's hundreds of users last week”.
What I don't understand is why the `sub` claim is not consistent for those users at Google. To my understanding of the OIDC protocol the `sub` should be unique for a specific user.
Additionally as far as I understand if you take over a defunct domain and create a new google workspace with new users those new user account should get assigned a new `sub`.
I agree, in my limited experience the sub remains consistent even when changing the Google Workspace domain. So the email changes but sub remains the same. The issue seems to be clients matching on email/hd claim instead.
I wonder what action is causing the sub to change like the author suggests is happening.
At my current company, if an employee leave and come back, they'll keep the same OID in Entra but they'll get a new `sub` in Google workspace. We had to put in place a process to be able to use an internal tool that used the login with Google.
That's most likely dependant on how the IT department handled the deprovisioning/provisioning of users in our Google Workspace, I unfortunately don't have the details for that.
> I wonder what action is causing the sub to change like the author suggests is happening.
Indeed this would be very interesting.
This issue is also very similar to CVE-2024-25618.
What we did to mitigate this is the following: - Federated login with OIDC - Look for a user based on the sub claim - If they are found: authenticate that user and optionally update their profile (email, name, ...) based on then new id claims. - Else look for a user matching on the `email` claim and link the `sub` to that user - If no user is found create a new one
That sub identifier changing without the underlying user changing may be the core of the bug. It's not fully clear from the post IMO. I posted a little breakdown here: https://news.ycombinator.com/item?id=42701030
I don't think the article clearly states one way or another whether those 0.04% of sub claims changing are legitimate end user changes or not. If those sub changes are legitimately the Google Account changing, I don't think that's a bug on Google's part, but rather is a problem on the clients' side for not validating the sub claim.
If the sub is changing without the Google Account user actually changing, maybe something internally at Google is swapping out those IDs when it shouldn't be. It doesn't look like a UUID from the example I saw, so maybe there's some code somewhere to just change the user's ID if there's a collision or something?
Interestingly, I don't think Google claims that Sign in with Google is fully OIDC-compliant. At least in the overview I linked in my comment, they compare the implementation to OAuth+OIDC, but I'm not sure they claim to be 100% spec compliant.
How would the clients tell if the account has a valid sub change or not if the only piece of information provided is that the sub claim changes? For this particular attack, without having some kind of Google Workspace account identifier for the domain, the sub claim doesn't sound sufficient to validate that it's the same Google account from the client's side. I'm guessing the engineer at the major tech company didn't provide that stat without checking if those users were valid, active accounts.
> They also classified the issue as a “Fraud and abuse” issue, rather than an Oauth/login issue.
I can _kinda_ see that, I agree they should mitigate it as best as possible too though, especially since they're Google after all.
> I thought this would be the end of the story, but 3 months later, they re-opened my ticket (after my Shmoocon talk was accepted), paid a $1337 bounty, and said they were working on a fix.
Sad how the only way to get Google's attention to make enough noise about something...
$1337 is a very curious amount. Surely, someone was being clever???
yeah, google likes to have fun with the Bug Bounty program. See: https://bughunters.google.com/about/rules/google-friends/662...
it's funny that all of the soul has not been sucked out of the entire company
It's leetspeak
It spells "leet" - see https://en.wikipedia.org/wiki/Leet
You're too young to have used BBSes :)
What are you on about? You clearly missed the sarcasm.
Your comment didn't seem obviously sarcastic. Intent is hard to convey over text.
Assuming most of the problem is that federated services only rely on email address from OidC login, I wonder if Google could start offering user+unique-workspace-id@company.domain as the email address for any new workspace/user/federated login? Obviously, Google needs to control "unique-workspace-id".
Does not solve it for existing customers and any federated service they logged into, though if included alongside the simple email, federated services could gradually move to it and announce once they are "ready" so workspace admins can configure them to start accepting only new, safe email addresses.
How's this any different than buying an expired domain and doing a password reset .. anywhere?
-- Probably someone else already mentioned this in the comments here, I just couldn't see.
In iSCSI, there are a few options for addressing the block storage devices, with one being the iSCSI Qualified Name (IQN) that uses the domain name. But it includes a timestamp:
- A date code, in yyyy-mm format. This date MUST be a date
during which the naming authority owned the domain name used in
this format, and SHOULD be the first month in which the domain
name was owned by this naming authority at 00:01 GMT of the
first day of the month. This date code uses the Gregorian
calendar. All four digits in the year must be present. Both
digits of the month must be present, with January == "01" and
December == "12". The dash must be included.
* https://en.wikipedia.org/wiki/ISCSI#Addressing
iqn.2001-04.com.example.storage:tape.sys1.xyz
iqn.2000-02.edu.example.cs:users.oaks:proto.target4
Clever idea to note change of ownership situations.
Seems like we need a chain of custody for domains, not just current ownership. Kinda like real estate. And then accounts are tied not just to the ownership state, but also the ownership instance ID.
Just don't tell the web 3.0 folks cause I don't necessarily mean a blockchain.
Web 3 already tried this, it was called Handshake. I think their downfall is they wanted to replace ICANN / DNS rather than work with and alongside the current system.
shame, ICANN sucks
I would argue the crypto bros are worse
You don't need a whole chain of custody, you just need to know that the current owner is the one you care about. TXT DNS records or magic URLs can do the trick.
> Just don't tell the web 3.0 folks cause I don't necessarily mean a blockchain.
Isn't this literally what certificate chains are for? Establishing a chain or custody - or rather, a chain of liability?
This isn't anything new, or exclusive to Google, or exclusive to OAuth.
My workmate hadn't used Facebook for many years and it was associated to an email on a custom domain. Said domain expired and somehow hackers/script kiddies/bots must have an email list, run a whois, buy the domain if expired, setup email, and do password resets across social media and common websites. His Facebook was stolen.
We take the OAuth2 token and make a call to the People API [0] to get the Person resourceName. Not sure if this is more stable than "sub" but we completely ignore that.
[0] https://developers.google.com/people/api/rest/v1/people/get
It seems like there have been a number of security stories lately that essentially amount to. "no tech companies ever does cleanup" Which everyone on HN with a couple years or more experience should know is the norm. I expect a lot more of this to keep happening since companies will never really care about security without consequences ie a $10 fine for making $100 literally incentivies lack of security.
Related maybe, I got a new phone number and I was able to use it to sign into the previous owner's account at a site that uses 2FA. I noticed it when I tried to setup a new account for myself. The website wouldn't allow it but it happily allowed to me access the previous owner's account. It was inconsistent, removing the phone from their account required their password but changing many other important fields didn't.
This feels similar in spirit to some service providers (*cough* Atlassian -- especially bad when they acquire other companies and push that bullshit on previously functional authentication systems) using ownership of a domain to assert ownership of an account using that domain.
E.g.:
1. You create an account using an email associated with a domain.
2. The domain owner coughs up their protection money to Atlassian and proves ownership of the domain.
3. Your password-protected account is assigned to the domain owner. Your password and 2FA invalid. The domain owner can access your private data without your password and 2FA.
Both TFA and the thing I described rely on broken, simplistic views of "domain ownership == account ownership".
I don't think this has anything to do with Google's OAuth. This issue is literally with every single expired domain name out there. All one has to do is register the expired domain and look at all the emails sent to that domain.
Granted, Google "could" do something, but I don't think it's Google's responsibility to police expired domain names. What am I missing here?
If oauth makes an authenticity claim, it should be true. Saying it's the same user when it's not is bad, clearly.
in other words: Google could make a more accurate authenticity claim than they currently do.
This problem would be worse without oauth, though, right? With plain email login, all they would need to do is "forgot password" and there wouldn't even be a way to tell.
in other words: Email login would never be able to make a more accurate authenticity claim.
Thankfully, apparently this only affects Americans.
:-)
One of the unintended side effects of Noscript is this sort of unintended hilarity:
> These claims usually include:
> hd (hosted domain): Specifies the domain, e.g., example.com.
> email: The user's email address, e.g., [email protected].
The placeholder, hypothetical email address is being "protected" from web scraping.
In the context of an article on a security website about identities being spoofed through email accounts.
Don't let domains you use lapse. Never get rid of a phone number.
A lot of these new TLDs have variable registration fees. They're going to price a lot of people out of renewals.
I would be curious if any other Identity Providers that implement OAuth have mitigations for this issue. If you used sign in with Microsoft or Okta or PingFederate, would those provide different claims to the service?
Ideally the trust with the SP would be using a cert. This requires the SP to implement this portion of the standard and the IT department to follow through with implementing a cert. The cert can be self-generated, so there isn't a cost for the cert itself.
Every provider has their own set of flaws.
Microsoft had a flaw for awhile where you could just change a user’s email to anything with no verification.. and if the SSO implementer was only checking the email field, you could impersonate anyone.
I think the problem is here in the doc:
I have worked with a few of these downstream providers to look for a solution. There is a documented unique user identifier (the sub claim) that could theoretically prevent this issue, but in practice, it's unreliable ... According to a staff engineer at a major tech company: “The sub claim changes in about 0.04% of logins from Log in with Google. For us, that's hundreds of users last week”.
The real solution is to use the sub claim, and not hd/email. If sub is unreliable, let's fix that. That said, I'm surprised sub is documented as unreliable here; I wonder if there is perhaps something else going on. I've done a lot of work with oauth and I haven't found sub to be so mutable, but then again, who knows maybe I just missed it?
And yeah, lapsed domains => domain takeover => email takeover and impersonation will always be a baseline problem, independent of oauth.
The email returned by OAuth providers is not always verified and thus cannot be trusted, anyone relies on the email as the ID is open to compromise. The sub claim should always be used as the ID, if you can prove that "the sub claim changes in about 0.04% of logins" then it's Google flaw and they should fix it, otherwise it isn't and there is no need to add another ID to the claim.
This is a stupid article. 1. It starts with “Millions of Americans”. How does this relate in any way to Americans? 2. Gmail is not the problem 3. Mitigation: Consider registering a domain for 10 years. When the startup fails, delete all dns entries
Well they have kind of known about this kind of problem for 14 years https://issuetracker.google.com/issues/35889152?pli=1 although this is not a domain issue it is one where people can't unlink from company project ownership.
Avoiding this scenario is why Google will renew all the domains for every startup it has ever acquired in perpetuity.
How many recovery processes would depend on email. And it coming from right domain? Or changes in DNS. Or some other factor that domain holder could do. I don't think possible alternative scenarios would go down too well.
So the "sub" claim does work but the "staff engineer at a major tech company" doesn't want to fix it because it happens too often?
This is a stupid article. 1. It starts with “Millions of Americans”. How does this relate in any way to Americans? 2. Gmail is not the problem 3. Consider registering a domain for 10 years. When the startup fails, delete all dns entries
I agree. Additionally, the authors haven’t considered data deletion policies. Most SaaS providers delete data within 30 to 90 days after contract termination. It’s unlikely that data from 100,000 failed startups, each using 20 to 30 SaaS applications, still exists.
I'm not sure I agree that it's a flaw in Google's OAuth implementation. Domain ownership is the source of truth for many systems and, as the article alludes to, expired domains can be abused in a variety of ways.
We've also seen this in the dependency management ecosystem.
Apparently Google removes pagerank from expired domains to prevent gamification of their Search Index.
Same should apply to OAuth, you would think.
If you think this is bad, wait until you move physical addresses and don’t notify all your creditors!
Literally ANYONE who moves into your old house will have access to your bank statements, credit card bills, tax forms, auto registration and who knows what else. And it will all come plaintext and unencrypted!
Phone numbers too. Ah, but it's illegal to open mail addressed to someone else. (Although what if you change your legal name to match???!) Perhaps we need an electronic messaging service backed by the USPS.
Okay, this made me curious if there was a technical solution to this that people could be providing using the existing tools available, and I think there is.
RDAP and WHOIS will return the creation date of a domain. These fields are controlled by the registrar not the registrant. That creation/registration date gets reset when the domain is lapsed and picked up by somebody new.
So, when doing any domain-name-based authentication (like email password resets) authenticators should look up the registration date of the domain name. If it's newer than the last time the user logged in using the domain name auth? That might be somebody who snatched the domain name.
Sadly, this isn't fool-proof. Domains can go up for auction or backorder on a registrar, and they won't update the registration date if the domain is purchased this way since the registrar can consider this a transfer. It's a signal, for sure, but it will miss cases. It will also miss transfers sometimes, depending on the registrar.
> Shmoocon talk accepted - Dec 9, 2024, Google re-opens issue - Dec 19, 2024
Wondering what the internal process there was at Google? Is someone's job to cruise around the various conferences and dev events to look for any Google mentions and report back. Did Google really consider it an issue or just wanted it avoid looking bad in public after the talk was accepted.
Yes, there are roles where people do nothing but search their names/products/etc specifically to see what people are saying. Hell, some people do it on themselves to the point there are phrases like "vanity search" or "egosurfing"
> Is someone's job to cruise around the various conferences and dev events to look for any Google mentions and report back.
Someone in Google's security team likely saw the talk, or the original blog post, and pushed internally for them to award something.
Yea. The leet amount of payout seems like this was someone saying that the classification was technically correct, but it's worth fixing either way and it'd be nice to pay an amount.
Google may have quickly closed the original report because a company like that is soooo overwhelmed with garbage security reports and at first glance it is noise, not an actionable item to fix. there's a term for this that I can't quite recall off the top of my head. Bug Bounty Trolling?
what a shitty headline and opening paragraph.
anyway, if you use SSO on a domain and then let the domain drop, you're an idiot and there will be bad consequences for your data.
Domains are for life, you can never let them lapse