This a good example of why manifest v3 sucks. By definition, it can't do anything like this...no live code hueristics are possible.
It's a war of escalation with advertisers. Google is the arms dealer to both sides. They won't give you what you would need to win.
There's no reason why a declarative manifest v3 API couldn't offer this. If I'm reading the commit details correctly, it could work even better by being better integrated into the request flow to block the request on the actual IP address used before anything is sent to the servers.
Of course, this all relies on Google wanting to add this API. Doing this imperatively with "live code" allows for innovations in userland before browser makers add built in support for it.
>This a good example of why manifest v3 sucks. By definition, it can't do anything like this...
Technically manifest v3 has nothing to do with APIs that the browser makes available to extensions. On firefox manifest v3 is supported with blocking web request[1], which is the filtering api prior to "manifest v3". Therefore the statement that it certain functionality "by definition" is false.
[1] https://blog.mozilla.org/addons/2022/05/18/manifest-v3-in-fi...
Exactly this. There's some good stuff in MV3, but Google decided to take the opportunity to smuggle in some self-serving changes, similar to how Congress likes to sneak controversial laws under the radar as part of unrelated bills*.
OK so Google's near monopoly implementation of V3 sucks. Technically a difference, but practically not so much.
I'm confused, isn't the Manifest V3 essentially just API spec?
Abandon Chrome, embrace Firefox.
The title seems to be wrong, uBlock Origin supported it for many years at this point (only on Firefox). This seems to be a refactor of that code, not a whole new feature.
Well, it does support it now. It supported it before, too :P
i used to use ublock origin. i still do, but i used to, too
There must have been some Monty Python... ah you get it.
I believe that is a reference to Mitch Hedberg R.I.P. [1]
[1] - https://www.youtube.com/watch?v=VqHA5CIL0fg [video][10 seconds]
And there are at least 3 fish
It did not hit me yet, but I'm already rewriting my extensions to firefox to switch if Chrome really axes uBO
It's not if. It's when. It has been 'when' since 2020. It is coming. It is not going to not come. It will be here in mere releases. Get ready.
You're probably right, but FWIW it's not unheard of for google to announce, continually delay, and eventually completely backtrack on things like this, like third party cookie deprecation.
This time it affects their bottom line in a profound way so wishful thinking is probably not going to work unfortunately.
> affects their bottom line in a profound way
Something around 8% of total digital ad spend.
To be fair, that's tremendous
Yeah, hence why I started already migrating, slowly.
I have a simple tab organizer extension and some greasemonkey scripts that should work perfectly fine on Firefox without any changes.
I am switching family over to Brave. They don’t even notice the difference and I’m more confident the browser will continue to support user centric content filtering.
Which is fine so long as what they have built into the browser is all you need (and so long as Google does not sabotage those efforts). If ever you might need more such as what uBO offers, though, Brave is also subject to whatever changes Google makes (such as Manifest V3).
Is that going to help? Brave is still blink. They might have some filtering baked in but I’m not sure if it’s as powerful (or can be) as UBO.
I don't think the new manifest rules are because of any actual restriction or the engine, maybe I'm wrong but it was my impression that the change is mostly just Google wanting to control what users can do with their browsers even more (and always in ways that make them profit of course). So they should be able to keep V2 support if they're willing to be on the hook for keeping it maintained themselves.
Brave will keep Manifest V2 compatibility as long as they can, specifically focusing on "AdGuard AdBlocker, NoScript, uBlock Origin, and uMatrix". https://brave.com/blog/brave-shields-manifest-v3/
I guess I'm curious what "as long as they can" actually means. If it means they can't pull upstream Blink without losing v2 support, that's bad.
I think Firefox is the only viable solution to continue using UBO at this point.
It's already axed in canary release
I'm still at the "This extension may soon no longer be supported" warning
For those unaware, what is uBO and how would it affect most extensions?
Maybe it’s a telling sign of the new wave of HN users, but I’m genuinely surprised to read that you don’t know what uBlock is..
uBO (uBlock Origin) is a popular open source ad blocker. There is a change coming to the way Google Chrome and some other browsers host extensions called manifest v3, which for (stated) security and privacy purposes limits a lot of the functionality that makes ad blockers work the way they do. There are workarounds but they are suboptimal. This has been an ongoing fight for years and there are plenty of accusations that Google is doing this because they want to cripple ad blockers since they make so much money from advertising. Firefox has explicitly stated they will not force these changes on extension developers, and thus a lot of people have been threatening to move to Firefox whenever Google finally makes this change for real.
"uBO" is an abbreviation of "uBlock Origin".
uBlock Origin is what makes Firefox even greater and definitely one big reason I use Firefox over Chrome etc. It make the Internet browsable.
That may change since Mozilla is becoming an ad company
I moved many years ago to this combo, and never saw a single reason to switch away. Same for android phone, the only usable mobile web experience I've seen. Those few sites over a decade that had some display issues had issues also under chrome.
Plus I personally consider ads a cancer of modern society. White and not so white lies, manipulation... nothing respectable regardless (or because ) of tremendous money circulating in it.
I really wish I could agree but sadly this has not been my experience with Firefox, and I have so many issues I've started to switch away recently. Wasting way too much time fighting with websites that turn out to work perfectly fine on Chrome, and the captchas I get on Firefox are becoming genuinely impossible for me to solve. I'm with you on the ads though, and glad it's working out for someone at least!
I mean there are appropriate applications for advertising (like classifieds in a newspaper), but there is no reason why advertising should be so pervasive that it requires a massive surveillance apparatus like it does today. Advertisements are the reason why everyone switched from TV to Netflix, and that's back when cable TV was a paid service.
secushare[1] makes the case that this is because the internet lacks a secure micropayments layer, so the funding model for everything has to be advertising-based instead of patronage-based. Paypal and the like are exploited as cash cows because of their centralized nature. Cryptocurrencies were later tried but have technical limitations that broadly prohibit this use case (even with payment channels/LN).
Ugh, micropayments again! I have no desire to pay for content or software -- torrents and OSS are all anyone needs.
CNAME cloaking? Does this mean an ad site may use a randomly generated subdomain pointing to a wildcard record?
That's part of it.
Normally when you visit contentsite.com which serves ads from adsite.com. Adblocker rules can just block adsite.com and the ads won't be shown. CNAME cloaking would have the main site have a subdomain like adsite.contentsite.com point to adsite.com, now the adblockers have the impossible task of blocking millions of subdomains that seemingly belong to legit sites, this also allows the legit sites to keep changing the subdomain since the adblocker will have no idea which subdomains serve legit content vs ads. As a bonus since the content is being served from the same domain, they can bypass certain cookie browser policies and track users even better.
This update allows you to set rules so that you can filter by resolved ip.
i hope that this results in sites that host malicious ads and use wildcard session-cookies get hacked to all hell by their ads.
this reminds me of domainfronting, who was a super smart way to get around of ads and other sites blockers, not sure if it's all 'fixed' now.
Yes. Ads and analytics providers have started doing this to get around third-party cookie protections.
I always find this development curious. About a decade ago I worked in this space. When someone brought up ad blockers I just said "put the analytics on our main domain. No one is going to block the entire website". The answer I got was "no one would ever do that because of the implications of serving advertising from your main domain". Yet, here we are.
They use a third party domain just because that way they can track the user actions with cookies, for example Google can track your navigation across multiple websites, and thus propose to you more relevant ads. Also using a different domain was simpler and cheaper, since you don't have to host the AD content and metadata, just include the JS from the AD provider somewhere in your HTML.
Now that thanks to EU laws and browser imposing restrictions about third-party cookies it's more difficult, the whole "serve ads from other domain" may not be that relevant anyway.
If you use a random wildcard subdomain... just serve them from the main website, what is the difference? On the other side with a proxy just route the AD requests to another server if it needs to be, of course you have to find a way to distinguish which requests are for AD and which are not, something you can do with some sort of signature in the filename, so that only the server can know which requests shall be handled locally and which one forwarded to the AD provider server.
News payers used to all serve their own ads including in house sales and design. Frankly with how key advertising is I don't understand why anyone would out source it.
Advertisers trust other advertisers not to lie but not the content providers. well except Google they trust Google. So you have to use these hella shady and networks that are fly by night and security and privacy nightmares across many domains. Instead of many walled gardens of ads like you're saying.
This. Everyone and their grandma decided it's cool for Google and others to decide what should display on your website next to your content because of "magic online advertising".
How much of the efficiency of online advertising comes from the actual "art" of tracking users and their preferences to display "personalized" ads vs the "efficiencies" from firing/outsourcing your marketing, ad sales and creative workforce.
cost and effectiveness.
selling ad space was always a lot of work. algorithms do it cheaper and in general better.
next step is just to run a GoogleAds lib/proxy...
Particularly with the reams of evidence that fraud is rampant, both in advertising content and in claimed click/view rates.
What are the implications?
If third party ad servers get access to your main domain's cookies, they can impersonate your signed-in users and steal their data.
...Unless you're savvy. Thank goodness for the availability of https://publicsuffix.org/ (as long as you only use your main domain and don't need to share cookies with your own subdomains), and the includeSubDomains directive to HSTS! But - if you already set this up, you probably are savvy enough to avoid the problems created (or your provider is)
HSTS won't prevent this at all; the advertiser merely needs to also set up TLS by getting a certificate for that subdomain, which they can already do precisely because it goes to their web server -- not yours. This also lets them steal cookies marked secure (sent over HTTPS only).
Edit: A combination of DNS CAA with an account identifier restriction in the record would prevent this. Then the advertiser would complain, and any ads served would have to be over plaintext, which would cause browser warnings about mixed content and allow MITM injection of (more) malicious content.
Presumably that adblockers (or rather their users) would object to blocking domains that folks might actually want to load content from. I can’t imagine “domain” is the only signal one could use to identify ads, though. To truly befuddle them you’d make advertisements truly indistinguishable from content. This is not trivial.
Not entirely true. If you lower the quality of your content enough the advertisements are in fact indistinguishable. I often enjoy reading the "chumbox" at the bottom of the news article more than the reporting itself
It more or less boiled down to "we would be labeled an advertiser and not a destination for information on the internet". Like being an advertiser stopped people from using Google search or something
This is such an intrusion of privacy. I wish I could just disable cookies entirely but the usability of many webpages just goes down. I should not be punished for not wanting 3rd party trackers.
I run all the time with first-party cookies disabled.
Most of the web works. Anything that does not, and I care about, gets blessed.
The only content I allow by default, even in low-security browser profiles, and even from first-party domains, are HTML, images and CSS.
I consider the occasional broken page to be a successful test of my configuration. If I care, I adjust permissions.
Before I get too alarmed someone would have to tell me how an adsite.com cookie is being sent to adsite.example.com. This workaround seems to let adsite.com profile me as well as example.com already can, but it loses the ability to correlate my activity across example2.com and example.com with a single cookie.
(I guess ad providers have gotten good enough to not need cookies? Like they know my browser window size, installed fonts, GPU vendor and model, IP address, geolocation, header order, etc. so they don't even need cookies anymore to track my activity across the web? I suppose it was only a matter of time.)
The correlation is happening through an API connection between adsite.com and example.com and not through cookies. So even if you block all third party cookies and scripts your activities are being tracked through the first party.
cookie is for each site, but that's enough... sure, maybe no retargeting ads, but those were creepy anyways (and likely not more effective)
Browser profiling has been a thing for at least a decade if I'm not mistaken.
Makes sense. "I am session abcdef12345" always seemed significantly guaranteed to me, but in a world with ad blockers and third-party cookie restrictions, using heuristics is the only way forward.
It's somewhat scary how much information our browsers leak to unknown parties.
(I don't really take sides on this. I use an ad blocker and am very anti-ad, but am impressed when ad companies come up with tech to thwart them. The cat-and-mouse game is entertaining to read about.)
it's more than enough. especially that the competition is also only using the same tech
Randomly generated domains are a major red flag for abuse and malware detection, and seems to have become a rather large part of how the domain industry manage abuse. Domain "credit score" is also something that is used in the email industry to score links and thus spam values. A large part of providing score values is behind security companies that offer their service as a paid services, but as with a lot of this stuff there are a lot of movement to offer it for free similar to spam block lists.
It will be interesting when this kind of technology moves down to browser add-ons.
There is a part of me that, at a high level, appreciates the back and forth between the user and the ad industry. On a personal level, I am slowly getting to the point, where I am less.. uhh.. understanding.
That said, the average person's conception of what acceptable needs to change. I did briefly think that they need suffer through more ad-infestation first, but I realized that the answer is more in line with what my wife seemed to have gone through. The low exposure to ads made her less willing to deal with them. This might be the way forward.
It is hard for a person used to existing ecosystem to even imagine, there could be something better.
Certainly, streaming services have ruined broadcast television for me. I don’t know how I used to spend over $100 a month on cable TV to be advertised to. Spending four and a half days a year (44 minutes out of every two hours) watching ads is not for me.
Don't some DNS servers implement something that acts like a server-resolved CNAME, where the admin puts in a record that points to some other DNS name but the client just sees an A (out AAAA) record?
I think you are referring to ALIAS records
Yup. some implementations provide similar ANAME, and Cloudflare has flattened CNAME which is probably the best implementation I came across in years of supporting folks trying to use these kinds of records on a large CDN.
Is chrome going to block uBO im never up to date on the latest. I do know theyre allowing 3rd party cookies now... so maybe theres a chance
For right now, uBlock Origin is still on the Chrome Web Store for Chromium browsers which support Manifest V2. If you use a Manifest V3 only version of Chromium, it is hidden.
They're not blocking uBO, they're removing the features in the browser that allowed uBO to work by releasing new plugin APIs, "Manifest v3". They're eliminating the key APIs needed for uBO to identify things that it shouldn't load, and then not load them. Google claims this was for "performance" or "security" reasons. Of course, the only major 'performance' or 'security' affected is the ability to identify, intercept, and stop harmful or ad related downloads before they start.
Does this affect extensions that know every website you visit even if it doesnt need to know, and has nothing to do with the extension’s functionality? (ie the ones that Similarweb buys)
Not updating your browser is also hazardous - much better to switch to FF, and have a browser that gets updates and also fully supports uBO.
They’re doing a slow phase-out over a long time to try to avert a wave of bad publicity that threatens their browser monopoly, but that timeline has already started as of June.
https://developer.chrome.com/docs/extensions/develop/migrate...
https://www.bleepingcomputer.com/news/google/google-chrome-w...
Honestly, it probably is going to depend on whether the US continues to have an administration that's willing to take blatant monopolists to court.
What is the uBI status on Brave, Edge and Opera?
I don't care about the two proprietary browsers you've mentioned, but Brave is going to (partially) support manifest v2 and maintain uBO compatibility for as long as they're able to:
https://brave.com/blog/brave-shields-manifest-v3/
Not that you really need it as Brave has its own very capable built-in ad blocker with -- last time I checked -- higher performance than uBO (since it's compiled into native code) and full support for same ad lists.
Brave is open source instead of proprietary now? I knew they were Chromium based (like the others) but I hadn't realized they switched over on all of the customizations on top.
Brave is like Firefox: it's open source, But it connects to some closed source servers to serve up contentious features (IMO this is a bigger problem on Brave than Firefox, since many of them cannot be fully hidden).
As an example of what CNAME cloaking is, let's say that a SAAS provider A wants to provide you, company Q, with fancy ad tracking software. In the olden days, they'd tell you to embed a script at e.g. https://A-ads-tracking.example into your website at address https://q-company.example
To block those ads, blocklists that uBlock Origin use have rules then that say "block requests being made to the domain name A-ads-tracking.example", which blocks the ads.
CNAME cloaking is where SAAS provider A sets up their ad-tracking services not on domain A-ads-tracking.example, but instead at a specific IP address of e.g. 29.1.2.3; then (and here's the important part) SAAS A tells you Company Q that you need to set up a subdomain of q-company.example which has a CNAME record pointing to 23.1.2.3, a subdomain with an innocuous name like media.q-company.example; once you've set up that CNAME, you at Company Q add a script tag to your website for `media.q-company.example` and now SAAS A is able to track all the users on your site. This indirection allows for effectively infinite cat-and-mouse on the part of you the owner of the Q Company vs the blocklists that the public assemble.
To get around this CNAME cloaking problem, the software powering extensions like uBlock Origin need to be able to see not only the destination domain of requests by browsers, but the underlying IP addresses of those domains as well. This commit makes that behavior possible, or at least is related to making that code work better.
That's not quite right; as the name suggests, it uses CNAMEs (which point to other records), not A records (which point to IPs). So you would have something like `media.q-company.example` as a CNAME to `q-company.ads-tracking.example` which then has an A record to give an IP.
Browsers might not offer intermediate DNS names to extensions (I don't know), so something like uBlock might need to rely on IP lists, but DNS-based filtering like pihole should just block it by a rule against `ads-tracking.example`. In any case, it's good to use both browser based and DNS based malware blockers.
Gah, it's all right there! Amazing what you can forget/mistake due to what you've been working on lately.
Thank you for the breakdown!
And this is a good reason to block all JavaScript in unlock advanced and slowly whitelist the scripts you see until the site works properly. Slow and error prone but once you get used to it it's a breeze. And you're completely immune to this sort of shittery.
Is there a public list of known legit, whitelisted scripts?