The title seems to be wrong, uBlock Origin supported it for many years at this point (only on Firefox). This seems to be a refactor of that code, not a whole new feature.
Well, it does support it now. It supported it before, too :P
i used to use ublock origin. i still do, but i used to, too
There must have been some Monty Python... ah you get it.
I believe that is a reference to Mitch Hedberg R.I.P. [1]
[1] - https://www.youtube.com/watch?v=VqHA5CIL0fg [video][10 seconds]
Thank you for providing the video length, it is the factor that made me click.
And there are at least 3 fish
It sounds to me like more than just a refactor, it now allows blocking based on ip earlier, before the request is actually made. Although, that isn't perfect because it doesn't know which ip address the browser will choose if there are multiple ips for a single domain.
Ok, I've reverted the title to that of the page. Submitted title was "uBlock Origin supports filtering CNAME cloaking sites on Firefox now". If someone wants to suggest a more accurate and neutral title, we can change it again. Github commits without additional context don't usually make for great HN threads though...
Something akin to "uBlock Origin CNAME uncloaking now supports filtering by IP address" should be fine.
Ok, I've switched to that - thanks!
It did not hit me yet, but I'm already rewriting my extensions to firefox to switch if Chrome really axes uBO
It's not if. It's when. It has been 'when' since 2020. It is coming. It is not going to not come. It will be here in mere releases. Get ready.
You're probably right, but FWIW it's not unheard of for google to announce, continually delay, and eventually completely backtrack on things like this, like third party cookie deprecation.
This time it affects their bottom line in a profound way so wishful thinking is probably not going to work unfortunately.
> affects their bottom line in a profound way
Something around 8% of total digital ad spend.
To be fair, that's tremendous
Source?
>third party cookie deprecation
It's because they were literally sued and are not allowed to remove it, not because they don't want to.
Yeah, hence why I started already migrating, slowly.
I have a simple tab organizer extension and some greasemonkey scripts that should work perfectly fine on Firefox without any changes.
I am switching family over to Brave. They don’t even notice the difference and I’m more confident the browser will continue to support user centric content filtering.
Which is fine so long as what they have built into the browser is all you need (and so long as Google does not sabotage those efforts). If ever you might need more such as what uBO offers, though, Brave is also subject to whatever changes Google makes (such as Manifest V3).
Is that going to help? Brave is still blink. They might have some filtering baked in but I’m not sure if it’s as powerful (or can be) as UBO.
Brave will keep Manifest V2 compatibility as long as they can, specifically focusing on "AdGuard AdBlocker, NoScript, uBlock Origin, and uMatrix". https://brave.com/blog/brave-shields-manifest-v3/
I guess I'm curious what "as long as they can" actually means. If it means they can't pull upstream Blink without losing v2 support, that's bad.
I think Firefox is the only viable solution to continue using UBO at this point.
I seem to recall they said as long as it was technically AND economically feasible. It depends on how much google attempts to spread the cancer of mv3 throughout their code base. Eventually brave won’t have the manpower to hack v2 in is the likely result without going bankrupt if google really wants to go down that path.
I don't think the new manifest rules are because of any actual restriction or the engine, maybe I'm wrong but it was my impression that the change is mostly just Google wanting to control what users can do with their browsers even more (and always in ways that make them profit of course). So they should be able to keep V2 support if they're willing to be on the hook for keeping it maintained themselves.
And if the removal of manifest v2 allows google to abandon/remove/refactor code exposed through it (and google could even be inclined to do so deliberately)? At some point Brave would end up having to maintain its own fork, which they probably don't have the resources to do.
It's already axed in canary release
I'm still at the "This extension may soon no longer be supported" warning
2 days ago: "Chrome Canary just killed uBlock Origin and other Manifest V2 extensions" https://news.ycombinator.com/item?id=41757178
> I'm already rewriting my extensions to firefox
What does that mean? Firefox uses the same API. At most you have to change `background.service_worker` to `background.scripts` (literally just rename the key)
For those unaware, what is uBO and how would it affect most extensions?
uBO (uBlock Origin) is a popular open source ad blocker. There is a change coming to the way Google Chrome and some other browsers host extensions called manifest v3, which for (stated) security and privacy purposes limits a lot of the functionality that makes ad blockers work the way they do. There are workarounds but they are suboptimal. This has been an ongoing fight for years and there are plenty of accusations that Google is doing this because they want to cripple ad blockers since they make so much money from advertising. Firefox has explicitly stated they will not force these changes on extension developers, and thus a lot of people have been threatening to move to Firefox whenever Google finally makes this change for real.
Maybe it’s a telling sign of the new wave of HN users, but I’m genuinely surprised to read that you don’t know what uBlock is..
I'm happy youngsters still use this ol' forum
It’s good that we are still getting Youngbloods who see the superiority of text in communications rather than one sided talking head videos.
"uBO" is an abbreviation of "uBlock Origin".
uBlock Origin is what makes Firefox even greater and definitely one big reason I use Firefox over Chrome etc. It make the Internet browsable.
I moved many years ago to this combo, and never saw a single reason to switch away. Same for android phone, the only usable mobile web experience I've seen. Those few sites over a decade that had some display issues had issues also under chrome.
Plus I personally consider ads a cancer of modern society. White and not so white lies, manipulation... nothing respectable regardless (or because ) of tremendous money circulating in it.
I really wish I could agree but sadly this has not been my experience with Firefox, and I have so many issues I've started to switch away recently. Wasting way too much time fighting with websites that turn out to work perfectly fine on Chrome, and the captchas I get on Firefox are becoming genuinely impossible for me to solve. I'm with you on the ads though, and glad it's working out for someone at least!
Try Firefox serving Chrome's UserAgent. You'll be shocked how many issues disappear.
Can you recommend an extension for that? I use “User agent switcher and manager” but it seems overly complicated and aimed at web devs. I just want a simple interface and easy switching between OS and browser combinations
I have very few issues with Firefox. The two that I suspect are:
1) Google-owned sites seem to just chew CPU on Firefox. In particular I'm thinking of GMail and Youtube, both of which I'm a heavy user of, and also Maps. But no non-google sites seem to have this problem.
2) I'm constantly getting websites saying "This is your first time using this device, are you sure you're you?", and I haven't tried whether it's better on Chrome, but it's pretty crazy because I've literally never used your stupid site with any other device, and I used it with THIS device just last month you idiots. I'm just blind guessing that this is some kind of problem as a result of Firefox privacy choices, like maybe the site doesn't know how to use cookies in a way that doesn't trigger anti-tracking. For example banks.
But Firefox can keep thousands of tabs open at once (thousands. plural. not kidding, not exaggerating.), it has working uBO, and the frequency of "just because we wanted to" UX changes is much lower. It's just a better choice all around.
I mean there are appropriate applications for advertising (like classifieds in a newspaper), but there is no reason why advertising should be so pervasive that it requires a massive surveillance apparatus like it does today. Advertisements are the reason why everyone switched from TV to Netflix, and that's back when cable TV was a paid service.
secushare[1] makes the case that this is because the internet lacks a secure micropayments layer, so the funding model for everything has to be advertising-based instead of patronage-based. Paypal and the like are exploited as cash cows because of their centralized nature. Cryptocurrencies were later tried but have technical limitations that broadly prohibit this use case (even with payment channels/LN).
Ugh, micropayments again! I have no desire to pay for content or software -- torrents and OSS are all anyone needs.
It's mostly about paying for the bandwidth, not the information itself. In the client-server model, the the server owner has to pay for bandwidth and operation costs, which is the justification for advertisements and such.
Even in torrents you have private trackers and all of these annoying incentive systems for people to host content. If you had a good reward system on top of bittorrent/IPFS I think that idea could take over the world, but it is not efficient or decentralized to do so.
That may change since Mozilla is becoming an ad company
They are not, but they are adding in support for more anonymous ads as they see it as a “compromise”, I don’t but I also don’t think they are as malevolent as a lot of people on HN and Reddit like to make them out to be.
A "compromise" between a usable non-ad-company product and what?
Honestly I’ve used brave and Firefox and don’t see a huge difference. I still prefer Firefox though because of its philosophy and status as coming from a nonprofit. Brave is a quality project too though, and is my back up, although sometimes I throw Vivaldi in the mix because of its windows splitting and much superior tab management.
Firefox is developed by the for profit company not the nonprofit org.
CNAME cloaking? Does this mean an ad site may use a randomly generated subdomain pointing to a wildcard record?
That's part of it.
Normally when you visit contentsite.com which serves ads from adsite.com. Adblocker rules can just block adsite.com and the ads won't be shown. CNAME cloaking would have the main site have a subdomain like adsite.contentsite.com point to adsite.com, now the adblockers have the impossible task of blocking millions of subdomains that seemingly belong to legit sites, this also allows the legit sites to keep changing the subdomain since the adblocker will have no idea which subdomains serve legit content vs ads. As a bonus since the content is being served from the same domain, they can bypass certain cookie browser policies and track users even better.
This update allows you to set rules so that you can filter by resolved ip.
i hope that this results in sites that host malicious ads and use wildcard session-cookies get hacked to all hell by their ads.
I would hope that this results in websites hosting malicious ads which harm users, which then results in a big lawsuit against these websites with a huge payout for the harmed users. After all, if the malware ad is being effectively hosted by the site, then the site should be legally responsible.
The upside is that it would only really impact their interaction with that specific site, not with anything else (unless there's opportunity for lateral moves because of SSO or the site hosts email and so they could hack password resets or something).
My dream scenario would be this happening to an in-company administrative user with the keys to the kingdom. Imagine an ad-ridden site like Fandom.com getting hacked in that way.
this reminds me of domainfronting, who was a super smart way to get around of ads and other sites blockers, not sure if it's all 'fixed' now.
Yes. Ads and analytics providers have started doing this to get around third-party cookie protections.
I always find this development curious. About a decade ago I worked in this space. When someone brought up ad blockers I just said "put the analytics on our main domain. No one is going to block the entire website". The answer I got was "no one would ever do that because of the implications of serving advertising from your main domain". Yet, here we are.
They use a third party domain just because that way they can track the user actions with cookies, for example Google can track your navigation across multiple websites, and thus propose to you more relevant ads. Also using a different domain was simpler and cheaper, since you don't have to host the AD content and metadata, just include the JS from the AD provider somewhere in your HTML.
Now that thanks to EU laws and browser imposing restrictions about third-party cookies it's more difficult, the whole "serve ads from other domain" may not be that relevant anyway.
If you use a random wildcard subdomain... just serve them from the main website, what is the difference? On the other side with a proxy just route the AD requests to another server if it needs to be, of course you have to find a way to distinguish which requests are for AD and which are not, something you can do with some sort of signature in the filename, so that only the server can know which requests shall be handled locally and which one forwarded to the AD provider server.
News payers used to all serve their own ads including in house sales and design. Frankly with how key advertising is I don't understand why anyone would out source it.
This. Everyone and their grandma decided it's cool for Google and others to decide what should display on your website next to your content because of "magic online advertising".
How much of the efficiency of online advertising comes from the actual "art" of tracking users and their preferences to display "personalized" ads vs the "efficiencies" from firing/outsourcing your marketing, ad sales and creative workforce.
Advertisers trust other advertisers not to lie but not the content providers. well except Google they trust Google. So you have to use these hella shady and networks that are fly by night and security and privacy nightmares across many domains. Instead of many walled gardens of ads like you're saying.
cost and effectiveness.
selling ad space was always a lot of work. algorithms do it cheaper and in general better.
next step is just to run a GoogleAds lib/proxy...
Until the algorith associates you with an ad for something negative to your audience. Scams for example are common for algorithms to allow while a human can validate some legitimentch
The opportunity cost of lost sales due to unwanted associations is a drop in the bucked compared to the cost of having humans in the loop. And when the most of the market moves to a low-cost regime eventually almost everyone has to because being a holdout now means you can't even find enough other counterparties (to sell your ad space to, or to buy ad space from).
Particularly with the reams of evidence that fraud is rampant, both in advertising content and in claimed click/view rates.
Well, it's just a question of priorities. What do you care about more, security on your site, or getting your ads past ad-blockers?
I'm not surprised there are people who prioritize the latter, especially for small sites where they may not have someone who fully understands the risks.
What are the implications?
If third party ad servers get access to your main domain's cookies, they can impersonate your signed-in users and steal their data.
...Unless you're savvy. Thank goodness for the availability of https://publicsuffix.org/ (as long as you only use your main domain and don't need to share cookies with your own subdomains), and the includeSubDomains directive to HSTS! But - if you already set this up, you probably are savvy enough to avoid the problems created (or your provider is)
HSTS won't prevent this at all; the advertiser merely needs to also set up TLS by getting a certificate for that subdomain, which they can already do precisely because it goes to their web server -- not yours. This also lets them steal cookies marked secure (sent over HTTPS only).
Edit: A combination of DNS CAA with an account identifier restriction in the record would prevent this. Then the advertiser would complain, and any ads served would have to be over plaintext, which would cause browser warnings about mixed content and allow MITM injection of (more) malicious content.
Presumably that adblockers (or rather their users) would object to blocking domains that folks might actually want to load content from. I can’t imagine “domain” is the only signal one could use to identify ads, though. To truly befuddle them you’d make advertisements truly indistinguishable from content. This is not trivial.
Not entirely true. If you lower the quality of your content enough the advertisements are in fact indistinguishable. I often enjoy reading the "chumbox" at the bottom of the news article more than the reporting itself
I think what we're asking is what are the implications for the advertisement company.
And yeah, I can trivially block stuff in uBO by using CSS rules for example, so that's still on the table.
> I think what we're asking is what are the implications for the advertisement company.
Higher impressions? Higher integration cost? I guess I'm not sure what the confusion might be. Advertisers obviously want to ram their bullshit down as many eyesockets as they can find.
It more or less boiled down to "we would be labeled an advertiser and not a destination for information on the internet". Like being an advertiser stopped people from using Google search or something
Or newspapers, both before and after it. They've always been vast advertising platforms, but don't have anywhere near the same stigma that online advertisers have acquired (for extremely good reasons imo - they're as invasive as possible, while printed media has rather tight limits)
They could have become the dominant advertisers online too, and then no doubt they'd be just as nasty. But they lost that war multiple times, first to doubleclick-likes and then to social media.
There is a part of me that, at a high level, appreciates the back and forth between the user and the ad industry. On a personal level, I am slowly getting to the point, where I am less.. uhh.. understanding.
That said, the average person's conception of what acceptable needs to change. I did briefly think that they need suffer through more ad-infestation first, but I realized that the answer is more in line with what my wife seemed to have gone through. The low exposure to ads made her less willing to deal with them. This might be the way forward.
It is hard for a person used to existing ecosystem to even imagine, there could be something better.
Certainly, streaming services have ruined broadcast television for me. I don’t know how I used to spend over $100 a month on cable TV to be advertised to. Spending four and a half days a year (44 minutes out of every two hours) watching ads is not for me.
Randomly generated domains are a major red flag for abuse and malware detection, and seems to have become a rather large part of how the domain industry manage abuse. Domain "credit score" is also something that is used in the email industry to score links and thus spam values. A large part of providing score values is behind security companies that offer their service as a paid services, but as with a lot of this stuff there are a lot of movement to offer it for free similar to spam block lists.
It will be interesting when this kind of technology moves down to browser add-ons.
This is such an intrusion of privacy. I wish I could just disable cookies entirely but the usability of many webpages just goes down. I should not be punished for not wanting 3rd party trackers.
I run all the time with first-party cookies disabled.
Most of the web works. Anything that does not, and I care about, gets blessed.
The only content I allow by default, even in low-security browser profiles, and even from first-party domains, are HTML, CSS, and images.
I consider the occasional broken page to be a successful test of my configuration. If I care, I adjust permissions.
What do you use to enforce this? Is it something that's going to break with Manifest V3?
I use uMatrix. On Firefox, so no current concerns about MV3.
Before I get too alarmed someone would have to tell me how an adsite.com cookie is being sent to adsite.example.com. This workaround seems to let adsite.com profile me as well as example.com already can, but it loses the ability to correlate my activity across example2.com and example.com with a single cookie.
(I guess ad providers have gotten good enough to not need cookies? Like they know my browser window size, installed fonts, GPU vendor and model, IP address, geolocation, header order, etc. so they don't even need cookies anymore to track my activity across the web? I suppose it was only a matter of time.)
The correlation is happening through an API connection between adsite.com and example.com and not through cookies. So even if you block all third party cookies and scripts your activities are being tracked through the first party.
Browser profiling has been a thing for at least a decade if I'm not mistaken.
Makes sense. "I am session abcdef12345" always seemed significantly guaranteed to me, but in a world with ad blockers and third-party cookie restrictions, using heuristics is the only way forward.
It's somewhat scary how much information our browsers leak to unknown parties.
(I don't really take sides on this. I use an ad blocker and am very anti-ad, but am impressed when ad companies come up with tech to thwart them. The cat-and-mouse game is entertaining to read about.)
it's more than enough. especially that the competition is also only using the same tech
cookie is for each site, but that's enough... sure, maybe no retargeting ads, but those were creepy anyways (and likely not more effective)
This a good example of why manifest v3 sucks. By definition, it can't do anything like this...no live code hueristics are possible.
It's a war of escalation with advertisers. Google is the arms dealer to both sides. They won't give you what you would need to win.
There's no reason why a declarative manifest v3 API couldn't offer this. If I'm reading the commit details correctly, it could work even better by being better integrated into the request flow to block the request on the actual IP address used before anything is sent to the servers.
Of course, this all relies on browser vendor (Google) wanting to add this API. Doing this imperatively with "live code" allows for innovations in userland before browser makers add built in support for it.
It could. Google won't do that for chrome.
Had they not taken away onBeforeRequest with manifest V3, plugins could implement it themselves. Which is the thing you're suggesting...before the request goes.
The commit message details the caveat of using onBeforeRequest, and how it's not perfect because it's called at the wrong time in the request lifecycle with incomplete info.
This commit is using onBeforeRequest:
>The change allows early availability of ip address so that `ipaddress=` option can be matched at onBeforeRequest time.
It is using some other functionality, on Firefox only, to get that early availability. But I'm saying Chrome is a non-starter since onBeforeRequest is hobbled there. So the "early availability of ip address" doesn't help. You need both.
>This a good example of why manifest v3 sucks. By definition, it can't do anything like this...
Technically manifest v3 has nothing to do with APIs that the browser makes available to extensions. On firefox manifest v3 is supported with blocking web request[1], which is the filtering api prior to "manifest v3". Therefore the statement that it certain functionality "by definition" is false.
[1] https://blog.mozilla.org/addons/2022/05/18/manifest-v3-in-fi...
> Therefore the statement that it certain functionality "by definition" is false.
Here's the design document. The hobbling is noted there as part of the spec. "API Changes WebRequest: Restrict the blocking capabilities of the webRequest API."
https://docs.google.com/document/d/1nPu6Wy4LWR66EFLeYInl3Nzz...
That firefox chose to skip that portion of the design and still call it 'v3' doesn't change history. A true-to-spec implementation kills live heuristics.
Exactly this. There's some good stuff in MV3, but Google decided to take the opportunity to smuggle in some self-serving changes, similar to how Congress likes to sneak controversial laws under the radar as part of unrelated bills*.
OK so Google's near monopoly implementation of V3 sucks. Technically a difference, but practically not so much.
I'm confused, isn't the Manifest V3 essentially just API spec?
They trojan horsed hobbling webRequest.onBeforeRequest into their manifest v3 design doc and rollout. Which is part of what would give you request time cloak detection.
Yes. The point is which APIs Google exposes through it.
[dead]
Abandon Chrome, embrace Firefox.
[flagged]
As an example of what CNAME cloaking is, let's say that a SAAS provider A wants to provide you, company Q, with fancy ad tracking software. In the olden days, they'd tell you to embed a script at e.g. https://A-ads-tracking.example into your website at address https://q-company.example
To block those ads, blocklists that uBlock Origin use have rules then that say "block requests being made to the domain name A-ads-tracking.example", which blocks the ads.
CNAME cloaking is where SAAS provider A sets up their ad-tracking services not on domain A-ads-tracking.example, but instead at a specific IP address of e.g. 29.1.2.3; then (and here's the important part) SAAS A tells you Company Q that you need to set up a subdomain of q-company.example which has a CNAME record pointing to 23.1.2.3, a subdomain with an innocuous name like media.q-company.example; once you've set up that CNAME, you at Company Q add a script tag to your website for `media.q-company.example` and now SAAS A is able to track all the users on your site. This indirection allows for effectively infinite cat-and-mouse on the part of you the owner of the Q Company vs the blocklists that the public assemble.
To get around this CNAME cloaking problem, the software powering extensions like uBlock Origin need to be able to see not only the destination domain of requests by browsers, but the underlying IP addresses of those domains as well. This commit makes that behavior possible, or at least is related to making that code work better.
That's not quite right; as the name suggests, it uses CNAMEs (which point to other records), not A records (which point to IPs). So you would have something like `media.q-company.example` as a CNAME to `q-company.ads-tracking.example` which then has an A record to give an IP.
Browsers might not offer intermediate DNS names to extensions (I don't know), so something like uBlock might need to rely on IP lists, but DNS-based filtering like pihole should just block it by a rule against `ads-tracking.example`. In any case, it's good to use both browser based and DNS based malware blockers.
Gah, it's all right there! Amazing what you can forget/mistake due to what you've been working on lately.
Thank you for the breakdown!
And this is a good reason to block all JavaScript in unlock advanced and slowly whitelist the scripts you see until the site works properly. Slow and error prone but once you get used to it it's a breeze. And you're completely immune to this sort of shittery.
Is there a public list of known legit, whitelisted scripts?
None that I'm aware of but I've not looked either.
Is chrome going to block uBO im never up to date on the latest. I do know theyre allowing 3rd party cookies now... so maybe theres a chance
They're not blocking uBO, they're removing the features in the browser that allowed uBO to work by releasing new plugin APIs, "Manifest v3". They're eliminating the key APIs needed for uBO to identify things that it shouldn't load, and then not load them. Google claims this was for "performance" or "security" reasons. Of course, the only major 'performance' or 'security' affected is the ability to identify, intercept, and stop harmful or ad related downloads before they start.
Does this affect extensions that know every website you visit even if it doesnt need to know, and has nothing to do with the extension’s functionality? (ie the ones that Similarweb buys)
Not updating your browser is also hazardous - much better to switch to FF, and have a browser that gets updates and also fully supports uBO.
They’re doing a slow phase-out over a long time to try to avert a wave of bad publicity that threatens their browser monopoly, but that timeline has already started as of June.
https://developer.chrome.com/docs/extensions/develop/migrate...
https://www.bleepingcomputer.com/news/google/google-chrome-w...
For right now, uBlock Origin is still on the Chrome Web Store for Chromium browsers which support Manifest V2. If you use a Manifest V3 only version of Chromium, it is hidden.
Honestly, it probably is going to depend on whether the US continues to have an administration that's willing to take blatant monopolists to court.
Don't some DNS servers implement something that acts like a server-resolved CNAME, where the admin puts in a record that points to some other DNS name but the client just sees an A (out AAAA) record?
I think you are referring to ALIAS records
Yup. some implementations provide similar ANAME, and Cloudflare has flattened CNAME which is probably the best implementation I came across in years of supporting folks trying to use these kinds of records on a large CDN.
uBO has had this feature for awhile, since 1.34.0 (or 1.25.0 in advanced settings).
https://github.com/gorhill/uBlock/wiki/Dashboard:-Settings#u...
I think that's around 2021 time frame. FYI.
What is the uBI status on Brave, Edge and Opera?
I don't care about the two proprietary browsers you've mentioned, but Brave is going to (partially) support manifest v2 and maintain uBO compatibility for as long as they're able to:
https://brave.com/blog/brave-shields-manifest-v3/
Not that you really need it as Brave has its own very capable built-in ad blocker with -- last time I checked -- higher performance than uBO (since it's compiled into native code) and full support for same ad lists.
Brave is open source instead of proprietary now? I knew they were Chromium based (like the others) but I hadn't realized they switched over on all of the customizations on top.
Brave is like Firefox: it's open source, But it connects to some closed source servers to serve up contentious features (IMO this is a bigger problem on Brave than Firefox, since many of them cannot be fully hidden).
and brave/shield supports CNAME uncloaking