The same thing happens every 3-5 years. HowStuffWorks, About.com (now like 10 different domains), many IAC acquired properties, RedVenture sites, even random sites like LiveStrong.com will be wildly prominent when the domains historically aren't relevant or authoritative for a given niche.
Even recently, sites like CNN were using subdomains with affiliate offers managed by third parties(1). These sites weren't being de-ranked algorithmically-- someone at Google would have to apply a manual action to remove them from the SERPs. What incentive would there be to do so if a prior agreement was in place?
Google doesn't really care about discoverability for smaller domains that may have good content. They are either being risk averse (avoiding potential spammers, junk AI content) by favoring trusted domains, favoring brands who are likely to spend on display or search ads, or maybe a combination of these.
1) https://searchengineland.com/google-begins-enforcement-of-si...
It's really frustrating. I currently want to buy a mattress and a refrigerator. The results for those are so awful as to be useless.
A lot of classic software essentially worked more like a database. In the last 10-15 years it's all moving to an algorithm.
Here is what I mean. Photos apps used to let you search through your photos using filters.
The same kinds of things are happening on the web which already happened to apps (desktop and mobile).
In the modern world, some marketing company wants to tell YOU which of YOUR photos you wanted, so they can sell you some prints, harvest your data, or something.
I would like any apps that have to do with collections of files, photos, music, etc to be more of a deterministic DATABASE and less of a nondeterministic algorithm.
> A lot of classic software essentially worked more like a database. In the last 10-15 years it's all moving to an algorithm.
You just described what I missed about the older software. Older software gives users control over sorting and show data in a tabular format. Modern software sorts data with an algorithm, with ads mixed in, and shows data in a card format, making it a lot less usable.
Exactly. My related observation: half of the SaaS products I see would be more useful and ergonomic for the user if they were implemented as an Excel sheet.
(I actually worked for one of such "better off as an .xls file" startup in the past, and its main competitor was an incumbent that sold the same stuff as an Excel extension. Trying to replace that with a React app is not a worthwhile use of life.)
Algorithms are fine. I'll happily apply the most advanced ones I can get. The problem is with who applies them to what - as you and GP said, it's about user control - or, currently, lack of.
Deterministic software puts the user in control of the product. Nondeterministic algos put the products in control of the user. Naturally companies want the latter and under guises of the ‘now better’ give the user worse and charge them more. A new generation isn’t even aware they’re being fleeced because they don’t have anything to compare with. And the frog boils slowly…
Mattresses have been especially bad for a long time. For refrigerators, you can look at consumer reports and wirecutter--and you can reasonably do some evaluation at your local big box appliance store. I wouldn't buy based on a random web search though.
It has always felt to me that Wirecutter focuses on only one end of the Pareto curve ("what is the very best XXXX that money can buy, within reason") and ignores the middle of the curve where most people are actually shopping ("what is the best XXXX that I can get for $XXX"). It also seems to reliably ignore brands from Mainland China (Hisense, Midea, etc). I guess It makes obvious sense to court rich (or at least price-insensitive) readers.
Their top pick for an air conditioner is a Midea: https://www.nytimes.com/wirecutter/reviews/best-air-conditio...
Whether or not it started that way, yes, it makes sense to recommend brands that New York Times subscribers are familiar and comfortable with. I'll buy a GE Profile refrigerator or Bosch dishwasher. Not some Chinese brand I've never heard of and have no idea what the service situation will be with. Makes perfect sense to me and I'm in that demographic. Especially with major appliances and things I can buy at the local big box store seems to make perfect sense to not buy things you have to go to Alibaba to obtain.
It's not about being price insensitive but recommending things that are relatively mainstream and that don't seem risky, especially for major purchases that have to be installed and potentially serviced.
(Did have a service issue on my recent GE Profile refrigerator but it took one phone call and was a no-brainer.)
But you're probably right in general. Wirecutter mostly doesn't recommend unknowns it thinks are potentially bargains. Which I probably wouldn't do in its position either.
Wirecutter's gone downhill after the NYT purchase as well. The Spruce seems somewhat better (but is also part of a huge web site family, so caveat emptor)
Either you do deep research, or you find a trusted friend to advise you. The Internet is largely useless at this point.
I think Wirecutter is still a decent source; they probably won't steer you too far wrong if you're not too picky. But nothing, including your trusted friends, is an all-knowing oracle if only because their tastes and priorities are probably different from yours. Certainly pre-Internet there were few enough reliable sources of recommendations--maybe some specialist magazines but even those were far from perfect.
> or you find a trusted friend to advise you
I think there's an opportunity here for a review platform that only shows you reviews from individuals that you personally trust. "Find a trusted friend" but for the internet.
The only signal I use is warranty. So I tend to go to Costco, and avoid Samsung.
It's not a bad idea to pair that with lawsuits related to such warranties.
Costco and Samsung are big enough that you can achieve reasonable signal.
When your local car dealer offers you a full drivetrain warranty though (assuming it's a full warranty and not one of the other ways people are often fleeced), will they honor it when the lemon they sold you breaks the first time? the 2nd? the 3rd? Will they, instead, note that most people buying that car don't have much money (or, if you used any form of dealer financing, know for a fact you don't have much money) and require you to retain a lawyer and sue them to recover any damages?
Try naplab.
How do we know you're not an ad?
>The same thing happens every 3-5 years. [...] Google doesn't really care about discoverability for smaller domains that may have good content.
What's galling is that (ostensibly) they used to care. So much for "organizing the world's information" and "don't be evil".
Well they are certainly organizing the world's information, and continue to be incentivized to do so. Deeply. For advertisers.
I get flashbacks to the exodus of Digg, when the admins basically said "Look, we get a lot of junk content and a lot of common source content so we are going to start fast tracking the common source content from trusted providers"
We all know how that went over.
Investopedia is another one. Same for bankrate.com Other ones included wikihow and genius.com
Forbes as a whole basically sold its soul for clicks. It used to be one of the three top business magazines depending on your preferences. After the web became dominant, at some point after Malcolm Forbes died, you ended up with a ton of blog writers--with plenty of biases and axes to grind--and essentially advertorial content.
It's a really good example of why one should not name your company after your surname. At some point if you sell your company, you are putting the surname of all your descendants in the hands of some other entity outside of your/their direct control.
Bit of a microcosm of the entire business world nowadays. Forbes made something - a magazine that produced enough good content to gain a reputation. In the new school of business, that's an asset, and assets are things we turn into cash as quickly as possible, so now Forbes sells CBD, and now anyone who sees "forbes.com" in the URL knows it's useless crap, but hey, someone made some money, and now they can go find the next thing to flip for a couple bucks.
Well what did you expect them to do? Magazine subscriptions fell off a cliff as the Internet matured and no one wants to pay for online content.
fortune.com too. same thing . these brands realized they can cash in on their domain authority to create content farms full of ads. scales really well too
Fortune's case was mostly the whole Time-Life magazine empire going to the dogs partly by way of TWB. Forbes just kind of faded away post-Malcolm (and with a distracted Steven Forbes).
But, in general, the magazine and journalism businesses aren't what they were so most of the relatively mass market magazines pretty much cashed in on their brands to the degree their owners decided to keep them around.
One of the best things about Kagi search (https://kagi.com) is you can ban domains from search results. Forbes was one of the first I entered!
The second best feature is that you can specify rewrite rules for URLs in results, using regular expressions. The example one is the most important one for me: rewriting reddit.com to old.reddit.com.
You can do a similar thing for Google with the uBlackList extension: https://chromewebstore.google.com/detail/ublacklist/pncfbmia...
This should come preconfigured with pinterest and quora blacklisted.
and pinterest; scourge of the web
I had honestly forgotten it existed until you mentioned it. Thanks, Kagi!
Very good article. Not clear to me why Google has let parasite SEO become so successful. Possibly they are starved of human generated content kept to a certain quality level. But it's very strange to see sites leveraging a legacy brand to expand far beyond their expertise. Forbes is the most prominent example.
On why Google allows bad quality results nowadays:
https://www.wheresyoured.at/the-men-who-killed-google/
Also discussed on HN earlier
One of the commenters on wheresyoured seemed insightful: "wonder if organic search results being worse generates more ad clicks, as the ads are more likely to be more useful than the actual search results".
Someone once described the state of mobile gaming on Android like this. Games that are good make less money. Games that are just good enough to get you to open them but are also just shitty enough that when you hit an ad in-game, you click on it and leave, make more money.
Google team has long given up its user’s needs for incremental revenue goals. See the 10 ads that come before any result
Since there is no competition and people will keep using Google whatever happens, might as well push the ad-filled garbage site than the ad-free handwritten blogpost. The former probably makes them more money, everything else humanity holds dear be damned.
Nobody has come up with a scalable metric for determining quality that can't be appropriated by SEO. Pagerank was one of the best for awhile (number sites that link to your site, weighted by their rank). Whether it be clicks, time on page, percentage of people who clicked onto the page then ended the session, etc it all gets gamed.
Like it or not, it's what the people want. The "trashy" movies, books, music, etc. all sell like wildfire, why do most people on hn think that the internet should be any different?
Complacency? Google has such a dominance in search that their name is used as a verb. Combine that with their culture of automating everything to an extreme degree. And the end result seems to be: search that is just good enough that people keep using it and requires little human fine tuning/curation making it cheap at scale.
Not to mention how flawed the current search tool really is. If you search for something, page 1 shows results from page 7 to some infinite number. But click on that large number, and you find out that the last page was page 3.
That was there for many years
It has to do with these old brands exploiting domain authority, plus buying tons of backlinks. Investopedia.com is another example of this. Google assigns too much weight to authority domains. Google doesn't actually penalize paid backlinks for old domains, I think.
Because Google makes money through all this. These move ads. That's all they care about at this stage. I had stated a few years back Google is dying. It will take a while and it's going to be painful but we will get over this soon. 20 years is a good run.
google search result can be shit and they will still make tons of money from 3rd party/publisher ads and youtube, cloud, gmail, atc.
There are 2-3 very detailed articles on how only a few media companies that own top few hundred domains have spammed SEO and hijacked top spots in search results. I made a list of block-able domains (dot dash meredith sites only). I have roughly explained how I searched these domains. https://gist.github.com/SMUsamaShah/6573b27441d99a0a0c792431...
Just copy paste this list to UBlacklist (or other tool). Need to sit down and search and add more sites including forbes someday.
I miss the days of a searchable internet
It's not just Forbes that is using this strategy. Many traditional media sites, including CNN and USA Today are running the same type of content. And of course they'll not report on this issue, which might just well be why Google is doing this, a kind of kickback for traditional media.
Actually googling some of the terms from his post and seeing Forbes up there is oddly surprising, even after reading it all.
"So we have $29M in annual revenue on an average of 3.4M searches per month in 2021." Is this real? That averages out to 40m searches, so .75 per. It seems insane to get close to $1 per search. I figured the return was closer to a penny or even a fraction of one.
When I did search (AdWords) for Thomas Cook, a now defunct but then major travel agency, we'd pay £10 or more to Google for a single click sometimes. (Full story: https://news.ycombinator.com/item?id=21046731)
Seems likely that the articles in question include affiliate links to the products they're advertising
Many types of sites get ~$1 per visitor from search engines. Quite possible, yeah.
if they are doing affiliate marketing than it's realistic
Is this an US thing? This has to be an US thing, right? How come I've literally never seen this in the EU?
I usually search in English and find SEO spam somewhat often, but never from these brands.
Just searched "best pet insurance", am inside Europe. Forbes is #1. I distinctly recall seeing this from time to time. Interestingly they're also the #1 for pet insurance on duckduckgo.
For me in the UK "best pet insurance" is squatted by The Telegraph and "best cbd gummies" has Forbes top and The Independent second.
I assume all media companies that have a "trusted domain" and are already involved in aggressive SEO are using this as a revenue generator.
The sites that turned exclusively into link farms like about.com could be whacked by Google eventually and everyone was happy. But if they try that with well known media brands there will be cries of censorship--whether it is collateral damage to some genuine journalistic content, or Google "taking away" a revenue stream.
"best cbd gummies" is interesting for me. I get a mix of "health" and vendors with some old media brands.
I get Forbes 4th and 5th
Chicago Magazine is 9th
The Independent, Kirkland Reporter and The Observer are 12th, 13th and 14th
About 10th in the UK with that search, so probably this isn't across every geography.
Just start browsing search results from the second page.
In Turkey, all searches hits to newspaper sites. Its like a sad joke. Related page is full of repetitive garbage where information is hidden somewhere.
This is not that different from this guy who posted how he 'stole' CEO traffic from his competitor.
https://news.ycombinator.com/item?id=38433856
And it is not that different (albeit at a smaller scale) from what websites like mini partition wizard has been doing. Their sitemaps are full of articles that don't relate at all to their tool:
https://www.partitionwizard.com/news_en_sitemap.xml
https://www.partitionwizard.com/partitionmagic_en_sitemap.xm...
All these 'articles' pollute search engines.
Damn, didn't realise that Forbes Marketplace was run separately to Forbes itself. Knew it was always a parasite SEO operation, but the idea of it being a separate company entirely (and how much they tried to hide the fact) is really interesting here.
But yeah, it's still crazy that this site is even allowed in Google, and that they've shown no signs of cracking down on these types of parasite SEO schemes.
Cory Doctorow article nails it https://doctorow.medium.com/the-specific-process-by-which-go...
It is Google "do no evil" to blame.
he really never misses
It's coming for all the old "legacy" web names with strong domain ranking and decades of backlinks. Private equity is snapping them up as fast as possible, loading them with ads, and bleeding the brand dry.
Right. I get offers for my decades-old domains every day or two now.
bingo. this is 100% the reason
So, Google can kill their whole business if they simply stop giving Forbes unfair prioritisation in the search results.
And then get negative press from Forbes for the next few years! Is that something useful for Google? No, better let them keep part of the money.
> negative press from Forbes for the next few years
Who the fuck reads Forbes anymore? You seen the garbage they're shilling on their website these days?
Sure, and Google and FB have killed businesses before by changing algorithms. But what is fair prioritization? Non-trivial Q.
I just did the “best pet insurance” search and once reputable sites came up.
- US News and World Reports
- CBS News
- Forbes
- Motley Fool
The entire web is a shit show.
I implemented something similar years ago for the publisher I worked for. Like 2006ish. It was after Katrina, I'd never had a full time dev job, and I created it as a POC to show to my employer that they could invest in me full time as a developer (I was helping put their magazines together when I wasn't working on their CRM).
I created a marketplace with finely tuned SEO for my employer to advertise (and charge) companies in niche industries. My SEO was better than the SEO of the developers who worked on their sites, and our audience was obviously much larger than theirs, so we ranked higher. Any time you would search for the company name or the product type in a certain geographic area, you'd find links to our pages dominating the search results.
One of the interesting things is the shenanigans some of these companies would pull to show up first in our local results. A whole lot of A1 and AAA names began to spring up as they decided that if the list was going to be alphabetical by default, then they needed to be the first in their category.
> One of the interesting things is the shenanigans some of these companies would pull to show up first in our local results. A whole lot of A1 and AAA names began to spring up as they decided that if the list was going to be alphabetical by default, then they needed to be the first in their category.
This well predates Google, though; it was a common trick for placement in the (physical) phone book.
This is such a big story and yet most of HN just doesn't care. It should make the WSJ though.
It isn't exactly news, though. This isn't a Forbes issue, or a Google issue. Pretty much every single large company is actively being ruined by parasites. We're dealing with a generation of CEOs / CFOs who were taught to care about nothing except short-term shareholder value. Quality and reputation doesn't matter anymore, so you replace your products with cheap garbage and hope nobody notices. When that inevitably fails, every single part of the company including its name is being torn apart and sold piece by piece, until nothing is left but an empty shell with a lot of debt.
We're intentionally ruining our economies and praising the people doing it. If the "Western" world gets economically steamrolled by Asia in the next couple of decades, we've got nobody to blame for it but ourselves.
> It isn't exactly news, though.
It's exactly news. It spots the issue, dives into it, exposes the source of it, and details the structure of how it came into existence. That's what news is. That you're not surprised by it is not material.
> we've got nobody to blame for it but ourselves.
Ironically you are the one who characterized this article as "not news."
> If the "Western" world gets economically steamrolled by Asia in the next couple of decades, we've got nobody to blame for it but ourselves.
Implicit in that statement is that only the "Western world" has that "short erm shareholder value" ethos. I'd say that is quite debatable.
> This isn't a Forbes issue, or a Google issue.
That's wrong. This is very much a Google monopoly issue.
Google has zero incentive to improve search for users since there is no competition. Google has every incentive to maximize the amount of money that search makes them.
Simply busting up companies with monopolies would fix 80%+ of the problems.
>We're dealing with a generation of CEOs / CFOs who were taught to care about nothing except short-term shareholder value. Quality and reputation doesn't matter anymore, so you replace your products with cheap garbage and hope nobody notices.
There is a line where you really do need to compromise on quality and even reputation to keep costs down, though. If you can't or refuse to do that, you end up stagnant and irrelevant like Japan.
Customers ultimately don't care how much sincerity and effort was infused into a product as long as it's past a certain "good enough" threshold.
Yes, there's a lot of crap out there but I also don't want to pay a huge premium for everything so that it lasts a lifetime (and will probably be outdated or out of fashion long before that).
Yes, this is a big deal, but most of us have simply stopped using Google and moved on to other tools.
> most of us have simply stopped using Google
Citation needed
This man is surprised that his google results are terrible. I have to assume this is a frog boiling thing and he's only just noticed the water heating up. Having switched to Kagi years ago, I'm immediately horrified by the state of Google if I ever end up on it. It's appalling and has been for a few years now.
Try those same search terms in Kagi and you'll see Forbes at the top of the results. I use Kagi and like it a lot but you should be aware that most of their results come from Google.
You can block Forbes in Kagi search, but not in Google.
Google does a lot of personalization of search results. But if you don't give it your searches, it won't know what you want. So when you come back after switching, it's much worse.
It's getting closer to the point where we will compare Google to Kagi like web-browsing without an ad-blocker. It's hell surfing the web with ad-blocking off.
Reading this comment just above your comment is funny:
Lol, guess you can't really escape. One benefit of Kagi is you can block websites from appearing. Maybe website blocking lists will appear similar to ad blocking lists. Maybe Kagi already lets you do this.
BlackHat SEO's have insiders at many of these companies that'll publish your article for $X amount of money. Or edit existing articles and insert your URL.
You don’t need an insider for you know where to just buy the link. They hang their shingle out on at least one link buying marketplace.
I suspect that's how geeksforgeeks got so dominant with their student written drivel.
I need the author to tell me again who runs Forbes Marketplace. I assume it's Forbes. It's not?