Feb 23 2011
To Improve Search Quality, Google Must Penalize Sources of Link Pollution
If you’ve been in SEO for more than a few weeks, it’s a safe bet you’ve seen poorly moderated blog comments or forum threads that have been polluted with spammy links. It’s also likely you’ve found evidence of “link hacking,” the unauthorized placement of links on an otherwise quality website via some sort of hacking mechanism. These are forms of link pollution, and much like real-world pollution, they can damage our search environment if left unchecked.
A must-read example of link pollution is documented in an SEOMoz blog post from late January, which shows that blog comment spam, forum spam, and hacked links placed on trusted websites led to spammy Google search results on a variety of competitive terms. This post is one of dozens that illustrate the immense task faced by Google and Bing: not only must they deliver excellent search results, but they must do so while dealing with an ever-increasing amount of link pollution.
While some people propose algorithmic adjustments to counter the effects of link pollution, I think it’s time to put a simpler and more obvious solution on the table: force website owners to take responsibility for allowing this pollution in the first place.
To be blunt, link pollution is often caused by poor and/or incompetent website management.
- Moderation of blog comments and forum posts has never been easier – there is no excuse for spammy links on blogs and forums, yet this problem doesn’t seem to be going away.
- When we find links that were placed by hackers, we’re reminded that password security is often the culprit. A 2010 password study shows that nearly 50% of Internet users have easily compromised passwords, and it’s a safe bet that many FTP passwords, WordPress passwords, etc. fall into the “easily compromised” category. How else could all these hacked links be explained?
While it’s true that even the best security and anti-spam measures can be compromised, it’s also true that these occurrences are relatively rare and – if a site is properly managed – easily corrected. If Google created a system that penalized irresponsible website management, link pollution would be drastically curtailed. Here is what I would propose:
1. Google should only index sites registered with Webmaster Tools. Assuming that all website owners valued Google search traffic, Webmaster Tools registration would be nearly universal. This would give Google a direct connection to website managers.
2. Mandate regular interaction. If Google were to send website managers a monthly “suspicious link report” – and then require the website manager to acknowledge receipt of this report – they would encourage website owners to actively manage the security and quality of their website.
3. Alert website managers to obvious spam links. While it is impossible for Google to detect every instance of link spam, there are certain occasions when a site has obviously been spammed. In these instances, Google should email the website manager immediately and request that the site be fixed ASAP.
4. De-index sites that fail to acknowledge alerts. Once Google has contact information for each and every website owner, there’s no excuse for failing to respond to Google spam link alerts in a timely manner. If a website manager fails to remedy a warning within a certain time frame, the site should be de-indexed for a period of days or weeks. Repeat offenders should be de-indexed for longer and longer periods until they either a) fix the problems or b) give up and go away.
While my plan to de-index non-conforming websites might sound draconian, imagine the benefits:
- Webmasters would close security gaps, reduce the incentive for hackers to attack websites, and subsequently make all of our sites a little safer.
- Blogs and forums would improve their moderation systems.
- A reduction in link spam would probably make paid links easier to detect.
- The rewards for ethical link-building would become even greater.
- Social sites like Twitter might do more to reduce incredible amount of spam generated on their platforms.
- Most importantly, Google would provide consumers with a better search experience.
To those who would argue against my plan on the idea that universal webmaster registration would make Google too powerful, I would submit that the current “unknown webmaster” free-for-all has no basis in reality. If we can’t drive a car without a driver’s license, why should we expect Google (or any other search engine) to send visitors to our website if we don’t identify ourselves to them first?
Besides, it’s not as if participation in my plan is compulsory. If a website owner doesn’t want to register with Google, they don’t have to…they just won’t be listed in Google search results.
To be clear, I’m not picking on Google. Google’s results are only as good as the websites they index. If tens of thousands of poorly managed websites are compromised with spammy links, it’s unreasonable to expect any search engine to overcome this problem algorithmically. My plan is focused on Google because of their prominence in the marketplace, but Bing could just as easily take action. Perhaps Google and Bing could even collaborate.
Whatever the solution, it’s clear that the current system has a fundamental flaw: there is no penalty levied against websites that are a source of link pollution. While our search environment is resilient enough to deal with some link pollution, regulation is the only way to prevent long-term damage.
Written By:
Jason Lancaster | Denver Internet Marketing | @sporkmarketing
Jason Lancaster is President of Spork Marketing, a Denver Internet marketing company specializing in search engine optimization, marketing, and web design.
More Posts By Jason Lancaster
A brilliant post on how to tackle the problem of link pollution.
I hope the search engines adopt the suggestions mentioned in this post which would automatically punish the spammers and reward the ethical methods and thereby ensure quality results in SERPS which would be fair to the SEOs, website owners and the search engines.
As the saying goes about better mousetraps and smarter mice, Google (and other search engines) find themselves facing an uphill battle. Every time they try to clean up the search results, someone is going to find a new way to game the system. I do agree that website owners should take a more active role in managing the links posted to their site, but it’s going to take the combined efforts of website owners and Google to help clean up the search results.
Absolutely correct. Just like changes to the algo aren’t enough to combat spam, registration isn’t enough by itself either. Both are needed.
Thanks!
We’re working in the very first years of what will someday be a massive, established industry, and my guess is that someday people will look back and marvel at how an anonymous website with no caretaker could possibly receive search traffic.
Hey Jason. Interesting article with some valid points. I think in general it highlights problems with the whole model of a link economy for determining websites’ rankings. While Google et al. have made a some advances in revising this model (for example, by now factoring in social signals as a new “vote” metric for users’ views of a web resource’s trustworthiness and usefulness), the basic model is still in some ways based on relatively primitive bean-counting – as evidenced by the recent JC Penny fiasco.
But I have to say that your proposed solutions are preposterous – most of all the requirement that website owners be required to register with Webmaster Tools in order to be included in Google’s index (and your remaining solution bullet points are predicated on this).
Sure, Google could do this – and pretty much immediately cease to be a search engine that anyone looking for comprehensive information on the web would ever use. Google is in the business of discovering, indexing and ranking as many possible web-delivered resources as it possibly can, regardless of whether or not site owners even care if search engines exist. With this goal in mind, Google can’t afford to be draconian in who gets into their index: they’re after information, not compliant web owners. Millions upon millions of site owners have never heard of SEO, don’t care (even if they “should”) about search-engine derived traffic, or otherwise see their output as a part of the search landscape. This in no way means that their content isn’t potentially valuable – and in fact may be produced solely with engaged users in mind, regardless of a presence in search. Google wants this content in its index, regardless of whether or not the owners of this content care about it’s existence. This is why Google has an opt-out mechanism (robots.txt or meta robots exclusion), but not an opt-in mechanism (if we can find your resource we’re putting it in our index unless you specifically tell us not to). The single compelling reason why a user might agree to mandatory WMT registration – because Google is the biggest, best and most comprehensive of the enterprise search engines – would immediately be voided by this very requirement, as it would cease to be comprehensive, and by dint of that no longer the best, as it could be usurped by any non-authoritarian engine with a decent ranking algorithm.
With your proposed solution you’re not actually describing what *Google* should do, but what *site owners* should do in order to correct deficiencies in Google’s algorithm: be listed in a register so they can be made to comply with Google’s demands. Yes, Google needs to do something about comment spam, including penalizing site owners that violate their TOS – but most site owners (especially sites that don’t have a comment mechanism or security vulnerabilities – probably the vast majority) are already in compliance, and shouldn’t have to prove it. The ball is in Google’s court.
Of course you make an excellent point – Google would cease to be the dominant search engine in the marketplace if they no longer indexed all information.
However, I would say that:
1. The rule would be implemented slowly – webmasters would have months to comply. It’s not as if things would disappear overnight.
2. Good quality sites would comply immediately. I can’t imagine that anyone with a business that relied upon search engine traffic would hesitate to play alone, let along publishing companies, individual bloggers, small business owners, etc. I guess what I’m saying is that every who matters would participate. If they don’t want to register, there’s a good chance they don’t matter.
3. Google wouldn’t have to go it alone. Link pollution is a serious problem, and by some accounts it’s getting worse every year. Google and Bing both need to get rid of link pollution sources if they want to improve the quality of their product. If Google and Bing both mandated registration, the market would have to comply.
4. Regulations, like taxes, are used to influence individual behavior. While you’re correct in saying this isn’t about Google should do, it’s really just semantics. Google is the big player in the search world, and until they create a penalty for poor site management – a “tax” if you will – insecure and unmoderated sites are going to hurt our search index. It’s incumbent upon Google and Bing to combat this problem, because they are the main beneficiaries of the current system.
The bottom line is that it’s impossible to solve link spam problems with an algo, especially when link spam is very carefully placed. Google’s results are only as good as the data they process…spam in = spam out.
Also, you touched on the idea that social signals could be used to help evaluate quality, but I see that as trading one form of spam for another. Until social profiles are verified and tied to specific individuals – and until all social data is open (i.e. Facebook), there’s no way social signals are enough to evaluate quality. The long and short of it is that links are still the backbone of the search index, and I don’t see that changing.
Thanks for the comment.
Some great points Jason and i do appreciate your efforts and i really wish that search engine adopt this in order have better search engine for tomorrow… but it didnt seems easy…
Take your 4th point de-index the website from the search engine if the webmaster is failed to respond to search engines email… i personally believe by this you are limiting the search engine…. what if there is a great informative website on the web and contain some spam links… search engine reports the owner then there can be several of reasons why one get failed to respond to search engine.. one of the legit excuse is some uncertain problem (Newzeland earthquake)
same goes for the point that says Google should only index the site that is registered by Google Webmaster tool… don’t you think by this search engine will be limiting to search engine only?
Over all an interesting Article!
It would be limiting if we assume that people wouldn’t choose to participate.
I agree that there are some implementation questions, but at the end of the day something needs to be done. I’d like to see Google and Bing work together…and perhaps start in North America and eventually move worldwide.
The hardest thing about requiring something like webmaster tools, is that there is a lot of web content out there that is not moderated by a “webmaster” but rather a normal less-knowledgeable website owner who built their site via a free WYSIWYG application such as weebly or Godaddy.
Great article though, and I agree with you that something needs to be done.
Matt – To be quite honest, I think it would be great if the cheap, poorly constructed and poorly maintained “website tonight” websites didn’t exist. In my opinion, there’s no excuse when a business doesn’t put forth a minimum investment to build a legitimate, quality website. I hear business owners tell me that they can’t afford a website, then I watch them spend hundreds of dollars on an ad in the Yellow Pages.
In other words, I agree that less-knowledgeable website owners will struggle – I think that’s one of the benefits! 🙂
I’ll play a little bit of devils advocate on this one.
I do agree with you on the fact that the web needs less junk and more qualified pages but there are exceptions. For example, I love cooking/recipe blogs, these blogs are often not ran by a webmaster, a professional chef or a business, but by an individual with a hobby and desire to share their goodness.
In this regard, the web becomes less “free” (which is good/bad), but content regulation must be put into place at some point.
Thanks for the reply!
I hear that, and I think some of my favorite blogs wouldn’t make it under this new regime. Still, if Google made the process easy…maybe hooked up with ICANN…this wouldn’t have to be a stumbling point.
I was thinking earlier today that Google could offer business owners free custom email addresses (via Google apps) when they register for Webmaster tools. That would give them another incentive and also help expand Google’s reach.
All of the comments make one thing clear: No solution is perfect. Mine could definitely use some tweaking, but ultimately I would argue that it’s absurd for Google to send traffic to sites that it can’t verify as “real”. At some point this will have to change, and registration is a great way to prove a site is legit, cut down on spam, reduce the impact and likelihood of paid links, etc.
This is a troll right?
Ridiculous.
To Improve Search Quality, Google Must Penalize Sources of Link Pollution | Search Engine Journal
Leave a reply
