This article provides free SEO advice to assist in the diagnosis and recovery from Google penalties and algorithmic filters, with a useful penalty checker allowing you to check your website for any non-compliances with Google’s Webmaster Guidelines.
KSL Consulting also offer a thorough Google penalty audit (including on and off-page SEO factors) followed by a one hour Skype consultation. The audit aims to provide an accurate diagnosis of the causes of the penalty and offers practical recovery advice. Visit our Penalty Consultancy Page for more information.
Finding the Causes of a Sudden Drop in Ranking
To check for a Google penalty with any degree of certainty requires analysis of recent events. If your website experiences a sudden fall in ranking for its main keywords, it may have been caused solely by an Google algorithm change such as the recent Penguin and Panda updates or by periodic organic search result fluctuations between Google data centers, influencing the SERP.
When any Google algorithm change is released, there are always winners and losers, and when a sudden drop in rankings are experienced Google penalties are often incorrectly blamed.
If you suspect a Google penalty, it first makes sense to cross-reference the Google algorithm change history on SEOMoz to determine when changes have been made and which one/s caused the problem.
When the traffic reduction from non-paid search is very extreme, as pictured above (from Google Analytics data – traffic sources > search engines > Google) then a penalty is much more likely, and the cause is typically due to the site contravening one or more Google Webmaster Guidelines.
There are a growing number of filters now built into the Google algorithm which aim to detect violations of Google Webmaster Guidelines in order to help improve the quality of the search results (SERP) for any given query. One such algorithmic filter is thought to have caused the massive drop of traffic pictured above.
Google Link Devaluation
When considering the causes of a sudden drop in Google ranking, it’s worth noting that the search engines continually apply link devaluation to links from known manipulative sources such as link networks that Black Hat SEO’s are using to manipulate rankings (the recent demise of the Build My Rank Network is one example).
Paid links are a another target for Google’s spam team. Hence they apply continual algorithm tweaks to combat this form of link spam, targeting both link buying and selling.
Sites using Black Hat do-follow forum signature spam as well as blog comment spam were targetted in the Penguin Update of 24th April 12.
When link devaluation is applied, as it has with reciprocal links as well as links from many paid text links, low quality web directories and link farms, the recipient website may suffer a reduction in Google ranking. The severity of any ranking fall is usually synonymous with the website’s reliance on that particular type of linking.
Links from low quality web directories have also been devalued as have backlinks from unrelated theme sites – so if your site heavily relies on these types of links, then it too may experience a sudden drop in Google ranking.
The most severe Google penalties lead to total website ban and de-indexing (I.E. the site doesn’t return any results when the site: command is used). Where the SEO misdemeanour is serious, a site ban may be imposed by Google accompanied by loss of Page Rank (grey indication).
With Google filters the website usually remains indexed with similar Page Rank, but SERP filtering is applied which all but removes the site for all or specific keywords.
Recovery from a Google penalty or filter is a challenge and our SEO checklist will help identify likely causes and reasons for any sudden reduction in Google ranking or major drop in SERP position for your main keywords.
Initial Test for a Penalty
When a penalty is suspected, start by checking the number of indexed URL’s. This can be accomplished by using the site:yourdomainname.com command within a Google search window. If no URL’s are indexed then there is a high probability of a penalty, especially if your site used to be indexed.
Where your site or certain pages have been dropped from the Google index, always check your Robots.txt file for syntax errors (if you have one) and investigate any recent Web Server outages. Google will remove websites which return a 404 server response when Googlebot crawling takes place. Also check your Meta “Robots” tags are set to ‘index’ and follow rather than ‘noindex’ and ‘nofollow’ your pages.
A quick Google penalty check can be made by searching for your exact company name and domain name. If you no longer rank for these terms, where previously you ranked well for your own brand name, a penalty is likely. The exception to this rule is a new website with few backlinks, which may not be Google indexed or lack enough trust to rank well.
Not all Google penalties result in a loss of Page Rank. For example, various filters can be triggered by unnatural irregularities in backlinks (detected by the automated Google algorithm) or by excessive link buying or reciprocal link exchange, particularly when using similar keyword optimised anchor text in your links. The example above shows a typical reduction in website traffic caused by an automated algorithmic penalty.
Another check that a site is under penalty is to take a unique paragraph of text from a popular page on the affected site and searching for it in Google. If the page doesn’t come back as #1 and the page is still showing as cached using cache:www.mydomain.com/page.htm, then this is a good indication that a penalty or SERP filter has been placed on the domain.
To avoid a Google penalty or SERP filter, take particular care when embarking on any link building program. In particular, avoid “reciprocal link exchange” and other link schemes. If you must buy links ensure that they are not placed in the footer of the site you’re listed on or under sub-headings such as ‘Our Sponsors’, ‘Our Partners’, or ‘Featured Sites’. Links placed naturally within text content on relevant sites are much more sensible and less detectable.
You should also check your site for non compliances to Google’s webmaster guidelines and (if appropriate) request website reconsideration from your Webmaster Tools account.
Interestingly, in a recent move by Google, some web sites which are in violation of Google’s webmaster guidelines or terms of service may receive an e-mail from Google advising them to clean up their act, warning of unnatural linking. 700,000 such emails were sent out in Q1 2012.
When the breach of Google’s terms (e.g. link spam or hidden text) is removed from the offending site, they will usually automatically clear the penalty and re-index the site as many so-called penalties are actually ‘filters’ triggered by irregularities found by the algorithm. Google uses a “machine learning algorithm” to identify low quality sites and poor user experience signals.
If your website has suffered a Google penalty, we provide a checklist to help diagnose the cause and solve the problem below. We also suggest watching the Google reconsideration tips video to help prepare a successful reconsideration request.
For personalised assistance with Google penalties see our Penalty consultancy service.
- Links from or to Google banned sites –Run a test on all outbound links from your site to see if you are linking to any sites which have Google penalties or bans. Banned sites have no cached pages in the Google index, so check this with the site:www.domain commandfrom a Google search window. Sites under penalty will not rank well for their target keywords.Conversely, if your site gets a significant number of inbound links from banned sites or manipulative Black Hat link networks, then your website ranking may drop as a result. Always try to remove links from Black Hat link farms or banned domains if possible. Above all adopt ethical link building practices to avoid these problems!
- Linking to bad neighbourhoods –Check you are not linking to any bad neighbourhoods (neighborhoods – US spelling), link farms or doorway pages. Bad neighbourhoods typically include spam sites and doorway pages, whilst link farms are sites consisting predominantly built for linking, with no original or useful content.If in doubt, we recommend quality checking all of your outbound links to external sites using the Bad Neighborhood detection tool. Whilst this SEO tool isn’t perfect, it may spot “problem sites”. Another good tip is to do a Google search for the HTML homepage title of sites that you link to. If the sites don’t come up in the top 20 of the SERP, then they are almost certainly low trust domains and linking to them should be avoided.
- Automated query penalty –Problems can be caused by using automated query tools which make use of Google’s API, particularly when such queries are made from the same IP address that hosts your website. These tools break Google’s terms of service (as laid out in their Webmaster Guidelines). Google allows certain automated queries into its database using its analytic tools and when accessing through a registered API account. Unauthorised types of automated query can cause problems, particularly when used excessively.
- Over optimization penalties and filters –In-bound link over-optimisation is the single most common causes of a Google penalty. Poor SEO techniques such as aggressive link building using the same keywords in link anchor text often causes problems, especially where there has been a high link accrual rate and the website is relatively new.When managing link building campaigns always vary the link text used and incorporate a variety of different keyword terms. Make sure you have a fair spread of back-links which use your official company name or your URL in them and that links are acquired not just to your website homepage.
Use a backlink anchor text analyser tool to check backlinks for sufficient keyword spread. Optimising for high paying (often abused) keywords like “Viagra” can further elevate risk, so mix in some long tail keywords into the equation. For brand new domains, be sensible and add a few one way backlinks a week and use deep linking to website internal pages, rather than just homepage link building. Above all, always vary your link anchor text to incorporate different keywords, your brand name and URL – not just incorporating variations on the same keyword!
There is strong evidence that Google has recently introduced some new automatic over optimisation filters into their algorithm. These seem to have the effect of applying a penalty to a page which has been over optimised for the same keyword by link building. See Google filters for more information or contact KSL Consulting for assistance (fees apply).
- Website cross linking & detectable link schemes –If you run more than one website and the Google penalty hits all sites at the same time, check the interlinking (cross linking) between those sites. Extensive keyword optimised interlinking of websites, particularly if they are on the same C Class IP address (same ISP) can be viewed as “link schemes”, breaking their terms of service.In addition, link schemes offering paid link placement in the footer section of webpages (even on high Page Rank pages) are detectable search engine spam and best avoided.
Site-wide links such as those offered in a Blogroll should also be avoided, particularly if keyword optimised. The reality is that site-wide links (when compared to a single link from the same source) do little to further improve visibility in the SERP, nor do they improve Page Rank more than a single link, as Google only counts one link from a site to another. KSL Consulting also believe that Yahoo! now applies a similar policy. There is some evidence that the extensive use of unnatural looking site-wide links can lower the Google trust of a site, which can subsequently reduce ranking.
- Duplicate Content problems –Duplicate content will be picked up by search engine algorithms. Avoid content scraping (almalgamating non-original content from multiple sources) and using large amounts of off-the-shelf “brochure content” to describe products, if this content is used elsewhere. News feeds (through their very nature) can also cause problems through content non-originality.
- Duplicate Websites & Domain Aliases-Many webmasters fail to realise the negative effect of having multiple sites on the web serving up the same content. This causes massive problems for search engines – which endeavour to decide which page/s to index. Having duplicate content domains also dilutes your link equity as there is a tendency for people to link to more than one domain, spreading links across multiple domains. Aliased domains must be removed (retaining the domain with the most search engine visibility) and each one 301 redirected to the domain being kept. If multiple domains are being used to target different geographic regions then the content of each needs to be unique.Algorithm changes made around the time of the Google Panda update have tightened up on duplicate content and Alias Domains can trigger the removal of website content from the Google index. Search engines seek to maintain the quality of the SERP and duplicate content is removed wherever possible.
- Hidden text or links –Remove any hidden text in your content and remove any hidden keywords. Such content may be hidden from view using CSS or alternatively, text may have been coded to be the same colour as the page background, rendering it invisible. These risky SEO techniques often lead to a Google penalty or web site ban and should be removed immediately. The same applies to hidden links, which Matt Cutts has openly stated break Google’s webmaster guidelines.
- Keyword stuffing (spamming) –Remove excessive keyword stuffing in your website content (unnatural repetitions of the same phrase in body text). Always use natural, well written web copywriting techniques.
- Check for Malware Problems –It is worthwhile carrying out a check to see if Google has blacklisted your site as unsafe for browsing. To assess whether this is the case visit
replacing ‘mydomain.co.uk’ with your domain.
- Link buying or selling –Check for any paid links (I.E. link buying of text links from known link suppliers or text link brokers). There is significant evidence that link buying can hurt rankings and Matt Cutts (head of Google’s spam team) mentions this on his SEO blog. Matt states that Google will also devalue links from companies selling text links, such that they offer zero value to the recipient in terms for improving website ranking or Page Rank. In the past they have applied a Page Rank penalty to known link sellers and many low quality directories who “sell Page Rank”.
- Reciprocal link building campaigns –Excessive reciprocal linking may trigger a Google penalty or cause a SERP filter to be applied when the same or very similar link anchor text is used and large numbers of reciprocal links are added in a relatively short time, leading to a high link accrual rate. We recommend that reciprocal linking be restricted to companies you have some business relationship with, rather than being done solely for SEO benefit. Optimised anchor text should be avoided. For example two companies who work together may need to link to one another legitimately.Lots of reciprocal link building with low quality sites or websites which have an unrelated theme is not recommended. This can lead to a Backlink Over Optimisation Penalty (known as a BLOOP to SEO experts!). This causes a sudden drop in SERP ranking (often severe). To avoid the problem, reciprocal link exchange should only be used in moderation as part of a more sustainable SEO strategy which also builds quality one way links to original website content.
- Paid links on Commercial Directories –Some leading online web directories offer paid links on multiple directory pages. These can be keyword optimised anchor text and search engine accessible (I.E. they have no “nofollow” tag).If you have optimised the same keyword elsewhere in your SEO campaign, adding hundreds of links from directories all with the same or similar anchor text can cause serious problems, as search engines monitor the link accrual rate as part of its algorithm. In extreme cases we’ve seen these kinds of directory links trigger a Google filter.
- Thin Affiliates and “Made for Adsense MFA sites” –It’s a well known fact that Google dislikes affiliate sites with thin content and the same applies to “made to Adsense” sites. Always make sure affiliate sites have quality original content if you don’t want to get them filtered out of the search results when someone completes a Google spam report.
- Content Feeds and I-Frames –Whilst content feeds (including RSS) are widely used on the web, there is some evidence that pulling in large amounts of duplicate content through such feeds may have an adverse effect on ranking; particularly after the Google Panda update. In particular, the use of I-frames to pull in affiliate content should be avoided where possible. Consider the use of banners and text links as an alternative.
- Same Registrant Domains –As Google has access to the WHOIS records for domains and is known to use this information, it is possible that a penalty applied to one website may reduce the ranking of other websites with the same registrant, although most filters only affect one domain.
- Non compliance with Google Webmaster Guidelines –Read the Google Webmaster Guidelines and check website compliance in all respects. Since early 2007, Google may alert webmasters via the Webmaster Consolewho they feel might have unknowingly broken their guidelines.However, blatant spam or significant breaches of Google’s rules will often result in a site being banned or filtered from the SERP, with no Webmaster Console notification. Where notification of a violation of webmaster guidelines is received, it usually encourages the webmaster to correct the problem/s and then submit a Re-inclusion Request (now referred to as a ‘reconsideration request’ in Webmaster Tools). From my experience, after this is done the website will usually regain its original ranking in around 14 days, assuming that all violations of Google’s terms and conditions have been resolved.
- Check Google Webmaster Tools –Matt Cutts’s Blog reported some time ago that Google has improved webmaster communication with respect to banned sites and penalties. They are now informing some (but not all) webmasters the cause of a website ban or penalty, via their excellent new Webmaster Console. In addition, a Google re-consideration request can be made from the same interface. For this reason, if you’ve been hit by a web site ban or penalty, it is worthwhile signing up for Google Webmaster Tools and uploading an XML Sitemap onto your site and then to check site status in the Webmaster Console. This is an easy 15 minute job and may help to identify the cause and fix for the problem!
- Preparing Your Site for Google Reconsideration –Google recently prepared a Google reconsideration video tutorial on how to create a good reconsideration request, including tips on what they look for when assessing the re-inclusion of any website. The video tutorial is presented by actual members of their reconsideration team and is very helpful to any webmaster looking to successfully prepare a reconsideration request.
May 2010 Update – There is clear evidence that over-optimising a single keyword by adding too many backlinks or site-wide links can trigger a Google filter whereby the recipient page of these links no longer ranks in the organic SERP for the keyword being optimised.
Affected page/s are still Google indexed and cached (using the site:domain.com command). The Trust Rank of the website may be affected leading to a ranking reduction for other keywords. Interestingly though, affected websites can retain ranking for other long tail keywords which have not been over optimised, particularly on pages which have not been subject to aggressive link building, but may have one or two decent natural links.
One other fact worth noting is that affected pages seem to have high keyword density to the point of being over-optimised.
To assess whether your website is affected by a Google SERP filter, do a site-wide backlink anchor text analysis using Majestic SEO, Link Diagnosis (free) or a paid SEO tool like SEOMoz Linkscape and check the spread of keywords used in links to your page look natural. Check your keyword density too excluding Meta tags.
Google Penalty Recovery Strategies
Recovering from a Google penalty normally involves fixing the cause of the problem/s and then waiting for them to remove any over optimisation penalties or SERP filters. To fully recover Google ranking may take around 2-3 months after all website problems are corrected, although we have seen penalty recovery in a matter of weeks following full and thorough resolution of the Webmaster Guidelines infringements.
The Google algorithm can automatically remove penalties if the affected website is still indexed. To check whether a particular website is indexed, refer to our Google indexing page. If your website has been de-indexed and lost Page Rank, then you will need to make a re-consideration request. Where the reason for the penalty is clear, it helps to provide details of any changes you’ve made to correct violations of the Google Webmaster Guidelines.
The best recovery strategy from any Google penalty is to thoroughly familiarise yourself with their Webmaster Guidelines. Don’t forget to check your link building strategy as poor SEO strategies often cause Google penalties. Start by removing any reciprocal links to low quality websites, or sites having no relevance to your website theme.
Preparing for a Reconsideration Request
We recommend you start by watching the Google reconsideration tips video.
If your site has been affected by a Google penalty, correct the problem and then apply to be re-included in the Google index by submitting a re-consideration request from your Webmaster Tools account. More information about this is provided in Google Webmaster Help. This is now undertaken from your Webmaster Tools account.
How long does site reconsideration take?
By submitting a reconsideration request to Google you enter the queue for the manual review process whereby your site is manually checked for violations of their Webmaster Guidelines. This can take several weeks. At the end of the process, an Inbox message is usually sent to the Webmaster to confirm that the reconsideration has been processed. This will be visible by logging into Webmaster Tools and then checking your Inbox under ‘Messages’.
A typical message (as shown above) merely says that the domain in question has been reviewed and that it will be considered for re-inclusion in the Google SERP assuming that all violations of Webmaster Guidelines have been corrected.
Following this, if the domain does not recover from the penalty, the Webmaster should undertake a more thorough website audit paying particular attention to backlink anchor text, recent backlink acquisition rate and the removal of any known paid links.
Majestic SEO can help in this regard by producing graphical representations of link trends. Note however that Majestic’s backlink data may not be quite up to date.
If you’d like more help with your specific Google penalty, filter or indexing problem, contact KSL Consulting for professional SEO advice. Penalty consultations are normally held over the telephone or by Skype at a pre-arranged time/date. The cost is £250 + VAT which includes a 2 hour SEO audit and an hour’s conference call to discuss the findings. These sessions are paid in advance by Paypal or Google Checkout. Terms and conditions apply.