Robot like status 2013 spikes

I know, I know, this whole thing quickly spirals into the annoyingly tech-y realm of nerd that most website owners probably want to avoid. I mean, if you just want to write on your blog and maybe collect some adsense revenue, this whole process is a huge pain. You know what those bots might be doing in addition to screwing with your Google Analytics visits? Without going into too much detail about Adsense, if Google thinks you are trying to manipulate your Ad revenue through a surge of automated bots?

That said, since all of the server log and robots. For now, I hope this helps answer a lot of your questions about cleaning up your Google Analytics after a spam bot spike. Or feel free to mail the disgust to my wife and let her revel in the power of the multitude. Quick question — if a site is hosted on Shopify, could a bot attack of this sort cause any potential damage to the server or any other issues that we should be aware of?

Generally speaking, analytics spam like this does not do any damage to the server. BUT, not all bots are created equal, and obviously malicious spam can be a real concern. If you can deduce where the bots are coming from, and then potentially match an IP to your server logs a lot easier said than done , that should help put your mind at ease. If you have any PPC specific landing pages unique URL parameters, etc , you could check if these are included in the direct traffic surge.

My guess would be they are not. If you really want to play it safe, I found this post very helpful in terms of managing scams. Dave, thanks so much for this article. Then I decided to do some research and found you. Thanks for the comment, Tony — really glad it helped! Definitely a confusing situation, but fortunately one that can be managed.

Hi Dave, just been looking at this very scenario, and found your article a useful support guide. Hi Blaine, glad it helped! First time visiting, great information. I sometime do feel like Will Smith lol. Seem like when one is corrected he goes and get his other friends and just do a number on my site.

Thanks for the help! Hi Dave, Thanks for confirming that this is a problem! We are seeing an increasing amount of what we expect is bot traffic from various hosting providers — aws amazon being one of them.

We have the issue of applying many filters to multiple profiles per account — not very scalable. We are seeing this issue. We found, for January alone — 15 different IP addrs that are causing 1.

I was wondering how we could permanently block the direct fake traffic to our websites. Also, In an effort to stop the surge coming on my clients website, we anlayzed Google Analytics and figured out that visitors came from network domain: If we permanently block this domain amazonaws.

In an effort to start controlling fake traffic we have added the following major spammers to. We have the same situation here. So what can we do to avoid amazonaws. The bounce rate question is an interesting one. Google Webspam is on record as saying GA data does not impact rankings: We have used the following code in.

Order Deny,Allow Deny from amazonaws. Thanks for your article. It is good to share with people with the same problems. Those are very frustrating and the most frustrating is that, even though we can exclude this traffic, we will never know its real origin!

Much like everyone in here, I got a sudden surge in traffic from Bellingham WA where around users would constantly browse different pages of my site.

After following your instructions, I was able to narrow down the domain name to one single domain and guess what it was.. Not sure if I should filter this domain out or let it be since its msn. Your site will still get crawled just fine. What other solutions can be used?

Just search around and find this post, what i wanna ask is if i got some fake bot visit per day, is this will do a harm to my adsense? I mean, if this count for every unique IP, does adsense will punish me about this? My thinking is that Adsense and Google are savvy enough to discern between most spamming bot impressions. If the spamming really gets out of hand, it seems like a possibility, but generally speaking you should be safe.

Hello and thanks for the article. We manage 2 sites here and both have seen a massive increase in visits over the last month from Bellingham WA, from msn.

Any idea what is going on in Bellingham, who is running bots from there and why? How does one do this in Blogger? I hate that I am so pathetic at knowing what to do. You may need to create a robots.

The Google guide is here. Excuse my inexperience on these matters but I run a magazine-type news-based website on a specific industry sector. On average, about 5 items are added per day. Three days ago, a relatively low-grade in terms of newsworthiness news item started attracting unusually large numbers of visitors almost immediately after it was posted.

Also, while paying careful attention to the Real Time activity in googlestats, I watched the particular item being viewed from a number of specific cities. Some of these cities now do not show in the stats for this particular news item. This was super helpful. I found 3 networks that are the main cause of my huge spike in traffic and they appear to be bots. Could you provide some insight into where I can find that?

Hi Dave, thanks for the great post! Is there any way to filter out visitors from a network, using option 2? It would appear that the direct traffic started when our web platform Adobe Business Catalyst moved its hosting to Amazon in I would be interested in determining why Amazon Web Service would be visiting our site and why the sudden drop — have asked Adobe BC for an explanation.

I was left scratching my head a bit, as our traffic spike looked like a bot attack, but it is not coming from one source. It is all registering as direct traffic and is all bouncing, but it is coming from a wide range of US cities and networks.

Thanks for providing this detailed understanding of sudden spike in traffic. I was also digging down this issue and found my alexa rank has dropped from 9k to 30k following this spike. Putting filters or segments in analytics wont stop bots to crawl my website, if i use robot. Looking forward to hear from you. The numbers to the site are virtually the same for each.

This is only happening to client sites that are using the same VPS hosting provider. We looked at other clients using different hosts and they are not experiencing the same issue… Next stop, contacting the hosting provider and looking at server logs. Two of the clients are local-only bricks and mortar, so segmenting out the visits from Ashburn will help in Analytics, but I DO want to block our little friend from Ashburn on the server side!

I too get a heck lot of spam traffic according to the web statistics plugin I have installed for my website. Fortunately, Google Analytics is showing me only real traffic which is quite lesser, I must say. Great article with exact steps on how to drill down to locate the bot activity and how to filter to show real world traffic. A couple of places in your article sort of threw me and wanted to clarify. From the wording it was almost like GA was the target and not the website.

The bots were pointed at the website itself right? Google does not have an option to prevent bots from the site itself just the ability to filter the results after the fact. Is this correct or am I missing something within GA dashboard? It was the website that got spammed not GA? Or that the bots are actually targeting the Analytics instead of the actual website.

That is just to tell good spiders where not to go. Bad bots will take no notice of it. Thanks for taking the time to write this article.

The locations are spread evenly throughout the U. The people who opted in are real people. I found LinkedIn profiles that match the employer emails they used. Thanks for the info, but can you tell us what to click on in order to get to exclude filters? A big thank you Dave for this thread. I was infuriated to find Moscow was in Russia and why in the hell are they looking at a Lawn Service company …. I may use your info here to filter them, if it continues to grow.

Aside from making GA data a bit misleading, are there any other problems that these kinds of spam bots are causing? You mention a waste of server resources—are there any other issues that could come from this particularly from an SEO perspective?

A lot of it depends on how search engine algorithms register these bots. Why does it happen? Why these spam visitors? This is the magic question. I discovered a huge spike in traffic to my site, starting this weekend and growing daily. It is all coming from one of our affiliaties, but I was worried he was using bots or some other dodgy method to do this.. Having analysed the visits now though, Im unsure what to think! Our bounce rate has stayed low — and looking at these visits in isolation, their bounce rate too, has been OK.

In addition, there is not one, or two, locations the visitors are coming from, but they are scattered all over. Many in the US, but no obvious concentration in state or city. At the same time as this has occurred, our visibility in google has gone up rapidly after a dip following a restructure of our site.

Can I assume the traffic is actually genuine? Thank you for the great article! Now I understand why I have so much traffic from Moscow although my website is in English and my second one is in Japanese. Thanks for the article.

We recently started getting a ton of spam referrals from semalt, ilovevitaly and similar websites. They all come from Samara, Russia. We blocked their IP adress the one we could locate in wordpress, also filtered some of them out in GA. However, we are still getting these spam referrals. Does anyone know if there is something that can be done in wordpress to permanently remove all referrals from specific city or country?

My GA is a mess with the bulk of traffic coming to my sites being reported as Brazilian. I applied a filter to block all traffic from Brazil, hopefully that will make the analytics more useful.

Is there a way to track what they do or are looking for on your site? Thanks for this detailed article. I wonder though, would this work if you block Google? I did delete a secondary menu and moved the content to the primary menu. I got tons of traffic to all my sites using GA. All from Russia, now I excluded all traffic from Russia. Thanks for the great post! If you have an ECommerce site. We got hit 5 days after with stolen credit cards.

A great overview when it comes to bots who hit your website. However, with Google Analytics and other similar software you are also prone to a much worse attack: If this repeats, say an average of ten times every month, for over a year, are these likely to be real people visiting and not just bots?

I am going to work through this guide, I am getting traffic from the usa, 1 page view, percent bounce to my site.

More than direct hits and visitors from the US, location not set. Thanks Dave, Well written. Found it about 6 articles down on the first results page.

I discovered that my site has been targeted by bots for a long time. I found a neat trick that worked though. If you create a segmented list and specify that the session must be greater the 1 sec. Obviously thew bots are still crawling about though but do you think there is any problem with my simple method?

I hate to rain on the parade, but the catch here is that a visit that bounces will not register any average time on site. For example, running site: This was a maddening exercise for many SEOs, since the number shown could radically change and quickly.

Instead, it might raise red flags and prompt more questions. This report alone can signal potential problems. If you are continaully building content, the number should keep increasing.

If you are weeding out content, the number might decrease. For example, this is a relatively normal indexation graph. The site might have weeded out some older content and then began building new content which is why you see a slight decrease first, and then it increases down the line:.

But what about a trending graph that shows spikes and valleys? If you see something like the graph below, it very well could mean you are experiencing indexation issues. There may be legitimate reasons why this is happening, based on changes you made to your site. Once again, this is why SEO Audits are so powerful. For example, if you see the number of pages blocked by robots. But if this surprises you, then you should dig deeper. Depending on how you structure a robots. It would be smart to analyze your robots.

Speak with your developers to better understand the changes that were made, and why. You never know what you are going to find. Security Breach Index Status can also flag potential hacking scenarios. If you notice the number of pages indexed spike or drop significantly, then it could mean that someone or some bot is adding or deleting pages from your site.