Imagine working hard on a brilliant story or article, only to find out someone else has taken it as their own. This is what happens when someone steals your website content.
Content theft, or "plagiarism," is a big problem for website owners. These people are pirates who copy your work and use it on their own sites, sometimes even passing it off as their own work. It's really frustrating and really unfair.
In this article, we’ll cover what blog scraping is, how to reduce and prevent it, and even how to use scrapers to your advantage.
What is Blog Content Scraping in WordPress?
Blog scraping is the process of grabbing content from multiple sources and republishing it on other websites. Often this is done automatically through the blog's RSS feed.
Unfortunately, it’s easy and common for your WordPress blog content to be stolen. If this has ever happened to you, you know how frustrating and stressful it can be.
Sometimes your content is copied and pasted directly onto another website, including your formatting, images, videos, etc.
Sometimes your content will be reposted with credit and a link back to your site, but without your permission. While this helps your SEO, you may want to only host original content on your site.
Why do content scrapers steal content?
Some users ask us why scrapers steal content. Usually, the main motivation for content stealing is to profit from your hard work:
- Affiliate Commissions: Dishonest affiliate marketers may use your content to drive traffic to their website via search engines in order to promote their niche products.
- Lead Generation: Lawyers and real estate agents may pay someone to add content and gain authority in their community without realizing that the information is being scraped from other sources.
- Ad revenue: A blog owner might scrape content to create a knowledge hub in a particular area for the “benefit of the community” and then plaster the site with ads.
Is it possible to prevent content scraping altogether?
In this article, we’ll show you some steps you can take to reduce and prevent content scraping. But unfortunately, there’s no way to completely stop those persistent thieves.
That’s why we’ve dedicated a section at the end of this article to taking advantage of content scrapers. While you can’t stop thieves completely, you might be able to generate some traffic and revenue from the content they steal.
What should you do when you discover someone has scraped your content?
Since it’s impossible to completely block scrapers, you might one day find out that someone is using content stolen from your blog. You might be wondering what to do if this happens.
Here are a few approaches people take when dealing with content scrapers:
- Do nothing: Fighting crawlers can be time-consuming, so some popular bloggers decide to do nothing. Google already considers well-known sites as authoritative, but this is not the case for smaller sites. Therefore, we believe this approach is not always the best option.
- Removal: You can contact the crawler and ask them to remove the content. If they refuse, you can submit a takedown notice.
- Take Advantage: While we actively work on removing content scraped from WPBeginner, we also use some tricks to get traffic and make money from scrapers. You can learn how to do this in the “Take Advantage of Content Scrapers” section below.
That being said, let’s take a look at how to prevent your blog data from being scraped in WordPress. Since this is a comprehensive guide, we’ve included a table of contents for your convenience:
- Copyright or trademark of your blog name and logo
- Make your RSS feed harder to crawl
- Disable Trackback and Pingback
- Blocking bots from accessing your WordPress site
- Preventing Image Theft in WordPress
- Prevent manual copying of your content
- Leverage content scraping tools
1. Copyright or Trademark of Your Blog Name and Logo
Trademark and copyright laws protect your intellectual property, brand and business from a number of legal challenges. This includes copying and unlawful use of your copyrighted material or brand name and logo.
You should clearly display a copyright notice on your website. While your website content is automatically protected by copyright law, displaying the notice lets people know that your content is copyrighted and that they cannot use your protected property for commercial purposes.
For example, you can add a copyright notice with a dynamic date in your WordPress footer. This will keep your copyright notice up to date.
This may deter some users from stealing content. It will also help if you do need to send a cease and desist letter or file a complaint with the DCMA to get the stolen content removed.
You can also apply for copyright registration online. The process can be complicated, but fortunately there are low-cost legal services available to help small businesses and individuals.
2. Make your RSS feed harder to crawl
Since blog content scraping is usually done automatically through the blog's RSS feed, let's look at some useful changes you can make to the feed.
Don’t Include Full Post Content in WordPress RSS Feed
You can include only a summary of each article in your RSS feed, rather than the full content. Summary content includes an excerpt as well as article metadata such as date, author, and category.
There is a real debate in the blogging community about whether you should use full or summary RSS feeds. We won't get into that right now, except to say that one advantage of using only summary RSS feeds is that it helps prevent your content from being scraped.
You can change the settings by going to Settings » Reading in your WordPress admin panel . You need to select the Excerpts option and then click on the Save Changes button.
Right now, RSS feeds will only show excerpts of articles. If someone were to steal your content through your RSS feed, they would only see the excerpt, not the full article.
Optimize your RSS feeds to prevent scraping
There are other ways you can optimize your WordPress RSS feeds to protect your content, get more backlinks, increase web traffic, and more. One of the best ways is to delay the display of articles in your RSS feeds.
The benefit is that when you delay the post appearing in your RSS feed, search engines have time to crawl and index your content before it appears elsewhere (such as on crawler sites). This way, search engines will see your site as authoritative.
The safest and easiest way is to use WPCode , as it has a recipe that automatically adds the correct custom code to WordPress.
3. Disable Trackback, Pingback and REST API
In the early days of blogging, trackbacks and pingbacks were a way for bloggers to notify each other of links. When someone linked to a post on your blog, their site would automatically send a ping message to yours.
The pingback will appear in your blog’s comment moderation queue with a link to their site. If you approve it, they’ll get a backlink and a mention on your site.
This creates an incentive for spammers to crawl your site and send trackbacks. Fortunately, you can disable trackbacks and pingbacks, giving scrapers less reason to steal your content.
Disabling the WordPress REST API
In addition to trackbacks and pingbacks, we also recommend disabling the WordPress REST API as it can make it easier for spammers to scrape your content.
All you need to do is install and activate the free WPCode plugin , then use its pre-made code snippet to disable the REST API.
4. Block bots from accessing your WordPress website
One way to stop crawlers from stealing your content is to ban them from accessing your website. You can do this manually by blocking their IP addresses, but most users will find it more convenient to use a security plugin such as a web application firewall.
Use a security plugin to block crawlers (recommended)
Manually blocking bots is tricky and a lot of work. Especially considering that many hacker attacks are carried out using a large number of random IP addresses from all over the world. Keeping track of all those random IP addresses is nearly impossible.
That’s why you need a Web Application Firewall (WAF) like Wordfence or Securi. They act as a barrier between your website and all incoming traffic by monitoring your website traffic and blocking common security threats before they reach your WordPress site.
For WPBeginner website, we use Sucuri. It is a website security service that protects your website from such attacks using a website application firewall.
Basically, all traffic to your website goes through the security service's servers, where it's checked for suspicious activity. They automatically block suspicious IP addresses from accessing your website.
Manually block or redirect the bot's IP address
Advanced users may also want to manually block the IP address of a crawler. This requires more work, but once you know the IP address of a crawler, you can target it with pinpoint accuracy. Web developer Jeff Star recommends this approach when he writes about how to deal with content crawlers.
Note: Adding code to your website files can be dangerous. Even a small mistake can cause serious errors on your website. Therefore, we only recommend this method for advanced users.
You can find the IP addresses of the crawlers by accessing the Raw Access Logs in your hosting account's cPanel control panel. You need to find the IP addresses that have an unusually high number of requests and make a note of them, for example by copying them into a separate text file.
Tip: You need to make sure you are not blocking yourself, legitimate users, or search engines from accessing your site. Copy the suspicious IP address and use an online IP lookup tool to learn more.
Once you are sure that the IP address belongs to a crawler, you can block it using the cPanel "IP Blocker" tool or by adding the following code to your root .htaccess file:
Deny from 123.456.789
Make sure to replace the IP address in the code with the IP address you want to block. You can block multiple IP addresses by entering them on the same line, separated by spaces.
Jeff suggests that instead of blocking the crawlers outright, you could send them some fake RSS feeds. You could create some RSS feeds full of Lorem Ipsum and annoying images, or even send them directly back to their own site, causing an infinite loop and crashing the server.
To redirect them to the dummy feed, you need to add code like this to your .htaccess file:
RewriteCond %{REMOTE_ADDR} 123\.456\.789\.``RewriteRule .* http://dummyfeed.com/feed [R,L]`
5. Prevent Image Theft in WordPress
Not only do you need to protect your written content, you should also prevent images in WordPress from being stolen.
As with text, there’s no way to completely stop people from stealing your images, but there are ways to stop image theft on your WordPress site.
For example, you can disable hotlinking of WordPress images. This means that if someone scrapes your HTML content, their images will not load on their site.
It will also reduce your server load and bandwidth usage, improving your WordPress speed and performance .
Alternatively, you can add a watermark to the image to indicate your copyright. This will clearly indicate that the scraper has stolen your content.
6. Discourage manual copying of your content
While most scrapers use automated tools, some content thieves may try to manually copy all or part of your content.
One way to make this more difficult is to prevent them from copying and pasting your text. You can make it harder for them to select the text on your site.
However, this does not completely protect your content. Keep in mind that tech-savvy users can still view the source code or use inspection tools to copy whatever they want. Also, this method does not work on all web browsers.
Also, remember that not everyone who copies your text is a content thief. For example, someone might just want to copy the headline so they can share your post on social media.
That’s why we recommend that you use this method only if you think your website really needs it.
7. Use content scraping tools
As the blog has grown in size, it has become nearly impossible to block or keep track of all content scrapers. We still send DMCA complaints. However, we know that there are a ton of other sites that are stealing our content, and we simply can't keep up.
Instead, our approach is to try to exploit content scrapers. It feels good when you find yourself making money from stolen content or getting a lot of traffic from the scraper's site.
Make a habit of internal linking to get traffic and backlinks from crawlers
In our ultimate guide to SEO , we recommend that you make a habit of building internal links. By adding links to other content in your blog posts, you can increase traffic to your site and reduce bounce rates.
However, there is a secondary benefit to scraping content. Internal linking can help you gain valuable backlinks from those who plagiarize your content. Search engines like Google use backlinks as a ranking indicator, so additional backlinks can be very beneficial to your SEO.
Finally, these internal links can also help you grab the audience of crawlers. Good bloggers will put links on interesting keywords to attract users to click. Visitors to the crawler website will also click on these links and return directly to your website.
Automatically link keywords with affiliate links to make money from crawlers
If you monetize your site through affiliate marketing , we recommend that you enable the autolink feature in your RSS feeds. This will help you maximize your revenue from readers who only read your site through an RSS reader.
Even better, it can help you make money from sites that steal your content.
Simply use a WordPress plugin like ThirstyAffiliates , which will automatically replace the specified keywords with affiliate links. We’ll show you exactly how to do this in our guide on automatically associating keywords with affiliate links in WordPress.
Promote your website in the RSS footer
You can use the All in One SEO plugin to add custom items to your RSS footer.
For example, you can add a banner promoting your own products, services, or content.
The best part is that these banners will appear on the crawler’s website as well.
In our case, we always add a small disclaimer at the bottom of the articles in our RSS feeds. This way we get backlinks to the original article from the crawler's website.
This will let Google and other search engines know that we are an authoritative site, and it will also let their users know that the site is stealing our content.
We hope this tutorial helped you learn how to prevent blog content from being scraped in WordPress. You may also want to check out our ultimate WordPress security guide , or our expert picks for the best WordPress content protection plugins .
If you liked this article, you can share it on Twitter and Facebook.
Disclosure: Some of the links in this article contain affiliate links, which means we may earn a commission if you click through to visit us, at no extra cost to you. See how SidelinePlay is funded, why it’s important, and how you can support us.
Was this helpful?