Back

Disable Crawler Traffic

Description

The following snippet is a urlRewrite rule to disable traffic from a crawler.  Just replace crawleragent with the User Agent of the crawler. The crawler will be sent a 403 Forbidden Response.

Code

<rule>
        <name>Disable Crawler</name>
        <from>^.*${esc.dollar}</from>
        <condition name="user-agent">crawleragent</condition>
        <set type="status">403</set>
        <to>null</to>
    </rule>

Have you created code you'd like to share with the dotCMS community?

Share your code Become a Contributor