.svg)
Botify Analytics is flexible. It's very easy to setup a website crawl and get advanced indicators. Very little information is needed to get started. But if you take just a few additional minutes to prove you own or manage the website, there are settings you gain control over, to crawl as fast as you want and set your own rules.
What you can do without verifying ownership of a website:
- Crawl the website with the following settings:
- Crawl speed: no more than 3 pages per second (updated)
- Number of URLS crawled per analysis: no limit
- Crawler user-agent: Botify user-agent
- Disallow rules: rules applying to Googlebot
- Emergency stop feature: enabled. The website owner is able to stop the crawl.(updated)
- Connect a Google Analytics account you have credentials for, to import traffic data and integrate it to the analysis.
- Get a full featured report: all sections of the report are available, as well as all indicators. The report is exactly the same as for validated websites.
Validate, and crawl Harder, Better, Faster, Stronger!
(updated)1) Crawl faster: no speed limitThe only limit there is is that of your web servers' performance and the load you are willing to apply on them. If you have a high-performance test environment, with no actual users, you can probably afford to turn up the crawling speed to hundreds of pages per second.2) Access to advanced option: ** Custom User-Agent**You can define the user-agent character string that the Botify crawler will use while exploring your website. This allows you to use a white-listed internal user-agent that gets special treatment, for instance.3) Access to advanced option: ** Virtual Robots.txt **You can instruct the Botify crawler to ignore your website's robots.txt file and follow, instead, the set of rules from the Virtual Robots.txt you have defined in Botify Analytics.4) Emergency stop feature removed: no need for this, you are in control through the Botify interface.
How to validate website ownership: choose from 4 easy ways
In your project (website) settings, select a validation method. Three of the proposed validation methods are automated. You have probably done this before, as these validation methods are similar to those provided by Google to validate site ownership in Google Webmaster Tools and Google Apps.1) A DNS TXT field:You will need to login to your domain account with your host provider and create the TXT record provided by Botify Analytics:2) A file at the website's root:Create an empty file named as indicated and place it at your site's root directory.3) A tag on the website's home page:Place the meta-tag provided by Botify in the section of your home page's code.Once you have entered the validation information on your Web server or in your domain account settings, go back to your project home page (website) anc click on "check" to validate the verification:4) Last option: an email to support@botify.com.If you encounter any problem using an automated validation method, don't hesitate to simply ask us! Email us from an email address on the same domain, or explain.You're all set to crawl with no limits!
.svg)
.svg)
.svg)
