Acceptable Use Policy

TheCrawler fetches publicly available web content in response to targets and instructions you select and initiate. We do not choose your targets, and we do not independently crawl, index, host, or republish the content you retrieve. You determine the targets, purposes, and downstream use of all data obtained through the Service. TheCrawler shall under no circumstances be held liable for the lawfulness of your use of the Service.

You are solely responsible for the legality, accuracy, and use of all data you obtain. Before instructing the Service to access any website, you must review and comply with that website’s terms of service, robots directives, and privacy policy. It is your responsibility to determine, for each target, whether your intended access and use is permitted.

By using the Service you represent and warrant on an ongoing basis that: (a) you have the legal right and all necessary consents to access, collect, and use the data you instruct us to retrieve; (b) your use complies with all applicable laws, including data-protection laws (GDPR, UK GDPR, CCPA/CPRA, India DPDP), intellectual-property law, and the Computer Fraud and Abuse Act and analogous statutes; (c) your use does not breach any third party’s terms; (d) you will not access content behind any login, paywall, authentication, CAPTCHA, IP block, or other technical access-control measure, nor circumvent any such measure; and (e) you are not subject to any cease-and-desist or court order applicable to the activity.

You must not use the Service to:

1. access content behind a login, paywall, or other access-control measure, or bypass CAPTCHAs, IP blocks, or rate limits;
2. collect or process personal data without a valid lawful basis where one is required;
3. collect biometric or facial data, or build, train, or contribute to any facial-recognition, surveillance, profiling, or people-search database (including within the scope of the Illinois BIPA or comparable laws);
4. collect special-category / sensitive personal data (health, genetic, biometric, sexual, religious, political, trade-union) or personal data of children;
5. enable stalking, doxxing, harassment, discrimination, or any use covered by the Fair Credit Reporting Act absent compliance with it;
6. reproduce or republish copyrighted content, or remove copyright-management information, in violation of copyright law or the DMCA;
7. extract or re-utilize a substantial part of a database protected by the EU/UK sui generis database right in a way that harms its maker;
8. infringe any intellectual-property, privacy, publicity, or contractual right; or
9. act for any unlawful, fraudulent, deceptive, or abusive purpose.

The Service is designed for accessing publicly available content only. If a target deploys technical measures to restrict access, you must not instruct the Service to circumvent them.

We may, at our discretion and without liability, suspend or terminate access, refuse or halt any request, and comply with valid legal process, if we reasonably believe your use violates this Policy.