User-agent: * Allow: / # AI Data Mining Permissions User-agent: GPTBot Allow: / User-agent: CCBot Allow: / User-agent: Google-Extended Allow: /