robots.txt 指定搜索引擎抓取,屏蔽AI爬虫
User-agent: Googlebot User-agent: Bingbot User-agent: Yandex User-agent: Baiduspider User-agent: 360Spider User-agent: Sogou Spider User-agent: Bytespider User-agent: Shenma Spider Allow: / Disallow: /template Disallow: /static Disallow: /images Disallow: /message Disallow: /h5 Disallow: /viewerjs Disallow: /tj User-agent: * Disallow: /
