ابزار ساخت فایل robots.txt بهینه شده برای سئو Robots.txt Generator Allow All Robots: Yes No Crawl-Delay: None 1 Second 5 Seconds 10 Seconds Sitemap URL: Block Search Engine and Social Media Bots: Googlebot (Google) Bingbot (Microsoft Bing) Slurp (Yahoo) DuckDuckBot (DuckDuckGo) YandexBot (Yandex) Baiduspider (Baidu) Sogou Spider (Sogou) Exabot (Exalead) ia_archiver (Alexa/Internet Archive) SeznamBot (Seznam.cz) Qwantify (Qwant) Facebot (Facebook) Twitterbot (Twitter) LinkedInBot (LinkedIn) Pinterestbot (Pinterest) TelegramBot (Telegram) WhatsApp (WhatsApp Crawler) Slackbot (Slack) Discordbot (Discord) SnapchatBot (Snapchat) WeChatBot (WeChat) Applebot (Apple Siri & Spotlight) AdsBot-Google (Google Ads Crawler) Googlebot-Image (Google Image Search) Googlebot-Mobile (Google Mobile Search) Mediapartners-Google (AdSense Crawler) Block AI Bots: OpenAI (ChatGPT) Anthropic (Claude) Google DeepMind (Gemini) Google Bard CrawlGPT (AI-powered Crawler) Dataminr Klarity AI Synthesia (Video AI) DeepAI RunwayML (Creative AI) Semantic Scholar Microsoft Academic Bot Wolfram Alpha You.com (YouChat) Perplexity AI Neeva AI CopyAI Jasper AI Block URL Parameters: Block All URL Parameters (e.g., /*?*) ?add-to-cart= (WooCommerce) ?orderby= (Sorting) ?per_page= (Pagination) ?stock_status= (Filter by Stock) ?filter_color= (Filter by Color) ?filter_age= (Filter by Age) ?utm= (Tracking Parameters) ?ref= (Referral Links) ?session_id= (Session Identifiers) ?fbclid= (Facebook Click ID) ?gclid= (Google Ads Click ID) ?campaign_id= (Campaign Tracking) ?tracking_id= (Custom Tracking IDs) ?s= (Search Queries) ?page= (Pagination) ?lang= (Language Selector) ?currency= (Currency Selector) ?preview= (Preview Parameters) Block File Extensions: .php .html .xml .js .css .jpg .jpeg .png .webp .avif .svg .gif .pdf .docx .doc .pptx .txt .mp3 .mp4 .ogg .wav .avi .mov .flv .exe .bat .jar .iso .dll .apk .psd Block Common WordPress Paths: /wp-admin/ /wp-includes/ /wp-login.php /wp-register.php /wp-content/ /wp-content/uploads/ /wp-content/cache/ /wp-content/themes/ /wp-content/plugins/ /wp-admin/admin-ajax.php /wp-admin/admin.php /wp-admin/edit.php /wp-admin/options-general.php /wp-admin/tools.php /wp-json/ /wp-admin/install.php /wp-cron.php /xmlrpc.php /wp-activate.php Block Additional WordPress Paths: /feed/ /search/ /author/ /comments/ /wp-json/ /trackback/ Block or Allow File Formats: .jpg .jpeg .png .webp .svg .gif .pdf .docx .doc .pptx .txt .xls .csv .mp3 .ogg .wav .flac .mp4 .avi .mov .webm .flv .zip .rar .tar .gz .7z .exe .apk .psd Generate Robots.txt
ارسال دیدگاه
ایمیل شما منتشر نخواهد شد.