ابزار ساخت فایل robots.txt بهینه شده برای سئو

Robots.txt Generator

Allow All Robots: Crawl-Delay: Sitemap URL:

Block Search Engine and Social Media Bots: Googlebot (Google) Bingbot (Microsoft Bing) Slurp (Yahoo) DuckDuckBot (DuckDuckGo) YandexBot (Yandex) Baiduspider (Baidu) Sogou Spider (Sogou) Exabot (Exalead) ia_archiver (Alexa/Internet Archive) SeznamBot (Seznam.cz) Qwantify (Qwant) Facebot (Facebook) Twitterbot (Twitter) LinkedInBot (LinkedIn) Pinterestbot (Pinterest) TelegramBot (Telegram) WhatsApp (WhatsApp Crawler) Slackbot (Slack) Discordbot (Discord) SnapchatBot (Snapchat) WeChatBot (WeChat) Applebot (Apple Siri & Spotlight) AdsBot-Google (Google Ads Crawler) Googlebot-Image (Google Image Search) Googlebot-Mobile (Google Mobile Search) Mediapartners-Google (AdSense Crawler)

Block AI Bots: OpenAI (ChatGPT) Anthropic (Claude) Google DeepMind (Gemini) Google Bard CrawlGPT (AI-powered Crawler) Dataminr Klarity AI Synthesia (Video AI) DeepAI RunwayML (Creative AI) Semantic Scholar Microsoft Academic Bot Wolfram Alpha You.com (YouChat) Perplexity AI Neeva AI CopyAI Jasper AI

Block URL Parameters: Block All URL Parameters (e.g., /*?*) ?add-to-cart= (WooCommerce) ?orderby= (Sorting) ?per_page= (Pagination) ?stock_status= (Filter by Stock) ?filter_color= (Filter by Color) ?filter_age= (Filter by Age) ?utm= (Tracking Parameters) ?ref= (Referral Links) ?session_id= (Session Identifiers) ?fbclid= (Facebook Click ID) ?gclid= (Google Ads Click ID) ?campaign_id= (Campaign Tracking) ?tracking_id= (Custom Tracking IDs) ?s= (Search Queries) ?page= (Pagination) ?lang= (Language Selector) ?currency= (Currency Selector) ?preview= (Preview Parameters)

Block File Extensions: .php .html .xml .js .css .jpg .jpeg .png .webp .avif .svg .gif .pdf .docx .doc .pptx .txt .mp3 .mp4 .ogg .wav .avi .mov .flv .exe .bat .jar .iso .dll .apk .psd

Block Common WordPress Paths: /wp-admin/ /wp-includes/ /wp-login.php /wp-register.php /wp-content/ /wp-content/uploads/ /wp-content/cache/ /wp-content/themes/ /wp-content/plugins/ /wp-admin/admin-ajax.php /wp-admin/admin.php /wp-admin/edit.php /wp-admin/options-general.php /wp-admin/tools.php /wp-json/ /wp-admin/install.php /wp-cron.php /xmlrpc.php /wp-activate.php

Block Additional WordPress Paths: /feed/ /search/ /author/ /comments/ /wp-json/ /trackback/

Block or Allow File Formats: .jpg .jpeg .png .webp .svg .gif .pdf .docx .doc .pptx .txt .xls .csv .mp3 .ogg .wav .flac .mp4 .avi .mov .webm .flv .zip .rar .tar .gz .7z .exe .apk .psd

ابزار ساخت فایل robots.txt بهینه شده برای سئو

Robots.txt Generator

دیدگاهتان را بنویسید لغو پاسخ