Robots.txt file is just a text document you can create using Windows programs and name it as robots with a file extension .txt however, when you place rules within this document and place it on web server root folder, then, you can better control crawling of search engine bots such as Googlebot. Important to Note… Continue reading robots.txt File
Tag: robots.txt
WordPress robots.txt File
Using robots.txt file on your WordPress built site? If not, you should. How to create it? Easy, just open a Microsoft Notepad and create a text file and name it robots.txt Then upload this to your WordPress installation. Login to cPanel > File Manager > public_html (path may be different for your website setup) >… Continue reading WordPress robots.txt File