"Create Robots.txt File & Sitemap.xml" #With Example
In this video, we will show you how to create a robots.txt file using cPanel.

So let ’s see how to create write and upload the custom robots.txt file For your website using cPanel.

you can also tell the search engine, what you want to allow for indexing to their search engine crawlers.

Here is a common example of an incorrectly set ROBOTS.TXT file:
--------------------------------------------------------------------------------------------------

Basic format:

User-agent: [user-agent name]
Disallow: [URL string not to be crawled]

examples of robots.txt

1.Disallowing all web crawlers access to all content

User-agent: *
Disallow: /

2.Allowing all web crawlers access to all content

User-agent: *
Disallow:

3.Blocking a specific web crawler from a specific folder

User-agent: Googlebot
Disallow: /example-subfolder/

4.Blocking a specific web crawler from a specific web page

User-agent: Bingbot
Disallow: /example-subfolder/blocked-page.html

2.Allowing all web crawlers access to all content except one specific file

User-agent: *
Disallow: /example-subfolder/blocked-page.html

sitemap-
To mention the sitemap for your website, sitemap uses a robots.txt file. and sitemap always is in XML format.
XML Sitemaps Generator: Create your Google Sitemap Online
https://www.xml-sitemaps.com/

================================================
Here are the steps to editing the ROBOTS.TXT file:
================================================

Login to the Cpanel.

1). Open-File Manager and go to the root directory of your website.

2). The ROBOTS.TXT file should be in the same location as the index file (the first page) of your website.

3). Click on the EDITOR to make any necessary changes. Add code

Click SAVE CHANGES and close the file

Congratulation, Now you know how to create a robots.txt file using cPanel.

Just enter your website URL to create a sitemap

https://www.xml-sitemaps.com/

#robots.txt#sitemap.xml

Sitemap syntax
sitemap: https://yoursitename/sitemap.xml

----------------------------------------------------------------------------
"Create Robots.txt File For Wordpress" #2min (With Sitemap.xml)
https://www.youtube.com/watch?v=KSDhNi3Uyc4