Generate the robots.txt
file for a Next.js app
Just copy and paste a file that will generate the robots.txt
file for the site, allowing all search engines to index every route and specifying the location of the sitemap.
Requirements
Before using the file, make sure your project meets the following requirements:
- Next.js 13.3.0 or higher
- Next.js App Router
Usage
Add the file to your project
Copy the robots.ts
or robots.js
file to your project's app folder.
Edit the baseUrl
variable
Update the baseUrl
variable in the robots.ts
or robots.js
file with your site's base URL.
const baseUrl: string = "https://novajs.dev";