Skip to content
Nuclei Templates

robots.txt file

ID: robots-txt

Severity: info

Author: CasperGN,TheZakMan

Tags: miscellaneous,misc,generic

Description

Section titled “Description”

YAML Source

Section titled “YAML Source”
id: robots-txt


info:
  name: robots.txt file
  author: CasperGN,TheZakMan
  severity: info
  metadata:
    max-request: 1
  tags: miscellaneous,misc,generic


http:
  - method: GET
    path:
      - "{{BaseURL}}/robots.txt"


    host-redirects: true
    max-redirects: 2


    matchers-condition: and
    matchers:
      - type: word
        words:
          - 'User-agent:'
          - 'Disallow:'
          - 'Allow:'


      - type: word
        part: header
        words:
          - text/plain


      - type: dsl
        dsl:
          - "len(body)>=140 && status_code==200"
# digest: 490a004630440220483c89872ac45742f715d053eb97b7b97c1a23e981027e2cb467df256ea0dca4022047fbfda47c570bf41d34c4ddf5d76df4d2fa6bc27a4856d0ff95cdb944aa1203:922c64590222798bb761d5b6d8e72950

Guide to check the vulnerabilities

Section titled “Guide to check the vulnerabilities”

This template is used to detect vulnerabilities in web applications. It can be used with the Nuclei tool to scan for specific patterns or behaviors.

Terminal window
$ nuclei -u "URL" -t "http/miscellaneous/robots-txt.yaml"

View on Github