5 min read enumeration

A Complete Guide to Web Enumeration with Dirb

If you are doing a Hack The Box challenge, bug bounty, or a “real world” security assessment, you use Burp Suite or manually browse a website to find linked pages. But what about pages that are unlinked, such as a ‘secret’ portal that admins know to navigate to directly?

That’s where web content scanners come in. They use wordlists to enumerate (potentially hidden) webpages and directories, and give you more attack surface to analyze.

This article is about dirb, which is a popular web content scanner made by The Dark Raver.

Note: only use dirb on systems that you have legal permission for. If you are not sure if you have permission, then you probably don’t. There are plenty of sites (such as HackTheBox) specifically set up to give you a legal environment in which to practice your security skills.

How Does Dirb Work?

If a webpage or directory is not linked to from other pages, how does dirb know it’s there? In short, it guesses.

To use dirb, you provide a wordlist. This wordlist might contain common endpoints, such as “index.html”, “admin.jsp”, “login.php”, to name a few examples. It might also include common directories like /.git/, /cgi-bin/, and so on.

Dirb takes the provided URL (ex: http://site-being-tested.com), appends the endpoints one at a time, and then makes a request.

The HTTP response code tells dirb about the webpage’s existence (or not):

  • If the endpoint does not exist, the webserver should return a 404, in which case, dirb ignores it.
  • If it returns a HTTP 200, then the webpage exists and is viewable.
  • If it’s a HTTP 30X, the server returned a response, redirecting dirb to another location.
  • If it’s a HTTP 40X, the server has indicated that authentication is required, and that dirb is forbidden or unauthorized.

Dirb does not “spider” websites (follow links), nor does it download files or run typical vulnerability scans. It just tries out all the endpoints provided in the wordlist.

You can read dirb’s source code here if you’d like.

We’ll cover specific use cases later in this article, but here is the basic dirb command format:

dirb <url_base> [<wordlist_file(s)>] [options]

How to Install Dirb

Ubuntu:

sudo apt install dirb

Here’s an install script for MacOS.

Dirb Wordlists

By default, dirb looks in /usr/share/dirb/wordlists/ for a wordlist named common.txt.

How to fix “Error opening wordlist”

If you get this error: (!) FATAL: Error opening wordlist file: /usr/share/dirb/wordlists/common.txt then you have two options:

  1. You can move a wordlist with the name common.txt into /usr/share/dirb/wordlists.
  2. Or, you can supply the path to a wordlist after the URL, such as
    dirb http://site-being-tested.com /path/to/wordlist

This second option will come in useful, since it’s a good idea to tailor your wordlist based on the website’s tech stack.

If a website is written in PHP, use a PHP-specific wordlist. Same goes for other languages. You can also use wordlists that target a specific website platform, like IIS.

Where to find good wordlists for dirb

Dirb is only as good as the wordlists you provide it. While common.txt is a good place to start (and will typically find you the endpoints for common CTF challenges within HackTheBox, etc), you will want to branch out and tailor your wordlists to your application.

My two top picks for wordlists are:

  1. Daniel Miessler’s SecLists on Github
  2. AssetNote Wordlists

Each of these resources has a variety of lists for different uses. I suggest that you save local copies and add to them as you discover new web technologies and related endpoints.

You can use multiple wordlists, just comma-separate them! For example,

dirb http://site-being-tested.com /path/to/wordlist1,/path/to/wordlist2

Dirb Hotkeys

As dirb is running, you can use hotkeys to skip ahead, pause the scan, or show stats:

======================== HOTKEYS ========================
'n' -> Go to next directory.
'q' -> Stop scan. (Saving state for resume)
'r' -> Remaining scan stats.

This can be used to exclude a directory, by pressing n after the directory scan has started. Dirb will jump to the next directory in the list.

If you want dirb to ask ahead of time, you can use the -R flag for interactive recursion, meaning that dirb will ask you before it begins each directory.

Dirb Examples

Here are some common dirb use cases:

Dirb with a specific port

If you provide a URL beginning with HTTP and no further port info, the port used by dirb will be “80”. Likewise, HTTPS will use 443 by default.

To provide a specific port, use the format http://url:port or https://url:port. For example:

dirb http://site-being-tested.com:8080

Authentication with dirb

If you have a website that requires basic authentication, use -u <username:password>. For example:

dirb http://site-being-tested.com /path/to/wordlist -u admin:secret

You can provide a cookie instead, if you have logged into the website (and then used Dev Tools, Burp Suite, or similar to find the relevant authentication cookie).

dirb http://site-being-tested.com /path/to/wordlist -c <cookie value here>

If you have a client certificate to provide, use the -E flag:

dirb http://site-being-tested.com /path/to/wordlist -E /path/to/certificate

If those options don’t cover your needs, you can always add other custom headers with -H <header value here> after the URL and wordlist.

Using dirb with a proxy

If you’re using dirb with a proxy, you can use -p <proxy[:port]>

If authentication is required, add -P <proxy_username:proxy_password> (uppercase P).

Ignore certain HTTP responses

If a website returns a certain HTTP status code that you’d like to ignore, use -N <HTTP code>. For example, if a website returns a 301 instead of a 401/403 response, you can ignore that with

dirb http://site-being-tested.com /path/to/wordlist -N 301

On the other hand, if you want to see NOT_FOUND pages, you can use the -v flag (warning, this will likely be very noisy).

Try variations on a found filename

If dirb finds a file from the wordlist, you can instruct it to automatically try variations with the -M flag. For example:

dirb http://site-being-tested.com /path/to/wordlist -M .tmp,.backup

If http://site-being-tested.com/db was found, dirb would automatically try http://site-being-tested.com/db.tmp and http://site-being-tested.com/db.backup.

This may not be available on all versions of dirb.

Add file extensions to wordlist contents

If you have a wordlist that is only filenames without extensions, you can use -X <file extension> to have dirb append the file extension for you.

dirb http://site-being-tested.com /path/to/wordlist -X .php

This might be useful in a scenario where there’s a catalog of information within a certain topic. For example, /dog.html, /cat.html, and so on.

Alternatives to Dirb

Two popular alternatives to dirb are wfuzz and gobuster. If you’d like a GUI version of dirb, use dirbuster.