4 min read appsec

A Brief Guide to Using curl

curl is a command-line tool that lets you create network requests. As with many command line tools, you can read the man page or look at the cURL project on Github (encompassing both the libcurl library and the command-line tool curl).

What this post offers is a beginner guide with enough curl information to get you started, and different options might be useful in a security context. Then, we'll go through some examples at the end.

Note that curl can be used to send requests for all sorts of protocols, including FTP, HTTP(S), IMAP(PS), LDAP(S), MQTT, POP3(S), RTMP(S), RTSP, SCP, SFTP, SMB(S), SMTP(S), Telnet, and TFTP. This guide is just going to cover HTTP and HTTPS.

Anatomy of a curl request

The basic usage is

Usage: curl [options...] <url>

But there is a LOT that can fit in the [options...] part. Options cover:

The HTTP Request method

GET, POST, PUT, DELETE, HEAD, OPTIONS, etc (defaults to GET if left blank). This is done by adding -X <Method type> such as -X POST. You might find this useful when testing endpoints.

Validation of requests and/or processing of data might vary between HTTP method types (to be clear, this shouldn’t happen, but it sometimes does, and is worth checking). And beginner CTFs may include very over-the-top examples of this to drive that point home.

HTTP Headers

Headers can encompass everything from User Agent to Content Encoding, Type, and Length, to Cookie information, Host and Referer, and Cache-Control to name a few, plus custom-defined headers for a given framework or server. See Wikipedia for a more complete list.

Here are a few that are useful from a security perspective:

  • Authorization:Basic <base64 string of username:password> to send a Basic Authentication set of credentials. This is the command line equivalent of when you visit a website and it pops up a window. See Example 10 here.
  • Content-Length and Transfer-Encoding: can be used in HTTP Smuggling attacks.
  • User-Agent: tells the server information about your browser, and can be used for XSS attacks (example here).
  • X-Forwarded-For: which can be used to spoof your IP address (CTF challenge example)
  • Host: which can be used for injection attacks.
  • Accept: which specifies the allowed content-types to return, and can also be used in attacks.
  • Referer: for referer spoofing.

Another one to try that doesn’t have direct security implications is Accept-Language: <lang>. Try picking a website and changing the header value from Accept-Language: en-US to use fr-CA and see what comes back.

Examples using curl:

Make a GET Request (GET the URL for a given webpage):

curl http://website.com/

Make a GET Request and see headers (verbose mode)

curl -v http://website.com/

You can also use -i to view headers:

curl -i http://website.com/

Alternatively if you want to only view headers, and not the page content:

curl -I http://website.com/

GET a webpage and store the output as a file

curl http://website.com/index.html -o index.html

You can also use standard output redirection (>)

curl https://website.com > index.html

Make a POST request

curl -X POST http://website.com/

Make a POST request with JSON data

curl -X POST http://website.com/ -H "Content-Type: application/json" -d '{"param1": 123, "param2": "something else"}'

Make a POST request with URL-encoded data

curl -X POST http://website.com/ -d "param1=123&param2=something%20else"

Get a URL and follow all redirects

curl -L http://website.com/

Make a GET request for a specific language

curl http://website.com/endpoint -H 'accept-language: <language string>'

Make a GET request for a specific content type

curl http://website.com/endpoint -H 'accept: <content type string>'

Make a multi-line curl request:

Use \ at the end of each line (except the very last line)

curl "http://website.com" \ 
   -H 'upgrade-insecure-requests: 1' \ 
   -H 'user-agent: Mozilla/5.0 <full UA string here>' \ 
   -H 'accept: text/html,' \ 
   -H 'referer: https://www.website.com/' \ 
   -H 'accept-language: en-US,en;q=0.9' \ 
   --compressed

Make a request and ignore SSL/TLS Errors

curl --insecure https://website.com/

Make a GET request that includes Basic Authentication information

curl -u username:password https://website.com

or

curl -H 'Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQ=' https://website.com

Where dXNlcm5hbWU6cGFzc3dvcmQ=  is username:password with base64 encoding. These two requests should be equivalent (with curl base64-encoding the credentials for you in the first example).

Make a request through a proxy

curl -x <proxy IP or URL>:<proxy port> http://website.com

Copy from Dev Tools

If you are making a request in your browser, you can open up Dev Tools, go to the Network tab, then right-click and choose Copy as cURL. This will give you a multi-line curl command that is equivalent to the request made by your browser (including cookies, user-agent, etc).

Try it out:

Another article recommending PicoCTF 🙂 Give GET aHEAD, Cookies, and Who are you? a try using curl.

References