[Bash] curl command

curl is a powerful command-line tool for transferring data with URL syntax.

It supports various protocols including HTTP, HTTPS, FTP, and many others.

curl [options] [URL]

Basic HTTP GET Request:

curl http://example.com

Save Output to a File:

curl -o output.html http://example.com

The -o option saves the content to a file named output.html.

Follow Redirects:

curl -L http://example.com

The -L option tells curl to follow HTTP redirects.

Include Headers in the Output:

$ curl -i http://example.com

HTTP/1.1 200 OK
Accept-Ranges: bytes
Age: 425101
Cache-Control: max-age=604800
Content-Type: text/html; charset=UTF-8
Date: Tue, 28 May 2024 16:46:42 GMT
Etag: "3147526947+gzip"
Expires: Tue, 04 Jun 2024 16:46:42 GMT
Last-Modified: Thu, 17 Oct 2019 07:18:26 GMT
Server: ECAcc (nyd/D191)
Vary: Accept-Encoding
X-Cache: HIT
Content-Length: 1256

The -i option includes the HTTP response headers in the output.

Send a POST Request:

curl -X POST -d "param1=value1&param2=value2" http://example.com

The -X POST option specifies the request method as POST, and -d specifies the data to send.

Send a JSON POST Request:

curl -X POST -H "Content-Type: application/json" -d '{"key1":"value1", "key2":"value2"}' http://example.com

The -H option adds a header to specify the content type as JSON.

Download a File:

curl -O http://example.com/file.zip

The -O option saves the file with its original name.

Send Custom Headers:

curl -H "Authorization: Bearer YOUR_TOKEN" http://example.com

The -H option sends custom headers, such as an authorization token.

Get Only HTTP Headers:

curl -I http://example.com

The -I option fetches only the HTTP headers.

Use a Proxy:

curl -x http://proxy.example.com:8080 http://example.com

The -x option specifies a proxy to use for the request.

Example

curl -s http://www.gutenberg.org/cache/epub/2701/pg2701.txt | gunzip | sed -r 's/\s+/\n/g' | grep -i whale | wc -l

Explanation:
curl -s http://www.gutenberg.org/files/2701/2701-0.txt:

Fetches the content of "Moby Dick" from Project Gutenberg silently (-s for silent mode).

sed 's/[^a-zA-Z]/\n/g':

Uses sed to replace any non-alphabet character with a newline, effectively placing each word on a new line.

grep -i whale:

Searches for the word "whale" in a case-insensitive manner (-i for case-insensitive).

wc -l:

Counts the number of lines, which corresponds to the number of occurrences of the word "whale".