The curl command-line tool is an indispensable utility for anyone working with network protocols. It stands as a robust and versatile client-side URL transfer library, capable of interacting with a multitude of protocols beyond just HTTP. This guide dissects the curl command, illustrating its fundamental and advanced capabilities through practical examples. By the end of this article, you will possess a critical understanding of curl‘s power, enabling efficient data transfer, debugging web services, and automating network tasks directly from your Linux terminal.
To effectively follow this guide, you should have:
- Access to a Linux terminal (e.g., Ubuntu, Debian, Fedora, CentOS).
- Basic familiarity with command-line interface operations.
curlinstalled on your system. Most modern Linux distributions includecurlby default. If not, install it using your distribution’s package manager (e.g.,sudo apt install curlon Debian/Ubuntu,sudo dnf install curlon Fedora/CentOS/almalinux).
Perform a Basic HTTP GET Request
The most fundamental use of curl is to retrieve the content of a URL, typically an HTML page. This operation defaults to an HTTP GET request.
curl https://example.comThis command will output the entire HTML content of https://example.com directly to your terminal’s standard output. It’s a raw, unformatted stream, valuable for quick content inspection.
Pro-Tip: View HTTP Headers
To inspect the HTTP response headers alongside the body, use the -i or --include option. This is crucial for debugging, as it reveals status codes, content types, and caching directives.
curl -i https://example.comWarning: Without additional flags, curl will print everything to stdout. For large responses, this can quickly overwhelm your terminal. Consider piping output to a pager like less (e.g., curl https://example.com | less) or redirecting it to a file.
Download Files from a URL
curl excels at downloading files, offering more control and reliability than simple browser downloads, especially for scripting or large files.
Save Output to a File
To save the output to a file with the same name as the remote file, use the -O (capital O) option.
curl -O https://example.com/path/to/remote_file.zipThis command will download remote_file.zip and save it in your current directory under the same name.
Specify a Custom Filename
For greater control, use the -o (lowercase o) option to specify a local filename for the downloaded content.
curl -o my_local_archive.zip https://example.com/path/to/remote_file.zipPractical Tip: When downloading large files, curl displays a progress bar by default. If you need to resume an interrupted download, add the -C - option. This tells curl to continue from where it left off, which is invaluable for unstable connections.
curl -C - -O https://example.com/large_file.isoSend Data with POST Requests
Interacting with APIs or submitting web forms often requires sending data via POST requests. curl provides robust options for this.
Send Form-Encoded Data
Use the -d or --data option to send form-encoded data, mimicking an HTML form submission. Multiple -d options can be used.
curl -d "param1=value1¶m2=value2" https://api.example.com/submitBy default, curl sends Content-Type: application/x-www-form-urlencoded with -d. If you need to specify JSON, you must explicitly set the header.
Send JSON Data
When interacting with RESTful APIs, sending JSON payloads is common. Combine -d with the -H (header) option to set the correct Content-Type.
curl -X POST -H "Content-Type: application/json" -d '{"username":"john.doe","password":"securepassword"}' https://api.example.com/loginWarning: Ensure your JSON data is correctly escaped if it contains special characters, or use single quotes around the JSON string to prevent shell interpretation issues.
Manage HTTP Headers
Customizing HTTP headers is essential for authentication, content negotiation, and API interaction. The -H option allows you to send arbitrary headers.
Add a Custom Header
To send an API key or a specific user-agent:
curl -H "Authorization: Bearer YOUR_API_TOKEN" https://api.example.com/datacurl -H "User-Agent: MyCustomApp/1.0" https://example.comPro-Tip: You can also use -H to override default headers or remove them by providing an empty value (e.g., -H "Accept:" to remove the Accept header).
Handle Authentication
curl supports various authentication methods, including basic HTTP authentication.
Basic HTTP Authentication
Use the -u or --user option to provide a username and password for basic authentication.
curl -u "username:password" https://protected.example.com/resourceSecurity Warning: Providing credentials directly in the command line can expose them in your shell history. For sensitive operations, consider omitting the password (-u "username:") to be prompted securely, or use environment variables where appropriate.
Follow HTTP Redirects
Many web servers issue HTTP redirects (3xx status codes) to guide clients to a new URL. By default, curl does not follow these redirects; it simply reports the redirect response.
Enable Redirect Following
To instruct curl to automatically follow redirects, use the -L or --location option.
curl -L https://shorturl.example.com/xyzThis command will follow any 3xx HTTP redirects until it reaches the final destination, then fetch and display its content.
Practical Tip: If you need to see the entire redirect chain, combine -L with -v (verbose mode) to view all requests and responses, including the redirect headers.
Debug Requests with Verbose Output
When troubleshooting network issues or API integrations, verbose output is invaluable. The -v or --verbose option provides a detailed log of the request and response process.
curl -v https://api.example.com/statusThis output includes connection attempts, SSL/TLS handshake details, sent request headers, and received response headers. It’s an essential tool for understanding exactly what curl is doing under the hood.
Next Steps
The curl command’s capabilities extend far beyond these examples, encompassing FTP, SFTP, FTPS, SMTP, POP3, IMAP, and more. Explore the official curl man page (man curl) for an exhaustive list of options, including features like proxy support, cookie handling, and advanced SSL certificate validation. Experiment with different protocols and options to solidify your mastery of this critical data transfer utility.
