By Jason Snell
March 4, 2016 9:12 AM PT
Last updated July 27, 2020
Automate This: Connect to the Web with cURL
Warning: This story has not been updated in several years and may contain out-of-date information.
One of the best utilities on the Mac is one you might not be familiar with, because it requires the use of Terminal. It’s cURL, and it’s a valuable part of my scripting, automation, and debugging toolbox.
curl is a command-line utility that works kind of like a web browser. I use it mostly to download web content or ping web services. Using cURL for a lot of basic tasks is quite easy. Just type
curl followed by a web address, and you’ll see that web page spewed across your Terminal window. type
curl https://sixcolors.com/ > sixcolors.html and you’ll have dumped the contents of that URL to a file in your user folder.
As a person who does a lot of podcast-related stuff, I am occasionally called upon to debug a podcast’s feed. Podcast feed files are XML files, and don’t display in a friendly way in a web browser. No problem — I use cURL to download the file and open it in my text editor of choice, BBEdit.
curl http://feeds.theincomparable.com/tpk > tpk.rss ; bbedit tpk.rss
This is the command I usually use — it’s actually two commands in one, separated by a semicolon. The first command downloads the RSS feed to a text file; the second uses the
bbedit command-line utility1 to open the resulting file directly in BBEdit.
You could also omit the BBEdit step and direct cURL to save the file to a more convenient location, like the Desktop:
curl http://feeds.theincomparable.com/tpk > ~/Desktop/tpk.rss
When I wanted to use IFTTT to automate some stuff in my house, I needed my computer to activate one of IFTTT’s “custom triggers” simply by loading a particular URL, in the format https://maker.ifttt.com/trigger/trigger-name-here/with/key/key-goes-here. In other words:
The app I’m using for this particular trigger can’t run Terminal commands itself, but it can launch apps. One great thing about terminal commands is that they’re easily integrated into AppleScript or Automator workflows. In this case, the “app” my app launches is this:
do shell script "curl https://maker.ifttt.com/trigger/mytrigger/with/key/this-is-my-key"
That’s it. With cURL and the AppleScript command “do shell script” (or the Automator equivalent, the Run Shell Script action), you can quickly and cleanly automate Web connections without using a web browser. And scripting aside, cURL is quite useful in downloading content from the Internet that you want to view in something other than a web browser2.
If you appreciate articles like this one, support us by becoming a Six Colors subscriber. Subscribers get access to an exclusive podcast, members-only stories, and a special community.