Posted Friday, 25-Mar-2022 by Ingo Karkat
Despite waning support after the discontinuation of Google Reader in 2013, I still like to consume blogs and other regular information like build statuses via RSS or Atom feeds. (This blog has an Atom feed, too.) Some notifications can only be delivered via email, though. To consume those in the same way, I've registered a free Gmail account, and then subscribe to a feed of that email inbox. Problem solved.
Well, except that it wasn't working any longer. (I can't tell exactly when it broke, because due to the migration of my main computer, I didn't follow the feeds for quite some time — how to consume the feeds from several clients while keeping a consistent history will be a topic for another blog post.) The feed reader indicated that there's a problem with the feed, and a manual invocation confirmed this:
$ wget --user ✱✱✱✱ --password ✱✱✱✱ https://mail.google.com/mail/feed/atom --2022-03-24 09:13:28-- https://mail.google.com/mail/feed/atom Resolving mail.google.com (mail.google.com)... 2a00:1450:4001:808::2005, 142.250.185.101 Connecting to mail.google.com (mail.google.com)|2a00:1450:4001:808::2005|:443... connected. HTTP request sent, awaiting response... 401 Unauthorized Authentication selected: BASIC realm="mail.google.com" Reusing existing connection to [mail.google.com]:443. HTTP request sent, awaiting response... 401 Unauthorized Username/Password Authentication Failed.
But in the browser, this is still working fine — of course, there I log in first with the corresponding Google account. That's great news, because that means that there's nothing wrong with Google's Atom feed; it's "only" an authentication problem on programmatic access. (Unlike Microsoft, Google does not hesitate to discontinue features or APIs. I had to switch from the Google Web Search API to Google Custom Search API some years ago.)
A comment in a GitHub issue soon puts me on the right track. Poking around in that user's Google account settings, I soon find less secure app access (under Security), and it's off. Basic authentication can be considered a less secure sign-in technology (though over HTTPS the password is encrypted in transit), so this looks like the solution.
After enabling this option, a repeat of the wget command indeed succeeds:
$ wget --user ✱✱✱✱ --password ✱✱✱✱ https://mail.google.com/mail/feed/atom --2022-03-24 09:01:00-- https://mail.google.com/mail/feed/atom Resolving mail.google.com (mail.google.com)... 2a00:1450:4001:82f::2005, 142.250.184.197 Connecting to mail.google.com (mail.google.com)|2a00:1450:4001:82f::2005|:443... connected. HTTP request sent, awaiting response... 401 Unauthorized Authentication selected: BASIC realm="mail.google.com" Reusing existing connection to [mail.google.com]:443. HTTP request sent, awaiting response... 200 OK Length: unspecified [text/xml] Saving to: ‘atom’
So, we're done, right? Well, we're only good for two months; then Google will disable that option (see screenshot above). Alright, so we need to learn about the go-forward solution, and better implement it now.
The note goes on to say that App Passwords (GitHub calls these Authorized OAuth Apps) are the successor. Basically, a token is generated and that is used instead of your user's password. (The added security benefit is that these tokens may be limited to certain operations (and certainly cannot be used for a login in the browser), and can be managed (revoked) separately.) Problem is… I don't find the setting.
Ah, this feature requires two-factor authentication (called 2-Step Verification by Google). I don't care much about this account (it's just used for registering elsewhere and receiving emails), but it's no big deal; just one more entry in my Authenticator app, right?
Except that… it won't let me choose TOTP authentication. Sigh. It either requires a hardware key device (which I don't have), or a phone number. Fortunately, there it accepts both SMS and calls, so I can enter my home phone number — I already have my mobile number associated with my "real" Google account, and don't want to connect the two.
… A moment later, Google's calling, a computer voice spells out a six-digit number, I enter it; done.
Even better, after the initial 2FA setup, I can add TOTP as an alternative, and then am able to remove the original phone number. So, through this little detour, I eventually got my preferred way. (But I don't understand why Google wouldn't let me start with this.)
Now, under Signing in to Google, I finally have the App Passwords option.
The token generation requires a name (feed reader for me) and confusingly also a device. As I use this with software on Linux, none of the options (which funnily still include Windows Phone (R.I.P.)) really apply, so I choose Other. I wonder for what they need this, anyway.
And there's the password! I briefly verify that it works with the manual query from above (same user, but instead of the user's password, the generated token is now used).
$ wget --user ✱✱✱✱ --password qh✱✱✱✱✱✱✱✱✱✱✱✱wh https://mail.google.com/mail/feed/atom --2022-03-24 09:16:20-- https://mail.google.com/mail/feed/atom Resolving mail.google.com (mail.google.com)... 2a00:1450:4001:830::2005, 172.217.16.133 Connecting to mail.google.com (mail.google.com)|2a00:1450:4001:830::2005|:443... connected. HTTP request sent, awaiting response... 401 Unauthorized Authentication selected: BASIC realm="mail.google.com" Reusing existing connection to [mail.google.com]:443. HTTP request sent, awaiting response... 200 OK Length: unspecified [text/xml] Saving to: ‘atom.1’
All that's left to do is updating the password in my feed reader application.
Ingo Karkat, 25-Mar-2022
ingo's blog is licensed under Attribution-ShareAlike 4.0 International
