Rachel Kroll
Oh boy, it's been a while, so let's talk about feed readers. This whole scoring project has been running since the end of May, and much data has been collected. It's logged over 70K requests to tagged feed URLs in that period from 97 distinct keys - the unique IDs which should be used by a single install of a single program.
Two of them have sent over 6000 requests in the time they've been running. Another half-dozen or so have sent over 2000. Keep in mind that not all of these started at the end of May, either. While some of them go back to the beginning, quite a few are much newer than that.
I've gone through and summarized anything that's still checking in, using an arbitrary cutoff of "has polled in the past week".
Once again, while any one entry does not reflect a program since a user might've configured it really badly (or really well), the patterns become obvious pretty quickly when you see multiple instances of the same thing.
Unlike before, now I'm actually going to list the names and versions. It's time.
[IMS = If-Modified-Since header. INM = If-None-Match header.]
...
Netvibes. Has some kind of caching error - stuck on old IMS/INM values for the past month basically. I'm going to kick it again to see what happens, but it doesn't bode well. Has some dumb 3 second rapid-fire thing at feed init, too.
SpaceCowboys Android RSS Reader / 2.4.15 (previously aka "sweatpants", more or less, apparently). This thing has a polling interval that's all over the place and it likes to just drop the IMS/INM and do unconditional polls now and then. No idea why.
NextCloud-News/1.0. Super broken caching. Terrible mess.
NewsBlur. Rapid-fire thing at startup, otherwise seems fine.
Friendica/2024.06-dev. No conditionals, ever. Also bum rushes the server when first adding a feed by hitting a bunch of goofy not-the-feed URLs (which can't be seen in the report, but *I* can see it in the logs).
Roy and Rianne's Righteously Royalty-Free RSS Reader v 0.2. Seems to have settled down into a nice polling pattern. No complaints.
Tiny Tiny RSS/23.04. Nothing but unconditionals. But then it also does this < 1 second "OMG LET'S POLL AGAIN" thing. What's the point? Annoying.
ureq/2.9.1. Seems to be in a good pace/pattern now.
Feedbin. Hella broken IMS/INM caching, terrible pacing, regular forced unconditional polls, etc. They trip 429s in prod constantly. There are a bunch of these and they're all broken the same way, so I chalk that up to the program/service and not any one user's config.
NetNewsWire. Bigtime brokenness in their IMS/INM caching. No idea why, but it makes it generate unconditionals constantly.
haven. All unconditional requests. Pacing is all over the place.
Miniflux/2.1.4 (plus older versions previously). They had some kind of caching bug but it looks like it's good and dead with this version. Hooray for doing the right thing!
inforss/2.3.2.0. Good pacing, proper caching behavior. Nice.
Miniflux/2.1.3 and then 2.1.4. As above. Yay fixes.
Some random thing that's just curl. Seems generally okay. Does a cheeky thing every week where it drops the IMS/INM, perhaps to see if the web server admin is a clown. (Spoiler: I definitely am.)
Inoreader/1.0. All over the place with timing and unconditionals. Also shoves a useless referrer down the pipe every time, as if that is going to accomplish anything. Has a really nasty rapid-fire startup sequence that'll probably get it autoblocked before it can even get subscribed properly.
Feedbin. Bleh. See above.
Feedbin. Meh.
Miniflux/2.1.0 that needs to be upgraded to 2.1.4. Currently isn't showing signs of cache insanity because I haven't deliberately triggered it in a while.
NetNewsWire. Just like the earlier one, broken caching and crazy timing.
Liferea/1.15.3. Has a tendency to just drop everything and start making unconditional requests... one second apart? Not great.
rawdog/2.24rc1. Behavior is spot on. More like this, please.
NetNewsWire. Same bad caching and timing stuff.
Something with no User-Agent, or IMS, or INM headers. It's like... okay, what exactly is going on here? At least it polls super slowly.
NewsBlur. Just like the other one.
FreshRSS/1.24.0. This one latched bad IMS/INM values back in *May* and is still sending them. Hella broken.
Miniflux/2.1.3 and 2.1.4. Seems fine now that it's been upgraded.
Tiny Tiny RSS - multiple 24.05 and 24.06 builds. Slightly better than the 23.04 one mentioned earlier. This one does send conditional requests, but it also sends tons of unconditionals, and it does the "OMG 1 SECOND HAS ELAPSED SO I MUST GO GET IT AGAIN" thing. Not cool.
Tiny Tiny RSS 24.02 and 24.05 - various builds. Just like the previous entry.
CommaFeed/4.4.0, 4.4.1, 4.5.0, 4.6.0. This had some caching bugs for a while and was sending bad IMS values. It's been behaving lately. It's probably time for me to deliberately do some weird (but totally valid) behavior again.
Vienna/8052, 8219. Timing is all over the place, but at least it sends conditional requests.
Some claims-to-be-a-browser thing (probably a browser extension). Sends derpy 2000-01-01 IMS at startup for no good reason. Caching headers jump around and disappear sometimes, like maybe multiple instances are warring over the state storage or something. Far too many < 1s polls for my liking.
Feedbin. Feh.
Tiny Tiny RSS/24.04. As explained above, OMGs and all.
theoldreader.com. I'm pretty sure Google Reader's poller did conditional requests. This thing doesn't. Ever. They seem to get around blocks in production by polling every 6-7 hours and using multiple hosts.
Some unidentified "Go-http-client/1.1" that seems to have a decent pace and which sends INM headers. No idea what it is, but it seems to be polite about the job it's doing.
Rapids. This one got upgraded and is now doing both IMS and INM and looks great. Thanks to the author for the effort in changing it.
er0k feeds. This one also seems to have been upgraded at some point and seems to be keeping up just fine now. I definitely need to do some "inject some crazy" testing to see how it does.
NetNewsWire. Broken, as above.
walrss/0.3.7. Just works. Nice.
Yarr/1.0. Startup had a bunch of unconditionals but then it settled into sending IMS and INM headers. Timing seems a little squirrelly, like it's not actually waiting for at least an hour to elapse, but rather just starts on some schedule and is subject to however long the last run took.
Bad data. Different programs using the same key.
FreshRSS/1.24.0, 1.24.1. Bad caching, bad timing.
Some other "might be a web browser extension" thing. Does the same 2000-01-01 IMS rapid-fire stuff at startup. Sending caching values but it's running a bit quick.
Feedly/1.0. This one is just abusive. At some point it switched to sending nothing but unconditionals every 30 minutes. It's like, okay, this *fake* feed doesn't say much. So chill out, already. Comes from a bunch of different hosts, too. Annoying.
NetNewsWire. Bro-ken.
feedparser/6.0.2. Does cached requests fine, but the timing is squirrelly here just like some of the others. I wrote a post about why you can't just say 'start me every hour' and leave it at that a few weeks back.
Reeder, various builds. This thing is bizarre. It'll just flip and flop between INM values (things I served up as an ETag) from weeks in the past. Tell me that's not some kind of wicked caching bug. Also polls FAR too fast. Some programs have timing that is merely squirrelly. This is a whole cage of them.
Unusable data - multiple programs.
Unusable data - multiple programs.
Unread RSS Reader. Godawful poll timing. 6103 requests in 52 days is about one poll every 736 seconds _on average_, but they're hugely spread out. WTF? Put it this way: the list of unique intervals (nn seconds, nn minutes, ...) is *four pages tall* on my web browser.
Feedly. Also went to 30 minute all-unconditionals. Bah.
FreshRSS/1.23.1. Bad caching, like the others.
Yarr/1.0. Just like the other one: settled into sending IMS/INM but the timing is all over the place, and often far too fast.
Unusable data - multiple programs.
Miniflux/2.1.3 that needs to be upgraded to 2.1.4. As above.
Miniflux/2.1.3. Same.
Another unidentified "Go-http-client/1.1", but this one seems to have the weird timing issues like so many others.
newsraft/0.25. This one got into some bad places with stuck caching headers. Given the version number hasn't shifted, I assume it would happen again if I ran another test from my side. Wacky timing, too.
NetNewsWire. Broken record.
feedparser-perl/6.0.10 is doing conditionals but it's happening too quickly, and the timing is mighty inconsistent.
Mojolicious (Perl). Just works. No issues. Thanks.
NextCloud-News/1.0. Very broken caching. It's not even using the right timing to generate If-Modified-Since requests, and this has been a problem for years. This is the infamous "1800-01-01" IMS sender.
com.vanniktech.rssreader, various versions. Timing is all over the place, frequently far too fast - 2 seconds, 10 seconds? Also seems to "lose" the INM header now and then.
bdrss/4.0. Sends conditional requests properly. No idea how the timing works on this, but at least it's on the "long interval" side of things and not the "super fast polls over and over" like most of the others in this list. Maybe it throttles back when a feed is quiet? That's great, if so.
Broadsheet/0.1. Seems fine.
feedbase-fetcher.pl/0.5. Also seems fine.
Artykul/1.0. I installed an app on my Mac and it started checking in, and this stupid clown thing started checking in as well. The Mac app is long gone, but this albatross won't go away. Sends unconditional requests every single time, and it's visiting URLs you probably didn't intend to share with them! Wonderful. I wrote a post about this a while back.
...
There you go: a bunch of bright spots and then a bit of sadness.
Choose wisely.