We’re no longer allowing (new) FTP/SHELL users to have an email address associated with them.
[…]
Fortunately, this change should be more or less invisible to everybody! The only thing lost is the ability to see and manipulate your mail files via FTP/Shell… (and even that is only for new users from now on). Whoop-dee-do, I say!

Right, why would anyone want to use their own SpamAssassin, procmail, or a Unix mail client? I never had a problem with overselling at Dreamhost — in fact, I’d go so far as to say that I’m happy to take advantage of it — but I don’t think that’s their problem. I think they’ve just let themselves grow until they’re deep over their heads.

(And yes, that doesn’t affect existing shell accounts there, but I imagine that’s just a matter of time, because it’s not like running two parallel mail architectures is going to help them much.)

I’ve hosted this blog and a bunch of other websites on Dreamhost since 2004, and I’ve referred enough people to them that my hosting there has been free for years. But most of those four years have been spent just below the “I need to do something about this” level of dissatisfaction.

As of this last week, though — which featured a 12h planned outage followed by the rest of the week trying to recover from NFS problems which left sites unresponsive or just plain missing — I’ve had enough, and I’ve bought a virtual private server at Linode instead. I’ll post more about Linode later on, but last night Dreamhost resolved their NFS issues and I had a brief moment of reconsideration. After all, it’s free…

So I brought up my support history and read through it, and once again I’ve convinced myself it’s time to move critical services away from there. But the more I read it, the more I realized I should share the highlights of my experience. Like last time with IStop, my awful Ottawa ISP, I’m left wondering why I stuck around so long!

The details are after the cut.

May 21, 2004: Apache configuration prevents files named “README.txt” from showing in directory listings. Support writes,

Can’t you just rename it to something that will not be filtered like that?

May 23, 2004: I complain that the installed SpamAssassin is ancient. Support writes:

[W]e may upgrade at some point, but you’d really want to install your own version if you want to stay current at all. […] I would definitely not suggest using 2.20 for anything at this point.

Sep 15, 2004: Payment fails:

Current Balance: -$9.94
Amount Due: $9.94
Due Date: 2004-09-15

Failure! Please correct the errors below.
THERE IS A $9.95 MINIMUM FOR PAYMENTS

Feb 5, 2005: Fileserver damage. I get five copies of a “Your data has been restored!” form mail. I point this out to support, since I assumed at that point I’d keep getting copies of the form mail forever. They reply:

One of our server clusters was having fileserver issues on Friday. The account you’re writing in from was not affected.

I point out that no, it was affected, the restore was indeed successful, and I want to stop receiving mail about it. They reply apologizing for the multiple messages but continue to insist that my data (which was missing 24h previously) was not affected.

May 8, 2005: I get mail at 2AM:

I had to disable your database chiffbb. It was using enough of the CPUs on the server to justify having its own server. If you want to continue running your bulletin board, you should consider a dedicated server.

I point out that it has been running at a constant load for months, except that recently they replaced the database server:

So, now I have no access to the data, no more access to the conuery statistics to see if something went wrong recently or if it’s been building up slowly, no idea what the problem queries were, no way to make sure the indices I needed were there — nothing at all to work with aside from “enough of the CPUs”.Nothing has changed on the forums in the past year, so I’m a bit confused as to what might have happened overnight to get your attention.

Please let me know what I’m supposed to do at this point to figure out if things can simply be scaled back, given that “blindly trust you that I have to give you more money to get my data back” is an unacceptable option.

(Incidentally, shutting things down with no grace period at 2 AM on Sunday — and Mother’s Day no less — doesn’t really seem to match the whole “we’ll be nice about it” from your conueries knowledge base page. I’d recommend either going back to a hard quota which people can compare to their usage, or giving a grace period for this sor, otherwise it is impossible for users to actually manage their usage.)

They eventually re-enable the service and tell me their MySQL guy will get in touch with me to figure out what’s going on. That never happens and I don’t hear anything about the forums again for a while.

Nov 15, 2005: One of four IMAP servers isn’t authenticating. Mail problems are the new black.

Jan 16, 2006: Someone’s added “dnsalias.com” and a dozen other dyndns.org second-level domains to their account, making Dreamhost’s nameservers (which are both their customer authoritative nameservers and their resolvers) refuse to believe that anyone else’s dyndns hostnames exist. They remove “dnsalias.com”.

Apr 26, 2006: Home directories under /home disappear on the mail server cluster, although the actual mountpoint at some long undocumented path still works.

Jun 27, 2006: “crontab -e” reports “Permission denied”.

Jun 30, 2006: Home directories under /home disappear on the mail cluster again.

Aug 3, 2006: Home directories under /home disappear on the mail cluster AGAIN.

Aug 5, 2006: Internal reverse DNS fails, and suddenly nothing can authenticate to MySQL, which has grants to ‘user’@’hostname’.

Aug 11, 2006: Home directories under /home disappear on the mail cluster. Again.

Aug 16, 2006: Home directories… yeah. Support reply begins:

I was actually going to email you to let you know that we had a problems with those machines, however, I couldn’t remember your ID.

Aug 20, 2006: The mail problem from the 16th is resolved four days later.

Oct 2, 2006: Remember back in January where someone claimed “dnsalias.com”? It happened again. They ask me to provide a full list of “the domains I own”, even though I explained what dyndns.org was in my request. At least this time they add all of dyndns.org’s domains to their list.

Feb 24, 2006: Home directories. Mail. Yep.

Mar 16, 2007: Remember back in 2005 when they shut down the whistle forums because of load? Guess what! Again I point out that the load has been constant for months and that the only change was a new server on their end. They’re less angry this time, at least, and they again reconsider:

Actually, I’m still digging into the load on this server, and the more I dig, the more I see that the throttle on your site is pointless :) I’m very sorry about that, I actually went ahead and removed the throttle as it wasn’t bringing the load down at all. I’m still looking into the load and will let you know when I pin it down. Don’t worry though, you were a false alarm, at the time, you were the busiest site, and you were the best candidate, however, the wrong one. I’m very sorry about that!

This time I convince them to put a note on my account which basically says “This is the first site you’ll notice when this cluster gets slow, but it’s not the root cause”.

May 8, 2007: Dreamhost gets listed in the CBL.

May 18, 2007: Mail server won’t accept mail. “450 Server configuration problem.”

Jun 8, 2007: Mail bounces with “unknown user”.

Jun 9, 2007: Mail server home directories again. Not the usual root cause, though:

We had an issue with our mail updating system where the server responsible for password updates cut off the password file short.

Jul 18, 2007: They accidentally disable relaying from localhost on the webserver’s mail server. Suddenly no web apps that use SMTP can send mail. Reply in part:

Looks like this was an accidental change made to the mail config when one of the admins altered something else.

Aug 11, 2007: Mail server home directories.

Sep 19, 2007: Mail spool fills up. “452 Insufficient system storage”, complains Postfix. Reply in part:

Sorry about that! Our systems normally notify us before there are any problems, but it’s been rather busy lately with larger issues.

Oct 25, 2007: pop3 logins failing. I love this part of the (auto)reply:

Note: This was an announcement due to a large support incident. Sorry if you did not get callback support.

Jan 18, 2008: Dreamhost accidentally bills customers for their entire next year of hosting, a $7.5-million error that ends up costing DreamHost over $500,000.

Mar 21, 2008: The outage that prompted this post and my move: growth issues necessitates moving my webserver’s cluster to a new data centre, which involves a 12-hour scheduled outage. Were that not enough, following the move, load hovers around 10, and disk operations take seconds to complete. The load/IO problem isn’t resolved until Mar 26.

Dreamhost will still be handy for the whistle forums and anything I need to host a lot of noncritical but big data for, but I’m looking forward to a change.

Something went wrong – -32300 : transport error – HTTP status code was not 200

Well, that’s certainly helpful. I wonder what the HTTP status code was, or what URL was being requested that returned said status code. But at least I know that something went wrong.

1. Take photo
2. Send to Flickr photo-posting address from phone

How I send photos to Flickr on my new Virgin Mobile phone:

1. Take photo
2. Send to an email address of mine which feeds into a program, which
3. Extracts the HTML part from that message
4. Pulls the message text and a URL out of that HTML part
5. Retrieves that URL, which links to a thumbnail of the image
6. Pulls a second URL out of Javascript source on that page
7. Retrieves the second URL, which links to the full-sized image’s page
8. Pulls the image URL out of the full-sized image’s page
9. Requests the image, but with curl(1) instead of libwww-perl, since libwww-perl mysteriously produces 500 Internal Server Errors on that page only, even though the previous two pages loaded fine
10. Bundles up a new MIME message with the text from (2) and the image
11. Sends that to the Flickr photo-posting address

Real elegant. Oh, well, at least it works. I knew going in that Virgin Mobile makes data-related things a bit difficult, but that just kept getting sillier and sillier.

One of these was the Dell Latitude 700M , a consumer-class 12.1″ widescreen laptop with built-in DVD. (At that size, optical drives are usually in external USB boxes.) The 700M was a bit outside what I was after because it was 1.5″ thick, a bit heavy, and didn’t have a Trackpoint mouse, but I hadn’t quite eliminated it from consideration because I liked the widescreen display and the DVD.

And then I saw the keyboard:

I’ve grown accustomed to expecting keyboard idiocy from Dell, but squishing together punctuation keys, leaving a big Shift key, and then leaving empty space beside the Shift key? Is the cursor diamond inviolable?

Ah, well, it’s out of the running now. (Current contenders, by the way, are the Dell Latitude D400, IBM Thinkpad X31, and HP nc4010, all 2-3 year old 12″ 1.4-1.6GHz Pentium M general-purpose ultraportables that don’t get into the ultraslim, 1.8″ disk, low-voltage madness. The Thinkpad’s winning for general engineering, the Dell for performance and familiarity, and the HP for price. Dell X300 and IBM X40 are too small.)

Since I have a list of around a hundred books in my to-read list, I don’t open requests for all of them at once. Instead, I use the “My List” feature that lets you maintain lists of catalogue entries without requesting them. It was getting a bit difficult to find books in there, though, so I decided to use a related feature that lets you create multiple lists to sort them by subject area. That way I could keep one title each from a bunch of subject areas on request, and then when those requests were filled, grab another.

This worked well for the first few lists. And then:

Oooookay. I can store as many books as I like on my three lists, but a fourth list is apparently out of the question. I know that it’s a given that online catalogues are brain-dead software — think “enterprise software” without the incentive of profit — but I can’t begin to imagine why they’d think it was a good idea to limit users to THREE LISTS. If you’re going to do that, and not, say, 63 or 255 or unlimited lists, why bother implementing multiple lists at all?

Luckily, My Opinion Counts! so I think I’m going to tell them my opinion of limiting me to three lists now.

Last night I went to move the LiveJournal friends I read regularly into Bloglines, since I’d found that I had been missing a lot of their posts because I don’t read far enough back on my friends page sometimes. Bloglines supports HTTP auth, but only after importing a couple hundred feeds did I learn that it doesn’t support digest auth fails to authenticate against LiveJournal. Without auth, I can’t see friends-only posts, so that’s no good. A deal-breaker, in fact.

So I set about finding a new feed reader, and am trying four: NewsGator, a Web-based reader like Bloglines; FeedDemon, a standalone Windows application from the NewsGator people; and Sage and Brief, two Firefox extensions. Below the cut you can find my thoughts on those four plus Bloglines.

My requirements for RSS readers are pretty specific. They must work in newspaper format (when reading a feed, all of the articles shown together in one pane), not in email format (a list of subjects, and you can read one at a time). They must support OPML import and export. They must let me store feeds in folders. They must support some sort of authorization that lets me read LiveJournal friends. They must handle hundreds of feeds and thousands of unread items without grinding to a halt.

On top of that I’ve got a bunch of things I prefer but don’t require: reading a feed should mark it as read without having to click on individual articles or a “mark all read” button. Clicking an article should open it up in its own browser tab. Feeds that don’t have any new articles should still appear in the feed list.

Bloglines

Pluses: Very fast. Since it’s online, checks feeds for you even when you’re not there. Designed for newspaper-style reading. Handles opening originals in new tabs intuitively. Good subscription-editing tools. Nice simple easy-to-read layout. Frames and AJAX make it work like an application instead of a series of webpages.

Minuses: Doesn’t support any kind of authorization other than Basic Won’t authenticate with LiveJournal, which is a deal-breaker for me.

Summary: If it weren’t for the authorization problem I’d still be using it, and this whole effort is to find something like Bloglines but better. It’s like throwing out a favorite pair of jeans.

NewsGator

Pluses: Works very much like Bloglines: online, newspaper-style, sensible handling of opening new articles. Authentication support is explicit, with username and password fields for feeds instead of just having to supply a http://user:pass@hostname URL.

Minuses: Slower than molasses. Not frame- or AJAX-based, so every click on a feed forces the entire page to reload, and scrolling down to read a busy feed makes the list of feeds scroll off the top of the screen. Doesn’t mark a feed as read when you display all unread articles in it. (What else could “read” mean?) Feeds with no unread articles don’t appear in the feed list.

Summary: The performance problems are a deal-breaker. The navigation issues can be worked around in part by GreaseMonkey. Maybe if it loaded faster the complete page loads would be a non-issue, but it doesn’t.

Sage

Pluses: Browser-based, so it’s always right there. Can use Firefox’s own cookies for authentication. Updates feeds pretty fast for a local reader (and even faster if you pair it with Live Bookmarks, I’m told). Feed editing done via Manage Bookmarms panel. Minimal featureset, very much in the “do one thing well” vein. Allows restyling of article view with user-supplied CSS. Marks a feed as read as soon as you bring it up, in newspaper style. (Also supports email style if that’s your kink.)

Minuses: Sits in the sidebar, so while it’s open, it’s open in every browser tab. Since it’s browser-based, it has to update when you open it instead of showing up ready to read. Doesn’t show number of posts unread in feed list.

Summary: Pretty darn close to Just Right, but having it stick around in all tabs is a real pain. Browser integration makes a lot of things easy, particularly feed management and authentication. There’s apparently a patch that makes it display number of unread posts, but I haven’t found it yet.

Brief

Pluses: Browser-based, so shares most of the benefits of Sage — but this one is its own XUL page, so it doesn’t have the sidebar-everywhere issue that Sage does. Shows number unread. Accepts user CSS styles for newspaper view, but not the same styles as Sage (sigh). Provides total number unread in the status bar.

Minuses: Very, very slow; about a minute just to display 300 feeds with 3000 or so unread items. Folders don’t collapse. No “mark all as read”. Since it’s the other Firefox feed reader, there’s a better chance it will fade into obscurity at some point in the future.

Summary: The performance issues and having to click a button on every single post to mark it read make this a non-starter, which is a shame because the single-XUL-page interface worked out very well.

FeedDemon

Pluses: Standalone application can sit there updating while I use my browser for other things; even then, it handles 300 feeds with 3000+ unread with ease. Newspaper view is intuitive. Shows number of unread posts. Accepts user styles (but not CSS). Supports digest authentication, with a separate config panel for passwords.

Minuses: Many, many features I’ll never use, like podcasting, synchronization with Outlook, and a built-in browser. Synchronization with Newsgator doesn’t seem to maintain folder structure. Doesn’t mark a feed read when viewing it. Commercial software with trial period. Feeds and folders will only sort alphabetically. Built-in tabbed browser is IE.

Summary: I can see why a lot of people consider FeedDemon essential, and it’s close to what I’m after, but a standalone application takes second place to something that sits in the browser. That might just take getting used to, though.

* * *

So as it stands I think it’s going to be a battle between Sage and FeedDemon, although I’m also considering writing a Digest Auth proxy and pointing Bloglines at that to get it to read friends-locked posts I’d be able to see on LiveJournal itself. I’m sure I’ve missed an option or two I should check out, though — if any readers have a favorite Windows RSS reader which works newspaper-style, handles digest auth, and meets the rest of my requirements, or if anyone has any input on the ones I’ve listed above, let me know!