A Fistful of Feeds

The man with no nameCue up the Ennio Morricone music and head for the hills! There’s been some recent talking among the town folk, about the feeding frenzy that’s happening out there on the wild web of the west. Let’s just say San Miguel will never be the same once the schema with no XSD enters town.

Just when you thought it was safe in digital listing land, it’s going to get a little wilder. You see, Jesse ‘Zillow’ James has got a new six shooter and is getting ready to take your listings, publish them for the world to see, and give the town sheriff something else to think about. Right now, Jesse is just at target practice, but high noon at the O.K. Corral is coming soon enough.

Even better, Jesse has been taking marksmanship lessons from Wild Bill Gates’ old play book. It’s every bit as clever as the lead shield old Clint used in the movies. You see, Zillow’s doing 2 things which show they’ve learned the “embrace and extend” tactics from yesterday’s web slinging masters.

First, Zillow is embracing Trulia’s feed format – This move means that anybody who already has a Trulia listings feed will be able to get their listings onto Zillow with than less than 10 minutes of effort (the amount of time it takes to fill out a form with your feed url). It’s entirely possible that by doing this, Trulia’s feed format will become the “de-facto” industry standard. (Which wouldn’t be all bad)

Secondly, Zillow’s extending the purpose of Trulia’s format, by coming out with their own feed format – OK, some of you are already thinking, oh great, another XML format I need to implement and support. However, I think Zillow will be able to garner support for their ZIF format because of the following reasons.

  1. Their spec is simple to understand. Unlike GoogleBase & edgeio, which seem to be trying to win an Obfuscated XML code contest with their name spaced RSS mess, Zillow’s feed documentation is every bit as clear as the current industry feed leader, Trulia.
  2. Their spec is comprehensive. The only industry schema that compares to the breadth and depth of the Zillow’s XML Schema is RETS. Except Zillow has the benefit of not having to getting 900 MLS boards to play nice together.
  3. Doc ‘Trulia’ Holliday is not dumb. A master gun fighter in his own right, nothing is preventing Trulia from embracing the Zillow feed standard as their V2 spec. If that happens, RETS may suffer the same fate as Lester Moore. Out here on the wild web of the west, there’s the quick and the dead.
  4. Oh yeah, they also get about 4+ million monthly visitors on their web site.

Anyway, grab the popcorn; it’s going to be show!

Adventures in digital listing land

Recently, one of my clients (Real Property Associates) asked me to automate the process of submitting (or advertising) their real estate listings and rental properties on Trulia, Google Base, and Craigslist. After implementing the feature, I thought sharing my experiences would a make an interesting blog post. (So here we are)…

[photopress:Feed_1_2.gif,full,alignright]As you may know, there are 2 ways of getting your listings on Trulia. The easiest is just to let Trulia crawl your site. Unfortunately this method doesn’t work very well since there are an infinite number of ways to present listings on a web page, and Trulia’s engineers haven’t been able to spend the requisite infinite amount of time required to handle all the cases. This isn’t a knock on Trulia, since Google Base doesn’t even attempt to do this, but just a reminder that there are a lot of things software just can’t do yet. If this method works for you, your lucky.

The recommended way is far more reliable. You merely need to host an XML file on your web site that contains the listings you want to promote, and then once day or so, Trulia’s web farm will request your file, parse it, and import onto their site for the whole world to see.

In my case, since I already export MLS searches via RSS (I knew writing that feature was a good idea), I merely had to spend a couple hours tweaking the output of my MLS RSS feed pages to match Trulia’s schema, register the URL on Trulia, and in 48 hours, we had listings on Trulia. And in 72 hours, I noticed referrals from Trulia was already generating about 4% of the site’s traffic!

By comparison, Google Base was easier in some respects and more cumbersome in others. The nice thing about the Google Base file format is that it is standard RSS. Or rather, it’s standard in the same way the Microsoft Word exports standard HTML. It’s RSS with a bunch of namespaced items for the custom attributes that Google Base uses for it’s Housing item type. Anyway, if you have already have an MLS RSS feed, tweaking the output to match Google’s schema is pretty straight forward. I should note that Google appeared to be more particular about the XML it gets than Trulia appeared to be, so you’ll probably be spending more time getting things onto Google Base.

The problem with Google Base isn’t creating the feed, it’s getting it up there. You see, Google Base does not download an URL like Trulia does, therefore you have to upload your data to the GooglePlex. There are 2 ways to upload your data, via a web browser or via ftp. I ended up writing a script on my server that would download a Google Base feed from my web server, and then upload it to Google in the middle of the night.

Automating Craigslist from a web page was an interesting challenge. They have a very aggressive anti-spamming policy, CAPTCHAs, have no supported way of submitting a post programatically, and the web browser’s cross domain security model certainly doesn’t make things easier. Fortunately, I found a way around everything but the CAPTCHA, but it required some IE only technology since Firefox on Windows still doesn’t support COM automation. (BTW, if any developers out there know if XUL applications on Firefox/Mozilla can accomplish everything IE based HTA’s can, drop me line. I’d love to talk with you)

After serving up listings to “the major players”, I decided to see what the beast from Redmond was up to. Turns out they want in on the action too (big surprise), and the 1-2 punch of Windows Live Expo and Live Product Upload appears to be Microsoft’s answer to both Craigslist and GoogleBase. I’ve signed up for the Live Product Upload Beta, and I’m looking forward to adding support for their service once they get their act together. It looks promising, but currently their upload service is more designed for merchants selling products, instead of real estate professionals selling homes.

Hopefully, the Live Product Upload team will correct this oversight and support multiple item types for upload. They better not wait too long to get that feature implemented, because I’ve recently discovered that Propsmart, Oodle, Edgeio, already have web feed programs in place for XML formatted listing submission. It looks like I’m going to be busy…

So, what sites do you use for listing promotion (or just reading classifieds)? Backpage.com looks like a promising up and comer. Anybody use postlets.com to assist in your online classified ad management? Anybody using Zillow, SubmitYourListings.com, or Ebay for listings promotion? Is paid advertising worth the expense when the free online classified marketplace is exploding?