Adventures in digital listing land

Recently, one of my clients (Real Property Associates) asked me to automate the process of submitting (or advertising) their real estate listings and rental properties on Trulia, Google Base, and Craigslist. After implementing the feature, I thought sharing my experiences would a make an interesting blog post. (So here we are)…

[photopress:Feed_1_2.gif,full,alignright]As you may know, there are 2 ways of getting your listings on Trulia. The easiest is just to let Trulia crawl your site. Unfortunately this method doesn’t work very well since there are an infinite number of ways to present listings on a web page, and Trulia’s engineers haven’t been able to spend the requisite infinite amount of time required to handle all the cases. This isn’t a knock on Trulia, since Google Base doesn’t even attempt to do this, but just a reminder that there are a lot of things software just can’t do yet. If this method works for you, your lucky.

The recommended way is far more reliable. You merely need to host an XML file on your web site that contains the listings you want to promote, and then once day or so, Trulia’s web farm will request your file, parse it, and import onto their site for the whole world to see.

In my case, since I already export MLS searches via RSS (I knew writing that feature was a good idea), I merely had to spend a couple hours tweaking the output of my MLS RSS feed pages to match Trulia’s schema, register the URL on Trulia, and in 48 hours, we had listings on Trulia. And in 72 hours, I noticed referrals from Trulia was already generating about 4% of the site’s traffic!

By comparison, Google Base was easier in some respects and more cumbersome in others. The nice thing about the Google Base file format is that it is standard RSS. Or rather, it’s standard in the same way the Microsoft Word exports standard HTML. It’s RSS with a bunch of namespaced items for the custom attributes that Google Base uses for it’s Housing item type. Anyway, if you have already have an MLS RSS feed, tweaking the output to match Google’s schema is pretty straight forward. I should note that Google appeared to be more particular about the XML it gets than Trulia appeared to be, so you’ll probably be spending more time getting things onto Google Base.

The problem with Google Base isn’t creating the feed, it’s getting it up there. You see, Google Base does not download an URL like Trulia does, therefore you have to upload your data to the GooglePlex. There are 2 ways to upload your data, via a web browser or via ftp. I ended up writing a script on my server that would download a Google Base feed from my web server, and then upload it to Google in the middle of the night.

Automating Craigslist from a web page was an interesting challenge. They have a very aggressive anti-spamming policy, CAPTCHAs, have no supported way of submitting a post programatically, and the web browser’s cross domain security model certainly doesn’t make things easier. Fortunately, I found a way around everything but the CAPTCHA, but it required some IE only technology since Firefox on Windows still doesn’t support COM automation. (BTW, if any developers out there know if XUL applications on Firefox/Mozilla can accomplish everything IE based HTA’s can, drop me line. I’d love to talk with you)

After serving up listings to “the major players”, I decided to see what the beast from Redmond was up to. Turns out they want in on the action too (big surprise), and the 1-2 punch of Windows Live Expo and Live Product Upload appears to be Microsoft’s answer to both Craigslist and GoogleBase. I’ve signed up for the Live Product Upload Beta, and I’m looking forward to adding support for their service once they get their act together. It looks promising, but currently their upload service is more designed for merchants selling products, instead of real estate professionals selling homes.

Hopefully, the Live Product Upload team will correct this oversight and support multiple item types for upload. They better not wait too long to get that feature implemented, because I’ve recently discovered that Propsmart, Oodle, Edgeio, already have web feed programs in place for XML formatted listing submission. It looks like I’m going to be busy…

So, what sites do you use for listing promotion (or just reading classifieds)? Backpage.com looks like a promising up and comer. Anybody use postlets.com to assist in your online classified ad management? Anybody using Zillow, SubmitYourListings.com, or Ebay for listings promotion? Is paid advertising worth the expense when the free online classified marketplace is exploding?