Where should the MLS end and the IDX begin?

The whole ruckus over the NWMLS no longer sending its member’s listings to realtor.com inspired many unlit pixels of commentary and many more wasted bytes of hard drive space. As I pondered a while ago, the industry appears to have a healthy appetite for technology. However, one of the comments was really insightful….

I still feel that this decision made by the board was wrong. As was the decision last year to disable the client email updates from Locator. We have the technology but are unwilling to use it. I have no love of REALTOR.com but I see no problem with sharing a limited set of data with them and offering our sellers maximum exposure of their listing. In fact, perhaps one of the reasons they discontinued the feed was because as Galen said, “Realtor.com was given the exclusive non-broker feed…” and they were getting pressure from Google and others to get a similar feed. I say give it to them. NWMLS has the ability to provide its members, all of them, with the technology usually reserved only for those with very deep pockets.

The whole thing got me wondering if this just a tactic for the big brokers to keep their upstarts at bay? Because of the MLS system, the big brokers share their listings inventory, with the smaller and independent brokers. However, perhaps the big brokers want the technology out of the MLS, because it harms the smaller upstarts without withholding listings from them?

Maybe there’s a less nefarious motivation. Since the NWMLS board appears to be dominated by members that belong to big brokers, perhaps they don’t want the NWMLS spending its’ limited computing resources (at the end of the day, even the Google’s & Microsoft’s have limited budgets, they just have a few more zeros at the end than most of us do) in areas where a big broker’s IT department or a motivated IDX vendor could do a better job. Regardless of the motivations, it does bring an interesting issue to light.

What should the MLS responsibilities be in terms of listing change notification, statistics/reporting, automated listing distribution, listing access via mobile devices or any number of things that either an MLS or an IDX vendor could provide?

I’m sure the big brokers are less enthusiastic about this type of thing because some have probably already invented these kind of technologies in house years ago (and paid for it out of their own pocket). They probably also see the MLS as competition for viewer eyeballs and would rather the MLS make it easier to combine listings data across their empires instead of being a shared technology provider. MLS regionalization is probably much higher on their MLS IT wish list. After all, the point of an MLS is to share listings data, not share listings technology.

The independent agents and the smaller brokers, probably want the MLS to provide these services, so they don’t have invest any more money in their IDX vendor / IT infrastructure that they don’t have to. I also suspect a lot people in that market segment see technology as an expense and not as an investment. They only want it, if they don’t have to pay for it.

As for me, I’m just an IDX vendor (I don’t have a dog in the fight). From my biased perspective, the less the MLS does, the more valuable my technology becomes, the more useful my services become, and the more opportunities for paying customers I get. I want you to spend money on your IT infrastructure and your IDX vendor! Apparently, the big brokers want you to do the same via their MLS policy direction!

Adventures in digital listing land

Recently, one of my clients (Real Property Associates) asked me to automate the process of submitting (or advertising) their real estate listings and rental properties on Trulia, Google Base, and Craigslist. After implementing the feature, I thought sharing my experiences would a make an interesting blog post. (So here we are)…

[photopress:Feed_1_2.gif,full,alignright]As you may know, there are 2 ways of getting your listings on Trulia. The easiest is just to let Trulia crawl your site. Unfortunately this method doesn’t work very well since there are an infinite number of ways to present listings on a web page, and Trulia’s engineers haven’t been able to spend the requisite infinite amount of time required to handle all the cases. This isn’t a knock on Trulia, since Google Base doesn’t even attempt to do this, but just a reminder that there are a lot of things software just can’t do yet. If this method works for you, your lucky.

The recommended way is far more reliable. You merely need to host an XML file on your web site that contains the listings you want to promote, and then once day or so, Trulia’s web farm will request your file, parse it, and import onto their site for the whole world to see.

In my case, since I already export MLS searches via RSS (I knew writing that feature was a good idea), I merely had to spend a couple hours tweaking the output of my MLS RSS feed pages to match Trulia’s schema, register the URL on Trulia, and in 48 hours, we had listings on Trulia. And in 72 hours, I noticed referrals from Trulia was already generating about 4% of the site’s traffic!

By comparison, Google Base was easier in some respects and more cumbersome in others. The nice thing about the Google Base file format is that it is standard RSS. Or rather, it’s standard in the same way the Microsoft Word exports standard HTML. It’s RSS with a bunch of namespaced items for the custom attributes that Google Base uses for it’s Housing item type. Anyway, if you have already have an MLS RSS feed, tweaking the output to match Google’s schema is pretty straight forward. I should note that Google appeared to be more particular about the XML it gets than Trulia appeared to be, so you’ll probably be spending more time getting things onto Google Base.

The problem with Google Base isn’t creating the feed, it’s getting it up there. You see, Google Base does not download an URL like Trulia does, therefore you have to upload your data to the GooglePlex. There are 2 ways to upload your data, via a web browser or via ftp. I ended up writing a script on my server that would download a Google Base feed from my web server, and then upload it to Google in the middle of the night.

Automating Craigslist from a web page was an interesting challenge. They have a very aggressive anti-spamming policy, CAPTCHAs, have no supported way of submitting a post programatically, and the web browser’s cross domain security model certainly doesn’t make things easier. Fortunately, I found a way around everything but the CAPTCHA, but it required some IE only technology since Firefox on Windows still doesn’t support COM automation. (BTW, if any developers out there know if XUL applications on Firefox/Mozilla can accomplish everything IE based HTA’s can, drop me line. I’d love to talk with you)

After serving up listings to “the major players”, I decided to see what the beast from Redmond was up to. Turns out they want in on the action too (big surprise), and the 1-2 punch of Windows Live Expo and Live Product Upload appears to be Microsoft’s answer to both Craigslist and GoogleBase. I’ve signed up for the Live Product Upload Beta, and I’m looking forward to adding support for their service once they get their act together. It looks promising, but currently their upload service is more designed for merchants selling products, instead of real estate professionals selling homes.

Hopefully, the Live Product Upload team will correct this oversight and support multiple item types for upload. They better not wait too long to get that feature implemented, because I’ve recently discovered that Propsmart, Oodle, Edgeio, already have web feed programs in place for XML formatted listing submission. It looks like I’m going to be busy…

So, what sites do you use for listing promotion (or just reading classifieds)? Backpage.com looks like a promising up and comer. Anybody use postlets.com to assist in your online classified ad management? Anybody using Zillow, SubmitYourListings.com, or Ebay for listings promotion? Is paid advertising worth the expense when the free online classified marketplace is exploding?