Adventures in digital listing land

Recently, one of my clients (Real Property Associates) asked me to automate the process of submitting (or advertising) their real estate listings and rental properties on Trulia, Google Base, and Craigslist. After implementing the feature, I thought sharing my experiences would a make an interesting blog post. (So here we are)…

[photopress:Feed_1_2.gif,full,alignright]As you may know, there are 2 ways of getting your listings on Trulia. The easiest is just to let Trulia crawl your site. Unfortunately this method doesn’t work very well since there are an infinite number of ways to present listings on a web page, and Trulia’s engineers haven’t been able to spend the requisite infinite amount of time required to handle all the cases. This isn’t a knock on Trulia, since Google Base doesn’t even attempt to do this, but just a reminder that there are a lot of things software just can’t do yet. If this method works for you, your lucky.

The recommended way is far more reliable. You merely need to host an XML file on your web site that contains the listings you want to promote, and then once day or so, Trulia’s web farm will request your file, parse it, and import onto their site for the whole world to see.

In my case, since I already export MLS searches via RSS (I knew writing that feature was a good idea), I merely had to spend a couple hours tweaking the output of my MLS RSS feed pages to match Trulia’s schema, register the URL on Trulia, and in 48 hours, we had listings on Trulia. And in 72 hours, I noticed referrals from Trulia was already generating about 4% of the site’s traffic!

By comparison, Google Base was easier in some respects and more cumbersome in others. The nice thing about the Google Base file format is that it is standard RSS. Or rather, it’s standard in the same way the Microsoft Word exports standard HTML. It’s RSS with a bunch of namespaced items for the custom attributes that Google Base uses for it’s Housing item type. Anyway, if you have already have an MLS RSS feed, tweaking the output to match Google’s schema is pretty straight forward. I should note that Google appeared to be more particular about the XML it gets than Trulia appeared to be, so you’ll probably be spending more time getting things onto Google Base.

The problem with Google Base isn’t creating the feed, it’s getting it up there. You see, Google Base does not download an URL like Trulia does, therefore you have to upload your data to the GooglePlex. There are 2 ways to upload your data, via a web browser or via ftp. I ended up writing a script on my server that would download a Google Base feed from my web server, and then upload it to Google in the middle of the night.

Automating Craigslist from a web page was an interesting challenge. They have a very aggressive anti-spamming policy, CAPTCHAs, have no supported way of submitting a post programatically, and the web browser’s cross domain security model certainly doesn’t make things easier. Fortunately, I found a way around everything but the CAPTCHA, but it required some IE only technology since Firefox on Windows still doesn’t support COM automation. (BTW, if any developers out there know if XUL applications on Firefox/Mozilla can accomplish everything IE based HTA’s can, drop me line. I’d love to talk with you)

After serving up listings to “the major players”, I decided to see what the beast from Redmond was up to. Turns out they want in on the action too (big surprise), and the 1-2 punch of Windows Live Expo and Live Product Upload appears to be Microsoft’s answer to both Craigslist and GoogleBase. I’ve signed up for the Live Product Upload Beta, and I’m looking forward to adding support for their service once they get their act together. It looks promising, but currently their upload service is more designed for merchants selling products, instead of real estate professionals selling homes.

Hopefully, the Live Product Upload team will correct this oversight and support multiple item types for upload. They better not wait too long to get that feature implemented, because I’ve recently discovered that Propsmart, Oodle, Edgeio, already have web feed programs in place for XML formatted listing submission. It looks like I’m going to be busy…

So, what sites do you use for listing promotion (or just reading classifieds)? Backpage.com looks like a promising up and comer. Anybody use postlets.com to assist in your online classified ad management? Anybody using Zillow, SubmitYourListings.com, or Ebay for listings promotion? Is paid advertising worth the expense when the free online classified marketplace is exploding?

Just because users can post doesn’t mean they will

In spite of all the hype for social websites, there have been a rash of noble attempts that come up short recently. No one has quite figured out the right way to start an open dialog about individual homes yet.

ShackYack, “the only real estate site where you can comment and rate the product, just like amazon, netflix, ebay etc.,” launched in late May. Apparently they have found that houses aren’t mass produced and sold by the millions like iPods and people really don’t have much to post about them. That said, part of the reluctance to comment on properties is probably the buggy interface and their newness on the market.

However! Propsmart’s interface isn’t buggy. In fact, it’s one of the best out there and they encourage comments too. But a random sampling of homes on the site shows no comments.

Why would you want to publicly comment on a house unless you’re the owner or an agent looking for leads? The house, along with your comments, is going to be off the market in a few weeks and, unless you’re an agent, you’re going to be out of the market in a short period of time too. And if you’re an agent, blogging seems like a more public and permanent way to go.

That said, Propsmart’s blogging service appears to be stillborn, with the last post being posted over 2 months ago. Even blogging systems aren’t always a hit.

I feel like I’m picking on PropSmart here, but I’m really not. Whenever you introduce new services, some will be hits and some won’t. Their search site and their interface in particular are great and they are definitely the most straight-talking real estate search company out there (possibly the only one?).

Real estate search sites hit data control sore spot

Inman news had an interesting article today about how some of the innovative real estate search sites are causing headaches across the country. Considering all the conversations on Rain City Guide recently about the future of “data” this seems quite relevant and is a very interesting read. However, Inman will make the article subscriber-only after today, so catch it fast if you’re not a subscriber (or just read below for some of the more interesting quotes):

Out of companies that have released real estate search tools, the two most interesting (to me) are definitely Trulia and PropSmart. (The Inman article also features some background on Oodle, but they don’t seem to have the laser-like focus on real estate like the other two)

Here is Ron Hornbaker of PropSmart view of the situation:

While (Ron) said the response to Propsmart has been mostly positive from agents and brokers, he has received a bit of pushback from a few people in the multiple listing service community. “First of all, they’re using the word ‘illegal’ and … ‘copyright infringement.’ I’ve just been polite and talked to them. We don’t want to do anything people don’t want us to do.” Hornbaker said he has complied with the wishes of industry professionals who wish to remove their listings from being included at the site.

“My concern is they are speaking for way more interest than they should be. I do not believe we are a Napster-like model, and that’s what we’re being equated to,” he said, referring to Napster’s past problems in offering up a service that allowed users to illegally download music. “I would ask the people who say we’re doing illegal things to point out where the harm is — who are we damaging?”

That last quote would be sure to get a few comments from concerned agents should Ron have posted it on a blog… 🙂

The property listings displayed at Propsmart, he said, “are being used simply for display to consumers to point them to a home for sale. I would think that the opposite side of this could be argued more effectively: By restricting consumer access to the listings, that has more (potential) for being illegal.

“I believe it’s the seller who’s being forgotten in this situation. It is the seller we’re trying to stand up for and do the right thing for,” Hornbaker said, adding that sellers may not be aware — and may not approve — of efforts by brokers to block the listing from being viewed on some Web sites.

“I don’t buy the ‘copyright infringement’ for a second. They see their old model slipping away and they’re grasping at what they can,” he added. “We’re not trying to make money from the brokers. We’re not a middleman. We’re just trying to be a useful tool for consumers. I think this is going to backfire on the (opponents) in time.”

The gang at Trulia also gave some background on their view of where Trulia fits into the big picture:

Sami Inkinen, COO and co-founder at Trulia, said the site’s development team made a concerted effort to bring brokers to the table. “Most importantly, even before we launched we talked to all of the key brokers in advance to make sure the product was acceptable to what they want.” Inkinen said that Trulia is also careful not to access IDX, or Internet Data Exchange property listings information that is protected through broker agreements with MLSs.

While brokers seem to get it, Trulia CEO and co-founder Pete Flint said MLS’s are “frankly a mixed bag. Some of them have yet to understand … that this is in the best interest of their members. Some of them are not friends of innovation. Really what we’re focused on for now … (is) communicating how search is really positive to the real estate industry.”

Inkinen said, “We understand who we’re serving and we’re serving the brokers. We don’t want to own content. Our motto has always been the search-engine approach, the search-engine model — help brokers to place the digital yard sign on the Internet and then point to the actual source.”

Flint added, “We’re very aware that listings are a very delicate matter. We’re not looking to reinvent the industry in any way. We’re trying to improve things a little bit for consumers, and … a little bit for brokers.”

(Thanks to Jim of Central Virginia Real Estate News for tipping me off about this interesting article)

Propsmart does some stuff right!

I really like some of the tools that Propsmart has just released!

They’ve added all the features that one would expect from a web2.0 real estate site (map-based searching and RSS feeds). Some of the great features I’ve found so far include:

  • Nationwide residential, commercial, land and apartments
  • Map-based searching over Google Maps
  • RSS feeds for each search
  • Exports to Google Earth
  • Comments on listings
  • Forums for general discussions

Overall, it is an impressive set of features and includes a fun, easy-to-use interface.

Propsmart Screenshot

The problem?
There were also some minor software bugs (like when I tried to export one of my searches to Google Earth, I ended up with a different set of listings), but I’m sure those will be worked out in the near future.

Their big problem is that they don’t have MLS data. I know this is by design (and even a central part of their manifesto), but without a more comprehensive database of information, I just can’t see this site getting much traction.

Let there be no doubt that they could get a small slice of a very big pie, but like Trulia, I just wouldn’t recommend people to the site unless they can get “all” the listings!