Buyers want a house; Sellers want a buyer

The National Association of Realtors held their Mid-Year conference this week. Often the Mid-Year Conference kicks up more dirt than the Annual event, as the Mid-Year conference focuses more on the business end and the mls issues (legislative sessions).  Less partying…more in depth issue discussions.

The big new issue is about Trulia and Zillow and the like. “Scraping” data vs. being spoon fed “appropriate” data. The big old issue is Dual Agency and what are “we” going to do about “it”.

The only news we must all and always be mindful of during these discussions is “we” are not the only ones in the “room”, and never will be.  Remember in the Bible when the Powers That Be of the time tried to trick Jesus into boiling everything He said in His whole life down to ONE major and single Rule?  Jesus didn’t skip a beat. If you don’t know what His one line answer was, well…go Google that. This is a Real Estate blog .

The one and only single rule of real estate, from which all other rules should follow is this: Buyers want a house at the lowest overall cost; Sellers want a buyer who gives them the highest net return.

Now take every mls rule, every State’s agency law, everything any broker wants and doesn’t want, everything any agent wants or doesn’t want, and hold it up against that one measurement…that one rule above all rules. Does what you want help buyers get their house at the lowest overall cost to them? Does what you want help sellers sell their house at the highest net return possible? If the answer is no…then change what you want.

The Big “New” Issue is about control of the mls data, control of the inventory, data scraping vs. direct feed via what insiders call IDX.  IDX is what you see when you search property on any agent site. Simple as that.

The sticky wicket for issue number one is that buyers want to see all the houses, including For Sale by Owner homes, preferably all on one site. That is why a Public MLS (kind of what Trulia and Zillow may turn into) serves the needs of buyers better than a private and Broker controlled site. That will continue to be true until and unless the Brokers fill the need of buyers (and to some extent sellers) by permitting listings that have no listing agent. Don’t hold your breath on that one.

The answer, as I see it, is two sources and not one.  One that has all listed property via any Brokerage site (as we all have access to all via IDX) and one that has all UN-listed property…and nothing else. Until then, buyer’s of homes will be confused into thinking that Zillow and Trulia and the like have all of the listed property PLUS…which it doesn’t. That means some portion of the public is always being mislead. Those that use only a brokerage site, and miss a choice For Sale By Owner property, and those that use Trulia or Zillow or Realtor.com, and miss a choice listed property. [One additional site for all rental property would be nice too. There’s a need someone should fill. But there never seems to be enough money generated by rental fees to support it actually happening.]

The Big “Old” Issue is Dual Agency. We already have that answer to some extent, it’s called Designated Agency. We simply need more time and practice and experience in the actual practice of Designated Agency…and that as they say is the SLOG of it. Until California adopts Designated Agency…there is no answer beyond the slog of it. When and IF California adopts Designated Agency, we’ll be able to make quicker progress.

Eradicating Dual Agency is not The NAR’s prerogative (Jim Duncan). Why? Look at The main rule of real estate according to ARDELL, characteristically in BOLD lettering in this post at paragraph four.

Sometimes and often, the buyer’s best way to get the house and/or get lowest overall cost, is by using the listing agent.  Not always, but sometimes and often. The State can’t…the NAR can’t…remove that option from the buying public. In reality what a buyer wants is full representation, from the person who knows the most about the house, and at the lowest possible cost which is free (or what they sometimes perceive to be free).

Sometimes and often, the seller’s best way to get a buyer to buy his house and get the highest net return is to cut out one of (or both of) the agents in the process.The State can’t…the NAR can’t…remove that option from the selling public In reality what a seller wants is ready access to all buyers in the marketplace without having to pay two agents, AND they want the buyer agent fee to come back to them vs. it being given to the buyer, if the buyer has no agent. They also don’t want to pay a buyer agent to tell the buyer that the house is overpriced or inadequate. They also want the agent they hire to be free to bring them a buyer direct (dual agency).

All of the answers with regard to Dual Agency are done with from the NAR’s perspective. They discourage agents from practicing it, until and unless it is absolutely necessary (when the buyer and seller want it). Each State has a long way to go on agency issues, like explaining “no agency” in it’s required agency disclosure noting it as an option. Until States stop asking for the real estate industry to approve and help with it’s agency options, “No Agency” will not appear as a fully explained option for their constituency.

Zillow widens gap on Trulia, traffic soars to 7.5 million visitors

Thanks to John Cook at TechFlash for the article (and picture)zillow-month.  I overheard Rich Barton talking to some friends a couple weeks ago at a TechFlash Launch Party talking about the record number of hits they were receiving.

I guess using Zillow today, was like watching the NASDAQ 8 years ago.

Dear Zillow-meisters – Better start makin’ copies of the Trulia-nator

The folks at Trulia, have just released a new feature that is cooler than the frozen tundra of Lambeau Field. In fact, I think Zillow and Realtor.com will need to order more photocopiers. Trulia has just released their Trulia Publisher Platform, and the coolness of this feature is that it lets publishers use their search technology, with your listings, with a publisher’s co-branding, and at no cost to the publisher. This is the real estate equivalent of Google AdSense and will change the nature of the real estate web advertising game, perhaps drastically.

Currently, Trulia has signed up Kiplinger, American Towns, and perhaps most interesting, Seattle Weekly. Getting Seattle Weekly as a publishing partner has to annoy Zillow more than Apple giving free computers to Redmond-area schools annoyed Microsoft. If Trulia can sign up more publishers (frankly the value proposition is so simple & compelling for small to medium sized publishers, I can’t think of a reason why they won’t sign up) they are going to have to upgrade their servers to handle the increase in traffic.

What’s this mean for publishers? Well if they are small or medium sized, they just got a much more effective way of associating their brand and increasing real estate related web traffic. Granted, Trulia controls the listings and the technology, but if your core competency isn’t real estate search, getting a co-branded search tool is much more cost effective. And since Trulia has over 2 million listings, the publishers will probably get more traffic & ad revenue too. Seems like an easy decision to me.

If you’re a big publisher, it’s a much harder decision. But since developing technology is expensive and getting listings critical mass is difficult, I suspect the desire to partner w/ Trulia got much stronger unless you’re a direct Trulia competitor. If Trulia gets big web media players to partner with it, things could get very interesting.

What’s this mean for Realtor.com? I dunno, but it’s increasingly looking like they are going to get HouseValued (yes, I just made that verb up), if they don’t show some brain activity.

What’s this mean for Zillow? Just when Zillow’s listings feed program was getting off the ground, Trulia does this! I’m guessing the sales & engineering departments just learned what they are going to be working on for the next several months.

What’s this mean for the broker in the trenches or realtor on the street? Well, if you have a Trulia listings feed, you just got more free exposure. If you pay Trulia to feature your listings more prominently, well you just got a much better return on your investment. If you liked the free traffic Trulia gave you before, you’re going to LOVE them now. Perhaps even more than the 12th man, loves his Seahawks.

A Fistful of Feeds

The man with no nameCue up the Ennio Morricone music and head for the hills! There’s been some recent talking among the town folk, about the feeding frenzy that’s happening out there on the wild web of the west. Let’s just say San Miguel will never be the same once the schema with no XSD enters town.

Just when you thought it was safe in digital listing land, it’s going to get a little wilder. You see, Jesse ‘Zillow’ James has got a new six shooter and is getting ready to take your listings, publish them for the world to see, and give the town sheriff something else to think about. Right now, Jesse is just at target practice, but high noon at the O.K. Corral is coming soon enough.

Even better, Jesse has been taking marksmanship lessons from Wild Bill Gates’ old play book. It’s every bit as clever as the lead shield old Clint used in the movies. You see, Zillow’s doing 2 things which show they’ve learned the “embrace and extend” tactics from yesterday’s web slinging masters.

First, Zillow is embracing Trulia’s feed format – This move means that anybody who already has a Trulia listings feed will be able to get their listings onto Zillow with than less than 10 minutes of effort (the amount of time it takes to fill out a form with your feed url). It’s entirely possible that by doing this, Trulia’s feed format will become the “de-facto” industry standard. (Which wouldn’t be all bad)

Secondly, Zillow’s extending the purpose of Trulia’s format, by coming out with their own feed format – OK, some of you are already thinking, oh great, another XML format I need to implement and support. However, I think Zillow will be able to garner support for their ZIF format because of the following reasons.

  1. Their spec is simple to understand. Unlike GoogleBase & edgeio, which seem to be trying to win an Obfuscated XML code contest with their name spaced RSS mess, Zillow’s feed documentation is every bit as clear as the current industry feed leader, Trulia.
  2. Their spec is comprehensive. The only industry schema that compares to the breadth and depth of the Zillow’s XML Schema is RETS. Except Zillow has the benefit of not having to getting 900 MLS boards to play nice together.
  3. Doc ‘Trulia’ Holliday is not dumb. A master gun fighter in his own right, nothing is preventing Trulia from embracing the Zillow feed standard as their V2 spec. If that happens, RETS may suffer the same fate as Lester Moore. Out here on the wild web of the west, there’s the quick and the dead.
  4. Oh yeah, they also get about 4+ million monthly visitors on their web site.

Anyway, grab the popcorn; it’s going to be show!

Tech Thursday: Are you addicted yet?

After a Wacky Wednesday, I thought it might be time to return to real estate technology…

ShackPrices adds mass transit to their listing search and Greg continues to be impressed(so am I)

USA Today provides an idea for a potential update to ShackPrices… What if Galen included the emotional map of each area?

Speaking of new online mapping tools, Joel has a nice write up on a new home search site out of Toronto called Real Estate Plus that was built by Fraser Beach

The vFlyer folks published a huge list of Web2.0 sites… There are some obvious omissions (I would have found a place for sites like Cyberhomes, Sellsius, PropertyShark, RealEstateShows, HomeHugg and, of course, Shackprices), but overall, it was a valiant effort to capture the cutting edge of the online real estate front…

[photopress:dustin_reptile.jpg,full,alignright]The Real Estate Zealot gives some good background on using Yahoo’s JumpCut to edit and stream real estate videos… (If YouTube made the previous list, then JumpCut appears to have earned a spot as well…)

Nothing too big, but I have been working with some others to build some new themes and widgets for a WordPress website for a Move Trends website that went up a little bit ago… (Note: I also took control over the “hat” at the top of Move.com, so don’t be too surprised if I start sending traffic to random places! LOL!)

The release of the updated Google Analytics has been a real joy! I spent way too much time this evening clicking on the “Entrance Sources” option for popular pages on RCG (it feels much more informative than the previous layout). In the process, I’ve learned a ton about where and how traffic is reaching the site and I’ve actually learned that some of my previous assumptions were completely wrong. (However, considering I’m not using any of the goal tracking or funnel analysis, Seth thinks I should just quit… but I’m having way too much fun to quit…)

I’ve also been wasting spending way too much time on Facebook recently (it ramped up after Joel’s recent post). Fight it if you wish, but I predict online social networking is in your future…

UPDATE: Shortly after hitting publish, Trulia announced some major enhancements to their websiteBloodhound has the details (including a podcast by Bryan).

Death by a thousand paper cuts

[photopress:papercut.jpg,thumb,alignright]Every once in a while a realtor or broker from out of state will ask me to develop an IDX web site for them. Unfortunately, supporting a new MLS is very similar to supporting a foreign language. It is a large software engineering task that takes a lot of time, and since I don’t already have the code written and don’t already have access to their MLS’s feed, I inform them that time is money and the conversation usually ends there. Someday, that may not be the case, but I’d rather be small & profitable than large & broke.

The problem is made worse by the fact that many Realtors don’t know what format or protocol their MLS uses for data downloads or even who to contact in their MLS to get a feed for an IDX vendor. If you ever want to change IDX vendors, hire a software engineer or are crazy enough to do it yourself, you should know this. Knowing how your MLS distributes your listing data is like knowing how to change the oil in your car or how to defragment your hard drive. You don’t have to know, but it’s good to know. It may seem like I’m ranting about some MLS techie mumbo jumbo thing again, but it is preventing the industry from taking advantage of the low cost IT innovations that could be. I don’t think folks fully appreciate the challenges that an IDX vendor faces and how those challenges are retarding the industry’s growth and health.

For example, the NWMLS (Northwest Multiple Listing Service – serves mainly Seattle, WA and western Washington) uses software from Rapattoni. It provides listing data via a proprietary SOAP interface and all the photos are accessible via an FTP server. Listing data is updated constantly (a new listing usually appears in our feeds about 15-20 minutes after it’s been entered into NWMLS by a member as I understand it).

By contrast, EBRD (East Bay Regional Data – serves mainly Oakland, CA and the east bay area) uses Paragon by Fidelity MLS Systems provides it’s listing data via nightly updated CSV text files, down-loadable by FTP. The new and updated listings images are accessible via ZIPed files via FTP. The photos for active listings which haven’t been recently added or changed are not available (unless you bug the IT dept).

The only way they could make their systems more different is if the EBRD encoded their listings in EBCDIC! In order to support both, I need to develop 2 very different programs for downloading the listing data onto my server, importing the listing data in my database, dealing with differences in the listing schema (for example, the EBRD doesn’t contain a “Number of Photos” field or a “Community Name” field), dealing with differences in the photo location downloading (the NWMLS stores all photos in an uncompressed format in one of a thousand sub directories while the EBRD just stores the fresh photos in one big zip file). So I can spend my limited time improving my software for new markets (that have no customers) or improving my software for my home market (which has paying customers). Unfortunately, given the current market realities I can only afford to support my home market at this time since MLS IDX programs can be very different and there is no place like home (so far as I know anyway).

I keep waiting for RETS to save me from this madness, but until it happens in Seattle or the East Bay, I’m not holding my breath. After all, if two of the larger MLSes in the country in the two most tech savy areas of the nation don’t support it yet, I interpret it to be a vote of no confidence. I suppose, RETS could be going great guns in the rest of the country, but if it was, I’d expect the NWMLS & EBRD to be all over it, like the establishment on Redfin.

The Center for REALTOR® Technology Web Log, paints a rosy a picture regarding RETS deployment in the industry. Unfortunately, according to Clareity Consulting, an IT consulting firm that serves MLSes and other parts of the real estate eco-system, RETS is the NAR’s unfunded mandate. Although, everybody wants the benefits of RETS, nobody is willing to pay for it. Furthermore, it appears back in days before I got sucked into real estate technology, there was an effort to promote the DxM standard and that went nowhere (which is a bad omen). What’s worse is that they keep moving the goal posts. We don’t even have widespead RETS 1.0 support, and they’ve already depreciated that standard going full bore on RETS Lite and RETS 2.0. It seems the biggest problem is one of vision and scope. They keeping adding more features to cover more scenarios, when we don’t even have wide deployment of the existing standard (assuming that we had standards to begin with at all). It reminds of the recent software industry debacle that is known as “Longhorn reset“. The problem is that RETS is just too complicated, in an environment with too many legacy systems in place, too few resources to support it, and excessive aspirations. The idea of RETS is great, it’s the implementation and deployment that’s disappointing and at least Microsoft pulled Vista out if it’s death spiral…

[photopress:pappercutter.jpg,thumb,alignleft]The sad thing is that computer industry already has great tools for moving data around over the Internet in efficient and well supported (if sometimes proprietary ways). They allow you to query, slice, and dice your data in a near infinite number of ways. They’re called database servers. They are made by multiple software vendors and there are even some excellent open source ones out there. They let you set permissions on what accounts can see what tables or views (gee, sounds like something an MLS would want). The better ones, even have this level of security to the field level. Even better, most of these so called database servers have the ability of exporting data into spreadsheets, reporting tools, and even GIS systems. All of them provide a well defined and often times well implemented API that software developers can use and exploit to implement what hasn’t been invented yet!

Why doesn’t the NAR & the MLSes save us all the trouble, standardize on a few good database platforms (I’m a fan of MS SQL Server and MySQL, but I’d settle for anything that has ODBC, .net & Java support at this point), and provide everybody RDBMS accounts? It’d lower the cost for us IDX vendors (less code to write, since everything is just SQL), it’d lower the costs for MLS vendors (since data access, security, programmability, and scalability is now the RDBMS vendor’s problem), provide more choices for agents and brokers (since getting Excel talking to MS SQL Server is a cakewalk compared to RETS) and it will lower IT costs for the MLS (because the MLS vendors don’t need to invent an industry specific solution to a problem that’s been largely solved already and I’m betting that the MLS vendors already use somebody else’s RDBMS to implement their solutions anyway). Granted, a SQL Server won’t enable all the scenarios that RETS wants to enable (if RETS was ever well implemented and widely deployed enough for that happen). However, I’m of the belief that it’s not going to happen until after Trulia or Google Base becomes the de facto nationwide MLS by providing a single schema with a simple REST like web services interface.

So, what does your MLS do to support IDX vendors? Do they provide all the data all the time, or just daily updates? Have they deployed RETS yet? Are they going to? Who is their MLS software vendor or do they have a home gown solution? What do you want to do, that you can’t do today because the data is in a format that you can’t use easily? Would you be willing to pay more in membership dues for better software or better service from your MLS? Are we at the dawning of the RETS revolution, or is it too little, too late?

PS – Anybody, know anybody from an MLS / IDX dept or MLS vendor that blogs? I’d love to know what things are really like on their side of the listing data fence.

Adventures in digital listing land

Recently, one of my clients (Real Property Associates) asked me to automate the process of submitting (or advertising) their real estate listings and rental properties on Trulia, Google Base, and Craigslist. After implementing the feature, I thought sharing my experiences would a make an interesting blog post. (So here we are)…

[photopress:Feed_1_2.gif,full,alignright]As you may know, there are 2 ways of getting your listings on Trulia. The easiest is just to let Trulia crawl your site. Unfortunately this method doesn’t work very well since there are an infinite number of ways to present listings on a web page, and Trulia’s engineers haven’t been able to spend the requisite infinite amount of time required to handle all the cases. This isn’t a knock on Trulia, since Google Base doesn’t even attempt to do this, but just a reminder that there are a lot of things software just can’t do yet. If this method works for you, your lucky.

The recommended way is far more reliable. You merely need to host an XML file on your web site that contains the listings you want to promote, and then once day or so, Trulia’s web farm will request your file, parse it, and import onto their site for the whole world to see.

In my case, since I already export MLS searches via RSS (I knew writing that feature was a good idea), I merely had to spend a couple hours tweaking the output of my MLS RSS feed pages to match Trulia’s schema, register the URL on Trulia, and in 48 hours, we had listings on Trulia. And in 72 hours, I noticed referrals from Trulia was already generating about 4% of the site’s traffic!

By comparison, Google Base was easier in some respects and more cumbersome in others. The nice thing about the Google Base file format is that it is standard RSS. Or rather, it’s standard in the same way the Microsoft Word exports standard HTML. It’s RSS with a bunch of namespaced items for the custom attributes that Google Base uses for it’s Housing item type. Anyway, if you have already have an MLS RSS feed, tweaking the output to match Google’s schema is pretty straight forward. I should note that Google appeared to be more particular about the XML it gets than Trulia appeared to be, so you’ll probably be spending more time getting things onto Google Base.

The problem with Google Base isn’t creating the feed, it’s getting it up there. You see, Google Base does not download an URL like Trulia does, therefore you have to upload your data to the GooglePlex. There are 2 ways to upload your data, via a web browser or via ftp. I ended up writing a script on my server that would download a Google Base feed from my web server, and then upload it to Google in the middle of the night.

Automating Craigslist from a web page was an interesting challenge. They have a very aggressive anti-spamming policy, CAPTCHAs, have no supported way of submitting a post programatically, and the web browser’s cross domain security model certainly doesn’t make things easier. Fortunately, I found a way around everything but the CAPTCHA, but it required some IE only technology since Firefox on Windows still doesn’t support COM automation. (BTW, if any developers out there know if XUL applications on Firefox/Mozilla can accomplish everything IE based HTA’s can, drop me line. I’d love to talk with you)

After serving up listings to “the major players”, I decided to see what the beast from Redmond was up to. Turns out they want in on the action too (big surprise), and the 1-2 punch of Windows Live Expo and Live Product Upload appears to be Microsoft’s answer to both Craigslist and GoogleBase. I’ve signed up for the Live Product Upload Beta, and I’m looking forward to adding support for their service once they get their act together. It looks promising, but currently their upload service is more designed for merchants selling products, instead of real estate professionals selling homes.

Hopefully, the Live Product Upload team will correct this oversight and support multiple item types for upload. They better not wait too long to get that feature implemented, because I’ve recently discovered that Propsmart, Oodle, Edgeio, already have web feed programs in place for XML formatted listing submission. It looks like I’m going to be busy…

So, what sites do you use for listing promotion (or just reading classifieds)? Backpage.com looks like a promising up and comer. Anybody use postlets.com to assist in your online classified ad management? Anybody using Zillow, SubmitYourListings.com, or Ebay for listings promotion? Is paid advertising worth the expense when the free online classified marketplace is exploding?

Realtor.com takes a step closer to Trulia

Come spring, Realtor.com won’t have access to the Northwest MLS, so only houses from agents who pay to participate will show up. Trulia.com, a site the “crawls” broker web sites to fill its database, will probably have more listings from western Washington from that point going forward (the NWMLS still has the most).

Usually I believe that access trumps all, but it is unseemly for an organization like the NWMLS to give its data away for free to a corporate giant, but block all other interested parties. Realtor.com, I’d like to be the first to welcome you to the rest of the world, where you either have to work with a licensed broker and play by the rules, accept listings for free, or crawl the web to find listings.

Is Trulia totally clueless?

In a post I’ve been meaning to blog about, Trulia asks Is the MLS Totally Clueless? Why is “the MLS” totally clueless (they mean all 1,000+ MLS systems)? According to Trulia, because they don’t allow For Sale By Owner properties to be shown alongside MLS listings. (Is Trulia right, is this dumb? Probably, but I’ll leave that for another discussion.) What is Trulia’s solution? They don’t allow For Sale By Owner properties to be shown alongside their broker listings (see answer number 1). Sounds like someone shouldn’t be throwing stones.

Will Trulia ever allow FSBO listings? My guess is the moment that Trulia gets enough traffic for consumers to care if their house is listed on Trulia, the tables will turn on the brokers they are so eagerly courting right now. FSBO: check. Smaller links to broker sites: check. Data added by users: check. I don’t think they’ll do this out of greed, rather they’ll do it because they have to: if they keep the site the way it is, with limited information about properties and links to agent sites, the rest of the industry is going to pass them by.

Note: Trulia is not a member of “the MLS” and does not need to follow any MLS rules. There was some reader confusion about this.