Tag Archives: RPA

OpenSearch is beyond cool – it’s the new cold

Posted on by
32

I was reading Redfin’s Developer Blog and the IE blog a few months ago and I got this desire to write my own OpenSearch provider. OpenSearch was originally created by A9.com (an Amazon.com company) and was primarily designed as a way for web developers to publish search results in a standard and accessible format. This turns out to be a good idea because different types of content require different types of search engines. The best search engine for a particular type of content is frequently the search engine written by the people that know the content the best. Google is great at searching unstructured content on the internet, but when it comes to structured search on a single web site there are much better options (Endeca, FAST, Autonomy, Solr, my favorite SQL database, etc). The other benefit of OpenSearch providers is that it shifts the balance of power away from Google and back toward web browser vendors & web site developers.

Both of the major web browsers support the OpenSearch Referrer extension. IE 7+, Firefox 2+ & Chrome allows you to add search engines to your browser without leaving the web page. The best place to get started is from the browsers vendors themselves. You can add search providers from Microsoft’s site or you can add search providers from Firefox’s add-ons site. In the interest of full disclosure, Opera allows you to add search engines manually, and Safari currently does not support this feature in any form (unless you count using vi to edit the Safari executable or changing your OS’s hosts file as support, which I do not recommend).

Anyway, our developer friends at Redfin wrote a blog post about their OpenSearch provider on their dev blog some time ago. Of course, they took the easy way out by not developing an OpenSearch Suggestions extension (slackers). I decided that a search provider without suggestion support is lame, so I took a stab at creating one. I think what inspired me to write an OpenSearch suggestions provider is that the IE 8 team blogged about their new Visual Search feature (which embraces & extends the OpenSearch suggestions work that Firefox pioneered) and I could leverage the work to improve the search experience for both IE 8 & Firefox 2+ users. (And the satisfaction of having a cool feature that Redfin & Estately haven’t implemented yet was probably another factor).

This functionality is typically exposed to users, via the search engine bar, next to the address bar in your web browser. So in your page markup, you’ll add something like this that tells the browser that your web site has a search service.

<link title="RPA Real Estate Search" type="application/opensearchdescription+xml" rel="search" href="http://www.seattlehouses.com/Feeds/OpenSearch.ashx"/>

The above element points to your site OpenSearch Description XML file which describes your search service in a way the browser can understand. When you visit RPA’s site, the browser will read RPA’s OpenSearch Description file located here and unobtrusively let you add the site’s search providers.

Assuming everything is working correctly, the user should be able to visit RPA’s web site, click on the browser’s search bar to add our search provider like so… (IE’s screen captures are on the left, Firefox’s are on the right).

I’ve also added a button in RPA’s search bar (see above right) in case site visitors don’t discover our search provider via the browser (I suspect most users would miss it otherwise).

After you’ve registered RPA’s search provider with your web browser, you can select it and just start typing. Since I’ve implemented a suggestions service, it will auto complete cities, school districts & neighborhoods as you type them (Didn’t I say this was cool?). I should note that although IE 7 & Chrome support OpenSearch, only IE 8 and Firefox currently support the suggestions providers. Anyway, if you wanted to look for listings in Bellevue, here’s what it currently looks like.

As you’ll notice, IE 8 & Firefox 3 displays suggestions differently on RPA’s site. This is intentional because IE 8 supports a newer version of the OpenSearch standards (Microsoft calls it Visual Search) and I designed RPA’s search provider to exploit this fact. In Firefox, the browser can only handle plain text suggestions, which can lead to ambiguous searches. For example, let’s say you search for Riverview. Riverview is both a neighborhood in Kent and a school district in Carnation / Duvall, so in Firefox there is no means for the user to tell the web site in which context they meant to search for when they typed in Riverview. I suppose one could create a “Did you mean” results page for cases like this, but I think that somewhat defeats the purpose of having suggestions support.

However, in IE 8, if a term has multiple contexts, the search provider can display them all and the user can select the one they meant. Also in IE 8, the search provider can display thumbnails next to the suggestions, which further helps the user quickly find what they are looking for. Although, I haven’t implemented that feature yet (mostly because I wasn’t sure what picture I should put up there for search terms that return multiple results), other web sites have. For example, if you wanted to buy a movie from Amazon or learn more about our 16th president from Wikipedia, the IE 8 search provider experience looks like this…

As the Redfin developers stated, implementing OpenSearch Referrer extensions are surprisingly easy (so I think users will soon request them from all web sites once the word gets out). The OpenSearch Suggestions extensions are more difficult to implement because every single keystroke is essentially a REST web service call. If you aren’t careful, you could bring your web server to its knees real quick. However, given all the AJAX map based tricks today’s real estate web sites perform, this isn’t anything that a professional software engineer can’t handle.

Call me crazy, but I think OpenSearch providers are going to become bigger than RSS feeds over the next year. If IE 8’s forth coming release doesn’t launch them into the mainstream, I think future releases of Firefox & Chrome will improve upon IE 8’s good ideas. Maybe you should think of it as browser favorites on steroids? If search is sticky, then OpenSearch is superglue and duct tape. If Firefox’s suggestions support were the tip of the iceberg, then IE 8’s implementation is cooler than Barrow, Alaska. The future of OpenSearch looks bright, even if it’s cold outside.

Grasping for pebbles & listening for grasshoppers

Posted on by

[photopress:grasshopper.jpg,thumb,alignright]Gordon Stephenson of RPA asked me the other day for blog setup advice. Although I consider myself good friends with the master of real estate blogging, I am a blogging grasshopper compared to many folks. You see, I have never set up a blog before. Sure, I’m a contributor on RCG, I comment on popular real estate blogs, and I even write Virtual Earth based IDX/MLS systems (w/ geocoded RSS feeds) for fun and profit. However, I’m sure Dustin sets up more blogs before 10 AM in a typical day, than I do in a year. Hence, the reason for my post. Besides, as another real estate blogging master has pointed out to me, blogging about blogging is always a good topic.

Anyway, I essentially told Gordon the following…

  1. I’d pick up a copy of the Realty Blogging book and read it. It’s probably out of date, (like most technology related books are) but I suspect it’s as good a starting point as any.
  2. I’d recommend finding a good shared hosting company that sets up & hosts WordPress or Typepad blogs.
  3. Most importantly, I’d ask other bloggers what they think!

I feel that’s good advice, however my clients deserve superior advice, and I don’t feel qualified to give it. Besides, Master Dustin took his pebbles when he left Seattle. 😉

Anyway, I really don’t want to host their blog. Besides, Zillow hosts their blog on Typepad, Move hosts their WordPress blog with Inmotion Hosting, and ShackPrices hosts their WordPress blog with Dreamhost, so I know I’m not alone in my thinking that they are better off out-sourcing to a shared hosting company that does blogs. So I have the following questions for the blogging masters.

Real Estate Blogging Questions

  1. What value / features do Real Estate targeted blogging platforms (Blogging Systems, Ubertor, etc) provide over general purpose alternatives (WordPress, Typepad, Community Server, dasBlog, subText)?
  2. ActiveRain or Real Town Blogs – To join or not to join, that’s my question…
  3. Are free blogging platforms, such as Blogger or Live Spaces, considered the blogging equivalent of using aol.com or hotmail.com for your e-mail?

General Blogging Questions

  1. What blogging platforms do you like or dislike and why? I know most of the masters recommend WordPress, so I’m more interested in hearing from folks who use something else and are happy with it.
  2. What company would you recommend or avoid for blog hosting? There’s a million of them out there, everybody uses somebody different.
  3. Do these companies provide tools that make it easy to setup? What about analytics, back-up, comment spam blocking, and everything else you want to do?
  4. Any good books or blog posts you’d recommend?

Geek Blogging Questions

  1. If I were crazy enough to host my own blog, what’s the best blog platform if you want to host on IIS/SQL Server/Windows?
  2. What blog platforms or plug-ins do you recommend for code syntax highlighting?
  3. Is there a way to get WordPress to not mangle, HTML/Javascript in a blog post?

Thanks for your help,

Your humble grasshopper

Is the mobile web the next big thing?

Posted on by
20

I was recently inspired by Joel Burslem’s, House Hunting On the Go blog posting. A lot of folks have voiced their desire for having MLS search tools designed for their mobile devices. Since I had some spare time last weekend, I went ahead and designed a mobile version of Real Property Associates’ web site. If you’re curious to see the final results, I encourage you to visit http://mobile.rpare.com and let me know what you think.

Ironically, as I was reading the post’s comments, I personally agreed most with Greg Tracy’s assertion (of Blue Roof fame), that his clients didn’t really care. The experience on a 3 GHz computer, with fast & reliable network connections and a 21″ LCD monitor is far more compelling, than the experience on a 200 Mhz Phone, with slow & intermittent network connections and 2″ LCD screen. I suspect a typical real estate consumer probably doesn’t care about mobile home search capabilities because they typically only look for homes for a few weeks every 5 or so years.

[photopress:mobile_zearch.jpg,full,alignright]Despite my reservations, I realize that real estate professionals are often very mobile and generally look for houses every week of the year. Since my customers are real estate professionals first and real estate consumers second, I suspect that my semi-pessimistic outlook is due to the fact that I’m not a mobile professional myself. That and the mobile web’s predecessor, WML, leaves a bad taste in my mouth like the New Coke did.

Mobile devices are a challenge to design for because the CPUs are slow, the browsers aren’t usually as powerful as their desktop equivalents (although Opera for Windows PocketPC currently smokes Internet Explorer Mobile), the screens are small, the network speeds can be glacial, and the user input mechanisms are slow and cumbersome. That said, having the whole MLS (w/ photos, static maps & Zillow Zestimates) and the office employee directory (w/ email addresses & photos) in your coat pocket is pretty dang slick, despite all the limitations.

Disclaimer: I designed this site for devices that support HTML and have screens that are 240 pixels wide. Since newer smart phones (Samsung Blackjack, Motorola Q, etc) and PocketPC’s have “big” screens and high speed networking, I decided that was the smallest device I wanted to support and still have the features I wanted. If you use a device with a smaller screen, you may be scrolling around more than you’d like.

So, what features would you like to see in your mobile real estate apps? Do you prefer text messaging apps (like Zillow Mobile) over real mobile web apps? What kinds of mobile devices do you plan on using to access mobile web apps? Is the future bigger, PDA like phones (like Windows Mobile devices, Blackberry, and the upcoming Apple iPhone) or smaller, more limited devices like the ubiquitous Motorola RAZR V3? Do think this trend will take consumers by storm or be restricted to professional use only? Would you pay extra for this technology if your MLS or IDX vendor offered it? Have you bought your .mobi domain names yet?

Adventures in digital listing land

Posted on by

Recently, one of my clients (Real Property Associates) asked me to automate the process of submitting (or advertising) their real estate listings and rental properties on Trulia, Google Base, and Craigslist. After implementing the feature, I thought sharing my experiences would a make an interesting blog post. (So here we are)…

[photopress:Feed_1_2.gif,full,alignright]As you may know, there are 2 ways of getting your listings on Trulia. The easiest is just to let Trulia crawl your site. Unfortunately this method doesn’t work very well since there are an infinite number of ways to present listings on a web page, and Trulia’s engineers haven’t been able to spend the requisite infinite amount of time required to handle all the cases. This isn’t a knock on Trulia, since Google Base doesn’t even attempt to do this, but just a reminder that there are a lot of things software just can’t do yet. If this method works for you, your lucky.

The recommended way is far more reliable. You merely need to host an XML file on your web site that contains the listings you want to promote, and then once day or so, Trulia’s web farm will request your file, parse it, and import onto their site for the whole world to see.

In my case, since I already export MLS searches via RSS (I knew writing that feature was a good idea), I merely had to spend a couple hours tweaking the output of my MLS RSS feed pages to match Trulia’s schema, register the URL on Trulia, and in 48 hours, we had listings on Trulia. And in 72 hours, I noticed referrals from Trulia was already generating about 4% of the site’s traffic!

By comparison, Google Base was easier in some respects and more cumbersome in others. The nice thing about the Google Base file format is that it is standard RSS. Or rather, it’s standard in the same way the Microsoft Word exports standard HTML. It’s RSS with a bunch of namespaced items for the custom attributes that Google Base uses for it’s Housing item type. Anyway, if you have already have an MLS RSS feed, tweaking the output to match Google’s schema is pretty straight forward. I should note that Google appeared to be more particular about the XML it gets than Trulia appeared to be, so you’ll probably be spending more time getting things onto Google Base.

The problem with Google Base isn’t creating the feed, it’s getting it up there. You see, Google Base does not download an URL like Trulia does, therefore you have to upload your data to the GooglePlex. There are 2 ways to upload your data, via a web browser or via ftp. I ended up writing a script on my server that would download a Google Base feed from my web server, and then upload it to Google in the middle of the night.

Automating Craigslist from a web page was an interesting challenge. They have a very aggressive anti-spamming policy, CAPTCHAs, have no supported way of submitting a post programatically, and the web browser’s cross domain security model certainly doesn’t make things easier. Fortunately, I found a way around everything but the CAPTCHA, but it required some IE only technology since Firefox on Windows still doesn’t support COM automation. (BTW, if any developers out there know if XUL applications on Firefox/Mozilla can accomplish everything IE based HTA’s can, drop me line. I’d love to talk with you)

After serving up listings to “the major players”, I decided to see what the beast from Redmond was up to. Turns out they want in on the action too (big surprise), and the 1-2 punch of Windows Live Expo and Live Product Upload appears to be Microsoft’s answer to both Craigslist and GoogleBase. I’ve signed up for the Live Product Upload Beta, and I’m looking forward to adding support for their service once they get their act together. It looks promising, but currently their upload service is more designed for merchants selling products, instead of real estate professionals selling homes.

Hopefully, the Live Product Upload team will correct this oversight and support multiple item types for upload. They better not wait too long to get that feature implemented, because I’ve recently discovered that Propsmart, Oodle, Edgeio, already have web feed programs in place for XML formatted listing submission. It looks like I’m going to be busy…

So, what sites do you use for listing promotion (or just reading classifieds)? Backpage.com looks like a promising up and comer. Anybody use postlets.com to assist in your online classified ad management? Anybody using Zillow, SubmitYourListings.com, or Ebay for listings promotion? Is paid advertising worth the expense when the free online classified marketplace is exploding?

RPA Zearch – Now with Turbo Zillow!

Posted on by
26

OK, I admit it. I got early access to the Zillow API. 🙂 And it’s pretty interesting stuff, it provides Zestimate values, comparable properties, Zestimates charts, and Zindex charts. Anyway, everybody knows I’ve done Zestimates before, but the charts are a new wrinkle I haven’t had the opprotunity to explore yet.

As some of you know, I’ve been working with Gordon & Jay of Real Property Associates (old site) to develop their new site (beta). Although the site is about a month away from going live, I thought I’d let the world know so they beta test my favorite new Zearch feature which I call “Turbo Zillow”.

So if you run a search for Eastside communities, below the map (sorry about the lack of pushpins folks – the server is having a bad geo-coding day), you’ll notice the new Zillow control. The control will populate with every city & zip code that was in your search results. (PS – Will the agent who entered a 00000 zip code into the MLS for MLS# 25147354, please fix it, don’t get me started). It will then let you plot a Zindex chart based off location, dollar/percent appreciation, and 1/5/10 year durations. So the control, looks like something like this…

 [photopress:TurboZillow.gif,full,centered]

This is really cool, because getting a new chart, is as simple and changing the drop downs to what your interested in, and the watching the chart change. Comparing city & zip codes median price histories has never been this easy on Zillow. The details page of a listing will also have a Zillow control that will show the chart of the listing, the zip, the city, the state, and the USA in the same way. Currently. the details version of the control appears to have a bug with getting the USA chart if Zillow can’t find the Zestimate. So if you see something that is way off. it could be my bug, or it could be Zillow’s bad Zestimate. Either way, I think charts & data visualization are the next big thing for MLS searches after everybody gets the AJAX maps out of their system.

On the Zillow site, to get this information, I have to click here for Bellevue, click back, click here for Redmond, and then back, and then click here for Kirkland. Why do they make getting Zindex charts so hard? I have to scroll to the bottom page, for everything and then click? Why can’t you do some Web 2.0 map magic instead of a sea of links (or just put the links it at the top of the page)?

OK, enough mini-flaming, I have to give credit were credit is due and I thank the crew at Zillow for having the guts to release an API to the public and having the courage to let me put it through it’s paces. Perhaps my experiments will inspire them to greater things, more APIs and a better UI for the Zindex pages? Until then, I’m using “Turbo Zillow” for my ZIndex fix.

Visit http://www.rpare.com/search.aspx, do your thing, have fun the fast lane my friends!