Leave it to Marlow…

…to come up with the most provocative (and interesting!) commentary on Redfin’s press release from today

Some more opinions from around the web:

68 thoughts on “Leave it to Marlow…”

1. Marlow flat out indicted their statistical analysis in the most basic of analyses.

2. LOL, that is so funny. The real reason is not in any of those explanations. I’d have to run the stats to prove my hypothesis…after I do the post in answer to Brian’s question.

3. I didn’t want to include my back-of-the-envelope calculations in the post because I felt like it was rubbing salt in Redfin’s wounds… but play with me here…

If Redfin claims that their buyers saved \$4,474 by shaving slightly less than 1% off the typical transaction, then a simple math equation tells us that if in fact the average buyer OVERPAYED by over 17% (99.32% – 81.96% = 17.37%), then using Redfin cost the average buyer almost \$86,000! Wow!

I’m more than happy to learn that I’m butchering the numbers, but it seems pretty straight-forward to me…

4. They keep telling me they used “a different metric”. Redfin says the NWMLS compared median list price to median sale price, where they “compared that ratio for each and every deal and gave them equal weight.”

I don’t know what that means.

Does that mean the NWMLS chart showing the Median List Price and the Median Sale Price is wrong? Or were they measuring something completely different?

Unfortunately, it’s very difficult to replicate their results and that is compounded by the remoteness of the data, that is not accessible to non-members. I do have access to the statistics, and I’m still not seeing what they see.

If number claims like that can’t be proven, checked or duplicated, it seems irresponsible to include them in a press release as fact.

And, of course, irresponsible of journalists to re-publish these press releases as fact, without doing further checking and investigation.

5. Marlow,

Think about it. Most homes that sell at better than asking are the best homes that sell within hours. The Redfin model generally doesn’t operate that way or that quickly. It is more likely a Redfin buyer will be in the lag market, buying the property that sells for less than asking, with longer days on market.

I think the stats are correct. But it isn’t about “negotiating”, it’s about the quality of the property. I’ll have to check the stats, but I’ll bet it’s because Redfin buyers aren’t winning the bidding wars on the best homes or grabbing the best ones the minute they hit the market…for the most part.

It’s more like getting the leftovers at a discount, than negotiating.

6. These guy really are great with their PR.

My take on this… Quantifying a broker’s/agent’s negotiating skill is difficult to do because it is so heavily affected by the client. In Redfin’s case (and TerritoryRE’s) our clients tend to fit a certain prototype and from my experience they negotiate much more aggressively than the “average” client. On top of that, Redfin presumably did the majority of their deals in the latter part of the year when the market was in its downward spiral this could have biased the results in their favor.

Are the statistics enough to really prove they are better negotiators? I doubt it. Is it good enough to convey that impression to the press and the general public? Probably.

Either way after you throw in the rebate there’s no way their clients didn’t get a better deal.

For comparisons sake on transactions where the buyer has been represented by a TerritoryRE Guide the average ratio between List Price and Selling Price is 93%. Average for our market is 96%. Add an average rebate of 1.8% and we are looking at a difference in purchase price of almost 5% vs working with the average agent.

7. We acknowledged in the initial study that we may attract more data-driven, deal-oriented clients.

But there is no reason to think that Redfin customers don’t pursue attractive properties; last I checked, our customers buy properties ever-so-slightly more expensive than the average.

We compete in most deals; in those deals, any agent’s ability to negotiate price is limited. What an agent can do is advise a client to walk from a deal that has become a bad deal. This is what an agent does when representing a friend instead of a client. We believe we are more likely to do this because our agents aren’t paid to close. When someone really wants a house, we do everything we can to make sure he or she gets it.

In deals where there is not a competing offer, an agent has the latitude to negotiate a lower price. We believe our agents are more aggressive in negotiating that price, again because the agents are not paid a commission.

Finally, we will publish later today an analysis showing that we still have an advantage for the final three months of the study period, so arguments attributing our advantage to a ramp-up in the midst of a market slowdown don’t seem true.

Can we at least admit that what kind of agent Redfin recruits and how we pay our agents might be a contributing factor in our ability to negotiate a lower price?

What we find shocking is that RCG — Seattle’s greatest real estate blog — would re-publish Marlow’s claim that homes tend to sell for 81% of list price. This is a misrepresentation of an NWMLS report. No responsible Seattle-area broker believes this could be true. Our response to Marlow’s post is below; we are still trying to reach the NWMLS.

We are happy to send our data set to any broker, agent or journalist; for journalists, we will have to seek NWMLS permission but we are confident we can get this. Kevin Boer, Greg Swann and Marlow Harris all have the data; all are prolific bloggers who would prefer that Redfin were wrong. We would love to send anyone at RCG the data. All of you are also welcome to access the data directly from the NWMLS.

For the sake of your own reputation, you have an obligation to assess the data available to you before attacking its credibility.

*~*~*~*Response on Marlow’s Blog*~*~*~*
Marlow, I am not sure you believe your own argument. We all know that a 19% discount to listing price is not the norm for any brokerage; for every \$500,000 home sold at list price, another would have to sell at nearly \$200,000 below list price.

It may be that the NWMLS delta is the result of comparing two data sets: one data set would be homes listed between January 1, 2006 and December 31, 2006; the other data set would be homes that closed between January 1, 2006 and December 31, 2006.

So it may be that homes that were listed on December 31, 2006 listed at a higher price than those that closed on January 1, 2006, with the 19% difference simply reflecting appreciation. It may be that the NWMLS median list price includes homes that never sold because they were overpriced.

The truth is, we don’t know how the NWMLS calculated this 19% difference. The NWMLS did not publish its methodology. Rather than speculating any further, we’ll do our best to find out, and we’ll let you know when we do.

We do know how we calculated our numbers. We evaluated only pairs of data: the list price of a property that closed, and the final price of that property. And we evaluated every single condominium and single-family home that closed in King County between February 6, 2006 and February 5, 2007. Surely we all agree that this is the most precise way to assess whether a brokerage tends to negotiate above or below list price on a particular property for its buyers.

For these reasons, I am not sure if you are actually standing behind the NWMLS numbers you cite as an accurate reflection of agents’ negotiating ability. But we are standing behind ours.

If your goal is to validate or invalidate our findings, you can simply review the data for each transaction. We provided instructions for replicating the data set from the NWMLS, so you do not have to trust us. At your request, we’ve also sent you the raw data for every closed transaction between February 6, 2006 and February 5, 2007, so you don’t have to go to any trouble.

But attacking our integrity creates an obligation. If your findings are different than ours, we are sure you will publish our mistake. If your findings are the same as ours, we hope that you will publish that result as well. We operate under the same obligation: if we can find out how the NWMLS calculated its 19% difference, we will explain the discrepancy.

But let’s not fling mud at each other just to obscure the truth. In real estate, all records are public, and the facts can readily be established.

8. Glenn,

Thanks for stopping by… I’m surprised you are shocked people would question your numbers… Of course they are going to question your data (that’s what bloggers do!), but we, on RCG, always welcome your response as I think it makes us all better.

I definitely hope that Marlow dives deeper into the data and I look forward to reading her analysis.

9. Dustin, it’s totally legit to question why the numbers are what they are, and whether they really mean what we say they mean, as Ardell and Greg have done. We think they make valid points and we love the debate.

But to say the numbers themselves are wrong when they can be so easily checked from a legal, verifiable source — it’s like arguing whether it’s raining outside. Why not just open a window? We are sending you the data. If it’s wrong, you should say so. If it’s right, you should say so too.

Aside from when you’re challenging a fact, we don’t expect you to be factual; let the fur fly. But when you publicized the statement that Redfin customers overpay on average by \$84,000 for a property, did you believe it was even conceivable that it could be true?

The world expects better from RCG.

10. Glenn,

My gut tells me to just turn the question around on you… Do you really believe that Redfin buyers saved “\$4,474” on average? I’m not under the impression that your study was meant to further real estate knowledge in general, but rather, you answered questions that furthered your agenda… By the way, I don’t think there is anything wrong with that type of press release, but don’t put yourself on a pedestal as doing unbiased research.

Even the folks at the Freakonomics blog (not exactly friends of the traditional agent) don’t buy that the “negotiating skills” of Redfin agents explains the differences as you so eloquently claim in your blog post. In other words, if you’re going to use this data to make claims that are clearly unwarranted, then don’t be surprised if the rest of us ask unwarranted “what if” questions.

11. I’m just coming up to speed on all of this….

If Marlow’s numbers are correct, what does that say for the collective valuation aptitude of listing agents who, on average, overpriced properties by almost 23%.

At last report, many real estate bloggers were horrified with Zillow’s lack of accuracy but rarely were the errors claimed to be 23%.

Hmmm….

12. I just did a study from the bankrate.com lenders that confirmed I am less expensive and more effective than 86% of the big banks in San Diego. I’ll probably be posting my findings in a press release Thursday or Friday.

I pray Rhonda Porter doesn’t dig into the data I used.

13. Russ, that seems like a silly assertion – agents (should) price properties to get the best deal for their clients. I can think of a million reasons why an rational, successful agent would price a property above (or below) market price in order to help their client get the best deal, can’t you?

14. Galen

23%???

Let’s just say that most agents would readily admit that the “average” house that is priced 23% above its eventual sales price is WAY overpriced, at least if you want to sell it within a reasonable time frame.

Russ

15. Dustin, will you answer the question: did you ever believe that what you said was true? As for me, the answer is yes.

This isn’t mere stubbornness. We both know that, at other times, when I have regretted a combative quote, that I have apologized for my tone. If I said anything that actually turned out to be untrue, I would retract it. This is what I think you should do.

It isn’t a question of belief that our buyers on average paid \$4,474 less than customers of other brokerages, as compared to the list price of a home; it is a fact. Not only do I believe in it, I am not sure you or I could choose not to believe in it, unless there are countervailing facts.

If the data we have sent you is in any way biased, you are obligated at this point to say how it is biased, or to produce an unbiased result. This is what we are doing with the NWMLS report that Marlow cited.

The claims that we make from the NWMLS data are disputable but not “clearly unwarranted.

16. PS, when I said that the claims we make from the NWMLS data are disputable, I meant that only the claims are disputable — Redfin agents negotiate better — not the data itself, which shows the \$4,474 savings.

17. Glenn: “it’s totally legit to question why the numbers are what they are, and whether they really mean what we say they mean, as Ardell and Greg have done. We think they make valid points and we love the debate.”

I did a quick spot check this morning and almost fell off my chair. I tested your real numbers against the one office I know is the worst at negotiating. Your numbers were spot on. Identical. Amazingly accurate. But they are known to be the worst negotiators in the market, and we so love getting offers from them 🙂

The reason they are the worst negotiators however is not because they want the extra pittance of commission, it’s because they want to be done with showing. If a buyer comes to you with house in hand, not much skin off your nose if they don’t get that house. But when an agent has dragged all over town with someone for three months and the buyer finally finds something they like, the agent wants to close it out. Best excuse I had from an agent this year for bringing a fullprice offer “I’m leaving for vacation and want to get this done before I go.” LOVED that one…so did my sellers.

I’ll run some more stats, but so far, yes you are better than the worst by the exact degree you claim. Let’s see how you stack up against the “not worst”. You know I’ll post it as I see it and as it IS. Now stop blaming me for Dustin’s math 🙂 Say you are surprised at Dustin…not “RCG”. Thanks.

18. Glenn,
You are trying very hard to point out that the numbers don’t lie. Ok.
According to the NWMLS, there were 69,407 closed transactions for residential and condo homes in 2006 (in King, Pierce, and Snohomish Counties). In the same tri-county area, Redfin was a part (listing or selling) of 224 transactions. That equates to 0.3% of the sales.

You may be perfectly correct in your interpretation of the data that an average Redfin buyer paid \$4,474 less, but you are using 224 transactions out of 69,407 to make your point. Whatever the point that is … it is …in my opinion…so statistically insignificant to have any weight.

Furthermore, I also have an issue of lumping all non-Redfin agents together and comparing it to Redfin agents. So I did some number pulling from the NWMLS. I wanted to see how Redfin agents on the selling (buyer’s) side compared directly to other brands. I chose:
Century 21 North Homes
Windermere Real Estate Co.
John L. Scott Inc.
Coldwell Banker Bain
ReMax Metro Realty

I pulled closed Residential (no condos) sales with your time frame of 2/6/06 to 2/5/07. I did not limit it to King County. When pulling data I checked the option to include all company sales ..not just the office ID I input since one company can own several offices. Here’s what I got.

Median Sale Price to Median List Price comparison:
100.97% – Redfin – based on 149 closed sales
100.31% – Century 21 North Homes based on 575 closed sales
99.72% – Windermere Real Estate Co based on 1032 closed sales
100.00% – John L. Scott Inc. based on 5274 closed sales
99.43% – Coldwell Banker Bain based on 4093 closed sales
100.19% – ReMax Metro Realty based on 940 closed sales

Let’s translate that into dollars. Based on the same data, I averaged the Median selling price of all 5 companies and got \$412,850. Round up to \$415k for easier number to use.
Century 21 North Homes buyer pays \$416,306
Windermere Real Estate Co. buyer pays \$413,828
John L. Scott Inc. buyer pays \$415,000
Coldwell Banker Bain buyer pays \$412,646
ReMax Metro Realty buyer pays \$415,776

….continuing the number analysis..
I broke down the Sale Price to List Price percentage based on Cumulative Days on Market. With the same number of closed sales per company ….Homes on the market:

Between 0-30 days, Redfin did better than the rest at 99.86% while the others ranged from 100.07% to 100.90%.

Between 31-60, Redfin’s 98% was 2nd to Windermere’s 97.58% with the highest at 99.17%.

Between 61-90, Redfin’s 98.74% was 2nd to LAST with only Century 21 North Home’s 98.89% worse than it. Best was Windermere’s 96.44%.

Between 91-120, Redfin’s 99.08% was again 2nd to LAST with only Century 21 North Home’s 99.32% worse than it. Best was Windermere’s 97.48%.

From 120+ days, Redfin stole the show with 93.45%. The others ranged between 96.13% and 98.04%.

Wow, ..93.45% ..something is way out of line. So I looked at the 149 closed homes Redfin had and filtered out the ones that were 120+ CDOM. There is an element of error here that I must mention …CDOM is based on the listing agent accurately entering when the house actually went pending. Those in the business knows…this does not always happen. Which may explain when I actually looked at the individual listings ..there were 18 homes showing 120+ days in the CDOM with an average SP/LP of 95.93%. Now…of these 18, four of them were \$1 million + homes. For homes that high in price, huge price reductions in final sale price is not uncommon. So I simplified it. I eliminated all four of them. The remaining 14 homes now had an average SP/LP of 97.42%.

So… looks like brand new listings ..Redfin agents did slightly better. However, as the listing ages, ..the Redfin agents did progressively worse.

I’d be happy to share my spreadsheet that I have all this data in.

19. As someone who lives inside the real estate numbers I thought I would add my 2 cents to this post.

I have reviewed Redfin’s numbers and their numbers match their statement. (Don’t get excited yet, Glen)

Here is my opinion on the numbers that Redfin presented.

Redfin is comparing 126 residential sales to the 27,000 + residential sales that occurred in the same time frame (this is a very small sampling). If you look at all the offices that had a sale during the time frame that Redfin is using you will find that 951 offices had a selling side transaction. If you ordered those 951 offices by list price /sale price ratio you would see that Redfin falls in at 351 from the top. That means that approximately 1/3 of all companies that sold properties during that time frame had a better list price to sale price ratio for their buyers then Redfin did.

As far as the NWMLS numbers that Marlow quoted she is 100% correct it what she stated. I do believe there is a missing number in the report that the NWMLS puts out which would give a closer representation of list price to sale price ratio. That would be the medial list price of the homes that sold. By comparing the Median list price of sold homes to the median sale price you would get the actual list price to sale price ratio.

20. I have to make a correction…my eyes apparently got crossed.
for year 2006, a redfin agent took part in 224 transactions in the tri-county area (King, Pierce, Snohomish). That part is still correct. However the total number of transactions was 69,048 …not what I said in my post above. …this changes the percent of redfin transactions from 0.3227% (which I simply rounded to 0.3% to the new number of 0.3244%. …..my apologies for this blunder, but the percentage doesn’t change.

Which actually goes to show how insignificant 224 transactions is. I misquoted or misread the number of transactions by 359. And it changed increased Redfin’s transaction percentage of the whole by 0.0017%. So …again, the point being…how can any analysis you draw from that small of a subset of transactions carry any weight?
This was only from the Tri-county area I outlined earlier. How will the numbers look if you included the entire region the NWMLS covers? Shouldn’t matter that Redfin only has a focus on King County …Redfin is an online brokerage ..location shouldn’t matter right?

Allen Benson’s analysis is what I was trying to get at. If Redfin wants to compare itself against the competition, then it would make more sense to compare it against individual companies.

21. Thanks to everyone who is diving into this data in much more detail than myself. I’m honestly too busy to do my own analysis at the moment, so I’m glad that others are doing this work.

Glenn,

Far from retracting my earlier comments, I’ll try to clarify my position. You have said (numerous times) that Redfin agents are better at negotiating than a traditional agent. You give a bunch of reasons, but the most inflammatory being that the traditional agents have a disincentive to make a good offer.

The connection you make between list price and sale price can be explained by numerous reasons (you list many good ones in your most recent post, although I would have highlighted the concept that Redfin clients tend to be people who are more price conscious and less concerned about the ethics of using the time of other real estate agents in order to save themselves a buck), but your explanation that Redfin agents are better negotiators is the most illogical argument of the bunch! Redfin agents are not paid to negotiate in any meaningful way. They are paid to fill in the blank on an offer that a client “pre-fills” on an internet form. To act like this is negotiation is disingenuous at best…

Hence, the numbers to me don’t really matter (i.e. I don’t care if it is \$4K one way or \$86K the other), your explanation of “negotiation skills” doesn’t make any sense to me and reeks of being an attempt to seek publicity. On that note, congratulations! As you note in your recent blog post, your efforts were not wasted!

22. My data is in. No surprises.

In hot markets, more than half of the Redfin transactions were at or higher than asking price. Pretty much the same as everyone else’s stats.

Those that were sold significantly under asking price were either on market for a very long time, or were ones where the seller was having an exceptionally difficult time selling for one reason or another. In other words, properties that other agents were not bringing offers to, so selling at less than asking, if property is on market for four months, was not exactly a huge negotiating boon.

A couple of “on market long time” that sold way under asking, became the “per transaction average” which gives a less than accurate representation of what really transpired.

Having read Glenn’s whole piece I can clearly understand his need to rebut the Dalton claim, but I am saddened at the same time. Two wrongs don’t make a right. Broad accusations answered with similarly erroneous broad accusations, may be the only way to save face, but I’m disappointed nonetheless. I don’t think Glenn should have been on stage with Dalton in the first place. Now it’s come to this, and no good can come of it.

I hope that this trend of pulling data to cast invalid aspersions is done with, for both sides of this battle. Redfin is better than that, Glenn. Don’t let them take you down to their level…again. Stay on the high road. We don’t need battles…we need advancements. The EBAs got dragged down that road before you. I have higher hopes for you and Buyer Agency in general. Don’t let them drag you in the direction the EBAs took and don’t play in the mud with them. They win that game…that’s why they started it in the first place.

Your value is in the fact that you are not them…don’t play their game. Stay true to yourself…because “yourself” is better, whether it shows in the profits & stats or not.

23. Ardell-

I agree. A hearty British parliamentary ‘here, here!’ At he end of the day, Redfin’s satisfied clients have more money in their wallets. Wouldn’t people agree that is the bigger story?

24. The bigger story is that Redfin represents a sense of dignity for the buyer client heretofore not afforded buyers, by and large. It’s not about the money so much really. It is that they recognize the fact that the buyer does in fact pay his buyer agent. Regardless of the amount, the recognition of that fact is why I support them.

25. >Regardless of the amount, the recognition of that fact is why I support them.

Agreed. Good point, Ardell.

If redfin.com consistently advertises that it gives back ,67% of what they earn as a buyers commission and most real estate agents average a “giveback” of say…5-10% of what they earn..doesn’t that make redfin.com a discounter?

and what’s wrong with being a discounter? I say embrace your model and call it what it is! Clippers owner, Donald Sterling, made a veritable fortune as a dollar store owner. Sam Walton built an empire on his model.

26. I do agree Brian that it is strange that Redfin doesn’t want to be called a discounter. I don’t know why the term offends them.

27. Ardell said;

I don’t know why the term offends them.

Maybe because there hasn’t been a discounter that has been able to put a dent into a market and stay in business. In order to keep the doors open you have to make a profit. Even Glenn himself continues to say that under the current model they haven’t been able to break even.

Consumers want to know that the company they are choosing to do business with can actually stay in business. But there are some consumers that just want the discount so bad they aren’t thinking about the what ifs.

Lets take this hypothetical:

A new real estate company goes into operation they represent a buyer and during the transaction there are errors made. The buyer a few months later seeks legal advice and their attorney feels their representation failed them and recommends they litigate. When they go back to sue the broker and the company is out of business that discount doesn’t look that appealing.

This is not an unfounded hypothetical maybe Craig or Russ can chime in on this.

28. Ardell: we have struggled to find a way to compete while still communicating respectfully, and we always appreciate your guidance in this regard. Our intent here was not to publish any misleading, erroneous or invalid data, or to attack others. We honestly believe the data shows that Redfin customers tended to save a few thousand dollars on top of the commission refund, and that it was the kind of thing any brokerage would want the world to know about. At some point it felt foolish arguing every day over whether customers lost their refund at the negotiating table when the data showing otherwise was in the public record.

Dustin, my goodness, are you standing by the \$86,000 claim or just shifting ground? We have always acknowledged that our customers are deal-savvy but on what basis do you argue that Redfin agents don’t negotiate? The idea that commissions can have the affect of encouraging buyer’s agents to close, regardless of price, may be upsetting, but do you think it’s true? Many factors may motivate buyer’s agents to represent clients’ best interests; the commission paid by the seller is just not one of them.

James, thank you for your analysis. A colleague, Sasha Aickin, is responding to it. I’ll dive in if I have anything to add…

29. Glenn,

I agree the entire argument is silly. It was silly when Dalton started it, and he looked foolish doing it, and you should have left it at that in my opinion.

No one is buying that Redfin agents are better at negotiating than most or even many of the other agents. You can’t take one client who “saved” \$20,000, and spread it over 5 clients and say they each saved \$4,000. That is just not a credible use of data. It’s false info, and gives your “competition” a chance to throw their poison darts at you.

Better to say 2 out of 5 people were able to negotiate below list price, than to pretend that all five participated in the other 2 client’s savings. That’s just false advertising.

30. Glenn,

I’m surprised you want to continue this conversation, but here goes…

1st off… I never made the \$86K claim. I said if Marlow’s numbers are correct, then \$86K is the logical fallout. I put a number to Marlow’s speculation… That’s all I did. (Notice I did use the “if” word!)

After that, I don’t follow you at all… I never claimed that the commission encourages a faster or cheaper closing.

What I claim is that you making some outrageous and untrue claims based a very unscientific-collected data source. Before you can make claims that Redfin negotiated on our buyer’s behalf for XYZ number of dollars over other agents you NEED to understand if we are talking about the same type of buyers.

Based on the nature of the Redfin buyers that have contacted my wife trying to get a “free” showing, I feel comfortable saying that Redfin buyers tend to be young, tech-savvy, and cheap. (I feel pretty comfortable saying we probably get more Redfin buyer’s contacting us for a showing than anyone else on the internet!) It is ridiculous to think that you can compare a Redfin buyer’s habits (and they way that they self-negotiate using internet tools) to the buying habits of the general population in Seattle.

Again, it has nothing to do with the commission (being intuitively in the “discount” world, I think you are stuck on that). The difference has to do with the type of buyer that is using your service. You are not taking into account (hence: the unscientific data collection on your part) that Redfin buyers do not have the same needs or demands as the general internet buying population. (By the way, money is only one of the needs and/or demands in a typical transaction as buyers have different time, emotional, aesthetic, etc. requirements that can all effect the eventual purchase price)

I could try to speculate on the type of buyer (beyond young, tech-savvy and cheap) that Redfin attracts, but I won’t. The research report that Redfin released is probably right-on-target (Kudos to the hard work of your analytical-staff). It is your claim, Glenn, that Redfin agent’s negotiating skills account for the \$4K difference that irks me (plus I think somewhere I read that you claimed this number was a scientifically valid fact! OUCH, you clearly don’t understand statistics either!). It is obvious to me that you do not understand the appropriate conclusions that can be drawn from the work of your researchers.

31. Dustin, agreed that money is only one of the needs in a transaction.

I wrote the original report, which plainly stated that our customers are one reason we have competed as well as we have. We continue to feel that it would be unreasonable to conclude that how we pay our agents is not another reason.

You will note that no one at Redfin has ever said our agents are more skilled or capable than other agents, though obviously we are very proud of the team we have here, only that they got better results than their counterparts.

If we could have figured out a scientific way for us to eliminate customer self-selection as a factor, we would have. If you know of one, please let us know.

The only statistical claim we have made is that it would be extremely unlikely (a p-value of .03) to see the results in the NWMLS data that we did unless Redfin customers did in fact tend to get a better deal as compared to list price.