February 11, 2004 Consumer Reports WebWatch readers respond to our research report Booking and Bidding Sight Unseen: A Consumer's Guide to Opaque Travel Web Sites, published Dec. 8, 2003.
Dear Consumer Reports WebWatch,
I thought the article on "blind" travel sites was rudimentary and the authors seemed clueless as to the importance of bidding strategy. They seemed unaware, for example, of sites such as betterbidding.com or biddingfortravel.com which significantly enhance one's chances of getting a superlative deal on blind bidding sites, especially on hotels and rent-a-cars.
Armed with the knowledge of the amounts paid by other, successful Priceline bidders, it has become routine for me to obtain $31 rooms at Hiltons; $21 full-size Hertz cars; and I recently stayed at a four-star Marriott Renaissance in Paris, France for $65. Like any skill, your blind bidding will improve with a little practice.
Mike Stone
Ed.: Our report's intent was to examine the claims made by the blind-bidding sites themselves, and not to endorse other sites as tools to use them. However, we think it's important consumers are aware of such sites as BiddingforTravel.com and BetterBidding.com as aids to travel planning. Note that BiddingforTravel.com accepts advertising from hotels, for-profit reservation sites, and from Orbitz, which has a marketing partnership with Hotwire — which, in turn, has been acquired by IAC/InterActiveCorp (USA Interactive), the parent company of Expedia.
January 22, 2004 Dear Consumer Reports WebWatch,
We are confident it was not Consumer Reports WebWatch's intent to be misleading in its December 8, 2003, report, "Booking and Bidding in the Blind: An In-Depth Examination of Opaque Travel Web Sites."
However, the way the Sabre computer system was used to benchmark fares would lead some readers to inappropriate conclusions about the ability of "brick and mortar" travel agencies using the system to obtain the lowest fares for their travelers.
[We believe that the] methods used to search the Sabre global distribution system (GDS) [resulted in the report comparing] "apples to oranges." The report compares negotiated fares and rates from the online agencies, including Travelocity, Expedia, Orbitz and Priceline, to published fares and rates from the Sabre GDS. Offline agencies also negotiate low airfares and hotel and car rental rates with travel suppliers that are incorporated into the Sabre system for their individual use, in addition to the published fares and rates.
In late 2002 and 2003, special agreements were reached between Sabre Travel Network (the Sabre GDS company) and the "Big Six" global airlines based in the U.S., as well as a number of other smaller airlines, where the airlines provide Web fares for the Sabre system and offline travel agents in return for a discount on the fees they pay to distribute their products in the system. Most of these Web fares were loaded into the Sabre system only a few weeks before the testing was done in late August and early September. We believe it's likely that these Web fares were not included in the GDS search.
Offline travel agents, enabled by the Sabre GDS, have a full portfolio of tools and products to use to provide their customers with the lowest and best fares and rates for their travel experiences, in addition to personalized services. Any future comparisons that include the Sabre GDS should reflect this.
Sincerely,
Nanci Williams
Sabre Travel Network
Ed.: Consumer Reports WebWatch and the Consumer Reports Travel Letter have employed Sabre as a benchmark on several occasions during the past year and a half while testing and comparing airline, hotel and car rental prices available on the Web. That includes our most recent report December 8 on "opaque" sites such as Priceline and Hotwire. Consumer Reports WebWatch stands behind this research.
Report author William J. McGee responds:
Travel distribution has undergone many changes in recent years, and Consumer Reports WebWatch is aware of the ways that both online and offline travel agencies have adapted to those changes. However, Consumer Reports WebWatch's evaluations of the travel industry have always focused solely on products available 1) directly to consumers and 2) via the Internet. Consumer Reports WebWatch has never evaluated consumer products not available via the Internet. When results from Sabre or any other global distribution system (GDS) were included, the GDS's role in such testing has always been to serve as a benchmark, not a direct competitor of the integrated sites. Sabre was chosen most often because it is the largest GDS in North America.
Fares and rates from Sabre have always been obtained in real time by an outside consultant with many years' experience in travel distribution, but they were never obtained from a travel agency with negotiated rates. Furthermore, the consultant was never asked to obtain negotiated rates, but instead was asked to obtain base or "rack" rates (available to all Sabre-subscribed travel agencies that do not have negotiated rate agreements). Until recently, Consumer Reports WebWatch used Sabre for travel Web site testing so that fares from the regulated GDS environment could be used as benchmarks when comparing fares obtained on unregulated integrated travel Web sites. The rapid growth of "Web-only" fares led us to discontinue the practice of including Sabre in such testing. And since the U.S. Department of Transportation announced earlier this month that it will soon end regulation of GDSs, it would seem that fares and rates obtained from Sabre would provide
little value as benchmarks in the future.
However, when the decision was made to test opaque travel Web sites in the Summer of 2003, the inclusion of the bidding site Priceline presented Consumer Reports WebWatch with a unique challenge. Consumer Reports WebWatch decided to use Sabre as a benchmark once again — this time to obtain fares and rates in real time so that we could determine reasonable starting bids for Priceline products. Sabre was included in the testing solely for this reason.
It should be noted that despite Consumer Reports WebWatch's specific mandate to evaluate Web-based products, Consumer Reports WebWatch has noted repeatedly that travel agents have many ways of obtaining fares and rates for their clients. For example, the report on hotel booking sites released in April 2003 included the following: "An important point: Sabre's poor showing should not lead one to conclude that travel agencies cannot compete with independent travel Web sites. Concurrent with the growth of online travel sites, travel agencies have received many new booking tools as well. For example, Sabre and other GDSs now provide search engines for the Internet, and many travel agencies have secured exclusive deals with online travel Web sites as well as directly with travel suppliers. Savvy travel agents have found ways to find the lowest fares both on and off the Web."
December 28, 2003 Dear Consumer Reports WebWatch,
I cannot ascertain how you determined the Priceline hotel rates. Did you actually bid below Hotwire's rates? Did you receive identical hotels? Are you aware that the star ratings for Hotwire are inflated? That is, a four-star [hotel] on Hotwire is often a three-star [hotel] on Priceline, and so on. I have successfully bid dozens of times for Priceline hotel rooms, and I have always paid less than similar available Hotwire offerings. Do you understand how to rebid using "free" rebids — that is, rebidding by adding zones that have lower star ratings than the desired zone? Using this strategy helps you get closer to the minimum successful bid.
Thank you,
Edna Kelly
Ed.: Producing a detailed bidding strategy guide for Priceline and Hotwire was beyond the scope of the report, and outside Consumer Reports WebWatch's mission. Our report sought to investigate, through extensive performance testing, the credibility of marketing claims made by opaque sites. Our conclusions are that while deals certainly can be had on Priceline and Hotwire, for the average consumer, the cons of using them may outweigh the pros, and consumers should shop among a broad variety of sites for the best deal. We realize "best deal" may not always mean, to everyone, the lowest price, but we believe most consumers consider price a major criterion of choice. We used Sabre, the largest global distribution system (GDS) in North America, so our testers would get ballpark bidding figures in real-time. In addition, we also used the three largest non-opaque, integrated travel Web sites — Expedia, Orbitz, and Travelocity — as benchmarks for opaque fares and rates. Finally, a discount hotel-booking site, Quikbook, was included as another benchmark, but for hotel tests only.
December 12, 2003 Mr. Beau Brendler
Director, Consumer Reports WebWatch
Dear Mr. Brendler,
As someone who has built a career on consumer research, I have long admired Consumer Reports for its dedication to reliable and fair testing methodology. That's why I was seriously disappointed with Consumer Reports magazine's story on "blind" travel sites in the January 2004 issue ("A clear view of 'blind' travel sites").
The report [based on a Consumer Reports WebWatch research project] sought to determine which discount travel Web site consistently offers the lowest prices — Hotwire.com or Priceline.com. In the study, you purchased 135 airline tickets, hotel rooms and rental car reservations in order to compare prices and determine which discount Web site offers lower fares and rates.
Your stated intention was to compare the sites' pricing under
"real-time, apples-to-apples conditions." I agree that an apples-to-apples comparison is imperative when comparing airline fares or car rental dates, because as most consumers know, a change in the day you fly or the airport you use can make an enormous difference in price.
This is why I was extremely disappointed to read that you did not compare the two companies' prices "apples-to-apples." Instead, you state that with 40 percent of your airfare comparisons, and 43 percent of your rental car comparisons, you searched and purchased a completely different itinerary on the two sites. In the case of airfares, you had to change the date of travel, the airport, or both in order for Priceline.com to accept your bid. As a result, in four out of ten examples you compared reservations for travel on completely different
days and/or including completely different locations.
Is this a fair comparison? Not according to your report's own author, William McGee. On December 10, 2003, Mr. McGee wrote a column in USAToday stating that for airfares, "not all travel dates are equal," and prices can vary widely by date. Of hotel rooms, he writes that "changing an itinerary by a day or two can save a lot."
Given Mr. McGee's own advice to travelers, it is baffling to me that he would then compare different travel itineraries on different sites with the intent to conclude one source is cheaper. Any seasoned or even occasional traveler knows prices can vary enormously from day to day, location to location. It is thus extremely disappointing that a respected and trusted publication like Consumer Reports could make such a glaring methodological error when conducting an extensive (and expensive) study such as this.
Regards,
Cheryl Law
Senior Director of Consumer Research
Hotwire.com
Editor's Note: Consumer Reports WebWatch stands behind the report.
Author William J. McGee responds:
Ms. Law is correct in noting that a difference in airports or travel dates (or in some cases, even time of day) can produce an enormous difference in the fares and rates consumers obtain when shopping online for airline seats, hotel rooms, and rental cars. This is why Consumer Reports WebWatch has spent hundreds of hours developing and perfecting its testing methodologies, to ensure such queries are conducted under
real-time conditions using identical itineraries.
The testing Consumer Reports WebWatch has conducted since 2002 builds on procedures developed by Consumer Reports Travel Letter beginning in 2000, after consultation with outside industry consultants and Consumers Union experts, specifically in the fields of research, survey and statistical methods. These methodologies have been continually honed and improved. The cornerstone of this testing has always been the simultaneous search for fares and rates from multiple Web sites in real time.
Consumer Reports WebWatch does not discuss specific methodologies or testing dates with the travel Web sites it examines, nor does it communicate with these companies during testing periods. However, several months prior to the crafting of these tests, I met with executives from Hotwire and Priceline to
discuss the particular testing challenges presented by opaque
Web sites and to solicit feedback. I stated that Consumer Reports WebWatch’s standard testing procedures would need to be amended.
This examination of opaque Web sites was by far the most challenging, time-consuming, and expensive travel project undertaken by Consumer Reports WebWatch. As noted in the research report, it required the purchase of nearly $38,000 in travel products from Hotwire and Priceline. It also required key changes in standard testing methodologies.
The first change was that an outside consultant obtained fares and rates from Sabre, a leading global distribution system used by travel agencies, and supplied them to Consumer Reports WebWatch testers so that benchmarks could be established prior to placing bids on the Priceline site. Then, other testers simultaneously queried fares and rates for identical itineraries in Hotwire and the other travel sites.
The second change was specific to Priceline’s booking architecture. As Ms. Law noted, Priceline often requires a user to make key changes to an itinerary prior to booking, and we found this to be the case with about four out of every ten airline and car rental bookings. The notification time to process these changes varied considerably throughout the testing. It was clear that once Priceline’s itinerary changes were accommodated, it would be impossible to communicate those changes to other Consumer Reports WebWatch testers and maintain real-time conditions.
Therefore, during the preliminary testing phase of the project, it was determined that a) Priceline’s fares and rates would be included with the itinerary changes and b) any itinerary changes would be prominently displayed in text and figures in Consumer Reports WebWatch's research report.
The conclusion Consumer Reports WebWatch reached: Priceline’s lower fares and rates require significant trade-offs consumers should examine prior to booking. This key finding of the project was stated repeatedly in the report's abstract, executive summary, and the report itself.
October 24, 2003 Your recent study comparing car rental Web sites was very insightful, but it contained inaccurate statements about the ability to find taxes and fees for cars displayed on Travelocity. Your report said taxes and fees were "NOT CALCULATED" on Travelocity, and you made several other statements indicating Travelocity rarely, if ever, showed customers the taxes and fees on their car rental.
For more than a year, Travelocity has had Total Pricing for Cars, which gives customers the daily rate, adds in the taxes and fees on the car and shows the TotalPriceTM. Only Travelocity shows customers both the daily rates and the TotalPriceTM across a range of car types and car companies. We even break out the specific taxes and fees for every car available anywhere in the world…every fee is there, right down to the Cactus League Spring Training Tax in Tucson, Ariz.
No one has invested as much time, money and energy as Travelocity to ensure customers have the information they need so there are no surprises at the rental counter. Your report was incorrect, it overlooked our team's tremendous investment for our customers and the strong product features that put us head and shoulders above our competitors, and we believe it deserves your consideration for a correction.
Thank you.
Al Comeaux
Vice President, Public Relations
Travelocity
Editor's note: Indeed, Travelocity's calculations of taxes, fees and surcharges for car rentals should rank it at the top in that category of evaluation. Results in Consumer Reports WebWatch's report are derived from real-time, "apples-to-apples" testing and queries, and though Travelocity's listing of taxes, fees and surcharges do not appear on the results pages the testers used, the site calculates and displays them a click away from that page. Consumer Reports WebWatch has corrected the listing in the report; the full correction appears here.
November 4, 2002 This refers to our phone conversation on Thursday, Oct. 31. We discussed then the Wall Street Journal article on medical Web sites on Oct. 29, 2002, which, according to you, correctly characterized the position stated by your organization Consumer Reports WebWatch, as to our site. Specifically, the article stated, "...Health Bulletin, which [our] experts ranked 10th, didn't include author credentials or article citations."
We should add that we only report on developments in health matters referring to foods and nutrients. We do not insert our opinions, positively or negatively. We only report the news from recognized medical and scientific publications. There is some antagonism among medical practitioners who have a lag time before application of from 5 to 10 years after the discoveries are made, which we do point out.
Also discoveries about foods and nutrients often are ignored due to medical practice's preference for drugs and invasive procedures. This may explain the negative, even false assessment of our site by them.
As to their assessment, every article that has ever appeared on our site references the authors, the medical or scientific journal from which it has been extracted, the journal's date, and the study author's affiliation. Affiliations are medical schools, hospitals, NIH employment, etc.
We would appreciate your making the necessary corrections.
Most cordially,
William A. Kent
www.healthbulletin.org
Ed.: This letter is in response to the Consumer Reports WebWatch research report, Experts vs. Online Consumers: A Comparative Credible Study of Health and Finance Web Sites, in which 15 experts from the health and finance fields were asked to assess the credibility of Web sites in their respective fields. Healthbulletin.org was one of the sites included in this study. Consumer Reports WebWatch stands by its research and reporting. We cannot control the content of what other news organizations write about our work, but we also believe the Wall Street Journal's reporting was fair and accurate.
