Spa Benchmarking: Reliable Research Versus Press Release Research | By Judy Singer

HFD conducted a study of companies that are performing spa benchmarking research. Our goal was to determine if the studies are “real research” or “press-release” research, e.g., are they following the best practices for valid, reliable research or are they providing information that get people’s attention. There are lots of established, well-known as well as new research and consulting companies that are conducting various types of spa benchmarking studies, but their reputation and expertise in one area does not mean they have the experience, industry knowledge, objectivity, etc. to conduct reliable spa research.

HFD contacted several reputable and respected companies that have received a substantial degree of media attention for their spa research studies. We read almost every press release related to companies that are conducting spa benchmarking studies. We carefully examined the actual research reports and presentations. We then contacted the research companies to hear first-hand what they are doing and how they are doing it so that we could have as complete an understanding as possible. We gave each company the opportunity to explain their approach so that the end-users (asset managers, directors of finance, general managers, developers, investors, consultants, spa operators, etc.) could read spa research with a “critical eye” which would help them understand and determine the nuances attached to the various ways in which information is gathered, analyzed and reported. We want the end-users to be able to tell if the study is a “real” study verses a “marketing” study, e.g., are the findings reliable, valid, useful, representative of the industry, etc. In the process of analyzing what the companies do and how they do it, it was obvious that there is lots of room for error and lots of ways data are compromised in which case findings are unreliable yet they are still “reportable.”

Based on all the information we gathered and analyzed, HFD selected three companies that represent a good cross-section of the reality verses marketability of what is happening in the much needed area of reliable spa benchmarking. These companies approach spa benchmarking in very different ways and, as a result, their reports differ significantly in terms of data integrity and “usefulness.”

Who's Doing Research & What are They Doing?

• Julie Garrow, Managing Director, Intelligent Spas
  Intelligent Spas is the only dedicated spa-specific research company that does nothing other than collect, analyze and report findings. They have been doing spa research since 2001 which is longer than any other company, do it globally, collect the most amount of data and have the greatest number of metrics. They collect raw data and have a proprietary software program to analyze the data. They are the only company that does not provide any consulting; therefore, there are no conflicts of interest in terms of how the data is used. They define and provide metrics that are niche-specific for the various spa segments and only release research which achieves a response rate of at least 5% of the spa population, which provides a statistically reliable sample size in industry research terms.
  
  
• Gabrielle Lerner, Associate, PKF
  PKF under the direction of R. Mark Woodworth (President, PKF Hospitality Research) and Bruce Baltin (Senior Vice President, PKF Consulting-LA) produced its first Spa Trends report in 2007. PKF collects financial statements from US-based lodging-related spas located in resort and urban areas that generate over $500,000 in spa revenue and have been in operation for a full two years prior to the report’s release. PKF identified spa benchmarking as an un-met need within the hospitality industry. As a result, they embarked upon spa research as an extension of their “core” business of hospitality consulting and research. Their sample size is limited in terms of number of participating spas. This may be due to the fact that spas may be somewhat hesitant to provide information to consulting companies. It would seem, however, that with their methodology, it would be easy to have a higher participation rate. PKF uses calculated, self-reported data from financial statements and follows the Uniform System of Accounts for the Lodging Industry rather than the Uniform System of Financial Reporting for Spas. Directors of Finance tend to follow the lodging reporting format rather than the more detailed spa reporting format. Regardless of which accounting system they follow, we have seldom seen spas categorize their information in a uniform way; therefore, consolidated data may not provide an accurate “apples to apples” comparison. On the positive side, however, conducting spa research allows PKF to be a better consultant and have a competitive edge over other consultants because they have a greater understanding of the “spa numbers” and can give better advice as part of their “core” hotel consulting services.
  
  
• Jan Freitag, Vice President, STR
  Over the past 2 years, STR has conducted two spa studies based on 9 data points collected from 40 or fewer luxury-based spas. Neither of these two studies resulted in a printed report on the spa industry; however, press releases were plentiful. STR is planning to create a more extensive spa benchmarking program, but no information has been published on the details, launch date, etc. We understand that as part of STR’s potential future research endeavors, they will be offering consulting services on how to use the data. As with PKF, this may be perceived as a conflict of interest, and it could compromise the response rate of future research.

Summary Chart

The attached chart is a summary of the key differentiators that need to be understood in order to determine the value and usefulness of what each company provides. This chart represents what has actually been done verses what is being planned or contemplated in the near or long-term future. Although STR is planning to do a more intensive research program, it has not yet been done; therefore, their future plans are not reflected on the chart.

Is the Research Reliable? Do the Numbers Make Sense?

It is obvious from the above that there are significant differences in the approach to spa benchmarking; therefore, you need to really look at the numbers and see if they make sense. When numbers are printed either in a report, presentation or in a press release, don’t assume they are always accurate. It is very easy for a company to have a slight mathematical error, for a writer to mis-type a number, for a spa to submit an inaccurate number, for the sample size to be too small to accurately represent the spa industry, etc.

People who rely on accurate metrics and benchmarks (asset managers, directors of finance, general managers, spa directors) need to take the time to do their own “double check” to see if the numbers seem reasonable. If a research company does not have experience within a certain discipline (like spas), they may not notice if a number does not “make sense.” If a spa submits a mistaken number and if the research company does not catch it, the research findings can be totally skewed. The research companies should have someone from within the spa industry review the numbers to see if anything appears to be questionable in which case the research company can determine if there’s a formula error, not a large enough sample size for statistically accurate data, an erroneous number being reported by a spa, etc.

Here’s a prime example of why someone who understands spa economics and operations should look at the findings and recommendations before the information is made public via a report, presentation, article or a press release. A Director of Finance for a hotel and spa read a press release on some numbers from the 2008 STR spa study. STR reported that the Revenue per Available Treatment Hour (RevPATH) is $136.79. This number did not make sense to the Director of Finance so he prepared the following analysis which he sent to HFD:

• a. Take a spa that has 10 rooms and offers services 10 hours per day. That is 100 available treatment hours.
• b. This spa would therefore have to generate $13,679 in revenue on that day.
• c. This would equate to $4,964,000 in revenue for a spa with 10 rooms.
• d. In order to achieve this, it would be like having 100% room utilization AND an average price per treatment of $136.79.
• e. Further in the STR press release, treatment room utilization is 30.8%. If this is the case, then the spa would have to have performed as follows: Total Revenue $4,964,000; Available Treatment Hours/Year=10 Hours/day x 10 Rooms x 365 days = 36,500 AVAILABLE TREATMENT ROOM HOURS; 36,500 ATRH x 30.8% utilization = 11,242 treatment hours booked $4,964,000 / 11,242 = $441.56 per treatment!

HFD sent the above to Jan Freitag. He is a Vice President at STR and oversees STR’s spa research. Jan’s response was “the data is indeed not for RevPATH, but for average treatment revenue per hour used (not available). Thanks for bringing it to my attention.” Jan made the correction and sent HFD a note saying “the RevPATH number is more in the range of $5.” A RevPATH of $5 is quite different from the reported number of $136.79. Jan then proceeded to look at the other numbers and found some more errors which he has corrected. HFD assumes that STR will be issuing an up-dated, corrected set of spa benchmarks so that those who are being held accountable for the spa financials are not being judged on a number that is erroneous.

In comparison, Julie Garrow, Managing Director of Intelligent Spas, explained she personally takes time to "clean" the data to ensure the survey responses are correct: "This is an important part of our research process which requires experience, it cannot be conducted by just anyone. Obvious input mistakes such as a single spa entering 199,534 in the number of employees field can easily be corrected. However, I also check for less obvious errors by cross referencing specific survey questions to ensure the data is realistic, for example, spa size versus number of treatment rooms and average revenue per visit versus pricing."

Gabrielle Lerner, PKF Associate, explained some of the challenges PKF has with accurate data collection. “In collecting data for our Trends in the Hotel Spa Industry, we ran into obstacles in terms of increasing the number of participants in the study, as well as reviewing and analyzing financial statements. I think a challenge is that hotel spa operators are not always aware of the benefits of benchmarking nor do they have the support of higher management to share their data. Furthermore, in order to properly use the industry research, spas need to streamline their revenue and expense line items according to one universal rule of thumb. Adapting financial statements to the Universal System of Accounts can take time, effort, and communication within the industry. But once more spas are on board and operators have standardized their accounting practices to ensure consistent reporting, we believe more meaningful data can be created and the spa industry will be better for it.”

As you can see, it is really challenging to do accurate spa research so the lesson learned is that when you look at numbers, don't just accept them at face value and don't automatically assume that the reputation of the company in one sector assures the accuracy of the research conducted for another sector.

Where Do We Go From Here?

Based on extensive due diligence, HFD concluded that statistically reliable, quality research is scarce. While each of the companies in this article has good intentions and is trying to provide useful information, a company’s “core competency,” credibility and reputation in one area is not the automatic “stamp of approval” when going into new areas of study.

HFD believes that there will be an increasing amount of information on spa metrics and benchmarking, but “reader be wary” because much of it may be unreliable and questionable. You cannot blindly believe or accept the findings as accurate just because they are quoted in a press release, a printed report, a presentation or conducted by a “big name” company. Professional market research principles and standards need to be applied to industry benchmarking studies, e.g., definition of a spa and market segments, sample size, type of data collected, etc.

The evolution, integrity and efficacy of spa benchmarking and the ability to determine reliable metrics will be critical to the economic potential and reality of the spa industry. The spa industry needs reliable spa industry benchmarks which are comparable world-wide and which are segment-specific. There are so many nuances between urban, resort and destination spas as well as day spas, club spas, medi spas, etc. The spa industry is not homogeneous so numbers need to be specific to the spa segment, and there needs to be enough spas within the segment to make the numbers statistically reliable.

HFD believes that as benchmarking studies evolve, companies will collect, analyze, report and monitor information so that the metrics become increasingly more reliable, useful and universal. Right now there is only one company, Intelligent Spas, that is totally dedicated to spa research. They are the pioneer and leader and have set the benchmark for other research companies that aspire to contribute to the growth of the spa industry.

At some point in the not too distant future, HFD thinks that spa research will consist of web-based programs whereby spas can enter their data on-line and will allow the spas to produce their own realtime reports so that they can react to market conditions in a timely manner. This web-based program could also help spas within a chain to compare how their spas are performing with one another and to set up “best practice” principles. In addition, spas would be able to print out their own competitive set analysis on a pre-set schedule assuming there are enough spas within the set to make the information useful. Accurate and “at will, on-call” spa benchmarks will give spas the information they need to respond in a timely manner to market conditions, therefore, providing the potential for them to perform better and maximize their profit potential.

The future of spa research is full of challenges as well as potential and opportunities. HFD has contacted asset managers, directors of finance, general managers, investors and spa operators and they all expressed the need and desire to have accurate benchmarks that can help them monitor their business and determine how well their spa(s) are doing compared to others within their chain and/or competitive set. The need is there and the time is “now.” The spa industry is ready to work with research companies that are prepared and dedicated to meet this need.

Judith L. Singer, Ed.D., ISHC, is the President & Co-Owner of Health Fitness Dynamics, Inc. (HFD) an internationally recognized spa consulting company. HFD is actively involved in conducting economic and consumer spa research. Since its inception in 1983, HFD has been the consulting firm to over $650 million of completed spa projects. Dr. Singer is also the past chairperson of The International Society of Hospitality Consultants and was on the ISPA Committee for the inaugural edition of the Uniform System of Financial Reporting for Spas. Ms. Singer can be contacted at 954 -942-0049 or judysinger@hfdspa.com

HotelExecutive.com retains the copyright to the articles published in the Hotel Business Review. Articles cannot be republished without prior written consent by HotelExecutive.com