1. Define Clear Objectives

• To measure customer satisfaction with identified services
• To identify weaknesses in the customer contact cycle
• To set realistic benchmarks for performance standards
• To test customers reaction to new products or services
• To profile customers and potential customers for business development purposes

If management decisions are to be made arising from the survey results, it is critical that the sample size will provide an acceptable level of statistical accuracy.

For example, with a customer base of 5,000 upwards and a desired level of accuracy of 90% with an error margin of 5%, the sample size would need to be around 270 surveys.

However if the survey is primarily to be used for targeted marketing initiatives, then the entire population will need to be surveyed

Each survey project will have three distinct stages as follows:

• Design and scripting
• Outbound calling and data collection
• Analysis and Report Preparation

The classic steps involved in any customer service performance benchmark project would typically involve:

• Identifying the key drivers of customer satisfaction.
• Ranking and weighting the key drivers in order of importance for:
• Measuring customer performance rating by key driver
• Measuring competitor performance rating by key driver
• Setting minimum and desired benchmark ratings for each driver
• Periodic reporting of performance vs. benchmarks and competitors

The development and design of a suitable questionnaire ideally should be team involving the key internal customers who will use the results. A workshop format often works well. Also a kick-off briefing meeting with subsequent iterative review and amendment can be equally effective.

The more input from the internal customer than the better will be the final results.

For complex surveys, consideration could be given to a mailing of customer questionnaires. A telephone follow-up will almost certainly be necessary to complete the necessary survey sample size by telephone. Online surveys are becoming increasingly popular and are very cost effective. In some case a mix of telephone and online data collection is best if specific quotas are required.

See online survey case study at Appendix 1.

Incentives often work as an incentive to participants to complete and respond to surveys. In the case of benchmarking, the best incentive is to offer a complimentary copy of the survey to each participant.

Online Internet surveys will sometimes work, particularly where a regular relationship exists with the service provider. However be careful that such surveys are representative and action may need to be taken to collect sample responses by telephone from respondents who do not have internet access.

Cold calling can be highly successful, particularly for short surveys of fewer than 15 questions and can provide a useful dipstick to compliment other methods of customer feedback such as focus groups.

The following factors should be considered when designing the survey:

• The use of open questions with probing and prompting by the interviewer (Useful for measuring perceptions, or for determining reasons for specific actions such as moving to a competitor). The advantage of open questioning techniques is that it avoids giving false trends through using lists of prompt options which allow the interviewee to close the call without giving an honest answer to a question
• The use of a post survey classification process to identify trends from open questions
• The use of nested question techniques to drill down further or probe the reasons for specific answers
• The use of prompt lists where appropriate
• The use of multiple choice vs. single choice questions

On completion of the data collection, the data needs to be analysed and formatted in a report for presentation to management. A number of software packages are available to report the results of the survey.

Reports can be in hardcopy, softcopy with and without graphics. If open questions are involved, these should be category coded prior to preparing a report. Reports can be published on an intranet with drill down options for detailed review of a customer record.

Performance Marketing can offer clients the following benefits:

• Survey respondents usually are more open with third parties
• We have considerable experience in survey design
• Our experience of doing telephone survey projects include:
  
  • Multiple market research surveys for O2
  • Network Survey for a network product vendor
  • World Bank survey of Managing Directors of Irish companies
  • Survey of Canteen service level at Vodafone
  • Survey of Telecommunication Industry trends for Canalys
  • Produkt and Markt Rural Development Survey in Ireland
  • Survey of IT Directors for HP
  • Compaq Computer (Customer service satisfaction)
  • Eircom (Internal IT helpdesk service quality measurement)
  • Eircom IT (User survey)
  • Le Bas (Aircraft leasing customer satisfaction)
  • Superquinn (Customer satisfaction)
  • Golden Pages (Custom loss review)
  • Irish Times (Advertiser survey, newsagent survey)
  • Telecom Phonewatch (Customer satisfaction )
    
• Multiple market research surveys for O2
• Network Survey for a network product vendor
• World Bank survey of Managing Directors of Irish companies
• Survey of Canteen service level at Vodafone
• Survey of Telecommunication Industry trends for Canalys
• Produkt and Markt Rural Development Survey in Ireland
• Survey of IT Directors for HP
• Compaq Computer (Customer service satisfaction)
• Eircom (Internal IT helpdesk service quality measurement)
• Eircom IT (User survey)
• Le Bas (Aircraft leasing customer satisfaction)
• Superquinn (Customer satisfaction)
• Golden Pages (Custom loss review)
• Irish Times (Advertiser survey, newsagent survey)
• Telecom Phonewatch (Customer satisfaction )
• Our low overhead structure means that our costs are competitive
• Our special survey software (Raosoft) automates the data analysis and has a powerful graphic facility to enable professional report production

Project Summary:

Canalys are a market analyst company, based in the UK , covering the European IT market, with a specific focus on Converged Telephony. The company are currently trying to build a panel within the European telephony reseller/distributor community.

The need is to recruit approximately 20 candidates (per country), from 7 different geographies, such as France , Germany , Italy , UK and Nordics, who would agree to complete an online questionnaire for 4 consecutive quarters. Panel members will be asked to give their opinions on topical European channel issues through a short online survey (around 10 questions). In return, respondents will receive a full set of survey results, with analysis, plus complimentary copies of Canalys reports, which provide insight on important European telecom trends. In addition, they will obtain access to an appropriate part of our website, giving them access to top level telephony data and other useful information only currently available to our clients.

Panellist Recruitment Process:

List of approx. 2,000 contact details of interview panellists to be supplied and uploaded to Telescript from Excel spreadsheet

• Outbound calling Programme:
  • Verify company activity matches desired profile
  • Identify the correct decision maker contact
  • Make contact and present Canalys offer
    • Free research reports
    • Access to relevant statistics on member area of website
• Recruit panellists for quarterly Converged Telephony surveys
• Obtain panellist contact details for quarterly survey including:
  • Direct number
  • Email address

• 7 countries with a min and max quota per country
• Send initial welcome email to new panellists
• Online access to report of panellists recruited to-date:
• Sub-total By country

Web Survey Process:

• List of approx. 200 contact details of interview panellists to be uploaded to online MS Access database from Excel spreadsheet
• Survey Id
• Country id
• Panellist id
• Panellist password
• Panellist contact details
• Each interviewee to be allocated a unique identifier(Username) and password to be used for access when completing the survey
• Emails to be sent to each panellist as follows:
• Quarterly survey activation email with URL to questionnaire in relevant local language
• 1 st reminder email to non respondants with URL
• 2 nd reminder email to non respondants with URL
• Regular review of response rates and activation of emails as required
• Summary report data:
• Quarterly survey id. (year, quarter number)
• Total survey panellists by country
• Total responses to-date by country
• View of completed surveys by survey id/country
• Non respondent records to be downloaded to Excel for telephone follow-up stage
• On completion download all completed survey records for despatch to client.