Case Study on Software evaluation in Companies

Hello Readers:)

This  week we are going to present you the below article on software evaluation model performed on major companies and its efficiency. "A maturity model and an evaluation system of software customer satisfaction: the case of software companies in Korea"

Objective:
Mainly this paper aims to introduce an evaluation model of software that reflects software specialty as well as customer feedback. A maturity model of software customer satisfaction is presented, and an evaluation system is developed based on the maturity model. As a case study, the maturity model and the evaluation system are applied to software companies and their customers to show evaluation results.

Existing Software Evaluation Models and their limitations:
There are many kinds of software process evaluation models such as CMM, SPICE, PSP, BOOTSTRAP, ISO 900‐3 and TickIT related to ISO 900‐39  and SEPRM . Among them, CMM  and SPICE are the two most popular and basic software evaluation models. They are designed to evaluate the capability level of software with the goal of improving software development processes. IT service CMM  is a process maturity model of organization that delivers IT services to customers.  However, the traditional software evaluation models such as CMM, SPICE, and IT Service CMM were not developed from the customer point of view. Those models are focused on software development process in terms of the software developers, providers, and software auditors.  Thus, it is necessary to review evaluation models about general customer satisfaction in order to develop an evaluation model of software customer satisfaction.

Maturity Model :

The maturity model of software customer satisfaction considers both software products and related services, because customer satisfaction is determined by the combined perception of a software product and its service. The maturity levels of software customer satisfaction can be classified according to the extent of collection and reflection of customer opinions about software product and service.

Four maturity levels of software customer satisfaction are defined as

• Initial level: no consideration for customer feedback concerning products and services

• Ready‐made level: providing a ready‐made and general product and service to customers. Making strategies for the enhancement of customer satisfaction by targeting one market group of general customers

• Tailored level: providing various products and services to a segmented market that considers its corresponding customer characteristics. Making strategies for the enhancement of customer satisfaction by targeting differentiated or segmented customers

• Customer‐oriented level: providing customized or personalized products and services to an individual customer. Making strategies for the enhancement of customer satisfaction by targeting each individual customer. Emphasizing change management to improve product, service and organization according to customer's dynamic requirements

Each maturity level of software customer satisfaction can be described in three aspects:

• the extent of collection and reflection of customer opinions
• the decision factors of product customer satisfaction
• the decision factors of service customer satisfaction

The below figure presents a schematic diagram that compares the maturity model of software customer satisfaction with other related works.

1. 
   
2. 
3. 
   

Evaluation Model:

The evaluation framework of software customer satisfaction is divided into two parts:

1. 1. Product satisfaction
2. 2. Service satisfaction

This framework has an underlying assumption that total customer satisfaction should be measured by evaluating product satisfaction and service satisfaction together. Technology and functions are measured in software product satisfaction. Various software service issues such as installation support, education, maintenance and repair, and customer care are measured in service satisfaction.

In the framework, the first‐level and the second‐level attributes are developed for product satisfaction and service satisfaction. Evaluation indices for measurement are generated from the second‐level attributes. The attributes and evaluation indices of the evaluation system are presented below

Case Study:

To verify practical values of the maturity model and the evaluation system of software customer satisfaction,its been applied to software companies and their customers in Korea. 

In the project, software companies are classified into four areas:

1. 1. e‐Business solution
2. 2. development, management and security software
3. 3. private and office software
4. 4. industry and science software

A total of 62 software companies from the four areas participated in the project. Their data were investigated for the project's quality and reliability in advance.Finally, 18 companies were selected for the application of the evaluation system and 180 of their customers released meaningful data through the questionnaires and interviews. In addition, 17 experts on software evaluation participated in assigning the weights of evaluation attributes in the framework.

Results:

• Software companies that acquire high product customer satisfaction also have high service customer satisfaction – the rank in product satisfaction is revealed to be identical to that in service satisfaction.
• Customer satisfaction regarding development, management and security software is higher than that of any other software areas. One can deduce that development, management and security software such as DBMS, case tools, and middleware are relatively stable in market and technology trends. Thus, their results in customer satisfaction seem to be superior to other software areas.
• The standard deviation of service satisfaction is 0.22, while that of product satisfaction is 2.13. This indicates that relative ranks of customer satisfaction among software companies depend more on product satisfaction. Customers still perceive software quality more in terms of product quality than service.
• The average product satisfaction is lower than service satisfaction by 2.676. The resulting customer satisfaction tends to decrease easily, as customer expected values increase. Thus, it can be interpreted that software users still have higher expectations of satisfaction in software itself than in the related services