INTRODUCTION
With the huge investment in developing training strategies and
programmes, the question is no longer whether organisations should train
or not, but rather it is about whether or not training is worthwhile and
effective (Mann 1996). Redshaw (2000) argues, that although T&D
plays a critical role in organisational performance, it is viewed
differently in different organisations. It may be considered as an
unnecessary, underused and unrecognised function. Some organisations
find that T&D does not contribute to employees' commitment, and
sometimes it is considered as a waste of money and time. Unfortunately,
in many other organisations the management does not believe in T&D
as a function to contribute in improving the overall organisational
performance. They believe that T&D does not work since it is too
difficult to prove its effectiveness in terms of calculating the Return
on Investment (ROI) of T&D programmes.
A review of the T&D literature shows in many Arab
organisations, including Jordanian organisations, training is still not
considered as an important organisational function, which contributes to
the organisation's success. Instead, it is viewed as a vacation
activity or leisure time pursuit, which is given to some people,
normally to the managers' relatives and friends. Moreover, the
literature suggests that in some Arab organisations evaluation is rarely
undertaken, because training is being seen as an overhead and not an
investment to be evaluated (Al-Athari & Zairi 2002). In many other
Arab organisations T&D evaluation is not conducted in a professional
manner. In this regard, Abdalla and Al-Homoud (1995), Bahar, Peterson
and Taylor (1996), and Abdalla, Maghrabi & Raggad (1998) argue that
many Arab organisational practices, in terms of their T&D
management, are deficient because of the lack of systematic planning,
implementation and the evaluation phases. Moreover, Atiyyah (1993)
argues that evaluation methods in Arab organisations are highly
subjective and their results have limited impact on improving the on
going programmes or even the designing of new programmes. Atiyyah (1993)
also contends that T&D programmes were usually evaluated at the
reaction level via questionnaires distributed to the trainees to give
their opinions on the quality of instruction, programmes materials, and
suitability of the training techniques.
From a broader landscape the international T&D literature
reveals that the T&D evaluation stage faces many vexations and
challenges. In this regard Shandler (1996), Redshaw (2000), as well as
Burrow and Berardinelli (2003) argue that proving T&D effectiveness
in achieving organisational performance criteria, such as profit,
quality, customer satisfaction, ROI and market share, is not an easy
task. Consequently, the evaluation stage in the training process is
often ignored. There are many factors that underpin the reluctance to
engage evaluation initiatives. For instance, T&D benefits accrue
over a long period of time, there are many other intervening variables,
and a great deal of factors can influence an employee's
performance, while most T&D outcomes are subjective, complex and
difficult to measure. Support for these contentions can be gleaned from
the work of Abdalla and Al-Homoud (1995), and Al-Athari and Zairi
(2002), who found that the most common evaluation challenges in the
surveyed Arab organisations are the cost of conducting this process,
difficulty in finding evaluation methods, difficulty in finding
evaluation quantitative financially criteria or language, time required
to accomplish this process and the lack of information needed for
evaluation.
The purpose of this study is two fold. First, is to attenuate the
gap in the T&D literature in Arab financial entities and in
particular Jordanian organisations by exploring the current management
T&D evaluation practices and challenges in the Jordanian banking
industry. Second, to suggest practical solutions for T&D evaluation
problems in these organisations. A review of T&D literature shows
that there is a lack of relevant studies in the field of T&D
practices and management in Jordanian organisations and in Arab
organisations, generally. Moreover, most of the previous studies
conducted in Arab organisation did not target the banking industry in
particular, which is considered as a people oriented industry. Indeed,
it can be claimed that a firm's success comes from the people who
work for these institutions. Jordanian banking organisations spend a lot
of money in T&D programmes, but these companies do not appear to
know the extent of effectiveness of these programmes (Altarawneh 2005).
Arguably, this study is different from other previous studies in terms
of being conducted in the context of Jordanian organisations, which have
not been previously targeted. In addition, it is different in terms of
employing the banking industry, the study sample, as this sector has
been neglected in the previous Arab research, despite claims of
extensive investment in T&D programmes.
This paper has three main parts. In the first part is provided an
overview of the most relevant T&D evaluation literature presenting
the definition and the importance of T&D evaluation; main T&D
evaluation challenges and problems; T&D evaluation models; and
T&D evaluation in Arab organisations. The second part of the paper
describes the findings of the study related to current T&D
evaluation practice and challenges in Jordanian banks. Thus, much of the
second part is also summarised on informative figures. The concluding
section, the third part of the paper, succinctly integrates the two
preceding parts, and in addition suggests practical solutions to the
most common T&D evaluation problems found in Jordanian banks.
THEORETICAL BACKGROUND OF THE STUDY
An Overview of T&D
T&D is the most important subsystem or element of human
resource development. It concerns increasing, improving, enhancing and
modifying employees', and managers' skills, abilities,
capabilities and knowledge to enable current and future jobs to be more
effectively conducted. These desirable achievements are likely to
increase an individual's as well as an organisation's growth
and performance. Generally, there are two main ways for conducting
T&D programmes. First, in house T&D designed initiatives that
are conducted within the organisation, which could be directly on the
job experience. A second way for conducting T&D programmes is off
the job or out of house T&D designed that is conducted outside the
organisations through external providers who will be responsible for
evaluating T&D effectiveness through many objectives. These outcomes
are likely to embrace such topics as developing the skills and abilities
of employees and managers to improve their performance; familiarising
employees or/ and managers with new systems, procedures and methods of
working; helping employees and new starters to become familiar with the
requirements of particular jobs in the organisation; and guiding the
organisation to improve customer service, satisfaction and total quality
in the organisation.
A fundamental enigma for Jordanian organisations is the
determination of adequate resources as well as a willingness to deal
with the requirements for comprehensive examination of prominent issues
associated with T&D. Clearly, T&D is an important and crucial
activity for any organisation intending to improve the productivity and
competitiveness of operations. In fact, the need for T&D has
increased dramatically due to the rapidly expanding use of technology
within industries and businesses, and the continuous threats of
knowledge and technology obsolescence. In addition, the growing emphasis
on quality and customer satisfaction has compelled organisations to
recognise the importance of T&D in terms of job satisfaction,
productivity and overall profitability. Whatever the type of T&D,
and which ever are the dominant objectives, evaluating T&D
effectiveness to achieve these primary outcomes within a mechanism that
is based on systematic criteria is a very important stage in the T&D
process. The next section will show why and how.
What is Training Evaluation and Why?
As organisations are spending more and more on training, evaluating
the effectiveness of training has become critical. Hesseling (1966)
suggests that one of the main tasks of the trainer is to test for
training effectiveness and to confirm that the selected training methods
have achieved the desired results. It could be argued that evaluating
training effectiveness is the starting point when talking about T&D
benefits and contributions in any organisation. Training is a tool used
to change people's behaviour, while evaluating training
effectiveness is centred on measuring that change.
The Manpower Services Commission (MSC) (1981) defines training
evaluation as the assessment of a total value and benefits of the
training process, course or programme in social as well as financial
terms. MSC distinguishes evaluation from validation in that it attempts
to measure the overall cost benefit of the course or programme and not
just the achievement of its laid down objectives. The term is also used
in a general judgmental sense of the continuous monitoring of a
programme or of the training function as a whole.
Training evaluation helps to collect all the descriptive and
judgmental information required to make effective training decisions.
Burrow and Berardinelli (2003) suggest that the evaluation stage should
be viewed as part of an effective training process and a base to improve
organisational decision making about human performance improvement. The
evaluation activity identifies where a T&D department could
demonstrate that it adds valuable service or adds value to the
organisation's performance. From training evaluation an
organisation can make a judgment of what employees gain from training
and what is transferred and implemented to their work. In this context
Bee and Bee (1994) state that it is necessary to evaluate T&D
programmes, since the evaluation process improves the efficiency and
effectiveness of training programmes, the results display the
organisation's impression about trainers' performance in the
programme and helps them to improve their training skills as well as
justifying the expenditure on the programmes. Therefore, it could be
said, that as long as an organisation pays attention and emphasises
training evaluation to identify training contributions, training is
viewed by this organisation as an important organisational function that
improves or helps to improve the organisational performance.
Training Evaluation Challenges and Problems
Although training evaluation is a very important phase for a
successful T&D programme, this phase is the most neglected one. The
caution in endorsing evaluation methods can be due to a lack of time and
resources to complete the evaluation process, or may be a lack of top
management support, ineffective design of the evaluation process and
unclear evaluation criteria. Thus, the data is likely to be
misunderstood, misused and misinterpreted (Beardwell & Holden 1994,
Houlton 1996, Combs & Falletta 2000). In this regard, Hale (2003)
has included two practical examples that demonstrate an insufficient
evaluation process. A recent survey undertaken in the USA has shown that
57 per cent of the surveyed American organisations consider evaluation
as important, but only 27 per cent use an action plan after training;
just 16 per cent use 'follow up' from T&D and only seven
per cent of the surveyed organisations evaluated the ROI in training.
These findings imply that the T&D evaluation stage was not used in
many organisations or in fact, after engaging the rigors of a T&D
initiative the investment is not wisely used.
Measuring the gains of organisational effectiveness that result
from training is probably the most difficult and problematic task of all
of the training stages. It is described by Foxon (1986) as "the art
of the impossible". Also, Grove and Ostroff (1990), Redshaw (2000),
and Shandler (1996) argue that evaluating training gains or influences
on organisational effectiveness and performance is a very hard task for
at least the following three main reasons.
* There are many other factors which can have an influence,
positively or negatively, on the organisation's performance. Such
as market forces, competitive activity, new technology, legislation, and
the economy (Shandler 1996, Redshaw 2000, Burrow & Berardinelli
2003), as well as luck, industry, power and size.
* The impact of training may take a considerable time to show up in
overall results (Redshaw 2000).
* Training results and outcomes are usually complex and difficult
to be measured or quantified, such as employee satisfaction, commitment,
motivation, and changes of behaviour and attitudes. A plethora of
outcomes can cause confusion between accountants and line managers who
view training evaluation differently from the way trainers view the
process. For instance, outputs can be compared with inputs, while the
fact is that the relationship between training intervention and
organisational performance cannot be easily identified.
Another important training evaluation challenge is that it is
difficult to measure the relationships between training and
organisational performance. A primary acknowledged difficulty is the
cause and effect relationships are not necessarily straightforward and
hardly ever clearly defined (Yeo 2003). For instance, learning occurs in
all organisational levels thus, there are many intervening variables.
And there are other organisational factors which may influence the
organisational outcomes, such as leadership, workplace arrangements, and
organisation structure.
To overcome most of the evaluation difficulties, Burrow and
Berardinelli (2003), and Redshaw (2000) suggest that there are two
important dimensions to be taken into account. First, line managers
should be involved. Indeed, effective T&D programmes require top and
line management support, commitment and leadership. Thus, line managers
should participate in determining the training objectives and evaluation
criteria, and is more likely to occur when trainers are consulting with
line managers, who may ask them questions such as, what they want to
improve, how they will recognise success and how they will measure this
success. During this process all the other factors that influence the
organisation's performance would be identified for agreement.
Second, because it is hard to connect training interventions to the
organisation's performance measurements directly, these
measurements need to be revised, refined and resized to document the
impact of planned learning (T&D) on organisational performance.
Training Evaluation Models
In order to understand how the T&D evaluation stage should be
conducted successfully, it needs to be based on a particular T&D
evaluation model. There are many training evaluation approaches and
techniques. However, there are four main complementary evaluation
frameworks: (a) Kirkpatrick, (b) the Bell system, (c) Parker, and (d)
CIRO. However, Kirkpatrick's evaluation model is the one most
commonly used by many organisations (Hale 2003).
According to Kirkpatrick (1996), there are four stages to be
considered when evaluating training effectiveness: reaction, learning,
behaviour and results level. The reaction level answers whether people
are happy with the training inputs (Hall 2003). It evaluates
participants' reactions, opinions, impressions and attitudes toward
the programme. The learning level answers, "what do people remember
from the training session?" (Hale 2003). To what extent have the
participants learned the material and the particular skills or
'know how' that are contained in the programme? This
measurement is made through special standardised tests at the end of the
programme, such as pre, post, paper and pencil tests, skill practice,
workshops and job simulation. The behaviour level addresses the issue of
"whether people use what they know at work?" (Hale 2003), and
the changes in the participants' behaviour, skills, patterns of
work, relationships and abilities that are necessary to undertake the
task at hand. These indications could be collected through observations,
survey, interview and comments of supervisors, and colleagues and from
performance appraisal reviews. Finally, the results level determines
"what are the outcomes of applications on the job over a period of
time?" (Hale 2003). This level of evaluation focuses on the impact
of behaviour change on the organisation's performance. Because
changing an employee's behaviour and attitudes is not the final
objective of T&D, the end results should include important elements
such as, improved productivity, better quality, lower costs, more speed,
fewer accidents, improved morale, lower turnover, and ultimately, more
profit and better service. To determine the cost efficiency of training,
recent ROI is used to establish whether the monetary values of the
results exceed the cost of the programme. In this regard, Mann (1996)
cited two empirical studies that reviewed the most popular evaluation
levels used by USA organisations when evaluating their training
programmes. Both of the studies found that most of the surveyed
organisations (over half) use assessment of their participants'
reaction and satisfaction with the programme as the key methods of
evaluating training.
All of the evaluation models focus on the following important
criteria for determining training effectiveness.
* Stakeholder perceptions (level one: reaction)
* Learning gain (level two: learning)
* On the job performance improvement (level three: behaviour)
* Organisational improvement and ROI (level four: results).
Stakeholder reaction provides insight into trainees'
motivation and satisfaction, but does not directly measure training
results. Historically, training was conducted through measuring the
number of trainees, as well as their perceptions and attitude towards
the programme. Nevertheless, while trainees' attendances,
perceptions, motivation and attitude towards the programme are important
outcomes of any training programme, there has yet to be reported a
significant relationship between these outcomes and achieving the
programme objectives, including improving an organisation's
performance (Combs & Falletta 2000, Green 2001). Assessing training
effectiveness based on trainees' perceptions is described by James
and Roffe (2000), and Acton and Gloden (2003) as ad hoc, unsystematic,
informal and unstructured evaluations of training programmes, which
tends to be post training appraisals rather than approaching the
evaluation of training programmes from their design stages.
Training effectiveness no longer focuses on trainees'
perceptions, but is directed at other important factors. These factors
incorporate such questions as: Did employee performance improve as a
result of training? How did training contribute to achieving the
organisation's critical goals? (Burrow & Berardinelli 2003).
Now there is a stronger acknowledgement for T&D results to be
connected to the organisational performance measurements, such as ROI,
changes in productivity, quality improvement, customer satisfaction and
market share (Miller 2002). Alternative outcomes are less valued and
valuable. Therefore, of the identified evaluation criteria there are two
critical and essential levels that affect other evaluation levels, and
they are (a) learning gain (acquired skills, competencies, attitudes and
capabilities); and (b) on the job performance improvement achieved
through changing trainees' behaviour and work patterns, which is
often called training transfer. Bramley and Kitson (1994) contend that
the appropriate training evaluation model is the model that facilitates
evaluating the third and fourth levels of the Kirkpatrick evaluation
model (behaviour change and results). These training needs and
objectives are defined in terms of the changed behaviour and increased
effectiveness, rather than just increasing knowledge, skills and
abilities. Also, top management and line managers are involved in the
whole training process and are able to evaluate the changes in behaviour
and effectiveness, which are occurring as a result of training.
Training Evaluation and Organisational Performance (Level Four)
The foremost strengths of a particular T&D evaluation model are
especially those that emphasise the role of T&D on organisation
performance and show a ROI from T&D. These perceptions are found in
the Kirkpatrick evaluation model, which introduces the relationship
between T&D evaluation and organisation performance. Assessing level
four of the Kirkpatrick evaluation model demonstrates these important
features.
Valued results are connected to the organisational performance
measurements, such as ROI, changes in productivity, quality improvement,
customer satisfaction and market share (Miller 2002). Continuous
training programmes are essential for achieving higher productivity,
better on the job performance and improved quality (Tennant, Boonkrong
& Roberts 2002). These social scientists also pointed out that
Japanese organisations emphasise the roles of training more than USA and
European manufacturing organisations. This is revealed in the automotive
sector with 380.3 training hours per year for Japanese workers, compared
with 46.4 for USA workers and 173.3 for European workers. This heavy
investment in T&D has led to dramatic performance improvement, in
terms of cost, quality and delivery in Japanese organisations. Horwitz
(1999) contends that T&D contributes to the organisation's
performance, success and competitiveness through providing the capacity
to enhance individual abilities and competencies as necessary
requirements for effective organisational performance. Armstrong (1989)
also refers to performance related training designed to develop
competencies that impact directly on the bottom line by promoting
flexibility and supporting innovation. He also argues, that if the
learning that is derived from training is not associated with enhanced
job performance or raised capability, then the credibility of both HRD
specialist and the process are at risk.
The effectiveness of T&D programmes on organisational
performance improvement is a difficult, vexatious and problematic task.
Thus, training professionals have been challenged to document or
demonstrate the results of training on organisational performance
improvement (level four) by using organisational performance impact
measures, such as ROI, changes in productivity, customer satisfaction,
and quality and market share. In this regard, Muhlemeyer and Clarke
(1997) argue that it is difficult to measure the correlation between the
implementation of T&D and the overall success of the organisation,
especially in the case where the organisation's success is measured
in financial terms, because there is a little evidence to suggest that
training per se can improve the financial performance.
Arguably, evaluation should focus on all organisational
stakeholders, comprehensively, and on the long-term results, and should
also use the most suitable information to make judgments about training
effectiveness (Russ & Preskill 2001, Burrow & Berardinelli
2003). It is also suggested that evaluation could be very focused and
short term or quite comprehensive with a focus on the long term results
depending on the type of training and its objectives (Russ &
Preskill 2001, Collins 2002, Burrow & Berardinelli 2003). Evaluation
should be based on a clear identification of the purpose and results
expected from the programme. Thus, if the programme is designed to
respond to a particular problem, or meet a specific requirement,
evaluation should be used to determine if that specific goal was
successfully achieved and not focus on broader expectations. According
to Brinkerhoff (1988), good evaluation is likely to provide convincing
evidence that the programme is aimed at important organisational
benefits. Specifically, robust evaluations are likely to lead to
participants reporting greater job satisfaction, the achievement of
important job reverent skills, a gaining of knowledge and improved work
attitude as well as effective transference to the job of learned skills
acquired from the T&D programme. However, much of the training
efforts in organisations are not specifically related to organisational
final outcomes.
Training Evaluation Stage in Arab Organisations
Despite widespread acknowledgement of the importance of the
evaluated stage of T&D programmes this element is regularly
neglected and seldom undertaken professionally. Indeed, some Arab
organisations rarely conduct T&D evaluation. Often training is seen
as an overhead and not an investment. When Al-Athari and Zairi (2002)
conducted an empirical study in Kuwait organisations, they found that a
minority of managers believed that evaluation is an important task.
Moreover, a majority of organisations occasionally evaluate their
training programmes through questionnaires, observation and performance
records. And regarding the evaluation models, these researchers observed
that most of the surveyed organisations rely on level one
(trainee's reaction) of the Kirkpatrick's (1996) evaluation
model when evaluating their T&D programmes. Al-Athari and Zairi
(2002) also reported that evaluation is not considered to be the most
important stage in the training cycle for most Kuwaiti organisations,
which rely on evaluating the level of trainees' reaction towards
the programme. Furthermore, Abdalla and Al-Homoud (1995) argue that
there are no specific 'follow up' procedures for evaluating
the effectiveness of T&D programmes. In most Arab organisations the
effectiveness of training programmes is evaluated based on the reaction
level, instead of focusing on the results level and on the transferred
knowledge to the workplace. In spite of these findings generally,
managers endorse T&D programmes.
Atiyyah (1993) argues that evaluation methods in Arab organisations
are highly subjective and their results have limited impacts on
improving the on going programmes or even designing new programmes. He
also argues that the programmes were usually evaluated at the reaction
level via questionnaires distributed to the trainees to give their
opinions on the quality of instruction, programmes materials, and
suitability of the training techniques. Moreover, Abdalla and Al-Homoud
(1995) contend that there are no specific 'follow up'
procedures for evaluating the effectiveness of T&D programmes, as in
most Arab organisations the effectiveness of training programmes are
evaluated based on the reaction level, instead of focusing on the
results of training and the transferred knowledge to the workplace,
which are considered, according to Kirkpatrick, as the best evaluation
systems, and focus on the effects of the application of information and
learned concepts on the organisation performance. The most common
evaluation challenges in Arab organisations are the cost of conducting
this process, difficulty in finding evaluation methods, difficulty in
finding evaluation quantitative financially criteria or language, time
required to accomplish this process and lack of information needed for
evaluation.
METHODOLOGY
Participants and Site
The population of the present study is defined as all the banking
organisations operating in Jordan that were listed and licensed as banks
at the Association of Banks in Jordan, Central Bank of Jordan according
to their reports for the year 2004 and listed also in Amman Stock
Exchange (ASE) as banking organisations. Therefore, the targeted
organisations reflect the whole population rather than a sample of the
population. The main reason for choosing the entire population is to
ensure that the sample is representative and not biased. Appendix 1
shows the name, groups and the number of the banking organisations. In
total 22 banks were listed as banks in 2003/2004 and all were in this
study. There were 14 commercial banks (of which five are foreign bank
branches), six investment banks and two Islamic banks.
The rationales behind targeting these organisations as the subject
of this study refer to the important role they play in enhancing
Jordanian economic performance. The banking sector, which is a very
active sector in Jordan, is one of the main pillars of the national
wealth and it is 'pushing' the whole economy forward.
Moreover, banking organisations in Jordan are known for their commitment
to develop their employees to offer better quality customer services,
these financial institutions pay more attention to T&D and expend
more effort and money in T&D than any other Jordanian organisations.
Due to the importance of human resources in economic growth and social
development, Jordanian banking organisations pay more attention to
improving the performance of their employees, as well as their
abilities, education and professional skills, through continuous T&D
and updating them with changes and development, so that they can react
properly to the process of economic growth to enable the banking sector
to better face the challenges of the coming decade.
A purposive or judgmental sampling technique was applied in
determining the participants that will best enable the researcher to
answer the research questions and to meet the research objectives. Thus,
the study targeted the banking organisations T&D, HRM and top
managers. These people were chosen to be the respondents of this study
because they are the people who have the required information about
T&D issues, management, problems and its importance for
organisational success, and also for their critical roles in their
banks.
The rational behind choosing top corporate managers in addition to
HRM and T&D managers as participants in this study is related to the
fact that top managers are considered by many researchers in the field
of management T&D as being the most important group in the
organisation to secure the success for T&D functions/programmes.
Their support, commitment, opinions and attitudes towards the importance
of T&D functions/programmes in the organisation decide the success
of these functions/programmes. In addition, these managers are the key
influential decision makers in the organisation. They are the key
strategy makers. Moreover, their appreciation and admiration of the
benefits of T&D, which are very important indications that T&D
really is working, will guarantee their commitment and support to this
important function. The term 'top managers' used in this study
reflects those who are on the board of directors or who report to the
managing director (e.g., Chairman, CEO, GM, MD, Deputy MD/GM, Director,
and Executive Manager).
This study was conducted in the headquarters of the banking
organisations with T&D, HRM and top managers rather than targeting
the banks branches because all Jordanian banking organisations have
headquarters in Amman and many other branches in Amman and other cities
in Jordan. However, in most of these branches there is no T&D
department/section or unit and in many branches there is no section or
unit for HRM. Therefore, the banks' branches do not have sufficient
information about T&D. All T&D decisions and issues are
formulated in, or centralised to, the banks' headquarters in Amman.
Thus, the headquarters are the main source of the desired data and
information about T&D in each bank; in these headquarters T&D
managers and/or HRM/HR managers are responsible for T&D issues,
rather than any other managers.
Procedure
In this study, primary data as well as secondary sources of data
are used. For the primary data this study employed the multi methods
approach (method triangulation) conducted through survey self
administered questionnaires and semi structured interviews in a
complementary, and supplementary way, rather than in competition with
each other, that was an appropriate and flexible way to conduct this
research.
Employing triangulation in one study has a number of advantages.
For instance, the technique provides a kind of convergence of results
that complements findings reached from analysing various observations
and enhances the scope and breadth of a study. Triangulation increases
validity because it ensures that the variable variance is attributed to
the trait of the subject examined rather than to the method used for
investigation. In short, triangulation consists of crosschecking data
for internal consistency and external validity, which are matters of
concern for any study. The small study sample encourages the use of
triangulation.
The logic behind using the multi method approach as a research
strategy for the current study refers to two main reasons. First, the
small size of the study sample and population necessitated collecting as
much rich and deep data as feasible by both qualitative and quantitative
data collection methods. The qualitative responses in this study
elucidated the meaning of the quantitative results (the sample size of N
= 38 is too small to make confidence predictions). The second reason is
that the research includes social behavioural factors, beliefs and
attitudes that need to be explored and explained in detail. Therefore,
applying the chosen research design helps the researcher to address all
qualitative and quantitative data that provide more flexibility to meet
multiple research interests and needs. Using different quantitative and
qualitative data collection methods in one study facilitates the
generation of richer and deeper facts and data that have potential to
enrich the study findings.
The self administered questionnaire, has a number of benefits.
These benefits have been identified as a prompt and relatively low cost
strategy for obtaining information in a context that is likely to
establish a good rapport with respondents. The rationale for using semi
structured interviews in this study is very successful in Arab
organisations, where managers prefer to talk rather than to complete a
questionnaire. Thus, it was decided that top managers would be
interviewed, rather than asked to complete a questionnaire on which they
may be reluctant to spend time providing written explanatory answers. In
addition, from the outset the researcher was convinced of the value of
using face-to-face interviews because of their advantages in exploring a
wide range of issues related to the research questions. In addition,
they complement the questionnaires and can explore or explain, indepth,
any further details, information, themes and facts under investigation
behind the questionnaire's responses.
A total of 44 questionnaires were distributed by the researcher in
late March 2004 to the HRM and T&D managers. An initial stage of the
study procedures was a personal visit to Amman Financial Market and the
Association of Banks in Jordan to get the organisations' addresses.
This action was followed by personal visits to each organisation's
headquarter by the researcher to introduce herself, to outline the
boundary of the study, and to build a strong trust with the executive
decision makers. This kind of visit is very important to build trust
with participants, to encourage them to participate and to determine a
deadline for completing the questionnaires. In addition, through these
visits the researcher was able to determine the name of the persons to
be interviewed. An informal or an unplanned discussion relating to
T&D practices and problems in Jordan accrued during these visits.
Moreover, appointments to distribute the questionnaires and to conduct
the interviews were decided during these visits.
A number of actions were undertaken to optimise the collection of
questionnaire data. At each visit the researcher delivered the
questionnaire to the participants' offices to make sure that the
questionnaire would be completed by them. A deadline for completing the
questionnaire was identified by asking the participant when s/he would
be able to complete the document. The researcher left a phone number to
advise when the questionnaire was ready to be collected, and during the
completion period the researcher phoned the participants to ask whether
or not they had finished. When collecting the questionnaire the
researcher made sure that she saw the participants who had completed the
questionnaire, and usually, she had the opportunities to have another
conversation (chat) about the questionnaire contents or any other
comments about the questionnaire. Some participants, after completing
the questionnaire, offered more documentary data which meant that they
were interested and understood the questionnaire.
The personal letter (covering letter) from the researcher urging
the participants to cooperate with the researcher was very useful to get
the participants' assistance in completing the questionnaire. In
this letter the researcher stressed that the information provided by the
participants would be treated with the utmost confidentiality and no
personal or organisations' names would be mentioned in the report
of the study. The researcher also stated in her letter that a brief
report of the final analysis would be given to the participant if
requested.
Appointments with top managers to conduct semi structured
interviews were set up by the researcher's effort through visiting
the secretaries' offices and building trust with them to set
appointments with their bosses, or by questionnaire participants in each
organisation. However, 'follow up' telephone calls were made
to the secretaries' offices to secure the appointments with the top
managers. All the interviews in this study were conducted in the
respondents' offices either in the morning or in the afternoon.
Nevertheless, the interviewees chose the place and the time of
interviews, since they were so busy it was a good decision to conduct
the interviews at times convenient to them. The schedule allowed the
researcher to spend between one and two hours maximum with each
respondent. The questions were mainly open ended with some close ended
questions. In conducting the semi structured interviews, the researcher
took notes in addition to recording the whole interview in order to
remember all the details. In each interview the researcher thanked the
respondents for their time, gave a brief overview of the research
provided and exchanged business cards. The researcher assured utmost
confidentially of the provided data. Finally, it is worth mentioning
that close questions were written on cards to be provided to the
interviewees giving them the opportunity to read these questions to
avoid any kind of bias.
A total of 38 questionnaires were returned which was about 86 per
cent response rate and 15 interviews were conducted by the end of July
2004. The reason for the realtively high response rate may be due to the
fact that the questionnaire were distributed and collected by the
researcher in addition to her 'follow up' letter and visits to
the banks. A total of four questionnaires were returned from two foreign
banks with a letter explaining that their bank policy does not allow
them respond to surveys and two questionnaires were unusable because of
missing data.
In sum, 22 banks were the population of the current study, 44
questionnaires targeted T&D and HRM managers in each bank and 15
semi structured interviews were done with 15 top managers. Despite the
intention to interview 22 top managers (of whom 15 responded). The
inputs from the executive management can be considered as favourable.
The other managers declined to take part in this study because they were
too busy, or because their banks' policies do not allow them to
give any information about T&D policies and management.
Measures
This study employed two primary data collection procedures (a) self
administered questionnaires distributed to T&D and HRM managers, and
(b) semi structured interviews conducted with top managers. The items
included in the questionnaire were adapted from previous research
studies (Garavan 1991, Albahussain 2000, Al-Atheri 2000, Al-Athari &
Zairi 2002, Tennant, et al. 2002) that were related to T&D and HRM.
The semi structured interviews assessed four dimensions of T&D
evaluation. These elements were (a) the position of T&D evaluation
in the organisations, (b) the significance of T&D evaluation stage,
(c) T&D evaluation models and criteria, and (d) T&D evaluation
problems and obstacles. Appendix 2 shows the items that were employed in
the questionnaire to obtain the self administered survey data and the
questions of the semi structured interviews.
The self administered questionnaire has six main sections. The
first section contains one item measuring the importance of T&D
evaluation stage in Jordanian banks using a five point scale (that
ranged from 1 = Not at all important, to 5 = very important). The second
section contains an item that measures the frequency of conducting
T&D evaluation using a five point scale (that ranged from 1 = never
to 5 = always). The third section of the questionnaire has five items
that measure evaluation methods and techniques using a five point scale
(that ranged from 1 = never to 5 = always). The fourth and fifth
sections were about T&D evaluation models and criteria that were
assessed with a five point scale (that ranged from 1 = to a very small
extent to 5 = to a very great extent). Finally, the last section
contains 12 items asking for T&D evaluation challenges and problems
in Jordanian banks using a five point Likert scale (that ranged from 1 =
strongly disagree to 5 = strongly agree).
Analysis
The qualitative data of the interviews was evaluated by content
analysis. Initially, the data are partitioned. This step is followed by
recognising relationships and developing categories by designing or
reorganising the data in a suitable matrix and placing the information
within cells.
Descriptive statistics include frequencies, measure of central
tendency (mean, median and mode) and measure of dispersion (range,
Standard deviation) was the most appropriate statistic for analysing
questionnaire questions. In addition, Kruskal-Wallis H test and
Chi-Square were used to establish the difference among bank groups.
RESULTS
This section presents the results in two parts. First, is given
findings from the questionnaire responses (quantitative data). Second,
the results of the responses from the interviews (qualitative data) are
delineated.
Quantitative
In this study the four Jordanian banking groups participated in
this study as shown in Table 1. Commercial banks represent the majority
of the participating banks (47.4 per cent). All of the investment and
the Islamic banks participated in this study; they represented 26.3 per
cent and 10.5 per cent of the study participants, respectively. Three of
the foreign banks agreed to participate, which represented 16 per cent
of the overall participants.
Table 1 also shows that all the questionnaire participants were the
managers of T&D or those who were in charge of T&D activities in
each bank, in addition to the managers of HRM and HR department in each
bank. A total of 66 per cent of them categorised their job status in the
middle management levels, as functional managers or senior non board
managers in their organisations. However, the Chi-Square test showed the
following result: [chi square] = 21.235, df = 9, sig = .012, p <
0.05, which indicates that there was a significant difference among bank
categories in terms of participants' status in the banks, ranking
foreign and commercial banks in the first and second levels, which means
that people responsible for T&D were in the higher status within
foreign and commercial banks. A further feature of the data is that 68
per cent of the participants had work experience in the field of T&D
of between one and seven years. The overall average work experience in
the field of T&D in the Jordanian banks was just 8.4 years. The
Kruskal Wallis H test result was [chi square] = 3.504, df = 3, sig =
.320, p < 0.05, which implies that there was a significant difference
among bank groups in terms of their participants' work experience
in the field of T&D, ranking the commercial banks and the Islamic
banks T&D work experience in the first and second levels. Indicating
that the people who were in charge of T&D issues in the commercial
and Islamic banks were more experienced in the field of T&D than in
other bank groups.
Table 2 lists mean scores and standard deviations for the other
items used in this study described in the measure sub section and shown
in Appendix 2. Regarding to the importance of T&D evaluation stage
in the participated organisations, Table 2 shows that most of the
participants asserted that their organisations viewed training
evaluation as a very important stage, (mean of 4.52). The Kruskal-Wallis
H test shows that there was a non significant difference among bank
groups in terms of whether they view training evaluation as important or
not ([chi square] = 6.553, df = 3, sig = .088, p > 0.05). In
addition, Table 2 shows that most of the participants believed that
their organisations evaluate their T&D programmes on a regular
basis, with a non significant difference among bank groups in terms of
the frequencies of conducting training evaluation ([chi square] = 6.553,
df = 3, sig = .088, p > 0.05).
In terms of the T&D evaluation methods and models in Jordanian
banks Table 2 shows that most of the participants agreed that completing
a questionnaire at the end of any training programmes was the most
frequent used evaluation method (mean 4.47). This result compares
favourably with a study that was conducted by Al-Ali (1999) who also
found that the most commonly used evaluation method by Kuwaiti
organisations was the questionnaire. In addition, about 40 per cent of
the participants declared that they almost rarely tested the trainees
before and after any training programmes, while 16 per cent said they
never did. Table 2 also shows a mean score of 3.47 for the Kirkpatrick
evaluation model, which is used to a considerable extent in the
organisations as revealed by the non significant difference among bank
groups in the mean ranking frequencies of using the Kirkpatrick
evaluation model ([chi square] = . 545, df = 3, sig = . 959, p <
0.05).
Table 2 shows that most of the participants stated, that when
evaluating the effectiveness of T&D programmes, they depend firstly
and greatly on trainees' reaction level given an average of 4.50.
This observation suggests that trainees reaction is the level which is
focused on to a very great extent when evaluating T&D programmes. In
addition, 63 per cent of the respondents affirmed that they depend
greatly on evaluating learning outcomes level, while 50 per cent of the
participants agreed that their organisations depend to some extent on
behavioural level when evaluating T&D effectiveness, yet the Results
had a value of 2.39, which indicated that this level is neglected when
evaluating T&D programmes.
Table 2 also shows that most of the participants agreed T&D
evaluation and 'follow up' stages face many difficulties and
challenges. The prominent items were (a) lack of quantitative measures
(4.03), (b) difficulties in measuring the change of trainees'
behaviours over a short period of time (3.71), (c) difficulties in
separating the impact of training on the final results from other
activities impacts (3.50), and (d) difficulties in getting managers to
participate in the evaluation process (3.29). Alternatively,
participants disagreed that the absence of job description and the lack
of knowledge about evaluation were difficulties facing evaluation
process in the Jordanian banks. These findings regarding to T&D
difficulties in the Jordanian banks are consistent with other research
findings (Atiyyah 1993, Abdalla & Al-Homoud 1995, Al-Athari &
Zairi 2002).
Qualitative
A total of 15 executive managers provided qualitative data. Six top
managers were from six commercial banks, one manager was from a foreign
bank, and a further six top managers were from investment banks, and two
top managers were from the two Jordanian Islamic banks. All of the
interviewed managers believed that T&D evaluation is a very
important stage. For instance, two of them stated,
"Training is a long term investment. We spend a lot of money
and time on it, thus we need to evaluate its results ..."
(Commercial bank)
"We actually spend a lot of money on T&D every year. We
offer all facilities for T&D activities, but it seems that these
programmes do not work and we do not know why?" (Islamic bank)
All of the interviewed top managers declared that they evaluate
their T&D regularly. However, 13 top managers stated that external
providers conduct most of the T&D programmes outside the banks. And
furthermore, these banks depend on those external providers to do the
evaluation for them. In this context two of the interviewed managers
said,
"T&D evaluation is conducted by the external centre which
conducts T&D programmes and I think they do it through
questionnaire." (Commercial bank)
"Thus, after attending T&D programmes, our employees came
back to their workplace trained and evaluated. Actually, we indeed,
trust those external providers because they have long work experiences,
they are specialists or professionals." (Investment bank)
The top managers held different perspectives about how T&D
programmes were evaluated. Some managers stated that in most times
external providers evaluated their T&D programmes, but most of the
managers were not sure whether those providers depend on the Kirkpatrick
model or not. However, when they were asked about T&D evaluation
methods, criteria or levels, most of them knew that external providers
evaluate T&D programmes through distribution of a particular
questionnaire for each trainee at the end of training session asking for
their opinions of the programmes. Also, the evaluations of the T&D
programmes ask trainers to fill in a particular questionnaire and to
evaluate the course as well as the trainees. Some top managers said
trainers observe the trainees rather than filling in a questionnaire. In
respect to this matter one comment was,
"In evaluating the trainees we depend on their reaction toward
the T&D course in addition to asking the trainers about the
trainees, what is more we sometimes ask those trainees to do a
presentation for what he/she learnt from the course, but of course that
is not always." (Commercial bank)
Finally, managers were asked about what were T&D evaluation
challenges and problems. The responses from these managers showed a
belief there were two important challenges facing evaluation process,
which were 1) lack of qualified internal T&D staff; and 2)
difficulties in measuring T&D outcomes. But the managers did not
consider leaving external providers to evaluate their T&D programmes
was a substantive issue.
DISCUSSION
The results of this study provide some disparate findings in terms
of evaluated dimensions of T&D programmes in Jordanian banks.
Despite a considerable extent of projection by a great number of
commentators (Garavan 1991, Bee & Bee 1994, Taylor 1996, Harrison
1997, Redshaw 2000, Russ & Preskill 2001, Burrow & Berardinelli
2003, Hale 2003), who have evaluated procedures that show there are
responsible elements of T&D initiatives, the study findings
disclose, that many managers (of these banks) have an ambivalence about
this part of the T&D programmes. Indeed, often bank policy is to
outsource the activity and then to defer responsibility for the
evaluation of the proceedings to the same body that is administering the
T&D programmes.
In order to understand how the evaluation stage is considered and
conducted in Jordanian banks, the study targeted all Jordanian banks.
Within these banks those people who are in charge of the T&D
programmes were the study respondents. These respondents were asked
about the level of importance of the evaluation process in their
organisations, the regularity of conducting evaluation, the evaluation
methods and techniques used in their organisations, the evaluation model
and levels of outcomes they depend on when evaluating T&D programmes
and the challenges and difficulties that face of evaluating the T&D
programmes, conducted by their banks. Also, executive managers were
asked about the evaluation stage in many contexts during their
interviews.
Interestingly, it was found, that all participating banks
represented by (T&D and HRM managers in each bank, the questionnaire
participants) believed the T&D evaluation is an important stage.
Therefore, the majority of Jordanian banks conducted T&D evaluation
on a regular basis. These findings, together with earlier literature
suggest that in Jordanian banks T&D is conducted more frequently
than in other Arab organisations. In fact, earlier Al-Athari (2000)
found that although the majority of the government and private Kuwaiti
organisations usually conducted T&D evaluation, 35 per cent of
government and 19 per cent of private organisations rarely evaluate
their T&D programmes. In addition, Albahussain (2000) found that
just nine per cent of the medium sized and 31 per cent of the large
sized surveyed Saudi organisations conducted evaluation on a regular
basis, whereas the majority of organisations conduct evaluative
procedures only sometime. Consequently, it might be inferred that
Jordanian banks believed in the importance of T&D evaluation and
conducted these procedures regularly.
The study revealed the most commonly used evaluation method by
Jordanian banks is the questionnaire. According to T&D and HRM
managers, questionnaires are used to measure perceptual responses from
trainees' managers or supervisors for their observation and
assessment of the trainees' learning as a result of attending
training programmes. Interviewing the trainees was occasionally used as
an evaluation method, while testing the trainees before and after the
programme was the most rarely used evaluation technique. Interestingly,
it was found from the interviews that some commercial banks had their
own ways of evaluating T&D programmes. One way was by sending
particular customers to particular branches to observe how employees
conduct their jobs and see how they serve the customers. Another manager
said they ask the trainees to do presentations of what they get from
T&D programmes. Yet another manager refused to talk about their own
ways of evaluating T&D outcomes since they consider them highly
confidential issues. However, all of these methods of evaluation are
subjective in nature and their assessment value for T&D
effectiveness is subject to personal aspects and preferences.
The interviewed top managers declared that most external T&D
programmes are evaluated by the external providers. Wilkins (2001)
argues, that in Arab countries, educational institutions are not
generally in the position to assess workplace performance so must find
other ways to measure the effectiveness of their programmes. In
Jordanian banks they know that external providers evaluate T&D
programmes through distributing a particular questionnaire for each
trainee asking for their opinions of the programmes. Another way to
evaluate external T&D programmes, in addition to asking the external
providers to evaluate them, is to ask the trainees' direct managers
about the trainees' performance after they have finished the
programmes. In addition, asking external providers to evaluate T&D
programmes reflects that Jordanian banks do not know exactly what they
want from a T&D programme.
The choice of an evaluation model to determine the effectiveness of
a T&D programme is critical. An evaluation model is a systematic way
to benchmark the organisation's training activity to find out how
much the organisation benefited from the T&D programme, what the
employees' reaction is, what they learned and how much they learned
and if they improved their performance, work related behaviours and
attitudes. In Jordanian banks this study found that most of the
participants (the people who are in charge of T&D) were not familiar
with the international evaluation models that were provided. However,
when the researcher defined each model and identified the criteria for
each evaluation model, the majority of them thought they used the
Kirkpatrick model criteria when evaluating their in house T&D
programmes. The unfamiliarity of the evaluation models reflects how much
they know about how to conduct an effective evaluation process by using
a systematic evaluation model. The shortage of knowledge of the
evaluation models could be explained by many reasons. The over
dependency on the external providers to evaluate a T&D course, lack
of well qualified T&D staff who are supposed to be familiar with,
and knowledge about evaluation issues, the lack of a T&D department
or section in most of the banks, and finally, because some banks found
their own way of evaluating their T&D programmes.
Results of the study showed the most common level of evaluation for
Jordanian banks was the trainee's reaction level. However, some of
the commercial banks concentrate on the behavioural changes and the
Results outcomes. All of the interviewed top managers disagreed that
they focused on cost effectiveness training evaluation, because
investing in training is a long term investment with fewer tangible and
complex benefits. Nevertheless, it is likely that some successful large
commercial banks have recognised the importance of measuring the
benefits of T&D.
The study findings have a degree of concurrence with previous
research findings. The most commonly used evaluation method was found to
be the questionnaire, which included trainee's evaluation of
T&D programmes, which is often assessed on a 'smile
sheet'. In this context, Robinson and Robinson (1989) state that
almost all HRD professionals provide end of course questionnaires that
are completed by participants and given to the trainer. However, these
reaction evaluations, frequently, are poorly designed and yield
minimally useful information. Finally, in terms of evaluation problems
and challenges in Jordanian banks it was found that in this study and
according to the people who were in charge of T&D, the evaluation
stage faced some difficulties and challenges. These perplexing issues
can manifest as the lack of quantitative measures, difficulties in
measuring the change of trainees' behaviours over a short period of
time, difficulties in separating the impacts of training on
organisational final results from other the impacts of other
organisational activities, and difficulties in getting managers to
participate in the evaluation process. Interestingly, participants
disagreed that the absence of job description and lack of knowledge in
the evaluation process, were difficulties for the evaluation stage in
Jordanian banks. Moreover, the interviewed banks' top managers
revealed some challenges and problems related to the evaluation process.
They said that frequently there were no specific qualified people to
evaluate T&D effectiveness. Thus, the responsibility for training
evaluation was formally left to the training coordinators or external
providers, who usually evaluate the programmes just for reporting to the
banks.
CONCLUSION
This study examined the current management T&D evaluation
practices in Jordanian banks; and identified the most important
concerns, problems, and challenges that are facing the T&D
evaluation stage in these organisations. Investigating the Jordanian
banking industry, which has not been targeted before, within a caveat
that T&D practices are assumed to be managed better than in any
other Jordanian industry has potential to contribute relevant knowledge
to the existing literature. The research findings were strengthened by
employing interviews to elucidate the questionnaire data. Arguably, this
study shows that the T&D evaluation stage in the Jordanian banking
industry is not in a better situation than in other Arab organisations.
Also, there is a lack of sustainable evidence that the T&D
evaluation stage in the Jordanian banking industry is likely to be
conducted differently than when the activity is undertaken in other Arab
organisations. Furthermore, there is an absence of argument that T&D
in the Jordanian banking industry (as services organisations) should be
managed differently from other Arab organisations.
The study revealed that although all Jordanian banks believe in the
importance of the evaluation process they conduct somewhat limited
evaluation processes for their in house T&D programmes. Despite
regular evaluations of the T&D programmes these assessments are
constrained to the use of questionnaires, asking trainees' managers
or supervisors about their observation and assessments of trainees'
learning. These methods of evaluating trainees are subjective in nature,
and indeed, supervisors' assessment for T&D effectiveness can
be highly personalised.
In terms of the range of evaluation frameworks the Kirkpatrick
evaluation model is mainly used by Jordanian banks. Nevertheless, they
depend to a very large extent on the reaction level, which is about the
trainees' reaction, opinions, attitudes and satisfaction about the
T&D programmes, followed by learning outcomes which they evaluate
through asking the trainees' direct managers for their assessment
of the trainees' learning. Thus, evaluation is mainly based on
subjective assessments. These evaluations range across an
individual's perceptions, opinions and attitudes toward T&D
outcomes, rather than focusing on behavioural changes, organisational
relevant work practices and operational improvements. For practitioners
the revealed facts and findings have implications for contemporary
banking organisations. For instance, coordination between Jordanian
banks and external providers in managing their T&D programmes,
finding specific qualified people to be in charge of evaluation and/or
educating, and developing the people who are currently in charge of
T&D evaluation process. In addition, T&D evaluation should not
just be based on trainees' reactions, but rather it should be based
on learning and behavioural outcomes, which could be gained from
observing the trainees' performance after they complete T&D
programmes. Two important behavioural outcomes that might be observed
are whether they transfer what they learn from T&D into their
workplaces; and the extent to which on the job performance was improved.
Training evaluation faces many difficulties. A lack of quantitative
measures, difficulty in measuring the change of trainees' behaviour
over a short period of time, difficulty in separating training impact on
the final results from the impacts of other activities, difficulty in
getting managers to participate in the evaluation process, lack of
knowledge about evaluation, lack of job description, and time required
to do evaluation are some other difficulties. These issues are
exacerbated by a lack of well qualified people who are in charge of
T&D evaluation, lack of knowledge about the evaluation process and
over dependency on external providers to evaluate the T&D
programmes.
From a practical point of view, this research has provided several
major managerial contributions. As many organisations have suffered from
a low success rate with their T&D activities and most of them feel
that T&D does not work, this study has provided useful guidelines in
the form of the critical elements and factors that can enhance success
in T&D. The study results provide assessments for the effectiveness
of the current practices relating to the evaluation process of T&D
and reflect many of the related problems and challenges. From this study
T&D professionals have an opportunity to derive a better
understanding of the role of their T&D activities in the development
of the organisations, especially in today's business world, which
is characterised by ever increasing competition, globalisation and
change. The findings of this study may guide T&D professionals to
understand how they can better manage their T&D activities, what
they need to focus on, and what strategies may be employed to improve
T&D effectiveness.
The conclusions and implications derived from this study remain
tentative until such time as they are confirmed by similar studies using
identical or alternative research methodologies. A refinement of this
work would consist of verifying that all relevant and important aspects
of T&D have been covered thus, providing other researchers with an
opportunity to offer a more complete picture of the situation as it
stands. Arguably, the boundary conditions of this study could be viewed
as opportunities for future research.
Based on the results of this study suggestions are given for
organisations seeking to improve their T&D evaluation stage. For
instance, more attention and effort is required when evaluating T&D
programmes. Importantly, there is a need to evaluate the outcomes of
external T&D programmes rather than asking the external provider to
perform the examination phase. Organisations are encouraged to use a
systematic evaluation model. For more effective benefits an emphasis
might be to consider the Kirkpatrick evaluation model outcome levels
instead of an overemphasis on the reaction level, which do not provide
or reflect much about T&D effectiveness. Finally, evaluation needs
to be conducted based on the objectives of the planned T&D
programmes. The paradox for senior executives of the Jordanian banking
sector is how to become enlightened about the evaluative parameters of
T&D programmes that provide credible underpinning for generating
effectual T&D initiatives and valued outcomes.
APPENDICES
Appendix 2
Study Questionnaire and Semi Structured Interview Schedule with Top
Managers
Section A: Evaluation and follow up
A1. How important is measurement training evaluation in your bank?
[] Not at all important
[] Relatively unimportant
[] Somewhat important
[] Relatively important
[] Very important
A2. Does your bank evaluate training and development programmes?
[] Never
[] Rarely
[] Sometimes
[] Mostly
[] Always
A3. If your bank evaluates its training and development programmes,
what methods are usually used? Please tick ([check]) in the box that
best reflects your answer where:
1 = Never 2 = Rarely 3 = Sometimes 4 = Mostly 5 = Always
Evaluation tools and techniques 1 2 3 4 5
Asking employees to fill a questionnaire at the end of the
programme
Interviewing the trainees at the end of each training programme
Testing the trainees before and after the training programmes (pre
and post test)
Asking the trainees' managers or supervisors for their
assessment of the trainees' learning
Performance appraisal reports
Others, please specify....
A4. Please indicate which of the following training evaluation
models does your bank use when evaluating training and development
programmes? Please tick ([check]) in the box that best reflects your
answer where:
1 = To a very small extent
2 = To a small extent
3 = To a considerable extent
4 = To a great extent
5 = To a very great extent
Evaluation models 1 2 3 4 5
Kirkpatrick (trainees' reaction; learning outcomes; behaviour
outcomes, results the impact on profits, productivity, quality
CIRO (training context, training input, trainees' reaction;
and training outcomes)
Evaluation models 1 2 3 4 5
CIPP (training context, training input, training process, training
products or outcomes)
Other, please specify....
A5. To what extent do you evaluate the following levels of
outcomes? Please tick ([check]) in the box that best reflects your
answer where;
1 = To a very small extent
2 = To a small extent
3 = To a considerable extent
4 = To a great extent
5 = To a very great extent
Levels of outcomes 1 2 3 4 5
Trainees' reactions: Feeling and opinion of the trainees about
the programmes' material, facilities, methods, contents, trainers,
durations and relevance of the programmes.
Learning outcomes: The skills, knowledge and attitudes acquired
during the programme.
Behaviour change: The change in on the job performance, which can
be attached to the programme.
Results: The effect in the organisation's performance
resulting from the change of behaviour such as: cost saving, quality
improvement, customer satisfaction.
A6. To what extent do you agree or disagree with the following
possible difficulties and challenges that might face your bank when
evaluating training and development programmes? Please tick ([check]) in
the box that best reflects your answer where:
1 = Strongly disagree 2 = Disagree 3 = Not sure 4 = Agree 5 =
Strongly agree
Evaluation challenges and difficulties 1 2 3 4 5
Difficulties in measuring the performance's improvement in
certain jobs
Difficulties in measuring the change of trainees' behaviour
over a short period of time
Difficulties in identifying the appropriate quantitative measures
High cost evaluation process
Most of training outcomes are subjective in nature
Lack of knowledge about evaluation process
Absence of job description
Time required to do evaluation well
Difficulties to separate training influence on the final results
from other organisation's functions' influences
No specific body is in charge of evaluating the training programmes
Difficulties in getting managers to participate in evaluation
process
Training and development objectives are not clear
Evaluation challenges and difficulties 1 2 3 4 5
Other, please specify....
Section B: (General information). The purpose of this section is to
obtain general information related to your bank and yourself as a
participant in this research
B1. What is your bank category?
[] Commercial
[] Foreign
[] Investment
[] Islamic
B2. How long has your bank been in business for? [] Years
B3. Please indicate approximately, how many employees does your
bank employ in total? [] Employees
B4. Please indicate your present job title? (Please tick one)
1 Manager of training department/unit
2 Manager of human resource department
3 Senior manager
4 Other, please specify....
B5. Please indicate your status within the bank? (please tick one)
1 Top management (e.g. CEO, MD, or other Board member)
2 Middle management (e.g. functional manager, or other senior non
board member)
3 Lower management/ supervisory
4 Professional /technical
5 Other, please specify....
B6. How many years of work experience do you have in total? []
Years
B7. How many years of work experience do you have in the field of
training and development? [] Years
B8. Please indicate your highest education level: (please tick one)
1 High school
2 Community/ technical college
3 University Bachelor's degree in (please specify)....
4 Master degree or equivalent in (please specify)....
5 PhD or equivalent in (please specify)....
6 Other, please specify ....
B9. What is your age? []
B10. What is your sex?
[] Male
[] Female
Semi structured interview schedule with top managers
1. The position of T&D evaluation in the organisation
Q1. In general, are you satisfied with overall T&D evaluation
stage in your bank? Why? Q2. Do you have a T&D department in the
bank? If no, why? Do you think T&D function and programmes are
important for the bank's success? If yes why? Alternatively, do you
believe in the importance of T&D evaluation? Q3. Do you conduct
T&D evaluation? If yes, how and who conduct it? Q4. Could you please
give me your opinion in the following statement: T&D is a waste of
money and time?
2. The significance of T&D evaluation stage
Q5. Do you conduct T&D evaluation to measure T&D
effectiveness function? How often do you conduct T&D evaluation? How
and who conduct it? Q6. Do you use particular model when evaluating
T&D programme? What are the criteria that you focus on when
evaluating T&D programme? Q7. Do you think T&D influences the
bank performance in any way? If yes how. If no why? Q8. How significant
is the impact of T&D in relation to the following areas: 1 = high
impact 2 = low impact 3 = do not know 4 = no impact
3. T&D evaluation main problems and obstacles
Q9. What are the main problems of T&D evaluation in your bank?
Q10. To what extent do you agree or disagree with the following reasons
that might be challenges for T&D evaluation stage within your
organisation. 1 = strongly disagree, 2 = disagree, 3 = undecided, 4 =
agree, 5 = strongly agree. 1. [] High cost of T&D evaluation process
2. [] Lack of training centre 3. [] Lack of separate T&D department
4. [] Lack of experienced training staff 5. [] Difficulty in measuring
T&D outcomes 6. [] Training programmes introduced from external
sources off the house
REFERENCES
Abdalla, I., & Al-Homoud, M. (1995). A Survey of management
training and development practices in the state of Kuwait. Journal of
Management Development, 14(3), 14-25.
Abdalla, H., Maghrabi, A., & Raggad, B. (1998). Assessing the
perceptions of human resource managers toward nepotism: A cross-cultural
study. International Journal of Manpower, 19(8), 554-570.
Acton, T., & Golden, W. (2003). Training the knowledge worker:
a descriptive study of training practices in Irish software companies.
Journal of European Industrial Training, 27(4), 137-146.
Al-Athari, A. (2000). The impact of multimedia based training on
employee training effectiveness and organisational performance.
Unpublished PhD Thesis, Bradford University.
Al-Athari, A., & Zairi, M. (2002). Training evaluation: An
empirical study in Kuwait. Journal of European Industrial Training,
26(5), 241-251.
Al-Bahussain, A. (2000). Human resource development: An
investigation into the nature and extent of training and development in
the Saudi private manufacturing. Unpublished PhD thesis, Bradford
University.
Altarawneh, I. (2005). Training and development effectiveness:
Practices, roles and impacts on performance in Jordanian banking
organisations. Unpublished PhD thesis, Huddersfield University, UK.
Atiyyah, H. (1993). Management development in Arab countries: The
challenges of the 1990s. Journal of Management Development, 12(1), 3-12.
Armstrong, M. (1989). Personnel and the bottom line. London:
Institute of Personnel Management.
Bahar, A., Peterson, S., & Taylor, D. (1996). Managing training
and development in Bahrain: The influence of culture. Journal of
Managerial Psychology, 11(5), 26-32.
Beardwell, I., & Holden, L. (1994). Human resource management:
A contemporary perspective. London: Pitman Publishing.
Bee, F., & Bee, R. (1994). Training needs analysis and
evaluation. London: Institute of Personnel Management.
Bramley, P., & Kitson, B. (1994). Evaluating training against
business criteria. Journal of European Industrial Training, 18(1),
10-14.
Brinkerhoff, R. (1988). Achieving results from training. San
Francisco: Jossey-Bass.
Burrow J., & Berardinelli, P. (2003). Systematic performance
improvement-refining the space between learning and results. Journal of
Workplace Learning, 15(1), 6-13.
Combs, W., & Falletta, S. (2000). The targeted evaluation
process. Alexandria, VA: American Society for Training and Development.
Collins, D. (2002). Performance-level evaluation methods used in
management development studies from 19862000. Human Resource Development
Review, 1(1), 91-110.
Foxon, M. (1986). Evaluation of training: The art of the
impossible. Training Officer, May, 133-137.
Garavan, N. (1991). Strategic human resource development. Journal
of European Industrial Training, 15(1), 17-30.
Green, E. (2001). The influence of individual and work environment
characteristics on trainee motivation and training effectiveness
measures. Unpublished Doctoral Dissertation, North Carolina State
University, Raleigh, NC.
Grove, A., & Ostroff, C. (1990). Training programme evaluation.
In Wexley, N., & Hinrichs, R., Developing human resources, ASPA/BNA
series, Washington, DC: Bureau of National Affairs.
Hale, R. (2003). How training can add real value to the business.
Industrial and Commercial Training, 35(2), 49-52.
Harrison, R. (1997). Employee development. London: Institute of
Personnel and Development.
Horwitz, M. (1999). The emergence of strategic training and
development: The current state of play. Journal of European Industrial
Training, 23(4)180-190.
Houlton, E. (1996). The flawed four-level evaluation model. Human
Resource Development Quarterly, 7(1), 5-22.
James, C., & Roffe, I. (2000). The evaluation of goal and
goal-free training innovation. Journal of European Industrial Training,
24(1), 12-20.
Kirkpatrick, D. (1996). Four levels of evaluation. Journal of
American Society for Training and Development, 33(2), 78-92.
Mann, S. (1996). What should training evaluations evaluate? Journal
of European Industrial Training, 20(9), 14-20.
Manpower Service Commission (1981). Glossary of training term.
London: HMSO.
Miller, L., (2002). Perception of training and non-training
managers of organisational impact measures based on design intent.
Unpublished Doctoral Dissertation, North Carolina State University.
Muhlemeyer, P., & Clarke, M.(1997). The competitive factor:
Training and development as a strategic management task. Journal of
Workplace Learning, 9(1), 4-11.
Redshaw, B. (2000). Evaluating organisational effectiveness.
Industrial and Commercial Training, 32(7), 245-248.
Robinson, G., & Robinson, C. (1989). Training for impact.
California: Jossey-Bass.
Russ, D., & Preskill, H. (2001). Evaluation in organisation.
Cambridge, MA: Pereseus Publishing.
Shandler, D. (1996). Reengineering the training function. Delray
Beach: St Lucie Press.
Tennant, C., Boonkrong, M., & Roberts, P. (2002). The design of
a training programme measurement model. Journal of European Industrial
Training, 26(5), 230-240.
Taylor, D. (1996). Training and developing people. In Molander C.
Human resource at work (26-32). London: Chartwell Bratt.
Wilkins, S. (2001). Management development in the Arab Gulf States:
The influence of language and culture. Industrial and Commercial
Training, 33(7), 260-266.
Yeo, R. (2003). Linking organisational learning to organisational
performance and success: Singapore case studies. Leadership and
Organisation Development Journal, 24(2), 70-83.
Ikhlas Altarawneh, PhD, is an assistant professor in Al Hussien Bin
Talal University, department of business, business and economics school,
Jordan. She did her Master and PhD degrees in UK. She has particular
research interests in assessment and planning human resource management,
training and development, women leadership, job satisfaction and
performance appraisal.
Emails : dr.ikhlas@ahu.edu.jo; eklastara@yahoo.com
Appendix 1
The Study Population
A. Commercial banks Arab Bank PLC Arab Banking
Corporation (Jordan) Bank of Jordan PLC Cairo
Amman Bank Export & Finance Bank Jordan Gulf
Bank Jordan Kuwait Bank Jordan National Bank PLC
The Housing Bank for Trade & Finance
Commercial banks perform general financial activities,
such as: financing working capital, term finance, import
and export activities, and deposit services. They also
allocate and invest financial resources in a variety of
economic sectors. Most of the operating banks are
commercial banks.
B. Foreign banks Standard Chartered Grindlays
Bank Egyptian Arab Land Bank HSBC Bank Middle
East CitiBank Rafidain Bank
Foreign banks are branches for foreign international banks
investing in Jordan, but still directed by foreign
management. Two of them are British, one American, one
Egyptian and one Iraqi. All of them are commercial in
nature.
C. Islamic banks Islamic International Arab Bank
PLC Jordan Islamic Bank for Finance and Investment
Islamic banks are characterised by an approach that avoids
the payment and receipt of interest.
D. Investment banks Arab Jordan Investment Bank
Jordan Investment and Finance Bank Philadelphia
Investment Bank Union Bank for Saving & Investment
Industrial Development Bank Middle East Investment
Bank Societe Generale de Banque au Liban
Investment banks offer long term lending to a wide variety
of economic sectors. They finance equity for start-ups, and
thus, contribute to enlarging business activities in Jordan.
Source: The Association of Banks in Jordan, Central Bank of
Jordan, 2004.
Table 1 Demographics % (N = 38)
Bank category
Commercial 47.4
Foreign 15.8
Investment 26.3
Islamic 10.5
Job titles
Manager of T&D department 50.0
Manager of HRM 50.0
Participant status
Top management 7.9
Middle management 65.8
Lower management 18.4
Professional/technical 7.9
Work experiences (years)
1-8 15.8
9-16 23.7
17-24 34.2
25-32 21.1
33-40 5.2
Table 2 Mean scores, standard deviation for the study main variables
Study's variables Mean Std.
How important is measurement training
evaluation in your bank 4.52 0.60
How frequently does your bank evaluate
training and development 4.23 0.88
Evaluation methods
Filling in a questionnaire 4.47 0.95
Interviewing the trainees at the end
of programmes 2.61 0.75
Testing the trainees before and after
the programme 2.37 0.85
Asking trainees' managers or
supervisors for their assessment of
the trainees' learning 3.82 0.98
Employees' performance records 3.71 0.96
Evaluation models
Kirkpatrick 3.47 1.33
GIRO model 2.50 1.20
CIPP model 2.34 1.19
Evaluation levels
Trainees' reactions 4.50 0.69
Learning outcomes 3.74 0.86
Behaviour change 3.47 0.83
Results 2.39 1.00
Evaluation difficulties and challenges
Difficulties in measuring improvement
in a certain jobs 3.05 1.06
Difficulties in measuring the change
of trainees' behaviours over short
period of time 3.71 0.73
Lack of quantitative measures 4.03 0.49
High cost evaluation process 2.66 0.94
Training outcomes are subjective 3.11 1.01
Lack of knowledge about evaluation
process 2.71 1.25
Absence of job description 2.56 1.18
Time required to do evaluation 2.84 1.08
Difficulties to separate training
impact on the final results from
other actives impacts 3.50 0.80
No specific body in charge of
evaluation 2.90 1.20