An IRT Analysis Of The Indonesian Version Of Emotional Intelligence Questionnaire Among Indonesian Teachers
The unidimensional graded response model, an item response theory (IRT) model, can be used to improve the assessment of psychological trait and evaluate the psychometric properties of the scale. Typically, psychological measurement utilizes ‘classical’ statistical techniques (e. g. , reliability and factor analysis). However, implementing IRT-based methods can provide more details about individual questionnaire items, which is useful when determining the final item content of assessment. An analysis was conducted using an emotional intelligence questionnaire which was consisted 20 items for Indonesian teachers using data from previous research conduct by Lutfiyah (2018) highlights the main components for implementing the unidimensional graded response model. High school teachers (n = 303) completed the Emotional Intelligence Inteventory (EQ-i). Results from the unidimensional graded response model supported a unidimensional scoring process for the measurement. 13 items were deemed as worthy contributors to the measurement and 7 items wasn’t having good psychometrics properties. The graded response model can accommodate unidimensional scales and also can be used with relatively lower sample sizes. Use of this model can help to improve the quality of psychological-based scales being developed within the Educational Psychology theoretical framework.
Introduction
Becoming an emotionally intelligent teacher is a journey and process, not an arrival state or end result. Emotionally intelligent teachers are active in their orientation to students, work, and life. They are resilient in response to negative stress and less likely to overwhelm themselves with pessimism and strong, negative emotions. A growing number of studies have suggested that teachers' personal competencies, and more specifically Emotional Intelligence (EI), are particularly important for teacher effectiveness. Results typically indicated positive impacts, among them increased recognition of the importance of EI to schools; increased use of emotional information, both own emotions and those of students, in lesson plans and in the classroom; enhancing teachers’ sensitivity to students’ emotions in different situations; increasing their ability to respond constructively to students’ social-emotional needs; and acquiring SEL strategies.
However, Boyd (2005) and Corcoran and Tormey (2010) failed to find significant change in teachers’ own EI levels, attributing the results to the insufficient length of training or to the fact that it took place with teaching students rather than active teachers. Furthermore, it has been suggested that EI development should become part of general professional development programmes for teachers. Developing EI competencies could enable teachers to better understand what underlies their motivations and behaviours, and has the potential to enhance less-developed competencies; contribute to greater understanding of students’ emotions; improve teacher-student relationships; and promote effective teaching. But in Indonesia, it was so hard to find some research about this problem. So, the purpose of this study are to validate the measurement of Emotional Intelligence among Indonesian teachers using sophisticated methods in psychological measurement and also make an interpretation about the level of their emotional intelligence based on information from the measurement.
Defining Emotional Intelligence
David Caruso (2003) is a leading EQ thinker, test author, and practitioner. He had this to say about attempts to define EQ: “Just what is this thing called emotional intelligence (EI)? The answer, to a large extent, depends on who you ask. EI has served as a sort of conceptual inkblot [emphasis added], an unstructured notion that is open to a vast number of interpretations”. In other words, a number of people have taken their favorite ideas and called them EQ. So thereare numerous definitions, models, and related assessment tools. Today, three models dominate the EQ landscape: those developed by (1) Peter Salovey and JackMayer, and further refined in collaboration with David Caruso, (2) Daniel Goleman, and (3) ReuvenBar-On. Each defines EQ somewhat differently. Just as the definition of cognitive intelligence has been a moving target for the past century, thedefinition of EQ has varied, and depending on who defines it. Each definition has merit. It is prematureand probably unnecessary to settle on a universally accepted definition at this point. Different waysof thinking about EQ lead to different lines of research and practice, all of which promote learning (Ackley, 2016).
Measurement tools of EI
The Society of Psychologists in Management (SPIM) presented a workshop on measuring EQ. Two Air Force psychologists, Major George Munkachy, PhD, and Captain Keith White, PhD, reviewed the then-current state of development of EQ measures. The Mayer-Salovey-Caruso Emotional Intelligence Test (MSCEIT) — the Salovey, Mayer, Caruso instrument — would not be published until 2002, although research versions were available in 1999. The original Emotional Competency Inventory (ECI), based on Goleman’s work, and Bar-On’s measure, the Emotional Quotient Inventory (EQ-i), had been published about two years earlier. With respect to other ‘old’ measurement tools, in this study we only used the later.
Methods
Participants
The participants of this study were teachers in the province of Jakarta, amounting to 303 people who were selected with nonprobability sampling technique. The consideration in using the sampling technique is because of the limitations in terms of time to create a sampling frame that contains data about the teacher, so that the sampling technique that allows it to be used is nonprobability. The willingness of respondents to participate in this research is voluntary.
Instruments
The research instrument in this study was the Indonesian version of The Bar-On Emotional Quotient Inventory (EQ-i). The EQ-i 2. 0 user’s guide is impressive in its coverage of conceptual, practical (administration, scoring, interpretation, application to intervention), and technical aspects of the EQ-i 2. 0. Equally impressive arethe development efforts, pilot and standardization samples, and aspects of the psychometric evidence (e. g. , reliability, scale structure, convergent validity, and expected differences between groups). Somequestions remain, however, regarding structural validity evidence for the overall model that guided thedevelopment of the scale. In addition, data from predictive validity studies linking scores and recommendedinterventions (e. g. , coaching) to outcomes are needed to strengthen support for the use of thisinventory as a professional and personal development tool.
The items tested were 20 items from the selection of items that matched the condition of Indonesia. The translation results are tested for the readability aspect and the suitability of the content by the author with the help of a lecturer in the field of Psychology. The data collection process is carried out by including this questionnaire into data collection conducted by students of the Faculty of Psychology UIN Jakarta who are completing their thesis. The 20 items used have a 4-point Likert scale with choice of answers: Strongly Disagree (STS), Disagree (TS), Agree (S), and Strongly Agree (SS). Furthermore, the data that has been collected is analyzed with the Graded Response Model method with the aim to get information about the characteristics of the psychometric aspects of the adaptation items, before interpreting the emotional intelligence level score as for the GRM method will be explained below.
Graded Response Model (GRM)
The model that will be used to describe the IRT based on statistical modeling in this study is the Graded Response Model which will be estimated by the method of marginal maximum likelihood estimation (MML) with the IRTPRO 3 program. GRM is an IRT model used when measurements are made on an ordinal scale such as a Likert scale. GRM is also known as indirect IRT model because it is different from the model for dichotomous data, the probability of selecting a certain response category cannot be done directly with formula of 2-PL IRT so that to be able to calculate it in each response category.
The overall model fit index used is RMSEA with a value of 0. 05-0. 08 indicating that the model is acceptable fit, then the results of data analysis performed on the results of measurements with EQ-i will be explained.
Results and discussion
The results of the analysis of 20 items EQ-i showed that the model fit was obtained with RMSEA = 0. 08, the unidimensionality assumption of GRM in this study was fulfilled. If the model is not fit, then the use of GRM needs to be changed to another model. After obtaining that the model has been fit with the data, then the interpretation can be done on each item parameter. As can be seen there is no reverse threshold where from the lowest threshold the value continues to rise to the highest threshold and applies to all items. Although the overall model is fit for GRM, there are some items that are not fit, which can be seen based on the S-χ ^ 2 statistical test there are 7 items whose p-value is still in below 0. 05, but it should be noted that the interpretation of the fit or not of an item is not the same interpretation as CFA, items that have significant p-values are not excluded from the analysis but need further analysis which generally use standardized residual information which has not been used in this study.
The use of GRM also produces estimates of TIC. TIC provides information about the estimated reliability of the test (information value) for each level in the latent trait. These values are the same as the inverse of the standard error where the higher the value shows the more reliable estimation results. Along the range of -3. 00 logit (3 SD below the mean of trait) to -2. 5 logit (1. 5 SD below the mean of trait) the amount of information from the test is greater than 9 where it can be seen that the standard estimate the error is in the value value below 0. 40 which means that this instrument is very informative in almost all ranges of trait. This means that the measurement with the EQ-i is a very good measuring tool for measuring respondents who are in a very low range of trait where when the position is very high, the accuracy decreases. From a methodological perspective, because there are quite a number of studies in Indonesia that apply CFA as a method to test construct validity as well as to test psychometric characteristics of measuring instruments in the field of psychology, this study provides a new color to provide an overview of the application of GRM which has been created as an alternative for almost 50 years. The choice to know the psychometric properties of the measuring instrument used. Testing of psychometric characteristics of EQ-i shows the stages in analyzing or evaluating the characteristics of the measuring instrument by using the IRT fit model index which is required in reporting the results of the analysis using IRT.
Conclusion
Based on the results of this study, it can be concluded that the EQ-i measuring instrument shows good and proven psychometric characteristics that can be used to measure EI and also proved to be unidimensional with a fit model index that is proven fit. Overall this model can be applied to future research using the same analysis method. This is a great novelty for Indonesia that the positive psychology must have a greater attention, because this field are really useful to understand so many problems that so hard to explain without the usefulness of good instruments with excellent psychometrics characteristics. We hope that this questionnaire can be used for other research in Indonesia.
