Measuring Generic Competences in College Students. Psychometric Properties of the Generic Competences Questionnaire
Keywords:
Student evaluation, Skills development, Testing, University education.Abstract
In this paper is presented the development of the Students’ Generic Competences Questionnaire, as well as its initial metric properties. The questionnaire is based on the competencies identified by the Tuning project within the EEES frame. Standard Behavioral Observation Scales are used to develop it. In a first study (N = 741) is set to test the reliability of the measurements obtained, its content validity and dimensional structure. The results show that the scales making up the questionnaire have adequate reliability and high rates of consistency, and dimensional structure model seems to reproduce the starting model. In a second study (N = 46) the criterion validity of the questionnaire versus academic results is explored. The results show that different competencies assessed by the questionnaire correlated significantly with the results obtained by students in an objective knowledge test and continuous assessment indicators. These skills significantly increase the explained variance in academic qualifications by the personality and intelligence. Together this data allows us to guarantee the reliability and validity of measurements obtained with the questionnaire.
Downloads
References
Accenture. y Universia. (2007). Las competencias profesionales en los titulados. Contraste y diálogo universidad y empresa. Recuperado de http://www.accenture.com/Countries/Spain/About_Accenture/LasEmpresa.htm
Aguado, D. y Lucía, B. (s.f.). Guía breve de eValue. Informe técnico de investigación. Documento sin publicar.
Aguado, D., Lucia, B., Ponte, G. y Arranz, V. (2008). Análisis inicial de las propiedades psicométricas del cuestionario BFCP Internet para la evaluación de Big-Five. Revista Electrónica de Metodología Aplicada, 13(2), 1-15.
Akkermans, J., Schaufeli, W. B., Brenninkmeijer, V. y Blonk, R. W. B. (2013). The role of career competencies in the job demands-resources model. Journal of Vocational Behavior, 83(3), 356-366. doi:10.1016/j.jvb.2013.06.011
Atkinson, H. H., Lambros, A., Davis, B. R., Lawlor, J. S., Lovato, J., Sink, K. M. y Williamson, J. D. (2013). Teaching medical student geriatrics competencies in 1 week: An efficient model to teach and document selected competencies using clinical and community resources. Journal of the American Geriatrics Society, 61(7), 1182-1187. doi:10.1111/jgs.12314
Bartram, D. (2005). The great eight competencies: A criterion-centric approach to validation. Journal of Applied Psychology, 90(6), 1185-1203. doi:10.1037/0021-9010.90.6.1185.
Blömeke, S., Zlatkin-Troitschanskaia, O., Kuhn, C. y Fege, J. (Eds.) (2013). Modeling and measuring competencies in higher education. Berlin: SensePublishers.
Boam, R. y Sparrow, P. (1992). The rise and rationale of competency-based approaches. En R. Boam y P. Sparrow (Eds.). Designing and achieving competency (pp. 3-15). Londres: McGraw Hill.
Boyatzis, R. (1982). The competent manager. Nueva York, NY: Wiley.
Cabrerizo, J., Rubio, M. J. y Castillo, S. (2008). Programación por competencias. Formación y práctica. Madrid: Pearson Educación.
Costa, P. T. y McCrae, R. R. (1985). The NEO personality inventory manual. Odessa, FL: Psychological Assessment Resources.
Costa, P. T. y McCrae, R. R. (1992). The revised NEO personality inventory (NEO-PI-R) and NEO-five-factor inventory (NEO-FFI) professional manual. Odessa, FL: Psychological Assessment Resources.
Elosua, P. y Zumbo, B. (2008). Coeficientes de fiabilidad para escalas de respuesta categórica ordenada. Psicothema, 20(4), 896-901.
Feltham, R. (1992). Using competencies in selection and recruitment. En R. Boam y P. Sparrow (Eds.), Designing and achieving competency (pp. 89-103). Londres: McGraw Hill.
Furnham, A. y Chamorro, T. (2004). Personality and intelligence as predictors of statistics examinations grades. Personality and Individual Differences, 37, 943–955. doi:10.1016/j.paid.2003.10.016.
García, M. (2011). Una revisión constructiva de la gestión por competencias. Anales de Psicología, 27(2), 473-497. doi:10.6018/123101
Gómez, J. M. y Borrás, F. (2006). Competencias profesionales en los titulados UMH. Elche: Universidad Miguel Hernández
González, M. (2006). Currículo basado en competencias: una experiencia en educación universitaria. Eduación y Educadores, 9(2), 95-117.
González, J. y Wagenaar, R. (2003). Tuning educational estructures in Europe. Bilbao: Universidad de Deusto.
Jung, H. Y. (2015). The study on essential competencies for university students in consideration of university specialization and major: focusing on hotel and tourism management. Journal of Digital Convergence, 13(10), 19-32. doi:10.14400/jdc.2015.13.10.19
Khairullina, E. R., Pochinova, T. V., Khisamiyeva, L. G., Sakhipova, Z. M., Fedorova, L. V., Ablyasova, A. G. y Aksenova, N. N. (2015). The competences model of competitive process engineer. Journal of Sustainable Development, 8(3), 250-266. doi:10.5539/jsd.v8n3p250
Kurz, R. y Bartram, D. (2002). Competency and individual performance: Modelling the world of work. En T. Robertson, M. Callinan y D. Bartram (Eds.), Organizacional effectiveness. The role of psychology (pp. 227-255). Chichester: Routledge.
Latham, G. P. y Wexley K. N. (1981). Increasing productivity through performance appraisal. Reading, MA: Addison-Wesley.
Lawler, E. E. (1994). From job-based to competency-based organization. Journal of Organizacional Behaviour 15, 3-15. doi:10.1002/job.4030150103.
Lozano, L., García-Cueto, E. y Muñiz, J. (2008). Effect of the number of response categories on the reliability and validity of rating scales. Methodology, 4(2), 73-79. doi:10.1027/1614-2241.4.2.73
Lucia, A. D. y Lepsinger, R. (1999). Competency models. Pinpointing critical success factors in organizations. San Francisco, CA: Jossey Bass Pfeiffer.
McClelland, D. C. (1973). Testing for competence rather than for intelligence. American Psychologists, 28, 1-14. doi:10.1037/h0034092
Murphy, K., Martin, C. y García, M. (1982). Do behavioral observation scales measure observation? Journal of Applied Psychology, 67(5), 562-567. doi:10.1037/0021-9010.67.5.562
Nikolau, I. (2003). The development and validation of a measure of generic work competencies. International Journal of Testing, 3(4), 309-319. doi:10.1207/S15327574IJT0304_1
Nybo, G. (2004). Personnel development for dissolving jobs: Towards a competency-based approach? International Journal of Human Resource Management, 15(3), 549-564. doi:10.1080/0958519042000181250.
Osterlind, S. J. (1989). Constructing test items. Boston, MA: Kluwer Academic Publishers.
Palmer, A., Montaño, J. y Palou, M. (2009). Las competencias genéricas en la educación superior. Estudio comparativo entre la opinión de empleadores y académicos. Psicothema, 21(3), 433-438.
Poropat, A. (2009). A meta-analysis of the five-factor model of personality and academic performance. Psychological Bulletin, 135(2), 322-338. doi:10.1037/a0014996
Rieckmann, M. (2012). Future-oriented higher education: Which key competencies should be fostered through university teaching and learning? Futures, 44(2), 127-135. doi:10.1016/j.futures.2011.09.005
Robertson, I. y Kinder, A. (1993). Personality and job competentes: The criterion-related validity of some personality variables. Journal of Occupational and Organizational Psychology, 66, 225-244. doi: 10.1111/j.2044-8325.1993.tb00534.x
Robertson, I., Gibbons, P., Baron, H., MacIver, R. y Nyfield, G. (1999). Understanding management performance. British Journal of Management, 10(5), 5-12. doi:10.1111/1467-8551.00107
Rovinelli, R. y Hambleton, R. K. (1977). On the use of content specialists in the assessments of criterion-referenced test item validity. Dutch Journal of Educational Research, 2, 49-60.
Schmeiser, C. B. y Welch, C. J. (2006). Test development. En R. L. Brennan (Ed.), Educational measurement (pp. 45-76). Westport, CT: Praeger Publishers.
Shek, D. T. y Sun, R. C. (2012). Promoting psychosocial competencies in university students: Evaluation based on a one-group pre-test/post-test design. International Journal on Disability and Human Development, 11(3), 229–234. doi:10.1515/ijdhd-2012-0039
SHL Group. (1993a). Inventory of management competencies: Manual and user’s guide. Thames Ditton: SHL Group plc.
SHL Group. (1993b). OPQ concept model: Manual and user’s guide. Thames Ditton: SHL Group plc.
SHL Group. (1994). Perspectives on managerial competencies: User´s manual. Thames Ditton: SHL Group plc.
SHL Group. (1997). Customer contact: Manual and user´s guide. Thames Ditton: SHL Group plc.
SHL Group. (1999a). OPQ32: Manual and user’s guide. Thames Ditton: SHL Group plc.
SHL Group. (1999b). Work styles questionnaire, version n: Manual and user´s guide. Thames Ditton: SHL Group plc.
Solanes, A., Núñez, R. y Rodríguez, J. (2008). Elaboración de un cuestionario para la evaluación de competencias genéricas en estudiantes universitarios. Apuntes de Psicología, 26(1), 35-49.
Sparrow, P. (1997). Organizational competencies: Creating a strategic behavioural framework for selection and assessment. En N. Anderson y P. Herriot (Eds.), International Handbook of Selection and Assessment (pp. 343-368). Chichester: Wiley.
Sparrow, P. y Bognanno, M. (1993). Competency requirement forecasting: Issues for international selection and assessment. International Journal of Selection and Assessment, 1, 50-58. doi:10.1111/j.1468-2389.1993.tb00083.x
Spencer, L. M. y Spencer, S M. (1993). Competence at work, models for superior performance. New Cork: Wiley.
Tea Ediciones. (2008). SOSIA. Recuperado de http://web.teaediciones.com/SOSIA--GESTION-POR-COMPETENCIAS.aspx
Tea Ediciones. (2011). CompeTEA. Recuperado de http://web.teaediciones.com/COMPETEA.aspx
Tett, R., Guterman, H., Bleier, A. y Murphy, P. (2000). Development and content validation of a “hyperdimensional” taxonomy of managerial competence. Human Performance, 13(3), 205-251. doi:10.1207/S15327043HUP1303_1
Vivel-Búa, M., Fernández-López, S., Lado-Sestayo, R. y Otero-González, L. (2015). ¿Cómo mejorar la asimilación de los contenidos teóricos por parte del alumnado universitario? Una aplicación del one minute paper en Contabilidad. REICE. Revista Iberoamericana sobre Calidad, Eficacia y Cambio en Educación, 13(2), 67-84.
Weng, L. (2004). Impact of the number of response categories and anchor labels con coefficient alpha and test-retest reliability. Educational and Psychologycal Measurement, 64, 956-972.
Wiersma, U. y Latham, G. (1986). The practicality of behavioral observation scales, behavioral expectation scales and trait scales. Personnel Psychology, 39, 619-628. doi:10.1111/j.1744-6570.1986.tb00956.x
Zhao, L., Sun, T., Sun, B. Z., Zhao, Y. H., Norcini, J. y Chen, L. (2015). Identifying the competencies of doctors in China. BMC Medical Education, 15(1), 1-8. doi:10.1186/s12909-015-0495-y