The role of textual cohesion in the validity and psychometric functioning of items in poetry-based cloze tests for PFL

Abstract

This study explores the role of textual cohesion in the psychometric functioning of items in cloze tests based on poetry in the context of Portuguese as a Foreign Language (PFL). Based on the premise that cohesive relations can influence textual processing and, consequently, performance in textual reconstruction tasks, the study aims to investigate whether the density of lexical and grammatical cohesion mechanisms is associated with the difficulty and discriminative power of cloze items. The corpus includes three poems used as stimuli in cloze tests administered to 32 PFL learners, with a total of 28 items and 896 individual responses. Cohesion strategies were identified through linguistic coding and operationalized by measures of cohesive density at the verse and stanza levels. To analyze the relationship between cohesion and performance, generalized linear mixed models and linear regression models were estimated. The results indicate that cohesive density is not a systematic predictor of the probability of correct answers or the discriminative power of the items. The observed performance seems to be predominantly explained by individual differences in proficiency and the intrinsic difficulty of the items. These results indicate that, under the conditions and in the corpus analyzed, the variation in textual cohesion of the poems does not introduce significant psychometric bias. This study contributes not only to the discussion on construct validity and item construction in cloze tests applied to the assessment of PFL, but also to the validity of integrating poetry into PFL tests.