Participants

The data was collected from a non-probabilistic and incidental sample of 509 participants. 53 % of the participants were women residing in the Buenos Aires Metropolitan Area. Their average age was 44.3 (SD = 13); 47.2 % stated they were married or in a lawful union, 25 % were single, 15.3 % were divorced, 4.7 % were widowed, while 7.9 % chose the "Other" option.

Procedure

The data was gathered using a non-probabilistic sample design defined by accessibility. The protocol was administered in an online interview format, which included anonymous informed consent. In addition, it was clarified that data use was exclusively for research purposes and that participation was completely voluntary, with the possibility of interrupting it at any point.

Instruments

Sociodemographic Questionnaire. It consists of a series of ad hoc questions for the present research that inquired about gender, age, marital status, nationality and place of residence.

Buenos Aires Loneliness Scale ((BALS); ^{Auné, et al., 2019}). It is a seven-item instrument, where the response modality is specified using a Likert scale with four options (1 = Completely disagree, 2 = Slightly Agree, 3 = Fairly Agree, 4 = Completely Agree).

Satisfaction of IRT Model Assumptions

The verification of the unidimensionality assumption required by the GRM, GRM_R, GPCM, and PCM models that were to be compared was carried out through the optimal implementation of parallel analysis (^{Timmerman, & Lorenzo-Seva, 2011}) and the variance percentage explained by the first factor. Both indices are obtained by Exploratory Factor Analysis (^{Ferrando & Lorenzo-Seva, 2017}a).

In addition to the unidimensionality assumption, the GRM, GRM_R, GPCM, and PCM models also assume that, given a fixed θ trait level, item responses are independent of one another. The X² _LD index (^{Chen & Thissen, 1997}) is calculated for each item pair, and a score over 10 indicates a failure to satisfy the assumption.

Comparison of IRT Models

Multiple methods were used to compare the relative fit of the GRM, RGRM, GPCM, and PCM models as described by ^{De Ayala (2009}) and ^{Toland (2013}). On the one hand, the Likelihood Ratio Test (LRT) was implemented, which compares two nested models and which was used in conjunction with the statical test R 𝛥 2 (^{Haberman, 1978}). In this case, the RGRM model is a restriction of the GRM, PCM, and GPCM models. The LRT examines whether the complexity of the complete model with unrestricted values for the a parameter is necessary to improve the model's fit. It adopts a χ² distribution, where a non-significant χ 𝛥 2 statistic implies that the additional complexity of the unrestricted model is unnecessary to improve goodness of fit to the observed data. The statistic R 𝛥 2 measures by what percentage the complete model increases the explanation of item responses compared to the restricted model. The R 𝛥 2 is calculated as follows: (log likelihood of the restricted model - log likelihood of the complete model) / log likelihood of the restricted model (^{Toland, 2013}).

On the other hand, the Akaike Information Criterion (AIC) and the Bayesian Information Criterion (BIC) were calculated for each model, where smaller values of AIC and BIC indicate a better relative fit. Finally, the global fit and the existence of item misfit in each model were analysed. Global fit was calculated with the M₂ statistic (^{Maydeu Olivares & Joe, 2005}, ²⁰⁰⁶) and the associated RMSEA index, considering RMSEA ≤ .05 as a good fit. The lower the values of the M₂statistic, the better the fit. The S-χ² index was calculated to determine whether each item is explained by a model (^{Orlando & Thissen, 2000}, ²⁰⁰³). If the p-value associated with the S-χ² index is higher than .01, it indicates a good fit (^{Toland, 2013}).

Analysis of Differential Item Functioning

The existence of DIF by marital status was explored dividing the sample between those who were married or in a lawful union, on the one hand, and participants who selected other marital statuses, on the other. This analysis was carried out following the series of steps explained by ^{Woods (2009}). As a first step, each of the BALS items was verified using the modified Wald test (^{Cai, 2012}; Cai, Thissen, & du Toit, 2011; ^{Langer, 2008}) considering the rest as anchors. Subsequently, a second step was carried out where an item with potential DIF was tested with anchoring in the responses to the most certainly DIF-free item, thus avoiding contamination.

Evidence within the IRT Framework

Evidence of reliability was obtained within the IRT framework through the Item Information Function (IIF) and the Test Information Function (TIF). The IIF indicates the accuracy of a certain item in measuring each θ trait level. The sum of all IIFs makes up the TIF, which provides information about the scale's reliability according to the θ trait level.

Software Used

The unidimensionality indices were obtained using the FACTOR 10.5 software (^{Ferrando & Lorenzo-Seva, 2017}b). Local independence, DIF, and IRT modelling analyses were conducted using the IRTPRO 4.2 software (^{Cai et. al, 2011}).

Satisfaction of IRT Model Assumptions

The optimal implementation of parallel analysis indicated that the suggested number of factors is one, while the variance percentage explained by the first factor was 57.48 %. Therefore, the data can be considered essentially unidimensional. In the outputs of each of the GRM, RGRM, GPCM, and PCM models, the X² _LD index was less than 10 for each item pair. Therefore, for the four models, both unidimensionality and local independence assumptions can be considered satisfied.

Comparison of IRT Models

The fit indices for GRM, RGRM, GPCM, and PCM can be seen in Table 1. Although every model fitted at a global level, the GRM obtained lower values for the M₂ statistic, log likelihood, AIC, and BIC. The LRT comparing the GRM and the RGRM indicated that the additional complexity of the complete model is necessary to improve fit to the data considering χ 𝛥 2 (6) = 7108.19 - 7011.88 = 96.31, p = 7.07x10^-19. The relative change between these models was R 𝛥 2 = .0135, that is, the MRG provides a better explanation of the data than the MRG_R by 1.35 %. Highly similar results are obtained when comparing GPCM and PCM. The LRT between the GPCM and the PCM resulted in χ 𝛥 2 (6) = 5939.43-5866.85 = 72.58, p = 1.21x10^-13. In this case, R 𝛥 2 = .0122, indicating a 1.22 % better fit of the complete model. For every model, none of the items showed misfit, as the p-value associated with the S-χ² index was higher than .01 for all the items.

Because a model with a free a parameter is necessary to improve both the fit and the explanation of the observed data, and because the GRM is the model showing the best relative fit, GRM is selected for the TRI modelling of the BALS item responses.

Table 1: Model level fit (Comparison) 

BALS Modelling with GRM

Of the four models, the GRM proved the most appropriate one. It assumes that a single θ -in this case, the loneliness level- non-linearly explains the item responses. Because the response options are four, an a slope parameter and three b _m threshold parameters were calculated for each reagent. The a parameter provides information about the degree to which response categories distinguish between θ levels. Moreover, it has been compared against factorial loads, as it reflects the magnitude of the relationship between each scale item and the latent θ trait. Each b _m parameter provides information about the loneliness level -θ- necessary for the probability of selecting the m response category or a higher one to be equal (.50) to the probability of selecting the lower categories. Thus, the item response options are separated into a series of dichotomies, in each of which the ML2P is applied. The IRCCCs represent the probability of selecting each response category according to the θ trait level.

Analysis of Differential Item Functioning

The results obtained after exploring the presence of DIF for each item using the rest of the items as anchors are shown in Table 2. The analyses suggested that item 5, I am completely out of any social group, was likely to present non-uniform gender-based DIF, even though the p-value was very close to the 0.5 limit. The second step of the ^{Woods method (2009}) was then implemented, taking item 2 as an anchor between the two groups. This item has the lowest _Totalχ² value, so it is assumed that it is the most DIF-free item. The statistically insignificant result from this second step indicated that item 5 does not exhibit DIF (^𝜒2 _a = 2.6, gl = 1, p =.1048). Therefore, all the items on the scale can be considered DIF-free regarding marital status.

Table 2: DIF Statistics for marital status. Wald test 

Item calibration with the GRM

The results obtained after applying the GRM to the scale indicated that the model fitted both globally (M₂ = 391.86; gl = 182; p = 0.0001; RMSEA = 0.05) and at item level (p associated to S-x²> .01). 28 parameters were estimated, the values of which are shown in Table 3.

Item threshold parameters are distributed across a relatively broad range for the latent trait, from -1.43 (b₁ item 3) to 3.15 (b₃ item 7). A relative heterogeneity of b_1, can be observed, while b₂ parameters are found at medium or high levels of the trait and b₃ at even higher levels.

As for the a discrimination parameters, they presented values of 1.11 to 3.01. This indicates that the response categories are powerful in distinguishing between participants with different levels of loneliness, with a moderate discrimination capacity for items 3 and 7, high for item 6, and very high for items 1, 2, 4, and 5 (^{Baker & Kim, 2017}).

Table 3: Graded response model parameter estimates for the BALS 

Figure 1 shows the IRCCCs of item 3, the one with the lowest a parameter. As it can be observed, the IRCCCs corresponding to the central response categories show a flattened form. For this item, even though all response options are most probable at some level of the trait, the Slightly Agree category is probable only within a narrow range. Given that the average b parameter is -0.09, this item can be considered medium-difficulty. A very low level for the trait is enough to select the Slightly Agree category or a higher one; a medium level is enough to select the Fairly Agree or Completely Agree categories over the previous two, and a very high level of the trait is necessary to select the top Completely Agree category. Therefore, it is feasible to say that the response categories behave in an expected manner.

Figure 1: Graded Response Model Item Category Characteristic Curves for Item 3 

Figure 2 shows the IRCCCs of item 4, the one with the largest a parameter. In this case, the IRCCCs corresponding to the extreme response categories show a high form. Even though all response options are most probable at some level of the trait, the intermediate options are most probable only within a very short interval. Given that the average b parameter is 1.09, this item can be considered high-difficulty. Even though the response categories behave in an expected manner, this item would be suitable as part of a dichotomous test.

Figure 2: Graded Response Model Item Category Characteristic Curves for Item 4 

Evidence of Reliability

Table 4 shows specific values of the IIFs and TIF for certain levels of loneliness distributed along the continuum of the trait. For most of the items, as well as the complete test, there is evidence that they provide a higher level of information for medium and high levels of loneliness, while, in turn, the standard error decreases (standard error, s.e.). The items show certain parity at the maximum level of information they provide.

Table 4: Graded response model Item Information Function and Test Information Function 

Figure 3 shows the TIF. According to the GRM, the TIF reached its maximum value of 6.9983 at θ = 1.00 with a minimum s.e. value at that 0.378 point. The level of information was higher in the medium and high levels of the trait, decreasing considerably in the low levels of the trait, as well as in the extremely high ones.

Figure 3: Test Information Function of the Buenos Aires Loneliness Scale (Graded Response Model).