SciELO - Scientific Electronic Library Online

 
vol.13 número1Investigaciones contemporáneas en Psicología EducacionalIntervenciones psicológicas positivas para adultos mayores en Brasil índice de autoresíndice de materiabúsqueda de artículos
Home Pagelista alfabética de revistas  

Servicios Personalizados

Revista

Articulo

Links relacionados

Compartir


Ciencias Psicológicas

versión impresa ISSN 1688-4094versión On-line ISSN 1688-4221

Cienc. Psicol. vol.13 no.1 Montevideo jun. 2019

http://dx.doi.org/10.22235/cp.v13i1.1812 

Original Articles

Psychometric properties of the online global symptoms scale (ESGO)

1Facultad de Psicología y Ciencias Sociales, Universidad de Flores. Argentina marian.durao@gmail.com, genisegp@gmail.com

2CONICET. Argentina jungaretti@psi.uba.ar, edgardoetchezahar@psi.uba.ar

3Facultad de Psicología, Universidad de Buenos Aires. Argentina jungaretti@psi.uba.ar, edgardoetchezahar@psi.uba.ar

4Facultad de Ciencias Sociales, Universidad Nacional de Lomas de Zamora. Argentina jungaretti@psi.uba.ar, edgardoetchezahar@psi.uba.ar Correspondence: María Durao, Ada Elflein 2781. CP 1643. Provincia de Buenos Aires, Argentina. Joaquín Ungaretti, Gral. Juan Lavalle 2353, C1052AAA. Ciudad Autónoma de Buenos Aires, Argentina. Gabriel Genise, Severo García G. de Zequeira 6880, CABA. Edgardo Etchezahar, Tte. Gral. Juan Domingo Perón 2158, C1040AAH. Ciudad Autónoma de Buenos Aires, Argentina.

ABSTRACT

Abstract: With the development of new technologies in the field of psychotherapy comes the need for valid and reliable tests that allow us to research the presence of symptoms in an online format. The aim of the following paper was to develop a scale to be used in the virtual format, which allows us to measure symptomatology for anxiety, depression, interpersonal relationships and psychoticism. 550 adults participated, who answered an online questionnaire that included four dimensions (anxiety, depression, interpersonal relationships, psychoticism). Results shows an adequate internal consistency and construct validity for the four dimensions tested. Also, for the external validity significant relationships were found whit the OQ-45 and SA-45. The test we constructed seems to be a valid tool for psychotherapeutic use. However, it is necessary to continue analyzing its psychometrics, taking into account patients who have the specific diagnoses evaluated by the test.

Key words: psychometry; depression; anxiety; interpersonal relations; psychoticism; global symptoms

Introduction

Mental health in global terms is facing a significant crisis with respect to previous years, reflected in the still growing prevalence rates (World Health Organization, 2015). In this regard, the data pertaining to depression, anxiety, and other mental disorders are truly alarming (Baxter et al., 2014). For these reasons, psychotherapy, along with other practices such as meditation and religion, appear as a way to cope with this reality, which has grown in the last thirty years on a large scale (Weissman & Cuijpers, 2017). So, during the last century, the world has witnessed a significant increase in the pursuit of symptomatic relief, as well as in the range of offers (Fernandez Alvarez, Gomez, & Garcia, 2013).

In relation to psychotherapy as the more widely consumed treatment worldwide by those suffering from emotional disorders, Olfson and Pincus (1994) found that 80% of outpatient consultations in the United States provided by mental health professionals included a psychotherapeutic intervention. In this line, more recent research argues that the figures have remained the same (Mojtabai & Olfson, 2010).

According to the above, the follow-up of the psychological state of a consultant who is undergoing treatment is one of the most important and complex issues in a psychotherapeutic process (Feltham, Hanley, & Winter, 2017). Therefore, and in response to the challenges that have been raised regarding the validity of the evaluation of psychotherapy, emphasis has been placed on the consideration of the multiple variables that influence this process, on developing measurable operational criteria, and on having a sample of patients that allows us to obtain significant results (Valdivieso, 1994). In this sense, despite the considerable bias that comes with evaluating a patient, clinical observation has been one of the techniques most often used in the context of psychotherapy, along with psychological tests, which may be useful tools to detect mental changes in patients at the start or during the treatment process (Richards & Bergin, 2014). The latter provide the advantage of standardization, as well as the reduction of bias and the collection of information through a different method (Kline, 2013).

Receiving feedback from patients during the psychotherapeutic treatment is a real, but skewed and distorted process due to the cognitive heuristics and schemes displayed by the very same therapist (Blumenthal-Barby & Krieger, 2015). That is why, as several researchers of the psychotherapeutic process have argued (Gelo & Manzo, 2015; Hardy, & Llewelyn, 2015), the informal feedback that the therapist receives needs to be complemented by a standardized assessment system that is reliable. This leads to the measurement of the quality of the psychotherapy, as well as in the therapist’s professional development (Kelley & Bickman, 2009).

The evaluation of the individual changes during a psychotherapeutic process has been a significant issue in the research into psychotherapy (Gelo & Salvatore, 2016). In this sense, different methods were developed over the years such as the Achievement of Therapeutic Objectives Scale, in which patients are evaluated at the end of the session. However, in relation to the inclusion of evaluation in psychotherapy, it was unusual for these evaluations to take place outside the scope of behavior therapists. According to Lambert (2013), assessment procedures over the course of a psychotherapeutic treatment could facilitate symptomatic reduction during the same. In this way, not only would the proposed objective be achieved, but it would also benefit those patients who see a constant improvement, as well as the positive bond with the therapist and the psychotherapeutic process. Additionally, the possibility to continuously evaluate the therapeutic process would offer mental health professionals a key tool to consider the patient’s process, as well as also implement the changes they deem pertinent in view of the effectiveness of the treatment, which would be reflected in the symptomatic relief of the consultant (Levendosky & Hopwood, 2017).

Many of the current techniques (for example, the SCL-90, OQ-45, SA-45), investigate symptoms of anxiety, depression, interpersonal relations, and psychoticism, among others, which allow a first general screening of the consultant (Lara Muñoz, Espinosa de Santillana, Cárdenas, Fócil, & Cavazos, 2005). In this line, one of multidimensional self-report questionnaires that is most widely used for the overall assessment of mental health symptoms is the SCL-90 (Derogatis, & Cleary, 1977). This assessment was designed to allow the self-assessment of various dimensions of psychopathology, both in psychiatric patients and the general population. Although these instruments seem promising in the evaluation of the mental state of seriously ill patients, since they contain specific scales to measure psychotic and schizotypal symptoms, like the psychoticism or paranoid ideation scale, their validity has not yet been confirmed. For example, Wood (1982) found no evidence that patients with schizophrenia obtained higher scores on the psychoticism scale than patients without schizophrenia. For their part, Johnson, Chipp, Brems, and Neal (2008), found no difference in the psychoticism and paranoid ideation scales among patients diagnosed with schizophrenia and those without this diagnosis. In order to overcome these limitations, years later, Davison et al. (1997) developed the Symptom Assessment-45 Questionnaire (SA-45). The objective of this version was, while keeping the same dimensions of the SCL-90, to reduce the length of the questionnaire by half and ensure that all dimensions have the same number of items to obtain better psychometric indicators, while avoiding overlaps.

Another instrument used to evaluate symptoms in mental health is the Outcome Questionnaire - 45 (OQ-45) (Lambert et al., 1996). It is a self-report created for patients undergoing psychotherapy, consisting of three subscales: general symptoms, interpersonal relations, and a subscale for social role, which measures fitness for work, study, and leisure. The OQ-45 evaluates the progress of the patient undergoing therapeutic treatment, designed to be applied during the therapeutic process and at the end of it. It is a brief scale that has shown good psychometric properties in terms of reliability and validity, applicable to different populations (Doerfler, Addis, & Moran, 2002). Although this instrument is not used to establish diagnoses, its design targets decision making (Von Berger & De la Parra, 2002).

From the development of new technologies in the field of psychotherapy arises the need for valid and reliable evidence in which to investigate the presence of symptoms from an online format (Clough, & Casey, 2015; Titov et al., 2015). For over three decades, there have been comparative studies between psychometric assessments through the use of both online and face-to-face formats, proving that the former was deemed to be as accurate and reliable as the latter (Lexcen, Hawk, Herrick, & Blank, 2006; Vallejo, Jordán, Díaz, Comeche & Ortega, 2007). Moreover, it has been evidenced that online assessments relating to clinical aspects allow an adequate diagnostic discrimination, with low rates of false positives (Hyler, Gangure, & Batchelder, 2005).

In this context, and due to the lack of free access to psychometric instruments with the adequate psychometric properties, the goal of the present study was to develop the Online Global Symptoms Scale (Escala de Síntomas Globales Online or ESGO), which would allow us to study four dimensions: depression, anxiety, psychoticism, and interpersonal relations. It is envisioned as an instrument of self-report, whose purpose is to become a psychological resource for gathering information about the internal states of the individual initiating treatment or for the professional who needs to assess patients.

Materials and Methods

Participants

The participants were selected through nonprobability purposive sampling. A total of 550 adults participated in the study, belonging to the general, non-clinical population, residents of the autonomous city of Buenos Aires, between 14 and 73 years of age (M = 36.6; DT = 13.08) and of both sexes. Of the total number of participants, 17.46% positioned themselves in a middle to high socio-economic level, 64,72% in a middle level, and 17.82% in a low level. Moreover, of the total number of participants, 24.91% indicated that they had undergone or were undergoing psychological treatment, while 75.09% stated they had never had any type of psychological treatment.

Data Collection Techniques

- Online Global Symptoms Scale (ESGO): To assess the construct, we proceeded to carry out the construction and validation of the ESGO, composed in its final version of 20 items that address the four dimensions that comprise the global assessment of symptoms presented by individuals. Each of the dimensions that comprise the construct were represented by a series of five items: for Depression (for example, “I often feel that there is no solution to my problems”; “I don’t enjoy things as much as I used to”), Anxiety (for example, “At times I feel intense fear, and I think I'm going to die”; “I have sensations in my body that nobody can understand”), Interpersonal Relations (for example, “I tend to worry that I’ll be judged negatively”; “I sometimes feel I’m less than my peers or friends”), and the fourth dimension, Psychoticism (for example, “ I sometimes hear voices that others don’t hear”; “I sometimes feel that an external force is trying to control me.”) We propose to refer to it as a scale of global symptomatology, as the different items explored, although they were developed so as to represent a specific dimension, may also represent other pathologies. Moreover, the use of the term “global” allows us to show that the goal of the assessment is to approximate a description of clinical symptoms, but that this in no way results in a diagnostic presumption. The assessment is aimed at individuals over the age of 14 and of both sexes, its scope being the fields of clinical psychology and health. The format was a Likert scale questionnaire with five points, ranging from 1 = Strongly Disagree to 5 = Strongly Agree (the same answer format was used for the other scales used in this study).

- SA-45 (Symptom Assessment-45): We used the Spanish version created by Sandin, Valiente, Chorot, Santed, and Lostao (2008), adapted from the original instrument created by Davison et al. (1997), composed of 45 items. Participants are instructed to indicate to what extent each of the 45 symptoms were present over the past week, based on a Likert scale ranging between 1 = not at all to 5 = very or extremely. The questionnaire assesses the same dimensions as the SCL-90 and obtains adequate indicators of internal consistency: somatization (α = .80), obsessive-compulsive (α = .72), interpersonal sensitivity (α = .84), depression (α = .85), anxiety (α = .84), hostility (α =. 83), phobic anxiety (α = .71), paranoid ideation (α = .71), and psychoticism (α = .63).

- OQ-45 (Outcome Questionnaire-45): We used a version adapted for this study from the original version created by Lambert et al. (1996). The questionnaire is composed of 45 items grouped in three dimensions: Symptom Distress (SD), Interpersonal Relations (IR), and Social Role (SR), whose scores make up a total score for the scale. The instrument has shown a high internal consistency for use with patients in its original version, both for the totality of the scale (α = .93) and for each of its dimensions (SD = .91; IR = .74; SR = .71). However, for the present study, only the dimensions of Symptom Distress and Interpersonal Relations were considered in order to establish relationships with the ESGO.

- Socio-demographic Data Questionnaire: To collect this type of information, we inquired about sex, age, self-perceived socio-economic level, and if the individual has undergone or is currently under psychological treatment.

Procedure

For the construction of the ESGO and its validation within the Argentine context, we took into account the international methodological standards recommended by the International Test Commission (ITC) for the creation of new assessment instruments (Muñiz, Elosua, & Hambleton, 2013).

The initial items were purged until we reached a version with 60 reagents. Subsequently, it was presented to a panel of expert judges. After analysis from expert judges, regarding the correspondence of each item to the dimension being evaluated through Cohen's kappa coefficient (K = .73), a pilot test was conducted of the preliminary version of the ESGO with a small group in order to corroborate the degree of understanding of the reagents.

In a second phase, the items were edited based on the recommendations obtained from the judges.

Consent was requested from the participants. They were informed about the anonymity of their responses, the voluntary nature of their participation, the strictly scientific use of the information collected, and the protection of their personal data was guaranteed in compliance with the provisions set forth in the national law number 25.326 regarding the protection of personal data.

Data Analysis

The statistical analyses that guided the development of this study were carried out with the use of the SPSS software for Windows, version 19.0 (George & Mallery, 2010) and the EQS 6.1 program (Bentler, 2007) for the development of confirmatory factor analysis (CFA) of the structure of the ESGO.

Additionally, we analyzed the descriptive statistics for each of the items that make up the final version of the scale (mean, standard deviation, skewness, and kurtosis). Subsequently, two factor analyses were carried out to account for the construct validity of the assessment, one exploratory (EFA) and the other confirmatory (CFA), in the latter case using the maximum likelihood estimation to test the adjustment of the data to the model of four correlated dimensions. Next, we analyzed the internal consistency of the instrument through the use of Cronbach's alpha and, finally, with the goal of evaluating the validity of the criteria, we assessed the association between the ESGO, the SA-45 and the OQ-45 through the use of Pearson's r.

Results

First, the descriptive statistics for the scale were analyzed, mainly the mean, the deviation, the correlation between the item and the total, and the alpha if item deleted (Table 1).

As seen in table 1, all descriptive statistics, as well as the indicators of internal consistency (.67 < a < .83), turned out to be adequate, with the exception of the distribution of the items in the Psychoticism dimension, which is to be expected due to the low rate of occurrence of this symptomatology in the general population.

Next, a confirmatory factor analysis was conducted to analyze the construct validity of the scale. For this purpose, a model of four correlated dimensions was proposed, which turned out to be adequate (X2 (164) = 574,916; p < .001; CFI = .91; AGFI = .90; RMSEA = .068 (.062 - .074)), thus corroborating the construct validity of the scale.

Subsequently, the relationships between the dimensions of the scale and other related variables included in the SA-45 and the OQ-45 were analyzed. Moreover, the relationships between all the variables and the participants’ ages were studied (Table 2).

As seen in Table 2, the dimensions of our scale are largely related to the corresponding scales of the SA-45 and the OQ-45 in all cases. Additionally, low but significant relationships were observed with the participants’ ages, except for interpersonal relations, whose strength was a bit higher.

Subsequently, the levels of the four dimensions of the assessment were analyzed with respect to the having undergone therapy and not having done it at all (Table 3).

According to the results presented in Table 3, statistically significant differences were seen in the dimensions of Depression and Anxiety, but not for Interpersonal Relations and Psychoticism.

Table 1: Descriptive analysis of the items that comprise the dimensions of the ESGO scale and the reliability for each of the dimensions 

Table 2: Correlations between all the variables and Cronbach’s alpha for each dimension 

Table 3: Differences based on previous experience with psychotherapy in the levels of Depression, Anxiety, Interpersonal Relations, and Psychoticism of the ESGO scale 

Discussion

In order to address the needs identified by different authors on the subject (Clough, & Casey, 2015; Titov et al., 2015), the goal of the present research was to develop an online global symptoms scale that would allow us to assess symptoms of anxiety, depression, interpersonal relations, and psychoticism.

Firstly, the analyses of the reliability and the validity of the ESGO were carried out. As in most of the classic assessments considered for analysis of psychopathological symptomatology (Davison et al., 1997; Derogatis & Cleary, 1977; Doerfler et al., 2002), Doerfler et al., 2002), the adequate psychometric properties were obtained, and this led to a factor structure composed of four dimensions: depression, anxiety, relations, and psychoticism. For each of the dimensions, internal consistency was adequate, the sub-scale for psychoticism being the lowest (α = .67). The correlations between each of the items and its factor were adequate, with the criteria proposed by Hair, Black, Babin, Anderson, and Tatham (2010) as a reference. These results indicate that, unlike other forms of assessment of symptomatology, the ESGO presents itself as a valid and reliable tool that allows an adequate discrimination of different dimensions of symptoms.

Secondly, it was noted that the model of four correlated dimensions of symptomatology showed a suitable adjustment to the sample data. Moreover, after analyzing the associations between each of the dimensions of the ESGO, we observed that they are significant between each of them, although as the prior literature suggests (Johnson et al., 2008; Wood, 1982), they are lower for psychoticism.

There were relationships between the different dimensions of the ESGO, both with the comparable dimensions of the SA-45 and the OQ-45. For example, the strongest relationships were observed between the depression dimension of the ESGO and the depression dimension of the SA-45 (r = .79) and with the symptom distress dimension of the OQ-45 (r = .80). In this same vein, it was noted that the strongest relationships for the anxiety dimension of the ESGO occurred with the anxiety dimension of the SA-45 (r = .76) and with the symptom distress dimension of the OQ-45 (r = .74). Regarding the relations dimension, the ESGO presented strong relationships with the interpersonal relations dimension of the SA-45 (r = .70), as well as with the same dimension of the OQ-45 (r = -.54) (the relationship is negative because the OQ-45 scale items are inverted in this dimension). Lastly, the psychoticism dimension of the ESGO had a stronger relationship with its correlates in the other two assessments, both with the psychoticism dimension of the SA-45 (r = .59), and with symptom distress of the OQ-45 (r = .41). Therefore, we may argue that the results obtained in the present study, as well as the construct validity, provide sufficient evidence of the ESGO’s concurrent validity, as it is compared to similar dimensions present in two instruments that are widely used in the scientific literature for the assessment of symptomatology.

As for the participants’ ages, the relationships were significant but low, except for Interpersonal Relations, whose value was the highest obtained.

In the Depression and Anxiety dimensions, statistically significant differences were seen with respect to whether the participant had previously undergone therapy or not, but not with respect to Interpersonal Relations and Psychoticism. More specifically, the subjects with previous experience with psychotherapy, presented higher levels of anxiety and depression than those who did not have previous experience.

The study contributes to the assessment of symptomatology through the creation of an instrument in Spanish, one that is valid and reliable, but which can also be used online and with open access (free). This work was carried out only with a sample of the general, non-clinical, population. Due to this limitation, it is recommended that studies be carried out considering different populations to achieve a greater generalization and representativeness of the results, including subjects with different psychopathological diagnoses. Finally, while there is ample evidence that online and face-to-face assessments are equally accurate and reliable (Lexcen et al., 2006; Vallejo et al., 2007), it is necessary to continue the evaluation of the psychometric properties of the ESGO in a paper format to compare the psychometric properties of both formats.

REFERENCES

Baxter, A. J., Scott, K. M., Ferrari, A. J., Norman, R. E., Vos, T., & Whiteford, H. A. (2014). Challenging the myth of an “epidemic” of common mental disorders: trends in the global prevalence of anxiety and depression between 1990 and 2010. Depression and anxiety, 31(6), 506-516. doi: htpps://doi.org/10.1002/da.22230 [ Links ]

Bentler, P. M. (2007). On tests and indices for evaluating structural models. Personality and Individual differences, 42(5), 825-829. https://doi.org/10.1016/j.paid.2006.09.024 [ Links ]

Blumenthal-Barby, J. S., & Krieger, H. (2015). Cognitive biases and heuristics in medical decision making: a critical review using a systematic search strategy. Medical Decision Making, 35(4), 539-557. doi: https://doi.org/10.1177/0272989X14547740 [ Links ]

Clough, B. A., & Casey, L. M. (2015). The smart therapist: A look to the future of smartphones and Health technologies in psychotherapy. Professional Psychology: Research and Practice, 46(3), 147. doi: https://dx.doi.org/10.1037/pro0000011 [ Links ]

Davison, M.K., Bershadsky, B., Bieber, J., Silversmith, D., Maruish, M.E., & Kane, R.L. (1997). Development of a brief, multidimensional, self- report instrument for treatment outcomes assessment in psychiatric settings: Preliminary findings. Assessment, 4, 259-276. doi: https://doi.org/10.1177/107319119700400306 [ Links ]

Derogatis, L. R., & Cleary, P. A. (1977). Confirmation of the dimensional structure of the SCL‐90: a study in construct validation. Journal of clinical psychology, 33(4), 981-989. doi: https://doi.org/10.1002/1097-4679(197710)33:4%3C981::AID-JCLP2270330412%3E3.0.CO;2-0 [ Links ]

Doerfler, L. A., Addis, M. E., & Moran, P. W. (2002). Evaluating mental health outcomes in an inpatient setting: convergent and divergent validity of the OQ-45 and Basis-32. The Journal of Behavioral Health Services and Research, 29(4), 394-403. doi: https://doi.org/10.1007/BF02287346 [ Links ]

Feltham, C., Hanley, T., & Winter, L. A. (Eds.). (2017). The SAGE handbook of counselling and psychotherapy. EE.UU.: Sage. [ Links ]

Fernández Álvarez, H, Gómez, B. & García, F. (2013) Bridging the gap between research and practice in a clinical and training network: Aigle´s Program. Psychotherapy Research, 25(4), 84-94. doi: https://doi.org/10.1080/10503307.2013.856047 [ Links ]

Gelo, O. C. G., & Manzo, S. (2015). Quantitative approaches to treatment process, change process, and process-outcome research. En Psychotherapy Research (pp. 247-277). Springer: Vienna. doi: https://doi.org/10.1007/978-3-7091-1382-0_13 [ Links ]

Gelo, O. C. G., & Salvatore, S. (2016). A dynamic systems approach to psychotherapy: A meta-theoretical framework for explaining psychotherapy change processes. Journal of counseling psychology, 63(4), 379. doi: https://dx.doi.org/10.1037/cou0000150 [ Links ]

George, D., & Mallery, P. (2010). SPSS for Windows Step by Step: A Simple Guide and Reference 17.0 Update. 10th Edition. Pearson: Boston. [ Links ]

Hair, J. F., Black, W. C., Babin, B. J., Anderson, R. E., & Tatham, R. L. (2010). SEM: An introduction. Multivariate data analysis: A global perspective, 629-686. [ Links ]

Hardy, G. E., & Llewelyn, S. (2015). Introduction to psychotherapy process research. In Psychotherapy Research (pp. 183-194). Viena: Springer. doi: https://doi.org/10.1007/978-3-7091-1382-0_9 [ Links ]

Hyler, S. E., Gangure, D. P., & Batchelder, S. T. (2005). Can telepsychiatry replace in-person psychiatric assessments? A review and meta-analysis of comparison studies.CNS spectrums, 10(5), 403-415. doi: https://doi.org/10.1017/S109285290002277X [ Links ]

Johnson, M. E., Chipp, C. L., Brems, C., & Neal, D. B. (2008). Receiver operating characteristics for the brief symptom inventory depression, paranoid ideation, and psychoticism scales in a large sample of clinical inpatients. Psychological Reports, 102(3), 695-705. doi: https://doi.org/10.2466/pr0.102.3.695-705 [ Links ]

Kelley, S. & Bickman, L. (2009). Beyond outcomes monitoring: Measurement Feedback Systems (MFS) in child and adolescent clinical practice, Current Opinion in Psychiatry, 22(4), 363-368. doi: https://doi.org/10.1097/YCO.0b013e32832c9162 [ Links ]

Kline, P. (2013). Handbook of psychological testing. USA: Routledge. [ Links ]

Lambert, M. J., Hansen, N. B., Umphress, V., Lunnen, K., Okiishi, J., Burlingame, et al. (1996). Administration and scoring manual for the OQ-45.2. Stevenson, MD: American Professional Credentialing Services. https://doi.org/10.1097/YCO.0b013e32832c9162 [ Links ]

Lambert, M. J. (2013). Outcome in Psychotherapy: The Past and Important Advances. Psychotherapy, 50(1), 42-51. doi: https://doi.org/10.1037/a0030682 [ Links ]

Lara Muñoz, C., Espinosa de Santillana, I., Cárdenas, M. L., Fócil, M. & Cavazos, J. (2005). Confiabilidad y validez de la SCL-90 en la evaluación de psicopatología en mujeres. Salud Mental, 28(3), 42-50. [ Links ]

Lexcen, F. J., Hawk, J. L., Herrick, S., & Blank, M. B. (2006). Use of video conferencing for psychiatric and forensic evaluations. Psychiatric Services, 57, 713-715. doi: https://doi.org/10.1176/appi.ps.57.5.713 [ Links ]

Levendosky, A. A., & Hopwood, C. J. (2017). Terminating supervision. Psychotherapy, 54(1), 37. doi: https://dx.doi.org/10.10437/pst0000096 [ Links ]

Mojtabai, R., & Olfson, M. (2010). National trends in psychotropic medication polypharmacy in office-based psychiatry. Archives of General Psychiatry, 67(1), 26-36. https://dx.doi.org/10.1001/archgenpsychiatry.2009.175 [ Links ]

Muniz, J., Elosua, P., & Hambleton, R. K. (2013). International Test Commission guidelines for test translation and adaptation.Psicothema, 25(2), 151-157. https://doi.org/10.7334/psicothema2013.24 [ Links ]

Olfson, M., & Pincus, H. A. (1994). Outpatient psychotherapy in the United States, I: Volume, costs, and user characteristics. The American journal of psychiatry, 151(9), 1289-1294. [ Links ]

Richards, P., & Bergin, A. E. (2014). Handbook of psychotherapy and religious diversity. Washington, DC, US: American Psychological Association. [ Links ]

Sandín, B., Valiente, R. M., Chorot, P., Santed, M. A., & Lostao, L. (2008). SA-45: forma abreviada del SCL-90. Psicothema, 20(2), 290-297. [ Links ]

Titov, N., Dear, B. F., Ali, S., Zou, J. B., Lorian, C. N., Johnston, L., & Fogliati, V. J. (2015). Clinical and cost-effectiveness of therapist-guided internet-delivered cognitive behavior therapy for older adults with symptoms of depression: a randomized controlled trial. Behavior Therapy, 46(2), 193-205. doi: https://doi.org/10.1016/j.beth.2014.09.008 [ Links ]

Valdivieso, S. (1994). Fundamentos y límites de la Psicoterapia. Boletín Escuela de Medicina, 23, 97-102. Recuperado de http://www.scielo.org.ar/scielo.php?script=sci_arttext&pid=S1668-70272009000200006Links ]

Vallejo, M., Jordán, C., Díaz, M., Comeche, M. & Ortega, J. (2007). Psychological Assessment via the Internet: A Reliability and Validity Study of Online (vs Paper-and-Pencil) Versions of the General Health Questionnaire-28 (GHQ-28) and the Symptoms Check-List-90-Revised (SCL-90-R). Journal of Medical Internet Research, 9(1), e2. doi: https://10.2196/jmir.9.1.e2 [ Links ]

Von Bergen, A., & de la Parra, G. (2002). OQ-45.2, Outcome Questionnaire and evolution ofPsychotherapy: Adaptation, Validation and Guidelines for its implementation and interpretation. Terapia Psicológica, 20, 161-176. [ Links ]

Weissman, M., & Cuijpers, P. (2017). Psychotherapy over the Last Four Decades. Harvard Review of Psychiatry, 25(4), 155-158. doi: 10.1097/HRP.0000000000000165 [ Links ]

Wood, W. D. (1982). An attempt to validate the psychoticism scale of the brief symptom inventory. British Journal of Medical Psychology, 55(4), 367-373. doi: https://doi.org/10.1111/j.2044-8341.1982.tb01521x [ Links ]

World Health Organization. (2015). World health statistics 2015. World Health Organization. [ Links ]

Note: Authors' participation: a) Conception and design of the work; b) Data acquisition; c) Analysis and interpretation of data; d) Writing of the manuscript; e) Critical review of the manuscript. M.D. has contributed in a,b,c,d,e; J.U. in a,b,c,d,e; G.G. n a,b,c,d,e; E.E. in a,b,c,d,e.

Note: How to cite this article: Durao, M., Ungaretti, J., Genise, G., & Etchezahar, E. (2019). Psychometric properties of the online global symptoms scale (ESGO). Ciencias Psicoóglicas 13(1), 95-105. Doi: https://doi.org/10.22235/cp.v13i1.1812

Received: February 20, 2018; Revised: June 05, 2018; Accepted: February 06, 2019

Creative Commons License Este es un artículo publicado en acceso abierto bajo una licencia Creative Commons