Detection of insufficient effort and simulation of cognitive impairment during neuropsychological examination using RBANS and SIMS methods

Overview

One of the current issues in neuropsychological diagnostics is the detection of simulation of cognitive impairment and inadequate effort during the examination. The reason for the simulation of cognitive impairment or lack of effort may be to obtain various benefits, such as financial, material or attention from others. Different types of methods are used to detect simulation and lack of effort. These include tests of cognitive abilities and tests specifically designed to detect malingering. This article focuses on describing two specific methods and their psychometric properties –⁠ The Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) and the Structured Inventory of Malingered Symptomatology (SIMS). Within the RBANS neuropsychological test, effort can be assessed using several indicators. SIMS is an inventory that focuses not only on the assessment of simulated cognitive impairment, but also on the assessment of other areas of psychopathology.

Keywords:

Simulation – Cognitive impairment – insufficient effort – neuropsychological diagnostics – RBANS – SIMS

This is an unauthorised machine translation into English made using the DeepL Translate Pro translator. The editors do not guarantee that the content of the article corresponds fully to the original language version.

 

Introduction

Neuropsychological diagnosis of cognitive functions is an integral part of the examination of neurological patients with a wide range of diagnoses (brain tumours, epilepsy, MS, etc.), but also psychiatric patients. In neurological practice, the examination of cognitive function is performed by means of screening tests to map cognitive deficits and a detailed neuropsychological examination is recommended according to the results. In the context of comprehensive neuropsychological examination, either fixed neuropsychological batteries (e.g., Neuropsychological Assessment Battery [NAB], Halstead-Reitan Neuropsychological Battery [HRNB], etc.) or flexible batteries constructed according to the patient's focus and abilities from a larger number of tests targeting different cognitive functions are used.

In neuropsychological diagnostics it is essential to strive for the best possible validity of the results obtained. Therefore, one of the current topics in neuropsychological diagnostics is the detection of simulation or feigned cognitive impairment and lack of effort during the examination. The reason for simulating or feigning cognitive impairment may be to gain various advantages, e.g. financial or material [1]. Lack of effort can be caused by a variety of factors such as low motivation, fatigue, pain or lack of education by the professional. Detection of simulation or lack of effort during an examination is possible using different types of methods [1-7].

Specifically, the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) [3] and the Structured Inventory of Malingered Symptomatology (SIMS) [1], a specially developed method for detecting simulations of different types of difficulties, can be used as indicators of lack of effort.

 

The concepts of simulation deficit and lack of effort to give a good performance

The term simulation was officially defined in 1980 by the American Psychological Association in the third edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-III) [8]. Subsequent versions of the Diagnostic and Statistical Manual of Mental Disorders, incl. DSM-5, states that simulation is not considered a mental disorder, but should be a focus of professional attention as it has significant psychosocial, economic and legal implications. In the DSM-5, simulation is defined as the deliberate creation of false or highly accentuated physical or psychological symptoms, motivated by extrinsic incentives such as obtaining financial compensation or avoiding a prison sentence [8].

In the literature, the terms simulation, feigning, and malingering can be found in the context of deficit simulation [1,9-11]. These terms refer to a situation where the goal of the test subject is to create the impression of the presence of a disease. Individuals feigning illness report symptoms of illness to an exaggerated degree or complain of a disproportionate reduction in ability. Performance validity tests (PVTs) and symptom validity tests (SVTs) are used to assess the validity of the results [12]. PVTs refer to the determination of whether an individual is performing at a level consistent with their actual ability, and SVTs refer to the assessment of the validity of reported symptoms [13].

Different types of methods and their combinations are used in simulation detection. These include structured interviews, such as the Structured Interview of Reported Symptoms (SIRS) [2]. Selected subtests of neuropsychological and cognitive tests can be used to assess the validity of performance and effort during the examination (e.g., tasks in the Repeatable Battery for the Assessment of Neuropsychological Status [RBANS], tasks in the Wechsler Adult Intelligence Scale [WAIS-III], or the Rey Osterrieth Complex Figure Recognition Task) [14]. Tests specifically designed to detect simulation have also been used, including the Structured Inventory of Malingered Symptomatology (SIMS) [1].

 

RBANS and detection of insufficient effort during examination

The RBANS is a neuropsychological screening method used for a comprehensive and rapid assessment of cognitive function [3]. It is designed for individuals aged 12 to 89 years and takes 20-30 min to administer. The validity of the RBANS results can be assessed using several indicators. These are the Effort index (EI) [15], Effort scale (ES) [16], Performance validity index (PVI) and the Charleston revised index of effort for the RBANS (CRIER). These validity indicators can be used to assess whether the subject is performing at a level consistent with their actual ability during the examination.

 

Effort index

The Effort Index was designed based on studies examining subtests of neuropsychological methods suitable for detecting simulated neurocognitive impairment or lack of effort during examinations [17-20]. The EI is calculated based on the scores of two subtests of the RBANS method, Word Recall and Number Repetition. This is because these types of tasks have been shown to discriminate well between individuals who are simulating a deficit and those who are trying to give a good performance. The EI is obtained by converting the raw scores of the two subtests mentioned above into weighted scores. Based on the recommended cut-off score (> 3), it can then be evaluated whether the individual is giving a performance that is consistent with his or her actual ability [15]. The cut-off score was derived from a research population consisting of patients with different types of neurological brain disease (e.g., epilepsy, mild stage dementia, CMP).

 

Effort scale

Another indicator of the validity of RBANS results is the EC [16]. The EC has been proposed to assess the validity of results in elderly persons with suspected memory impairment (e.g., Alzheimer's disease). This is because EI-based assessments show a high false positive rate in the elderly population for simulated cognitive impairment [21-23]. ES is calculated using the following formula: RBANS ES = (Word Recall -⁠ (Word List Equipment + Story Equipment + Figure Equipment)) + Number Repetition. An ES score of less than 12 is considered an indicator of insufficient effort to give a good performance during the examination [16].

 

Performance validity index and Charleston revised index of effort for the RBANS

In response to the development of the EC index, additional indicators of performance validity in the RBANS, the PVI and CRIER, have been constructed specifically for the elderly population [22]. The PVI is calculated using the following formula: RBANS PVI = Word List Equipment + Story Equipment + Figure Equipment + Number Repetition + Word Recall. A PVI value of less than 42 [22] indicates that the subject is underperforming on the RBANS. In addition, compared to the PVI, the CRIER also accounts for the patients' level of depression as assessed by the Geriatric Depression Scale (GDS). The reason for incorporating the GDS score was the finding that patients who were administered a procedure that was considered to be not valid showed significantly higher levels of depression as assessed by the GDS questionnaire than patients whose procedure was assessed as valid. The CRIER is calculated according to the following formula: RBANS CRIER = Word List Fitting + Story Fitting + Figure Fitting + Number Repetition + Word Recall -⁠ GDS. CRIER values less than 24 indicate the possibility that the test subject is not giving a performance that matches his/her actual ability [22]. Comparing all four validity indicators -⁠ EI, ES, PVI and CRIER -⁠ it was found that CRIER showed the highest values of sensitivity, specificity and area under curve (AUC) [23].

 

SIMS and the detection of simulation of different areas of psychopathology

The SIMS is a 75-item inventory focusing on the simulation of 5 domains of psychopathology. The SIMS is a screening tool for simulating cognitive and psychiatric complaints in a broader context (e.g., forensic and neuropsychological testing). The SIMS was developed for individuals 18 years of age and older. Validation was conducted on a research cohort of forensic, psychiatric and healthy populations [1]. The adaptation of the SIMS method to the Czech environment took place between 2021 and 2023 [24].

The strengths of SIMS are its relative ease of administration and breadth of coverage in the assessment of simulated psychopathology [25]. Completion is done by the examinee independently, and administration is possible in groups or individually. The SIMS assessment can yield 5 outcomes: a total score and 5 subscale scores (Psychosis [P], Neurological Disorder [NI], Amnestic Disorder [AM], Low Intelligence [LI], Affective Disorder [AF]). The SIMS total score provides an estimate of the likelihood that an individual is presenting with symptoms of a psychiatric or cognitive disorder. Evaluation of the 5 subscales and further item-level analyses allow for qualitative analysis and more detailed assessment of the simulation across different domains of psychopathology [1]. It is recommended to use cut-off scores > 14 for the overall SIMS score; for individual subscales, the following cut-off scores are recommended: P > 1, NI > 2, AM > 2, LI > 2, AF > 5 [1,26].

The reliability of SIMS was assessed using internal consistency test-retest reliability. The overall SIMS score was shown to achieve adequate internal consistency with a Cronbach's alpha of 0.72 [27]. The overall SIMS score also showed an adequate test-retest reliability value (r = 0.72) [27].

The validity of the SIMS has been assessed mainly by simulation studies or known-groups studies. Simulation studies were conducted in such a way that the research population was assembled into subgroups according to the type of psychopathology the participants were to simulate [26-28]. Some studies also included research cohorts from forensic and psychiatric populations [27,29,30].

The SIMS was found to have very good discriminatory power, as assessed by receiver operating characteristic (ROC) analysis, with an AUC value of 0.96, which was measured by comparing the results of a group of undergraduate psychology students and a group of hospitalized psychiatric patients [27]. In a cohort of 196 undergraduate students, the SIMS was shown to have remarkable sensitivity and specificity values for both the total SIMS score (96.4% sensitivity; 91.3% specificity) and the SIMS subscale scores (91.5-100% sensitivity; 81.5-93.2% specificity) [28].

In a study involving 116 male offenders recruited from either the prison population or a prison psychiatric hospital, SIMS was shown to detect simulation with the greatest accuracy compared to other methods [30]. Similar results were reached by other authors who included 55 men undergoing court-ordered competency and sanity testing at the time of the offense in their study [29].

 

Conclusion

One of the current topics in neuropsychological diagnosis and forensic psychological examinations is the detection of simulated cognitive impairment and lack of effort during the examination. The reason for simulating cognitive impairment may be to gain various advantages, e.g., financial, material, or attention from others. In the case of insufficient effort, somatic diseases, fatigue, lack of examiner education, etc. may also have an impact. Different types of methods are used to detect simulation and lack of effort, e.g. cognitive ability tests or tests specifically designed to detect simulation. This article focused on the description of two specific methods adapted to the Czech environment and their psychometric properties -⁠ RBANS and SIMS. Within the neuropsychological test RBANS, effort can be assessed by several indicators such as EI, ES, PVI, CRIER. The SIMS is an inventory focusing not only on the assessment of simulated cognitive deficits, but also on the assessment of simulated other areas of psychopathology, namely psychosis, neurological symptoms, intellectual impairment or affective disorder. However, it should be noted that there are many negative consequences associated with a possible incorrect conclusion about the presence of simulation or lack of effort to give a good performance. Therefore, it is advisable to combine the results of multiple methods targeting this issue simultaneously. The authors aimed to enrich the spectrum of diagnostic tools designed to assess these phenomena and to familiarize practitioners with these new procedures.

 

Grant support

The publication was supported by the Grant Agency of Charles University (project no. 62624) and the Ministry of Health of the Czech Republic, grant IG193001 and IG223001.

 

Conflict of interest

Prof. PhDr. Lenka Krámská, Ph.D. is the author of the Czech adaptation of the RBANS method. Prof. PhDr. Lenka Krámská, Ph.D. and Mgr. Marie Krejčová are the authors of the Czech adaptation of the SIMS method.