Malingering, Feigning, and Negative Response Bias in

Gerald Young, Ph.D.
Ontario Psychological Association
68th Annual Conference
February 21, 2015
 Malingering
 Part I: New Literature Review
 Part II: Young (2014). Malingering, Feigning, and
Response Bias in Psychiatric/Psychological Injury:
Implications for Practice & Court.
Thanks to Joyce Chan and Anna Vehter for preparing these slides in such an
appealing and efficient manner.
New Literature Review
New Literature Review
(all paraphrased)
Bass & Halligan (2014) in The Lancet
 The challenge in abnormal health-care-seeking
behaviour is to establish the degree to which the
complainant’s reported symptoms are due to volitional
control, or psychopathology beyond volitional control,
or both.
 “Clinical skills” by themselves are not sufficient to
“detect malingering.”
New Literature Review
Bass & Halligan (2014) in The Lancet
 Non-genuine case of Posttraumatic stress disorder
have been noted (Rosen & Taylor, 2007), possibly
because the diagnosis is based especially on the
evaluee’s subjective report of symptomology (Hall &
Hall, 2006).
 In PTSD, its striking positive symptoms, such as
nightmares and flashbacks, are more readily described
(Hall & Hall, 2007).
New Literature Review
Bass & Halligan (2014) in The Lancet
 15-30% of evaluees with mild Traumatic Brain Injury
describe continuing non-specific symptoms (Ferrari,
 In a patients with complex regional pain syndrome
(type 1), evaluated in disability-seeking contexts, at
least three-quarters failed one indicator of
performance validity (Grieffenstein, Gervais, Baker,
Artiola, & Smith, 2013)
New Literature Review
Chafetz & Underhill (2013)
 The frequency of feigning of disabling illness in
evaluation of adult disability compensation in the
Social Security Disability (SSD) is 45.8%-59.7%.
Note: Does the evidence uniformly agree on the
prevalence/ base rate of malingering being this high.
Some estimates are even higher, others much lower (see
Young 2014a, 2014b)
New Literature Review
Chafetz & Underhill (2013)
 Feigning or exaggeration of symptoms for an external
incentive constitutes malingering.
 Chafetz (2008) found that 45.8% of disability evaluees
failed the TOMM (Tombaugh, 1996) at below-chance
or at chance levels or they failed both the TOMM and
the SVS (for Low Functioning Individuals) (Chafetz,
Abrahams, & Kohlmaier, 2007).
New Literature Review
Chafetz & Underhill (2013)
 In this study, we estimated “The costs of malingering
based on adult mental disorder data” at the Social
Security Administration totaled $20.02 billion in 2011.
New Literature Review
Chafetz & Underhill (2013)
 A letter from U.S. Senator Tom Coburn (2013),
supported by all of neuropsychology’s national (US)
organizations, strongly urged the funding of
performance validity testing in SSD evaluations.
[Note. Young et al. (2015) I will be talking at APA in Toronto on the
disability epidemic in the VA and SSA and the need for
comprehensive scientifically, informed impartial assessments in
disability determinations in these regards.]
New Literature Review
Russo (2014)
 “Conflicting ethical-moral and utilitarian-political
forces” that are inherent in the VA (Department of
Veteran Affairs) act to undermine “accuracy in
evaluation of military veterans’ symptoms by way of
both institutional-wide systemic practices and local
medical center-specific pressures towards collusive
New Literature Review
Russo (2014)
 We need to assess accurately military veteran symptom
validity because of our “personal integrity” in that
there is a lack of judicial overview and few external
consequences for not doing it.
New Literature Review
Russo (2014)
 VA psychologists should protect themselves from
retaliation by informing veterans with a
comprehensive, signed informed consent document
that includes that they (a) will be treated
professionally, and with courtesy and respect; and (b)
are expected to “give their best and most honest
New Literature Review
Wygant & Lareau (2015)
 The DSM definition of malingering includes that it
should be considered when the evaluee displays
symptoms of antisocial personality disorder.
 However, according to Rogers (2008), this screen is
likely to result in an unacceptably high level of false
positive determinations.
New Literature Review
Wygant & Lareau (2015)
 There are some important considerations when PTSD
is a possible diagnosis in a civil case.
 The PTSD criteria in DSM-5 allows for an extreme
number of permutations (636,120) of symptom
combinations in diagnosing PTSD (Galatzer-Levy &
Bryant, 2013).
New Literature Review
Wygant & Lareau (2015)
 Young, Lareau, and Pierre (2014) calculated the
symptom combinations allowed by a DSM-5 diagnosis
of PTSD and other common comorbid disorders.
 There are more than one quintillion different
symptoms combinations possible when dealing with a
PTSD diagnosis and common comorbidities.
 [Note. The comorbidities include the other major
psychological injuries, mTBI and chronic pain (SSD).]
New Literature Review
Wygant & Lareau (2015)
 The major personality inventories included embedded
scales that can help in forensic disability and related
 These include the MMPI-2, MMPI-2-FR, and the PAI
New Literature Review
Sleep, Petty, & Wygant (2015)
 One MMPI-2 over-reporting indicator is the
Infrequency (F) scale.
 It includes rare psychopathological symptoms
endorsed by < 10% of the original MMPI normative
New Literature Review
Sleep, Petty, & Wygant (2015)
 Some F scale items also appear in scales on
 Therefore, psychologically disturbed individuals might
endorse many of these items.
 Also, evaluees attempting to portray themselves in an
unrealistic negative light tend to endorse F scale items
(Graham, 2011).
New Literature Review
Sleep, Petty, & Wygant (2015)
 Arbisi and Ben-Porath (1995) developed the Fp scale to
supplement it.
 Fp was developed in order to detect exaggerated
psychological symptomology, Arbisi and Ben-Porath
 Its 27 items were endorsed “rarely” (by less than 20%)
in two samples of psychiatric inpatients, and also in
the MMPI-2 normative sample.
New Literature Review
Sleep, Petty, & Wygant (2015)
 The FBS includes 43 items that were rationally selected
toward assessing exaggerated post-injury emotional
distress, while also minimizing any preexisting
 [Note. The FBS had been referred to as the Fake Bad
Scale but now is referred to as the Symptom Validity
Scale, although the abbreviation FBS has been kept.]
New Literature Review
Sleep, Petty, & Wygant (2015)
 The MMPI-2-RF includes five symptom over-reporting
respondent validity scales.
 The infrequent Responses (F-r) scale consist of 32 items
distributed throughout the MMPI-2-RF (and is a
counterpart to the F scale).
 As with F, it is measure of general over-reporting.
 It includes items that are rarely endorsed (≤ 10%) in the
normative sample (Tellegen & Ben-Porath, 2008/ 2011).
New Literature Review
Sleep, Petty, & Wygant (2015)
 The Infrequent Somatic Responses (Fs) scale,
developed by Wygant, Ben-Porath, and Arbisi (2004),
is a 16-item scale new to the MMPI-2-RF.
 This scale was developed to help identify noncredible
reports of somatic symptoms.
 Its developers employed a rare-symptom approach.
 Fs includes items with somatic content that are rarely
endorsed by medical/ chronic pain patients.
New Literature Review
Sleep, Petty, & Wygant (2015)
 The FBS-r scale, unlike the FBS, includes 12 items on
other validity scales (Hoelzle et al., 2012).
 Elevated scores on FBS-r are indicative associated with
over-reporting of somatic and cognitive deficits.
New Literature Review
Sleep, Petty, & Wygant (2015)
 The Response Bias Scale (RBS; Gervais, Ben-Porath,
Wygant, & Green, 2007) was developed based on the
MMPI-2-RF but is compatible with the MMPI-2-RF.
 The RBS over-reporting scale was developed to
identify self-reported symptomology (regardless of
item content) that are associated with poor
performance on cognitive PVTs
New Literature Review
Sleep, Petty, & Wygant (2015)
 Wygant et al. (2011) found the Fs and FBS-r scales were
“good at identifying noncredible neurocognitive and
somatic symptoms” in evaluees undergoing litigation
related compensation-seeking disability evaluations
classified at maingering levels using related with the
MND (Malingered Neurocognitive Dysfunction; Slick
et al., 1999) and MPRD (Malingered Pain-Related
Disability; Bianchini et al., 2005) criteria.
New Literature Review
Buddin et al. (2014)
 The TOMM is the most used performance validity test
(PVT) in neuropsychology, but it does not include a
measure of response consistency, which is important
in the measurement of credible evaluee presentation.
 To address this need, Gunner, Miele, Lynch, and
McCaffrey (2012) developed the Albany Consistency
Index (ACI).
New Literature Review
Buddin et al. (2014)
 He developed the Invalid Forgetting Frequency Index
(IFFI) for the same purpose.
 In a retrospective case-control study of 59 forensic
cases from an outpatient clinic in Southern Kansas, we
found that the IFFI was superior psychometrically to
both the TOMM indexes and the ACI.
New Literature Review
Kulas, Axelrod, & Rinaldi (2014)
 The TOMM is among the more popular free-standing
performance validity measures (PVMs).
 New indices have been developed for it: Trial 1
(Denning, 2012); TOMMe10 (Denning, 2012); and
Albany Consistency Index (ACI; Gunner, Miele, Lynch,
& McCaffrey, 2012).
New Literature Review
Kulas, Axelrod, & Rinaldi (2014)
 We examined the performance of these measures in a
mixed clinical sample of military veterans who were
referred for neuropsychological assessment.
 All five examined measures allowed “good to excellent”
discrimination of evaluees who had failed two/ three
alternate measures of performance validity.
New Literature Review
Bashem et al. (2014)
 Their study examined five widely-used PVTs:
 The Test of Memory Malingering (TOMM),
 Medical Symptom Validity Test (MSVT),
 Reliable Digit Span (RDS),
 Word Choice Test (WCT), and
 California Verbal Learning Test – Forced Choice (CVLTFC).
New Literature Review
Bashem et al. (2014)
 They examined 51 adults with genuine moderate-to-
severe TBI, along with 58 demographicallycomparable healthy adults who were coached to
simulate memory impairment.
 The results showed nearly equivalent discrimination
ability as individual predictors of the TOMM, MSVT,
and CVLT-FC,and each of the tests “markedly
outperformed” the WCT and RDS in this regard.
New Literature Review
Bashem et al. (2014)
 They also found that combining PVTs using Bayesian
information criterion statistics showed that diagnostic
accuracy evidenced only small to modest growth when
the number of PVTs was increased beyond two.
[Note. But is their research supported in every case? For
a positive response to the question, see Larrabee (2014);
for a negative one, see Odland et al. (2015).]
New Literature Review
Larrabee (2014)
 PVT error rates using Monte Carlo simulation (see
Berthelson et al., 2013) were compared in two
nonmalingering clinical samples.
 Berthelson et al.’s findings had queried the validity of
using 2 or more PVT failures as representing probable
invalid clinical neuropsychological presentation.
New Literature Review
Larrabee (2014)
 In his work, at a per-test false-positive rate of 10%,
Monte Carlo simulation overestimated error rates.
 These clinical results support the practice of using the
threshold of ≥ 2 testing validity failures as
representative of probable invalid clinical
neuropsychological presentation.
New Literature Review
Crighton et al. (2014)
 Can two brief measures, Modified Somatic Perception
Questionnaire (MSPQ) and the Pain Disability Index
(PDI) screen effectively for malingering in relation to
the MPRD criteria?
New Literature Review
Crighton et al. (2014)
 They compared 144 disability litigants, predominantly
presenting a history of musculoskeletal injuries with
psychiatric overlay, with 167 nonlitigating pain patients,
predominantly in treatment for chronic back pain issues
and other musculoskeletal conditions
 The results suggested that both the MSPQ and PDI are
useful in screening pain malingering in forensic
 The MSPQ, though performed the better in differentiating
the two groups.
New Literature Review
Crighton et al. (2014)
 Although useful as screeners, the MSPQ and the PDI
should not be used as a definitive source to make
malingering determinations.
 In screening in clinical settings of individuals
evaluation of disability for pain, scores of ≥ 14 on the
MSPQ or ≥ 54 on the PDI should be used.
New Literature Review
Bianchini et al. (2014)
 They examined the accuracy of the MSPQ and PDI in
relation to classification of evaluees according to the
 They used a retrospective cohort of patients with
chronic pain, n = 328 and a simulator group (college
students, n = 98)
 Results showed that MSPQ and PCI accurately
differentiated Not-MPRD from MPRD cases.
New Literature Review
Bianchini et al. (2014)
 Their Table 7 showed the following for cut scores on the
MSPQ and PDI as screeners for comprehensive
psychological evaluation and/ or functional capacity
 Score Levels: MSPQ ≥ 17; PDI ≥ 62
 With these thresholds, the recommended interpretation is
that malingering is “likely involved” in evaluee
New Literature Review
Bianchini et al. (2014)
 The authors concluded that while high scores on these
screeners reflect an increased probability of
malingering, no matter how high, the scores are
insufficient for a diagnosis of MPRD.
New Literature Review
Odland et al. (2015)
 The aim of the Monte Carlo simulation is to provide base
rate data and recommendations for interpretation of
multiple validity indicators (assuming varying correlations
between each PVT ), at a range of specificity and sensitivity
 First, we validated Monte Carlo methodology across 24
embedded and standalone validity indicators in seven
compensation-seeking clinical samples found in prior
New Literature Review
Odland et al. (2015)
 Samples included evaluees with psychotic and non-
psychotic psychiatric disorders, as well as different
neurological conditions.
 The simulation that we undertook arrived at strategies
“for clinical integration of base rate data for advanced
administration and interpretation of multiple validity
indicators.” (p. 1)
New Literature Review
Odland et al. (2015)
 In this type of research, small sample sizes lower the
likelihood that significant findings represent a true
effect and also they exaggerate actual effects when they
exist (Button et al., 2013).
 The use of research designs that cross-validate
embedded PVTs using no-incentive samples selected
on the basis of passing other freestanding PVTs (e.g.,
Victor et al., 2009)also is inadequate.
New Literature Review
Odland et al. (2015)
 Procedures such as this bias the samples, because evaluees
who provide a credible test performance, but who
happened to fail PVTs for reasons unrelated to incentive,
are excluded from the final no-incentive sample (Bilder,
Sugar, & Hellemann, 2014).
 Such research may result in apparently well-validated PVTs
that, in clinical practice, actually misclassify more than
expected numbers of evaluees who provide credible test
New Literature Review
Odland et al. (2015)
 Other research arrives at different conclusions, such as
Larrabee (2014).
 A weakness of the approach advocated by Larrabee
(2014) is that combinations of embedded and/ or
standalone PVTs cannot be chained using Liklihood
Ratios (LRs) unless every PVT included is independent
from every other PVT used.
New Literature Review
Odland et al. (2015)
 However, the psychometric characteristics of the
validity indcators cited by Larrabee (2014) were
fundamentally different from those of Berthelson and
colleagues (2013) that he criticized [and also those in
the meta-analysis by Sollman and Berry (2011)].
 As more PVTs/ SVTs are used in evaluations, the
probability of increases in type I error rates increases.
New Literature Review
Odland et al. (2015)
 There does not appear to be fixed number of PVT or SVT
failures one should use in these type of assessments (e.g.,
two or three; e.g., Larrabee et al., 2007).
 We developed a decision-tree (Figure III; and Tables III-V
and Appendices I-VIII) that increases efficiency during
 It provides feedback regarding the costs/ benefits
associated with administering additional standalone PVTs
in light of the number of test passes/ failures already
New Literature Review
Bigler (2014)
 Significantly below chance performance on relevant
testing is the sine qua non indicator for malingering in
neuropsychological assessment.
 However, there are substantial interpretative problems
with SVT (symptom validity test) performance that is
below the cut-point yet far above chance.
New Literature Review
Bigler (2014)
 This intermediate, grey-zone performance on SVT measure
requires examining other data in an evaluation.
 Neuroimaging results may be “key” in understanding
better the meaning of such grey-zone SVT performance.
[Note. Does the evidence support this conclusion? Bigler
(2014) used case studies to support it.]
New Literature Review
Bush, Heilbronner, & Ruff (2014)
 The ASAPIL (Association for Scientific Advancement
in Psychological Injury and Law;
position statement on the need for effective
assessment and testing of evaluee negative response
bias and exaggerated/ malingered presentation and
performance is based on articles in the journal
Psychological Injury and Law ( by Bush
et al. (2014) and Young (2014a).
New Literature Review
Bush, Heilbronner, & Ruff (2014)
 In the following, summarize the key points of the two
 This summary highlights the ethical underpinnings in
doing this type of forensic and related work.
 It makes no specific test recommendations, though.
 This decision on the tests to use are the responsibility
of each practitioner.
New Literature Review
Bush, Heilbronner, & Ruff (2014)
 This ASAPIL position statement by Bush et al. (2014)
promotes ethical psychological practice in forensics,
legal contexts.
 It reviews issues in validity assessment and their
ethical foundations.
 Evaluees in psychological injury cases could have
strong incentives to minimize prior problems and to
emphasize postevent or posttrauma symptoms.
New Literature Review
Bush, Heilbronner, & Ruff (2014)
 Therefore, it is essential to assess evaluee validity as
part of forensic psychological evaluations.
 Psychological instruments have focused increasingly
on evaluee validity scales (see Heilbronner & Henry,
2013 for a review).
 However, a multi-method approach is needed to
determine the validity of an examinee’s overall
approach in an assessment.
New Literature Review
Bush, Heilbronner, & Ruff (2014)
 Appropriate methods in this regard commonly include
some combination of the following:
 Psychometric measures having respondent validity scales,
 Free-standing validity measures of validity,
 Embedded indices within tests of cognitive ability,
 Behavioral observations,
 Information in records, and
 Interviews of the evaluee and of collated sources.
New Literature Review
Bush, Heilbronner, & Ruff (2014)
 Even when test cutoff scores are reliable and valid, none
can “capture the intent” that underlies on examinee’s
invalid test results.
 Use of probabilistic language (e.g., possible, probable,
definite) based on structured diagnostic criteria should be
used in determinations of malingering (MND; Slick,
Sherman, & Iverson, 1999).
[Note. But how valid is the MND? (see Young, 2014b)]
New Literature Review
Bush, Heilbronner, & Ruff (2014)
 Or, in many situations, presenting invalid evaluee
results as representative of feigning should be used
instead of attributing to malingering.
 Only validity measures having appropriate
psychometric properties are used in malingering
 They should be selected based on the characteristics of
the evaluee and on the circumstance(s) of the referral.
New Literature Review
Bush, Heilbronner, & Ruff (2014)
 When interpreting of the results of testing, we need to
consider all the relevant reliable data.
 Conclusions in opinions and testimony are developed
that best fit the full data set in these regards.
 One’s conclusions are arrived at independently and
not for the desires of the referral source.
New Literature Review
Bush, Heilbronner, & Ruff (2014)
 When the evidence is insufficient with respect to
motivation, volition, intention, and consciousness ,
evaluators are wary of making inferences on these matters.
 However, evaluators do not avoid making a judgment on
these matters when sufficient evidence allows for it.
 “Best practices in forensic psychological evaluations consist
of a multi-method, evidence-based validity assessment
process that includes psychometric measures of validity.”
(p. 202)
New Literature Review
Bush, Heilbronner, & Ruff (2014)
 Young (2014a) provided resource material to the Bush et al.
(2014) authors to help in their writing ASAPIL’s position
statement on performance validity testing.
 These include material from the APA forensic practice
guidelines, the APA ethics code, prior statements on PVTs
and the 2014 standards for psychological testing and
 Young (2014a) organized material on the topic according to
a revised 10-prinicle model of ethics in psychology (the
APA code includes 5).
New Literature Review
Young (2014a)
 Note that, unlike the case for the DSM, other
approaches to defining malingering do not include
exaggeration in their definitions.
 Given the difficulties in clearly defining malingering, it
is not surprising that estimates of its base rate or
prevalence vary.
New Literature Review
Young (2014a)
 The estimates range from below 10% (even 1%) to over
 More likely, problematic presentations and
performances, in general, express the latter range, with
the percentage of outright malingering in the former
range (as reviewed in Young, 2014b).
New Literature Review
Young (2014a)
 DSM-IV-IR. The DSM-IV defines malingering as the
“intentional production” of “grossly exaggerated” or
“false” “psychological” and “physical” symptoms that
derives from “motivation by external incentives” for
example, in obtaining financial compensation.
New Literature Review
Young (2014b)
 According to Young (2014b), an improved definition of
malingering would involve exclusion of the term
“production,” given its connotation of symptomology
being evident, for the terms “presentation,” which is
neutral in this regard, and so allows for a completely
absence of genuine symptoms.
New Literature Review
Young (2014b)
 Therefore, malingering should be defined as: the
intentional presentation with false or grossly
exaggerated symptoms [physical, mental health, or
both; full or partial; mild, moderate, or severe], for
purposes of obtaining an external incentive, such as
monetary compensation for an injury and/ or
avoiding/ evading work, military duty, or criminal
New Literature Review
Kulas, Axelrod, & Rinaldi (2014)
a. Malingering-related tests: RDS, CVLT-FC, WMT
b. Malingering detection system: Failure on 2-3 of the
measures (suboptimal effort)
c. Sample: N = 126 military (US) veterans (outpatient,
d. Malingering-related groups: Optimal effort,
intermediate, suboptimal
e. % for each group: 41 (52), 49 (62), 10 (12)
New Literature Review
Buddin et al. (2014)
a. Malingering-related tests: RDS, FTT, VPA-II Recog,
VR-II recog; WMT (no/ GMIP), FBS, FBS-r
b. Malingering detection system: MND modified (need
to fall 2 PVTs)
c. Sample: N = 59 forensic outpatients
d. Malingering-related groups: 0, probable, definite
e. % for each group: 58 (34), 39 (23), 34 (2)
New Literature Review
Larrabee (2014)
a. Malingering-related tests: BVFD, FTT, RDS, CVMT;
CRM, WCST, FBS (note, raw score ≥ 21)
b. Malingering detection system: MND
c. Sample: N = 41 “malingering” (mTBI sample [and
N=54 clinical subjects, nonlitigating]
d. Malingering-related groups: Probable, Definite
e. % for each group: 41 (17), 59 (24)
New Literature Review
Crighton et al. (2014)
a. Malingering-related tests: MMPI-2-RF, TOMM, LMT,
b. Malingering detection system: MPRD
c. Sample: N = 133(5) forensic disability cases [and pain
d. Malingering-related groups: 0, possible, probable/
e. % for each group: 53 (N=71), 24 (N=32), 24 (N=32)
New Literature Review
Bianchini et al. (2014)
a. Malingering-related tests: CVLT (1, 2); MMPI-2; PDRT;
b. Malingering detection system: MPRD
c. Sample: N = 305 clinical pain patients with incentive [and
controls; simulators]
d. Malingering-related groups: 0; 1 ambiguity;
Indeterminate; Possible Malingering, probable,
e. % for each group: 10, 6, 11, 34, 27, 46%
Table 1 Cognitive Biases That Could Affect in
Forensic Evaluators (Adapted)
(also Conjunction
fallacy; Base rate
Overemphasizing evidence resembling a typical
prototype representation (also disregarding the
probability an outcome will occur (base rate) in
determining a specific outcome likelihood)
(also Confirmation
(What You See Is
All There Is))
Overestimating the probability of an event
occurrence when other instances of it are quite easy
to recall (also selective data gathering/
interpretation toward favored hypothesis)
Table 1 Cognitive Biases That Could Affect in
Forensic Evaluators (Adapted)
(also Framing/
Data first encountered are more influential than those
encountered later (also, arriving at a different
conclusion from the same data, depending on factors
such as how or by whom that data is presented)
Adapted from Neal & Grisso (2014)
New Literature Review
Murrie & Boccacciini (2015)
 They asked whether forensic experts can remain objective
and accurate, given that when they are retained by one side
or the other in adversarial legal proceedings?
 The authors summarized recent field and experimental
 They conclude that working for one or the other side in an
adversarial/ legal proceeding/ case, a “substantial portion”
of opinions offered by experts drift towards the referral
source, even when using apparently objective procedures
and instruments.
New Literature Review
Murrie & Boccacciini (2015)
 Murrie and Boccacciini called this process of the
adversarial divide affecting expert objectively adversarial
 The mechanisms that underlie this process among workers
in forensics are likely similar to the unconscious heuristics
and cognitive biases to apparently at work in arriving at
judgment in other settings, to understand.
 Further research is needed to ultimately reduce the process
of adversarial allegiance.
New Literature Review
Odland (2015) Emailed Review of Young (2014b) [cited with
 Thank you again for the PDF of your 2014 book, and my
compliments to you for constructing a very well designed
and comprehensive system.
 Your Diagnostic System for Malingering is the most
comprehensive and integrative available, it is presented
with meticulous detail and is intuitive, especially with the
supplementary materials. In particular, I believe there is
strong support for having degrees of certainty of response
bias included in your system.
New Literature Review
Odland Review of Young (2014b)
 The dimensional approach outlined in your work answers a
question I was not entirely certain of how to respond to as I
wrote the recent paper for PIL: How does one make sense out of a
certain number of PVT failures that is neither normal nor
indicative definite invalidity - the areas of gray defined in your
 Assigning varying degrees of certainty with regard to
classification ... intuitively seems to have improved ecological
validity over extant models.
 Re-analysis of already published data using your system, as I'm
sure you are aware, could reshape the fields conceptualization of
validity, base rates, and interpretive approach.
New Literature Review
Odland Review of Young (2014b)
 The set of 60 weighting rules that accompanies the system
is also straight forward, and in agreement with our recent
study in PIL.
 After reading through the rules, I found myself wondering
why such efforts had not previously been made given that
the utility of any model hinges on veracity of its underlying
 This is a long overdue component to any malingering
system, and is a major contribution in that it removes the
guesswork from determining what factors suffice in the
rubric of a larger classification system.
New Literature Review
Odland Review of Young (2014b)
 It is refreshing to see such precision in outlining how validity measures
are to be used within the system structure.
The 60 rules appear well supported by a large body of literature.
E.g., Rule 11: “…5-8 of them indicates significant doubt about the
credibility of the evaluee...”
This is consistent with population estimates of false-positive rates
associated with failing 5-8 out of10-15 measures, given previous
mention that all tests are valid and each have a false-positive rate less
than or equal to 10% (Sollman & Barry Metanalysis).
From my perspective, other logic-driven rules also have strong
psychometric support ... (e.g., Rule 33.).
As an aside, in addition to the Malingering Detection System, I found
your analysis of base rates in Chapter 2 quite enlightening.
Arbisi, P. A., & Ben-Porath, Y. S. (1995). An MMPI-2 infrequent response scale
for use with psychopathological populations: The infrequencypsychopathology scale, F(p). Psychological Assessment, 7, 424-431.
Bashem, J. R., Rapport, L. J., Miller, J. B., Hanks, R. A., Axelrod, B. N., & Millis, S.
R. (2014). Comparisons of five performance validity indices in Bona
fide and simulated
Neuropsychologist, 28, 851-875.
Bass, C., & Halligan, P. (2014). Factitious disorders and malingering: Challenges
for clinical assessment and management. The Lancet, 383, 1422-1432.
Berthelson, L., Mulchan, S. S., Odland, A. P., Miller, L. J., & Mittenberg, W. (2013).
False positive diagnosis of malingering due to the use of multiple effort
tests. Brain Injury, 27, 909-916.
Bianchini, K. J., Aguerrevere, L. E., Guise, B. J., Ord, J. S., Etherton, J. L., Meyers, J.
E., Soignier, R. D., Greve, K. W., Curtis, K. L., & Bui, J. (2014). Accuracy of
the modified somatic perception questionnaire and pain disability index
in the detection of malingered pain-related disability in chronic pain.
The Clinical Neuropsychologist, 28, 1376-1394.
Bianchini, K. J., Greve, K. W., & Glenn, G. (2005). Review article: On the diagnosis
of malingered pain- related disability: Lessons from cognitive
malingering research. The Spine Journal, 5, 404-417.
Bigler, E. D. (2014). Effort, symptom validity testing, performance validity testing and
traumatic brain injury. Brain Injury, 28, 1623-1638.
Bilder, R. M., Sugar, C. A., & Hellemann, G. S. (2014). Cumulative false positive rates given
multiple performance validity tests: Commentary on Davis and Millis (2014) and
Larrabee (2014). The Clinical Neuropsychologist, 28, 1212-1223.
Buddin Jr., W. H., Schroeder, R. W., Hargrave, D. D., Von Dran, E. J., Campbell, E. B.,
Brockman, C. J., Heinrichs, R. J., & Baade, L. E. (2014). An examination of the
frequency of invalid forgetting on the test of memory malingering. The Clinical
Neuropsychologist, 28, 525-542.
Bush, S. S., Heilbronner, R. L., & Ruff, R. M. (2014). Psychological assessment of symptom
and performance validity, response bias, and malingering: Official position of the
Association for Scientific Advancement in Psychological Injury and Law.
Psychological Injury and Law, 7, 197-205.
Button, K. S., Loannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., &
Munafo, M. R. (2013). Power failure: Why small sample size undermines the
reliability of neuroscience. Nature Reviews Neuroscience, 14, 365-376.
Chafetz, M. D. (2008). Malingering on the Social Security disability consultative
examination: Predictors and base rates. The Clinical Neuropsychologist, 22, 529546.
Chafetz, M., D., Abrahams, J. P., & Kohlmaier, J. (2007). Malingering on the Social Security
disability consultative examination: A new rating scale. Achieves of Clinical
Neuropsychology, 22, 1-14.
Chafetz, M., & Underhill, J. (2013). Estimated costs of malingered disability.
Archives of Clinical Neuropsychology, 28, 633-639.
Coburn, T. (2013). Letter to SSA Commissioner Michael Astrue urging the
requirement of validity testing in disability determinations. Retrieved
Crighton, A. H., Wygant, D. B., Applegate, K. C., Umlauf, R. L., & Granacher, R.
(2014). Can brief measures effectively screen for pain and somatic
malingering? Examination of the modified somatic perception
questionnaire and pain disability index. The Spine Journal, 14, 20422050.
Denning, J. H. (2012). The efficiency and accuracy of the Test of Memory
Malingering trial 1, errors on the first 10 items of the test of memory
malingering, and five embedded measures in predicting invalid test
performance. Archives of Clinical Neuropsychology, 27, 417-432.
Ferrari, R. (2011). Minor head injury: Do you get what you expect? Journal of
Neurology, Neurosurgery, & Psychiatry, 82, 826.
Galatzer-Levy, I. R., & Bryant, R. A. (2013). 636,120 ways to have posttraumatic
stress disorder. Perspectives on Psychological Science, 8, 651-662.
Gervais, R. O., Ben-Porath, Y. S., Wygant, D. B., & Green, P. (2007). Development
and validation of a Response Bias scale (RBS) for the MMPI-2.
Assessment, 14, 196-208.
Graham, J. R. (2011). The MMPI-2: Assessing personality and psychopathology (5th
ed.). New York: Oxford University Press.
Greiffenstein, M., Gervais, R., Baker, W. J., Artiola, L., & Smith, H. (2013).
Symptom validity testing in medically unexplained pain: A chronic
regional pain syndrome type 1 case series. The Clinical
Neuropsychologist, 27, 138-147.
Gunner, J. H., Miele, A. S., Lynch, J. K., & McCaffrey, R. J. (2012). The Albany
Consistency Index for the Test of Memory Malingering. Archives of
Clinical Neuropsychology, 27, 1-9.
Hall, R. C. W., & Hall, R. C. W. (2006). Malingering of PTSD: Forensic and
diagnostic considerations, characteristics of malingerers and clinical
presentations. General Hospital Psychiatry, 28, 525-535.
Hall, R. C. W., & Hall, R. C. W. (2007). Detection of malingered PTSD: An
overview of clinical, psychometric, and physiological assessment: Where
do we stand? Journal of Forensic Sciences, 52, 717-725.
Heilbronner, R. L., & Henry, G. K. (2013). Psychological assessment of symptom
magnification in mild traumatic brain injury cases. In D. A. Carone, & S.
S. Bush (Eds.), Mild traumatic brain injury: Symptom validity assessment
and malingering (pp. 193-202). New York: Springer Publishing Company.
Hoelzle, J. B., Nelson, N. W., & Arbisi, P. A. (2012). MMPI-2 and MMPI-2Restructured Form validity scales: Complementary approaches to
evaluate response validity. Psychological Injury and Law, 5, 174-191.
Kulas, J. F., Axelrod, B. N., & Rinaldi, A. R. (2014). Cross-validation of
supplemental test of memory malingering scores as performance validity
measures. Psychological Injury and Law, 7, 236-244.
Larrabee, G. J. (2014). False-positive rates associated with the use of multiple
performance and symptom validity tests. Archives of Clinical
Neuropsychology, 29, 364-373.
Larrabee, G. J., Greiffenstein, M. F., Greve, K. W., & Bianchini, K. J. (2007).
Refining diagnostic criteria for malingering. In G. J. Larrabee (Ed.),
Assessment of malingered neuropsychological deficits (pp. 334-372). New
York: Oxford University Press.
Murrie, D. C., & Boccaccini, M. T. (2015). Adversarial allegiance among forensic
evaluators. Annual Review of Law and Social Science, 15.
Neal, T. M. S., & Grisso, T. (2014). The cognitive underpinnings of bias in
forensic mental health evaluations. Psychology, Public Policy, and Law,
20, 200-211.
Odland, A. (2015). Emailed personal communication of his review of Young
Odland, A., Lammy, A., Martin, P., Grote, C., & Mittenberg, W. (2015). Advanced
administration and interpretation of multiple validity tests.
Psychological Injury and Law, 8.
Rogers, R. (2008). Detection strategies for malingering and defensiveness. In R.
Rogers (Ed.), Clinical assessment of malingering and deception (3rd ed.,
pp. 14-38). New York: The Guilford Press.
Rosen, G. M., & Taylor, S. (2007). Pseudo-PTSD. Journal of Anxiety Disorders, 21,
Russo, A. C. (2014). Assessing veteran symptom validity. Psychological Injury
and Law, 7, 178-190.
Sleep, C. E., Petty, J. A., & Wygant, D. B. (2015). Framing the results:
Assessment of response bias through select self-report measures in
psychological injury evaluations. Psychological Injury and Law, 8.
Sollman, M. J., & Berry, D. T. R. (2011). Detection of inadequate effort on
neuropsychological testing: A meta-analytic update and extension.
Archives of Clinical Neuropsychology, 26, 774-789.
Tellegen, A., & Ben-Porath, Y. S. (2008/ 2011). MMPI-2-RF (Minnesota Multiphasic
Personality Inventory-2 Restructured Form): Technical manual.
Minneapolis, MN: University of Minnesota Press.
Tombaugh, T. N. (1996). TOMM: Test of memory malingering. North Tonawanda,
NY: Multi-Health Systems.
Victor, T. L., Boone, K. B., Serpa, J. G., Buehler, J., & Ziegler, E. A. (2009).
Interpreting the meaning of multiple symptom validity test failure. The
Clinical Neuropsychologist, 23, 297-313.
Wygant, D. B., Anderson, J. L., Sellbom, M., Rapier, J. L., Allgeier, L. M., &
Granacher, R. P. (2011). Association of the MMPI-2 Restructured Form
(MMPI-2-RF) validity scales with structured malingering criteria.
Psychological Injury and Law, 4, 13-23.
Wygant, D. B., Ben-Porath, Y. S., & Arbisi, P. A. (2004, May). Development and
initial validation of a scale to detect infrequent somatic complaints.
Poster presented at the 39th Annual Symposium on Recent
Developments of the MMPI-2/ MMPI-A, Minneapolis, MN.
Wygant, D. B., & Lareau, C. R. (2015). Civil and criminal forensic psychological
assessment: Similarities and unique challenges. Psychological Injury and
Law, 8.
Young, G. (2014a). Resource material for ethical psychological assessment of
symptom and performance validity, including malingering.
Psychological Injury and Law, 7, 206-235.
Young, G. (2014b). Malingering, feigning, and response bias in psychiatric/
psychological injury: Implications for practice and court. Dordrecht:
Springer Science + Business Media.
Young, G., Lareau, C., & Pierre, B. (2014). One quintillion ways to have PTSD
comorbidity: Recommendations for the disordered DSM-5.
Psychological Injury and Law, 7, 61-74.
Young, G., Marx, B., & Evans, B. (2015). Toward balanced VA policies in
psychological injury disability assessment. Poster presented at the
Annual convention of the American Psychological Association, APA,
Toronto, August.
Implications for Practice and Court
Gerald Young, PhD
New York: Springer SBM
About this book
 What is psychological injury?
 PTSD, chronic pain, TBI (esp. mTBI)
 How to detect malingering, feigning and related
response biases in psychological/psychiatric injury
 Takes a look at approaches to and inconsistencies in
the field, even in defining malingering and
establishing its base rate
 Proposes solutions for these concerns in practice and
in court
Introduction to the Field of Psychological Injury
Field of Psychological Injury and Law
 Intersection of:
 forensic psychology
 law (e.g., evidence, tort, insurance)
 assessment/ testing, including of malingering
 disability and return to work
 trauma psychology
 chronic pain
 neuropsychology
 rehabilitation
 harassment/ discrimination
What is “Psychological Injury”?
 Psychological or psychiatric condition associated with
an event that leads, or may lead, to a lawsuit in tort
action or other legal-related claims.
 For example:
 Tort, e.g., after a motor vehicle collision, and
Worker Compensation,
Veteran’s Administration (VA), and
Social Security Administration (SSA)
What is “Psychological Injury”? (contd.)
 PTSD, mTBI, and persistent postconcussive symptoms
(PPCS, aftereffects of a concussion) and chronic pain
may be involved in psychological injury cases.
 Note that these are not necessarily DSM disorders
 Disorders that involve mood or emotions, such as
depression, anxiety, fear or phobia, and adjustment
disorder are also typically manifested.
 These conditions/ disorders may occur separately or in
combination (co-morbidity).
Claimable injuries
 They might result from events at issue only, such as a
motor vehicular collision or other negligent action.
 They might be exacerbations of pre-existing
conditions or vulnerabilities, and the event at issue is
not a sole cause but a material one in the
multifactorial causal nexus
 Functionally, they might cause impairments,
limitations, and disabilities
Legal definition
 Considered a:
 mental harm,
 suffering,
 damage,
 impairment, or
 dysfunction
 It is caused to a person as a direct result of some action
or failure to act by some individual, perhaps as an
exacerbation of a pre-existing condition
Admissibility in court
 Daubert v. Merrell Dow Pharmaceuticals, Inc. (1993)
 Supreme Court case in the United States of America that
provided basis for admissibility of scientific evidence in
 The Daubert criteria establish the parameters of goof
compared to poor or junk science, e.g., not just general
acceptance but also peer review, falsifiability, etc.
 Other cases/ rulings have been made and they
constitute the “Daubert trilogy”
Admissibility in court (contd.)
 General Electric Co. v. Joiner (1997) and Kumho Tire
Co. v. Carmichael (1999) are the two other cases
comprised of the “Daubert trilogy”
 R. v. Mohan (1994)
 Canadian case with similar outcome
 However, some states still abide by the Frye standard
og general acceptance for scientific admissibility in
court (Frye v. United States, 293 F. 1013, 34 ALR 145 (D.
C. Cir 1923).
Malingering Definitions and Base Rates
Definition of Malingering
 Psychiatrists and the DSM-5:
 […] “intentional production of false or grossly
exaggerated physical or psychological symptoms” that
derives from “motivation by external incentives” (in the
 For example: obtaining financial compensation
 APA (American Psychological Association) Dictionary:
 Exaggeration is not referenced in the definition of
Definition of Malingering (contd.)
 In the APA’s dictionary of psychological terms:
 Malingering is the deliberate feigning of an illness or
disability that is motivated to achieve a particular
specific external factor or outcome
 For example: faking illness in order to obtain financial
 Black’s law dictionary contains a similar definition:
 It includes feigning for external incentives, but there is
no reference to an exaggeration component.
 Malingering is further e;laborated; it may involve a
combination of four factors:
 (a) the referral context is medicolegal;
 (b) the objective findings are “markedly” discrepant with
the evaluee’s claimed “stress or disability;”
 (c) the evaluee exhibits a lack of cooperation with the
assessment procedure or with suggested treatments; and
 (d) he or she is diagnosed with antisocial personality
 If there is any combination of these factors,
malingering should be strongly suspected.
Base Rate Inconsistency in DSM
 Mittenberg et al. (2002) undertook an often-cited
study of the base rate of malingering. It involved a
survey of professionals in the field.
 Several inconsistencies were discovered in my reading:
 The definitions of malingering and exaggeration were
not provided to the respondents in the study
 Exaggeration was not specified for severity
 Malingering was conflated with exaggeration in the
percentages offered
Consistencies - Boone
 Boone (2011a) examined the psychological testing needed
to infer an attribution of malingering.
 Her references cited that failure on two or more tests of
effort can best discriminate between credible and noncredible populations (e.g., Victor, Boone, Serpa, Buehler, &
Ziegler, 2009).
 The more there are “failed indicators,” the more confidence
one can have in conclusions.
 Numerous failed tests can be used as irrefutable evidence
in court.
Boone (contd.)
 She referred to “differential diagnosis” in ruling
malingering in or out.
 I noted that according to the DSM-IV-TR, malingering
is not a diagnosis, but rather a class of behaviours
given a “V-code”.
 Further, the use of qualitative and idiographic data
gathered from interviews of evaluees does not mean
that “art” rather than science is being used when
conducting forensic mental health assessments.
Boone (contd.)
 In the practice of differential diagnosis of malingering
(Heilbrun et al., 2009, on FMHA):
 (a) all the relevant data are gathered in a comprehensive
manner, including from testing and interviews;
 (b) all possible hypotheses are considered for the
conclusions; and
 (c) the final conclusions reached are supported by both
the evidence gathered and the state-of-the-art science in
the literature that is applicable to the case at hand.
Consistencies – Rogers and Granacher
 They reviewed the conceptualization and assessment
of malingering.
 And specified that gross exaggeration in the DSM-IV-
TR’s definition of malingering is unlikely to involve
“minor or isolated amplifications of symptoms.”
 This is consistent with the present view that the DSM
approach to defining malingering as involving only
gross exaggerations and not also minor ones is valid.
Consistencies – Sollman & Berry (2011)
 The evidence of base rates for “suboptimal effort”
(generic term instead of malingering) in clinical
practice is equal to or greater than 40% in some
 They believed that mild exaggeration may also be
referred to in regards to suboptimal effort.
 By including all types of suboptimal effort and reasons
for them, the base rate may be more than 40%
Consistencies - Others
 Merckelbach, Jelicic, and Pieters (2011)
 In a study with undergraduates students, conscious
feigning may eventually lead to symptom conviction and
actual somatoform disorders.
 Merckelbach and Merten (2012) elaborated that:
 Conscious other-deception could turn into unconscious self-
 That being said, Medically Unexplained Symptoms
(MUS) might develop via anxiety or over-focus on the
Consistencies - Others (contd)
 Larrabee, Millis, and Meyers (2009)
 The standard base rate of malingering in the field
should be acknowledged as 40% plus/ minus 10.
 Larrabee (2007) & Mittenberg et al. (2002)
 Persistent neuropsychological deficit in cases of mTBI,
may increase the malingering rate to as high as 88%.
Consistencies – Others (contd.)
 However, I note that it is premature to presume
malingering if:
 (a) if the basic definition of malingering is unclear,
 (b) if intent is hard to assess,
 (c) if the assessment instruments themselves have
disparate even if relevant findings, etc.
 Thus, when results are not definitive, assessors should
use terms other than malingering for doubt about an
evaluee’s symptom presentation/ performance.
 For example: lack of credibility or feigning
Malingering Maximized
 Greve, Ord, Bianchini, and Curtis (2009) conducted a
review of over 500 consecutive referrals to a private
 Of the 508 patients, up to 36% were classified as
probable or definite malingerers, with 10.4% as definite
malingerers (using the MPRD).
 The authors concluded that the prevalence of
malingering to be between 20% and 50%, depending on
the type of analysis undertaken.
 However, I noted that authors’ estimated is actually
more toward 10% according to their own data.
Greve et al. (2009)
 They added that nearly half of their participants
showed evidence of “symptom magnification”.
 This concept is broader than malingering and includes
symptom exaggeration
 1/3 of the sample met the criteria for “possible” MPRD
(Malingered Pain-Related Disability)
 2/3 of the sample showed “some form of exaggeration”
 However, I note: not all exaggeration reflects
Wygant et al. (2011)
 They looked at 251 individual compensation-seeking
 They applied both the MND (Malingering of
Neurocognitive Dysfunction) and MPRD diagnostic
systems to classify individuals as:
 incentives only,
 possible malingering,
 probable malingering, and
 definite malingering
Wygant et al. (2011) - Results
 I calculated in their data that 30.7% were classified in
the probable/ definite malingering group
 Consistent with prior estimates that malingering-related
classifications should be in the 30-50% range.
 Definite malingering was found at only 8% in this
 Consistent with other research that the figure for
outright malingering should be about 10%.
Lee at al. (2012) – Gender differences
 They investigated gender differences on the FBS in claimants
who had undergone non-neurological medico-legal disability
 They used the Slick et al. MND criteria and SVT results (WMT,
TOMM, CARB, etc.)
 For definite malingering, they needed a score below chance on an
SVT and, for probable malingering, it involved a below cut score on
one or more SVTs.
 Of 1,209 patients,
 Over 30% met the criteria for non-credible responders (definite,
 But, only 1.5% (19) met the criteria for definite malingering.
Conclusions by Rogers and Bender (2012)
 Rogers and Bender (2012) suggested that the previous research
on malingering base rates may be accurate, but the publications
have conceptual and methodological limitations.
 They also described that there are multiple explanations for
incomplete/ suboptimal effort in testing other than the reason of
 Such as, pain, depression, stress, and expectation of failure on the
part of the evaluee and reaction to evaluator factors.
 Elhai et al. (2012) indicated that other evaluee factors, such as
being ill, poor sleep, and medication side effects, might also
affect results.
Detecting Malingering
MND – Malingering of Neurocognitive
Definition: Volitional exaggeration or fabrication of
cognitive dysfunction for the purpose of obtaining
substantial material gain, or avoiding or escaping
formal duty or responsibility.
MND (Contd.)
 Substantial material gain includes money, goods, or
services of nontrivial value (e.g., financial compensation of
personal injury).
 Formal duties are actions that people are legally obligated
to perform (e.g., prison, military, or public service, or child
support payments or other financial obligations).
 Formal responsibilities are those that involve
accountability or liability in legal proceedings (e.g.,
competency to stand trial).
Definite MND
 Individual presents clear and compelling evidence of
volitional (conscious) exaggeration or fabrication of
cognitive dysfunction and the absence of plausible
alternative explanations.
 There is specific diagnostic criteria to be met:
1. Presence of a substantial external incentive [Criterion A]
2. Definite negative response bias [Criterion B1]
3. Behaviors meeting necessary criteria from group B are not
fully accounted for by Psychiatric, Neurological, or
Developmental Factors [Criterion D]
MND criteria – Rogers vs Boone
 Rogers et al. (2011a) described and critically analyzed the
Slick et al. (1999) diagnostic criteria for MND
 They believed that the different levels in certainty of response
bias/ malingering (possible, probable, and definite) lead to
the over-classification of malingering.
 Rogers et al. (2011) conducted a literature review of MND
and found:
 The base rate for malingering over the studies was only 5.3%
on average.
 The rate for probable malingering was 21.2% and, further, it
was as high as 50% in one study.
MND criteria – Rogers vs Boone
 Boone (2011) argued that Rogers et al. (2011)
exaggerated the failings of the MND model.
 She also believed that the model is accurate in
identifying feigners/malingerers.
 However, she argued that there should be revision to
the B2 MND criterion to require failure on three or
more SVTs (“>2 SVTs”).
 She also recommended to stop the use of malingered
ND as a description of evaluees and rather use a term
such as “noncredible neurocognitive dysfunction.”
MND criteria – Rogers vs Boone
 Rogers et al. (2011b) noted that there is more
agreement than disagreement between Rogers et al.
(2011a) and Boone (2011) about the MND model.
 Overall, recommendations from both parties would go
very far in improving the MND definition and criteria.
Young (2008)
 Young (2008) recommended that the prevalence of
wider noncredible neurocognitive dysfunction, such as
regarding chronic pain and PTSD in tort claims for
personal injury and in litigation, should be considered
more broadly.
 He argued that the prevalence rate of wider
noncredible neurocognitive dysfunction and related
dissimulation could potentially be even higher than
50% by having a broader outlook than the narrow
construct of malingering.
Response Bias
 McGrath et al. (2010) reviewed response bias as a
source of error variance in clinical assessments.
 They reviewed response bias indicators as suppressors
or moderators of the validity of various substantive
psychological indicators.
 Only 12 out of 44 sets of data examined supported the
effectiveness of response bias measurement.
 However, Rohling et al (2011) provided multiple
reasons and referred to data why response bias
measurement in forensic disability cases is pertinent.
Wiggins et al. (2012)
 Wiggins et al.’s (2012) study supported the validity,
value, and need to verify response bias in forensic
disability and related assessments via the use of
MMPI-2-RF validity scales.
 They found a 25% base rate level for significant
negative response bias. This 25 % level found includes
malingering, per se, as only one possibility.
 Overall, the research is accumulating that the position
of McGrath et al., despite its contrary nature on the
matter, does not diminish the relevance of
psychometric testing in malingering determination.
Detection instruments
 There are three classes of instruments that permit
testers to identify malingering, feigning, and related
response biases:
 personality tests,
 stand-alone tests (forced-choice tests, structured
interviews, and others), and
 embedded neuropsychological tests
Personality tests – MMPI
 FBS - Symptom Validity Scale (formerly referred to as
the Fake Bad Scale)
 Nelson et al. (2010) conducted a meta-analysis on the
FBS via 32 studies.
 They found a large omnibus effect size. There were
large effect sizes when:
 Participant effort was known to be insufficient
 Assessments took place for traumatic brain injury (TBI)
 Thus, there is strong support for the use of the FBS in
forensic neuropsychology practice.
Reseach on the MMPI-2-RF
 Detection of feigned psychiatric disorders (Marion et
al., 2011).
 Discriminate a malinger group from controls (Wygant
et al., 2011).
 Used in research with cognitive impairments or
disorders related to epilepsy (Locke et al., 2010; Rogers
et al., 2011).
 Differentiate valid and invalid somatic and pain
complaints (Burchett & Ben-Porath, 2010, 2011; etc.)
 Used in a study of Attention Deficit Hyperactivity
Disorder (ADHD) (Harp et al., 2011).
Personality tests – PAI
 There is less substantive research for the PAI than
there is for the MMPI for psychological injury, however
it still has utility with:
 Pain-related samples
 PTSD samples
Personality test – MMCI-III
 MMCI-III - Millon Clinical Multiaxial Inventory, Third
 There are opposing opinions on its use in the field of
psychological injury:
 Kane and Dvoskin (2011) recommended against its usage
in the psychiatric/ psychological injury context.
 Whereas, Aguerrevere, Greve, Bianchini, and Ord (2011)
demonstrated that it may be useful in identifying
intentional symptom exaggeration in TBI claimants
Stand-Alone Test – SIRS & SIRS-2
 SIRS - Structured Interview of Reported Symptoms
 Rogers et al., (2009) indicated that SIRS may have some
utility in the psychiatric/psychological injury population.
 SIRS-2 has received some mix reviews and requires further
research and validation to identify its usefulness.
 In response to some negative reviews, Rogers & Bender
(2012) indicated that it has the ability to differentiate
feigned and genuine responding, with effect sizes being
large to very large.
Stand-Alone Test – TOMM
 TOMM – Test of Memory Malingering
 There has been a surge of research on the validity of
this test.
 Brooks et al. (2011) found the first TOMM trial to be a
valid indicator.
 [Note. The research on the TOMM is flourishing]
Green and SVTs
 Green developed several SVTs (Symptom Validity
 the WMT;
 the MSVT, Medical Symptom Validity Test; Green
 the NV-MSVT, Nonverbal Medical Symptom Validity
Test; Green, 2008
Briere and PTSD
 Briere developed tests that contain scales to evaluate
respondent validity when assessing PTSD.
 DAPS, Detailed Assessment of Posttraumatic Stress; the
TSI-2, Trauma Symptom Inventory, Second Edition;
Briere, 2011
 Gray et al. (2010) demonstrated that the Atypical
Response Scale of the TSI-2 helped discriminate
simulated from genuine PTSD
Embedded Neuropsychological
 There are many embedded neuropsychological indices
within commonly used assessments, which help
determine examinee credibility.
 Digit Span from WAIS-R and WAIS-III
 Reliable Digit Span (RDS), Logical Memory Recognition
(LMR) and Discriminant Function (DF) from WMS-R
Embedded Neuropsychological
tests (Contd.)
 There are also individual indices that can be
embedded into the used batteries.
 WCST – Wisconsin Card Sorting Test
 AVLT RMT – Rey Auditory Verbal Learning Test
Recognition Memory Test
 CVLT – California Verbal Learning Test
 FTT – Finger Tapping Test
 RCFT – Rey Complex Figure Test
Boone (2013) on Malingering
 1) Boone (2013) gave little importance to the
Malingered Neurocognitive Dysfunction (MND)
approach of Slick et al. (1999) for the detection of
malingering in the forensic neuropsychological
examination. [In contrast to Boone (2011).]
 2) She supported the use of the MMPI-2-RF to help in
malingering and related negative response bias
 3) She de-emphasized the specific calculation
procedures promoted by Larrabee (2008) in
combining SVTs to determine the probability of
Symptom Validity Tests (SVTs)
 Boone (2013) explained
 1) How do they work?
 2) How are they validated?
 3) Test selection
 4) Discounting failed and passed SVTs
1) How do SVTs work?
 SVTs can be in a “forced-choice” format, where an evaluee
must choose between two possible answers.
They have a 50% chance of selecting the correct answer.
Scores significantly below chance indicate noncredible
About 15% of real-world noncredible evaluees will score in
this “significantly below” range.
Two or more failures can provide a more accurate result.
On these tests, noncredible evaluees will score below the
probability level at p = .05, which translates to a score of
<19/ 50 on the TOMM.
2) How are SVTs validated?
 In order for a test to be effective, it needs to be highly
sensitive and specific.
 Sensitivity and specificity are in reciprocal balance (as
one gets higher, the other gets lower).
 Generally, specificity is set at ≥ 90%.
3) Test Selection
 For sensitivity, values of < 40 % are considered low,
whereas those at 40 – 69% are moderate, and those at
or above 70% are high.
 SVTs should be chosen to allow for repeated testing of
response bias throughout the evaluation (Boone,
 To avoid redundancy, SVTs could be minimally or
moderately correlated with each other, but not
 Some tests are easier to coach or are more readily
available on the internet for self-coaching
4) Discounting Failed and Passed SVTs
 According to Boone (2013), there are various factors
that could account for failed SVTs, such as:
 Lower intelligence or dementia, as opposed to feigning.
 Cultural factors may also be of influence
 However, depression and pain should not affect the
 In some cases, Boone believed that passed SVTs
should be discounted.
One issue with Boone (2013)
 She indicated that validity indicator failure, such as on
an F scale, should not be considered a cry for help, but
rather be considered an act of feigning/exaggeration.
 Iverson’s (2006) ethical stance about how to interpret
failed SVTs does not necessarily exclude explaining
them as a cry for help.
 Therefore, these scores could also be a sign of
catastrophizing or of valid desperation (cry for help).
Proposed Criteria for Diagnosis of
Malingered Pain-related Disability
 There are 5 proposed criteria to assist in an effective
diagnosis of malingered pain-related disability:
 Criteria A: Evidence of significant external incentive.
i.e., personal injury settlement or disability pension
 Criteria B: Evidence from physical evaluation.
Physical evaluations are consistent with exaggeration or
feigning of physical disability.
Proposed Criteria for MPRD
 Criteria C: Evidence from cognitive/perceptual
(neuropsychological) testing.
Patient’s cognitive capacities are consistent with exaggeration
or feigning of cognitive disability.
 Criteria D: Evidence from self-report.
Reported symptoms, complaints, or limitations are consistent
with exaggeration or feigning of physical, cognitive and
emotional disability.
Proposed Criteria for MPRD
 Criteria E: Behavior meeting necessary criteria from
groups B, C, and D are not fully accounted for by
psychiatric, neurologic, or developmental factors.
Likely volitional act aimed at achieving some secondary gain
The presence of a documented pathology, illness, or injury
(including psychiatric illness) does not automatically exclude
the possibility of a MPRD diagnosis.
Malingered PTSD Detection System
Covered In This Chapter
 Presents a diagnostic model to detect malingered PTSD in
forensic disability and related evaluations.
 There is no adequate malingered PTSD detection system,
thus Young (2014) based his recommendation on:
 The Slick et al. MND criteria and recommendations by
Rogers et al (2011a,b) and Boone (2011).
 The MPRD criteria system created by Bianchini et al. (2005).
 Suggestions made by Rubenzer (2009) to detect malingered
PTSD (he used a point system).
 Revisions of already-developed models for neurocognitive
and pain domains (MND and MPRD).
Model for Response Styles/Biases
 Young (2014) also offers a survey in the form a questionnaire to
help determine the prevalence/base rates for these response
styles and biases (Figure 5.2).
 There is a 7-point range of potential response styles and biases
derived from Slick et al. (1999) MND testing approach.
(a) definite malingering;
(b) definite response bias, to
(c) probable,
(d) probable/ possible (gray zone),
(e) possible, and then
(f) minimal negative response bias; and
(g) absent bias.
Comparison to Slick et al. (1999)
MND model
 Slick et al. (1999) terms
 Young (?) terms
 (a) overt malingering,
 (a) definite malingering;
 (b) noncredible gross
exaggeration/ inconsistency,
 (b) definite,
 (c) noncredible moderate
exaggeration/ inconsistency,
 (d) probable/ possible (gray
 (d) indeterminate gray zone,
 (e) credible but possible moderate
exaggeration/ inconsistency,
 (f) credible but mild exaggeration/
inconsistency, and
 (g) no exaggeration/ inconsistency.
 (c) probable,
 (e) possible,
 (f) minimal negative response
bias; and
 (g) absent bias.
Figure 5.2a Self-Unfavorable Presentations/ Performances
(Psychological, Psychiatric) in Evaluees According to Response
Biases (R/B) in Testing and/ or Inconsistencies/ Discrepancies (I/D).
Figure 5.2b Self-Unfavorable Presentations/ Performances
(Psychological, Psychiatric) in Evaluees According to Response
Biases (R/B) in Testing and/ or Inconsistencies/ Discrepancies (I/D).
Figure 5.2 Explained
 Presents an integrated model related to malingering
and other response styles/ biases and motivations.
 It also suggests an approximate normal distribution
that these styles, biases and motivations should take.
 The terms used in this figure acknowledge that there
are many cases in these assessments that can fall into
an indeterminate or gray zone.
 The most difficult cases to assess are those that fall
into these “gray zones”.
The “Gray Zone”
 The gray zone may vary in size and direction depending the
 Variance may depend on the plaintiff or the source of
 The margin of the gray zone may become better defined by:
 Conducting thorough research of both models.
 Apply the models with equal rigor across all sources of
 Therefore, there needs to be a comprehensive, impartial,
scientifically-informed approach to studying these models.
 This zone corresponds to the real world of evaluees and
evaluators –> ecological and face validity.
Inconsistencies/Discrepancies in the
MND and MPRD Systems (Contd.)
 It appears that multiple types of inconsistencies/
discrepancies used by Slick et al. and Bianchini et al.,
overlap in the two systems. They are related to:
 (a) standard test data;
 (b) self-report;
 (c) observations;
 (d) known patterns of brain functioning;
 (e) known patterns of physiological functioning;
 (f ) collateral information; and
 (g) documented information.
Inconsistencies/Discrepancies in the
MND and MPRD Systems (Contd.)
 Information in these inconsistency/ discrepancy
categories could be about pre-event, event, or postevent factors. It might refer to either pre-event history,
such as prior police or criminal record, or event/ postevent symptoms, impairments, dysfunctions, and
disabilities, if any.
 The inconsistencies/ discrepancies could be
compelling/ marked/ substantial or otherwise, but no
clear guidelines are offered to help distinguish the
compelling type.
Inconsistencies/Discrepancies in the
MND and MPRD Systems (Contd.)
 Test data for the systems derive from measures of
exaggeration, fabrication, and suspected malingering,
such as in SVTs (symptom validity tests), but also tests
like the MMPIs, which include clinical scales, as well.
 Better ways of combining the different types of tests
data in detecting malingering need to be created.
Young (2014) Detection Model
 Proposed are more types and more combinations of
inconsistencies/ discrepancies, as well as permitting
their notation within categories.
 Better definition and clarification of terminology.
 Adopted a three-level system:
 First tier of compelling inconsistencies into less and
more extreme versions
 The third tier relates to moderate and nontrivial
inconsistencies/ discrepancies.
Feigned Posttraumatic Stress Disorder
Disability/ Dysfunction system (F-PTSDR-D)
 The model proposed for evaluating whether there is
non-credible, feigned, or malingered PTSD-related
presentation or performance response bias is called
the Feigned Posttraumatic Stress Disorder
Disability/ Dysfunction system (F-PTSDR-D).
 7 principles were used in its construction.
The 7 Principles behind the
 Principle 1:
 The range of malingering and related biases is expanded by
placing them on a continuum of seven categories –
(a) definite malingering;
(b) definite,
(c) probable,
(d) probable/ possible (gray zone),
(e) possible,
(f) minimal negative response bias; and
(g) absent bias.
 In between the probable and possible negative bias points,
there is so-called gray zone.
The 7 Principles behind the
F-PTSDR-D (Contd.)
 Principle 2:
The F-PTSDR-D system has more extensive clarification on:
How to test results related to failing/ missing critical thresholds
Inconsistencies/ discrepancies in evaluee presentation and
performance that will be use to determine whether there is a presence
of malingering and related biases.
The model was created similar for PTSD, pain and TBI, but PTSD-
specific examples were included.
These examples concerned response to psychological and
pharmacological interventions, in particular.
The 7 Principles behind the
F-PTSDR-D (Contd.)
 Principle 3:
Within the one rating scheme of the F-PTSDR-D system,
there are various types of psychological comprehensive
and scaled measures. For example:
(i) personality inventories, such as the MMPI family ones;
(ii) stand-alone validity/ effort tests, including forced-choice
ones that have two relevant criteria -- at or below-chance
accuracy level (e.g., in a two-alternative test) and a less
rigorous pass-fail level (related to cut scores); and
(iii) embedded measures in cognitive/ neurological tests,
such as those related to digit span.
The 7 Principles behind the
F-PTSDR-D (Contd.)
 Principle 4:
The present system provides a comprehensive list of 60
rules for weighing the tests/ measures/ scales/ indicators
so that they are used effectively.
 Principle 5:
There are elaborate cautions provided at the end of the
system, which are meant to assure reliability and validity.
The 7 Principles behind the
F-PTSDR-D (Contd.)
 Principle 6:
Normally, 5-8 failed test results are needed for malingering and
related attributions when there is nothing else in the assessment at
However, personality inventories, such as the MMPI-2-RF, can
contribute up to four of the five validity indicator failures.
Moreover, even clinical patterns on them can be used in system
The 7 Principles behind the
F-PTSDR-D (Contd.)
Aside from cases with extremely compelling evidence, such as frank
admission or indisputable videographic evidence,
definite malingering can be attributed in cases in which
(a) two or more forced-choice measures are failed at the below-chance level,
(b) there are five or more test failures on other valid psychometric measures,
(c) there are three or more compelling inconsistencies,
(d) any combinations of these types of evidence are found, or
(e) other evidence replaces the weighting of these three types of evidence,
such as extreme scores on valid psychometric tests or an overall judgment
of the file that adds weight.
When the latter obtains then, when numerical data can be gathered, three
test failures could be sufficient to attribute malingering, everything else
being equal.
The 7 Principles behind the
F-PTSDR-D (Contd.)
As for assigning definite response bias, the criteria above apply,
except that they involve one-forced choice test, not two, four other
tests, not five or more, and two compelling inconsistencies, not
three or more, with none of the extreme nature involved.
In terms of probable response bias, the criteria exclude forced-choice
test failure, but consider three other test failures, not four, and one
compelling inconsistency, not two.
To conclude, the reader will note that Larrabee (2012) emphasized
three if not two failures on relevant tests as very strong evidence of
malingering., All things considered, the present system arrives at a
protocol that might give a comparable weighting to such test
The 7 Principles behind the
F-PTSDR-D (Contd.)
 Principle 7:
There is a three-level system of degree. The levels of
inconsistencies/ discrepancies in the present system that
(a) most or extremely compelling, as per frank admission,
videographic evidence, etc.;
(b) compelling with respect to other file material that is to the
level of a marked/ substantial inconsistency/ discrepancy; and
(c) moderate/ nontrivial ones.
10 Specific Changes to the MND/
MPRD Systems
(1) Aside from below-chance performance on a forcedchoice measure, definite negative response bias can be
assigned based on performing below cut-off on five or
more well-validated tests designed to measure
psychiatric/ psychological exaggeration or fabrication.
(2) The sequence of definite, probable, and possible
response bias involves failing four, three, and two such
tests, respectively.
10 Specific Changes to the
MND/ MPRD Systems (Contd.)
(3) The measures to detect feigning/ malingering and
related biases might derive from any of personality
inventories, stand-alone tests, and those aimed at
detecting improbable symptoms and the like (e.g., SIRS2).
(4) Other measures might be informative in this regard,
such as PTSD-dedicated ones (DAPS, Detailed
Assessment of Posttraumatic Stress; Briere, 2001) and
embedded cognitive (neuropsychological) indices.
10 Specific Changes to the
MND/ MPRD Systems (Contd.)
(5) Where warranted, and if properly validated for the
question at hand, the most recent, valid tests should be
used, such as the MMPI-2-RF, the SIRS-2, and the TSI-2
(Trauma Symptom Inventory, Second Edition; Briere,
[Note. As of 2014, the evidence supports use of the
MMPI-2-RF in the present system but not yet the SIRS-2
or the TSI-2.]
10 Specific Changes to the MND/
MPRD Systems (Contd.)
(6) Inconsistencies/ discrepancies in self-report, reliable
documents, collateral information, behavioral observations,
etc., that are compelling, marked, and substantial, in
particular, are adjunct sources of valid data in malingering
When psychological testing is impossible, inconsistencies/
discrepancies can be used by themselves to determine
malingering and other response bias. This would allow
psychiatrists and other mental health workers to use the
system, albeit with less data available.
10 Specific Changes to the
MND/ MPRD Systems (Contd.)
(7) Causality needs to be considered, as well, as part of
non-testing factors; for example, pre-existing and/ or
extraneous, nonevent-related concurrent causal factors
could fully explain an evaluee’s presentation and
performance after an index event.
(8) Provisos are added that the diagnostic system should
be used prudently and conservatively because of the
harm that could be caused by false attributions of
malingering and related biases.
10 Specific Changes to the
MND/ MPRD Systems (Contd.)
(9) The data set gathered should be comprehensive,
scientifically-informed, and impartial, and
interpretations should consider all the reliable data from
a scientific reasoning basis.
10 Specific Changes to the MND/
MPRD Systems (Contd.)
(10) Motivation should not be imputed, for example, that
malingering is present, without irrefutable or
incontrovertible evidence.
However, the astute assessor will know how to use language
that denies the credibility of the patient, and even to
significant degrees, when the data warrant this conclusion.
In this regard, the system is meant to cover the full range of
response biases, from mild exaggeration to clearly
malingered, so that unlike the case for MND and MPRD, its
title involves the word “feigned” instead of “malinger.”
Table 5.4 Outline of Proposed Criteria for Non-credible Feigned
Posttraumatic Stress Disorder and Related Disability/
Dysfunction (F-PTSDR-D)
Criterion A: Evidence of significant external
Criterion B: Evidence from psychological
Table 5.4 Outline of Proposed Criteria for Non-credible Feigned
Posttraumatic Stress Disorder and Related Disability/
Dysfunction (F-PTSDR-D)
A. Different Degrees of Certainty of Response Bias,
According to Psychological Testing
A1) Definite Malingering.
The evidence is incontrovertible
A2) Definite negative response bias
e.g., Below chance performance (p<.05) on one forced
choice measure
A3) Probable negative response bias.
Table 5.4 Outline of Proposed Criteria for Non-credible Feigned
Posttraumatic Stress Disorder and Related Disability/
Dysfunction (F-PTSDR-D)
A. Different Degrees of Certainty of Response Bias,
According to Psychological Testing
A3-4) Intermediate (Probable to possible, gray zone)
negative response bias
A4) Possible negative response bias.
A5) Minimal negative response bias.
A6) No evident response bias.
Table 5.4 Outline of Proposed Criteria for Non-credible Feigned
Posttraumatic Stress Disorder and Related Disability/
Dysfunction (F-PTSDR-D)
Weighting Rules for Test Batteries
60 rules are quite explicit:
Rule 1: Two pathways; Rule 2: Forced-choice; Rule 3: Tests; Rule 4: MMPI family; Rule 5: Other tests
needed; Rule 6: Improbable symptoms, etc.; Rule 7: PTSD; Rule 8: Pain; Rule 9: Cognitive (embedded);
Rule 10: 10-15 Primary; Rule 11: 5-8 Critical; Rule 12: Not at cut-off; Rule 13: Neuropsychology; Rule 14:
Supplementary tests; Rule 15: Secondary information; Rule 16: Pattern analysis; Rule 17: Limited cognitive
testing; Rule 18: Neuropsychological path; Rule 19: Test independence; Rule 20: Prioritizing; Rule 21:
Exception 1; Rule 22: Exception 2; Rule 23: Exception 3; Rule 24: Exception 4; Rule 25: Maximum use 1;
Rule 26: Omnibus tests; Rule 27: Dedicated tests; Rule 28: Nondedicated tests; Rule 29: Maximum use 2;
Rule 30: Adjusted rating, lowering it; Rule 31: Adjusted rating, raising it; Rule 32: Patterns; Rule 33:
Preselection; Rule 34: Fishing expeditions; Rule 35: No exceptions; Rule 36: Ecological validity; Rule 37:
Warnings; Rule 38: Qualifications; Rule 39: State-of-the-art; Rule 40: No harm; Rule 41: Cognitive/
Neuropsychological testing; Rule 42: Rating cognitive/ neuropsychological tests; Rule 43: Cognitive/
Neuropsychological and Regular rating; Rule 44: Positive results for only one of the two paths; Rule 45:
Cognitive/ Neuropsychological path alone; Rule 46: Test selection; Rule 47: Minimal testing; Rule 48:
Less than minimal testing; Rule 49: Less testing yet doing enough; Rule 50: Larrabee (2012); Rule 51:
Justify less testing; Rule 52: Supplementary evaluators; Rule 53: Seconding team work; Rule 54: Leading
team work; Rule 55: Interdisciplinary assessments; Rule 56: Specific dedicated tests; Rule 57: Altering
rules on testing and test battery; Rule 58: Special populations; Rule 59: Consider whole file; Rule 60:
Combining test data with inconsistencies/ discrepancies.
Table 5.4 Outline of Proposed Criteria for Non-credible Feigned
Posttraumatic Stress Disorder and Related Disability/
Dysfunction (F-PTSDR-D)
Criterion C: Evidence from Inconsistencies/
a) Inconsistencies/ Discrepancies in Conjunction with Testing
a1) Inconsistency/ Discrepancy between cognitive/ neurocognitive test
data and known patterns of brain functioning. (Inconsistency #1)
a2) Inconsistency/ Discrepancy, either marked/ substantial or moderate/
nontrivial, between test data of PTSD-related symptoms after event at
claim and known patterns of physiological reactivity. (Inconsistency #2)
a3) Inconsistency/ Discrepancy, either marked/ substantial or moderate/
nontrivial, between test data and self-report. (Inconsistency #3)
Table 5.4 Outline of Proposed Criteria for Non-credible
Feigned Posttraumatic Stress Disorder and Related
Disability/ Dysfunction (F-PTSDR-D)
a4) Inconsistency/ Discrepancy, either marked/ substantial or
moderate/ nontrivial, between test data of PTSD-related symptoms
after event at claim and verbal and/ or nonverbal observed
behavior/ symptoms/ complaints/ limitations/ functions.
(Inconsistency #4)
a5) Inconsistency/ Discrepancy, either marked/ substantial or
moderate/ nontrivial, between test data and information reported
by reliable informants/ collaterals. (Inconsistency #5)
a6) Inconsistency/ Discrepancy, either marked/ substantial or
moderate/ nontrivial, between test data of PTSD-related symptoms
after event at claim and information reported in reliable
documents. (Inconsistency #6)
Table 5.4 Outline of Proposed Criteria for Non-credible
Feigned Posttraumatic Stress Disorder and Related
Disability/ Dysfunction (F-PTSDR-D)
b) Inconsistencies/ Discrepancies in Conjunction with SelfReport (other than with testing)
Inconsistency/ Discrepancy between such self-report and any of
the following:
b1) Known patterns of brain function. (Inconsistency #7)
b2) Known patterns of physiological function. (Inconsistency #8)
b3) Observed behavior/ symptoms/ complaints/ limitations/
functions. (Inconsistency #9)
b4) Information reported by reliable informants/ collaterals, such as
primary care physicians and spouses. (Inconsistency #10)
b5) Information reported in reliable documents, such as by primary
care physicians and other mental health professionals. (Inconsistency
Table 5.4 Outline of Proposed Criteria for Non-credible
Feigned Posttraumatic Stress Disorder and Related
Disability/ Dysfunction (F-PTSDR-D)
c) Inconsistencies/ Discrepancies in Conjunction with
Observations (other than with testing and with self-report)
Inconsistency/ Discrepancy between such observations and
any of the following:
c1) Known patterns of brain function. (Inconsistency #12)
c2) Known patterns of physiological function. (Inconsistency
c3) Information reported by reliable informants/ collaterals.
(Inconsistency #14)
c4) Information reported in reliable documents. (Inconsistency
Table 5.4 Outline of Proposed Criteria for Non-credible
Feigned Posttraumatic Stress Disorder and Related
Disability/ Dysfunction (F-PTSDR-D)
d) Inconsistencies/ Discrepancies in Conjunction with
Collateral Information (other than with testing, self-report,
and observations)
Inconsistency/ Discrepancy between such information and
any of the following:
d1) Known patterns of brain function. (Inconsistency #16)
d2) Known patterns of physiological function.
(Inconsistency #17)
d3) Information reported in reliable documents.
(Inconsistency #18)
Table 5.4 Outline of Proposed Criteria for Non-credible
Feigned Posttraumatic Stress Disorder and Related
Disability/ Dysfunction (F-PTSDR-D)
e) Inconsistencies/ Discrepancies in Conjunction with
Documentation (other than with testing, self-report,
observations, and collateral information)
Inconsistency/ Discrepancy between such documentation
and any of the following:
e1) Known patterns of brain function. (Inconsistency #19)
e2) Known patterns of physiological function.
(Inconsistency #20)
Table 5.4 Outline of Proposed Criteria for Non-credible
Feigned Posttraumatic Stress Disorder and Related
Disability/ Dysfunction (F-PTSDR-D)
f) Inconsistencies/ Discrepancies Within Major Data
Sources (not between them which are scored above)
f1) Known patterns of brain function (Inconsistency #21)
f2) Known patterns of physiological function. (Inconsistency
f3) Self-report. (Inconsistency #23)
f4) Observed behavior/ symptoms/ complaints/ limitations/
functions. (Inconsistency #24)
f5) Information reported by reliable informants/ collaterals.
(Inconsistency #25)
f6) Information reported in reliable documents. (Inconsistency
Table 5.4 Outline of Proposed Criteria for Non-credible
Feigned Posttraumatic Stress Disorder and Related
Disability/ Dysfunction (F-PTSDR-D)
g) Other, Miscellaneous Inconsistencies/
g1) No causality attributable to the event at claim, despite
the evaluee’s insistence. (Inconsistency #27)
g2) Only minimal causality attributable. (Inconsistency
g3) Material-level causality but not to the degree insisted.
(Inconsistency #29)
g4) Other. (Inconsistency #30)
Table 5.4 Outline of Proposed Criteria for Non-credible
Feigned Posttraumatic Stress Disorder and Related
Disability/ Dysfunction (F-PTSDR-D)
B. Different Degrees of Certainty of Response Bias,
According to Inconsistencies/ Discrepancies
B1) Definite Malingering.
B2) Definite negative response bias.
B3) Probable negative response bias.
B3-4) Intermediate (Probable to possible, gray zone)
negative response bias.
Table 5.4 Outline of Proposed Criteria for Non-credible
Feigned Posttraumatic Stress Disorder and Related
Disability/ Dysfunction (F-PTSDR-D)
This list can be used in Intermediate Negative Response Bias
a) Personality disorder of a problematic nature.
b) Blaming everyone and anything, overly suspicious.
c) Not trying to mitigate loss.
d) Unduly adopting the sick role.
e) Somatization.
f) Failure to treat substance abuse impeding progress.
g) Failure to take recommended medications.
h) Refusing a work-hardening trial, modified duties, retraining.
i) Catastrophizing/ crying out for help.
j) Any other confound that is documentable, such as attorney or
similar coaching.
Table 5.4 Outline of Proposed Criteria for Non-credible
Feigned Posttraumatic Stress Disorder and Related
Disability/ Dysfunction (F-PTSDR-D)
As well, five factors derived from the pre-event:
k) Psychiatric/ self harm/ substance abuse history.
l) Criminal/ legal/ problematic military history; history of
deceit/ fraud.
m) History of irregularity in/ dissatisfaction with work or
other role at issue.
n) History of irregularity in/ dissatisfaction with family,
o) History of financial stresses/ bankruptcies/ unsupported
Table 5.4 Outline of Proposed Criteria for Non-credible
Feigned Posttraumatic Stress Disorder and Related
Disability/ Dysfunction (F-PTSDR-D)
B4) Possible negative response bias.
B5) Minimal negative response bias.
B6) No evident response bias.
Criterion D: Behaviors meeting necessary criteria from
groups B and C are not fully accounted for by psychiatric,
neurologic, developmental, or other factors.
Abbreviations. PTSD = posttraumatic stress disorder.
Adapted from Bianchini, Greve, & Glynn (2005), which in turn was
adapted from Slick, Sherman, & Iverson (1999).
Ambiguity of Malingering by Faust et al. (2012a, b)
Faust et al. (2012a, b) Commentary
 In comparison to definitive or nearly definitive cases of
malingering, there is not nearly enough research on
the ambiguous cases.
 Most research centers around the extremes on the
continuum: definite malingering and absent bias.
 Due to this lack of research, there is a lack of
representativeness and generalization.
 Faust et al. (2012a) believe that there should be focus
on results stemming from both clinical practice and
research using known-groups.
Faust et al. (2012a, b) Commentary
 They believed that malingering is a hypothetical
construct that is inferred from data rather than being
directly observed.
 The field needs clear definitions.
 Nevertheless, in identifying malingering, practitioners
should not overvalue operational definitions or the
provided diagnostic criteria due to the lack of
scientific research available.
 This is especially true in the legal setting.
Diagnostic Continuum
 Faust et al. (2012a) believed that an evaluator may find
themselves on a continuum of completely accurate to
completely inaccurate in their malingering
assessments due to many factors. Such as:
 Inaccurate data due to measurement error
 Evaluee’s lack of sleep the night prior to the assessment
 Overall, the evaluator might be wrong to prematurely
conclude whether there was a case of malingering or
its absence.
How to improve assessment?
 Faust et al. (2012a) believe that there needs to be better
understanding of test results of malingering, effort, clinical
and forensic neuropsychological functioning and their
 All sources of inaccuracy, misrepresentation, and their
intentional or non-intentional bases must be considered
when evaluating forensic and clinical cases involving
potential malingering.
 Overall, inaccuracies can stem from both evaluators and
evaluees. Therefore, I add that there is a need for
comprehensive, impartial, and scientifically-informed
Evaluator and Evaluee Errors
 Faust et al. separated evaluator and evaluee factors
that could influence a diagnosis.
 Evaluators: Most sources of error are avoidable due to
the underutilization of the available scientific
 i.e. Testing an evaluee excessively – lowering their effort
and motivation to complete the task.
Could lead to a malingering diagnosis
Four Biasing Evaluator Factors
 Faust et al. (2012a) described 4 biasing evaluator
factors in undertaking assessments and arriving at
conclusions, such as:
 1) Confirmatory bias
Tendency for an evaluator to maintain a belief despite
“convincing” counter-evidence.
Four Biasing Evaluator Factors
 2) Premature closure
 Arriving at first conclusions too rapidly and evaluator selffulfilling prophecies.
 There may be a biased selection of instruments leading to
greater/lesser false-positive or false-negative errors.
 Evaluator behaviour may influence how the evaluee behaves
in order to confirm their hypothesis.
 There needs to be systematic use of the available resources
and procedures in order to decrease potential evaluator bias.
Four Biasing Evaluator Factors
 3) Illusory correlation
For example, an evaluator might believe that an evaluee is
nervous because he or she is malingering, even though it may
be an ordinary reaction to an assessment that holds a lot at
Evaluators may apply potential malingering indicators prior to
using well-validated indicators of methods in detection of
Four Biasing Evaluator Factors
 4) Overconfidence
 Potential for dangerous “pernicious” assessments.
 Some evaluators may use “an arbitrary or inconsistent”
procedure for malingering determination:
 The more malingering measures used, the greater the
possibility that errors across the tests will be
Potential Assessment Errors
 Another assessment error in this regard is to give more
tests than is appropriate and then counterbalance that
decision by setting “high cut-offs” for each of the tests.
 The tests have not been studied in combination, so by
combining tests in an assessment, the evaluator is not
working with a “known accuracy rate.”
Integrating All Data
 Some evaluators may believe that the best way to assess
clients would be to integrate all available data in order to
arrive at their conclusion.
 Faust et al. (2012a) argued that this method is inefficient
and may lead to include weak predictors, consider validity
as cumulative, and not consider validity as incremental.
 This may in fact cause more harm than good.
 Thus, they suggest that evaluators only include the
available information that increases accuracy and should
exclude any information that does not.
The Work of Richard Rogers
Rarity of Malingering
 Rogers dealt with misconceptions and fallacies in the
field, such as:
 Malingering is rare  false
Tables 11.8 states it is rare and 11.7 states it is very rare
 He stated that “possible malingering” could be over
 The base rate could however be as low as 10%
Fallacies and Misconceptions
 Rogers and Bender (2012) identified other new
 Malingering is common
 Exact diagnostic capacity of cut scores
 Exact diagnostic capacity of the DSM-IV-TR
Assessing Psychological Injuries and Malingering:
Disability and Report Writing
 Impairment:
 Important deviation, loss, or loss of use of a psychological/
psychiatric function.
 Disability:
 The functional consequences of the impairment.
For example, in term of activity limitations, participation
restrictions, or both.
 Handicap:
 More of a social rather than legal term.
For example, how one perceives oneself or how society perceives the
person with impairment/disability.
Impairment and Disability
 Peterson and Paul (2009) said that in order to
understand impairment and disability, one has to
consider the interaction of the relevant symptoms and
functional effects in terms of the context and
 They must be compared to generally accepted norms
and the general population.
Figure 14.4 The Six-Step Process of
a Disability Evaluation
Figure 14.4 Caption
A model of the steps in disability evaluation that
includes standards, functions, causes and impairments.
Adapted from Piechowski (2011)
Figure 14.4 Explanation
 Piechowski offered an approach to evaluate disability
 She said the evaluator must consider:
 Occupational standard involved,
 The components of the relevant job duty,
 The relationship of the residual functional abilities with the
work demands, etc.
 In the workers compensation context, relating functional
impairments to work demands is critical in disability
Table 14.7 Factors Contributing to
Difficulties in Evaluations: Assessment
Not get all data needed
Ignore certain data
Mental status
Work/ school/ role
Social/ family
Other (i.e., finances)
Records/ collaterals
All requested records/ documents
All collaterals (personal, work, role, professional) consulted
Only reliable ones used
Tests/ measures
Chosen to fit question at hand
Multitrait, multimethod
Psychometrically sound
With appropriate norms, cut-offs, etc., for question
Symptom Validity
Tests (SVTs)
Two-alternative forced choice
Embedded in personality inventories
Embedded in neurocognitive batteries
Structured interview ones
(within, across)
each of the
Reliable records/ documents
Reliable collateral information
Known effects/ expected symptoms
Event at claim
Fact vs. perception
Dose-response relationship, absence
Biases at play?
Adversarial divide at play?
Blaming victim/ extreme entitlement in victim?
Magnify/ minimize pre-event status
Magnify/ minimize event and immediate reaction
Magnify/ minimize post-event symptoms and functions
Evaluee (verbal)
Reliable historian/ respondent?
Inconsistent/ discrepant/ vague?
Evasive/ uncooperative/ resisting/ refusing?
Past treatments
Therapies followed?
Medications taken?
Evaluee (other)
Crying for help?
Response bias
Feigning, fabrication
Gross exaggeration
Ambiguous/ gray zone
Intent in deception Deliberate/ conscious, unconscious?
For secondary gain?
Consider evaluee as individual and according to normative research
Individual differences
Cultural, minority differences
Sex differences
Absence of relevant research?
Relevant research analyzed?
Scientific process
Methods scientifically informed?
Scientific reasoning in conclusions?
Consider all symptoms/ functions/ roles in arriving at disorders/ diagnoses/ disabilities/
dysfunctions/ impairments
If there are any implicated roles/ disorders/ diagnoses/ disabilities/ dysfunctions/ impairments,
do all data support them?
Present all evidence for the favored conclusion, for and against, and all the evidence for other
conclusions rejected, for and against
Genuine conditions
Related conditions to malingering (i.e., factitious disorder)
DSM difficulties
Polytrama/ comorbidities
Subsyndromal/ partial/ features
In remission
Job/ role duties
Residual abilities, impairments
Transferable skills, retrainable?
Probable course?
Causal factors
Pre-event related
Post-event related
Extraneous/ unrelated/ auxiliary
Blaming event at issue for everything
Whitewashing past problems
Insurance process
Event at claim material contributor?
Thin/ crumbing skill considered? (i.e., pre-existing responsible in full, in part)
Abbreviation. DSM = Diagnostic and Statistical Manual
of Mental Disorders (American Psychiatric Association,
Beginning an Assessment
 Psychological assessments must be:
 impartial,
 comprehensive, and
 scientifically-informed
 Evaluator must aim to get as much data as possible and not
ignore any.
 It is important to start by interviewing the evaluee to see if
they have a good mental status for psychological testing.
 The interview should take a holistic approach and
examine the whole person in their context.
 Not only should classic tests like the MMPI-2 be used,
but symptom validity tests (SVTs) as well.
 Clinicians must not only look for malingering
characteristics throughout the interviews and tests,
but must also search for inconsistencies in all sources
and documents available.
Assessment (Contd.)
 Clinicians must verify whether there is a dose-response
relationship between the injuries sustained and the
psychological effects reported.
 Clinicians must also verify the reliability of past
treatment providers and verify their own biases.
 They must critically analyze evaluee responses and
compare them to the factual data and information.
 It is possible that poor test performance and effort can
be due to other factors than simply malingering.
 Other factors may be:
 being overwhelmed,
 catastrophizing, and
 crying out for help.
 Many evaluees will present and perform their
symptomology in ambiguous, mixed and uncertain
ways  grey-zone
 An evaluee that presents symptoms in the “grey-zone” is
 How to minimize the uncertainty?
 know the scientific literature well,
 use scientifically-informed methods and procedures,
 use scientific reasoning,
 respect the individual differences
 evaluator needs to consider distinctions between symptoms
versus impairments, and
 Consider distinctions between disorders/ diagnoses versus
disabilities/ dysfunctions
Grey-zone (Contd.)
 Finally, has all the reliable evidence been considered?
 Has the event at claim has been a material contributor
to the psychological condition presented?
 Are pre-existing factors responsible in full or in part?
 Table 14.8 examines the range of pre-existing factors
that might influence disability determinations in the
forensic context.
 Table 14.9 emphasizes that events at claim might lead
to physical injuries, psychological injuries, or both,
and the injuries might be at either minor or major
 Table 14.10 points that the evaluee might also be
unduly influenced by the litigation process and
iatrogenic factors.
Legal Aspects and Testimony
 After the referral, the evaluator must assess not only
the details of the event at claim but also its credibility.
 After the completion of the evaluation and the written
report, in court judges may determine the testimony/
report’s admissibility.
 Legal decisions will hinge on issues of functional
impairment, disability, permanence of the damages.
 Table 14.12 further specifies that the functional
outcome in disability cases could examine:
 quality of life,
 pain and suffering, and
 the catastrophic nature of the injury involved
 The causality analysis might lead to conclusions that
malingering, more than anything else, had been
Effective Forensic Writing
 DeMier (2013) noted 3 central points:
 (a) First, essential points need to be included, such as
use of third-party information and description of
functional abilities.
 (b) Second, the report should show clearly how clinically
findings relate to the legal question at hand, so that
psycholegal opinions are clearly justified.
 (c) Third, the issue of whether forensic psychologists
should address ultimate issues is actually secondary to
the quality of the data gathered and justification of the
interpretations and conclusions presented.
Most Recent Journal Article Review
Bigler & Larrabee (2012a,b)
 Engaged in dialogue about symptom validity testing in
neuropsychological assessment.
 Bigler:
 SVT testing can help infer symptom and performance
 Below-chance SVT scores are clear and indisputable
indices of invalid test performance.
Bigler & Larrabee (2012a,b)
 He queried the meaning and interpretation of SVTs.
 Just fail SVTs, also known as near-pass SVT
performance, might do so for valid reasons
 i.e. valid underlying neuropathology
 He noted that there is no systematic research of the
effects of lesions on SVT performance.
SVT Selection and Use
 There are no clean guidelines that evaluators use to
select SVTs
 There is no known number to use, order, context or what
do with passes and failures on some tests.
 Most evaluators will end up using their subjective
personal judgement.
 Cut-score selection should be wary of one-size-fits-all
Bigler & Larrabee (2012a,b)
 For Larrabee (2012a, b), the science and research behind
the approach to use dichotomous pass-fail cut scores and
also algorithms to combine tests are valid and avoid
inappropriate attributions.
 So, factors such as expectation and stress alleged to explain
poor performance on the tests have little weight.
Bigler & Larrabee (2012a,b)
 Both conduct research in the area of malingering
detection, but have contrasting opinions.
 Either more scientific refinement is needed to reduce
such contrasting opinions.
 OR, both parties’ arguments require careful scrutiny for
errors of omission and commission.
 This book offers many guidelines to help resolve these
Bigler (2012a,b)
 He refers to litigation science to help improve SVT
 Although I agree that this type of research could present
biases, the same applies to any science, even if non-litigation.
 As long as litigation science is conducted transparently, it
should not be labeled beforehand as invalid
 Bigler’s most important contribution with respect to SVTs
in neuropsychological assessment relates to the lack of
guidelines for their proper use in practice.
 **These cautions are similar to the ones I have raised in the
present book.
Larrabee (2012a,b)
 Larrabee (2012a, b) added:
 Certain types of case control research designs meet the
highest quality of standards in research.
 High degree of replicability of the results in the research on
symptom validity.
 Effect sizes in this type of research are quite large -- for
example, for the RDS (Reliable Digit Span), the MMPI-2’s
FBS, and the DMT.
 He cited research showing illness behavior and diagnosis
threat do not appear to affect performance on SVTs.
Hall and Hall (2012)/ Silver (2012)
 Hall and Hall (2012) called for attribution of
compensation neurosis when it seems warranted in
assessments, a construct which is conceptually related
to malingering on the continuum of possible response
biases (see Figure 17.1).
 In contrast, Silver (2012) believed it’s too difficult to
attribute malingering, since so many factors can
contribute to poor effort on SVTs.
Never the Twain Shall Meet
 Compensation neurosis concerns symptom
exaggeration related to not only the prospect of
secondary gain but also to internal motivations (e.g.,
stress from the case, or from treatment issues, and its
effects on somatization and aspects of personality,
such as dependence).
 Compensation neurosis is different from malingering
as it also takes internal motivations into account on
top of external motivation, unlike malingering.
Never the Twain Shall Meet
 Compensation neurosis does not refer to symptom
absence, there are physical symptoms involved, but
the causes for the symptoms do not involve real
injuries related to the event at hand
 they reflect psychosomatic processes at work.
 Individuals might be prone to react to events at claim
this way.
 The stress of the case includes conscious and
unconscious pressures not to improve.
 The legal and disability arena is iatrogenic.
Hall and Hall (2012)
 I note: Not only can the iatrogenic effect be due to
conscious or unconscious motivation for financial
compensation, but also from:
 Insurance pressures, IEs and, Unjust denials of claims
 An even-handed approach to the question would
acknowledge the presence of stress for the evaluee from all
corners of the system.
 It would be difficult to diagnose compensation neurosis
due to the need to differentiate conscious from
unconscious motivations, and internal from external
incentives, etc.
 The process of symptom hardening is complex and it might
exclude the event at claim as a cause.
Silver (2012)
 Silver (2012) noted that symptom severity is influenced by
multiple non-TBI factors, pre-existing factors, etc. For example:
 Expectations that symptoms reflect TBI
 Stereotypic threat, and
 Ego depletion (which might be a form of stereotypic threat)
 The compensation/ insurance/ litigation process includes an
adversarial component.
 This may increase psychological costs (more anger, wanting
revenge, loss aversion, i.e., generally the reward to loss ratio should
be about 2:1), which can affect symptoms.
 Thus, cheating a “little” might be normal in these circumstances,
as well. “A lot” of cheating is not the norm.
Silver (2012) Contd.
 Suboptimal effort or symptom magnification is evident in
neuropsychological assessment.
 They may occur for many reasons other than conscious effort
and malingering.
 Stress of the compensation/ insurance/ litigation process
might lead evaluees to try too hard rather than less hard.
 They would possibly use a thinking process that is slower,
deliberate, and conscious on tests of effort, which normally
should elicit thinking that is fast, non-effortful, and
 Their altered cognitive style might give a false impression
of malingering.
Young (2014)
 Silver (2012) has not considered certain factors in his arguments
rendering the attribution of malingering to be very difficult:
Insurance process might be stressful or effortful not only because of
trying harder, but because of efforts to falsely present or produce
There is no empirical evidence to support the statement that only a
“little cheating” can be expected in forensic disability and related
contexts, in this case for assessing MTBI.
Throughout the present book, I have argued that better surveys on this matter
need to be conducted.
To conclude, there are alternate interpretations of poor effort
unrelated to negative response bias, when evaluees perform poorly on
Book Conclusions
Disability Evaluations: Psychologist
 Piechowski (2012) noted that disability evaluations differ
from evaluations conducted by treatment providers
especially in the emphasis on functional capacity
evaluation compared to diagnosis.
 She stated that disability is defined functionally, as an
inability to undertake behaviors of a specified task or role
in context.
 As for causality, the assessor must show that the disability
is causally related to the condition of the patient.
 She added that secondary factors such as “financial
problems, personal lifestyle choice, legal issues, and family
demands” might affect work functioning.
Table 34.1 Topics for the Interview
Social history
Childhood, family children, etc.
Educational history
Academic and behavioral performance
Occupational history
Satisfaction and dissatisfaction with work, etc.
Legal history
Involvement with the criminal justice system, etc.
Medical history
Current or past health problems, etc.
Mental health history
psychotropic medications, etc.
Substance abuse history Use of alcohol, illegal drugs, abuse of prescription medications,
Job duties
Detailed description of duties, working conditions, schedule,
and pace of work done just prior to the onset of the claimed
Current daily activities
How the claimant currently spends the day
Disability onset
Detailed description of the onset of the difficulties
Functional impairments Detailed description of how functioning has been affected
Disability Evaluations: Psychologist
 Samuel and Mittenberg (2005) found that estimates of
the base rate for malingering in disability claimants
varied between 7.5 and 33%.
 Also, Sumanti, Boone, Savodnik, and Gorsuch (2006)
investigated “non-credible” symptoms in workers
claiming “stress”, they found that 9 to 29% of the
workers endorsed non-credible psychiatric symptoms,
along with 8 to 15% for non-credible cognitive
Malingering and SVTs
 Failing to meet the threshold on these tests does not
automatically imply that malingering has taken place
(Lilienfeld et al., 2013).
 Furthermore, malingering is not a dichotomous
concept (present, absent), rather it is dimensional.
 Malingering testing yields only moderate correlations,
at best, and several separate factors, depending on the
 Psychopathological and cultural influences have not
been sufficiently investigated, among others.
Malingering and SVTs
 Lilienfeld et al. (2013) concluded that such tests
“surely” assess variance related not only to response
sets but also to “genuine psychopathology.”
 Thus, the “precise meaning” of scores obtained on
many SVTs need “clarification.”
 In addition, the manner in which they can be
combined has not been conclusively established.
 In fact, any new information that they may provide
might “worsen” clinical judgment and prediction.
 e.g., if the information is of nonexistent or negligible
 If SVTs are going to demonstrate their clinical utility, the
“V” portion (or validity portion) of their intent must be
better demonstrated.
 I would add this refers to:
 (a) their capacity to differentiate in research “known”
malingerers from “genuine” responders,
 (b) the research base on their clinical utility in applied
practice (do they add “incremental validity” in malingering
attribution), and
 (c) their ability to meet the challenge posed by McGrath,
Mitchell, Kim, and Hough (2010) that they have yet to
demonstrate sufficient “convergent” validity (but see the
response by Rohling, Larrabee, Greiffenstein, Ben-Porath,
Lees-Haley, Green, & Greve, 2011).
Malingering and SVTs
 Overall, the Lilenfeld et al. (2013) article cautions the way in how
SVTs are used and interpreted for court purposes.
 Perhaps it is wise to conclude a book on possible
malingering by the evaluee with a note of caution on
possible bias in the evaluator.
 Kassin, Dror, and Kukucka (2013) referred to a forensic
confirmation bias, Murrie, Boccaccini, Guarnera, and
Rufino (2013) to an allegiance effect, and Stanovich, West,
and Toplak (2013) to a myside bias.
 These studies were not related to the forensic civil
disability, and related context, but their concerns resonate
for this area of practice.
Malingering and SVTs
In essence, evaluators, evaluees, and
third party stakeholders form an
integrated system in which science
must be the best source of evidence for
court to dispel bias from any side of
the process in court.
Who We Are
 Association for Scientific Advancement in Psychological
Injury and Law: A Society (
 Psychological Injury and Law (PIL): Our Journal
 For mental health professionals and legal professionals
working together
Thank You
Gerald Young, Ph.D., C. Psych.
Editor-in-Chief, Psychological Injury and Law
President of ASAPIL Association
Email: [email protected]
Phone: 416-247-1625 clinical office/ 416-726-2709 cell
Please contact me if you want more slides.

similar documents