Understanding Sources of Differences from Interagency Research

Understanding Sources of Differences from
Interagency Research:
ACS-GQ and BJS Correctional Population Estimates
FCSM Statistical Policy Seminar
December 4, 2012
William J. Sabol
Bureau of Justice Statistics
Motivation: Census report from 2006 ACS on
correctional populations
• September 2007: Census report from the 2006 ACS
• Census’ “first in depth look at people in prisons [sic]
since the 1980 Census.” (Washington Post)
• Census reported the 2006 correctional population at
2.1 million and reported a 4% change in the corr.
pop. since 2000.
• Census figures differed from BJS, which reported
more than 2.3 million in prison or jail.
Motivation: Responses to Census report
• Pew Foundation: Letter to editor of WP; claimed the
Census report gave a false impression about the
growth of correctional populations, which they had
forecast to increase more rapidly than implied by the
Census numbers; pointed out that Census’ counts were
lower than BJS counts.
• BJS: I wondered why the ACS had not coordinated its
release with BJS or perhaps why it had not attempted
to reconcile its statistics with BJS (then) twice-yearly
releases, especially since the Governments Division
collected the prison & jail count data for BJS.
Nature of discrepancies
• Estimates of inmates in correctional facilities
– ACS: 2.1 million inmates in correctional facilities in 2006
– BJS: 2.3 million
• Change estimates
– ACS: 4% increase from 2000-2006
– BJS: 15% increase from 2000-2006
• Characteristics of inmates
– Age, race
– Education, veterans’ status, disability
• Estimates tied to key policy issues in sentencing and
Differences between BJS and ACS-GQ correctional
population estimates have diminished over time…
Decrease in differences arises from different trends
BJS methods
• Establishment surveys of all 51 departments of
• Establishment survey of sample of all local jails
• Reference date is yearend (ACS is continuous
Estimates inform to key sentencing & corrections policy
• Size and cost of corrections—efforts to reduce the size
of the prison population while not harming public
safety and to repurpose funds for other uses;
• Fairness and equity of sentencing—racial composition
and disparities in punishment; long-standing issues in
sentencing & corrections
• Re-integration (re-entry) of former inmates—specific
needs (e.g., education & employment), specific groups
(e.g., veterans and persons with disabilities)
• Utility of ACS-GQ—potential source for ongoing,
routine estimates of key indicators of composition of
national and state-level correction population.
Comparable age distributions in 2006
Comparable age distributions in 2009; BJS age data
from administrative records and self-report survey
BJS and ACS-GQ differences on race revolve around
percent black and persons of 2 or more races
BJS estimates for non-Hispanic blacks may differ from ACS-GQ due to the
number of persons of two or more races in its surveys. BJS adjusts
establishment survey data on race by self-report inmate survey estimates.
Differences in racial composition has implications for
measures of racial disparity
Black-to-white racial disparity ratio (ratio of incarceration rates) is
lower with ACS-GQ estimates; no major differences for Hispanics.
ACS-GQ and BJS: Educational attainment questions
• What is the highest degree
or level of school you have
completed? [Mark one box.
If currently enrolled, mark
the previous grade or
highest degree received.]
BJS inmate survey
• Before your admission, what
was the highest grade of
school that you ever
• Did you complete that year?
• Do you have a GED, that is, a
high school equivalency
• Did you earn your GED while
serving time in a prison, jail, or
other correctional facility?
Census staff (Ewert & Widlhagen, 2011) have compared ACS and BJS inmate surveys
but limited comparison of differences primarily to differences in question wording
rather than reconciling differences.
ACS-GQ and BJS inmate surveys: Prisoners have high
rates of HS non-completion
Source: U.S. Census Bureau, American Community Survey, 2009; Bureau of Justice Statistics,
Survey of Inmates in State Correctional Facilities, 2004
ACS-GQ and BJS: School enrollment questions
• At any time IN THE LAST 3
MONTHS, have you
attended school or college?
[Include… [school] which
leads to a high school
diploma or a college degree.
• What grade or level were
you attending?
BJS inmate survey
• Since your admission, have
you ever been in any other
education program? Exclude
vocational training.
• What kind of program was
BJS inmate survey shows higher rates of enrollment
Source: U.S. Census Bureau, American Community Survey, 2009; Bureau of Justice Statistics,
Survey of Inmates in State Correctional Facilities, 2004
ACS-GQ and BJS: Veterans
BJS inmate surveys:
• Have you ever served on active
duty in the U.S. Armed Forces,
military reserves or National
• When did you serve on active
duty in the U.S. Armed Forces?
[Mark (x) a box for each period
• Do you have a VA serviceconnected disability rating?
• What is your service-connected
disability rating?
• Did you ever serve in the U.S.
Armed Forces?
• In what branch(es) of the Armed
Forces did you serve?
• When did you first enter the
Armed Forces?
• During this time did you see
combat in a combat or line unit?
• When were you last discharged?
• Altogether, how much time did
you serve in the Armed Forces?
• What type of discharge did you
ACS’ VA service-connected disability questions;
potentially high-value for VA medical services planning
and programs for criminal justice involved veterans.
NSDUH: Veterans questions
BJS: Percent of state/federal prisoners reporting prior
military service
Veteran status of criminal justice populations, various
surveys, 2007
Criminal justice involvement
NSDUH Probation supervision
Parole or supervised release
Local jail custody
State prison custody
Federal prison custody
Adult correctional population
Adult correctional population
BJS estimates from inmates surveys (prisons & jails). Item allocation (veteran status) on
the order of 10% in ACS; increasing over time; BJS missing on veteran status on the
order 2-3%.
Trends in ACS-GQ estimates of veterans in adult
correctional facilities diverge from NSDUH estimates
VA medical services: Planning for criminal justice system involved veterans;
requesting from BJS annual estimates of veterans under correctional
supervision; BJS & VA initiating efforts to determine feasibility of matching
BJS administrative records with VA data to identify veterans incarcerated.
ACS-GQ and BJS: Hearing and vision disability
• Are you deaf or do you have
a serious hearing difficulty?
• Are you blind or do you
have serious difficulty
seeing even when wearing
BJS inmate surveys:
• Do you have difficulty
seeing ordinary newsprint,
even when wearing glasses?
• Do you have difficulty
hearing a normal
conversation, even when
wearing a hearing aid?
ACS-GQ and BJS: Cognitive
• Because of a physical,
mental, or emotional
condition, do you have
serious difficulty
remembering, or making
BJS inmate survey:
• Does a mental health
condition keep you from
participating fully in work,
school, or other activities?
• Do you have a learning
disability, such as dyslexia
or attention deficit
ACS-GQ and BJS: Ambulatory, self-care, independent
• Do you have serious
difficulty walking or
climbing stairs?
• Do you have difficulty
dressing or bathing?
• Because of a physical,
mental, or emotional
condition, do you have
difficulty doing errands
alone such as visiting a
doctor’s office or shopping?
BJS inmate survey:
• Do you use a cane,
wheelchair, walker, hearing
aid, or other aids to help
you with your daily
BJS and ACS-GQ differ on most measures of disability
ACS-GQ, 2008-2010 (3-yr data)
BJS inmate survey
With a disability
Differences in items; BJS mobility questions differ from ACS mobility
BJS-Census project to identify and measure sources of
Definitional differences
Frame & sampling
Weighting & estimation
Self-report surveys, CAPI vs. ACASI
Establishment surveys
Administrative records
Variation in common questions
BJS’ surveys cover domains not in ACS (mental health, medical problems substance abuse, offenses &
criminal history, criminal justice status, etc.)
Mode of collection
Impact of Census Population Estimate Program control totals
Match BJS prison & jail frames to ACS frame, 2010 decennial
Large GQ populations and sampling (improvement over time in expected populations in large GQs)
Prisons vs. jails vs. total correctional population
Classification differences
Report on what is learned and what can be done to address discrepancies
BJS-Census effort to assess sources of differences
between ACS-GQ and BJS estimates
Effort related to issues arising in 2007 NAS report on the ACS
Recommendations from the 2007 NAS report on the
Comments from the 2007 NAS report :
• Census Bureau should give serious consideration to whether
the long-form-sample type data from the continuous ACS for
the institutional population…is needed to an extent that
justifies the costs.
• Work with user community to identify items to collect every
year and items that are not needed or for which data are not
likely to be of sufficient quality to be useful
– For example, utility of asking prisoners about income; quality of data
on prisoner income [NAS reported that in 2000, 78%...of
prisoners…had all of their income imputed because they did not
answer any of the income questions; by comparison 25% of household
residents had all of their income imputed.
• Additional item of high utility: Date of entry into facility.
Definitional issues: ACS Group Quarters
• Place where people live or stay, in a group living
arrangement, that is owned or managed by an entity
or organization providing housing or services for the
• Include places such as college dormitories &
residence halls, residential treatment centers, skilled
nursing facilities, group homes, military barracks,
correctional facilities, and workers’ dormitories.
Definitions: ACS-GQ correctional facilities for adults
• Correctional residential facilities: Community-based facilities
operated for correctional purposes; include halfway houses,
restitution centers, prerelease, work release, study centers.
• Federal detention centers: Provide short-term confinement for
adults pending adjudication or sentencing
• Federal & state prisons: Places where persons convicted of crimes
serve their sentences, such as prisons, penitentiaries, correctional
• Local jails/municipal confinement facilities: Operated for counties,
cities, and American Indians/Alaska Natives; hold adults pending
Definition: BJS correctional populations
• Prisons: Include prisons, penitentiaries, and correctional
institutions; boot camps, prison farms, reception, diagnostic and
classification centers; release centers, halfway houses; road,
conservation, forestry camps operated for prisoners; vocational
training; prison hospitals, and treatment facilities for prisoners.
[Some overlap with ACS-GQ residential correctional facilities.]
• Local jail facilities: Confinement facilities, which are usually
administered by a local (county/city) law enforcement agency that
are primarily intended for adults but sometimes hold juveniles
before or after adjudication. Primarily receive and hold individuals
pending arraignment, awaiting trial, conviction, or sentencing; may
hold inmates sentenced to short terms or in some cases to longer
terms to alleviate crowding in state prisons. Some jail facilities
operate community-based programs as alternatives to
Definitional differences
• BJS and ACS-GQ definitions of adult correctional
facilities generally comport with each other
• Some differences in classification:
– BJS includes some types of ACS-GQ correctional residential
facilities as prison facilities
– ACS-GQ includes federal detention centers; BJS does not
include these in its annual counts of prisoners
• Counting/classification differences would generally
lead to the expectation that BJS counts would be
lower than ACS-GQ estimates (due to BJS exclusion
of some CBCF and federal detention centers).
Frame and sampling
• ACS frames: Current (from 2006) and 2010 decennial; proceeds
from 2010 decennial not fully incorporated into ACS
• ACS improves GQ frame by annually updating the expected
population for GQs, especially large GQs.
– Between 2006-08, mean difference between expected and observed population in
GQs with populations greater than 400 persons declined from a 5.8% (mean
difference) to a 1.7% (mean difference). Mean absolute difference also declined
from 30% to 14% (Source: Beaghen and Stern (2009))
• BJS:
– Current: 2010 prison & jail facility frames
– In progress: 2012 prison facility census
– Planned: 2014 jail census
• Match frames
– Identify common/unique facilities
– Compare populations, especially in large GQs
Frame (as described in the 2007 NAS report):
Note: The Internet searching for new prisons is one particular issue that has given
rise to the current BJS-Census project on the ACS-GQ/BJS estimates.
2010 Decennial Census counts line up with BJS counts
fairly well
• 2010 decennial census counts of the correctional
population generally comport with BJS counts
• Major differences include:
– State prisoners in 2010 decennial census is about 60,000
fewer than BJS’s count (1.31 mn. in BJS surveys; 1.25 in
decennial (4.5%) low)
– Local jail inmates in 2010 decennial is low (682,000 vs.
749,000 in BJS (9% lower than BJS); but one-day counts of
jail inmates fluctuates widely, and April 1st vs. June 30th
counts could reflect normal variation.
• Federal prisoner counts are equivalent.
Sampling and weighting/estimation
• Sample selection; quality of ACS frame; coverage error,
relative to BJS frame
– ACS sampling: State by type of GQ stratification; within state type by
geography; corrections facilities identified by type (e.g., prisons, jails,
etc.); classified into 2 (3) groups (large, small, unknown pop.);
systematic sample of facilities based on groups of 10 persons; most
prisons & jails fall into the “large GQ” category (hold > 15 persons).
• BJS:
– Annual counts: Establishment surveys of all departments of
corrections (censuses of DOCs) and sample survey of jails
– Inmate surveys: Two-stage sampling, (1) facilities proportionate
to size and (2) systematic sample of inmates (approx. 40 per
facility); oversampling women; 4-states self-representative
• Control totals for ACS-GQ: PEP estimates of GQ populations
– Same form as ACS; allows for comparisons to household
– Concerns about relevancy of some items (e.g.,
employment in past week);
• BJS: Instrument completes ACS instrument with
some overlap
– Inmate survey covers 10 domains, most not in ACS
• Mode: ACS is primarily CAPI; BJS mode varies
depending upon survey, including ACASI.
Instrumentation: Policy-relevant topics in ACS not
annually covered by BJS surveys
• Frequently asked questions/topics of BJS:
– Non-U.S. citizens and country of birth/origin
– Welfare benefits
– Insurance coverage, especially in relation to the Affordable
Care Act
– Residence prior to incarceration; where prisoners should
be counted for purposes of the census
• BJS asks where inmates lived prior to admission
• ACS asks where inmates lived a year ago
– 1-year estimates report total correctional population;
public confusion that correctional populations = prisoners.
– PUF shows institutional vs. noninstitutional GQ;
– Census studies: report by type, e.g., prisons
• BJS:
– Reports total correctional population but distinguishes
between types (prisons vs. jails), given different functions
and population composition of these types
– Reports separately on inmates in prisons and jails from
Reporting: Different functions and compositional
differences between prisons and jails
• Prisons: Hold persons sentenced for felonies; mean
time served of about 2 years (across all
commitments), 3 years for those sentenced from the
courts; about 650,000 commitments per year on a
stock of about 1.5 million.
• Jails: Hold unconvicted persons primarily (2/3rds of
the jail population); average stay is about 3 weeks;
half spend two or fewer days in jail; 10-12 million
bookings per year on a stock of about 750,000.
Useful enhancements to both agencies’ efforts
– Use proceeds of the effort to assist in updating ACS-GQ
frame during inter-census periods
– Ask a question about the date of entry into the GQ
• Useful for estimating length of stay and turnover in correctional
• Important for issues related to prisoner reentry
– Employment
– Insurance coverage
– Expectations about receipt of Federal benefits
– Produce estimates adjusted to BJS control totals
Useful enhancements to both agencies’ efforts
– Use decennial census on GQ in lieu of one of its censuses
(saving costs to taxpayers)
– Modify inmate survey instruments to comport with ACS
and other national surveys (enhance comparability)
– Use modified ACS-GQ estimates to provide policy makers
with relevant statistical information on prison & jail inmate
– Use multi-year ACS-GQ for state-level estimates of
characteristics of prison & jail inmates
Importance of addressing discrepancies
• ACS-GQ data a valuable source of data on core
indicators of characteristics of prison & jail inmates
– BJS can use the ACS-GQ data for statistical reporting if BJS
and Census can identify, address and correct survey
– Comparability: BJS’ own surveys of prison & jail inmates
capture data in other domains and key estimates can be
pegged to ACS-GQ
• Large sample sizes and reliable estimates;
comparisons to noninstitutionalized populations
• Enhance utility of ACS-GQ data
Concluding thoughts
• Identifying discrepancies is the easy part; identifying
the phases in a survey’s lifecycle to focus on is more
• Persistence required in identifying the key personnel
to work with; interagency work is hard.
• Try to organize the work around achieving mutuallybeneficial goals for each agency.
• Allow for learning to motivate changes.
• Draw upon received wisdom.

similar documents