How Valid Are Medical Records and Patient Questionnaires for Physician Profiling and Health Services Research? A Comparison with Direct Observation of Patient Visits

Kurt C. Stange, Stephen J. Zyzanski, Tracy Fedirko Smith, Robert Kelly, Doreen M. Langa, Susan A. Flocke, Carlos R Jaen

Research output: Contribution to journalArticle

297 Citations (Scopus)

Abstract

OBJECTIVES. This study was designed to determine the optimal nonobservational method of measuring the delivery of outpatient medical services. METHODS. As part of a multimethod study of the content of primary care practice, research nurses directly observed consecutive patient visits to 138 practicing family physicians. Data on services delivered were collected using a direct observation checklist, medical record review, and patient exit questionnaires. For each medical service, the sensitivity, specificity, and Kappa statistic were calculated for medical record review and patient exit questionnaires compared with direct observation. Interrater reliability among eight research nurses was calculated using the Kappa statistic for a separate sample of videotaped visits and medical records. RESULTS. Visits by 4,454 patients were observed. Exit questionnaires were returned by 74% of patients. Research nurse interrater reliabilities were generally high. The specificity of both the medical record and the patient exit questionnaire was high for most services. The sensitivity of the medical record was low for measuring health habit counseling and moderate for physical examination, laboratory testing, and immunization. The patient exit questionnaire showed moderate to high sensitivity for health habit counseling and immunization and variable sensitivity for physical examination and laboratory services. CONCLUSIONS. The validity of the medical record and patient questionnaire for measuring delivery of different health services varied with the service. This report can be used to choose the optimal nonobservational method of measuring the delivery of specific ambulatory medical services for research and physician profiling and to interpret existing health services research studies using these common measures.

Original languageEnglish (US)
Pages (from-to)851-867
Number of pages17
JournalMedical Care
Volume36
Issue number6
StatePublished - Jun 1998
Externally publishedYes

Fingerprint

Health Services Research
Medical Records
health service
physician
Observation
Physicians
questionnaire
nurse
medical services
habits
counseling
Nurses
statistics
Physical Examination
Habits
medical service
Counseling
Immunization
family physician
examination

Keywords

  • Health services research
  • Medical record review
  • Physician profiling
  • Primary care
  • Research methodology
  • Survey research

ASJC Scopus subject areas

  • Public Health, Environmental and Occupational Health
  • Nursing(all)
  • Health(social science)
  • Health Professions(all)

Cite this

How Valid Are Medical Records and Patient Questionnaires for Physician Profiling and Health Services Research? A Comparison with Direct Observation of Patient Visits. / Stange, Kurt C.; Zyzanski, Stephen J.; Smith, Tracy Fedirko; Kelly, Robert; Langa, Doreen M.; Flocke, Susan A.; Jaen, Carlos R.

In: Medical Care, Vol. 36, No. 6, 06.1998, p. 851-867.

Research output: Contribution to journalArticle

Stange, Kurt C. ; Zyzanski, Stephen J. ; Smith, Tracy Fedirko ; Kelly, Robert ; Langa, Doreen M. ; Flocke, Susan A. ; Jaen, Carlos R. / How Valid Are Medical Records and Patient Questionnaires for Physician Profiling and Health Services Research? A Comparison with Direct Observation of Patient Visits. In: Medical Care. 1998 ; Vol. 36, No. 6. pp. 851-867.
@article{354b161233804b66809d403323051de8,
title = "How Valid Are Medical Records and Patient Questionnaires for Physician Profiling and Health Services Research? A Comparison with Direct Observation of Patient Visits",
abstract = "OBJECTIVES. This study was designed to determine the optimal nonobservational method of measuring the delivery of outpatient medical services. METHODS. As part of a multimethod study of the content of primary care practice, research nurses directly observed consecutive patient visits to 138 practicing family physicians. Data on services delivered were collected using a direct observation checklist, medical record review, and patient exit questionnaires. For each medical service, the sensitivity, specificity, and Kappa statistic were calculated for medical record review and patient exit questionnaires compared with direct observation. Interrater reliability among eight research nurses was calculated using the Kappa statistic for a separate sample of videotaped visits and medical records. RESULTS. Visits by 4,454 patients were observed. Exit questionnaires were returned by 74{\%} of patients. Research nurse interrater reliabilities were generally high. The specificity of both the medical record and the patient exit questionnaire was high for most services. The sensitivity of the medical record was low for measuring health habit counseling and moderate for physical examination, laboratory testing, and immunization. The patient exit questionnaire showed moderate to high sensitivity for health habit counseling and immunization and variable sensitivity for physical examination and laboratory services. CONCLUSIONS. The validity of the medical record and patient questionnaire for measuring delivery of different health services varied with the service. This report can be used to choose the optimal nonobservational method of measuring the delivery of specific ambulatory medical services for research and physician profiling and to interpret existing health services research studies using these common measures.",
keywords = "Health services research, Medical record review, Physician profiling, Primary care, Research methodology, Survey research",
author = "Stange, {Kurt C.} and Zyzanski, {Stephen J.} and Smith, {Tracy Fedirko} and Robert Kelly and Langa, {Doreen M.} and Flocke, {Susan A.} and Jaen, {Carlos R}",
year = "1998",
month = "6",
language = "English (US)",
volume = "36",
pages = "851--867",
journal = "Medical Care",
issn = "0025-7079",
publisher = "Lippincott Williams and Wilkins",
number = "6",

}

TY - JOUR

T1 - How Valid Are Medical Records and Patient Questionnaires for Physician Profiling and Health Services Research? A Comparison with Direct Observation of Patient Visits

AU - Stange, Kurt C.

AU - Zyzanski, Stephen J.

AU - Smith, Tracy Fedirko

AU - Kelly, Robert

AU - Langa, Doreen M.

AU - Flocke, Susan A.

AU - Jaen, Carlos R

PY - 1998/6

Y1 - 1998/6

N2 - OBJECTIVES. This study was designed to determine the optimal nonobservational method of measuring the delivery of outpatient medical services. METHODS. As part of a multimethod study of the content of primary care practice, research nurses directly observed consecutive patient visits to 138 practicing family physicians. Data on services delivered were collected using a direct observation checklist, medical record review, and patient exit questionnaires. For each medical service, the sensitivity, specificity, and Kappa statistic were calculated for medical record review and patient exit questionnaires compared with direct observation. Interrater reliability among eight research nurses was calculated using the Kappa statistic for a separate sample of videotaped visits and medical records. RESULTS. Visits by 4,454 patients were observed. Exit questionnaires were returned by 74% of patients. Research nurse interrater reliabilities were generally high. The specificity of both the medical record and the patient exit questionnaire was high for most services. The sensitivity of the medical record was low for measuring health habit counseling and moderate for physical examination, laboratory testing, and immunization. The patient exit questionnaire showed moderate to high sensitivity for health habit counseling and immunization and variable sensitivity for physical examination and laboratory services. CONCLUSIONS. The validity of the medical record and patient questionnaire for measuring delivery of different health services varied with the service. This report can be used to choose the optimal nonobservational method of measuring the delivery of specific ambulatory medical services for research and physician profiling and to interpret existing health services research studies using these common measures.

AB - OBJECTIVES. This study was designed to determine the optimal nonobservational method of measuring the delivery of outpatient medical services. METHODS. As part of a multimethod study of the content of primary care practice, research nurses directly observed consecutive patient visits to 138 practicing family physicians. Data on services delivered were collected using a direct observation checklist, medical record review, and patient exit questionnaires. For each medical service, the sensitivity, specificity, and Kappa statistic were calculated for medical record review and patient exit questionnaires compared with direct observation. Interrater reliability among eight research nurses was calculated using the Kappa statistic for a separate sample of videotaped visits and medical records. RESULTS. Visits by 4,454 patients were observed. Exit questionnaires were returned by 74% of patients. Research nurse interrater reliabilities were generally high. The specificity of both the medical record and the patient exit questionnaire was high for most services. The sensitivity of the medical record was low for measuring health habit counseling and moderate for physical examination, laboratory testing, and immunization. The patient exit questionnaire showed moderate to high sensitivity for health habit counseling and immunization and variable sensitivity for physical examination and laboratory services. CONCLUSIONS. The validity of the medical record and patient questionnaire for measuring delivery of different health services varied with the service. This report can be used to choose the optimal nonobservational method of measuring the delivery of specific ambulatory medical services for research and physician profiling and to interpret existing health services research studies using these common measures.

KW - Health services research

KW - Medical record review

KW - Physician profiling

KW - Primary care

KW - Research methodology

KW - Survey research

UR - http://www.scopus.com/inward/record.url?scp=0032089115&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0032089115&partnerID=8YFLogxK

M3 - Article

C2 - 9630127

AN - SCOPUS:0032089115

VL - 36

SP - 851

EP - 867

JO - Medical Care

JF - Medical Care

SN - 0025-7079

IS - 6

ER -