Random Error [1] • Non-systematic fluctuations in readings of a measurement device or administration of a measure Limited assessment of psychometric properties in prosthesis/orthosis users [5] • i.e. reliability , validity, responsiveness and sensitivity Floor and Ceiling Effects • The inability of a measure to capture an individuals lowest and highest performance/responses hence changes in condition may be missed. Learning (practice) Effects [1] • Improvement in patient performance of a measure with each attempt. Day to day fluctuation in the variable to be measured [5] What are outcome measures? An outcome measure is a standardized instrument used in clinical and research settings to evaluate change in health status of an individual, group or population that is attributable to an intervention or series of interventions [1]. Why use outcome measures? [1] Outcome measures may benefit clinical practice in a variety of ways: • Documenting patient status • Informing intervention planning • Documenting clinical care • Evaluating the effect of interventions • Auditing the quality of clinical care • Enhancing communication with the patient, clinical team, and third party payers How can outcome measures be classified? [2] Patient-reported measures • Questionnaire or survey of patient’s perception, e.g. satisfaction with service, quality of life, functional status, etc. Performance-based measures • Assessment of a patient’s physical ability to perform tasks • Require more space and equipment than patient-reported measures Condition-specific measures • Validated for a particular population or condition • Tests are less broadly applicable than generic measures Generic measures • Applicable across a variety of populations and conditions • Useful for comparison across populations How are outcome measures developed? [4] • Purpose of the instrument is defined • Test specifications are defined • Items are developed • Scoring criterion is developed • Expert reviews are conducted • Pilot studies are conducted • Psychometric properties are evaluated Outcome Measure Description Outcome Measures in Clinical Practice [1] Robinson, C. and Fatone, S. (2012). You've heard about outcome measures, so how do you use them? Integrating clinically relevant outcome measures in orthotic management of stroke. Prosthetics and Orthotics International, 37(1), 30-42. [2] Condie, E., Scott, H. and Treweek, S. (2006). Lower limb prosthetic outcome measures: A review of the literature 1995 to 2005. Journal of Prosthetics and Orthotics, 18(1S), 13-45. [3] Kahle, J.T. and Highsmith, M.J. (2009). Evidence-based practice for the individual with amputation. inMotion, 19(5), 26-28. [4] Heinemann, A.W. (2013). Survey instrument development. Research I. Lecture conducted at Northwestern University, Chicago, IL. [5] Miller, L.A. and McCay, J.A. (2006). Summary and conclusions from The Academy’s sixth state-of-the-science conference on lower limb prosthetic outcome measures. Journal of Prosthetics and Orthotics, 18(1S), 2-7. [6] Baer, H.R. and Wolf, S.L. (2001). Modified Emory Functional Ambulation Profile: An outcome measure for the rehabilitation of post-stroke gait dysfunction. Stroke, 32, 973-979. [7] Fatone, S. (2013). Simple Outcome Measures for Use in O&P. Assessment. Lecture conducted at Northwestern University, Chicago, IL. [8] Heinemann, A.W., Connelly, L., Ehrlich-Jones, L. and Fatone, S. (2014). Outcome instruments for prosthetics: Clinical applications. Physical Medicine and Rehabilitation Clinics of North America, 25, 179-198. [9] Heinemann, A.W., Bode, R.K. and O'Reilly, C. (2003). Development and measurement properties of the Orthotics and Prosthetics Users' Survey (OPUS): A comprehensive set of clinical outcome instruments. Prosthetics and Orthotics International, 27, 191-206. [10] “Functional outcome assessment instruments”. (n.d.). Retrieved on Feb 8, 2014 from http://www.oandp.org/olc/lessons/html/SSC_06/section_05.asp?frm [11] Hanspal, R.S., Fisher, K. and Nieveen, R. (2003). Prosthetic socket fit comfort score. Disability and Rehabilitation, 25 (22), 1278–1280. [12] ”Prosthetics”. (2013). Retrieved on Feb 8, 2014 from http://www.silipos.com/products/Prosthetics Sources of Information Acknowledgements: Poster development supported by the American Academy of Orthotists and Prosthetists Gait Society. Clinical Outcome Measures in Orthotics & Prosthetics Stephanie Fritschen, Brittany Mott, Stefania Fatone, PhD, BPO(Hons) (Faculty Mentor) Northwestern University Prosthetics-Orthotics Center (NUPOC) Suggested process for implementing outcome measures in clinical practice [1]: • Determine why an outcome measure(s) is needed, what is being evaluated, and what decisions need to be made to treat the patient. • Match treatment goals to an appropriate measure. • Determine when the measure needs to be administered (e.g. initial visit, delivery, follow-up). • Search for and assess measures based on their reliability, validity, and responsiveness for your patient population. • Online search engines (e.g. RehabilitationMeasures.org) may be used to find instruments related to a specific diagnosis and area of assessment. • Develop a plan for data collection and analysis. • Train all administrators on proper administration of the measure and interpretation of results/scores. • Evaluate the integration and usefulness of the measure into practice (e.g. efficiency of implementation, value of data per time spent, etc.). Issues Affecting Outcome Measures Administering outcome measure [3] modified Emory Functional Ambulation Profile (mEFAP) [6] • Performance-based measure for the post-stroke population. • Assesses functional mobility using an aggregate of 5 individually timed tests: (1) 5-meter walk on hard floor; (2) 5-meter walk on carpeted surface; (3) timed up-and-go test; (4) traversing a standardized obstacle course; and (5) ascending and descending 5 stairs. Benefits • Two of the sub-tests (5-meter walk and TUG) have been independently validated for many populations and can be used separately from the mEFAP [8]. • mEFAP allows score to be weighted based on amount of manual assistance or type of assistive device used, which allows for changes in the amount of assistance a stroke patient uses during the course of recovery/rehabilitation to be factored into scoring of performance. • Inter-rater reliability and test-retest reliability are high [6]. Limitations • Criterion validity remains questionable [6]. • Sensitivity to different mobility aids could benefit from more research [6]. Orthotics and Prosthetics Users’ Survey (OPUS) [9] • Self-report measure for prosthesis and orthosis users. • Assesses four domains: (1) functional status, (2) health- related quality of life, and (3) satisfaction with services and (4) device. • Specific functional status questionnaires for upper and lower limb device users. • 5 item response scale (e.g. strongly disagree to strongly agree). Benefits • Evaluation of satisfaction is required for facility accreditation by the American Board for Certification (ABC). • Survey detects a wide range of function, quality of life, and satisfaction with adequate internal validity [9]. • Each component may be administered separately. Limitations • More research required to assess sensitivity to change over time. Socket Comfort Score (SCS) [11] • Self-report measure of the comfort of a prosthetic socket. • Patient reports comfort on a scale of 0-10, 0 being the most uncomfortable socket imaginable to 10 being the most comfortable socket imaginable. Benefits • Easy to administer in a short amount of time. • Quantifies socket comfort, an important concept in prosthetic practice. • Repeatability, criterion related validity, and sensitivity to change reported [11]. • Strong relationship between poor clinical fit and low reported scores [11]. Limitations • Limited research establishing psychometric properties. Outcome Measures Relevant to O&P Tools to administer the mEFAP [7] Considerations There are no perfect outcome measures, and many could benefit from further development/refinement. Regardless, use of outcome measures can give added meaning and structure to clinical practice sometimes without adding substantial time to patient visits. Clinicians need to use their clinical judgment to evaluate outcome measure use in practice. Administering OPUS [10] Socket comfort graded from 0-10 [12]