Java程序辅导
C C++ Java Python Processing编程在线培训 程序编写 软件开发 视频讲解
C C++ Java Python Processing编程在线培训 程序编写 软件开发 视频讲解
1 © 2012 3M. All Rights Reserved. METHOD VALIDATION & VERIFICATION APEC-PTIN Project: CHILE 2013 2 © 2012 3M. All Rights Reserved. § Provide definitions of validation & verification from key resources § List some key performance characteristics for method validation § Provide an overview of 3rd party method validation schemes § Distinguish validation and verification § List some key performance characteristics for method verification § Distinguish verification: method verification vs “continuous” verification § Provide links to references for further use / information OBJECTIVES 3 © 2012 3M. All Rights Reserved. § CODEX - CAC/GL 74-2010: Guidelines on performance criteria and validation of methods for detection, identification and quantification of specific DNA sequences and specific proteins in foods § ISO/DIS 16140-1: Microbiology of food and animal feed, Method validation, Part 1: Vocabulary Currently under revision § USDA FSIS: Guidance for Test Kit Manufacturers, Laboratories: Evaluating the Performance of Pathogen Test Kit Methods; and Establishment Guidance for the Selection of a Commercial or Private Microbiological Testing Laboratory § US FDA: Guidelines for the Validation of Analytical Methods for the Detection of Microbial Pathogens in Foods § Health Canada: The Compendium of Analytical Methods, Volume 1, Development of Methods § ISO/IEC 17025:2005 General requirements for the competence of testing and calibration laboratories Main Validation & Verification Definition Sources: 4 © 2012 3M. All Rights Reserved. Reference Validation Definition Codex CAC/GL 741 Process to establish the performance characteristics and limitations of an analytical method: which analytes, in what kind of matrices, in the presence of which interference. Result = precision and trueness values of a certain analytical method under the examined conditions. 1 [pg 2] ISO 16140-12 Establishment of the performance characteristics of a method and provision of objective evidence that the performance requirements for a specified intended use are fulfilled.2 [pg 2] USDA FSIS3 Process to measure performance characteristics of a particular test, with the goal of determining whether the test is equivalent to the reference test for the intended conditions of use. “Equivalent” = the performance characteristics are statistically indistinguishable.3 [pg 2] US FDA4 Demonstration that adequate confidence is provided when the results obtained by the alternative method i.e. the commercially available kit, are comparable to or exceed those obtained using the reference method using the statistical criteria contained in the approved validation protocol.4 [pg 16] Health Canada5 Evaluation of the performance parameters of a new method in comparison to an accepted reference method using paired or unpaired samples. In the context of relative validation, the results of the reference method are assumed to reflect the true microbiological status of the samples and the performance parameters of the alternative method are calculated in relation to this. (Source: Part 4) ISO 17025:20056 The confirmation by examination and the provision of objective evidence that the particular requirements for a specific intended use are fulfilled.6 WHAT is Validation? 5 © 2012 3M. All Rights Reserved. Validation: 1. Performance characteristics are defined, then… 2. compared to a reference method, and then… 3. statistically evaluated to determine “equivalence” A method is validated for the matrices [& sample sizes] included in the validation Summary of ALL Validation reference definitions: 6 © 2012 3M. All Rights Reserved. Categories of Microbiological Test Methods Performance Characteristics Included in a Validation Study1 Performance Characteristic Quantitative Qualitative (presence/ absence) Relative Accuracy (% Recovery) Yes No Matrix Effects Yes Yes Precision Yes No Sensitivity Yes Yes Specificity Yes Yes Inclusivity Yes Yes Exclusivity Yes Yes False Positive Rate Yes Yes False Negative Rate Yes Yes LOD [POD or LOD50] Yes Yes LOQ Yes No Ruggedness Yes Yes Linearity/Range Yes No Dr. Michael Brodsky AOAC International 2013 Annual Meeting 7 © 2012 3M. All Rights Reserved. Methods requiring validation: ― New or original methods ― Modifications to validated methods ― Extension of scope to include additional analytes, matrices, or changes in intended use ― Changes involving new technology or automation ― Significant parameter changes: reagents/apparatus, time /temperature incubation periods, enrichment media, etc. It is possible that small changes/matrix extensions to a related product etc would not need a ‘full validation’ but may need some level of testing to assess for unintended differences. 8 © 2012 3M. All Rights Reserved. § “Validated” methods are required for conducting “Official Testing” ― Official = adjective, relating to an authority or public body and its activities and responsibilities [Oxford online dictionary] § European Union regulations specifically stipulate validated methods that have been collaboratively studied § Internationally, regulatory agencies require, or at least prefer, to use methods that have been validated and collaboratively studied. § Some global regulatory agencies require AOAC Official Methods of Analysis to receive Final Action, before they may be used as an “Official” method WHY are Validated methods important? 9 © 2012 3M. All Rights Reserved. Agency Method Validation Requirements Codex CAC/GL 218 5.2.1 Whenever possible, only methods for which the reliability (accuracy, reproducibility, inter- and intra-laboratory variation) has been statistically established in comparative or collaborative studies in several laboratories should be used. (EC) No 2073/20079 The use of alternative analytical methods is acceptable when the methods are validated against the reference method in Annex I and if a proprietary method, certified by a third party in accordance with the protocol set out in EN/ISO standard 16140 or other internationally accepted similar protocols, is used. Pg 9 USDA FSIS11 All analytical methods described in FSIS-MLG have been scientifically validated and are considered fit for their intended purpose. Laboratories may also use other validated testing methods that differ from the methods described in FSIS-MLG. For the intended conditions of use, the performance characteristics of the new method and the well characterized method should be statistically indistinguishable Pg 13 US FDA4 FDA-BAM presents the agency's preferred laboratory procedures for microbiological analyses of foods and cosmetics. Methods…fully validated in a collaborative study monitored and evaluated by an independent accrediting body e.g. AOAC-OMA, AFNOR, etc. require in-house verification for the “first use”… Health Canada5 HPB Methods and Laboratory Procedures are generally used to determine compliance with the Food and Drugs Act and Regulations. In general, methods published in the Compendium of Analytical Methods for which the "application" section is appropriate for the intended purpose are used for microbiological testing of food. REQUIREMENTS of methods for “Official” use 10 © 2012 3M. All Rights Reserved. § Official Methods = Methods used or accepted for use by a regulatory authority ― Traditional, cultural methods [may be jurisdiction specific] ― ISO Method ― ISO 16140 Validated Methods ― AOAC Official Methods ― FDA Bacteriological Analytical Methods ― USDA FSIS Microbiology Laboratory Methods ― HC Compendium of Analytical Methods, Volumes 1 to 5 § Any of the above methods MAY be accepted for use by regulatory agencies, IF: • They have been compared to an acceptable reference method • They have been validated as “fit for purpose” for the matrices analyzed What is considered an OFFICIAL method? 11 © 2012 3M. All Rights Reserved. § ISO ― AFNOR: French National Organization for Standardization ― MicroVal: European certification body for microbiology methods ― NordVal: Nordic certification body: Denmark, Finland, Iceland, Norway and Sweden § AOAC Research Institute [RI]: International Standards Organization ― Performance Tested Method (PTM) ― Official Method of Analysis (OMA) § Health Canada: Canadian certification through Microbiology Methods Committee (MMC) ― Official Methods (MFOs) ― HPB Methods (MFHPBs) ― Laboratory Procedures (MFLPs) Validation Schemes 12 © 2012 3M. All Rights Reserved. AFNOR and MicroVal Certification schemes based on EN/ISO 16140: § Comparison Study to reference method 1. Independent expert laboratory co-ordinates validation 2. Inter-laboratory study [collaborative study] 3. “All Foods” Claim § Regular audit of the quality system of the manufacturer [based on EN 29002 standard (ISO 9002)] § Regular verification of the certified methods post certification [every 4 years] 13 © 2012 3M. All Rights Reserved. AOAC Research Institute § AOAC Performance Tested Method [PTM] ― Pre-collaborative + 1 Independent lab: I/E; Ruggedness testing, paper review of quality system. ― Yearly renewal – paper process only ― NO all foods claim – only for matrices evaluated § AOAC PTM / Official Method of Analysis [OMA] Harmonization ― Complete AOAC PTM program / Initiate collaborative study to achieve OMA First Action status; Final Action status after 2 yrs use in the industry without issue § AOAC PTM / OMA/ MicroVal Harmonization ― Both programs run simultaneously – 1 collaborative study 14 © 2012 3M. All Rights Reserved. § Micro Food Lab Procedure [MFLP] ― Independent lab [pre-collaborative study] ― All Foods Claim ― Reviewed every 5 years ― Paper review of quality system § Health Protection Food Branch [MFHPB] Methods ― Must be an MFLP method for 1 year prior to collaborative study Health Canada 15 © 2012 3M. All Rights Reserved. § AFNOR http://www.afnor-validation.com/afnor-validation-validated-methods/validated-methods.html § MicroVal http://www.microval.org/validated-methods.html § AOAC http://www.aoac.org/iMIS15_Prod/AOAC_Member/Research_Institute/RI_Main.aspx?WebsiteKey=2e25ab5a-1f6d-4d78- a498-19b9763d11b4&hkey=33b744f6-f71e-456a-9305-552469587666&CCO=5 § Health Canada http://www.hc-sc.gc.ca/fn-an/res-rech/analy-meth/microbio/index-eng.php (Volumes 1 to 5) § NordVal http://www.nmkl.org/NordVal/NordValMetoder.htm Method Scheme Websites – list validated methods including claimed matrices, sample sizes 16 © 2012 3M. All Rights Reserved. § A global Stakeholder panel was formed [through AOAC RI] § Performance characteristics between agencies/schemes were compared § 5 areas for harmonization: 1. Reference methods – so much variation, some matrix specific 2. Definition of Fractional Positives – 25-75% positive out of 20 samples 3. Selection of Food/Category – Chart created for incorporation into ISO 16140-2 4. Statistical Analysis – RLOD, Chi-Square and/or POD models 5. Data Sets for Qualitative Collaborative Study/Sample Size – 10 labs / 25 g samples § Recent addition: “All Foods” claim now = “Broad range of foods” Harmonization – ISPAM International Stakeholder Panel on Analytical Methods 17 © 2012 3M. All Rights Reserved. Reference Verification Definition Codex Could not find a method verification definition Codex documents [only process verification] ISO 16140-1 Demonstration that a validated method functions in the user's hands according to the method's specifications determined in the validation study and is fit for purpose USDA FSIS Laboratory should demonstrate on-going competence in performing the method at their facility, which would include participating in proficiency test programs11 Pg 13 US FDA The confirmation by examination and the provision of objective evidence that specified requirements have been fulfilled.4 Pg 16 Health Canada Verification studies are required when a laboratory implements a fully validated, published or reference standard method. The laboratory should provide evidence, by conducting a secondary evaluation study, that it is capable of achieving the published performance characteristics of the method.5 Vol 1; Part 4; 3.4 ISO 17025:2005 The Laboratory shall confirm that it can properly operate standard methods before introducing the tests or calibrations Comparison of VERIFICATION definitions 18 © 2012 3M. All Rights Reserved. § Proof that the laboratory can perform the [validated] method as intended § FDA has detailed Guidance on in-house verification of a validated method before use4 Pg 16 Verification for USE This applies only to matrices and sample sizes that were included in the validation study 19 © 2012 3M. All Rights Reserved. § Validation ― Demonstrate the METHOD is “equivalent” to the reference method [for the matrices validated] based on defined method criteria § Verification ― Demonstrate in the hands of the USER that the method can be performed to meet the defined method criteria Distinguishing Validation and Verification 20 © 2012 3M. All Rights Reserved. Categories of Microbiological Test Methods Performance Characteristics Included in the Verification of a Validated Method1 Performance Characteristic VALIDATION Quantitative VALIDATION Qualitative (P/A) VERIFICATION (Where Applicable) Relative Accuracy (% Recovery) Yes No Yes Matrix Effects Yes Yes Yes Precision Yes No Yes Sensitivity Yes Yes Yes Specificity Yes Yes Yes Inclusivity Yes Yes No Exclusivity Yes Yes No False Positive Rate Yes Yes Yes False Negative Rate Yes Yes Yes LOD [POD or LOD50] Yes Yes No LOQ Yes No No Ruggedness Yes Yes No Linearity/Range Yes No No Dr. Michael Brodsky AOAC International 2013 Annual Meeting 21 © 2012 3M. All Rights Reserved. § Proficiency Testing: ― Should be done several times per year ― Should be handled as a routine sample – not specially handled ― Can help determine the performance of individual laboratories for specific tests ― Is used to monitor laboratories’ continuing performance ― Can demonstrate an analyst’s ability to properly perform the test § Test samples: ― Certified proficiency test providers ― Internal Ring tests CONTINOUS Verification - ISO 17025 requirement 22 © 2012 3M. All Rights Reserved. § Definitions of validation & verification § Key performance characteristics for method validation and verification § Several global 3rd party method validation schemes were outlined § Distinguish validation and verification § References for further use / information SUMMARY 23 © 2012 3M. All Rights Reserved. 24 © 2012 3M. All Rights Reserved. 1. CODEX - CAC/GL 74-2010 Guidelines on performance criteria and validation of methods for detection, identification and quantification of specific dna sequences and specific proteins in foods http://www.codexalimentarius.org/standards/list-of-standards/ 2. ISO/DIS 16140-1 Microbiology of food and animal feed — Method validation — Part 1: Vocabulary Currently under revision 3. USDA FSIS: Guidance for Test Kit Manufacturers, Laboratories: Evaluating the Performance of Pathogen Test Kit Methods http://www.fsis.usda.gov/shared/PDF/Validation_Studies_Pathogen_Detection_Methods.pdf 4. US FDA: Guidelines for the Validation of Analytical Methods for the Detection of Microbial Pathogens in Foods http://www.fda.gov/downloads/ScienceResearch/FieldScience/UCM273418.pdf; 5. Health Canada: Procedure for the Development and Management of Food Microbiological Methods; Part 1. Definitions and Explanations of Methods and General Procedures; 3. Definition of Terms, 3.1 Method http://www.hc-sc.gc.ca/fn-an/res-rech/analy-meth/microbio/volume1/method_development-eng.php 6. ISO 17025:2005 General requirements for the competence of testing and calibration laboratories NO LINK, this is copyrighted material 7. 2013 AOAC International Annual Meeting presentation, with permission from: Michael Brodsky, President, Brodsky Consultants, Thornhill, Ontario, Canada mhbrodsky@rogers.com 8. CODEX – CAC/GL 21- 1997 Principles for the Establishment and Application of Microbiological Criteria for Foods http://www.codexalimentarius.org/standards/list-of-standards/ 9. COMMISSION REGULATION (EC) No 2073/2005 of 15 November 2005 on microbiological criteria for foodstuffs : http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2005:338:0001:0026:EN:PDF 10. Health Products and Food Branch (HPFB) Standards and Guidelines for Microbiological Safety of Food - An Interpretive Summary 6. Equivalent Methods http://www.hc-sc.gc.ca/fn-an/res-rech/analy-meth/microbio/volume1/intsum-somexp-eng.php 11. USDA FSIS: Establishment Guidance for the Selection of a Commercial or Private Microbiological Testing Laboratory: http://www.fsis.usda.gov/shared/PDF/Guidance_Selecting_Micro_Testing_Lab.pdf 12. AOAC: AOAC INTERNATIONAL Methods Committee Guidelines for Validation of Microbiological Methods for Food and Environmental Surfaces http://www.aoac.org/imis15_prod/AOAC_Docs/StandardsDevelopment/AOAC_Validation_Guidelines_for_Food_Microbiology-Prepub_version.pdf 13. Official Methods for the Microbiological Analysis of Foods http://www.hc-sc.gc.ca/fn-an/res-rech/analy-meth/microbio/volume1/index-eng.php References