ASTM D3764 : 23

Standard Practice for Validation of the Performance of Process Stream Analyzer Systems

Standard Details

1.1 This practice describes procedures and methodologies based on the statistical principles of Practice D6708 to validate whether the degree of agreement between the results produced by a total analyzer system (or its subsystem), versus the results produced by an independent test method that purports to measure the same property, meets user-specified requirements. This is a performance-based validation, to be conducted using a set of materials that are not used a priori in the development of any correlation between the two measurement systems under investigation. A result from the independent test method is herein referred to as a Primary Test Method Result (PTMR).

1.1.1 The degree of agreement described in 1.1 can be either for PPTMRs and PTMRs measured on the same materials, or for PPTMRs measured on basestocks and PTMRs measured on these same basestocks after constant level additivation.

1.1.2 In some cases, a two-step procedure is employed. In the first step, the analyzer and PTM are applied to the measurement of the same blendstock material. If the analyzer employed in Step 1 is a multivariate spectrophotometric analyzer, then Practice D6122 is used to access the agreement between the PPTMRs and the PTMRs for this first step. Otherwise, this practice is used to compare the PPTMRs to the PTMRs measured for this blendstock to determine the degree of agreement. In a second step, the PPTMRs produced in Step 1 are used as inputs to a second model that predicts the results obtained when the PTM is applied to the analysis of the finished blended product. Since this second step does not use analyzer readings, the validation of the second step is done independently. Step 2 is only performed on valid Step 1 results. Note that the second model might accommodate variable levels or multiple material additions to the blendstock.

1.2 This practice assumes any correlation necessary to mitigate systemic biases between the analyzer system and PTM have been applied to the analyzer results. See Guide D7235 for procedures for establishing such correlations.

1.3 This practice assumes any modeling techniques employed have the necessary tuning to mitigate systemic biases between the analyzer PPTMR and PTMR have been applied to the model results. Model form and tuning is not covered by this practice, only the validation of the model output.

1.4 This practice requires that both the primary method against which the analyzer is compared to, and the analyzer system under investigation, are in statistical control. Practices described in Practice D6299 should be used to ensure this condition is met.

1.5 This practice applies if the process stream analyzer system and the primary test method are based on the same measurement principle(s), or, if the process stream analyzer system uses a direct and well-understood measurement principle that is similar to the measurement principle of the primary test method. This practice also applies if the process stream analyzer system uses a different measurement technology from the primary test method, provided that the calibration protocol for the direct output of the analyzer does not require use of the PTMRs (see Case 1 in Note 1).

1.6 This practice does not apply if the process stream analyzer system utilizes an indirect or mathematically modeled measurement principle such as chemometric or multivariate analysis techniques where PTMRs are required for the chemometric or multivariate model development. Users should refer to Practice D6122 for detailed validation procedures for these types of analyzer systems (see Case 2 in Note 1).

Note 1: For example, for the measurement of benzene in spark ignition fuels, comparison of a Mid-Infrared process analyzer system based on Test Method D6277 to a Test Method D3606 gas chromatography primary test method would be considered Case 1, and this practice would apply. For each sample, the Mid-Infrared spectrum is converted into a single analyzer result using methodology (Test Method D6277) that is independent of the primary test method (Test Method D3606). However, when the same analyzer uses a multivariate model to correlate the measured Mid-Infrared spectrum to Test Method D3606 reference values using the methodology of Practice D8321, it is considered Case 2 and Practice D6122 applies. In this case 2 example, the direct output of the analyzer is the spectrum, and the conversion of this multivariate output to an analyzer result require use of Practice D6122, hence it is not independent of the primary test method.

1.7 Performance Validation is conducted by calculating the precision and bias of the differences between results from the analyzer system (or subsystem) after the application of any necessary correlation, (such results are herein referred to as Predicted Primary Test Method Results (PPTMRs)), versus the PTMRs for the same sample set. Results used in the calculation are for samples that are not used in the development of the correlation. The calculated precision and bias are statistically compared to user-specified requirements for the analyzer system application.

1.7.1 For analyzers used in product release or product quality certification applications, the precision and bias requirement for the degree of agreement are typically based on the site or published precision of the Primary Test Method.

Note 2: In most applications of this type, the PTM is the specification-cited test method.

1.7.2 This practice does not describe procedures for establishing precision and bias requirements for analyzer system applications. Such requirements must be based on the criticality of the results to the intended business application and on contractual and regulatory requirements. The user must establish precision and bias requirements prior to initiating the validation procedures described herein.

1.8 Two procedures for validation are described: the line sample procedure and the validation reference material (VRM) injection procedure.

1.9 Only the analyzer system or subsystem downstream of the VRM injection point or the line sample extraction point is being validated by this practice.

1.10 The line sample procedure is limited to applications where material can be safely withdrawn from the sampling point of the analyzer unit without significantly altering the property of interest.

1.10.1 The line sample procedure is the primary option for when the validation is for (2b) materials including effect from additional treatment to the material.

1.11 Validation information obtained in the application of this practice is applicable only to the type and property range of the materials used to perform the validation.

1.12 Two types of validation are described: General Validation, and Level Specific Validation. These are typically conducted at installation or after major maintenance once the system mechanical fitness-for-use has been established.

1.12.1 General Validation is based on the statistical principles and methodology of Practice D6708. In most cases, General Validation is preferred, but may not always be possible if the variation in validation materials is insufficient. General Validation will validate analyzer operation over a wider operating range than Level Specific Validation.

1.12.2 When the variation in available validation materials is insufficient to satisfy the requirements of Practice D6708, a Level Specific Validation is done to validate analyzer operation over a limited range.

1.12.3 The validation outcome are considered valid only within the range covered by the validation material Data from several different Validations (general or level-specific) can potentially be combined for use in a General Validation.

1.13 Procedures for the continual validation of system performance are described. These procedures are typically applied at a frequency commensurate with the criticality of the application.

1.14 This practice does not address procedures for diagnosing causes of validation failure.

1.15 This standard does not purport to address all of the safety concerns, if any, associated with its use. It is the responsibility of the user of this standard to establish appropriate safety, health, and environmental practices and determine the applicability of regulatory limitations prior to use.

1.16 This international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for the Development of International StandardDetails, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade (TBT) Committee.