1.1
本指南涵盖了在感官和产品评估方面具有潜在不同功能角色和培训程度的一小群人(通常在3到10人之间)为特定目标评估产品或一系列产品的情况,需要做出预先确定的决定,但不使用正式的假设测试或统计。在产品测试行业,这些通常被称为“替补”、“切割”或“替补筛选”,或者,就食品而言,“品尝”、“非正式品尝”、“团队品尝”或“技术品尝”在本指南中,使用了术语“小群体产品评估”(SGPE)。
1.2
本指南的目的是提供最佳实践,以确保以足够严格的方式进行SGPE,以便在考虑风险的同时做出最合适的决策或获得所需的学习。
由于参与者在功能角色、对手头问题的了解、感官敏感性以及感官或产品评估培训程度方面可能存在异质性,因此无法保证就前进道路达成一致的可能性。此外,由于之前的知识或他们在组织中的角色,参与者可能对要决定的问题有某些偏见。可以通过正确规划和执行SGPE来解决这些潜在的脱轨问题。当SGPE是非结构化的、未聚焦的、实验错误和偏差不受控制时,SGPE的输出不会通知决策或以科学的方式提供所需的学习。本文件的目标是通过概述结构、提前定义决策标准并提供实施指南,利用现有的感官理论和方法,提高小群体产品评估的实践。
这些SGPE的输出用于通知决策并确定下一步,包括每一步涉及的风险。
SGPE被广泛使用,当正确进行时,是感官专业人员工具箱中的一个选项。只有在已知、说明和分担风险的情况下,才应进行SGPE。仅凭有限的时间和资源不足以成为利用SPGE测试和放弃正式感官测试的充分理由。这样做的风险必须得到所有相关方的明确沟通和同意。
SGPE的正确用途有几个:筛选变量、建立假设、获取有关产品集或类别的信息、在需要低风险产品决策时采取行动,或在整个开发计划中进行产品学习。
在所有这些情况下,团队必须接受由SGPE输出通知决策所带来的风险。SGPE涉及的一个风险是缺少产品之间的微小差异(β风险),而评估的目标是找到此类差异,特别是那些可能对消费者重要的差异。SGPE未能发现差异并不意味着建立了产品相似性或等效性,因为建立相似性/等效性需要比SPGE常见的样本量大得多。
1.3
本指南涵盖了规划和实施过程,包括目标设定、方法确定、参与者数量和类型、投票、样本制备、决策标准、要包含的产品、收集的信息审查和岗位管理-
产品评估讨论,在小组内达成决策。还包括记录和传达SGPE输出,以及无法做出决定时的下一步。包括食品、家庭和个人护理等行业的工作示例。涵盖的不同类型的SGPE包括那些通常执行的类型,但并不详尽。
1.4
本指南不包括在大规模测试实施之前使用小组评估来试验研究或测试协议。此外,本指南既不建议也不包括使用小组评估来替代包含正式假设测试和统计分析的较大评估,或替代享乐测试。
本指南也不包括出于演示或信息目的而进行的质量职能部门的常规活动和产品审查,这些活动没有定义决策标准。
1.5
看见
5.2
关于感官专业人员或受过培训的代表在结构化SGPE的规划、设计、实施或监督中的作用的最佳实践建议。
1.6
本标准并非旨在解决与其使用相关的所有安全问题(如有)。本标准的用户有责任在使用前制定适当的安全、健康和环境实践,并确定监管限制的适用性。
1.7
本国际标准是根据世界贸易组织技术性贸易壁垒(TBT)委员会发布的《关于制定国际标准、指南和建议的原则的决定》中确立的国际公认标准化原则制定的。
====意义和用途======
4.1
使用SGPE的最佳实践确保了决策将基于科学原则,获得的结果将比没有这种规划、结构、重点和最佳实践的评估会议更加客观。这些小组评估与更正式的产品测试形成对比,其中包括资格预审参与者样本、假设测试和统计分析。
如果没有最佳的感官实践和程序,SGPE可能是非结构化的、非系统的、难以管理的,并且可能导致输出不明确、不可信或被忽视。此外,使用适当的感官练习可以减少具有特定样本知识或希望推进议程的参与者之间的偏见。本指南提供了一个概念化、组织和执行这些SGPE的框架。
4.2
SGPE用于不需要正式的、假设驱动的产品评估的情况。其中包括决策风险很小或利益相关者对伴随风险的决策感到满意的情况,或两者兼而有之的情况。这些情况的示例可能包括样本或其他资源的可用性有限、潜在的专利暴露或目标人群的低发病率。
SGPE可以是更正式的产品测试之前的初始筛选步骤或前体测试。在适当的背景下,SGPE本身也可以成为决策工具。当少数人在没有受控条件的情况下评估产品时,使用本文提出的框架可以提供一定程度的严格性。2009年Pangborn感官科学研讨会上的海报
(
1.
)
3.
报告了SGPE的调查结果。59 % 92%的受访者(N=92)表示,在他们的工作场所,非感官专业人员通常会组织SGPE。
表1
总结了一个典型的非结构化产品评估与一个不遵循最佳实践的小组和一个遵循本指南中概述的最佳实践的SGPE之间的关键区别。
1.1
This guide covers those occasions in which a small group of individuals (generally between three and ten) with potentially different functional roles and degrees of training in sensory and product evaluation, evaluates a product or series of products for a specific objective, with a pre-identified decision to be made, but without the use of formal hypothesis testing or statistics. In the product testing industry, these are often referred to as “benchings,” “cuttings,” or “bench screenings” or, in the case of food products, “tastings,” “informal tastings,” “team tastings,” or “technical tastings.” In this guide, the term “Small Group Product Evaluation” (SGPE) is used.
1.2
The aim of this guide is to provide best practices to ensure that SGPE are conducted with sufficient rigor to enable the most appropriate decision or to yield the needed learning while considering the risk. Because the participants may be heterogeneous with respect to functional role, knowledge of the issue at hand, sensory sensitivity, and degree of sensory or product evaluation training, the likelihood of agreement on a path forward is not assured. Additionally, participants may have certain biases with respect to the issue to be decided, because of prior knowledge or their role within the organization. These potential derailers can be addressed through proper planning and execution of an SGPE. When SGPE are unstructured, unfocused and experimental error and biases uncontrolled, the outputs of SGPEs do not inform decisions or deliver the desired learning in a scientific manner. The goal of this document is to elevate the practice of small group product evaluations by outlining a structure, defining decision criteria in advance, and providing guidelines for implementation, drawing upon existing sensory theory and methods. Outputs from these SGPE are used to inform decisions and determine next steps including the risks involved with each of these.
SGPE are widely used, and when properly conducted, are an option in the sensory professional’s toolbox. SGPE should be conducted only when the risks are known, stated, and shared. Limited timing and resources alone are not adequate reasons to utilize SPGE testing and forgo formal sensory testing. Risks in doing so must be clearly communicated and agreed to by all involved parties.
The proper uses of SGPE are several: to screen variables, to establish hypotheses, to gain information about a product set or category, to take a course of action where a low risk product decision is needed or for product learning throughout a development program. In all of these cases, the team must accept the risks that come with having SGPE outputs to inform a decision. One risk involved in SGPE is missing small differences among products (beta risk), when the goal of the evaluation is to find such differences, particularly those differences that might be important to the consumer. An SGPE failure to find differences does not mean that product similarity or equivalence is established, since much larger sample sizes than are common to SPGE’s are required to establish similarity/equivalence.
1.3
This guide covers the planning and implementation processes, including objective setting, method determination, number and types of participants, ballots, sample preparation, decision criteria, products to be included, review of information collected, and management of the post-product evaluation discussion to arrive at a decision within the small group. Documenting and communicating SGPE outputs are also covered, as well as next steps if a decision cannot be reached. Worked examples across industries including food, household, and personal care are included. The different types of SGPE covered include those commonly executed but is not exhaustive.
1.4
This guide does not cover the use of small group evaluations to pilot research or test protocols before implementation in larger scale testing. In addition, the use of small group evaluations to substitute for larger evaluations that incorporate formal hypothesis testing and statistical analysis or to replace hedonic testing are neither recommended nor included within this guide. SGPE that are regular activities of a quality function and product reviews that are done for demonstration or informative purposes with no defined decision criteria are also not covered in this guide.
1.5
See
5.2
for a best practice recommendation for the role of the sensory professional or trained delegate in the planning, designing, conducting, or oversight of structured SGPE.
1.6
This standard does not purport to address all of the safety concerns, if any, associated with its use. It is the responsibility of the user of this standard to establish appropriate safety, health, and environmental practices and determine the applicability of regulatory limitations prior to use.
1.7
This international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for the Development of International Standards, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade (TBT) Committee.
====== Significance And Use ======
4.1
Using best practices for SGPE ensures that decisions made will be based on scientific principles, and the outputs obtained will be more objective than those evaluation sessions conducted without this planning, structure, focus, and best practices. These small group evaluations contrast with more formal product tests that include a prequalified participant sample, hypothesis testing, and statistical analysis. Without best sensory practices and procedures, SGPE may be unstructured, unsystematic, difficult to manage, and may lead to outputs that are unclear, not credible, or ignored. Additionally, the use of proper sensory practices reduces bias among participants with specific sample knowledge or a desire to advance an agenda. This guide provides a framework for conceptualizing, organizing, and executing these SGPE.
4.2
SGPE are used in situations in which formal, hypothesis-driven product evaluations are not required. These include situations in which the decision risk is small or stakeholders feel comfortable in making a decision with the attendant risks, or both. Examples of these situations may include limited availability of samples or other resources, potential patent exposure, or low incidence of target population. The SGPE could be an initial screening step or a precursor test before a more formal product test. In the proper context, SGPE can also be a decision-making tool in and of itself. Using the framework presented here provides a degree of rigor that may be absent when a few people evaluate a product without controlled conditions. A poster presented at the 2009 Pangborn Sensory Science Symposium
(
1
)
3
reported the results of a survey on SGPE. 59 % of respondents (N = 92) stated that, at their place of employment, typically, non-sensory professionals organized SGPE.
Table 1
summarizes key differences between a typical unstructured product evaluation with a small group not following best practices and an SGPE that follows the best practices outlined in this guide.