The “Affiliated” list emerged from the process described in this methods article. All other lists on this table already existed and were used for consultation in arriving at the “Affiliated” list in the far right-hand column.
The authors compiled a complete and accurate list of 108 CTSC-affiliated faculty members following several months of investigation using the process outlined above. The authors were highly confident that the list included all faculty members who belonged in the study population, while excluding all others. This “Affiliated” list was stratified by (1) clinical, (2) basic science, (3) pharmacy, or (4) nursing and other faculty members, before a random selection process began for the subsequent study process.
Plato and Aristotle, as well as many early physical scientists in more recent centuries such as Bacon and Mill, pioneered inductive approaches [ 5 ]. In its purest form, an inductivist approach begins “with a collection of data, empirical observations or measurements of some kind, and build[s] theoretical categories and propositions from [the] relationships discovered among the data” [ 6 ]. Pure inductivism tends to introduce biases, however [ 7 ]. More modern forms of inductivism have emerged in the form of techniques for developing clearly delineated categorizations and classifications that have high fidelity with describing logical groupings in the real world.
Induction contrasts with the other major form of logical reasoning known as deduction. Whereas induction assembles more and more specific observations in order to form a generalization, deduction begins with a generalization and then applies this generalization to specific instances [ 8 ]. Deduction assumes that the generalization of interest consists of an agreed-upon fact, principle, or theory [ 9 ].
Inductive approaches are more common than might be recognized immediately. While rarely noted explicitly, investigators often employ inductive approaches at different junctures of the research process. Investigators might use inductive processes, for example, to generate hypotheses or research questions. Investigators also apply inductive approaches to define study populations. Clinicians use inductive techniques when formulating patient diagnoses. Librarians and information science professionals, moreover, use inductive processes while engaged in classification processes, such as indexing or meta-tagging.
The analytic induction approach “involves an iterative testing and retesting of theoretical ideas using the data” [ 10 ]. Investigators use analytic induction to develop more clearly defined concepts or categories by examining individual cases against emerging concepts [ 11 ]. It seeks to remain faithful to empirical observations of a limited number of cases as any categorizations take shape, while ensuring that investigators avoid jumping to premature conclusions. It abstracts from some detailed observations and then, in a quasi-deductive way, it tests those abstractions against more and more cases. The abstractions are revised if they prove too inflexible or too narrow to incorporate any apparent deviant cases [ 12 ].
Analytic inductive approaches orient one unfamiliar with a new phenomenon to make sense out of it. In a way, analytic inductivism creates order out of disorder, or at least order amidst relative confusion. A zoologist, for example, upon encountering a new species, might initially categorize the animal by color. Eventually the zoologist finds this narrow color categorization scheme to be insufficient over successive cases to accurately describe additional examples of the new species, which are otherwise similar, except for the different colors. The zoologist instead shifts focus to the presence of vertebrae in the new species as a possibly more precise categorization system for uniquely defining the species [ 13 ].
Becker illustrated the similar use of analytic inductivism in the social sciences when he wrote: “Once we have isolated … a generic feature of some social relation or process and given it a name, and thus created a concept, we can look for the same phenomenon in places other than where we found it” [ 14 ]. Inductive analysis in a broader sense offers great flexibility in defining previously nebulous categories and classifications, whether in the physical sciences, in the social sciences, or in clinical settings [ 15 ]. Inductive analysis can only serve as a starting point that initiates further exploration to validate concepts or systems in the real world. Neither inductive nor deductive logic approaches alone will suffice in the research process [ 16 ]. Yet, inductivism offers the advantage of viewing even a familiar research problem from a fresh perspective [ 17 ].
In this study, the analytic inductive aspect of defining a study population relied on reacting to existing lists in order to more precisely define the final criteria for identifying all members of the study population. Questions arose about the current employment status of certain faculty members who had left the university when their names still appeared on a list of faculty members funded through the CTSC, to cite one example. As noted already, the simplified truth table displayed in Table 1 guided the dual definitional and identification processes.
The study population description needs to offer a clear definition as to who belongs to the study population and who should be excluded from the study population. The Uniform Requirements for Manuscripts Submitted to Biomedical Journals remind investigators to be explicit about the selection criteria for inclusion and exclusion that define a study population, adding that “The guiding principle should be clarity about how and why a study was done in a particular way” [ 18 ]. Some researchers recommend including a table that delineates inclusion criteria [ 19 ]. The left-hand column in Table 1 provides an example of such inclusion criteria.
Investigators should employ standardized, commonly understood definitions of groups whenever possible in developing a study population description. The population of affiliated faculty members described in this article adhered to common conceptions of faculty members. Imagine the potential complexity of defining a study population associated with more nebulous concepts. Comstock et al. reveal, in their analysis of multiple epidemiological studies, how different investigators employed their own operational definitions for the concepts of race and ethnicity, rather than standardized definitions of these concepts [ 20 ]. The net result of these multiple definitions undermined the utility of comparing the studies. Gobbens et al. similarly discovered that the commonly referenced phrase “medical frailty in the elderly” did not have a standard definition, despite some investigators’ misconceptions to the contrary [ 21 ]. The research community in such circumstances might need to convene a consensus conference to arrive at a commonly understood definition [ 22 ].
Investigators need to be alert to (and, if necessary, to correct for) the possibility that their inclusion and exclusion criteria reflect gender, racial, ethnic, economic, or information disparities biases [ 23 ]. The first author made such a mistake, years ago while conducting a study of rural health care providers. The author assumed that most rural inhabitants had access to the same high-quality Internet connections as urban residents. Not much later, the field informants for the study population fortunately corrected this misunderstanding [ 24 ]. Attitudes, behaviors, or skills within a study population might change over time in other circumstances. If that change itself serves as a focus of the study, this dynamic needs to be addressed in the research strategy. For example, if a research study wishes to gauge the effectiveness of library or informatics training on a defined population, but some members of the initial study population acquire the training via different venues, then the investigators need to identify and exclude these individuals from the study population.
Who applies the stated inclusion criteria to determine eligibility? The authors defined the study population and then recruited participants directly from that defined population in every study that they have conducted. One experiment involving first-year medical students used enrollment lists of students in good academic standing to define the study population [ 25 ]. What if a study recruits individuals by asking prospective participants to self-identify? In such circumstances, study population members’ own self-definitions might not align with investigators’ definitions. One study found that immigrant and refugee youth self-defined themselves in ways within the contexts of their own experiences [ 26 ]. Investigators must clearly define and promulgate their definition of a study population early in the study to avoid enrollment of ineligible individuals either later in the study or at other study sites [ 27 ].
Validity refers to the close proximity of a defined concept and what the investigators are measuring [ 28 ]. For the central example in this article, a poorly defined or recruited study population of putatively CTSC-affiliated faculty members that includes ineligible individuals such as staff members who are not engaged in the same types of roles will jeopardize the credibility of the study. The inadvertent inclusion of faculty not affiliated with the CTSC or retired faculty members would have similarly threatened the integrity of the study. Critics could justifiably deem this, or any other study, to be flawed due to sloppiness in the assembling of the study population.
The authors implemented this study in a highly controlled university environment where many resources were available to verify the accuracy of individual members of the defined study population. The authors had an online employee directory, a faculty contracts office, and departmental administrators who often could quickly resolve identity or status issues. The authors additionally had their own extensive 17 years of cumulated experience with the CTSC to aid in the resolution of identity issues with a high degree of fidelity. Many research studies occur in less-controlled “field” contexts that lack such verification resources or expert knowledge.
Simpson et al. have noted that “Although it is recommended that study eligibility criteria should be clear, objective and precise, they are often complex and open to interpretation” [ 29 ]. The authors hope that the recommendations in this article will help others to streamline and lend greater credibility to their own studies involving human populations. The authors could not locate a study to serve as an exemplar of a perfectly defined and identified study population. Instead, they identified a series of large scale cohort studies in the International Journal of Epidemiology that do offer definitions of study populations that are both concise and precise. The cohorts involve study populations as diverse as Japanese diabetics, Danish nurses, secondary school students in Amsterdam, twins, German uranium miners, and Thai university students [ 30 – 35 ]. Readers might find these examples helpful for their own research studies.
Institutions and organizations often function and build decisions on definitions of groups that are not well defined. Such definitions will serve episodic needs connected with an initiative or program in the short-run, rather than as master lists capable of serving long-term institutional needs. In the example at the University of New Mexico, many official CTSC communications contain the words “affiliated” or “affiliation” when referring to faculty with an institutional relationship, when no definition exists to guide one in interpreting what these words mean. All organizations reflect at least some ambiguity when defining affiliated faculty groups. Still, many investigators wanting to conduct organizationally related research often assume, at least initially, that the organizations have rigorously defined all group definitions.
Real world experience reveals wide gaps between aspirational goals and actual practices when defining populations. Motivating organizations to construct well-thought-out definitions poses challenges, since doing so might force these organizations to confront very difficult structural, hierarchal, political, or cultural conflicts. Some examples of these myriad conflicts may involve resolving ambiguities about budgeting, organizational authority, or political problems. As an example, defining who is affiliated with a CTSA at an institution may impact the authority academic departments or other research centers have over their own primarily affiliated faculty members. These kinds of probing discussions are often difficult to pursue smoothly and might be politically charged.
Despite these challenges, organizations that invest the time and resources to more rigorously define these group definitions will reap the previously described long-term benefits. Perhaps analogous to swallowing bad-tasting medicine, working out important group definitions will improve organizational health by helping the organization to engage in difficult, complex, or otherwise unpleasant issues.
Defining a study population early in the design stages of a research project will help to facilitate a smooth implementation phase. Clear definitions inform the value of applying research results to relevant populations for real world purposes. Importantly, a carefully and accurately defined study population enhances the completed study’s validity. This article describes the many challenges in defining a research study population and their potential solutions.
Funding Support
Health Sciences Library and Informatics Center, University of New Mexico, Albuquerque and Clinical and Translational Sciences Center, University of New Mexico. NIH Award # UL1TR000041.
IMAGES
VIDEO
COMMENTS
The target population is a subset of the general public identified as the targeted market for a given product, advertising, or research. It is a subset of the entire population chosen to serve as the objective audience.
In research, there are 2 kinds of populations: the target population and the accessible population. The accessible population is exactly what it sounds like, the subset of the target population that we can easily get our hands on to conduct our research.
This paper thoroughly explores the foundational principles governing population and target population concepts within research methodology. It delves into the essential roles these concepts...
Defining the target population is an essential part of protocol development to ensure that the study participants are well suited to the research question ( Hulley et al., 2013 ). The target population is the entire group of people who share a common condition (disease process) or characteristic the researcher is interested in studying ( Elfil ...
The study depicts how the research goal, contexts and assumptions can dictate the content and concentration of the target and accessible population in qualitative inquiry.
The target population is the group of individuals that the intervention intends to conduct research in and draw conclusions from. In cost-effectiveness analysis, characteristics of the target population and any subgroups should be described clearly.
The article emphasizes the significance of correctly defining the target population to ensure accurate research findings and appropriate generalizations.
Study population is the operational definition of target population (Henry, 1990; Bickman & Rog, 1998). Researchers are seldom in a position to study the entire target population, which is not always readily accessible.
Target population, study population and study sample. A population is a complete set of people with a specialized set of characteristics, and a sample is a subset of the population. The usual criteria we use in defining population are geographic, for example, “the population of Uttar Pradesh”.
Defining the study population in a research project involves inductive reasoning, critical thinking, and pragmatic project management skills. Yet, few research methods books or articles explicitly address the many issues related to rigorously defining and identifying members of study populations.