research methodology question papers for phd

We are the No 1 PhD Consultancy in South India

20+ students register for phd every month, coursework – research methodology -sample question paper with answers.

Multiple Choice Questions

• Conference proceedings are considered as.documents. a. Conventional b. Primary c. Secondary d. Tertiary Answer : b. Primary
• Informationis….. a. RawData b. Processed Data c. Inputdata d. Organized data Answer : b. Processed Data
• Information acquired by experience or experimentation is called as: a. Empirical b. Scientific c. Facts d. Scientific Evidence Answer : b. Scientific
• Abstract elements representing classes of phenomena within the field of study are called : a.Concepts b.Theories c.Variables d.Hypothesis Answer: a. Concepts
• All living things are made up of cells Blue whale is a living being, Thereforeblue whale is made up of cells’ The reasoning used here is a. Inductive b. Deductive c. Hypothetic deductive d. Both a and b Answer : b. Deductive
• Questionnaire is a: a. Research method b. Measurement technique c. Tool for data collection d. Data analysis technique Answer : b. Measurement Technique
• Mean, Median and Mode are a. Measures of deviation b. Ways of sampling c. Measure of control tendency d. None of the above Answer : c. Measure of control tendency
• The reasoning that uses general principle to predict specific results is calledas- a. Inductive b. Deductive c. Both a and b d. Hypothetic o-deductive Answer : b. Deductive
• A research paper is a brief report of research work based on a. Primary Data only b. Secondary Data only c. Both a and b d. None of the above Answer : c. Both a and b
• Research is a. Searching again and again b. Finding solutions to any problem c. Working in a scientific way to d. None -of the above Answer : c. Working in a scientific way to
• Multiple-choice questions are an example of a. OrdinalMeasure b. Nominal Measure c. RatioMeasure d. None of the above Answer : b. Nominal Measure
• Which of the variables cannot be expressed in quantitative terms a. Socio economic status b. Marital status c. Numerical aptitude d. Professional attitude Answer : d. Professional attitude
• The essential qualities of a researcher are : a. Spirit of free enquiry b. Reliance on observation c. Reliance on evidences d. All of the above Answer : d. All the above
• A research process starts with- a. Hypothesis b. Experiment to test hypothesis c. Observation d. None of the above Answer : a. Hypothesis
• Who was the proponent of deductive method- a. FrancisBacon b. Christian Huygenes c. Aristotle d. Isaac Newton Answer : b. Christian Huygenes
• The non-random sampling type that involves selecting a convenience sample from a population with a specific set of characteristics for your research study is called a. Convenience sampling b. Quota sampling c. Purposive sampling d. None of the above Answer : a. Convenience Sampling
• Which of the following is NOT an example of a non-random sampling technique? a. Purposive b. Quota c. Convenience d. Cluster Answer : c. Convenience
• The purpose of drawing sample from a population is known as a. Sampling b. Census c. Survey research d. None of the above Answer : a. Sampling
• Sampling in qualitative research is similar to which type of sampling in quantitative research a. Simple random sampling b. Systematic sampling c. Quotasampling d. Purposive sampling Answer : d. Purposive sampling
• A set of rules that govern overall data communications system is popularly known as……….. a. Protocol b. Agreement c. Pact d. Memorandum Answer : a. Protocol

Essay Questions

•  Basic Research: In this type of research, data is collected to enhance knowledge. The purpose is non-commercial research that is generally not used to invent anything.
•  Applied research: The focus of this research is to analyze and solve real-life problems. It prefers to help solve a practical problem with scientific methods.
•  Problem-Oriented research: It focuses on understanding the nature of the problem to find a relevant solution. The problem could be in various forms; this research analyses the situation.
•  Problem-solving research: Companies usually conduct this type of research to understand and resolve their problems. The research is to find a solution to an existing problem.
•  Qualitative research is a process of inquiry that helps to create an in-depth understanding of problems and issues. It has open ended questions
• State the purpose clearly
• Define the concepts used
• Describe the research procedure in sufficient detail that allows another researcher to make further advancement on the topic
• Design the procedure carefully to achieve desired results
• Data analysis should reveal adequate significance
• Appropriate analysis methods should be used.
• Carefully check the validity and reliability of the data.
• Conclusions should be confined to justify the research data and limit for the which data provides and adequate basis
• Systematic research: Conduct research in structured format with specified steps, rules while keeping in perspective the creative thinking.
•  Research is guided by logical reasoning and process of deduction and induction, which serves as a great value in carrying out research.
•  It is empirical: research is related to one or more than one aspects in real situation that deals with concrete data
•  It is replicable: the characteristics allow researchers to replicate study and building a sound basis for decisions.
• Observing Behaviors of Participants:
• Questionnaire Method
• Interview Method
• Schedules Method
• Information from Correspondents
• Identify the problem
• Review the Literature
• Clarify the Problem
• Clearly Define Terms and Concepts
• Define the Population
• Develop the Instrumentation Plan
• Collect Data
• Analyze the Data

For brochure with pricing details, please mention your contact details below and we will get back to you. Alternatively, you can also call us at  +91-8130872449 . If that number is busy, you can reach us at  91-9160743777  [Hyderabad] or  +91-7411845787  [Bangalore]

Our Branches

My Indian Academy, Vaswani Chambers, 1st floor 264-265, Dr. Annie Besant Rd, Worli, Mumbai, Maharashtra 400051. +91-8976286890

My Indian Academy, 1st floor, D-1/38, near Ramphal Chowk Road, Block D, Sector 7 Dwarka, New Delhi, Delhi 110075. +91-7678657219

My Indian Academy, Fairfield by Marriot, Sankalpa Apartment 2 Block-B Street Number 189, CB Block(Newtown), Action Area I, Newtown, New Town, West Bengal 700156. +91-7679482290

My Indian Academy, 91 Springboard, Gopala Krishna Complex, 45/3, Residency Road, MG Road, Shanthala Nagar, Ashok Nagar, Bangalore – 560025. +91-8548062559

My Indian Academy, HatchStation, 1-8-303, 48/15, PG Road, Begumpet, Hyderabad, Telangana 500003 Phone : +91-9177449218

• Faculty/Subject
• List of Research Guides
• Scholars Login
• Staff Login

Previous Year Question Papers

• Research and Publication Ethics
• Research Methodology -Stream I
• Research Methodology -Stream II
• Research Methodology Stream III
• Theory and Concept -Arabic
• Theory and Concept-Biosciences
• Theory and Concept-Botany
• Theory and Concept-Commerce
• Theory and Concept-Economics
• Theory and Concept-English
• Theory and Concept-History
• Theory and Concept-Home Science
• Theory and Concept-Islamic History
• Theory and Concept-Law
• Theory and Concept-Philosophy
• Theory and Concept-Physics
• Theory and Concept-Political Science
• Theory and Concept-Sociology
• Theory and Concept-Zoology

QUICK LINKS

© 2024 Priyadarsini Hills, Kottayam,Kerala,India, Pin: 686560

How To Write The Methodology Chapter

The what, why & how explained simply (with examples).

By: Jenna Crossley (PhD) | Reviewed By: Dr. Eunice Rautenbach | September 2021 (Updated April 2023)

So, you’ve pinned down your research topic and undertaken a review of the literature – now it’s time to write up the methodology section of your dissertation, thesis or research paper . But what exactly is the methodology chapter all about – and how do you go about writing one? In this post, we’ll unpack the topic, step by step .

Overview: The Methodology Chapter

• The purpose  of the methodology chapter
• Why you need to craft this chapter (really) well
• How to write and structure the chapter
• Methodology chapter example
• Essential takeaways

What (exactly) is the methodology chapter?

The methodology chapter is where you outline the philosophical underpinnings of your research and outline the specific methodological choices you’ve made. The point of the methodology chapter is to tell the reader exactly how you designed your study and, just as importantly, why you did it this way.

Importantly, this chapter should comprehensively describe and justify all the methodological choices you made in your study. For example, the approach you took to your research (i.e., qualitative, quantitative or mixed), who  you collected data from (i.e., your sampling strategy), how you collected your data and, of course, how you analysed it. If that sounds a little intimidating, don’t worry – we’ll explain all these methodological choices in this post .

Why is the methodology chapter important?

The methodology chapter plays two important roles in your dissertation or thesis:

Firstly, it demonstrates your understanding of research theory, which is what earns you marks. A flawed research design or methodology would mean flawed results. So, this chapter is vital as it allows you to show the marker that you know what you’re doing and that your results are credible .

Secondly, the methodology chapter is what helps to make your study replicable. In other words, it allows other researchers to undertake your study using the same methodological approach, and compare their findings to yours. This is very important within academic research, as each study builds on previous studies.

The methodology chapter is also important in that it allows you to identify and discuss any methodological issues or problems you encountered (i.e., research limitations ), and to explain how you mitigated the impacts of these. Every research project has its limitations , so it’s important to acknowledge these openly and highlight your study’s value despite its limitations . Doing so demonstrates your understanding of research design, which will earn you marks. We’ll discuss limitations in a bit more detail later in this post, so stay tuned!

Need a helping hand?

How to write up the methodology chapter

First off, it’s worth noting that the exact structure and contents of the methodology chapter will vary depending on the field of research (e.g., humanities, chemistry or engineering) as well as the university . So, be sure to always check the guidelines provided by your institution for clarity and, if possible, review past dissertations from your university. Here we’re going to discuss a generic structure for a methodology chapter typically found in the sciences.

Before you start writing, it’s always a good idea to draw up a rough outline to guide your writing. Don’t just start writing without knowing what you’ll discuss where. If you do, you’ll likely end up with a disjointed, ill-flowing narrative . You’ll then waste a lot of time rewriting in an attempt to try to stitch all the pieces together. Do yourself a favour and start with the end in mind .

Section 1 – Introduction

As with all chapters in your dissertation or thesis, the methodology chapter should have a brief introduction. In this section, you should remind your readers what the focus of your study is, especially the research aims . As we’ve discussed many times on the blog, your methodology needs to align with your research aims, objectives and research questions. Therefore, it’s useful to frontload this component to remind the reader (and yourself!) what you’re trying to achieve.

In this section, you can also briefly mention how you’ll structure the chapter. This will help orient the reader and provide a bit of a roadmap so that they know what to expect. You don’t need a lot of detail here – just a brief outline will do.

Section 2 – The Methodology

The next section of your chapter is where you’ll present the actual methodology. In this section, you need to detail and justify the key methodological choices you’ve made in a logical, intuitive fashion. Importantly, this is the heart of your methodology chapter, so you need to get specific – don’t hold back on the details here. This is not one of those “less is more” situations.

Let’s take a look at the most common components you’ll likely need to cover. 

Methodological Choice #1 – Research Philosophy

Research philosophy refers to the underlying beliefs (i.e., the worldview) regarding how data about a phenomenon should be gathered , analysed and used . The research philosophy will serve as the core of your study and underpin all of the other research design choices, so it’s critically important that you understand which philosophy you’ll adopt and why you made that choice. If you’re not clear on this, take the time to get clarity before you make any further methodological choices.

While several research philosophies exist, two commonly adopted ones are positivism and interpretivism . These two sit roughly on opposite sides of the research philosophy spectrum.

Positivism states that the researcher can observe reality objectively and that there is only one reality, which exists independently of the observer. As a consequence, it is quite commonly the underlying research philosophy in quantitative studies and is oftentimes the assumed philosophy in the physical sciences.

Contrasted with this, interpretivism , which is often the underlying research philosophy in qualitative studies, assumes that the researcher performs a role in observing the world around them and that reality is unique to each observer . In other words, reality is observed subjectively .

These are just two philosophies (there are many more), but they demonstrate significantly different approaches to research and have a significant impact on all the methodological choices. Therefore, it’s vital that you clearly outline and justify your research philosophy at the beginning of your methodology chapter, as it sets the scene for everything that follows.

Methodological Choice #2 – Research Type

The next thing you would typically discuss in your methodology section is the research type. The starting point for this is to indicate whether the research you conducted is inductive or deductive .

Inductive research takes a bottom-up approach , where the researcher begins with specific observations or data and then draws general conclusions or theories from those observations. Therefore these studies tend to be exploratory in terms of approach.

Conversely , d eductive research takes a top-down approach , where the researcher starts with a theory or hypothesis and then tests it using specific observations or data. Therefore these studies tend to be confirmatory in approach.

Related to this, you’ll need to indicate whether your study adopts a qualitative, quantitative or mixed  approach. As we’ve mentioned, there’s a strong link between this choice and your research philosophy, so make sure that your choices are tightly aligned . When you write this section up, remember to clearly justify your choices, as they form the foundation of your study.

Methodological Choice #3 – Research Strategy

Next, you’ll need to discuss your research strategy (also referred to as a research design ). This methodological choice refers to the broader strategy in terms of how you’ll conduct your research, based on the aims of your study.

Several research strategies exist, including experimental , case studies , ethnography , grounded theory, action research , and phenomenology . Let’s take a look at two of these, experimental and ethnographic, to see how they contrast.

Experimental research makes use of the scientific method , where one group is the control group (in which no variables are manipulated ) and another is the experimental group (in which a specific variable is manipulated). This type of research is undertaken under strict conditions in a controlled, artificial environment (e.g., a laboratory). By having firm control over the environment, experimental research typically allows the researcher to establish causation between variables. Therefore, it can be a good choice if you have research aims that involve identifying causal relationships.

Ethnographic research , on the other hand, involves observing and capturing the experiences and perceptions of participants in their natural environment (for example, at home or in the office). In other words, in an uncontrolled environment.  Naturally, this means that this research strategy would be far less suitable if your research aims involve identifying causation, but it would be very valuable if you’re looking to explore and examine a group culture, for example.

As you can see, the right research strategy will depend largely on your research aims and research questions – in other words, what you’re trying to figure out. Therefore, as with every other methodological choice, it’s essential to justify why you chose the research strategy you did.

Methodological Choice #4 – Time Horizon

The next thing you’ll need to detail in your methodology chapter is the time horizon. There are two options here: cross-sectional and longitudinal . In other words, whether the data for your study were all collected at one point in time (cross-sectional) or at multiple points in time (longitudinal).

The choice you make here depends again on your research aims, objectives and research questions. If, for example, you aim to assess how a specific group of people’s perspectives regarding a topic change over time , you’d likely adopt a longitudinal time horizon.

Another important factor to consider is simply whether you have the time necessary to adopt a longitudinal approach (which could involve collecting data over multiple months or even years). Oftentimes, the time pressures of your degree program will force your hand into adopting a cross-sectional time horizon, so keep this in mind.

Methodological Choice #5 – Sampling Strategy

Next, you’ll need to discuss your sampling strategy . There are two main categories of sampling, probability and non-probability sampling.

Probability sampling involves a random (and therefore representative) selection of participants from a population, whereas non-probability sampling entails selecting participants in a non-random  (and therefore non-representative) manner. For example, selecting participants based on ease of access (this is called a convenience sample).

The right sampling approach depends largely on what you’re trying to achieve in your study. Specifically, whether you trying to develop findings that are generalisable to a population or not. Practicalities and resource constraints also play a large role here, as it can oftentimes be challenging to gain access to a truly random sample. In the video below, we explore some of the most common sampling strategies.

Methodological Choice #6 – Data Collection Method

Next up, you’ll need to explain how you’ll go about collecting the necessary data for your study. Your data collection method (or methods) will depend on the type of data that you plan to collect – in other words, qualitative or quantitative data.

Typically, quantitative research relies on surveys , data generated by lab equipment, analytics software or existing datasets. Qualitative research, on the other hand, often makes use of collection methods such as interviews , focus groups , participant observations, and ethnography.

So, as you can see, there is a tight link between this section and the design choices you outlined in earlier sections. Strong alignment between these sections, as well as your research aims and questions is therefore very important.

Methodological Choice #7 – Data Analysis Methods/Techniques

The final major methodological choice that you need to address is that of analysis techniques . In other words, how you’ll go about analysing your date once you’ve collected it. Here it’s important to be very specific about your analysis methods and/or techniques – don’t leave any room for interpretation. Also, as with all choices in this chapter, you need to justify each choice you make.

What exactly you discuss here will depend largely on the type of study you’re conducting (i.e., qualitative, quantitative, or mixed methods). For qualitative studies, common analysis methods include content analysis , thematic analysis and discourse analysis . In the video below, we explain each of these in plain language.

For quantitative studies, you’ll almost always make use of descriptive statistics , and in many cases, you’ll also use inferential statistical techniques (e.g., correlation and regression analysis). In the video below, we unpack some of the core concepts involved in descriptive and inferential statistics.

In this section of your methodology chapter, it’s also important to discuss how you prepared your data for analysis, and what software you used (if any). For example, quantitative data will often require some initial preparation such as removing duplicates or incomplete responses . Similarly, qualitative data will often require transcription and perhaps even translation. As always, remember to state both what you did and why you did it.

Section 3 – The Methodological Limitations

With the key methodological choices outlined and justified, the next step is to discuss the limitations of your design. No research methodology is perfect – there will always be trade-offs between the “ideal” methodology and what’s practical and viable, given your constraints. Therefore, this section of your methodology chapter is where you’ll discuss the trade-offs you had to make, and why these were justified given the context.

Methodological limitations can vary greatly from study to study, ranging from common issues such as time and budget constraints to issues of sample or selection bias . For example, you may find that you didn’t manage to draw in enough respondents to achieve the desired sample size (and therefore, statistically significant results), or your sample may be skewed heavily towards a certain demographic, thereby negatively impacting representativeness .

In this section, it’s important to be critical of the shortcomings of your study. There’s no use trying to hide them (your marker will be aware of them regardless). By being critical, you’ll demonstrate to your marker that you have a strong understanding of research theory, so don’t be shy here. At the same time, don’t beat your study to death . State the limitations, why these were justified, how you mitigated their impacts to the best degree possible, and how your study still provides value despite these limitations .

Section 4 – Concluding Summary

Finally, it’s time to wrap up the methodology chapter with a brief concluding summary. In this section, you’ll want to concisely summarise what you’ve presented in the chapter. Here, it can be a good idea to use a figure to summarise the key decisions, especially if your university recommends using a specific model (for example, Saunders’ Research Onion ).

Importantly, this section needs to be brief – a paragraph or two maximum (it’s a summary, after all). Also, make sure that when you write up your concluding summary, you include only what you’ve already discussed in your chapter; don’t add any new information.

Methodology Chapter Example

In the video below, we walk you through an example of a high-quality research methodology chapter from a dissertation. We also unpack our free methodology chapter template so that you can see how best to structure your chapter.

Wrapping Up

And there you have it – the methodology chapter in a nutshell. As we’ve mentioned, the exact contents and structure of this chapter can vary between universities , so be sure to check in with your institution before you start writing. If possible, try to find dissertations or theses from former students of your specific degree program – this will give you a strong indication of the expectations and norms when it comes to the methodology chapter (and all the other chapters!).

Also, remember the golden rule of the methodology chapter – justify every choice ! Make sure that you clearly explain the “why” for every “what”, and reference credible methodology textbooks or academic sources to back up your justifications.

If you need a helping hand with your research methodology (or any other component of your research), be sure to check out our private coaching service , where we hold your hand through every step of the research journey. Until next time, good luck!

Psst... there’s more!

This post was based on one of our popular Research Bootcamps . If you're working on a research project, you'll definitely want to check this out ...

You Might Also Like:

52 Comments

highly appreciated.

This was very helpful!

This was helpful

Thanks ,it is a very useful idea.

Thanks ,it is very useful idea.

Thank you so much, this information is very useful.

Thank you very much. I must say the information presented was succinct, coherent and invaluable. It is well put together and easy to comprehend. I have a great guide to create the research methodology for my dissertation.

Highly clear and useful.

I understand a bit on the explanation above. I want to have some coach but I’m still student and don’t have any budget to hire one. A lot of question I want to ask.

Thank you so much. This concluded my day plan. Thank you so much.

Thanks it was helpful

Great information. It would be great though if you could show us practical examples.

Thanks so much for this information. God bless and be with you

Thank you so so much. Indeed it was helpful

This is EXCELLENT!

I was totally confused by other explanations. Thank you so much!.

justdoing my research now , thanks for the guidance.

Thank uuuu! These contents are really valued for me!

This is powerful …I really like it

Highly useful and clear, thank you so much.

Highly appreciated. Good guide

That was helpful. Thanks

This is very useful.Thank you

Very helpful information. Thank you

This is exactly what I was looking for. The explanation is so detailed and easy to comprehend. Well done and thank you.

Great job. You just summarised everything in the easiest and most comprehensible way possible. Thanks a lot.

Thank you very much for the ideas you have given this will really help me a lot. Thank you and God Bless.

Such great effort …….very grateful thank you

Please accept my sincere gratitude. I have to say that the information that was delivered was congruent, concise, and quite helpful. It is clear and straightforward, making it simple to understand. I am in possession of an excellent manual that will assist me in developing the research methods for my dissertation.

Thank you for your great explanation. It really helped me construct my methodology paper.

thank you for simplifieng the methodoly, It was realy helpful

Very helpful!

Thank you for your great explanation.

The explanation I have been looking for. So clear Thank you

Thank you very much .this was more enlightening.

helped me create the in depth and thorough methodology for my dissertation

Thank you for the great explaination.please construct one methodology for me

I appreciate you for the explanation of methodology. Please construct one methodology on the topic: The effects influencing students dropout among schools for my thesis

This helped me complete my methods section of my dissertation with ease. I have managed to write a thorough and concise methodology!

its so good in deed

wow …what an easy to follow presentation. very invaluable content shared. utmost important.

Peace be upon you, I am Dr. Ahmed Khedr, a former part-time professor at Al-Azhar University in Cairo, Egypt. I am currently teaching research methods, and I have been dealing with your esteemed site for several years, and I found that despite my long experience with research methods sites, it is one of the smoothest sites for evaluating the material for students, For this reason, I relied on it a lot in teaching and translated most of what was written into Arabic and published it on my own page on Facebook. Thank you all… Everything I posted on my page is provided with the names of the writers of Grad coach, the title of the article, and the site. My best regards.

A remarkably simple and useful guide, thank you kindly.

I real appriciate your short and remarkable chapter summary

Bravo! Very helpful guide.

Only true experts could provide such helpful, fantastic, and inspiring knowledge about Methodology. Thank you very much! God be with you and us all!

highly appreciate your effort.

This is a very well thought out post. Very informative and a great read.

THANKS SO MUCH FOR SHARING YOUR NICE IDEA

I love you Emma, you are simply amazing with clear explanations with complete information. GradCoach really helped me to do my assignment here in Auckland. Mostly, Emma make it so simple and enjoyable

Thank you very much for this informative and synthesised version.

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

• Print Friendly

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base
• Starting the research process
• Writing Strong Research Questions | Criteria & Examples

Writing Strong Research Questions | Criteria & Examples

Published on October 26, 2022 by Shona McCombes . Revised on November 21, 2023.

A research question pinpoints exactly what you want to find out in your work. A good research question is essential to guide your research paper , dissertation , or thesis .

All research questions should be:

• Focused on a single problem or issue
• Researchable using primary and/or secondary sources
• Feasible to answer within the timeframe and practical constraints
• Specific enough to answer thoroughly
• Complex enough to develop the answer over the space of a paper or thesis
• Relevant to your field of study and/or society more broadly

Table of contents

How to write a research question, what makes a strong research question, using sub-questions to strengthen your main research question, research questions quiz, other interesting articles, frequently asked questions about research questions.

You can follow these steps to develop a strong research question:

• Choose your topic
• Do some preliminary reading about the current state of the field
• Narrow your focus to a specific niche
• Identify the research problem that you will address

The way you frame your question depends on what your research aims to achieve. The table below shows some examples of how you might formulate questions for different purposes.

Using your research problem to develop your research question

Note that while most research questions can be answered with various types of research , the way you frame your question should help determine your choices.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

Research questions anchor your whole project, so it’s important to spend some time refining them. The criteria below can help you evaluate the strength of your research question.

Focused and researchable

Feasible and specific

Complex and arguable

Relevant and original

Chances are that your main research question likely can’t be answered all at once. That’s why sub-questions are important: they allow you to answer your main question in a step-by-step manner.

Good sub-questions should be:

• Less complex than the main question
• Focused only on 1 type of research
• Presented in a logical order

Here are a few examples of descriptive and framing questions:

• Descriptive: According to current government arguments, how should a European bank tax be implemented?
• Descriptive: Which countries have a bank tax/levy on financial transactions?
• Framing: How should a bank tax/levy on financial transactions look at a European level?

Keep in mind that sub-questions are by no means mandatory. They should only be asked if you need the findings to answer your main question. If your main question is simple enough to stand on its own, it’s okay to skip the sub-question part. As a rule of thumb, the more complex your subject, the more sub-questions you’ll need.

Try to limit yourself to 4 or 5 sub-questions, maximum. If you feel you need more than this, it may be indication that your main research question is not sufficiently specific. In this case, it’s is better to revisit your problem statement and try to tighten your main question up.

Prevent plagiarism. Run a free check.

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

Methodology

• Sampling methods
• Simple random sampling
• Stratified sampling
• Cluster sampling
• Likert scales
• Reproducibility

 Statistics

• Null hypothesis
• Statistical power
• Probability distribution
• Effect size
• Poisson distribution

Research bias

• Optimism bias
• Cognitive bias
• Implicit bias
• Hawthorne effect
• Anchoring bias
• Explicit bias

The way you present your research problem in your introduction varies depending on the nature of your research paper . A research paper that presents a sustained argument will usually encapsulate this argument in a thesis statement .

A research paper designed to present the results of empirical research tends to present a research question that it seeks to answer. It may also include a hypothesis —a prediction that will be confirmed or disproved by your research.

As you cannot possibly read every source related to your topic, it’s important to evaluate sources to assess their relevance. Use preliminary evaluation to determine whether a source is worth examining in more depth.

This involves:

• Reading abstracts , prefaces, introductions , and conclusions
• Looking at the table of contents to determine the scope of the work
• Consulting the index for key terms or the names of important scholars

A research hypothesis is your proposed answer to your research question. The research hypothesis usually includes an explanation (“ x affects y because …”).

A statistical hypothesis, on the other hand, is a mathematical statement about a population parameter. Statistical hypotheses always come in pairs: the null and alternative hypotheses . In a well-designed study , the statistical hypotheses correspond logically to the research hypothesis.

Formulating a main research question can be a difficult task. Overall, your question should contribute to solving the problem that you have defined in your problem statement .

However, it should also fulfill criteria in three main areas:

• Researchability
• Feasibility and specificity
• Relevance and originality

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, November 21). Writing Strong Research Questions | Criteria & Examples. Scribbr. Retrieved July 1, 2024, from https://www.scribbr.com/research-process/research-questions/

Is this article helpful?

Shona McCombes

Other students also liked, how to define a research problem | ideas & examples, how to write a problem statement | guide & examples, 10 research question examples to guide your research project, "i thought ai proofreading was useless but..".

I've been using Scribbr for years now and I know it's a service that won't disappoint. It does a good job spotting mistakes”

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

Enter the email address you signed up with and we'll email you a reset link.

• We're Hiring!
• Help Center

Final Exam Review for Research Methodology (RES301)

Research Methodology final exam review

Related Papers

JOHN KARANJA, PhD , Dr. John Karanja

research proposal is a comprehensive plan for a research project. It is a written description of a research plan that has to be undertaken. It determines the specific areas of research, states the purpose, scope, methodology, overall organization and limitations of the study. It also estimates its requirements for equipment (if necessary), finance and possible personnel.

pinky marie mendi gallano

Rebekka Tunombili

Rengganis Ernia

Dr RUKUNDO Levi, PhD

Dewi Nurbaeti Widianingsih

Nidhi Mishra

Fundamental_of_Research_Methods and Statisti

Hadi Pranoto

Loading Preview

Sorry, preview is currently unavailable. You can download the paper by clicking the button above.

RELATED TOPICS

•   We're Hiring!
•   Help Center
• Find new research papers in:
• Health Sciences
• Earth Sciences
• Cognitive Science
• Mathematics
• Computer Science
• Academia ©2024

By Gkseries see more questions

Research methodology mcqs | research methodology short questions & answers.

• General Knowledge /
• research-methodology-multiple-choice-questions-and-answers

Free download in PDF Research Methodology Multiple Choice Questions(MCQs) & Answers. These multiple choice questions on Research Methodology are very useful for PhD entrance exam.

View Answer

Answer: Helps those interested in further research and studying the problem from another angle

Answer: It depends on your point of view

DOWNLOAD CURRENT AFFAIRS PDF FROM APP

• 10 Important Books for UPSC Preparation
• 10 Important Books For SSC CGL, CPO, CHSL, and GD CONSTABLE Exam Preparation

Answer: Pr absorbs red light and becomes Pfr

Answer: Involves testing an explicitly defined hypothesis

Answer: A hypothesis

Answer: Sampling people, newspapers, television programmes etc.

Answer: Introduction; Literature review; Research methods; Results; Discussion; Conclusion

Answer: Research questions

Answer: Experimental studies

Answer: Laboratory and field experiments

Answer: Reference collection

Answer: Emphasis upon the control of the immediate situation

Answer: Longitudinal

Answer: Cross-sectional studies

Answer: Variables

Answer: All of these

Answer: General terms

Answer: Inadequate sample

Answer: Fact finding (historical) studies

Answer: Booth

ADRE 2.0 Free Mock Tests

Adre 2.0 full length mock test, take mock tests.

Random GK Questions

Advertisement.

65 Research Methodology Question Paper PDF Download Free

In case you are preparing for your final year dissertation, you might be interested in downloading a research methodology question paper PDF.

This article will discuss where to find such a question bank. We have given the direct links to download the question papers .

The links in this article will guide you through the steps needed to download and use the question bank. Moreover, it will help you understand what exactly goes into creating a good research methodology question paper. Here are some tips that will help you succeed in this course.

Interesting

• How to Complete Syllabus Fast Easily
• Top 20 Ways To Earn Money As A Student In India
• 2022s Best Diet for Students Preparing for Competitive Exams
• Best 11 ways How to Improve Handwriting Easily
• Effective Guide to Prepare for Exams in a Week

Research Methodology Question Paper PDF

The Research methodology MCQ questions are designed to test the knowledge of the students in their field. They are applicable to competitive and academic MBA, regular, and distance-learning courses. It is important to choose the research question carefully, as it determines the purpose of your study.

The Research Methodology Question Paper questions may be related to a gap in knowledge, analyzing academic assumptions, monitoring a practice development, testing theories, or comparing methods. Ensure that the research question is appropriate for the discipline and institution and is based on a problem-solving framework.

Research Methodology Question Bank pdf

The M.Com Accountancy (IDOL) Research Methodology question paper is available in pdf format for download. It is a good practice to download previous year’s question papers from the Internet as this will help you to learn the techniques used in solving them.

Also, you can look up the solutions of the sample papers to know how to do better in your own research. However, be sure to select the right question paper for your course.

The first step in any research is the formulation of a hypothesis. A hypothesis is a statement regarding a particular population’s size or behaviour. This is followed by research designs. Samples are the group of individuals chosen for a study.

The data collected is analyzed using various techniques. They may be observational or experimental. Regardless of the technique used, the study must answer two research questions. The data collection is based on several methods, including observational research, experimentation, simulation, and derived methodology.

Research Methodology Question Bank with Answers pdf

The first thing you need to do is figure out the research methodology. The research methodology includes the methods and procedures for conducting research. These Research Methodology Question Paper may include physical surveys, questionnaires, interviews, or participant observations. Once you have decided what method to use, you can begin preparing for the exam. Here are some things you need to know about this type of exam.

This article will help you decide what method you want to use. You can also use the research methodology question bank to study various subject aspects.

The objective of the Research Methodology Question Papers is to discover a frequency or an association. It involves collecting data and conducting research using a systematic, logical approach. It involves data analysis, interpretation, and formation of principles.

With the help of Research Methodology Question Papers, you will also need to figure out if the research problem is true or false and whether it’s valid. The process includes steps and criteria for valid research. You will need to know how to gather data, evaluate results, and evaluate findings correctly.

Assignment Questions

• Define and explain the term “Research”. Explain its importance and objectives.
• Define the main issues which should receive the attention of the researcher in formulating the research problem. Give suitable examples.
• Explain in brief the stages in the data processing.
• Briefly explain the significance of data processing. What is the problem associated with data processing?

Firstly, you need to determine whether you’re conducting an ex post facto study. This type of research design is used to investigate the effects of a situation after it happens.

In this case, the researcher can use data not relevant to the problem understudies, such as cash register receipts or customer grievance data. Alternatively, you can use a sample from a national survey.

This type of research is very useful for various situations, such as market research, because it allows you to compare different factors, such as cost, quality, and consistency.

Research Methodology Question Paper in English

Research Methodology Question Paper in Hindi

Ultimately, it would be best if you defined your research question. You need to know your objectives and goals. Once you’ve defined your research question, you need to define your variables. T

hese will result in your final result. By defining your variables and identifying the variables in a study, you will know what to focus on for the final analysis. It’s a must-have tool for any researcher who needs to conduct research. You may take the help of Research Methodology Question Papers.

Experimentation is another method you’ll need to know. In this method, you can establish a causal relationship among variables. The method is also known as experimental research. You can also perform qualitative research, which is mostly exploratory and relies on subjective data. By collecting data, you can develop testable hypotheses and make predictions about the outcome. These results then prove the hypotheses. This method can be used for any research, whether you’re experimenting with science or social work.

A problem is usually defined. Once you’ve defined the problem and chosen the methodology, you can begin collecting data. Research Methodology Question Paper will help you develop the problem, gather data, and test hypotheses. You’ll also need to write a study plan. After that, you’ll need to write the hypothesis, which outlines the methodology you’ll use for your research. If you’re unsure, refer to a research methodology question bank with answers pdf.

PhD Entrance Exam Question Paper For Research Methodology

The PhD entrance exam for research methodology focuses on the principles of a research project. It includes questions on the following: the characteristics of a good researcher, the types of research, case study research, and statistical applications. It also covers the types of research reports, their structure, and their reasoning. The exam will also cover current affairs. Here are some tips to help you ace the research methodology section of the entrance exam with the help of Research Methodology Question Papers.

Answer any TWO of the following

(2 x20 = 40 Marks)

• Write an essay on the preparation of  a manuscript for publications
• Describe various methods of collection of insects in the field.
• Describe any two types of Taxonomic keys with examples.
• Give an account of the types of preservation of insects.

Attempt Research Methodology Question Paper:  Attempt a PHD entrance exam in research methodology to familiarize yourself with the topics. By doing this, you will have an idea of the types of questions you will be asked during the examination.

This is also a good time to get advice from senior students. Ask them for help and clarification on any question that they feel is tricky. In this way, you can maximize your chances of acing the PhD entrance exam.

PhD Research Methodology Old Question Papers

Are you planning to write your PhD in Research Methodology? Are you wondering how to prepare for the PhD research methodology question paper? You can learn how to prepare from the below tips.

The research methodology syllabus consists of several topics that cover the qualities of a good researcher. Other topics in this section are the various types of research and their merits. The topics also cover logic and reasoning, the structure of a research report, and its components.

Class Test Questions

• What are ethical issues concerning the research participants of the research activity? Explain.
• What are the characteristics of research? Explain how quantitative researches differ from qualitative research.
• Explain the significance of primary data. What are the limitations of primary data? Explain in brief the stages in the data processing.
• Discuss the various methods of research? Explain the various factors to be considered in making the decision on the sampling method.

1. PhD Research Methodology Old Question Paper 2020

In the PhD research methodology question paper 2020, students must explain the main reasons behind their chosen data collection and analysis method.

They must also highlight the practical limitations of the method used. This information should be limited to the reasons for the choice of the methodology, which should not include irrelevant details.

For instance, basic procedures should only be explained when the research subject is unfamiliar with them. This will help them score a high mark in the research methodology section.

2. PhD Research Methodology Old Question Paper 2019

A key feature of a PHD research methodology question paper is that it should be based on an academic question that is of interest to researchers and practitioners in the subject. It should be derived from the literature, current situation, or practice of the subject.

Research Methodology Question Papersuld has a clearly stated purpose and can be anything from filling a knowledge gap to analyzing academic assumptions and monitoring developments in practice. It can also be about testing theories within a certain population. The question should also be appropriate for the discipline, institution, and community.

SECTION – A  

(10 x 2 = 20 Marks)

Answer ALL questions

• What are the advantages of Websites in literature collections?
• What is Curating?
• Distinguish conceptual from empirical research.
• What are Tautonyms?
• Define international code of Zoological Nomenclature.
• Define the term – Null hypothesis.
• Distinguish bar diagram from a histogram.
• What is a holotype?
• Mention any two objectives of the Research.
• Differentiate acclimation from acclimatisation

UGC NET Research Methodology Old Question Paper

In the recent UGC NET exam, questions were asked about the research methods. Research Methodology Question Papers covered critical features of research methods, the definition of the research question, hypotheses, data collection, analysis, and reporting procedures.

Students should familiarize themselves with these topics and the questions that will be asked during the examination. If you are preparing for the UGC NET exam, you should prepare by referring to UGC NET mock tests.

In addition to studying the UGC NET syllabus, it is also important to learn the different referencing styles. The UGC NET Research methodology question paper includes study notes on the various accepted referencing styles. The study notes also feature a list of helpful ICT tools to aid you in your preparation. These include tools such as Mendeley and IBM SPSS. Also, the notes include information about the application of ICT to research.

SECTION – B

(4X 10  = 40 Marks)

Answer any four of the following

• Explain briefly the types of sampling.
• What is Berlese Funnel?  Explain its use in the collection of arthropods.
• Write an account of types in insect taxonomy.
• Summarize statistical techniques used in experiments.
• Explain the impact of physicochemical parameters in stress experiments.
• Bring out the significance of current contents and review volumes in research.

MBA Research Methodology Old Question Paper

The MBA research methodology question paper is for the first year of the course and has a simple pattern. Each question carries the same number of marks and includes long answer types. The questions in the question paper cover different aspects of the research process, such as the types of experimental designs, the scales used for measuring variables, and the different factor analyses.

The research methodology question paper also includes the format and layout of a research report. To ensure a good score, candidates are advised to read previous year’s papers to familiarize themselves with the format and content of the exam.

Students should understand the importance of selecting the appropriate statistical technique for the analysis. They must be able to distinguish between non-parametric statistical techniques and parametric statistics. They must also be able to define the research report, thesis, and the significance of the research work.

The question paper also requires the students to understand the format and pattern of the Business Research methodology question paper. The syllabus of the course is available for download in pdf format.

Research Methodology Old Question Paper MCQs

Students preparing for MBA exams should familiarize themselves with Research methodology MCQ questions. Research Methodology Question Papers may be asked in the regular or distance mode. They can be asked about academic, competitive, and managerial research topics. Identifying the problem is the first step in designing a research study.

This process will help you develop a hypothesis and plan of investigation. Then you will have to reduce the problem to a more manageable one.

Moreover, you should understand that Secondary data does not necessarily have to be relevant to the problem under study. For example, you can use cash register receipts or customer grievances data.

Secondary data may also be obtained from national surveys, such as the Statistical Abstracts of India. The purpose of secondary data is to inform decision-makers and make them aware of the importance of their findings. The next step is to analyze the data.

• अनुसंधान पद्धति प्रश्न पत्र 2014
• अनुसंधान पद्धति प्रश्न पत्र 2015
• अनुसंधान पद्धति प्रश्न पत्र 2016
• अनुसंधान पद्धति प्रश्न पत्र 2017
• अनुसंधान पद्धति प्रश्न पत्र 2018
• अनुसं धा न पद्धति प्रश्न पत्र 2019
• अनुसंधान पद्धति प्रश्न पत्र 2020

How to Answer Research Methodology MCQs

This article will teach you how to answer the research methodology MCQs on the UGC NET and MBA research methodology question papers. Hopefully, this will make the task a little easier. But, first, you should be familiar with how the research methodology question paper is formatted.

Below are some questions to keep in mind. When possible, make sure to use an example question paper to illustrate the different methods. If possible, you should also record your research methods. This will ensure that you capture all the small details.

1. “One of the methods of logical reasoning process” is called

   a) Induction   b) Deduction   c) Research d) Experiment

2. “A systematic step-by-step Procedure following the logical process of reasoning” called

   a) Experiment b) Observation c) Deduction d) Scientific method

3. An essential Criterion of Scientific study is called

   a) Belief   b) Value    c) Objectivity d) Subjective

4. “Reasoning from general to particular “is called

   a) Induction   b) deduction c) Observation d) experience

5“Deduction and induction are a part of the system of reasoning” – stated by

a) Caroline   b) P.V.Young  c) Dewey John d) Emory

6 ……….. is a “systematically conceptual structure of interrelated elements in some schematic form”

a) Concept   b) Variable c) Model   d) Facts

7 The method by which a sample is chosen

a) Unit   b) design   c) Random   d) Census

8 Research conducted to find a solution for an immediate problem is ………….

a) Fundamental Research              b) Analytical Research

c) Survey.                                          d) Action Research

9 Fundamental Research is otherwise called

a) Action Research   b) Survey   c) Pilot study   d) Pure Research

10 A research which follows the case study method is called

a) Clinical or diagnostic    b) Causal

c) Analytical                        d) Qualitative

11 Research conducted in a classroom atmosphere is called

a) Field study                        b) Survey

c) Laboratory Research      d) Empirical Research

12. Research through experiment and observation is called

   a) Clinical Research             b) Experimental Research

   c) Laboratory Research       d) Empirical Research

13 ………….. is a way to systematically solve the research problem

About The Author

Question paper guru.

Guiding PhD Students and Connecting PhD Talent with Industry. Your one-stop shop for everything PhD-related.

Business Directory

Access phd entrance exam model question paper for effective preparation.

Access and download sample question papers for PhD-related exams to familiarize yourself with the format, content, and level of difficulty, helping you prepare effectively for your upcoming examinations. We provide entrance exam sample papers for individual subjects as wel as university-specific. To keep them relevant for you, we also include questions based on the PhD previous year question paper.

Search Required PhD Entrance Sample Question Paper Here

Research Paper Name

Search Your PhD Learning Requirement on Our Portal

Search Your Job on our Portal

Sample question paper.

VTU Research Methodology and IPR Question Papers 1st sem SSE 2020 scheme

20RMI17 Question Paper

VTU Research Methodology and IPR FEB 2021 Question Paper

Research methodology and ipr question papers, last updated : monday january 23, 2023.

• VTU last year question papers
• VTU E Learning
• VTU Time Table New
• Model Question Papers New
• VTU Syllabus New
• VTU Results
• VTU Question Bank
• VTU Revaluation Results
• SGPA to Percentage

VTU Updates

• VTU NON-CBCS Results New
• SSP Scholarship 2023 New
• Cloud Computing vtu question papers New
• Machine Learning Syllabus New
• 18CS71-AiML VTU Question Papers New
• Machine Learning VTU Question Papers New
• Web Technology Syllabus New
• VTU change of college Procedure New
• VTU MTech Syllabus New
• VTU MBA Results New
• VTU Notes New
• VTU PhD TimeTable New
• VTU Academic Calendar 2023 Odd Sem
• VTU Updates New
• Infosys Recruitment 2022 New
• Cyber Security Syllabus New
• MBA in USA for Engineering Students New

• Apply to UMaine

Human health, tourism, and nature

Studying the Relationship Between Tourism and Mosquito-Borne Diseases

 Interviewing Elizabeth Pellecer Rivera, PhD candidate at the University of Maine

[ Español ]

1. Can you tell us a bit about your personal background and how you started your PhD at the University of Maine?

Prior to starting my PhD, I studied sustainable tourism and development studies. Most of my professional experience focused on applied social science research with the Center for Health Studies, at the Universidad del Valle de Guatemala. I worked on researching vector-borne diseases from ecological, biological, and social approaches. This experience made me realize how complex and multifactorial problems are, and how interdisciplinary research can help inform and seek potential solutions. I had the privilege of working on a project related to the Zika virus during its initial emergence in the Americas.

When I learned about the opportunity to work on an interdisciplinary research project about the relationship between tourism and mosquito-borne diseases at the University of Maine, I was thrilled because it touched on many aspects of my personal and academic background. Hopefully this research project will have a positive impact in the public health and tourism sectors.

2. What is your project about and why is it important?

The project aims to provide insight into the dynamics between tourism and emerging diseases, as the world is more interconnected than ever, and tourism and travel represent a large part of human mobility. This project aims to better understand how travel can serve as a driver to spread diseases, how humans perceive the risk of emerging diseases, and what kind of actions and responses they take when facing this kind of risk. Originally the project focused on two mosquito-borne diseases, Zika and chikungunya, but given the extent of impact that the COVID-19 pandemic had, we have included it in the research.

The trends show that international tourism was steadily growing before 2020, and it is expected that outbreaks of emerging diseases will continue occurring. Understanding past experiences can only allow us to better prepare for future events. 

3. What questions did you set out to answer?

My central research questions are:

• What preventative and travel behaviors are being suggested in newsprint media targeting visitors?
• What business strategies, if any, have stakeholders used to respond to the threat that Zika and/or chikungunya, and COVID-19 pose to their business, and their clients?
• How do travelers perceive the potential risk of exposure to Zika, chikungunya or COVID-19 when traveling? What factors influence their risk perception?

With all the data we have gathered, most of our questions are being answered, and new questions have emerged. The challenge is to bring it all together and transform it into useful information for both academics and practitioners.

4. Could you share insights into your research methodology?

The social science part of the research focuses on doing a case study in Guatemala to understand the topic, from diverse perspectives: (1) what is being communicated in the news; (2) what are the travelers’ risk perceptions towards the diseases and what kind of travel-related and preventive actions they tend to adopt; and (3) how the tourism industry has been impacted and what coping and response strategies have they implemented. I am using mixed methods (quantitative and qualitative), including content analysis, surveys, and phenomenological interviews. 

5. What have you discovered so far?

Although my analysis is ongoing, my research has already revealed some interesting findings.We found that news media don’t necessarily balance the messages being communicated. More news included risk-elevating messages, focusing on the “scary” side of the diseases, but did not include as many messages that inform people what to do and how to prevent the diseases. From the tourism stakeholders’ perspective, we have identified some of the main strategies used to cope with mosquito-borne diseases and COVID-19, as well as some of the enabling and limiting factors to overcome the impact of these health-related risks.

6. What were some key challenges and highlights that you have encountered while conducting research?

One of the major challenges has been doing a PhD during the COVID-19 pandemic. The impact of this situation was twofold — it not only induced stress and isolation but also disrupted our initial research plans. We had intended to conduct international research, but the imposition of travel restrictions made it unfeasible. Much of my research had to be conducted virtually.

My favorite part of doing research is interacting with people and being able to listen and learn from their experiences. Therefore, I really enjoyed conducting qualitative interviews, and hearing from the tourism sector about their resilience and capacity to adapt when facing threats like mosquito-borne diseases and COVID-19, especially as the latter impacted the sector as never before. I found it surprising that some individuals pointed out positive outcomes resulting from the pandemic, such as the catalyzation of local organization among stakeholders at destinations and an enhanced appreciation for nature and green spaces among visitors. In conclusion, I was amazed by the diversity of responses and the high degree of optimism and hope.

7. How do you envision the practical applications or real-world impact of your research?

I think the COVID-19 pandemic opened everyone’s eyes to the extent of impact and spread of an emerging disease, both as a public health issue and because it halted human mobility. An integral practical application of my project is to promote intersectoral and interinstitutional relationships, public-private partnerships, academia-practitioner collaborations, and tourism-public health interactions. Overall, my investigation can improve understanding of the intricate relationship between tourism and disease.  

To learn more about my work, read my recent publication :

Elizabeth Pellecer Rivera, Sandra De Urioste-Stone, Laura N. Rickard, Andrea Caprara & Lorena N. Estrada (2024) Tourists and epidemics: how news media cover the risks of Zika virus and chikungunya outbreaks in the Americas, Current Issues in Tourism , DOI: 10.1080/13683500.2024.2309164

• Researchers
• Publications

• Open access
• Published: 25 June 2024

GPT-4 performance on querying scientific publications: reproducibility, accuracy, and impact of an instruction sheet

• Kaiming Tao 1 ,
• Zachary A. Osman 1 ,
• Philip L. Tzou 1 ,
• Soo-Yon Rhee 1 ,
• Vineet Ahluwalia 2 &
• Robert W. Shafer 1  

BMC Medical Research Methodology volume  24 , Article number:  139 ( 2024 ) Cite this article

210 Accesses

1 Altmetric

Metrics details

Large language models (LLMs) that can efficiently screen and identify studies meeting specific criteria would streamline literature reviews. Additionally, those capable of extracting data from publications would enhance knowledge discovery by reducing the burden on human reviewers.

We created an automated pipeline utilizing OpenAI GPT-4 32 K API version “2023–05-15” to evaluate the accuracy of the LLM GPT-4 responses to queries about published papers on HIV drug resistance (HIVDR) with and without an instruction sheet. The instruction sheet contained specialized knowledge designed to assist a person trying to answer questions about an HIVDR paper. We designed 60 questions pertaining to HIVDR and created markdown versions of 60 published HIVDR papers in PubMed. We presented the 60 papers to GPT-4 in four configurations: (1) all 60 questions simultaneously; (2) all 60 questions simultaneously with the instruction sheet; (3) each of the 60 questions individually; and (4) each of the 60 questions individually with the instruction sheet.

GPT-4 achieved a mean accuracy of 86.9% – 24.0% higher than when the answers to papers were permuted. The overall recall and precision were 72.5% and 87.4%, respectively. The standard deviation of three replicates for the 60 questions ranged from 0 to 5.3% with a median of 1.2%. The instruction sheet did not significantly increase GPT-4’s accuracy, recall, or precision. GPT-4 was more likely to provide false positive answers when the 60 questions were submitted individually compared to when they were submitted together.

Conclusions

GPT-4 reproducibly answered 3600 questions about 60 papers on HIVDR with moderately high accuracy, recall, and precision. The instruction sheet's failure to improve these metrics suggests that more sophisticated approaches are necessary. Either enhanced prompt engineering or finetuning an open-source model could further improve an LLM's ability to answer questions about highly specialized HIVDR papers.

Peer Review reports

The systematic review of data from multiple research studies is often required to answer many of the most significant biomedical questions. However, the literature searches required for a systematic review often suffer from low sensitivity (recall) and specificity (precision) in part as a result of the limitations of current search tools which rely on the Medical Subject Headings (MeSH) key words, the National Library of Medicine’s controlled vocabulary used for indexing articles [ 1 ]. Extracting data from relevant studies also requires painstaking review by highly trained human reviewers.

The use of automated software tools to assist in reviewing research papers has become a topic of increasing interest. Most such tools have used natural language processing (NLP) and machine learning (ML) algorithms primarily to screen the titles and abstracts of publications to determine whether they meet the search criteria for a systematic review [ 2 , 3 , 4 , 5 , 6 , 7 , 8 ]. Several studies have also described the potential for using the representational language model Bidirectional Encoder Representations from Transformers (BERT) and the Generative Pre-trained Transformer (GPT) large language models (LLMs) for reviewing the full text of published studies [ 9 , 10 , 11 , 12 ]. LLMs have also been evaluated for their ability to summarize research studies [ 13 , 14 ].

We have extensive experience reviewing published studies on the topic of human immunodeficiency virus (HIV) drug resistance having maintained the Stanford HIV Drug Resistance Database (HIVDB; https://hivdb.stanford.edu ) and performed multiple systematic literature reviews [ 15 , 16 ]. In this study, we evaluated the ability of GPT-4 to correctly answer questions about publications on HIV drug resistance with and without an instruction sheet designed to provide GPT-4 with specialized HIV drug resistance knowledge. We evaluated publications considered for inclusion in a curated database. This database primarily links mutations in the genetic targets of HIV therapy to the antiviral treatments of the persons from whom the sequences were obtained and to the impact of these mutations on the i n vitro susceptibility to individual HIV drugs.

HIV drug resistance questions

We designed 60 questions pertaining to HIV drug resistance reflecting the type of information typically extracted from published papers evaluated for possible addition to HIVDB. Most of the questions dealt with linking HIV genetic sequence data to two other forms of data: (1) the antiviral treatments received by the patients from whom the sequenced viruses were obtained and (2) the effect of mutations in these viruses on their susceptibility to antiviral drugs. The questions were of three types: Boolean, requiring yes or no answers; numerical, where the correct response was an integer; and list-based, where a series of items constituted the correct answer. The complete list of questions can be found in Supplementary File 1.

Published papers

We selected 60 published papers on HIV drug resistance identified in recent PubMed searches and in recent GenBank database submissions including 19 published after September 2021, the cut-off date for the dataset used to train the GPT-4 model that was used. Nearly two-thirds of the papers reported HIV genotypic resistance data (e.g., genetic sequence data or lists of HIV drug-resistance mutations). Nearly one-half reported that their sequences had been submitted to GenBank, the standard public repository for sequence data, and provided GenBank accession numbers. The selected papers often reported the antiviral treatment histories of patients undergoing virus sequencing, the samples submitted for sequencing, the technology used for sequencing, and the results of in vitro susceptibility testing. Two authors reviewed each paper to determine the answers to the 60 questions. A third author designated the correct answer when there was a disagreement between the first two authors. The complete list of papers can be found in Supplementary File 2.

Instruction sheet

The instruction sheet contained 2002 words that provided background knowledge about HIV drug resistance and the type of information that a human curator would need to know to identify the relevant data for inclusion in HIVDB (Supplementary File 3). This document encapsulated fundamental antiviral therapy and HIV drug resistance concepts, alongside a description of frequently used terms and abbreviations within the field. The instruction sheet was not designed to be a comprehensive treatise on antiviral therapy and HIV drug resistance but rather to offer practical guidance to human curators with some background HIV knowledge. The instruction sheet contained information considered useful to answering many of the 60 questions. However, it was not designed specifically to answer each of the questions developed for this study.

Automated query pipeline

We designed an automated pipeline utilizing OpenAI GPT-4 32K API version “2023–05-15” (Microsoft Azure, accessed Sep 15th, 2023) (Fig. 1 ). A Python script was used to transform a published paper to a markdown format containing the text of the study methods, study results, tabular data, and figure legends. The abstract, introduction, discussion, and references were excluded from the markdown version of the paper. The median number of tokens in each markdown paper was 5338 (range:1282 to 13,861). On average, one token is about 0.75 words or about four English language characters. We chose to not submit the introduction, discussion, and references to GPT-4 because these parts of a paper often refer to the work of other studies.

Automated query pipeline work flow. The first step involved developing 60 questions relevant to HIV drug resistance, identifying 60 published papers, and developing an approximately 2000 word instruction sheet with HIV drug resistance information. Each paper was reviewed by two human reviewers and a markdown version of each paper’s full text was created. The second step involved querying GPT-4: building a prompt that included (1) the marked down version of each paper, (2) all 60 questions, and (3) the instruction sheet. The third step evaluated the GPT-4 answers to assess whether they were the same as the answers determined by the human curators. Three sample questions are shown including one for which the correct answer was Yes or No, another for which the correct answer was a list of items, and a third for which the correct answer was a number

Each GPT-4 query consisted of one markdown paper plus one of the following: (1) all 60 questions presented simultaneously; (2) all 60 questions presented simultaneously with the instruction sheet; (3) each of the 60 questions presented individually; and (4) each of the 60 questions presented individually with the instruction sheet.

We refer to the process of submitting the 60 questions simultaneously as the multiple-question mode and the process of submitting each question individually as the single-question mode. We refer to the process of presenting all questions without the instruction sheet as the base model. The single-question mode necessitated repeatedly submitting the same markdown paper with each question. It was therefore much more time consuming and expensive than the multiple-question mode.

If GPT-4 failed to answer all 60 questions for a paper or if a time-out error occurred when questions were presented in the multiple-question mode, the unanswered questions were resubmitted along with the paper.

Supplementary File 4 provides an example GPT-4 prompt. Supplementary File 5 provides the Python code used to generate the GPT-4 prompts.

Automated response evaluation pipeline

We evaluated the accuracy of GPT-4 responses using the following approach: (1) for Boolean questions, a script was used to determine if the response began with “yes” or “no”; (2) for numerical questions, a script was used to determine if the response contained a single number; (3) all other responses were evaluated manually. Accuracy was defined as concordance between the correct answer and the GPT-4 response for Boolean and numerical questions. For list questions, we considered the GPT-4 response to be accurate if it identified at least one element of the correct list. The response was considered inaccurate if it did not identify any element of the correct list or if it identified elements that were not part of the correct list. A manual review of half of the responses to the Boolean and numerical questions confirmed that the script used to determine whether the response began with “yes” or “no” or contained a single number accurately gauged GPT-4’s answers to these questions.

Experimental design and analyses

To evaluate the performance of GPT-4 in answering questions about a paper, we designed a series of experiments: (1) We assessed the reproducibility of the base model in the multiple-question mode by performing each query in triplicate. (2) We calculated the recall, precision, and F1 score – the harmonic mean of precision and recall, calculated as 2 x (recall * precision) / (recall + precision) – for the base model in multiple-question mode. This analysis was performed on the median of the triplicate results and it was performed separately for results obtained with and without the instruction sheet. (3) We compared the accuracy – measured as the proportion of correct answers – of the base model in the multiple-question mode to its performance when the responses were from randomly permuted papers. In essence, we assessed the accuracy of GPT-4's responses to the submitted paper compared with its accuracy when the answers were drawn from ten randomly selected papers, distinct from the actual paper. (4) We compared the accuracy of the base model in the multiple-question mode to the accuracy with the instruction sheet in the multiple-question mode. (5) Finally, we compared the accuracy of the base model in the multiple-question mode with the accuracy of the base model in the single-question mode, also in triplicate.

Figure 2 displays triplicate determinations of the accuracy of GPT-4 on each of the 60 questions applied to each of the 60 papers in the multiple-question mode without the instruction sheet (i.e., base model). The median accuracy for the 60 questions over the three replicates was 91.8% (range: 50.7%-100%). The mean accuracy for the 60 questions over the three replicates was 86.9%. The mean accuracies were similar for Boolean (86.6%), numerical (84.7%), and list (90.2%) questions. The standard deviation (SD) of three replicates for the 60 questions ranged from 0 to 5.3% with a median SD of 1.2% across all questions. The coefficient of variation (CV) of three replicates for the 60 questions ranged from 0 to 0.068 with a median CV of 0.012. The maximum difference between any two of the three replicates was 6 for one question, 4 for two questions, and 3 for three questions.

Triplicate determinations of the accuracy of each of the 60 questions applied to each of the 60 papers in the multiple question mode (i.e., all 60 questions presented simultaneously) without the instruction sheet (i.e., base model). The Y-axis indicates the percentage of times in which the GPT4 response was accurate across the 60 papers. The X-axis shows the question ID in descending order of median accuracy. The three bars shown for each question ID indicate separate replicates. The median accuracy for the 60 questions over the three replicates was 91.8% (range: 50.7%-100%). The mean accuracy across all questions and all papers were 86.8%, 86.9%, and 87.1%. Different colors mean different replicates

Figure 3 compares the results of one of the three replicates for the base model in multiple-question mode with the results obtained when the answers to the 60 papers were permuted. The mean accuracy for 10 permutations of the papers was 62.9%. Therefore, the increased accuracy of GPT-4 on the actual papers was 24.0% higher than expected by chance on the permuted set of papers (95% CI: 18.6%-29.4%; p  < 0.000001; paired Student’s t-test). The surprisingly high level of accuracy for permuted answers is explained by the uniformity of responses across many papers. Specifically, for Boolean questions, the answers were not infrequently always ‘yes’ or ‘no.’ Similarly, for numerical questions, the answer was often 0, and for list questions, the answer was often an empty list. Figure 3 demonstrates this in 10 questions where ≥ 90% of the Boolean answers were either ‘yes’ or ‘no’, in two numerical questions where the answer was usually 0, and in two list questions where the answer was usually an empty list.

Figure 4 shows the precision, recall, and F1 score with and without the instruction sheet separately for the Boolean, numerical, and list questions. Without the instruction sheet, GPT-4 demonstrated a recall of 68.1% and a precision of 84.6% on the 2280 Boolean questions (i.e., 48 questions × 60 papers); a recall of 61.6% and a precision of 88.1% on the 660 numerical questions (i.e., 11 questions × 60 papers); and a recall of 88.6% and a precision of 91.9% on the 660 list questions (i.e., 11 questions × 60 papers). Of the 296 true positive answers for list questions, 273 (92.2%) were identical to the manual answers whereas 23 (7.8%) contained a subset of the manual answers.

Recall and precision of GPT-4 at answering questions about HIV drug resistance papers: comparison with manual curators. The manual result (obtained by two human curators and a third to break ties) was considered to be the correct answer. Each entry in the six sections containing raw data represents the median of 3 repeats. *Includes questions for which GPT reported a number > 0 when the correct answer was 0 and questions for which GPT reported an incorrect number (i.e., one that differed from the manual review). **The results were considered to be false positives when GPT-4 identified items not identified by manual review. Additionally, 12 of 16 answers obtained without the instruction sheet and 14 of 18 answers obtained with the instruction sheet were considered to be false negatives because GPT-4 also failed to identify any of the items that were identified by manual review. Abbreviations: GPT (GPT-4), TP (true positive), TN (true negative), FP (false positive), FN (false negative), F1 score = 2 * (Recall * Precision) / (Recall + Precision)

Figure 5 displays the triplicate determinations of the accuracy of GPT-4 in the multiple-question mode with and without the instruction sheet. Across the 60 questions, the mean net change in accuracy was + 1.2% resulting in an overall accuracy of 88.1% across all questions and papers with the instruction sheet. On average, across the three replicates, the instruction sheet improved the accuracy of 3.2% ( n  = 114) of questions that were initially answered incorrectly. Conversely, 2.0% ( n  = 72) of questions initially answered correctly were incorrect with the instruction sheet. For all 60 questions combined, recall (76.2% vs. 72.5%; p  = 0.08; Fisher Exact Test) and precision 87.4% vs. 87.1%; NS) were not significantly higher with the instruction sheet compared to without the instruction sheet (Fig. 4 ).

Triplicate determinations of the accuracy of GPT-4 in multiple-question mode with and without the instruction sheet. For each question, two histograms are shown. The left histogram shows the median of triplicate accuracy determinations without the instruction sheet. The right histogram shows the median of triplicate accuracy determinations with the instruction sheet. Increased accuracy associated with the instruction sheet is shown by coloring part of the right-sided histogram in blue while reductions in accuracy are shown by coloring part of the left-sided histogram in red. The sizes of the colored regions indicate the sizes of the increases or decreases in accuracy associated with the instruction sheet. The questions are shown in descending order of the increased accuracy associated with instruction sheet (i.e., the size of the blue histograms)

The instruction sheet significantly impacted three questions: one showed a net accuracy increase of 26.1%, resulting in 16 additional correct responses; another recorded an 8.3% improvement (5 papers); and the third saw a 6.7% increase (4 papers). The remaining questions displayed net changes in accuracy that were no greater or lower than three. The question “Does the paper report GenBank accession numbers for sequenced HIV isolates other than those for laboratory HIV isolates?" was the one associated with a net increased accuracy of 26.1%. The question “How many samples in the paper were reported to have undergone plasma virus sequencing?” was the one associated with a net increased accuracy of 8.3%.

In an attempt to edit the instruction sheet to increase GPT-4 accuracy for questions that were often answered incorrectly, we modified the query pipeline as follows. Rather than submitting each paper in multiple-question mode, we submitted each paper with just the one question that we were targeting for improvement (i.e., in single-question mode). After running several questions in both modes, we noticed marked differences in GPT-4 accuracy between the multiple-question and single-question modes. Figure 6 compares the accuracy of the multiple-question and single-question mode for all 60 questions without the instruction sheet. Each histogram represents the median of three replicates. Overall, the median and mean accuracy for the single-question mode were significantly lower than the multiple-question mode across all 60 questions: median (83.0% vs 91.8%, p  = 0.0006; Wilcoxon signed-rank test) and mean (77.6% vs 86.9%, p  = 0.0005; Wilcoxon signed-rank test).

Figure 6 groups the questions according to whether the accuracy was ≥ 10% lower in the single-question mode ( n  = 21 questions), ≥ 10% higher in the single-question mode ( n  = 3 questions), or less than 10% different between the multiple-question and single-question modes. The largest differences in accuracy between the two modes was for questions for which the answer was usually no for Boolean questions, 0 for numerical questions, and an empty list for list questions. We refer to these usually negative questions as No_0_Empty. Indeed, for the 21 questions that belonged to this category, the median accuracy was 91.0% in the multiple-question mode but only 60.8% in the single-question mode. There was also a strong correlation between the frequency of answers that were No_0_Empty and the reduced accuracy when questions were presented in the single-question mode ( r  = 0.45; p  = 0.0003).

The cost of using the GPT-4 API to obtain responses for 60 papers × 60 questions in the multiple-question mode without and with the instruction sheet was $240 and $300, respectively. The cost of obtaining responses for 60 papers × 60 questions in the single-question mode without and with the instruction sheet was $1500 and $2000, respectively. The overall cost of this study, considering that most experiments were performed in triplicate was $8120: $240 × 3 plus $300 × 3 plus $1500 × 3 plus $2000 × 1. After completing this study, OpenAI released a new model, GPT-4-turbo (gpt-4–1106-preview), on November 6, 2023. This model significantly reduced costs, decreasing from $0.06 to $0.01 per prompt and from $0.12 to $0.03 per completion. Consequently, the overall cost of this study would have been approximately five times lower.

We submitted the text of the methods, results, tables, and figure legends of 60 published papers on HIV drug resistance together with 60 questions related to HIV drug resistance with and without an instruction sheet to the GPT-4 API. We found that the accuracy of GPT-4 responses was approximately 87%, which was 24% greater than that obtained when the answers to the papers were permuted. With the exception of one question, the accuracy of GPT-4 was not improved with an approximately 2000 word instruction sheet. Notably, GPT-4 was also less likely to answer certain types of questions accurately when they were submitted individually (single-question mode) compared to when they were submitted together (multiple-question mode).

This study differs from most previous studies of automated software tools designed to assist with systematic reviews. First, we prompted the LLM GPT-4 to answer specific questions about entire papers whereas previous studies were often optimized for screening paper title and abstracts [ 5 , 6 , 7 , 8 ]. Second, we used GPT without providing training examples, whereas previous studies, were often interactive in that they combined NLP and ML algorithms with user feedback [ 5 , 6 , 9 , 10 , 11 ]. Finally, the results presented in this study were quantitative and transparent, whereas several previous studies, particularly those using LLMs, presented their results in a qualitative manner.

GPT-4 performs well at summarizing research papers because LLMs are adept at distilling and condensing information into simpler shorter formats [ 13 , 14 ]. However, answering specific questions can be challenging for LLMs because they only process a limited amount of text at once. This limitation hampers their ability to cross-reference details within longer documents effectively. Indeed, the questions that GPT-4 was likely to answer correctly were those for which the answer could be found in a single paragraph or sentence in a paper. In contrast, those questions that required reasoning about information found in different parts of a paper were less likely to be answered correctly. For example, the question “Does the paper provide complete ART history for all of the individuals in the study?” was answered correctly only about 50% of the time.

The instruction sheet contained information that would have been expected to be helpful for several questions such as “Does the paper report the results of HIV pol sequences?” and “Were the individuals in the study INSTI-naïve?”. Despite this, when GPT-4 was equipped with the information that 'pol' refers to the gene encoding the viral enzymes protease, reverse transcriptase, and integrase, 'INSTI' denotes integrase strand transfer inhibitors, and 'naïve' implies untreated, it still only correctly answered these questions 57% and 62% of the time, respectively.

After completing the experiments outlined in this study, we performed ten queries in an attempt to determine the extent of GPT-4’s HIV drug resistance knowledge. Supplementary File 6 lists each of the ten queries and the entire GPT-4 response (version last updated April 2023). The responses to these additional queries, demonstrated that GPT-4 possesses extensive information about HIV drug resistance. Although the experiments outlined in the results were performed using an earlier version of GPT-4 (last updated September 2021), all of the information included in the GPT-4 response had been publicly available prior to this earlier date. Given that GPT-4 already contained most of the information provided in the instruction sheet, enhancing its performance would likely hinge on providing prompts that demonstrate how to apply its knowledge to a published paper.

In an attempt to edit the instruction sheet to increase GPT-4 accuracy we modified the query pipeline by submitting each paper with just the one question that we were targeting for improvement. This modification led to the study’s second major new finding: when questions were presented individually, GPT-4 tended to provide incorrect affirmative answers to questions that generally warranted a negative response such as ‘no’, ‘0’, or an empty list. For instance, when the query "Which drugs were tested on phenotypic susceptibility in the paper?" was posed separately, there were 40 instances where GPT-4 erroneously referenced drugs that were administered to patients instead of those used in a susceptibility assay. This mistake was infrequent when all 60 questions were asked at once, indicating that presenting the full batch of questions improves GPT-4's understanding of each question's context. The enhanced accuracy observed when presenting multiple questions simultaneously may resemble automatic chain-of-thought prompting [ 17 ]. This technique, used in AI interactions, involves supplying step-by-step questions that guide the system through a logical thought sequence, thereby improving its comprehension of complex inquiries.

While enhancing the questions and instruction sheet was of interest, undertaking such revisions methodically would have required an open-ended approach beyond the scope of this study. Nonetheless, we observed that rephrasing two of the questions led to a significantly increased GPT-4 accuracy. For example, the question “Were sequences obtained from individuals with active HIV replication?” was true for 26 of the 60 papers. The median accuracy of GPT-4 on three replicates, with and without the instruction sheet, was 61%. However, the median accuracy of GPT-4 was 97% when we rephrased the question as follows: “Were sequences in the paper obtained from individuals with virological failure while receiving antiretroviral therapy?”. In contrast, the few changes we made to the instruction sheet did not yield substantial increases in GPT-4 accuracy for any of the questions.

GPT-4 possesses extensive knowledge about HIV drug resistance and it reproducibly answers Boolean, numerical, and list questions about HIV drug resistance papers. Its accuracy, recall, and precision of approximately 87%, 73%, and 87% without human feedback demonstrate its potential at performing this task. GPT-4 faced several challenges beginning with the specialized nature of the questions that were on topics that likely represented a small part of its training corpus [ 18 ]. In addition, addressing queries that necessitate making inferences, particularly when dealing with unsaid elements within the text, can be difficult. A more robust familiarity with the subject of HIV drug resistance would potentially have empowered GPT-4 to make better inferences. Finally, the instruction sheet was designed for human comprehension without the multiple examples usually necessary for optimizing a language model’s performance. The inability of GPT-4 to utilize the instruction sheet suggests that more sophisticated prompt engineering approaches or the finetuning of an open source model are likely required to improve accuracy when answering questions on highly specialized research papers.

Availability of data and materials

All data generated or analyzed during this study are included in this published article and its supplementary information files.

2R24AI13661806. The funder played no role in this review.

Jin Q, Kim W, Chen Q, Comeau DC, Yeganova L, Wilbur WJ, et al. MedCPT: contrastive pre-trained transformers with large-scale PubMed search logs for zero-shot biomedical information retrieval. Bioinformatics. 2023;39(11):btad651.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Cierco Jimenez R, Lee T, Rosillo N, Cordova R, Cree IA, Gonzalez A, et al. Machine learning computational tools to assist the performance of systematic reviews: A mapping review. BMC Med Res Methodol. 2022D 16;22(1):322.

Article   PubMed   PubMed Central   Google Scholar  

Blaizot A, Veettil SK, Saidoung P, Moreno-Garcia CF, Wiratunga N, Aceves-Martins M, et al. Using artificial intelligence methods for systematic review in health sciences: A systematic review. Res Synth Methods. 2022;13(3):353–62.

Article   PubMed   Google Scholar  

dos Santos ÁO, da Silva ES, Couto LM, Reis GVL, Belo VS. The use of artificial intelligence for automating or semi-automating biomedical literature analyses: A scoping review. J Biomed Inform. 2023J;1(142).

Article   Google Scholar  

van Dijk SHB, Brusse-Keizer MGJ, Bucsán CC, van der Palen J, Doggen CJM, Lenferink A. Artificial intelligence in systematic reviews: promising when appropriately used. BMJ Open. 2023J 1;13(7).

van de Schoot R, de Bruin J, Schram R, Zahedi P, de Boer J, Weijdema F, et al. An open source machine learning framework for efficient and transparent systematic reviews. Nat Mach Intell. 2021F;3(2):125–33.

Schopow N, Osterhoff G, Baur D. Applications of the Natural Language Processing Tool ChatGPT in Clinical Practice: Comparative Study and Augmented Systematic Review. JMIR Med Inform. 2023N 28;11(1).

Guo E, Gupta M, Deng J, Park YJ, Paget M, Naugler C. Automated paper screening for clinical reviews using large language models. J Med Internet Res. 2023 [cited 2023 Nov 26]; Available from: http://arxiv.org/abs/2305.00844 .

Weissenbacher D, O’Connor K, Klein A, Golder S, Flores I, Elyaderani A, et al. Text mining biomedical literature to identify extremely unbalanced data for digital epidemiology and systematic reviews: dataset and methods for a SARS-CoV-2 genomic epidemiology study. medRxiv. 2023 . 2023.07.29.23293370. [cited 2024 Jan 3]. Available from: https://www.medrxiv.org/content/https://doi.org/10.1101/2023.07.29.23293370v1 .

Syriani E, David I, Kumar G. Assessing the ability of ChatGPT to screen articles for systematic reviews. arXiv; 2023. [cited 2023 Nov 14]. Available from: https://arxiv.org/abs/2307.06464 .

Google Scholar  

Alshami A, Elsayed M, Ali E, Eltoukhy AEE, Zayed T. Harnessing the power of ChatGPT for automating systematic review process: methodology, case study, limitations, and future directions. Systems. 2023;11(7):351.

Khraisha Q, Put S, Kappenberg J, Warraitch A, Hadfield K. Can large language models replace humans in the systematic review process? Evaluating GPT-4’s efficacy in screening and extracting data from peer-reviewed and grey literature in multiple languages. 2023. [cited 2023 Nov 13]. Available from: http://arxiv.org/abs/2310.17526 .

Liang W, Zhang Y, Cao H, Wang B, Ding D, Yang X, et al. Can large language models provide useful feedback on research papers? A large-scale empirical analysis. arXiv; 2023. [cited 2023 Nov 14]. Available from: http://arxiv.org/abs/2310.01783 .

Liu R, Shah NB. ReviewerGPT? An exploratory study on using large language models for paper reviewing. arXiv; 2023. [cited 2024 Jan 5]. Available from: http://arxiv.org/abs/2306.00622 .

Rhee S-Y, Kassaye SG, Jordan MR, Kouamou V, Katzenstein D, Shafer RW. Public availability of HIV-1 drug resistance sequence and treatment data: a systematic review. Lancet Microbe. 2022;3:e392–8.

Tao K, Rhee SY, Chu C, Avalos A, Ahluwalia AK, Gupta RK, et al. Treatment Emergent Dolutegravir Resistance Mutations in Individuals Naïve to HIV-1 Integrase Inhibitors: A Rapid Scoping Review. Viruses. 2023S;15(9):1932.

Zhang Z, Zhang A, Li M, Smola A. Automatic chain of thought prompting in large language models. arXiv; 2022. [cited 2024 Jan 4]. Available from: http://arxiv.org/abs/2210.03493 .

Kandpal N, Deng H, Roberts A, Wallace E, Raffel C. Large language models struggle to learn long-tail knowledge. arXiv; 2023. [cited 2024 Jan 2]. Available from: http://arxiv.org/abs/2211.08411 .

Download references

Acknowledgements

Not applicable.

This work was funded by a grant from the National Institutes of Health:

Author information

Authors and affiliations.

Division of Infectious Diseases, Department of Medicine, Stanford University, Stanford, CA, USA

Kaiming Tao, Zachary A. Osman, Philip L. Tzou, Soo-Yon Rhee & Robert W. Shafer

Aphorism Labs, Palo Alto, CA, USA

Vineet Ahluwalia

You can also search for this author in PubMed   Google Scholar

Contributions

R.W.S contributed to the design of the work, interpretation of the data, and drafting the manuscript; K.T contributed to the design of the work, acquisition, analysis, and interpretation of the data, and creation of the software used in the work; Z.A.O and P.L.T contributed to the acquisition of the data; S.Y.R contributed to the design of the work and acquisition of the data; V.A contributed to conception of the work and revision of the manuscript.

Corresponding author

Correspondence to Robert W. Shafer .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

Robert W. Shafer has received honoraria for participation in advisory boards from Gilead Sciences and GlaxoSmithKline and speaking honoraria from Gilead Sciences and ViiV Healthcare. Other authors declare that they have no competing interest.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., supplementary material 2., supplementary material 3., supplementary material 4., supplementary material 5., supplementary material 6., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Tao, K., Osman, Z.A., Tzou, P.L. et al. GPT-4 performance on querying scientific publications: reproducibility, accuracy, and impact of an instruction sheet. BMC Med Res Methodol 24 , 139 (2024). https://doi.org/10.1186/s12874-024-02253-y

Download citation

Received : 09 January 2024

Accepted : 21 May 2024

Published : 25 June 2024

DOI : https://doi.org/10.1186/s12874-024-02253-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Large language model
• HIV drug resistance
• Systematic review
• Data extraction

BMC Medical Research Methodology

ISSN: 1471-2288

• General enquiries: [email protected]