Applying the Evidence-based Approach to the Analysis of a Case: Actionable Insight through Research Assignment, NUI,

“Applying the evidence-based approach to the analysis of a case”

OVERVIEW

This assignment is designed to help you to develop your research and decision-making skills by applying the principles of evidence-based management to the analysis of a case study. We will upload a selection of cases to Moodle and you can choose any one of them for your assignment. You cannot switch between cases, however. Your assignment has to be based on only one of the proposed cases. The assignment consists of three sections that follow the learnings from Sessions I, 2 and 3 of the module respectively; as well as the leanings from your group project. Below you will find information on each of the sections.

Step I Clarity of the problem:

• What is the problem? What are its’ causes and organizational consequences? What is the “logic” / causal model behind the problem? If some elements of the problem/model are not clear, please indicate this and explain it.
• What are the assumptions behind the problem statement? Note that these assumptions may differ between different characters in the case, as well as between different practitioners commenting on it. If that’s the case, clarify this.
• What is the PICOC for the problem?

Step 1.2: Evidence for the Problem

• What types of evidence were used by the characters in the case to diagnose a problem? What types of evidence were not used (i.e., not mentioned, if any)?
• Are these sources/types of evidence trustworthy and relevant?
• Does this evidence support the problem statement/causal model presented in the case?

Step 1.3: Clarity of the solution

• What is the solution offered? What is the “logic” / causal model behind the solution? If some elements of the solution/model are not clear, please indicate this and explain it.
• Do we know the costs and benefits of this solution?
• Were multiple options of the solution considered in the case?

Step 1.4: Evidence for the solution

• What types of evidence were used by the characters in the case or by commenting experts to support the solution? What types of evidence were not used/not mentioned?
• Are these sources/types of evidence trustworthy and relevant?
• Does this evidence support the solution presented in the case?

Step 1.5: Needs for additional evidence

• Based on the analysis above, identify two key pieces of evidence (i.e., related to the logic model underlying the problem or the solution) that are still missing and that the decision-makers could gather to make a better decision.

a) One piece of evidence needs to come from existing empirical evidence from the scientific literature. For example, if the missing evidence you identified concerns the performance effects of remote working, the research question could be: How does remote working affects individual work performance?

b) The other piece of evidence needs to be missing empirical evidence from the organization or the stakeholders that require the collection of ‘new’ primary data (i.e., not secondary data). For example, if the missing evidence you identified concerns the job satisfaction differences (if any) between work-from-office and work-from-home employees in your organization, the research question could be: What are the differences in job satisfaction between employees that work from home and that work from the office?

These two pieces of evidence will inform two research questions that you will explore in the Sections 2 and 3 of your report respectively. These two pieces of evidence may be closely related (e.g. evidence on what motivates workers as suggested by the scientific evidence: and on what motivates employees in your particular organization); or maybe less so. It is your choice to decide what are the key needs for evidence in the case that you analyze.

SECTION 2: IDENTIFYING AND ANALYSING THE EVIDENCE FROM THE SCIENTIFIC LITERATURE

Now collect and critically evaluate existing empirical evidence in the scientific literature that you identified as the key to the situation in the case in Section I- Step 1.5a. Follow the steps that were discussed in class during Session 2, as well as relevant checklists in the textbook and on the slides. The following questions will guide you in your analysis.

Step 2.I: Literature search strategy

• Develop the search strategy that will enable you to identify empirical evidence from the scientific literature for the research question defined in Step I.5a. In particular, decide on
  • Search strings to use;
  • Sources to search in;
  • Inclusion/exclusion criteria

• In your report, explain this search strategy. and report your interim steps and search results (e.g., what search strings you tried, how you modified your search strings based on your findings, how many papers you found using particular combinations of the search strings, what inclusion/exclusion criteria you used, etc.). This can be done well in a table.

Step 2.2: Select the evidence

• Among the scientific evidence that you found select two papers. Focus on empirical work; and — where possible – include systematic reviews and/or meta-analysis. Explain your choice. That is, how are the selected papers relevant to the problem or potential solution(s) in the case study?
• Cite the selected papers in your bibliography section using Harvard style (see below for details).

Step 2.3: Critically evaluate the evidence

• Analyze the evidence (i.e., the papers that you have selected in step 2.2). Use the checklists from the textbook (Chapter 7) and from the lecture slides for the Session 2, that are appropriate to the research design of the selected piece of evidence. In particular,
  • evaluate the fit of the research design used for the research question that a given article seeks to answer;
  • identify methodological weaknesses, if any;
  • evaluate the trustworthiness of the findings and conclusions based on the considerations above;
  • briefly summarize (i.e., I — 3 sentences) the main conclusion of this piece of evidence.

Stec. 2.4: Identify the conclusions

• Integrate and briefly (i.e., a few sentences) summarize your findings from step 2.3: what does the evidence from scientific literature say about your research question?

SECTION 3: DEVELOP A PLAN FOR GATHERING ADDITIONAL EVIDENCE FROM THE ORGANISATION OR THE STAKEHOLDERS

For this section, focus on one critical piece of evidence that is still missing, as identified in Section I — Step 1.5b. Explain how you would gather this missing piece of empirical evidence from the organization or stakeholders. Apply the considerations that were discussed in class during Session 3. The following questions will guide you in your thinking.

Step 3.1 Define the data collection strategy

• What source of evidence will you tab into (i.e.. organizational or stakeholders)?
• Which research design (i.e., qualitative or quantitative) and which specific method (e.g., cross-sectional survey or qualitative interviews) will you use and why?
• Who will you collect the data from? Please define precisely who your potential participants (i.e., the population of interest) will be and provide an explanation for why you think that this population is suitable.
• Often it is neither feasible nor necessary to ask all potential participants. Rather, you will collect data from a sub-sample of the population you are interested in. How will you select individuals for your study (i.e., what sampling strategy will you use) and why?
• What sample size will you go for (roughly) and why? To support your reasoning, please make a rough estimate of the overall size of the population of interest.
• When and where will you collect data from them and why?

Step 3.2: Questions to be asked

• Please briefly discuss what topics you will cover in this data collection.
• Develop 3 — 5 example questions that could be used in this data collection. These should be related to the topics of interest (i.e., age, gender, etc. do not count here). Often, it is desirable to use pit-existing measures/questions from the extant scientific literature. You do not necessarily have to do this, but it is absolutely OK to suggest pm-existing measures here instead of developing your own questions.
• For questions you developed yourself: Please explain your thought process when you developed these questions (e.g., their content and wording).
• For questions/measures from the scientific literature: Please provide references to the scientific papers that developed the measures/questions you are suggesting, discuss why they are suitable for your data collection, and evaluate the measures/questions using the criteria discussed in class.

Step 3.3: Evaluation of your data collection design

• Discuss how this data collection will be helpful in dealing with the situation in the case. That is how will the evidence collected help in understanding the problem or its solution(s) better?
• Please briefly reflect on the potential strengths and weaknesses of your proposed data collection design (e.g., general strengths or weaknesses of the method, the sampling procedure, etc. as discussed in class).