Technology Research Preparation Week 2

Reading wk2

What is Research

Given a problem or an issue, a research question is framed that when answered, will address the issue.

• focussed on a single well-defined problem
• researchable
• feasible to answer within timeframe and practical constraints
• specific enough to answer thoroughly
• complex enough to require developing and validating the answer
• the potential answer is open to debate rather than accepted facts
• Relevant to your field of IT expertise

Why do we need Research

Main purpose:

• inform action
• prove theory
• contribute to developing knowledge in a field or study
• A tool for building knowledge and for facilitate learning
• Understand various issues
• Increase public awareness
• Aid business success

Outcomes are able to independently replicated, can be applied anywhere ideas can be communicated and by anyone with the requisite skills.

Research Proposals

Choosing a Topic in Research Proposals - A Practical Guide, Appendix 4 (2012)

Choose a good topic, initial questions to ask yourself, narrow the field of choice (use articles, readings / get inspiration from others’ past work), topic must be justified for the evaluators, do not presume that they share the same enthusiasm (sell the idea),

‘good’ for evaluators

• relevant (fitting the remit for the research set by those to whom the proposal will be submitted)
• worthwhile (is necessary and offers suitable benefits)
• feasible (scope, available resources, access to data, researcher skills, ethics and legality)

The topic should fit well with the requirements of the academics, sponser or for whom the proposal is being written.

Researchable topics

Not	Researchable
Should Uk become a republic (political judgment required)	What would be the constitutional change needed to make the UK a republic? Is public opionin in favour of retaining the monarchy?
What is the best rock band in the world (based on aesthetic judgement or emotion)	What criteria do people use when choosing the best rock band?, What is the most popular rock band in the world based on annual earnings from record sales and live performances
Is euthanasia a good thing? (moral judgement)	Under what circumstances would members of the public support the practice of euthanasia?

Ethics and legality

Play safe, do not choose a topic that might lead you to break the law in the process of collecting and analysing data.

Consider the consequences, when choosing one, consider the ethical implications of doing the research, the consequences of the research for the participants

Be specific

Factors influencing the topics

Of interest to self, a means of personal development Social identity - a reflection of personal background & experiences Professional self-interest - a means of career development

these make a significant influence, can be a positive benefit.

Interest, involvement and bias

A passionate interest can threaten the prospects of producing an impartial objective work.

Researches should ask themselves:

• Do I have a vested interest in the findings from the research?
• Will I be able to approach the topic with an open mind
• Could I incorporate ideas/views I disagree with and be willing and able to consider both sides of the arguments?
• Am I too close / involved to the subject?
• Will my personal values, beliefs, and background lead to biased findings?
• What chance is there that my research will provide a fair and balanced picture?

Justifying the choice

Again, the topic may not persuade the reader that is worthwhile.

So it needs to argue that it will do:

• fill a gap in what is already known by adding some info, by applying existing theories and methods in new context
• examine some contradition
• contribute to debate or controversy
• provide a timely commentary on some significant contemporary issue
• examine a practical problem, with a view to providing a remedy
• produce guidelines for good practice.

the way topic fits with existing research knowledge can be used to persuade that it’s worthwhile the way the topic fits with personal identity & research background can be used to persuade of the feasibility of the project

Literature Review

Guide To Writing, Literature Review by Randolph (2009)

Purposes for writing a literature review

means of demonstrating an author’s knowledge about a particular field of study, including vocab, theories, key variables and phenomena, and its methods and history.

• delimiting the research problem
• seeking new lines of inquiry
• avoiding fruitless approaches
• gaining methodological insights
• identifying recommendations for further research
• seeking support for grounded theory.

(Gall, M. D., Borg, W. R., & Gall, J. P. (1996). Education research: An introduction (6th ed.). White Plains, NY: Longman)

• distinguishes what has been done from what needs to be done
• discovering important variables relevant to the topic
• synthesizing and gaining a new perspective
• identifying relationships between ideas and practices.
• establishing the context of the topic or problem
• rationalizing the significance of the problem
• enhancing and acquiring the subject vocab
• understanding the structure of the subject
• relating ideas and theory to applications
• identifying the main methodologies and research techniques that have been used
• placing the research in a historical context to show familiarity with the state of the art developements

(Hart, C. (1998). Doing a literature review: Releasing the social science research imagination. London: Sage.)

• provides a framework for relating new findings to previous findings in the discussion section of a disseration.

Taxonomy of LR

Cooper, H. M. (1988). Organizing knowledge synthesis: A taxonomy of literature reviews. Knowledge in Society, 1, 104-126.

Cooper suggests that LR can be classified to 6 characteristics (Focus, Goal, Perspective, Coverage, Organization, Audience)

Focus

4 potential foci

1. Research outcomes: may help identify a lack of info on a research outcome, thus establishing a justifiable need for an outcome study.
2. Method review: help identify methods’ strengths and weaknesses in a body of research, and examine how research practices differ.
3. Review of theories: help establish what theories already exist, the relationships,and to what degree it have been investigated. (Suitable for if it aims to advance a new theory.)
4. Practices or Applications: help establish a practical need not currently being met

LR may review all or some of the focis.

Goal

is to integrate and generalize findings across units, outcomes, settings, to resolve a debate or to bridge the language used across fields.

Meta-analysis: goal is to integrate quantitative outcomes across studies.

Perspective

Often authors reveal their own preexisting biases and discuss how it may affect the review. Since secondary research (review research) methods parallel primary research methods, it makes sense for the author of a qualitative review to follow the tradition and reveal biases, and quantitative review author to claim a neutral position.

Coverage

deciding how wide to cast the net.

1. Exhaustive review: promises to locate & consider every available piece of research.
2. Exhaustive review with selective citation: define the population in such a way that it’s bounded and the number of articles to review is managable.
3. Representative sample: considering a sample of articles and make inferences about the entire population of articles. (gather evidence that demonstrates that the sample is actually representative)
4. Purposive sample: examines only the central, pivotal articles in a field. (key is to convince the reader that the selected ones are the pivotal articles)

LR can be organised methodologically, (intro, method, results, discussion). - often used in meta-analytic reports.

Audience

For a dissertation, supervisor and reviewers are the primary audience. Scholars in the field are secondary audience.

Avoid writing the review for a general, non-academic audience.

How to conduct?

The stages for conducting and reporting a literature review parallel the process for conducting primary research.

secondary research (Literature Review) is a modified version of the primary.

Key components are:

1. a rationale for conducting the review
2. research questions or hypotheses that guide the research
3. an explicit plan for collecting data, including how units will be chosen
4. an explicit plan for analyzing data
5. a plan for presenting data

units: articles that are viewed

Validity and reliability issues, iterative stages and mixed stages both applies to primary & secondary

Problem formulation

Once the type of review has been identified, focus is on problem formulation.

Decides what questions it will answer and determines explicit criteria to dictate the inclusion or exclusion of an article included.

LR is the primary source of the empirical research question

1. Determination of the questions that will guide the LR.
   • if goal is to integrate research outcome - “From the prev literature, what is the effect of intervention X on outcomes Y and Z”
   • if goal is to analyze the research methods - “What research methods have been used in the past to investigate phenomenon X & what are the flaws of the methods?”
   • if goal is to identify central issues (theories) - “What are the central theories that have been used to explain phenomenon X”
2. Explicitly determine the criteria for inclusion and exclusion.
   • determine which articles will be included and not.
   • lots of trial and errors will happen,
   • Recursive pilot-testing is time-consuming but less than starting over.

Data collection

must actively document how the data were collected. describe the procedure with such detail, that people can follow and get the same data. collection can stop when the point of saturation is reached, sufficient evidence to convince the readers that everything can be reasonably done is taken.

Data evaluation

Reviewer devises a system for extracting data from the articles Document the types of data extracted and the process used. - the detail is so extensive, it is included as appendices

Data analysis and interpretation

Reviewer attempts to make sense of the extracted data

Public presentation

Author determines which info is more important and will be presented and left out.

Formulating & justifying empirical research questions

LR, combined with research problem should formulate an empirical research questions. The author explains, using evidence from the review, how to disseration makes a meaningful contribution to knowledge in the field.

If the study is a contribution to an established line of theory and empirical research: make clear of what the contributions are and how to the study contributes to testing, elaborating, or enriching that theoretical perspectice

If the study is intended to establish a new line of theory, make clear what new theory is, how it relates to existing ones, why the new one is needed and the intended scope.

If the study is motivated by practical concerns, make clear what the concerns are, why it’s important and how this investigation can address those.

If the study is motivated by a lack of info about a problem or issue, make clear what it is that is lacking, the importance and how it will address the need for info.

(American Education Research Association. (2006). Standards for reporting on empirical social science research in AERA publications. Educational Researcher, 35(6), 33-40)

Quantitative Literature Review

before meta-analysis became prevalent, almost all quantitative reviews were narrative.

Narrative review

emphasized better designed studies & organized their results to form a composite picture of the state of the knowledge on the problem or topic being reviewed. Number of statistically significant results, compared with the number of nonsignificant results, may have been noted. Each study may have been described separately in a few sentences or a paragraph.

Despite the frequent use, its significantly affected by the reviewer’s subjectivity. One narrative review can differe completely from another review written by a different author.

Meta-analytic review

1. collect a representative or comprehensive sample of articles
2. codes those articles according to a number of aspects (study quality, type of intervention used etc)
3. finds a common metric (standardized mean difference effect size) that allows the study outcomes to be synthesized
4. examines how the characteristics of a study covary with study outcomes

criticism:

• it is subject to publication bias
• too mechanistic

so it is wise to combine, both. quantitatively synthesize each study, but also provide a thorough narrative description of particularly relevant studies.

Qualitative Literature Review

When a body is primarily qualitative, or contains a mixture of quantitative and qualitative results, it may be necessary to conduct a qualitative review.

Ogawa and Malen

8 steps:

1. Create an aduit trail - documents all steps
2. Define the focus of the review - problem formation
3. Search for relevant literature - non-research reports (memo, meeting notes etc) should also be included
4. Classify the documents - classify documents according to the types of data
5. Create summary database - data evaluation stage
6. Identify constructs and hypothesized causal linkages - identify essential themes between themes. increase the understanding of the phenomena
7. Search for contrary findings and rival interpretations - (reread documents to search for contrary evidence)
8. Use colleagues or informants to corroborate findings - share draft with collegues and informants for critical review

(Ogawa, R. T. & Malen, B. (1991). Towards rigor in reviews of multivocal literature: Applying the exploratory case method. Review of Educational Research, 61, 265-286.)

Phenomenological method

Goal is to arrive at the essence of the lived experience of a phenomenon (Moustakas, C. (1994). Phenomenological research methods. Thousand Oaks, CA: Sage)

Applied as a review technique, goal is to arrive at the essence of researcher’s empirical experiences with a phenomenon. Individuals who have experienced a certain phenomenon are interviewed. Unit of analysis is the research report rather than an individual who experienced it. Data come from an empirical research report rather than interview data.

Steps of phenomenological review mirror the steps of phenomenological research.

1. Bracketing - identify the phenomenon, then bracket their experience by explaining it
2. Collecting data - interview, read reports, decide criteria for inclusion and define research strategy.
3. Identifying meaningful statements - highlighting empirical claims about the phenomenon of interest and collecting those claims, in some kind of spreadsheet or qualitative software to make it manageable
4. Giving meaning - give meanings to the statements or categories
5. Thick, rich description - of the essence of primary researchers’ experiences with the phenomenon.

Mistakes Commonly Made in Reviewing Research Literature

1. does not clearly relate the findings of the literature review to the researcher’s own study
2. does not take sufficient time to define the best descriptors and identify the best sources to use in review literature related to one’s topic
3. relies on secondary sources rather than on primary sources in reviewing the literature
4. uncritically accepts another researcher’s findings and interpretations as valid, rather than examining critically all aspects of the research design and analysis
5. does not report the search procedures that were used in the literature review
6. reports isolated statistical results rather than synthesizing them by chi-square or meta-analytic methods
7. does not consider contrary findings and alternative interpretations in synthesizing quantitative literature.

Evaluating a Literature Review

(Boote, D. N., & Beile, P. (2005). Scholars before researchers: On the centrality of the dissertation literature review in research preparation. Educational Researcher, 34(6), 3-15.)

Stakeholder Analysis

Stakeholders are

• people affected by the impact of an activity
• people who can influence the impact of an activity

can be individuals, groups, a community or an institution

USER GROUPS : people who use the resources or services in an area INTEREST GROUPS : people who have an interest in, an opinion about, or who can affect the use of, a resource or service BENEFICIARIES of the project DECISION-MAKERS THOSE OFTEN EXCLUDED from the decision-making process

people can be include in more groups.

Primary Stakeholders

who benefits from, or are adversely affected by, an activity. This term describes people whose well-being may be dependent on a resource or service or area that the project addresses. Usually vulnerable, reason why a project is carried out - the end users.

Secondary Stakeholders

all other people and institutions with an interest in the resources or area being considered. Means by which project objectives can be met, rather than an end in themselves.

If stakeholders are not identified at the project planning stage, its at risk of failure, as project cannot take into account the needs and aims of those who will come into contact with it.

What does it do?

It’s a useful tool for identifying stakeholders and describing the nature of their stake, roles and interests

It helps:

• improve the project’s understanding of the needs of those affected by a problem
• reveal how little we know as outsiders, which encourages those who do know to participate
• identify potential winners and losers as a result of the project
• reduce, or hopefully remove, potential negative project impacts
• identify those who have the rights, interests, resources, skills and abilities to take part in, or influence the course of, the project
• identify who should be encouraged to take part in the project planning and implementation
• identify useful alliances which can be built upon
• identify and reduce risks which might involve identifying possible conflicts of interest and expectation among stakeholders so that conflict is avoided.

Risks

• analysis is only good as info used. It’s difficult to get the necessary info and many assumptions will have to be made
• Tables can over simplify complex situations

Carrying out stakeholder analysis (Methods)

Stakeholder table

+ Potential positive impact on interest – Potential negative impact on interest +/– Possible positive and negative impact on interest ? Uncertain

1 highest priority

Table showing influence and importance of stakeholders

Important to not neglect the primary stakeholders even if they have low influence.

Influence: Power that stakeholder have over the project

Importance: Priority given by the project to satisfying the needs and interest of each stakeholder.

A: High importance to the project but with low influence. They need special initiatives to ensure their interests are protected. B: High importance and high influence, important to develop good working relationships with these stakeholders to ensure adequate support for the project C: High influence but low interest. May be a source of risk, relationship with them are important and need careful monitoring. N D: Low influence and low interest, but need limited monitoring and evaluation to check they have become high priority

Identify stakeholder participation

Participation is the involvement of people in development projects.

• attending a meeting, even tho they say nothing
• taking part in the decision-making process
• contributing materials, money or labour
• providing information
• answering questions for a survey

inactive involvement of stakeholders lead to unsuccessful projects. active participation results in:

• Improved effectiveness - increase sense of ownership by beneficiaries, increase the objectives being achieved.
• Enhanced responsiveness - likely to target effort and inputs at perceived needs
• Improved efficiency - if local knowledge and skills are drawn on, project is likely to be good in quality, stay within budget and time, mistakes avoided and disagreement minimised
• Improved sustainability and sustainable impact - more commitment carrying on activity after outside support has stopped
• Empowerment and increased self-reliance - develop skills and confidence amongst beneficiaries
• Improved transparency and accountability - stakeholders are given info and decision-making power
• Improved equity - if the needs, interests and abilities of all stakeholders are taken into account

active participation is likely to have more benefits, but doesn’t guarantee a success in a project. and full participation takes time and conflicting interest will arise.

the diagram shows, lowest level as involvement, higher up the diagram, greater the participation. Different levels of participation will be appropriate for different stakeholders at different stages of the project cycle.

Partnership

when 2 or more stakeholders share in decision-making and the management of the activity. ideal: project staff <-> beneficiaries but its difficult

possible issues:

• primary stakeholders may think its costly in time and money compared to benefits expected
• P stakeholders may lack appropriate info for effective decision-making
• P stakeholders may challenge the right of other groups to participate
• Organisation may have a management structure that does not encourage participation of P stakeholders.

Participation matrix

Important to keep revising this table, we might have unexpected changes

after completion, think of how the participation might actually happen.

Research

All dev work should be accurate, reliable and sufficient info. good information is important in order to:

• understand the context in which the project is taking place
• understand the causes and effects of the issue that is being addressed
• understand what others are doing in order to avoid duplication and to work together if appropriate
• ensure that the response takes into account all factors and is the most appropriate and effective for the situation
• understand how the context is changing so the response can address potential future needs or prevent problems from arising
• justify the course of action to our organisation, beneficiaries, donors and others we are working with
• learn from past successes and mistakes
• provide good evidence for the response

Thorough research should look at social, technical, economic, environmental and political factors. This might help to identify new stakeholders and risks to the project.

• the area’s history
• geography, climate, environment, eg: main features, map, communication, area, seasonal problems
• population – numbers, age and sex profile
• social systems and structures – religious divisions, status of women, social institutions

Tutorial Preparation

Research Interest

a) Business & Technology, (government IT projects, startups, enterprise software dev)
b) AI & Project Management
c) IT projects fail 70% of the time, due to delays, over-budget, failure, AI PM tools could predict and mitigate risks and improve decision-making
d) AI, Projects, Success rates, Software development, Project Failures, Agile, ML / AI in PM
e) AI - simulate human intelligence, AI PM TOOLS - predicts project risks, optimize schedules and workflows and assist decision-making. Project failure - failure in meeting project scope, budget, time and quality, leading in critical cost.
f) Management - it’s about AI supporting PM, aiding decision-making for PM Socio-technical - AI impacting human roles in PM, interaction between the tool and PM Technical - AI methods used in tool, AI risk management algorithms

Potential Stakeholders

a) Primary

those directly impacted

• PMs
• Software dev teams (front, back)
• Companies using AI tools for PM
• AI service providers

b) Secondary

those that may benefit indirectly

• Business Exec
• Clients, end users of project
• Investors of Companies

c) Internal

• PM & team
• AI software dev, data scientists
• Execs

External

• Investors
• Clients

d) Social

Social stakeholders include people and organisations who may be involved in your research project or may be associated via business interests.

• IT professionals
• Tech companies
• Investors

Non-social

Nonsocial stakeholders refer to entities such as animals, the environment, and generations of people who have not yet to be born.

• Job market

Lecture

Methodical: Critical Thinking, Critical Reading & Critical Writing

this is how you need to work on your assignments

Critical Thinking

is necessary when a choice is required

• uncertainty exists

• no one right answer

• be objective. Methodologically analyse the problem based on context and facts (Citations)
• Analysing facts to understand the problem
• Using triangulation. Using multiple sources of data or multiple approaches to analysing data to enhance the credibility of a research study through balance argument
• Address a situation based on all available facts and info. Using critical thinking includes sorting and organising facts, data and other info to define a problem and develop effective solutions

Skills of Critical Thinking

Observation

Starting point, quickly sense and identify new fact. Understand why something might be an issue, Tolerate ambiguity

Analysis

Knowing what facts, data or info about the problem are: importance, their context, cause and effect on potential solution

Collaboration

Explain, discuss and learn about issues and possible solutions Identify others’ positions, assertions and claims - and their limitations Evaluate evidence from alernative points of view (social triangulation) Weigh up arguments and evidence in a balanced way Recognize false logic and other persuasive devices Persevere through researching and drafting arguments

Inference

Drawing conclusions about info to develop an answer

Problem Solving

Execute the solution

Information Literacy - supporting critical thinking

• recognize a need for info
• Distinguish way to address info gaps
• Construct strategies for locating info
• Locate and access info
• Compare and evaluate info obtained from different sources
• Organize, apply and communicate info to others
• Synthesize and build upon existing info, contributing to the creation of new knowledge

Principles supporting critical reading

• Have respect
• Be open and fair
• Recognise your cultural, personal bias
• Appreciate difference
• Possess healthy scepticism
• Think about issues as a whole
• Avoid accepting generalisations
• Justify arguments through robust evaluation of evidence
• Develop different approaches to reading texts

Different types of Critical Thinking

Inspectional Reading

systematic skimming

1. Does this book deserve reading? Check TOC, indexes, references, quick skim
2. Superficial reading, read without stopping, gain overall impression
3. More detailed reading, select some sections you want to follow up and reread.

Analytical Reading

Make notes and diagrams of the book’s structure, key concepts, discussions between books

1. Classify/Prioritize book
2. Summarize book in a sentence or paragraph
3. Show book’s concept organisations
4. More details

Syntopical Reading

read numerous books on same subject, compare and contrast ideas

1. Build a bibliography
2. Find relevant passages and add simple notes to bibliography creating a knowledge map
3. Common agreements
4. Clarification of positions
5. Define issues
6. Analyse discussions
7. Both work in progress and your main reference

Check Author’s Credentials

• Who is the author, what are their area of expertise; number of citations; institutional connections
• When was the text published or updated, is it recent enough for your topic?
• How much of the content is fact and how much opinion? Is the language objective or emotive? Is it balanced?
• Is the argument supported by evidence? What kind of evidence? How is the argument developed?

Assessing Reliability

• Identify authors’ evidence for their main points of argument, hypotheses, results and conclusions (Refer back to your lists, tables or diagrams of these ideas with referencing info)
• A good way to identify if the results of an experiment are reliable is to check how many participants were involved in the study, and if the results were calculated appropriately for the context of the study.

Identifying bias

• check the info whether it has bias init, check if source analyses and evaluates a range of other perspective
• if bias is present, find another counter-argument source
• Check if article or author is sponsored by a company that would benefit financially from the results
• Do a brief online search for the companies listed to check for potential bias

Clarification

• Clarify your task, make sure it meets the requirement of the rubric
• How are you required to write? Review academic writing and get help, talk to your tutor
• Check that you understand the material you have read, Be sure to read and also review sources carefully, but also review lecture notes and assessment tasks handout.

Reviewing, organising content, planning

• make lists, tables or diagrams of ideas with referencing info (page numbers) so you can track where you retrieved info from
• Use a literature review matrix
• May need to revisit texts, understand and develop different approaches to reading texts (Inspectional, Analytical, Syntopical)

Submission

• Read carefully for grammar, syntax, spelling and layout
• Check in-text citations and bibliography, do you need better support for facts/methodology/reasoning? (if you make 3 mistakes on referencing you lose all marks on referencing.)
• Trace your arguments from initial assumptions to evaluation to logical conclusion, does it make sense?
• Make sure your arguments are reasoned, valid, and relevant to your research topic. Does the content fit the rubric requirements?
• Have you completely answered all the rubric requirements at the highest level?
• Have you entered all the ID required.