Quantitative Research that adopts the Quasi-Experimental Design

In designing a quantitative research project, would you prefer to use an experimental design or a survey? Say why.
A quantitatve research describes trends or explains relationship(s) between variables. In general, 2 designs could be adopted: Experimental Design or Survey Design. The choice is however dependent on the desired outcomes (i.e. the intent of the study).

Experimental design is used when we want to study the possible "Cause-and-Effect" of the relationship between variables. Hypothesis testing comes into the picture.

Survey designs does not carry out experiments. It describes trends in the data and focuses on attitudes, behaviours and characteristics of a population.

In my research study that studies the impact of Web 2.0 technologies on students' ability to communicate their mathematical understanding, I am looking at the "Cause-and-effect" relationship between the independent variable, "Web 2.0 technologies" on the dependent variable "students' ability to communicate their mathematical understanding", hence the experimental design is adopted.

In reference to your choice, discuss the intent of the research, the research questions, and the hypothesis and research objectives.
Give examples to illustrate your meaning.

The Intent of the research sets the direction of the entire studies.

• In my project, the intent is to find out the impact of Web 2.0 technologies on students' ability to communicate their mathematical understanding.

The Research Question relates the independent variable with the dependent variable, which brings out the "Cause-and-Effect" relationship that the study sets to examine

• The question is "What is the impact of Web 2.0 technologies on students' ability to communicate their mathematical understanding", which relates the independent variable "Web 2.0 technologies" to the dependent variable "students' ability to communicate their mathematical understanding"

The Hypothesis is the conjecture on the outcome. The prediction, in this case is

• The use of Web 2.0 technologies will improve students' ability to communicate their mathematical understanding
  

Discuss key factors to consider when designing the experimental research or the survey.
Discuss how you might manage these factors throughout the study through data collection and analysis. Give examples.

Several factors are be considered when adopting the Experimental Design:

1. Random Assignment - by assigning the individuals randomly to groups, it 'distributes' the variability across the groups

2. Control of extraneous factors that could influence the data collected, hence making it unsuitable for comparison

3. Manipulation of Treatment Conditions - in the experimental condition, researchers intervene to alter conditions experienced by the experimental unit

4. Outcome measures - the outcome is the dependent variable, which is the presumed effect on the treatment variable. It is also the effect predicted in the hypothesis.

5. Group comparison

6. Threats to validity- internal and external
Internal threats might include maturity of individuals over time, the procedures carried out
External threats could be the interaction between setting and treatment

Research - Why is it Important?

Explain the importance of research to a new research student.
Give reasons why research is important and outline the problems of doing research today.

Broadly, the are 4 important reasons to carry out research:

1. Research adds knowledge to educational issues

• It closes gaps of knowledge
  
• It also expands knowledge
• It adds one's voice to knowledge
  

2. Research improves practices

• Educators generate new ideas
• Educators gain deeper insights on teaching methods
• Educators also gain deeper insights and understanding of their students

3. Research informs policy debates

• Research helps policy makers to make sound and informed decisions
• It offers weights to different perspectives

4. It helps students to develop a set of Research skills, which comprises of

• Organisational skills
• Analytical skills
• Presentation skills
• Writing skills

Some problems:

1. Contradiction or vague findings
   
2. Questionaable Data
3. Unclear statement of the intent of the study
4. Lack of full disclosure of data collected
   

Outline typical research processes and detail the skills needed for research.
Draw a model that clearly outlines the research spiral and explain the importance of each step.

The typical research processes
1. Define a Research Problem

• Specify the problem
• Justify
• Suggest the need to study
  

2. Review Literatures

• Locate Literature/ Resources
  
• Selecting resources
• Summarising resources
  

3. Specify the purpose

• Identify the purpose statement
• Narrow down the purpose & therefore the research questions

4. Collect data

• Selecting individuals to study
• Obtain permissions
• Gather information
  

5. Analyse and interpret the data

• Break down the data
• Represent the data
• Explain the data
  

6. Evaluate the data and write the Report

• Decide on audiences
• Structure the Report
• Write the report sensitively
  

Skills needed as a Researcher

• Curiosity to solve 'puzzle' - to look at issues with an inquiry mind and think beyond the existing/ known boundary & depth
  
• Attention span - the patience to go through the literature and revisit and re-search, as well as the time needed during data collection
• Library skills - the use of resources - to search, to summarise and to write
• Writing and Editing skills to write in context with the relevant audience in mind
  

Designing a QUALITATIVE Research Study

Think of a Qualitative study in your field.
In designing this qualitative research project, answer the following questions and give an example.

(a) Who would you study?

• We identify people that can best help us understand our central phenomenon
  

(b) What permission would you need?
We would need to seek permission from

• the organisation and the site where the research would be carried out
• the personnel involved in the research (e.g. students, teachers)
• Parents, when students are below the legal age to give approval
• As qualitative research requires greater access to the participant and the data collection process may take a much longer period, the researcher therefore needs to submit a write-up to provide the context and a detail description of the research to the Institutional Review Board for approval.
  

(c) What information would you collect?
In qualitative research, data are recorded in self-designed protocols that helps us record information provided by the participants.

• Observations - comprises of field notes and drawings (descriptive field notes on the description of events and activities and reflective field notes which documents the personal reflection)
  
• Interviews and questionnaires - in the form of transcriptions
  
• Documentation - hand recorded notes
  
• Audio visual materials - pictures, photos, videotapes, sounds, objects
  

(d) How do you administer the data collection?

• Time is needed for data collection
• > Limit collection to one or two observations/ interviews
• > Time is needed to establish substantial database
• Obtain permission to use the materials
• Handle ethical issues with sensitivity where the anonymity of the participants would be protected and information gathered to be kept confidential and not shared with people outside the research study.
  

(e) How would you analyse the data?

• The analysis initially consists of developing a general sense of the data and then coding description and themes about the central phenomenon
• When collecting data, one may also be analyzing other information previously collected to look for major ideas.
• One may read the data several times and conduct analysis each time, to gain a deeper understanding about the information given by the participant.
• The phases are iterative - between data collection and analysis

(f) How would you report the results?

• Use multiple perspectives for each theme: Identify perspectives based on individuals
• Use quotes to capture feelings, emotions and the way people talk about their experiences
• Write in vivid detail
• Specify contradictions and tensions in one's experiences
• Use metaphors and analogies
  

Designing a QUANTITATIVE Research Study

Think of a Quantitative Study in your field.
In designing this quantitative research project, answer the following questions and give examples.

(a) Who would you study?

• Participants involved in 'generating' the data for analysis.
• This could be students and teachers in the school.
• The participants are identified through random sampling
  

In the case of my research study on "The impact of Web 2.0 technologies on students' ability to communicate their mathematical understanding", the students would be the ones we study.

(b) What permissions would you need?
Permissions from

• the organisation or institution
• the data collection site (e.g. if the organisation operates at different sites)
  
• the people involved (e.g. students, teachers),
  
• parents (in an event when the participants are below a legal age to sign the agreement to participate in the study)

In my project, I will need to seek permission from

• the school where the students under the study are (by writing to the Principal)
  
• the students - who will be involved in the study (control, experiment group)
• the parents - as the students are under the legal age to sign the agreement, the parents' permission need to be obtained
• the teachers of the participating classes

(c) What information would you collect?

• Information are data that would address to the research questions.
  
• There are 4 types:

1. Measure of Performance - e.g. scores from achievement tests
   
2. Measure of Attitude - e.g. a record of participants' feelings towards a topic (through unbias questioning)
   
3. Observation of Behaviour - observations of specific behaviour that are recorded in an instrument like the checklist
4. Factual Information - quantitative data that are available with the organisation, e.g. Exam entry scores of all Secondary 1 students
   

(d) What instruments would you use?

• Instruments are tools used to measure, observe or record quantitative data
• They have to be reliable and valid
• They come in 3 forms:

1. Self-developed instrument
2. Locate and Modify
3. Locate and use in entirety

Examples of instruments include:

• Behavioral checklist
• Achievement test
• Aptitude test
  
• Pre-Post test
  

(e) How would you administer the data collection?
Two aspects need to be addressed to when administering data collection:

• Standardisation: To standardise the data collection procedure in order to minimise the likelihood where data of individuals are uncomparable or unsuitable for analysis. For example, clear instructions on the dos and don'ts (e.g. no calculator is allowed in the pre-test) to a pre/post-test is written clearly on handout and invigilators were thoroughly briefed before they administer the test.
  
• Ethical Issues: The anonymity of the individuals are to be protected so that confidential information will not be shared with participants outside the research.
  

(f) How would you analyse the data?
There are 2 ways to analysis quantitative data:

• Descriptive Statistics that involves the description of
  

1. Measures of Central Tendency (e.g. Mean, Median, Mode)
2. Spread of the Scores (e.g. standard deviation)
3. Ranking of the Scores
   

• Inferential Statistics involves

1. Hypothesis testing
2. Confidence Level
3. Effect Size
   

(g) How would you report the results?

• Results can be reported in the following forms:

1. Tables
2. Figures (includes charts, diagrams)
3. Discussion of the statistical analysis

Report would

• summarise findings in general statement(s)
• explain why they occur
• conclude a studey to advance the limitations to the research
  
• highlight how future research can improve on the weaknesses of the study
  

Literature Review

Discuss the significance of literature review in designing research.

Carrying out literature review allows us to

• provide evidence on the need of our study - e.g. to identify new practices to improve learning in classrooms
• document how our study would add to the existing researches (to ensure it is not a duplicate on existing studies)

From the 'personal -development' perspective, it develops our aptitude as a researcher

• through building our research skills in the use of libraries (print and non-print)
• to develop skills to locate information/ materials in a timely manner
  
• to find examples and models that are useful to our study
• to learn how other educators compose their research studies
  

Is there a difference between a quantitative and qualitative literature review?
There are 3 primary difference in terms of

• the amount of literature reviewed and cited
• the use of the literature review at the beginning of the study
  
• the use of the literature review at the end of the study.

In quantitative research, a substantial amount of literature is cited at the start to justify the need and importance of the study, hence the purpose and rationale of the study to be carried out; wherease in qualitative research, minimal amount fo literature is cited and there is no extensive discussion at the beginning of the research so that participants would be able to generate emerging views without being constrainted or influence by the views of others.

At the end of the result, in quantitative research, the findings were to confirm or disconfirmed predictions drawn from the literature whereas in qualitative research, it findings are used to support or modify existing findings in the literature.

Outline the 5-step approach in conducting a literature review

They are:

1. Identifying key terms (that are relevant to the research studies, for search purpose)
2. Locate literature - through different databases (online or print)
3. Critically evaluate and select literatures that are relevant to the research study
4. Organise the literature - by taking notes and organising them in a literature map
5. Writing the literature review - to report on the summaries of the literature to be included in to report.

What is the relationship between a literature review and the identification of a research problem and the research questions?

• With the key terms drawn from the research problem and questions, we surface literatures related to evaluate.
• Through evaluation of the literatures, previously 'unknown' relationships between variables might emerge as we organise the information and map them in the literature map.
  
• The new relationships or unknown variables might require us to re-visit our research questions to clarify the research objectives, hence sharpening the research questions.
  
• This would be an iterative process until we are clear of the direction of the research study.

Difference between Quantitative & Qualitative Research

Highlight the difference between Quantitative and Qualitative research.
Show how th design process differs from each approach (Purpose. Design Question. Data Collection. Analysis).

Quantitiative research

• (purpose) is appropriate for research questions that requires a description of trends or an exploration of relationships between variables.
  
• Hence, the research questions are specific and narrow, and tends to seek for some measurable and observable data for the variables.
  
• To collect such data, instruments with pre-determined questions and responses are often used (e.g. Pre-post test). It is used to collect quantifiable data which are numeric and it is usually administered to a large number of participants.
• The analysis tends to consists of statistical analysis that describes the trends or compares groups differences to relate the variables. The interpretation tends to compare results with the prior prediction (i.e. hypothesis) or past research.

Qualitative research

• (purpose) is approprate when addresssing research questions that we do not know the variables and there is a need for exploration.
• the research questions are therefore general and broad, and seeks to find understand the participants' experiences.
  
• To collect data, forms with emerging questions are used, to permit the individuals to generate responses. The data collected is largely made up by text and images. This number of participants involved in small.
• The analysis tends to be text analysis, and involves developing a description or themes. The interpretation tends to consists of stating a larger meaning of the findings.

Give example of how you would use either quantitative or qualitative approach in designing a meaningful project in your area.

A purpose research: To study the impact of Web 2.0 technologies on students' communication skills in Math learning (in a 1-to-1 ICT-enabled classroom).

The intent is to find a "Cause-and-Effect" relationship between the use of Web 2.0 technologies on students' communication skills. Hence it is a quantitative research.
On the other hand, because it is carried out with an intact sample (i.e. an existing class), it is therefore a Quasi-Quantitative research.

In this research, there are 2 variables:

• Independent variable: Web 2.0 technologies
• Dependent variable: Students' communication skills in Math learning

The result question is "What is the impact of Web 2.0 technologies on students' communication skills to articulate their mathematical understanding?"

To collect the data, a pre-post test is designed and administered to both control and experimental groups, before and after the intervention. The size of each group would be between 40 to 45.

The data (i.e. results of both pre-and post-tests) would be entered into a computer system to generate statistical analysis, to find out if there exists any relationship between the variables.