Liberal Arts Political Science Political Theory Political Theory Notes
In the text below you will find notes from Political Theory. The notes cover Political Research Methods such as scientific inquiry, methodology, limitations, theory building, operationalization, research design, sampling and data sources, qualitative and quantitative approaches, ethics, statistics, and much more. The notes will help you with any Political Theory college course.
Note: If you are having trouble viewing this page or would prefer to view this information in a document viewer Click Here!
Research Methods in Political Science Notes
Lecture 1
Scientific Inquiry
I. The Nature of Scientific Inquiry
The Nature of Scientific Inquiry
• “Chaos is nature’s reality, order is man’s dream.”
• Curiosity and necessity motivate human inquiry.
. Nature of what science is all about
. Human innate desire to learn about their world driven by necessity or curiosity.
. End Point – understand world in which we live or improve it.
The Nature of Scientific Inquiry
• How do we know?
. A question of method.
. The primary interest of the course.
• How should we use what we know? (Application off scientific knowledge to use what we know)
. A question of ethics or preference.
. Addressed in part two.
NON-SCIENTIFIC INQUIRY
How Do We Learn (about the world)?
• Myth – create a shared set of beliefs (ex. G. Washington chopping down the cherry tree)
• Tradition – Shared knowledge developed from experience. Body of knowledge accepted by the community is not always correct (ex. Slavery)
• Authority – Accept knowledge from others (ex. Professor)
• Rationality – Use own logical reasoning to deduce truths about the world.
• Intuition – A gut feeling. Initial hunches may be incorrect.
• Personal experience
Problems with Non-Scientific Knowledge (with previous observations in learning about the world)
• Inaccurate observations
• Overgeneralization
• Selective observations
• Illogical reasoning
• Subjective or Normatively Based – Our own values may bias our inquires.
SCIENTIFIC INQUIRY
The Essence of Scientific Inquiry
1) • Causality
Future circumstances are caused by present conditions.
-Use scientific knowledge for prediction and understanding.
2) • Probabilistic Reasoning
The effect occurs more often when the causes occur than when they do not.
Characteristics of Scientific Knowledge
• Explicit – All rules for defining and examining are predetermined before the process begins.
• Systematic – Each item of evidence is linked by reason or observation by all other items of evidence.
• Controlled – Observed in as rigorous manner as possible
• Empirical – Seeks to explain how the world works as opposed to how it should work.
• Objective – Our own values do not bias our inquires.
Outputs of Social Scientific Knowledge
• Generalizable – Explain a broad class of phenomenon. A general explanation that can account for all similar types of cases.
• Predictive – Explains what has happened and what to expect in the future.
• Provisional – All scientific research is provisionally accepted until it is either disproved or improved by other researchers.
• (THEORY BUILDING IS…) Iterative – No single piece of research is accepted as the final word. Truth is accepted only after a body of evidence develops over time.
Scientific Research Methods
• (Definition of Scientific knowledge) -A process of testing theories and hypotheses by applying rules of analysis to the observation and interpretation of reality.
Scientific Inquiry
II. The Methodology of Social Scientific Research
STEP 1: Defining the Research Question
• All research starts with a question.
• Research questions should be narrowly defined. (easier to answer in a satisfactory manner)
• Often we address part of a much larger question.
• Many times we re-visit questions examined by prior research.
STEP 2: Developing a Theory
• Theories are proposed answers to our research questions.
• Theories are logical arguments that explain empirical relationships.
• Theories are not fixed, but evolve (over time).
• Theories seek to have explanatory and predictive power.
Types of Theories
•Normative theory seeks to explain how the world should work.
• Empirical theory seeks to explain how the world does work.
•Normative theory can inform our research, but the research must be empirical.
• The division between normative and empirical theorists is a point of contention.
Establishing Causality
• Theories explain how the independent variable causes the dependent variable to change.
• A bivariate theory posits a single iv affects the dv.
• A multivariate theory posits multiple ivs all partially affecting the dv.
Deducing Hypotheses (what we expect to find)
• Theories are not tested directly
• Instead we test hypotheses deduced from our theory.
• Hypotheses captures the essence of our causal argument in a single sentence.
Operationalization (From theory building to analysis)
• Operationalization is the process of moving from theory to analysis
.Data collection (existing or collect)
.Development of measures for variables in our theory (measurement – turn raw data into indicators that closely capture the theoretical concept of interest. (*Measurement is a crucial step in this process)
Application of either quantitative or qualitative methods
-Quantitative – thorough accounting of 1 or 2 cases but no generalibility.
-Qualitative – No depth, but generalibility.
Inference and Generalization
• The conclusions that we draw directly from our analysis are called inferences.
• Generalization assesses the larger implications of our research. (less context specific, but more general)
Scientific Inquiry
III. Limitations
Limitations
• Human beings – may defy systemic theorizing (idiosyncratic)
• Measurement – Some things we are interested in empirically validating may be difficult to measure.
• Observable – May not have access to all possible observations or actual decision making. (ex. Cannot enter the Oval Office)
• Small numbers of observations (n) – Difficult to study a small number of cases and develop and test theories that are generalizable.
• Subjectivity – In studying things you care about objectivity may be difficult to maintain in research.
Black Box Theories
Input
Input
Output
Input
* The box represents an educated guess on output (but we do not know what is in the box).
EXAMPLE:
Public Opinion
Legislative Priority
Veto or sign Bill
Partisan Control of Congress
Lecture 2:
Overview of Political Science
I. The Development of Political Science
Goals of Political Science
• The systematic and empirical study of political phenomena.
. This has not always been the case.
. Differences of opinion exist about what political scientists should accomplish.
1) The introduction of scientific methodology is a harmful illusion that trivializes basic truths about politics.
2) Critical Theory Approach – Political science is inseparable from society in general so we cannot simply look at political science so we have to take a broader look at what political science is.
3) You can study politics scientifically. Political science is about the systematic and empirical study of political phenomena.
The Evolution of Political Science
• As old as humankind (Some people argue: Plato’s Republic is the 1st political science book.)
• Towards a more empirical political
Science (Alexis de Tocqueville: observing democracy 100 years after it was established in the U.S. .
Book “Democracy In America”.
His research lacked a research method; it was rich in description, but no attention to political behavior.)
• The professionalization of political
Science (Modern era of political science 1) University of Chicago – In the 1920s and 1940s focused on organized empirical research program . Emphasis on barrowing from psychology and sociology to study political phenomena. First movement towards quantification. After WWII, behavioral revolution – University of Michigan application of scientific methods on the study of politics. Increased acceptance of quantitative methods. The discipline was maturing with the emerging of sub-fields, professional societies, and the creation of refereed journals. Entry of deductive and mathematical models (from economics). More recently, neo-institutionalism, capture the interdependence between institutional structures and how they affect and are affected by political actors. Important to remember, the traditional normative approach still survives.
• A hybrid discipline – The discipline has evolved by barrowing theories and methods from other disciplines such as psychology, sociology, anthropology, economics, philosophy, and journalism. Look to other disciplines for guidance.
Overview of Political Science
I. The Development of Political Science
II. Organization of the Discipline Organization of the Discipline
• American Politics
• Comparative Politics
• International Relations
• Political Methodology
• Political Theory
-Each sub-field:
-1) draw on different methodological and theoretical traditions.
-2) subject to its own methodological problems and difficulties.
-3) has its own journals and professional organizations.
-Because of the attention to sub-fields. Criticized for lacking integration as opposed to finding commonalities among sub-fields. This affects other disciplines to one degree or another.
American Politics (two areas of research)
• 1) Institutions – Focusing on the major institutions of the American governing process. (ex. Presidency, courts, legislatures, congress, state/federal relations.)
• 2) Political behavior – motivation and consequences of individual political activity. (ex. Voting behavior, campaigns and elections, public opinions, etc.)
• Public policy / Public Law – (These are unique to the other two) Public Law – focused on thick description and textual interpretation. Public Policy – brings together both the study of political behavior and political institutions.
Early institution studies ignored
Institutional changes and development and how political behavior is constrained by its institutions.
Neo-Institutionalists paradyme – tries to bridge the gap between instutions and political behavior; institutions themselves are a function of individual preferences. Paradyme tries to tease out how institutions are affecting individuals while at the same time individuals are shaping institutions to achieve their own goals.
Comparative Politics
• Focus on many of the same concerns as American Politics (only difference is that it takes a comparative perspective and tends to look at multiple countries.
• The sub-field is divided between area studies and comparative work (areas studies – a single country qualitative / comparative work – region or multiple countries quanitative)
International Relations (shapes by two concerns)
• 1) War and the decision making and institutions that support it
• 2) Political economy
• Presently much of the work in this area examines the impact that domestic politics exerts on international relations
Political Methodology – goal is to improve our ability to study politics scientifically.
• Philosophy of science – trying to determine what make science science and how we separate scientific knowledge from non-scientific knowledge.
• Improving research methods
• Statistics – Developing new statistical procedures to allow us to more accurately test our theoretical arguments.
Political Theory
• Political theory is becoming divorced from the rest of the discipline (it is more normative while political science itself moves more towards being empirical.)
• Political theory is normative in its orientation
. The Ancients (the Greeks)
. Modern (post-Machiavelli)
. Liberal political thought (post-Enlightenment) (ex. Locke, Hobbes)
. Contemporary political thought (more recent)
Theory Building I:
Formulating the Research Question
I. Role of the Research Questions
• All research begins with a question.
. The range of questions that can be addressed by political scientists is limitless.
. Many of the questions of interest to political scientists overlap with other disciplines.
Role of the Research Question (driven by two goals)
• All questions are asked for the same purposes:
. 1) Help us better understand the world in which we live.
. 2) Help us better anticipate or control future events. (using knowledge in our research to improve)
Sources for Research Questions
1. Personal observations.
2. Prior research. – Assures continuity. insures incremental advancement.
3. Personal interests and experiences. – Be careful of subjectivity.
4. Normative theory. – Actual research must be empirical (how the world works rather than how it should work).
5. Methodological preferences. – (NOT IDEAL) The research question should drive the method not the other way around.
Evaluating Research Questions (Not all questions are of equal value)
1. Contribution to knowledge.
2. Relevance to politics.
3. Potential for originality.
4. Feasibility. (Very important – can I handle the question in a manageable manner?)
5. Ethical issues.
** Not every question will address all of these criteria, but it would be wise for a researcher to consider them from the outset.
Formulating the Research Question
I. Role of the Research Questions
II. Defining and Focusing the
Research Question
Selecting a Topic
• A researcher may begin with a basic topic:
. International conflict.
. Campaign finance.
. Economic development.
• Very quickly, the researcher will need to focus their topic in order to develop a manageable research question.
** A general topic may motivate our research, but we must come up with a research question that is manageable.
Focusing the Research Question
• As the question is being focused, we think about potential explanations for what we observe.
(Theorizing – what is the potential answer)
• This leads to a working hypothesis, which lays the ground work for theory.
. At this point, we are thinking at the level of concepts (can vague or open to interpretation)
. We also want to think about the unit of analysis.
** Must be open to other points-of-view to avoid bias.
CONCEPTS – Term that can be used to describe objects, phenomena, or ideas.
UNIT OF ANALYSIS – What it is we are actually going to be studying.
EXAMPLE OF THE PROCESS:
Focusing a Research Question: Example
• General question: Why do some people participate in politics?
.What do I mean by political participation?
.What factors differentiate participants from nonparticipants?
. What factors differentiate participants from non-participants?
• Focused question: What affect do demographic characteristics have on voter turnout?
Book: Essentials of Political Research – Alan D. Monroe
Why is it important to know about research methodology?
1) To better understand past research results
2) You can better conduct original research of your own.
Definitions:
Science – an attempt to identify and test empirical generalizations.
Empirical – facts or the real world – that which can be known through the experiences of our senses (seen, touched, heard, smelled).
Normative – it reflects our judgments about what should be. Scientific methods cannot deal with non-empirical questions.
Objective – the results must not be dependant on any particular researcher’s biases. Intersubjective Testability – A finding cannot be accepted unless it can be replicated by others.
Generalization – Make a statement about an entire class of objects, not just individual cases, though the observation must be of individuals.
• The main purpose of science is to explain and predict.
• The generalizations made in social sciences are almost never absolute.
• Analytical statements refer to prepositions whose validity is completely dependant on a set of assumptions or definitions rather than on empirical observation.
Methods of reformulating normative questions into empirical questions: (pg. 6)
1) Change the frame of reference
2) Ask empirical questions about the assumptions behind normative judgments.
• Empirical research can never answer a normative question.
• Scientific research begins with the question the researcher intends to answer.
Topics to consider when formulating a research question:
1) Clarity
2) Testability
3) Theoretical Significance
4) Practical Relevance
5) Originality
Stages in the research process:
1) Formulate research questions
2) Formulate hypotheses
3) Research Design
4) Data Collection
5) Data Analysis
6) Draw Conclusions
Lecture 4
Theory Building II: Conducting the Literature Review
Conducting the Literature Review
I. The Role and Purpose of the Literature Review (Review existing body of knowledge)
-Political Inquiry – Has been around since the Ancient Greeks.
-Empirical Political Research – Roots begin with Machiavelli.
-Over 15,000 political scientists in the U.S. today.
The Role of the Literature Review
• Thorough knowledge of prior research is a key to theory building.
• What is a body of knowledge? (Developed slowly over a long period of time)
• Incremental contributions and iterative theory building. (Contribution is likely to be to an existing body of knowledge).
-Research is collective, not individual.
-Value of Research – Ability to build upon and extend existing literature incrementally adding to collective knowledge.
Purposes of the Literature Review (Read Wide and Knowledgeably)
• What research questions relevant to our topic have been addressed by others?
• How have other researchers gone about addressing these questions?
• What conclusions have these researchers found?
-Answering these questions will help…
o General understanding of past research relevant to our topic.
o Avoid problems down the road.
Purposes of the Literature Review (To get a firm understanding of research relevant to our topic)
• Uncover variables that we had not considered.
• Existence of relevant data.
• Measurement issues
• Rival or alternative explanations.
-Benefit from technology – Online Databases.
Conducting the Literature Review
I. The Role and Purpose of the Literature Review
II. Conducting and Organizing the Literature Review
Conducting the Literature Review
• Not exhaustive, but comprehensive
. The more work that is done up front the easier the rest of the process goes.
.The goal is to have a firm understanding of the accumulated knowledge in a given area.
. The role of technology.
Conducting the Literature Review
• Structuring the search
. Start broadly and move to more specific sources.
. Efficiency hints:
• Start with most pertinent and most recent.
• Stayed focused. (Do not read every word, scan for things relevant to our research)
• “Borrowing” from others.
American Political Science Association: A Volume .
The State of the Discipline
Summarizes all literature that has been done over the last decade.
Organizing the Literature Review
• Key research questions in the literature.
• Chronology.
• Competing explanations or conclusions. (Different camps of thought)
• Different methodologies. (Quantitative and Qualitative)
-Do not submit summaries of books (annotated bibliographies)
-Make as brief as possible
-Present crisp overview of literature relevant to your area of research.
Lecture 5
Theory Building III: The Logic of Theory Building
The Logic of Theory Building
I. What is a Theory? (Plausible explanation that will answer the research question.)
What is a Theory?
• The research question and theory building
. As we develop and refine our research question, we begin the process of theory building.
. This initial reasoning helps us to reduce the complexity of social life and puts us in a position to begin scientific inquiry.
What is a Theory?
• Purposes of theories
. Theories create possible explanations for observed events.
. Theories help to gain understanding of reality in order to better control it or adapt to it.
. Theories provide direction for how to determine if our understanding of events is correct
. Theories aid interpretation of events or data.
-Without theory we will not be able to determine if the data we observe is correct.
-Without theory, data are meaningless because theory provides context for why the data we observe exists in the first place.
What is A Theory?
• Theory defined
. Theories are logically related propositions that represent what we think occurs (e.g., intellectual tools).
. Theories are the proposed answers to our research questions.
. Theories are neither true nor false in any absolute sense, but are evaluated in terms of their usefulness.
. Theories are not found, but rather they are crafted.
-Theories go beyond description to address the WHY question.
-Theories begin with a thorough knowledge of what we want to explain (purpose of a literature review)
-Theory building can be thought of as a conundrum.
The Logic of Theory Building
I. What is a Theory?
II. Inductive Reasoning
Inductive Reasoning
PROCESS
Generalization (assumption)
Induction
Evidence (many specific facts)
EXAMPLE
All Republicans are conservative
Therefore
All the Republicans in Florida are conservative
-Induction – the purpose of generalizing from what we observe to what we have not/cannot observe (moving from the specific to the general)
-Inductive reasoning is considered weaker.
-Empirically Grounded – with inductive theory building observation of the data precedes our attempt to explain it.
-The main weakness – observe data in a specific context in trying to make a general claim devoid of that context.
-Goal – if we accurately tap into general assumptions it should apply to all.
The Logic of Theory Building
I. What is a Theory?
II. Inductive Reasoning
III. Deductive Reasoning
Deductive Reasoning
PROCESS
Generalization (assumption)
Deduction
Evidence (predictions about many specific facts)
EXAMPLE
The Republican Party attracts only conservatives
Therefore
All the Republicans in Florida are conservative
-Considered higher powered theory.
-Works opposite of induction.
-Reasoning from general to specific.
-If general assumption is on the mark should help to explain the specific.
The Logic of Theory Building
I. What is a Theory?
II. Inductive Reasoning
III. Deductive Reasoning
IV. The Confluence of Inductive and Deductive Reasoning
The Confluence of Induction and Deduction
1. Use induction to translate observations into general assumptions.
2. Then use deduction to develop predictions.
3. Test these predictions with new data.
4. Revise assumptions based upon analysis.
5. Repeat the process to refine and clarify the theory.
-Most theories involve an interaction between induction and deduction.
The Wheel of Science
Lecture 6:
Theory Building IV: The Building
Blocks of Theory
The Building Blocks of Theory
I. Concepts
Concepts
• What is a concept?
. Concepts are the initial building block of theories.
. Theories are composed of sets of concepts that are related by logical propositions.
. Concepts are merely a word or symbol that represents some idea.
Concepts
• Making concepts useful
. 1 -The concept must refer to a phenomena that is potentially observable (empirical referents – something we can observe that is indicative of the concept – concept itself is not observable ex. democracy).
. 2 -Concepts must be precisely defined.
. 3 -Concepts must have theoretical import.
The Building Blocks of Theory
I. Concepts
II. Variables
Variables (Similar to Concepts)
• From concepts to variables
. Concepts are mental constructs that need to be fleshed out.
. Once they have been defined and we have associated specific properties with them, they become variables.
-Variables are logical groupings of attributes.
Variables
• Variables and attributes
. Attributes are characteristics or qualities that describe an object.
. Variables are logical groupings of attributes.
. Attributes are the categories that make up a variable and represent our concepts.
Concepts vs. Variables vs. Attributes
Variables
• Variables and attributes
. Variables must have more than one attribute. (VARIABLES MUST VARY)
. Description versus explanation.
Variables
• Independent and dependent variables
. The independent variable is the factor that theory argues causes change in the relationship of interest
. The dependent variable is the effect (e.g., the behavior, decision, outcome of interest) we are trying to explain.
. Bivariate versus multivariate theories.
-Bivariate – (Simplest, rare, trivial) Single independent variable on a dependent variable.
-Multivariate – (Common in social science) More than one independent variable all affecting the dependent variable.
The Building Blocks of Theory
I. Concepts
II. Variables
III. Assumptions
Assumptions
• The role of assumptions
. Assumptions are the glue that hold our theory together.
. Because of their importance to theory building, assumptions need to be logically derived.
Assumptions
• Where do assumptions come from?
. With deduction, assumptions come first.
. With induction, assumptions are made after observation.
. To aid us in developing reasonable assumptions, we may turn to paradigms.
The Building Blocks of Theory
I. Concepts
II. Variables
III. Assumptions
IV. Paradigms
Paradigms
• Role of paradigms
. Paradigms are fundamental models used to organize our observations and reasoning.
. Paradigms can be difficult to recognize because they are often implicit.
. Paradigms provide researchers with sets of assumptions from which they can build their theory.
Paradigms
• Paradigms and scientific development
. Thomas Kuhn (The Structure of Scientific Revolutions). (1st to focus on paradigms)
. In the social sciences, paradigms may gain or lose popularity but they are seldom discarded.
. Instead, social sciences paradigms represent a variety of views, each of which offers insights the others lack while ignoring aspects of social life that the others reveal.
Examples of Paradigms in Political Science
• Realism – Used primarily in international politics. Internationally we are in a state of anarchy. Competitive.
• Rational choice – Barrows from economics. Decision making as a function of cost benefit analysis. (Cross disciplinary)
• Neo-institutionalism – Institutions are the rules of the game.
How humans construct institutions and then are affected by them in decisions.
• Political psychology – Study of political behavior mostly through the work of Walter Litman. Politics is a very low interest activity. Low cognitive effort to involve ones self.
Lecture 7
Theory Building V: Causality and Parsimony
Causality and Parsimony
I. Causality
Causality
• The centrality of causality
. Central to any theory is demonstrating causality between the independent and dependent variables.
. Social scientific theories focus on those independent variables or factors that have a strong and systematic effect on the dependent variable.
. Probabilistic versus deterministic reasoning.
-Probabilistic (mostly thought of in these terms) – when the independent variable is present it is probable in observing change in the dependent
variable.
-Deterministic – In every case where the independent variable is present that there should be the outcome we expect to see in the dependent variable (If one case doesn’t work the whole theory is thrown out).
Causality
• The concept of correlation
. The strength and direction of causal relationships.
. Direction can be either positive or negative.
. The strength of a relationship can be subjective.
. Collectively, we think of relationships between variables in terms of correlations.
-Correlation in itself does not guarantee causality.
Understanding Correlation
Positive – move in the same direction.
Negative – move in different directions.
Criteria for Causality
1. The variables are empirically correlated (a covariational relationship).
2. The cause precedes the effect.
3. A causal process that accounts for process of interest can be logically derived.
4. The correlation between the variables cannot be explained by a third variable (a spurious relationship).
-They are interrelated – Correlation (empirical phenomena that we can observe) – Causality (only comes from theory).
Problems in Establishing Causality
• Exogenous causal relationships
. Causation runs from the independent variables to the dependent variable (independent affects dependent)
. Example: The impact of inequality of wealth on revolution
. Example: The impact of age and education on voter turnout
Problems in Establishing Causality
• Endogenous causal relationships
.Causation runs back and forth between the independent and dependent variables. (Variables affect each other).
. Example: The relationship between public opinion and policy outcomes.
. Example: The relationship between campaign fundraising and election outcomes.
-Misspecified Theory – when you ignore endogenous relationships.
Causality
• Control variables
. A second class of independent variables a theory needs to account for are control variables.
. To demonstrate causality, we need to account for any other factors that may affect the relationship of interest.
. Past research and identification of control variables.
Causality and Parsimony
I. Causality
II. Parsimony
Parsimony
• Importance
. The development of general theories places a premium on parsimony.
. Parsimony is defined as “extreme or excessive economy or frugality.”
. Parsimonious explanations are simple, precise, and elegant.
Parsimony
• Occam’s razor
. Occam's razor is a logical principle attributed to the mediaeval philosopher William of Occam.
. It states that one should not increase, beyond what is necessary, the number of entities required to explain anything.
Causality and Parsimony
I. Causality
II. Parsimony
III. Characteristics of Useful Theories
Characteristics of Useful Theories
1. Testable – Needs to be able to be assessed with data.
2. Logically Sound – Consistent, cause precedes effect, etc.
3. Communicable – Can others understand your theory?
4. General – Predictative accuracy.
5. Parsimonious – Simple enough to be explained and applied.
-Useful Theories Provide: Clarity, simplification, and precision.
Building Generalizable Theories
Reading: Essentials of Political Research Chapter 2 – Alan D. Monroe
Theories, Hypotheses, and Operational Definitions:
-Science starts and ends with theories.
-Theories – A set of empirical generalizations about a topic.
-A theory consists of very general statements about how some phenomenon, such as voting decisions, economic developments, or outbreaks of war, occurs.
-But theories are too general to test directly because they make statements about the relationship between abstract concepts that are complex and not directly observable.
-To actually investigate the empirical applicability of a theory, it must be brought down to more specific terms.
-This is done by testing hypotheses.
-Hypotheses – An empirical statement derived from a theory.
-If a general theory is correct, then the more specific hypotheses derived from it ought to be true.
-If the hypothesis is confirmed by empirical observation, then our confidence in the general theory is increased.
-However, if a hypothesis is not confirmed, we must question the validity of the theory from which it was derived.
-Hypotheses are those answers to our research questions that seem to be the most promising on the basis of theory and past research.
-Hypotheses are statements about variables.
-Variable – an empirical property that can take on two or more different values.
-Each variable in a hypothesis must have an operational definition.
-Operational Definition – A set of directions as to how the variable is to be observed and measured.
AN OVERVIEW OF LEVELS OF RESEARCH: (p. 18)
LEVEL
THEORY: Concept 1 is related to Concept 2.
HYPOTHESES: Variable 1 is related to Variable 2.
OPERATIONAL: Operational Definition 1 is related to Operational Definition 2.
TYPES OF HYPOTHESES:
-Univariate Hypothesis – The hypothesis makes a statement about only one property or variable.
-Multivariate Hypothesis – Makes a statement about how two or more variables are related.
-Most scientific hypotheses are multivariate as well as directional.
-Directional – Specifies not just that the variables are related to one another but also what the direction of the relationship is.
-Positive/Direct Relationship – As one variable raises the other tends to rise (or as one falls the other falls).
-Negative/Inverse Relationship – As one variable rises the other tends to fall.
-Nominal Relationships – The hypothesis does predict direction, but one or both of the variables are such that they cannot be described in quantitative terms.
THEORETICAL ROLE:
-Theoretical Role – The presumed causal relationship between the variables are specified.
-Independent Variables – Those presumed in the theory underlying the hypothesis to be the cause.
-Dependent Variable – Are the effects or consequences.
-Often the nature of the relationship lies in the timing between variables.
-We usually presume that the social factors are independent variables and the behaviors are the dependent variables.
-Ultimately the decision as to which are the independent and which are the dependent variables is based on our theoretical understanding of the phenomena in question.
-Control Variable – Are additional variables that might affect the relationship between the independent and dependent variables.
UNIT OF ANALYSIS:
-Unit of Analysis – the objects that the hypothesis describes.
-Ecological Fallacy – Erroneously drawing conclusions about individuals from data on groups.
-The best way to avoid the problem is to draw conclusions only about the units of analysis for which the data were actually collected.
OPERATIONAL DEFINITIONS:
-Testing hypotheses requires precise operational definitions specifying just how each variable will be measured.
-Aggregates – Population groups.
-All variables in a hypothesis must be operationalized for the same unit of analysis.
-Two requirements in constructing the operational definition:
o 1 – What we want, and
o 2 – Where (or how) we will get it.
-If the unit of analysis is the individual then the source usually must be a survey.
-Standardized – Measured in a way that makes comparison of the different cases meaningful.
Reading: Reading Journal Articles:
-Journal articles provide very detailed and comprehensive perspectives of small pieces of much larger puzzles.
-The first thing to figure out is what the larger puzzle is.
-Often times the title, the introduction, or the conclusion can help.
-Journal articles organization – 1) Identifying the research question 2) the literature review 3) specification of theory 4) research design 5) analysis 6) discussion and conclusion.
Reading: Doing a Literature Review:
-A literature review summarizes and evaluates a body of writings about a specific topic.
-2 Key elements of a literature review: 1) concisely summarize the findings of prior research 2) reach a conclusion about how accurate and complete that knowledge is.
Lecture 8 Operationalization
Operationalization
I. Operationalization Overview
Operationalization Overview
* More abstract to more concrete.
Operationalization
I. Operationalization Overview
II. From Concepts to Indicators
From Concepts to Indicators
• Concepts and conceptualization revisited
. The process of conceptualization lays the ground work for measurement.
. Central to empirical research is the contention that any concept can be measured (through indicators).
-Concepts – description of mental images.
-use best words to describe concepts to properly convey meaning to others.
-Measurements – careful observation of the real world for the purpose of describing events and objects in terms of attributes composing of variables.
From Concepts to Indicators
• Concepts as constructs
. Scientists seek to measure three things:
• Direct observables (something we can observe directly ex. Color)
• Indirect observables (things we indirectly observe based upon information taken from another source ex. Survey Data)
• Constructs (theoretical creation based upon observation but we cannot observe directly ex. Measure Intelligence . (IQ Test)
. Because concepts are not real in any tangible sense, but rather they are descriptions of mental images, can they be measured?
From Concepts to Indicators
• Indicators
. Conceptualization gives definitive meaning to a concept by specifying one or more indicators of what we have in mind.
. An indicator is a sign of the presence or absence of the concept we are studying.
. Thus, concepts are not measured, but rather we measure indicators of these concepts.
. In many cases, there may be multiple indicators that might be used as proxies for concepts.
-Ex. Inflation . is an indication of the state of the economy.
-Ex. Multiple Indicators – Inflation, unemployment rate, and GDP . all indicators of the state of the economy.
From Concepts to Indicators
• Bringing it together
. Conceptual definitions give definitive meaning to the terms we are using,
. This lays the ground work for the operational definition, which define the procedures that will result in empirical observations of these concepts.
. In so doing, we precisely specify our variables and the indicators that we will analyze.
Operationalization
I. Operationalization Overview
II. From Concepts to Indicators
III. Formulation of Hypotheses
What is a Hypothesis?
• A statement of the expected probabilistic relationship between our independent and dependent variables.
• Hypotheses capture the essence of the causal relationship posited by our theory. (but do not explain the “why” part, the Theory does that)
• A separate hypothesis is needed for each independent variable.
Characteristics of a Good Hypotheses
• Empirical statements. (not normative)
• Focus on general as opposed to specific relationships.
• Based on logical reasoning. (Explanation as to why we expect the independent variable to effect the dependent variable)
• Stated in as specific terms as possible. (Directional – positive or negative)
• Testable. (Must be evidence or data in the real world that we can collect to test if hypotheses is correct or not)
Causal Diagram Example: Impact of Spending on Student Performance
Causal Diagram Example: Capitol Punishment Policy in the States
*Nothing is set in stone in the research process.
Lecture 9
Measurement (the process by which we develop rules for defining what the indicators of our variables are)
I. Introduction
Operationalization Overview
Introduction
• Importance
. The process by which the rules and procedures for defining our indicators for our theoretical concepts are defined.
. For every variable in our theory we need to develop an empirical indicator.
. Without valid and reliable measurement, empirical validation is impossible.
Introduction
• The nebulous nature of scientific concepts
. For many concepts that are commonly used, we do not give much thought to how they are measured.
. Measurement is of even greater import when working with abstract and complicated concepts.
. This problem plagues both the hard and soft sciences.
. The language of measurement provide the operational choices used to develop empirical indicators .
-Measurement is uncertain even for things we take for granted: (Ex.
Weights and measures .
Europe uses metrics)
-Pope Gregory 13th .
Gregorian Calendar
Measurement
I. Introduction
II. The Language of Measurement (Provides the operational choices to develop valid and reliable indicators)
The Language of Measurement
• Range of variation
. What is the potential range in values that a concept can have?
. To what extent am I willing to combine attributes into gross categories?
. What is the appropriate range that is needed to capture the variation I think occurs in the world?
-Ex. Income
The Language of Measurement
• Defining categories
. Conceptual and operational definitions specify our variables and their attributes.
-Ex. Measuring political affiliation:
. The attributes composing a variable should be exhaustive. (Republican, Democrat, or Independent)
. The attributes composing a variable should be mutually exclusive. (Only one Category – Ex. Republican)
The Language of Measurement
• Precision
. Precision focuses on the amount of information about a concept.
. Higher levels of measurement possible provides more information about the concept of interest.
. The level of measurement also has implications for the use of statistics.
Levels of Measurement
• Nominal – variables that are mutually exclusive and exhaustive (e.g., categories).
• Ordinal – variables that can be logically ranked. (There is order, but not a precise difference Ex. Age in terms of decades)
• Interval – indicates ranking and specifies exact difference between categories. (THIS IS IDEAL – actual distance separating the attributes Ex. Age in terms of years)
The Language of Measurement
• Reliability
. A measure is reliable to the extent that it gives the same result if the measurement is repeated. (Primary concern = Consistency)
. There are numerous sources of unreliability in social science data. (Ex. Attitudinal measurement – different interpretations cause problems)
. The best way to avoid unreliable measures is to develop precise conceptual and operational definitions prior to the fact.
Testing for Reliability
• Test-retest – repeating the measurement a second time. (Should get the same results)
• Multiple coders – different individuals measure the same concepts and then inter-coded reliability checks are used to assess reliability.
The Language of Measurement
• Validity (Primary concern = accuracy)
. How accurately does a measure captures the theoretical concept of interest?
. Validity is a bigger concern than reliability.
. Testing for validity is more difficult than testing for reliability.
Testing for Validity
• Face validity – on its face does the measure seem to be valid? (Most common but less rigorous – Self-evident)
• Construct validity – does the measure perform how we expect it to in relation to other concepts?
• Discriminant validity -how does the measure differ from indicators of other concepts it is unrelated to? (Ex. Trust of people in general vs. trust of government officials . should be different)
• Pragmatic validity – how well does the indicator perform compared to another measure that we know is valid? (Ex. More appealing candidates get a bigger share of the vote)
* THEORY DRIVES MEASUREMENT NOT THE OTHER WAY AROUND!
Measurement Error
• No measurement is 100% accurate.
• Measurement error stems from two considerations:
. Random error is a function of reliability. (Defies data randomly)
. Systematic error is a function of validity. (More problematic – Defies all data)
• Our goal is to eliminate systematic error and minimize random error.
Measurement
I. Introduction
II. The Language of
Measurement
III. Measurement Examples
Nominal Data Coding Example: Gender
Male = 1
Female = 0
*Dichotomist or Dummy variables.
Nominal Data Coding Example: Religious Affiliation
Catholic = 1
Jewish = 2
Protestant = 3
Muslim = 4
Ordinal Data Coding Example: Attitudes (Uses categories . not precise)
*This is called a Likert scale . captures rates of attitudes.
Ordinal Data Coding Example: Education
Ordinal Data Coding Example: Partisan Identification
Simple Interval Data Coding (Ranking and precise differences)
Examples: Age and Education
• Age measured in years
• Education measured in years
Complex Interval Data Coding
Examples: Media Exposure
A scale taken from the following NES question:
1. # of days per week respondent watches television news (0-7)
2. # of days per week respondent reads a newspaper (0-7)
3. How closely respondent followed campaign through television news (1-5)
4. How closely respondent followed campaign through newspaper (1-5)
The latter two variables are transformed into eight point scales to make them equivalent to the other two components of the index by subtracting one and multiplying by 7/4. The measure is then divided by 2.8 to create a scale ranging from 0 to 10.
Measurement
I. Introduction
II. The Language of Measurement
III. Measurement Examples
IV. Indexes, Scales, and Typologies
Indexes, Scales, and Typologies
• Used for concepts whose meanings are complex and varied, or when multiple indicators exist for a concept. (May need more complex measures . scales & typologies)
• Typically, scales and indexes are used in quantitative analyses.
• Typologies are used in both quantitative and qualitative research.
Indexes versus Scales
• Terms are used imprecisely and interchangeably.
• Both are interval measures and both are composite measures of indicators.
DIFFERECNE BETWEEN INDEXES AND SCALES:
• Indexes simply aggregate values from multiple indicators together.
• Scales do this as well as weight the individual indicators to reflect the degree to which each indicator taps the variable.
Index Construction Logic (Activism on some degree)
Scale Construction Logic (Different degrees of political activism)
Typologies
• Summarize the intersection of two variables to create categories or groupings.
• Typologies can be used as independent variables, but they do not work as dependent variables.
Typology Example
• Assessing the ideological tone of newspaper coverage of domestic and foreign policy.
Book: Monroe Pages 83-90:
Levels of Measurement:
-The term level of measurement refers to the classification or units that result when a variable has been operationally defined.
-There are three levels of measurement: nominal, ordinal, and interval data.
Nominal Variables:
-The “lowest level of measurement,” that is the least precise, is the nominal level.
-A nominal variable simply places each case into one of several unordered categories.
-Nominal variables contain information on “what kind” not “how much.”
Ordinal Variables:
-Ordinal variables rank cases in relation to each other.
-This can take two forms: 1) Rank order 2) Ordered categories.
-Rank order puts the cases in exact order according to some characteristic.
Rank order is not much used in analysis for research purposes.
-Ordered categories are not put into categories (like nominal variables) the categories have inherent order.
-Unlike nominal variables, ordinal variables, whether rank order or ordered categories, may be described in quantitative terms.
-In determining whether a set of categories may be considered as ordinal, it is important to remember that all categories must fit a pattern of high to low (or low to high) on the variable.
Interval Variables:
-This is the highest level of measurement.
-An interval variable provides an exact number of whatever is being measured.
-There is also a similar level of measurement called a ratio scale.
Examples of levels of measurement: (p. 86).
Rules for using levels of measurement: (Examples p. 89)
1) A variable may always be treated as a lower level of measurement. (Always can go down, but never up)
2) A dichotomy may be treated as any level of measurement. * A dichotomy is a variable that has two and only two possible values or categories (Ex. Male or female). In multivariate analyses a dummy variable is sometimes created using each category in a nominal variable to create new dichotomous variables.
Why levels of measurement are important:
-Because each of the many statistics designed for data analysis makes assumptions about the variables’ level of measurement. If you use an inappropriate statistic to evaluate your data, the results may be meaningless and lead you to draw erroneous conclusions.
-Always be aware of the level of measurement of your variables and of what levels the two rules will allow you to treat them as.
What is a statistic: P. 88 & 90.
Readings:
“The Multi-Layered Impact of Public Opinion”:
-One cannot understand the relationship between public opinion and policy without analyzing the interrelationships between the two over time.
-Without both a historical component and an investigation of implementation, policy theorists may underestimate the influence of public opinion on governmental programs in the states.
“The Poverty Measure”
-Economists have long argued that the poverty line should be revised to provide an accurate picture of who is actually poor.
-First developed in 1963 by Mollie Orshansky (an economist at the Social Security Administration). This was based on the affordability of an adequately nutritious diet. The cost was multiplied by three and if a family spent less than 1/3rd of their income on food then they were not poor.
-Hasn’t changed due to politics: decision to be made by the President, but does not affect a majority so is not a priority on the agenda.
“Bloomberg Seeks New Way to Decide who is Poor”
-Mayor Bloomberg of New York is developing and implementing a new poverty measure in hopes to gain national attention regarding the need for a new nationwide measure.
-Scholars say the new formula is likely to increase the poverty rate within the mayor’s city.
Lecture 10
Research Design (How research tends to fulfill the goal of a proposed study)
I. Role and Importance
Operationalization Overview
Research Design Definition
• The plan a researcher develops to fulfill the goals of a proposed study.
• The research design provides the empirical evidence necessary to evaluate the usefulness of a theory.
• There is continual back-and-forth and refinement between each stage of a research project.
-Imaginative process. Not “cookbook” research.
-Research design can make or break the entire research process.
Causality (one variable causes change in another)
• Empirical, theory-oriented research is exclusively concerned with assessing causal relationships.
• Research design is where causal relationships are formally evaluated.
-Causality only come from our theory.
Criteria for Causality
1. The variables are empirically correlated (a covariational relationship).
2. The cause precedes the effect.
3. A causal linkage that accounts for process of interest can be logically derived.
4. The empirical correlation between the variables cannot be explained by a third variable (a spurious relationship).
False Criteria for Causality
• Complete causation
. Because of parsimony, social scientific models do not seek to completely explain causality.
-Identify the variable that has the strongest effect.
• Exceptional cases
. Because of probabilistic reasoning, cases that do not comport to our theoretical expectations, do not disprove a causal relationship.
• Majority of cases
. Causal relationships can be true even if they do not apply to a majority of cases.
-Extreme cases (ex. An incumbent beat by a challenger because the challenger was able to raise tons of money)
Alternative or Rival Explanations
• A well-formulated research design also accounts for alternative explanations that could account for the causal process.
• A rival or alternative hypothesis is one that predicts the same outcome but asserts a different causal process.
• Typically, this is done by controlling for other independent variables suggested by prior research.
Example:
Controlling for Alternative Explanations: Gender and Vote Choice Example
• Theory: there are gender differences (the iv) in voting behavior (the dv).
• Past research suggests that education, income, and ideology also affect voting behavior.
• To insure that the differences in the dependent variable are due to gender we also need to account for the influence of these other independent variables.
• This is more difficult, but increases validity.
Sources for Rival Explanations
• Past research.
• Our own thinking.
• The review process.
Components of a Research Design
• 1 -Statement of research hypotheses to be tested.
• 2 -Discussion of sample and data sources.
• 3 -Discussion of the data collection.
• 4 -Precise definitions of the indicators used to measure each of our variables.
• 5 -Discussion of how the data that will be analyzed.
-Hypotheses: a single sentence statement that captures our theory.
The Unit of Analysis (what it is that we are studying)
• In social research there is virtually no limit to who or what can be studied.
• Units of analysis can be individuals, geographic entities, groups, outcomes.
• The ecological fallacy. (Drawing a conclusion on a level that your data was not measured at)
-Unit of analysis dictates what conclusions we can draw.
Research Design
I. Role and Importance
II. Research Design Templates
Research Design Templates
• Time and space
. Research designs seek to capture process that exists in either time or space.
. Thus, the relationship that we are interested in capturing may have a temporal or a spatial component (or in some cases both).
. Space refers to what is occurring in a particular place at a particular point in time.
. Time refers to process that develop over a period of time. (Ex. Bugeting, study of public opinion, etc. how relationships change over time)
The Basic Experimental Design (Strength: maximum leverage over causality)
-3 important elements:
1) Pre-test and post-test of dependent variable to measure change.
2) Experimental and control group
-Experimental: group experiences the dependent variable
-Control: does not experience the dependent variable (comparative category)
3) Independent variable is administered by the researcher at will.
-Quasi-experimental design: taking the logic of experimentation and applying it to non-experimental situations.
Cross Sectional Designs (most common because of the reliance on survey data)
• The independent and dependent variables are measured once and at the same time.
• The distribution of the independent variable creates quasi-experimental and control groups.
• Variation in the values of the independent variable are used to assess variation in the dependent variable.
Cross Sectional Designs
• Strengths
. Data collected in natural setting.
. Large and representative samples.
. Allow for easy control of rival explanations. (Through statistics)
Cross Sectional Designs
• Weaknesses
. Limited control over causality.
. Less precision. (In developing measures preferred)
. No point of comparison. (Major drawback)
Longitudinal Designs
• These designs examine the same phenomena over an extended period of time.
• Used in both qualitative and quantitative research.
• There are three type of longitudinal designs:
. Trend or time series studies.
. Cohort studies.
. Panel studies.
Trend or Time Series Designs (Interested in examining changes in population over time)
• Time series examines changes in trends that occur overtime in response to the introduction of the independent variable.
• The before and after represent quasiexperimental and control groups.
• These designs require numerous measures of the dependent variable before and after the independent variable is introduced.
• Problems with aggregated data.
-This design is common to public opinion studies.
Time Series Analysis
Time Series Analysis (key to this type of analysis – lots and lots of data)
Public Opinion and Abortion
-Aggregated data: you may miss stuff that occurs at a lower level of the data.
Cohort and Panel Designs
• Cohort designs examine specific subpopulations or cohorts over time. (Ex. Age groups)
. The same subjects are not analyzed in successive waves.
. instead, different samples from the cohort are taken.
-Cohort is more common in sociology.
• Panel studies are similar except the same subjects are analyzed at each wave.
. Strengths: better able to assess causality.
. Weaknesses: measurement error, cost, and panel morality. (Cost . need a large sample to start and need to keep track of all of the people.)
Advanced Designs
• Designs that attempt to overcome limitations associated with cross sectional and longitudinal designs.
• Pooled time series and pooled cross sectional approaches.
• Pooled cross sectional time series designs.
-These are attempts to get more validity out of the test.
-We don’t choose the research design we create one that is suitable for our purposes determined by creativity, resources available, and ethical concerns.
Lecture 11
Sampling and Data Sources
I. Sampling (this is used in all research not just in surveys)
• Every research design involves sampling
. A population is the universe of relevant observations.
. A case is a single observation taken from a population.
• Samples may be constrained by data and resource limitations.
• The ease with which samples can be gathered affects the type of research that gets done.
-Importance: this type of research wants to draw a conclusion for an entire population . often impossible . so you take a sample . and then make inferences.
-Limitations:
-Data (Ex. Campaign finance data is only available as far back as the 1970s)
-Cost
-The data available dictates the type of research that is done.
Sampling and Research Design
• Samples and Populations
. Researchers are interested in characteristics of populations.
. Because it is usually impossible to survey an entire population, a sample is taken from the population.
. What is learned from the sample is then used to make inferences about the population.
Sampling Logic
Characteristics of Good Samples
• A good sample is one that is representative of the population.
• A representative sample is one in which every attribute of interest in the population is present and roughly proportionate to the occurrence of each attribute in the population.
Sampling Methods:
1) Probability Sampling
• The preferred method because if it is done correctly, it yields representative samples.
• Every element of the population has a known, non-zero probability of being sampled.
• Random selection from the sampling frame.
• Allows for the calculation of the sampling error (e.g., the margin of error).
-We like this type because it allows for a margin of error to be calculated.
2) Non-probability Sampling
• Each element of the population has an unknown chance of being selected.
. Does not allow for sampling error to be calculated.
. Tend to be less representative. (and in many cases can be biased)
• Examples:
. Volunteer subjects. (Ex. Students volunteer for a professors study)
. Haphazard samples. (people happen to be where you are taking the sample)
. Quota samples. (research sample can meet certain quotas . (weight for a certain number of age, sex, race, etc. . problem is that it is not random)
. Purposive samples. (this is the only type that should be used in non-probability . it seeks out the sample (because they are rare)
The Process of Sampling
1. Identification of population of interest: who we are trying to learn about (defined by research question): Population: People likely to vote on Election Day
2. Select the sampling frame: a list of the target population from which sample is drawn (ideally, this is the same as the population): Sampling frame: List of phone numbers of registered voters
3. Draw sample using probability sampling method: Typically, this is accomplished via random digit dialing and screening questions.
Sampling
• Telephone Sampling (today most are done this way) . (downside: respondent fatigue – hang-up)
. The majority of samples are drawn via random digit dialing. (originally suspect because the poor did not have phones . not today because 98% have phones)
. Technological changes. (Problem today – cell phones, pagers, faxes, etc)
. Response rates (difficult to get a hold of people . certain people underrepresented)
Sampling
• Sample Weighting
. Samples may be weighted if the sample is biased due to problems with the response rate.
. When weighting is done, some respondents are weighted more or less so that the overall sample is reflective of the population.
. Weighting assumes that, for instance, sampled males are representative of unsampled males.
Sampling (never going to be 100% accurate)
• Sample Size and Sampling Error
. How can a sample of 1,500 accurately represent the views of nearly 300 million Americans? (major criticism)
. The precision that is lost is determined by the sample size and the sampling procedure
-2 sources of error:
. 1 -Systematic error: error that is built into the design that systematically biases results. (bad because it biases the entire sample)
. 2 -Random error: the margin of error. (due to sampling procedure)
Sampling
• Famous Incorrect Samples
. The 1936 Literary Digest presidential poll. (claimed incumbent FDR would lose . sample was not representative because they used car registrations and phone records . depression . sampled only the wealthy)
. The 1948 presidential election. Sampling Error: The 1948 (correct sampling methods but they stopped collecting too early)
Presidential Election
Sampling and Data Sources
I. Sampling
II. Data Sources
Data Sources
• The ubiquity of survey research. (surveys are commonly used . advantages – large number of observations, sample is representative, quantifiable, easy to use . problem – question development wording etc.)
• Government documents. (Federal, state, and local to a lesser extent Ex. Crime statistics
. strength – easy to gather, already in a quantified format . Weakness – no way for researcher to check validity)
• Direct observation. (usually used by other social sciences . strength – observed in a natural setting . weakness – unclear on how representative, ethics issues, concerns of objectivity)
• Primary documents. (used more often with quantitative work (diaries, etc) . Weakness – reliability, validity.
• Content analysis. (take written speech with an eye toward developing a quantitative measure . weak – reliability)
• Interviews. (similar to surveys . not representative sample . strength – comprehensive information, multiple perspectives)
-Book: Monroe Pages. 32 – 46
-Book: Monroe Chapter 5
Lecture 12
Qualitative and Quantitative Approaches
I. Qualitative Approaches
Qualitative Approaches (interested in studying a case in its entirety)
• Definition
• Qualitative research seeks to understand cases in their entirety.
• This leads to an emphasis on thick description.
. A case study can be thought of as a setting or group that the analyst treats as an integrated social unit that is studied holistically and in its particularity.
* Trades breadth for depth
Qualitative Approaches
• Characteristics of qualitative approaches
. Focus on a small number of cases.
. Generally, large units of analysis. (e.g. countries, states)
. Seek complete explanations of each case.
. Study phenomena in the context in which they occur.
. Focus on description and detail as opposed to numerical indicators.
Qualitative Approaches
• Examples of qualitative approaches
. Participant observation
• Field research.
• Developing a substantial relationship with people while they go about their normal activities. (researcher is immersed within the case)
• Overt versus covert observation.
-Covert – researcher blends in and does not divulge role as a researcher to get better quality data.
-Overt – everyone knows that the researcher is conducting research.
Qualitative Approaches
• Examples of qualitative approaches
. Intensive interviewing (open-ended, unstructured questioning)
• Relatively unstructured questioning that seeks to uncover in-depth information on the interviewee’s feelings, experiences, and perceptions.
• Much less structured as compared to a survey.
• Greater engagement between the subject and the interviewer.
Qualitative Approaches
• Examples of qualitative approaches
• Unstructured group interviews in which the group leader actively encourages discussion among participants on the topics of interest.
• Seeks to mimic the natural process of forming and expressing opinions.
• Unrepresentative samples. (Can undermine generalizability)
Qualitative Approaches
• Uses of qualitative approaches (5 reasons to use:)
. Testing theories.
. Creating theories. (theories are created while analyzing data . inductive approach)
. Identifying antecedent conditions.
. Testing the importance of antecedent conditions.
. Examining cases of great importance. (e.g. WWI & WWII)
Qualitative Approaches
• Qualitative approaches and the research process
. Defining the research question (dictates method)
. Theorizing (observation precedes theory)
. Research design (qusai-design less emphasis on causality)
. Sampling (insight to specific causes)
. Data collection (words as opposed to numbers)
. Data analysis (greater modification)
. Standards of evidence (lacks pre-defined rules to create a string argument)
. Reporting the results (narratives describing what was observed)
* Problem with selecting on the dependent variable: selecting cases that support your theory and avoiding cases that do not.
Qualitative Methods
• Controlling for rival explanations
. Research designs test both our theory and alternative explanations.
. With quantitative methods this is done via statistical control.
. With qualitative methods this is done via case selection.
• Most similar cases approach.
• Most different cases approach.
Qualitative and Quantitative Approaches
I. Qualitative Approaches
II. Quantitative Approaches
Quantitative Approaches (also referred to as large n studies)
• Characteristics of quantitative approaches
. Use of numbers and statistics to examine political phenomena.
. Emphasis on measurement of theoretical concepts.
. Seeks to develop parsimonious and generalizable explanations.
. Breadth versus depth.
Raw Data
Quantitative Approaches
• Goals of Quantitative Studies
. To test theories by examining patterns within our data. (not so concerned about a single case, but about the general pattern)
. Probabilistic as opposed to deterministic.
. Seek to generalize back to populations. (parsimonious theories & representative samples)
Quantitative Approaches
• How large does a large n study need to be?
. No magic number.
. More is better.
. Depends on statistical methods being used.
* Usually at least 120 cases for assumptions to be met.
Quantitative Approaches
• Quantitative Approaches and the Research Process
. Defining the research question
. Theorizing (pre-formulated theories)
. Research design
. Sampling (large and representative)
. Data collection (numerical measurement of variables)
. Data analysis (planned in advance of data collection . uses computers)
. Standards of evidence (clearer rules to establishing validity)
. Reporting the results (numerical data to present evidence)
Quantitative Approaches
• Strengths of quantitative approaches
. Control for alternative explanations.
. Representative samples. (Increases generalizability)
. Generalizability.
. Replication. (Increases validity)
Quantitative Approaches
• Weaknesses of Quantitative Approaches
. Lack of depth. (Breadth at the expense of depth)
. Abuse of statistics.
. Data driven research.
. Measurement issues.
Qualitative and Quantitative Approaches
I. Qualitative Approaches
II. Quantitative Approaches
III. The Qualitative/Quantitative Divide (This split is one of the biggest in the social sciences)
The Qualitative/Quantitative Divide
• Criticisms of Qualitative Approaches
. Where is the parsimony?
. Lack of generalizability.
. Lack of predictive power.
. Lack of attention to measurement.
. Replication.
. Controlling for alternative explanations.
* How representative is your sample?
The Qualitative/Quantitative Divide
• Criticisms of Quantitative Approaches
. Ignores the richness of politics.
. Lies, damn lies, and statistics.
. Some questions defy systematic explanations.
. Lack of accessibility of quantitative work. (math narrows focus of readership)
The Qualitative/Quantitative Divide
• The reality
. Both approaches are guided by the same principles. (Provide empirical measure of how the world works)
. The research question should determine the appropriate method.
. Regardless of which approach is used, imperfection is a given.
Lecture 13
Inferential Limitations
I. Validity
Validity
• Standards of Evidence
. After empirical analysis, we are in a position to assess how well the evidence supports the theory.
. This is a crucial step in the research process.
. However, there are no magic formulas that can help us to determine this.
. Rather, we assess the evidence in terms of the inferences and generalizations that we seek to make.
* @ types of conclusions we can draw: 1) Inference 2) Generalization.
Validity
• Inference – what specifically does our research tell us about the world?
. These are the facts that we put forth, which are drawn directly from our analysis.
. Inferences are narrowly drawn.
. We seek to infer from sample to population.
Validity
• Generalization – how do our theory and findings bear on other similar contexts?
. Generalizations are much more difficult than inferences because we are projecting our findings on to contexts that we did not examine.
. Generalization is aided by parsimony.
* 2 types of validity: 1) Internal 2) External.
Validity (determines types of conclusions to draw)
• Internal validity
. Bears on our ability to make inferences.
. How well does our research design test the causal process posited by our theory?
• Type I Error (false positive). (There is a relationship but it does not occur in the population)
• Type II Error (false negative). (Failed to detect a relationship in the population even though it does occur)
Validity
• External validity (refers to generalizability of our results)
. Do we expect to find the same causal process at work in other similar contexts.
. Bears on the generalizability of our research and is shaped by the parsimony of our research. (largely comes from parsimony)
Validity
• Which Is More Important? (Dr. Damore argues that internal validity is more important)
. Without internal validity neither accurate inferences about our sample cannot be made nor can we make inferences to the population of interest or other contexts in which our theory should be applicable.
. If we have internal validity, we still may have problems with external validity (e.g., experiments).
. Assessing internal and external validity is difficult because no rules exist to determine either.
. Both are shaped by a number considerations.
Inferential Limitations
I. Validity
II. Errors in the Research Process
* 2 types of errors: 1) Commission 2) Omission.
*Errors can undermine your internal or external validity.
Errors in the Research Process
• Errors of Commission
. Errors that result from actions taken by the researcher during the research process.
Errors in the Research Process
• Errors of Omission
. Errors that result from actions that the researcher should have taken during the research process.
Examples of Common Errors
• Focusing the research question. (causality)
• Reviewing the relevant literature. (too little/much focus on prior research)
• Development of theory. (valid assumptions)
• Deriving hypotheses. (failed to develop for each independent variable in a testable manner)
• Operationalization. (precise definition for concepts)
• Measurement.
• Data collection and research design. (what contexts to develop hypotheses)
Inferential Limitations
I. Validity
II. Errors in the Research Process
III. Dealing With Limitations
Dealing With Limitations
• Self-restraint. (be temperate in language to open opportunity for difference of opinion or conclusions . stress flaws because no research is perfect)
• The review process. (Journals: 1) present at professional conference 2) submit for publication . blind review . editor sends to 3 reviewers you do not know – Books: 1) send prospectus to publisher . multiple reviewers that may or may not be anonymous)
• Iterative research. (Only after bodies of evidence by numerous scholars have researched can it be accepted as common knowledge)
Readings:
1. Wars and American Politics
2. War and the Fate of Regimes
3. A Spiral of Cynicism for Some
Lecture 14
The Ethics of Social Research
I. What are ethics and where do they come from?
What are Ethics?
• Ethics are rules that structure behavior as it relates to a given activity.
• Ethics are ubiquitous to most activities that involve human interactions. (nearly every endeavor we are involved in has an ethical component)
Where Do Ethics Come From?
• Ethics tap into morality by defining what is “wrong” and “right.”
• The morality underlying ethics may come from a variety of sources. (e.g. religious, core values, pragmatic observations, political ideology, etc.)
• Ethics emerge out of an agreed code of conduct among practitioners of a group or profession.
Ethics and Social Research
• Social scientific ethics are agreements shared by researchers about what is proper and improper in the conduct of research. (necessary to accompany the research method)
• These ethics focus on three concerns:
. Subjects
. Reporting
. Application
The Ethics of Social Research
I. What are ethics and where do they come from?
II. Ethics and Subjects
Ethics and Subjects (to insure they are not taken advantage of)
• No harm to subjects. (Physical damage is unlikely in the social sciences, but maybe psychological . cause subject to question morality or self-worth)
• Voluntary participation. (A way to reduce exposure of researchers to subject harm . provide written information regarding study and provide to subject beforehand) <overt vs. covert>
• Anonymity and confidentiality. (no one should be identified by their responses
. use numbers to code for names and other identifiers . scientific research is not considered privileged information in courts)
• Deception. (hide true purposes of study to avoid canned responses . this must be justified on scientific grounds by de-briefing subjects and fully disclosing the purpose of the study after completed . subjects cannot leave being duped)
The Ethics of Social Research
I. What are ethics and where do they come from?
II. Ethics and Subjects
III. Ethics and Reporting
Ethics and Reporting
• Obligations to our fellow scientists regarding how we conduct and report our research.
• Science progresses through honesty and openness; not ego and deception.
Ethics and Reporting
• Report the shortcomings in our work. (e.g. in footnotes or appendix)
• Report negative as well as positive results. (Both of these findings contribute to knowledge . but negative results may be harder to get published)
• Do not misrepresent the research process. (Do not present unexpected results as if they were pre-planned hypotheses)
• Submission guidelines to journals and publishers. (Submit to one journal at a time . with books a manuscript to multiple publishers is acceptable)
The Ethics of Social Research
I. What are ethics and where do they come from?
II. Ethics and Subjects
III. Ethics and Reporting
IV. Ethics and Application
Ethics and Application
• The most controversial ethical debates center on the application of scientific research to the real world. (What do we do with the knowledge we created after we have created it e.g. cloning)
• Typically, there are no blanket rules for governing these concerns. (Between researchers, parties affected, and politicians)
• Often political considerations trump science.
The Ethics of Social Research
I. What are ethics?
II. Ethics and Subjects
III. Ethics and Reporting
IV. Ethics and Application
V. Institutions for Insuring Ethics
Institutions for Insuring Ethics
• The peer review process.
• Institutional review boards.
• Professional codes of ethics.
Readings:
-“The Politics of Government Funded Research”
-“Fingerprint Science on Trial”
-“Official Played Down Emissions’ Link to Global Warming”
-“Former Whitehouse Official Takes Exxon Job”
-“Doonesbury”
Lecture 15
Introduction to Statistics
I. Statistics and Social Research
II. Measurement Revisited
III. Organizing the Data
Introduction to Statistics
I. Statistics and Social Research
Statistics and Social Research
???? The ubiquity of statistics
???? Most people have an aversion to statistics. (Too difficult, think irrelevant, or problems with math)
???? Statistics are a used in all of the social, behavioral, and biological sciences, as well as in business and by the government.
???? We are focusing on one application of statistics: using statistics to test hypotheses derived from social scientific theories. (We are focusing on only one sector of statistics)
Statistics and Social Research
???? Statistics Defined
???? Statistics are tools and techniques that are used to describe, organize, and interpret data.
???? Anything that can be quantified is potential data.
???? Statistics can be either descriptive or inferential.
Statistics and Social Research
???? Statistics in practice
???? Statistical analysis occurs after the other stages of the research process have been completed.
???? However, in practice, many of the issues associated with data analysis are confronted throughout the research process. (Because often the types of analysis we want to perform affect earlier decisions we need to make)
Statistics and Social Research
???? Conceptualization versus doing the math
???? Being able to do the math is less important than understanding the concepts that underlie the math. (More important to understand the theory behind the math)
???? Learning and using statistics is an on- going process that necessitates practice.
Introduction to Statistics
I. Statistics and Social Research
II. Measurement Revisited
Measurement Revisited
???? Levels of Measurement
???? Nominal – classify or categorize. (Easiest to work with)
???? Ordinal – rank or order of categories.
???? Interval – order categories and indicate distances between categories. (Most complicated, but most useful)
- Statistics à assume that the numbers we provide represent the theoretical concept.
Measurement Revisited
???? Importance of Levels of Measurement
???? The level that data are measured at has two important consequences:
???? It determines the type of information and level (amount) of detail that is contained in the data.
???? It dictates the statistical methods that can be employed.
Nominal Data Coding (No ranking à values just differentiate)
Example: Gender
Male = 1
Female = 0
Nominal Data Coding
Example: Religious Affiliation
Catholic = 1
Jewish = 2
Protestant = 3
Muslim = 4
Ordinal Data Coding Example: (Separate categories à ranked on preference)
Attitudes
Ordinal Data Coding Example: (Separate categories)
Education
Ordinal Data Coding Example: (Order to the data, but do not represent exact differences)
Partisan Identification
Simple Interval Data Coding (Ordered with exact differences)
Examples: Age and Education
???? Age measured in years
???? Education measured in years
Complex Interval Data Coding (More commonly used in the social sciences)
Examples: Media Exposure
An index taken from the following NES question:
1. # of days per week respondent watches television news (0-7).
2. # of days per week respondent reads a newspaper (0-7).
3. How closely respondent followed campaign through television news (1-5).
4. How closely respondent followed campaign through newspaper (1-5).
The latter two variables are transformed into eight point scales to make them equivalent to the other two components of the index by subtracting one and multiplying by 7/4. Thus, the scale can range from 0 to 29. The index was then divided by 2.8 to create a scale ranging from 0 to 10.
Introduction to Statistics
I. Statistics and Social Research
II. Measurement Revisited
III. Organizing the Data
- In spreadsheets values for each case are placed in rows across values for each column.
- SPSS or DATA are types of statistical software.
Organizing the Data
???? Frequency distributions. (Helps to get a feel of what the data looks like)
Frequency distributions summarize a distribution of cases by their category value.
- Missing system = the number of respondents that did not answer the question. *These must be dropped from the analysis à the valid percent column takes this into account.
- Frequency distribution à better for ordinal data. Not good for interval data (if it is used you must categorize to make it manageable)
Organizing the Data
???? Frequency distributions.
???? Cross-Tabulations. (Relationship between two variables à better with nominal and ordinal level data)
Cross-Tab of Ideology and Gender (2000 ANES)
- Organization à Total is called “marginals” and the boxes where data appears are called “cells.”
Cross-Tab of Ideology and Gender (2000 ANES)
- Sometimes, percentages are easier to work with.
- Cross-Tabulations can be used for three or more variables, but it becomes difficult.
Organizing the Data
???? Frequency distributions.
???? Cross-Tabulations.
???? Graphical presentations.
The more common types of graphical representations:
Pie Chart of Ideology (2000 ANES)
Bar Graph of Ideology (2000 ANES)
Histogram of Bush Feeling Thermometer (2000 ANES)
- Histograms are related to the bar graph.
Cumulative Distribution of Bush Feeling Thermometer (2000 ANES)
Lecture 16 – Descriptive Statistics
Descriptive Statistics (2 types:)
I. Measures of Central Tendency à averages
II. Measures of Variability à range, standard deviation, etc.
Descriptive Statistics
I. Measures of Central Tendency à Used to learn more about the data.
The Mode (Use where there are categories à no ranking)
???? The mode is the most general and least precise average.
???? The mode is used for nominal data.
???? The mode is the most common value in a distribution.
Calculating the Mode: Partisan Identification Example
???? Raw data:
???? Democrat = 90
???? Republicans = 70
???? Independents = 140
???? The modal category is Independents because that category occurs with the greatest frequency
Unimodal Distribution (Only one high point in the distribution)
Bimodal Distribution (Two high points in the distribution)
The Median (Used at ordinal and interval levels of measurement)
???? The median is the midpoint of a distribution:
???? 50% of data is above.
???? 50% of data below.
???? Used with ordinal and interval data.
???? To calculate:
???? List a set of scores from highest to lowest
?? Find the midpoint
Calculating the Median
???? Raw data = 32, 45, 63, 77,55, 78, 25
???? Re-ordered data = 78, 77, 63, 55, 45, 32, 25 (Re-ordered from highest to lowest)
???? The median = 55
Calculating the Median
???? If there is an even number of values, then the median is the average of the two middle scores.
???? If the two middle most values are the same, then the median is simply that numerical value.
???? Percentiles are an extension of the median. (Percentage of data equal to or below the distribution)
The Mean
???? The mean is the most common measure of central tendency.
???? The mean is used with interval level data.
???? The mean is the arithmetical average.
Formula for the Mean
Comparing Measures of Central Tendency
???? The level of data (AND when most commonly used)
???? Mode – nominal.
???? Median – interval or ordinal.
???? Mean – interval.
???? The shape of the distribution (What can it tell us about the appropriate use according to levels of data)
Normally Distributed Data (The mean, median, and mode are more or less equivalent)
A Skewed Distribution (Use the median with skewed distributions à the mean is not a good measure)
Descriptive Statistics
I. Measures of Central Tendency
II. Measures of Variability (How scores differ from one another)
Variability Example
How Distributions Can Differ in Variability
The Range (This is the least precise measure of variability)
???? Provides a quick, but rough measure of variability.
???? Captures the difference between the lowest and highest values in a distribution.
???? The range is problematic because it only considers the largest and smallest cases.
Formula for the Standard Deviation (The most common measure of variability à measures the average given mean between data)
Calculating the Standard Deviation
1. List each score (the order of the scores does not matter).
2. Compute the mean for the distribution.
3. Subtract the mean from each score.
Calculating the Standard Deviation: Steps 1 - 3
Calculating the Standard Deviation
Calculating the Standard Deviation
6. Divide the sum by n-1(which is 9 in this case): 28/9 = 3.11.
7. Compute the square root to obtain the standard deviation: square root of 2.8 is 1.76.
Understanding the Standard Deviation
???? We do not add the deviations together because the result is zero.
???? We square each deviation to eliminate negative values.
???? The square root is taken to return to the original units.
Interpreting the Standard Deviation
???? The standard deviation tells us on average how far any score is from the mean.
???? Distributions with large standard deviations have greater variability.
???? Distributions with small standard deviations have less variability.
The Variance (Closely related to standard deviation)
???? The variance is closely related to the standard deviation.
???? It is calculated by following the steps of the standard deviation except taking the square root.
???? Because the variance captures variability in squared units, it is not all that useful in and of itself.
???? The variance is used with more sophisticated statistics. (Ex. Correlation, Regression analysis, etc.)
Lecture 17 – Probability
Probability (Lays the basis for statistical significance)
I. Probability and Probability Distributions
II. The Normal Curve
III. Z Scores
IV. The Central Limit Theorem
Probability
I. Probability and Probability Distributions
Probability and Probability Distributions
???? “Probability is expectation founded upon partial knowledge. A perfect acquaintance with all the circumstances affecting the occurrence of an event would change expectation into certainty, and leave neither room nor demand for a theory of probabilities.”
???? “Probability theory is nothing but common sense reduced to calculation.”
- In essence: Study of likelihood that an event will/will not occur given a set of circumstances.
Probability and Probability Distributions
???? Probability Defined
???? Probability is the study of the likelihood that an event will occur given a set of circumstances.
???? With enough information, the probability of any outcome occurring can be calculated.
Probability Distributions
???? Probability distributions
???? Probability distributions capture a theoretical distribution of likely outcomes in a population.
???? The probabilities for each value represent the likelihood of that outcome occurring.
- Frequency distributions focus on samples while probability distributions focus on populations.
Probability versus Frequency Distributions
???? Statistics for probability (population) distributions are symbolized with Greek letters
???? Mean = ? (mu)
???? Standard deviation = ? (sigma)
???? Variance = ?2 (sigma squared)
???? Statistics for frequency (sample) distributions are symbolized with English letters
???? Mean = x
???? Standard deviation = s
???? Variance = s2
Probability
I. Probability and Probability Distributions
II. The Normal Curve (A visual representation of a distribution of scores)
Characteristics of the Normal Curve
- Always symmetrical
- Mean, median, and mode are always the same.
- Tails continue for infinity
The Area Under the Normal Curve
- Cases at the ends of the curve (tail) have less than a 1% chance of occurring.
Probability
I. Probability and Probability Distributions
II. The Normal Curve
III. Z Scores
Formula for Z Scores (Allows us to make comparisons across distributions)
Calculating a Z Scores
* Means à A score of “20” is “3” standard deviations away from the mean of “50.”
What a Z Score Represents
* If the z-score is not a whole number a z-table is available.
The Z Score Table
???? If our z scores are not whole values we need to use the z score table.
???? The table tells us the area under the curve between the mean and any z score.
???? We can use the table to determine the area between two scores as well.
Probability
I. Probability and Probability Distributions
II. The Normal Curve
III. Z Scores
IV. The Central Limit Theorem
The Central Limit Theorem
???? If repeated samples are taken from a population and each sample mean is calculated and plotted, the distribution eventually would resemble the normal curve.
Characteristics of a Sampling Distribution of Means
1. The sampling distribution of means approximates the normal curve.
2. The mean of a sampling distribution of means is equal to the true population mean.
3. The standard deviation of sampling distribution of means is smaller than the standard deviation of the population
Understanding the CLT
???? Most of outcomes occur around the population mean.
???? Such events have a high probability of occurring
???? The farther we move from the mean, the less likely an outcome will occur.
???? Such events have a low likelihood of occurring
The Standard Error of the Mean
???? In practice, researchers draw a single sample.
???? This does not yield information about the population mean or standard deviation.
???? The standard error of the mean provides an estimate of the variation around the population mean.
Formula for the Standard Error of the Mean
Confidence Intervals
???? The standard error of the mean allows one to generate confidence intervals.
???? Confidence intervals are a range in which one might expect to find the true population mean.
???? These intervals allow one to assess the confidence that the sample mean is an accurate estimate of the population mean.
The 68% Confidence Interval for the True Population Mean
The 95% Confidence Interval for the True Population Mean
The 99% Confidence Interval for the True Population Mean
Estimating the Standard Error of the Mean
???? In practice, the standard error of the mean is estimated from sampling data.
???? This adds additional uncertainty beyond that which arises due to sample variability.
???? As a consequence, researchers want to have a wider range in estimating this value. (Purpose of n - 1)
???? This means that instead of using the normal curve, the t distribution is used.
The t Distributions
Estimating the Standard Error of the Mean with Sample Data (This is very important to understand because it is the basis for all interval statistics)
Lecture 18 – Statistical Significance
Statistical Significance
I. The Logic of Statistical Inference
II. Type I and Type II Errors
III. Conducting a Test of Significance
IV. Substantive versus Statistical Significance
Statistical Significance
I. The Logic of Statistical Inference
The Logic of Statistical Inference
???? In an ideal research setting, data from populations would be used to test theories.
???? In nearly all applied research, data from samples are analyzed.
???? However, researchers want to draw conclusions about populations.
The Logic of Statistical Inference
???? Researchers need leeway in how confident they can be in concluding that what was found in a sample holds in a population.
???? Statistical inference is the process by which researchers make this leap.
???? Statistical significance captures the level of risk one is willing to take in making this leap.
Significance Levels (Level of risk you are willing to take of making an incorrect inference)
???? Significance levels capture the likelihood of making an incorrect inference.
???? Significance = .05 means a 1 in 20 chance of making an incorrect inference.
???? Significance = .01 means a 1 in 100 chance of making an incorrect inference.
???? Significance = .001 means a 1 in 1000 chance of making an incorrect inference.
- Why not always use the highest? à If a more difficult threshold is used we may miss certain instances.
Statistical Significance (Tells you the likelihood of making a Type I error)
I. The Logic of Statistical Inference
II. Type I and Type II Errors
Type I and Type II Errors
???? The Null and Research Hypotheses
???? The research hypothesis states the causal relationship between the independent and dependent variables. (This is what we are interested in)
???? The null hypothesis states there is no relationship between the independent and dependent variables. (what occurs in the population not samples)
Type I Errors (Reject the null)
Type II Errors (Accept the null)
Type I and Type II Errors
???? Why Sample Size Matters
???? Large samples are more likely to be representative and are less likely to result in sample specific results.
???? With small samples there are fewer combinations of the values of the variables.
???? This increases the likelihood of obtaining by chance a combination of values suggesting a strong relationship.
The Ratio of Baby Boys to Baby Girls
- 120 babies are born in Hospital 1 each day
- 12 babies are born in Hospital 2 each day
???? On average, the ratio of boys to girls born in each hospital is 50/50
???? One day, twice as many girls were born in one of the hospitals
???? In which hospital is this more likely to occur? (Hospital 2 <12 babies> because it is a small sample)
Type I and Type II Errors
???? Why Sample Size Matters
???? The smaller the relationship the larger the sample needed to detect the relationship.
???? The larger the relationship the smaller the sample needed to detect the relationship.
- We don’t know relationship beforehand à Err on the side of caution and use larger samples.
Statistical Significance
I. The Logic of Statistical Inference
II. Type I and Type II Errors
III. Conducting a Test of Significance
Conducting a Test of Significance
1. State null and research hypotheses.
2. Set the significance level. (Anything above the .05 level is conventionally used)
3. Select the appropriate test statistic.
4. Compute the test statistic.
5. Determine the critical value needed to reject the null.
6. Compare the value of test statistic to the critical value and make a decision.
Comparing the Test Statistic to the Critical Value
Statistical Significance
I. The Logic of Statistical Inference
II. Type I and Type II Errors
III. Conducting a Test of Significance
IV. Substantive versus Statistical
Significance
Statistical versus Substantive Significance
Most common mistake: If one finds statistics that are significant, one assumes there is a substantive significance. A mistake because this only tells the likelihood of case occurring, not the substantive importance (what the numbers mean).
Statistical versus Substantive Significance
???? Statistical significance is a measure of technical success.
???? Substantive significance focuses on the interpretation of the statistical results in the context of our theory.
Lecture 19 – Inferential Statistics I: Difference of Means and ANOVA
Inferential Statistics I
I. Difference of Means (t Test)
II. ANOVA
Inferential Statistics I
I. Difference of Means (t Test)
-Inferential Statistics are used for hypothesis testing.
When Do We Use the Difference of Means or t Test?
???? We want to examine comparisons between groups.
???? The independent variable is nominal or ordinal.
???? The dependent variable is interval.
???? There are only two categories for the independent variable.
???? Equal variances among groups in the population.
???? The difference of means is also known as the t test.
- Are differences due to chance or is there a systematic difference between groups.
- Independent samples that are measured only once is most common with the difference of means test.
Formula for the Difference of Means or t Test
Standard Error of the Difference of Means
Determining the Critical Value
???? After t has been calculated, we need to determine if it is statistically significant.
???? The null is rejected if the value of the test statistic exceeds the critical value associated with the .05 level.
Comparing the Test Statistic to the Critical Value
- The (5% of all values) is rare and unlikely to occur by chance, but more likely to have a systematic differences between the dependent and independent variables.
The Critical Value and Degrees of Freedom (Sample sizes of both groups minus 2)
The t Distributions
- As sample sizes get larger we have more confidence.
Steps to Conducting a Difference of Means or t Test
1. Calculate the mean for each group.
2. Calculate the variance for each group.
3. Calculate the standard error of the difference of means.
4. Calculate the t ratio.
5. Determine the critical value, compare it to the obtained t value, and either accept or reject the null.
6. Interpret the substantive importance of the statistical analysis.
SPSS Output for t Test of Gender Differences in Thermometer Ratings of Bush (2000 ANES)
- Large t values indicate statistical and substantive significance.
SPSS Output for t Test of Racial Differences in Thermometer Ratings of Bush (2000 ANES)
Inferential Statistics I
I. Difference of Means (t Test)
II. ANOVA
When Do We Use ANOVA? (Analysis of variance)
???? ANOVA tests for significant differences in the mean values of the dependent variable for different categories of the independent variables.
???? ANOVA is used when there are more than two categories of the independent variable.
???? The dv is measured at the interval level.
???? The iv is measured at the nominal or ordinal level.
???? Equal variances among groups in the population.
- ANOVA can handle multiple categories in the independent variable.
Within Group Variation
- Difference between group means.
Between Group Variation
- Difference among group means.
ANOVA as an Extension of the t Test
Understanding Sum of Squares
???? ANOVA begins by considering the total variation in the dv.
???? This is the Total Sum of Squares (SStotal)
???? ANOVA separates SStotalinto two categories.
???? Variation in the dv resulting from differences within group (SSwithin)
???? Variation in the dv resulting from differences between groups (SSbetween)
Understanding Sum of Squares
Mean Squares
???? The various sums of squares increase with variation in the data and as sample size increases
???? Because the sums of squares increase with sample size, the number of scores used to calculate the sums of squares needs to be controlled for
???? This is done by dividing the SSwithin and SSbetween by their respective degrees of freedom to yield the mean square between (MSbetween) and mean square within (MSwithin)
Calculating Mean Square Between
Calculating Mean Square Within
The F Distribution
- Also known as a right skewed distribution.
- Tail shows when to reject null.
Calculating the F Ratio (If F value exceeds critical value à reject null)
Steps to Using ANOVA
1. Calculate the mean for each group.
2. Calculate the various sums of squares.
3. Calculate the within groups and between groups mean square.
4. Calculate the F ratio.
5. Determine the critical value, compare it to the obtained F ratio, and either accept or reject the null.
6. Interpret the substantive importance of the statistical analysis.
SPSS Output for ANOVA test of Regional Differences in Thermometer Ratings of Bush (2000 ANES)
