| Preface |
|
vii | |
| Part 1 Foundations |
|
1 | (38) |
|
|
|
3 | (36) |
|
|
|
4 | (10) |
|
|
|
4 | (1) |
|
|
|
5 | (1) |
|
|
|
5 | (1) |
|
|
|
6 | (1) |
|
The Nature of a Relationship |
|
|
6 | (1) |
|
Patterns of Relationships |
|
|
6 | (2) |
|
|
|
8 | (1) |
|
|
|
9 | (2) |
|
|
|
11 | (2) |
|
|
|
13 | (1) |
|
|
|
13 | (1) |
|
|
|
14 | (9) |
|
|
|
14 | (1) |
|
|
|
15 | (2) |
|
|
|
17 | (1) |
|
Positivism and Post-Positivism |
|
|
18 | (2) |
|
|
|
20 | (3) |
|
|
|
23 | (2) |
|
|
|
24 | (1) |
|
|
|
25 | (5) |
|
|
|
25 | (1) |
|
Where Research Topics Come From |
|
|
25 | (1) |
|
|
|
26 | (1) |
|
|
|
27 | (1) |
|
|
|
27 | (3) |
|
|
|
30 | (4) |
|
Introduction to Evaluation |
|
|
30 | (1) |
|
Definitions of Evaluation |
|
|
30 | (1) |
|
|
|
31 | (1) |
|
|
|
31 | (1) |
|
|
|
32 | (1) |
|
Evaluation Questions and Methods |
|
|
33 | (1) |
|
The Planning Evaluation Cycle |
|
|
34 | (1) |
|
|
|
34 | (5) |
| Part 2 Sampling |
|
39 | (22) |
|
|
|
41 | (20) |
|
|
|
42 | (2) |
|
Threats to External Validity |
|
|
43 | (1) |
|
Improving External Validity |
|
|
43 | (1) |
|
|
|
44 | (1) |
|
Statistical Terms in Sampling |
|
|
45 | (5) |
|
The Sampling Distribution |
|
|
46 | (1) |
|
|
|
47 | (1) |
|
The 65, 95, 99 Percent Rule |
|
|
48 | (2) |
|
|
|
50 | (5) |
|
|
|
50 | (1) |
|
|
|
50 | (1) |
|
Stratified Random Sampling |
|
|
51 | (2) |
|
Systematic Random Sampling |
|
|
53 | (1) |
|
Cluster (Area) Random Sampling |
|
|
54 | (1) |
|
|
|
55 | (1) |
|
|
|
55 | (3) |
|
Accidental, Haphazard, or Convenience Sampling |
|
|
56 | (1) |
|
|
|
56 | (1) |
|
|
|
56 | (1) |
|
|
|
57 | (1) |
|
|
|
57 | (1) |
|
|
|
58 | (1) |
|
|
|
58 | (1) |
|
|
|
58 | (3) |
| Part 3 Measurement |
|
61 | (108) |
|
The Theory of Measurement |
|
|
63 | (44) |
|
|
|
64 | (24) |
|
Measurement Validity Types |
|
|
65 | (1) |
|
|
|
66 | (1) |
|
Criterion-Related Validity |
|
|
67 | (2) |
|
Idea of Construct Validity |
|
|
69 | (2) |
|
Convergent and Discriminant Validity |
|
|
71 | (1) |
|
|
|
72 | (1) |
|
|
|
72 | (1) |
|
|
|
73 | (2) |
|
Threats to Construct Validity |
|
|
75 | (1) |
|
Inadequate Preoperational Explication of Constructs |
|
|
75 | (1) |
|
|
|
75 | (1) |
|
|
|
75 | (1) |
|
Interaction of Different Treatments |
|
|
76 | (1) |
|
Interaction of Testing and Treatment |
|
|
76 | (1) |
|
Restricted Generalizability across Constructs |
|
|
76 | (1) |
|
Confounding Constructs and Levels of Constructs |
|
|
77 | (1) |
|
The Social Threats to Construct Validity |
|
|
77 | (1) |
|
|
|
78 | (1) |
|
The Multitrait-Multimethod Matrix |
|
|
79 | (2) |
|
Principles of Interpretation |
|
|
81 | (2) |
|
Advantages and Disadvantages of MTMM |
|
|
83 | (1) |
|
A Modified MTMM--Leaving out the Methods Factor |
|
|
83 | (1) |
|
Pattern Matching for Construct Validity |
|
|
84 | (1) |
|
The Theory of Pattern Matching |
|
|
84 | (2) |
|
Pattern Matching and Construct Validity |
|
|
86 | (2) |
|
Advantages and Disadvantages of Pattern Matching |
|
|
88 | (1) |
|
|
|
88 | (15) |
|
|
|
89 | (1) |
|
|
|
90 | (1) |
|
|
|
90 | (1) |
|
What Is Systematic Error? |
|
|
91 | (1) |
|
Reducing Measurement Error |
|
|
91 | (1) |
|
|
|
92 | (4) |
|
|
|
96 | (1) |
|
Inter-Rater or Inter-Observer Reliability |
|
|
96 | (1) |
|
|
|
97 | (1) |
|
Parallel-Forms Reliability |
|
|
98 | (1) |
|
Internal Consistency Reliability |
|
|
99 | (2) |
|
|
|
101 | (2) |
|
|
|
103 | (2) |
|
Why Is Level of Measurement Important? |
|
|
104 | (1) |
|
|
|
105 | (2) |
|
Survey Research and Scaling |
|
|
107 | (44) |
|
|
|
108 | (24) |
|
|
|
108 | (1) |
|
|
|
108 | (1) |
|
|
|
109 | (1) |
|
Selecting the Survey Method |
|
|
109 | (1) |
|
|
|
109 | (1) |
|
|
|
110 | (1) |
|
|
|
111 | (1) |
|
|
|
111 | (1) |
|
|
|
112 | (1) |
|
|
|
112 | (1) |
|
|
|
113 | (1) |
|
|
|
113 | (4) |
|
|
|
117 | (2) |
|
|
|
119 | (3) |
|
|
|
122 | (2) |
|
|
|
124 | (1) |
|
|
|
125 | (1) |
|
|
|
125 | (1) |
|
The Role of the Interviewer |
|
|
125 | (1) |
|
Training the Interviewers |
|
|
126 | (1) |
|
|
|
127 | (1) |
|
|
|
127 | (5) |
|
Advantages and Disadvantages of Survey Methods |
|
|
132 | (1) |
|
|
|
132 | (18) |
|
General Issues in Scaling |
|
|
133 | (1) |
|
|
|
134 | (1) |
|
|
|
134 | (2) |
|
Unidimensional or Multidimensional? |
|
|
136 | (1) |
|
The Major Unidimensional Scale Types |
|
|
136 | (1) |
|
|
|
136 | (9) |
|
|
|
145 | (2) |
|
|
|
147 | (3) |
|
|
|
150 | (1) |
|
Qualitative and Unobtrusive Measures |
|
|
151 | (18) |
|
|
|
152 | (15) |
|
The Qualitative/Quantitative Debate |
|
|
154 | (1) |
|
Qualitative and Quantitative Data |
|
|
154 | (1) |
|
All Qualitative Data Can Be Coded Quantitatively |
|
|
155 | (2) |
|
All Quantitative Data Is Based on Qualitative Judgment |
|
|
157 | (1) |
|
Qualitative and Quantitative Assumptions |
|
|
158 | (1) |
|
|
|
159 | (1) |
|
|
|
159 | (1) |
|
|
|
159 | (1) |
|
|
|
159 | (1) |
|
|
|
160 | (1) |
|
|
|
160 | (1) |
|
|
|
161 | (1) |
|
|
|
161 | (1) |
|
|
|
161 | (1) |
|
Unstructured Interviewing |
|
|
161 | (1) |
|
|
|
161 | (1) |
|
|
|
162 | (1) |
|
|
|
162 | (1) |
|
|
|
162 | (1) |
|
|
|
162 | (1) |
|
|
|
163 | (1) |
|
|
|
164 | (1) |
|
|
|
164 | (1) |
|
|
|
165 | (1) |
|
Secondary Analysis of Data |
|
|
166 | (1) |
|
|
|
167 | (2) |
| Part 4 Design |
|
169 | (86) |
|
|
|
171 | (20) |
|
|
|
172 | (14) |
|
Establishing Cause and Effect |
|
|
173 | (1) |
|
|
|
173 | (1) |
|
Covariation of the Cause and Effect |
|
|
174 | (1) |
|
No Plausible Alternative Explanations |
|
|
174 | (1) |
|
|
|
175 | (3) |
|
|
|
178 | (4) |
|
|
|
182 | (3) |
|
Social Interaction Threats |
|
|
185 | (1) |
|
|
|
186 | (2) |
|
|
|
188 | (2) |
|
|
|
190 | (1) |
|
|
|
191 | (24) |
|
Introduction to Experimental Design |
|
|
191 | (5) |
|
Experimental Designs and Internal Validity |
|
|
192 | (1) |
|
Two-Group Experimental Designs |
|
|
193 | (2) |
|
Probabilistic Equivalence |
|
|
195 | (1) |
|
Random Selection and Assignment |
|
|
196 | (1) |
|
Classifying Experimental Designs |
|
|
196 | (1) |
|
|
|
197 | (8) |
|
The Basic 2 x 2 Factorial Design |
|
|
198 | (1) |
|
|
|
199 | (1) |
|
|
|
199 | (1) |
|
|
|
200 | (2) |
|
Factorial Design Variations |
|
|
202 | (1) |
|
|
|
202 | (1) |
|
|
|
203 | (1) |
|
Incomplete Factorial Design |
|
|
204 | (1) |
|
|
|
205 | (2) |
|
How Blocking Reduces Noise |
|
|
206 | (1) |
|
|
|
207 | (4) |
|
How Does a Covariate Reduce Noise? |
|
|
208 | (3) |
|
|
|
211 | (1) |
|
Hybrid Experimental Designs |
|
|
211 | (3) |
|
The Solomon Four-Group Design |
|
|
211 | (2) |
|
Switching-Replications Design |
|
|
213 | (1) |
|
|
|
214 | (1) |
|
Quasi-Experimental Design |
|
|
215 | (22) |
|
The Nonequivalent-Groups Design |
|
|
216 | (5) |
|
|
|
216 | (1) |
|
The Bivariate Distribution |
|
|
217 | (1) |
|
|
|
217 | (1) |
|
|
|
218 | (1) |
|
|
|
219 | (1) |
|
|
|
220 | (1) |
|
|
|
220 | (1) |
|
The Regression-Discontinuity Design |
|
|
221 | (7) |
|
|
|
222 | (2) |
|
The Logic of the RD Design |
|
|
224 | (2) |
|
The Role of the Comparison Group in RD Designs |
|
|
226 | (1) |
|
The Internal Validity of the RD Design |
|
|
226 | (2) |
|
The RD Design and Accountability |
|
|
228 | (1) |
|
Statistical Power and the RD Design |
|
|
228 | (1) |
|
|
|
228 | (1) |
|
Other Quasi-Experimental Designs |
|
|
228 | (7) |
|
|
|
228 | (1) |
|
The Separate Pre-Post Samples Design |
|
|
229 | (1) |
|
The Double-Pretest Design |
|
|
230 | (1) |
|
The Switching-Replications Design |
|
|
231 | (1) |
|
The Nonequivalent Dependent Variables (NEDV) Design |
|
|
231 | (1) |
|
The Pattern-Matching NEDV Design |
|
|
232 | (2) |
|
The Regression Point Displacement (RPD) Design |
|
|
234 | (1) |
|
|
|
235 | (2) |
|
|
|
237 | (18) |
|
Designing Designs for Research |
|
|
238 | (10) |
|
Minimizing Threats to Validity |
|
|
238 | (2) |
|
|
|
240 | (1) |
|
|
|
240 | (1) |
|
|
|
241 | (4) |
|
A Simple Strategy for Design Construction |
|
|
245 | (1) |
|
An Example of a Hybrid Design |
|
|
245 | (2) |
|
The Nature of Good Design |
|
|
247 | (1) |
|
Relationships among Pre-Post Designs |
|
|
248 | (2) |
|
Contemporary Issues in Research Design |
|
|
250 | (4) |
|
|
|
250 | (1) |
|
The Case for Tailored Designs |
|
|
251 | (1) |
|
The Crucial Role of Theory |
|
|
251 | (1) |
|
Attention to Program Implementation |
|
|
252 | (1) |
|
The Importance of Quality Control |
|
|
252 | (1) |
|
The Advantages of Multiple Perspectives |
|
|
252 | (1) |
|
Evolution of the Concept of Validity |
|
|
253 | (1) |
|
Development of Increasingly Complex Realistic Analytic Models |
|
|
253 | (1) |
|
|
|
254 | (1) |
| Part 5 Analysis |
|
255 | (90) |
|
|
|
257 | (24) |
|
|
|
258 | (8) |
|
Threats to Conclusion Validity |
|
|
259 | (1) |
|
Finding No Relationship When There Is One (or, Missing the Needle in the Haystack) |
|
|
259 | (1) |
|
Finding a Relationship When There Is Not One (or Seeing Things That Aren't There) |
|
|
260 | (1) |
|
Problems That Can Lead to Either Conclusion Error |
|
|
261 | (1) |
|
|
|
262 | (3) |
|
Improving Conclusion Validity |
|
|
265 | (1) |
|
|
|
266 | (2) |
|
|
|
266 | (1) |
|
Checking the Data for Accuracy |
|
|
266 | (1) |
|
Developing a Database Structure |
|
|
267 | (1) |
|
Entering the Data into the Computer |
|
|
267 | (1) |
|
|
|
267 | (1) |
|
|
|
268 | (11) |
|
|
|
269 | (1) |
|
|
|
270 | (1) |
|
Dispersion or Variability |
|
|
271 | (1) |
|
|
|
272 | (1) |
|
|
|
272 | (3) |
|
Calculating the Correlation |
|
|
275 | (2) |
|
Testing the Significance of a Correlation |
|
|
277 | (1) |
|
|
|
277 | (1) |
|
|
|
278 | (1) |
|
|
|
279 | (2) |
|
Analysis for Research Design |
|
|
281 | (36) |
|
|
|
282 | (1) |
|
|
|
282 | (5) |
|
The Two-Variable Linear Model |
|
|
282 | (2) |
|
Extending the General Linear Model to the General Case |
|
|
284 | (1) |
|
|
|
285 | (2) |
|
|
|
287 | (7) |
|
|
|
287 | (2) |
|
Statistical Analysis of the t-Test |
|
|
289 | (3) |
|
Factorial Design Analysis |
|
|
292 | (1) |
|
Randomized Block Analysis |
|
|
293 | (1) |
|
|
|
293 | (1) |
|
Quasi-Experimental Analysis |
|
|
294 | (21) |
|
Nonequivalent Groups Analysis |
|
|
294 | (1) |
|
|
|
295 | (1) |
|
|
|
296 | (4) |
|
|
|
300 | (3) |
|
|
|
303 | (1) |
|
Regression-Discontinuity Analysis |
|
|
304 | (1) |
|
Assumptions in the Analysis |
|
|
304 | (1) |
|
The Curvilinearity Problem |
|
|
305 | (1) |
|
|
|
306 | (2) |
|
|
|
308 | (1) |
|
|
|
308 | (3) |
|
|
|
311 | (2) |
|
Regression Point Displacement Analysis |
|
|
313 | (2) |
|
|
|
315 | (2) |
|
|
|
317 | (28) |
|
|
|
318 | (2) |
|
|
|
320 | (5) |
|
|
|
321 | (1) |
|
|
|
321 | (1) |
|
|
|
321 | (1) |
|
|
|
321 | (1) |
|
|
|
321 | (1) |
|
|
|
321 | (1) |
|
|
|
321 | (1) |
|
|
|
322 | (1) |
|
|
|
322 | (1) |
|
|
|
322 | (1) |
|
|
|
322 | (1) |
|
|
|
323 | (1) |
|
Reference Citations in the Text of your Paper |
|
|
323 | (1) |
|
Reference List in Reference Section |
|
|
323 | (2) |
|
|
|
325 | (1) |
|
|
|
325 | (1) |
|
|
|
325 | (1) |
|
|
|
325 | (20) |
| Glossary |
|
345 | (10) |
| Index |
|
355 | |