Psychology 206

Quantitative Methods for Reviewing Research

Fall Semester, 2016


Syllabus for Fall, 2016

Instructor:

Jack L. Vevea (psy206ma@gmail.com)
(Please note that this is a special email address for this class; I will not monitor it after the conclusion of the class. My regular email is jvevea@ucmerced.edu.)
Social Science and Management Building 306a
Office hours: Tuesdays, 1:00-4:00, or by appointment.
Telephone: (209) 658-1706 (but email is usually a much quicker way to reach me)

Text:

Lipsey, M.W. & Wilson, D.B. (2001). Practical Meta-Analysis. Newbury Park, California: Sage Publications.
Assorted handouts and manuals (see course outline).

Meeting times:

We will meet Tuesdays and Thursdays from 10:30 to 11:45 AM in room 272 of the Classroom Building.

Course description:

Quantitative procedures (meta-analysis) for reviewing research findings; techniques for locating and coding research studies, calculating effect sizes, and analyzing study findings.

Course learning goals:

In the class, you will:
  • Study the history of meta-analysis.
  • Learn about search procedures for locating relevant literature.
  • Learn about coding and assessment of coding accuracy.
  • Study methods for calculation of effect sizes.
  • Learn about basic (mean-only) meta-analysis.
  • Study the distinction between fixed- and random-effects models.
  • Learn graphical procedures related to meta-analysis.
  • Learn how to assess moderators of effect magnitude.
  • Survey methods for addressing the problem of publication bias.
  • Get an overview of advanced methods, including network meta-analysis, multi-level meta-analysis, multivariate models, and Bayesian approaches.
  • Learn how to present a meta-analysis.
  • Course learning outcomes:

    By the end of the class, you will be able to:
  • Describe the history and advantages of meta-analysis.
  • Formulate a meta-analytic question.
  • Search the literature for studies about that question.
  • Code effect sizes and other information from the studies.
  • Compute basic and selected advanced meta-analytic statistics:
  • Combine effect sizes using fixed- and random-effects models.
  • Estimate categorical and continuous moderation models.
  • Employ and report advanced models such as multivariate analyses and multilevel models.
  • Use several methods to detect and adjust for publication bias.
  • Use appropriate graphics to present resulsts.
  • Orally present the meta-analysis in a professional format.
  • Write a manuscript in the style expected for publication.
  • Evaluation:

    Your grade in this class will be based on a combination of your written class project (50%), in-class presentation of your project (30%), and participation in periodic in-class reporting of progress on the project (20%).

    Academic Integrity

    Students should be familiar with University policies on academic integrity. You will find relevant information here.

    In the overall context of that policy, the following information is specific to this class:

    Some assignments require that you work cooperatively with other class members. I strongly encourage cooperative work on the computational aspects of assignments or the class project. However, you should work independently on discussion and interpretation. The words you submit in your written assignments should be entirely your own.

    Students with Disabilities

    UC Merced has a variety of services available to accommodate students with disabilities. Information is available here.

    How to submit assignments

  • You should submit periodic assignments by emailing a document to the special email address) for this class.
  • Include the assignment number in your subject line.
  • Attach a file with a name that includes the assignment number and your name.
  • For example, if I were submitting Assignment One, I would type "Assignment One" on the subject line and I would attach a Word file named "Assignment One Jack Vevea.doc".
  • Some of you may prefer to work in LaTex and will submit a pdf. Whichever format you choose, please submit only one file!
  • Incorporate any graphics you include into that file.
  • Under no circumstances submit several files zipped together.

  • Course Outline

    The course outline is somewhat less detailed than is usual for my syllabi. That is because this is a new course for me, and details of the timing and flow are difficult to predict. The outline will be updated as we proceed through the semester.
    August 25
    No class meeting.
    August 30 - September 1
    Initial class meeting: introductions, overview of the class, using the class web page, discussion of the class project. History of meta-analysis. Problems with narrative reviews and vote counting. Time and costs. Gathering studies.
    Reading: L & W Chapters 1-2.
    Assignment One: write up the equivalent of what you just said in class: your program, your interests, your advisor, etc.
    Here are student responses.
    September 6-8
    Introduction to R and metafor. Initial discussion of project ideas. Coding manual and process.
    Reading: L & W Chapter 4.
    Assignment Two: By Sunday, September 11, send to the class email account a brief description of the meta-analysis you are contemplating for this class. Include a clear statement of the question of interest, and present some preliminary evidence that quantitative studies of this issue exist (probably from a search of bibliographic database such as psycinfo). Explain why this topic interests you. (Remember that your assignments will be shared in class. So, for example, you might not want to explain that you are interested in studies of recidivism among convicted axe murderers because you are a convicted axe murderer.)
    Here are student responses.
    September 13-15
    Discussion of draft coding manuals. Inter-rater reliability.
    Reading: Handout on inter-rater reliability (found here).
    September 20-22
    Effect sizes.
    Reading: L & W Chapter 3.
    Here are final project descriptions with comments added in class.
    Assignment Three is available here. See the assignment text for details of due dates, but be aware that the first installment is due this week.
    September 27-29
    Preliminary analyses. Graphics. Dependent effect sizes.
    Reading: L & W Chapters 5-6.
    October 4-6
    Combining effect sizes with fixed- and random-effects models.
    Reading: L & W Chapter 7.
    Here is a link to a pdf showing formulas for conditional variances for various non-categorical effect sizes.
    October 11-13
    Combining effect sizes with fixed- and random-effects models (continued). Diagnostics, assessing influence.
    Reading: L & W Chapter 8.
    October 18-20
    Assessing moderators of effect size. Multilevel models.
    Reading: Poke around in the metafor manual.
    October 25-27
    Publication bias.
    Reading: Poke around in the weightr manual.
    November 1-3
    Other sensitivity analyses. Power.
    November 8-10
    Reporting standards. Discussion of problems and issues in class projects.
    Reading: APA reporting standards (handouts here).
    November 15-17
    Discussion of problems and issues in class projects. Place holder to recognize the fact that we will undoubtedly be behind schedule at this point.
    November 22
    Class and office hours cancelled.
    November 24
    Thanksigiving holiday; no class meeting.
    November 29-December 1
    Presentation of class projects.
    December 6-8
    Presentation of class projects.