Logo

IZA/DFID Short Course on Program Evaluation

Organizers: David Lam (University of Michigan and IZA), Ahmed Elsayed (IZA)
Location:Hilton, Menelik II Ave, Addis Ababa 1164, Ethiopia
Date:December 08, 2018 - December 09, 2018

Event Manager:

Dominik Spitza (IZA)




Teaching Staff: Marco Caliendo (University of Potsdam and IZA)
Stefano Caria (University of Bristol)
Joan Hicks (University of Oklahoma)





Program

Saturday, December 08:

08:00 - 09:00   Registration and Distribution of Learning Materials

Teaching Materials
Stata Licenses, Installation

 
 
09:00 - 09:15   Welcome and Introduction
David Lam (University of Michigan and IZA), Marco Caliendo (University of Potsdam and IZA)
 
 
        Section A: IMPACT EVALUATION
 
09:15 - 10:30   Joan Hicks (University of Oklahoma), Stefano Caria (University of Bristol)

  • What is Evaluation? Why Evaluate?
  • What Questions Do You Want to Answer?
  • Different Questions and Research Strategies and Specific Methods for Evaluating Impact / Potential Outcome Framework
  • Include Practical (own) Examples
  •  
     
    10:30 - 11:00   Coffee Break
     
     
            Section B: EXPERIMENTAL DESIGN/RCT
     
    11:00 - 12:30   Stefano Caria (University of Bristol)
    "B1 How to Design an Experiment"

  • Methods of Randomization
  • Linking Experimental Design and Theory
  • Determining Sample Size Using Parametric and Non-parametric Power Calculations
  •  
     
     
    12:30 - 13:30   Lunch
     
     
    13:30 - 15:00   Stefano Caria (University of Bristol)
    "B2 Analyzing Data from an Experiment"

  • Average Effects
  • Heterogeneity of Treatment Effects
  • CACE (Complier Average Causal Effect)
  • LATE (Local Average Treatment Effect)
  •  
     
     
    15:00 - 15:30   Coffee Break
     
     
    15:30 - 17:00   Joan Hicks (University of Oklahoma)
    "B3 How to Implement an Experiment"

  • Finding Funding
  • Working with Partner Organizations
  • Data Collection (Including Innovative Methods of Data Collection)
  • Pitfalls to Avoid
  •  
     
    19:00       Dinner
     
     

    Sunday, December 09:

    08:30 - 10:00   Joan Hicks (University of Oklahoma)
    "B4 What Lessons to Draw from an Experiment"

  • Cost-benefit Analysis
  • Scale-up and General Equilibrium Effects
  • External Validity and the Importance of Context
  • Meta-analysis
  •  
     
    10:00 - 10:30   Coffee Break
     
     
            Section C: NON-/QUASI-EXPERIMENTAL DESIGNS
     
    10:30 - 12:00   Marco Caliendo (University of Potsdam and IZA)
    "C1 Propensity Score Matching"

  • What is (Propensity Score) Matching?
  • Choosing a Matching Algorithm
  • Assessing the Matching Quality
  • Effect Estimation
  •  
     
     
    12:00 - 13:00   Lunch
     
     
    13:00 - 14:30   Marco Caliendo (University of Potsdam and IZA)
    "C2 Difference-in-Differences"

  • Motivation and Intuition

  • Graphical Illustration and Formalization
  • Did with Regressions

  • Practical Advice and Sensitivity Analysis
  •  
     
    14:30 - 15:00   Coffee Break
     
     
            Section D: PRACTICAL IMPLICATIONS AND ACADEMIC ADVICE
     
    15:00 - 15:45   Girum Abebe (Ethiopian Development Research Institute)
    "Practical Advice for Young Researchers in Developing Countries"
     
    15:45 - 17:00   Marco Caliendo (University of Potsdam and IZA), Stefano Caria (University of Bristol), Joan Hicks (University of Oklahoma),
    David Lam (University of Michigan and IZA), Ahmed Elsayed (IZA), Girum Abebe (Ethiopian Development Research Institute)

  • Hands on Project Work (tbd)
  • Break Out into Groups (tbd)
  • Advice on How to Publish

  •  




    Suggested Reading

    • Impact Evaluation:
      Khandker, S.R., Koowal, G.B. & Samad, H.A. (2010): Handbook of Impact Evaluation : Quantitative Methods and Practices, Chapter 2, World Bank Publications.

    • Difference in Differences:
      Card, D. & Krueger, A.B. (1994): Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania , The American Economic Review 84, 772-793.

    • Matching:
      Caliendo, M. & Kopeinig, S. (2008): Some Practical Guidance for the Implementation of Propensity Score Matching, Journal of Economic Surveys 22, 31-72.
      Dehejia, R.H. & Wahba, S. (1999): Causal Effects in Nonexperimental Studies: Reevaluation of Training Programs, Journal of the American Statistical Association 94, 1053-1062.

    • Randomized Evaluations:
      Khandker, S.R., Koowal, G.B. & Samad, H.A. (2010): Handbook of Impact Evaluation: Quantitative Methods and Practices, Chapter 3, World Bank Publications.
      Duflo, E., Glennester, R. & Kremer, M. (2006): Using Randomization in Development Economics Research: A Toolkit, National Bureau of Economic Research (NBER) Technical Working Paper 333.
      Muralidharan, K. & Niehaus, P. (2017): Experimentation at Scale, National Bureau of Economic Research (NBER) Working Paper 23957.

    • Regression Discontinuity Design:
      Meyersson, E. (2014): Islamic Rule and the Empowerment of the Poor and Pious, Econometrica 82, 229-269.