1st April 2009 at 11:42 pm #6254Caitlin HughesParticipant
Does anyone know of good resources regarding how to conduct comparative policy analyses? We would like to know how to conduct rigorous state by state comparisons into the impacts of drug policies. Any relevant resources on for example cross-national comparison methods would be appreciated.4th April 2009 at 6:15 am #6258Berkay OzcanParticipant
Can you give a little bit more information about what kind of data you will be using? Are you planning to compare policies or policy outcomes? How are these outcomes measured? Unit of analysis and your sample size make a huge difference in the type of comparative analysis you might want to use. For example, if you have a cross-national (multi-country) surveys where you have individual observations with large sample sizes (i.e.many individuals, on a few/ or many countries) then you can apply standard econometric techniques . It is even better if you have these surveys over time (or panels).Especially Differences- in Differences methods are pretty standard for policy evaluations. But if your unit of analysis is countries and all your data is at the aggregate level then you might want to get as longer periods as possible. If this is the case for you, then I can suggest you some key readings about cross-national comparative research (i.e. pooled cross-sectional time series analysis).5th April 2009 at 9:53 pm #6257Caitlin HughesParticipant
At this stage we are developing a methodology for undertaking state by state comparative policy analyses within Australia, hence it is a little difficult to get into the specifics. (There are 8 states within Australia). Nevertheless our preference is to compare different policies or policy types. For example to compare states with cannabis decriminalisation versus those with cannabis criminalisation or states with broad availability of needle syringe programs versus those with restrictive availability. That said if states have the same policy we would compare on the basis of outcomes. For the two examples cited our main outcomes are prevalence of cannabis use and prevalence of daily use (for the cannabis decrim vs crim comparison) and prevalence of HIV (for the needle syringe program availability comparison). Our preference is to assess the impacts using time series analysis. For example data on the prevalence of cannabis use had been collected every four years in Australia since the first state introduced cannabis decriminalisation. We therefore have 8 waves of data.
The challenge for us is to see how different disciplines have approached comparative policy analyses, what types of data analysis have been used, and how best to deal with issues such as how to control for relevant confounders.8th April 2009 at 1:41 am #6256Berkay OzcanParticipant
OK, I think you have an aggregate data (such as percentage of cannabis users in a population) rather than individual data. I don’t agree with your sentence “…That said if states have the same policy we would compare on the basis of outcomes.” It is great that you have some control states where you don’t have the same policy. So you can attribute better the changes in outcomes is to a specific policy. If the states had the same policy then the differences in outcomes would be most probably due to something else than the policies adopted everywhere equally. But you have 8 waves for 8 states which makes 64 observations. I still think differences-in- differences is the ideal research for an implementation of a law. Although your case sounds like an ideal one because there’s a nice variation both at the cross-sectional( e.g among states) and across time (over a long period), with few observations and four years time gap, it can become tricky. You can read more about it in many standard econometrics text books. The most intuitive one I found is the chapter 5 of “Mostly Harmless Econometrics” by Joshua Angrist and Steffen Pischke. Here I paste the first paragraph of a famous paper on this method by Bertrand, Dufflo and Mullainathan (2004) Quarterly Journal of Economics Vol 119. :
” Differences-in-Differences (DD) estimation has become an increasingly popular way to estimate causal relationships. DD estimation consists of identifying a specific intervention or treatment (often the passage of a law). One then compares the difference in outcomes after and before the intervention for groups affected by the intervention to the same difference for unaffected groups. For example, to identify the incentive effects of social insurance, one might first isolate states that have raised unemployment insurance benefits. One would then compare changes in unemployment duration for residents of states raising benefits to residents of states not raising benefits.” p.249
I hope you find this helpful.
Berkay28th October 2009 at 11:06 am #6255Ilonka Sargent-CapronMember
what about CULTURE and the context in which the documents are used..and how YOU interpret them? What about the the events out of the ordinary that may have been a contributing factor(s)?
Just some food for thought.
- The forum ‘Default Forum’ is closed to new topics and replies.