Stephen Quiz Week 6

Question 1: Data Interpretation

Below is a chart showing the pre-treatment GDP trends for California and several potential control states:

Based on this pre-treatment data, which state would make the best single control unit for a Synthetic Control Method analysis of California?

Nevada Oregon Washington Arizona

Question 2: Multiple Choice

Which treatment effect explains why a tutoring program works better for STEM students than humanities students?

Average Treatment Effect (ATE) Heterogeneous Treatment Effect (HTE) Local Average Treatment Effect (LATE) Treatment Effect on the Treated (TT)

Question 3: Multiple Choice

In the context of SHAP values, what does a negative SHAP value for a feature indicate?

The feature is not important for the prediction The feature decreases the predicted value compared to the baseline The feature has a negative correlation with other features The model failed to learn the relationship for this feature

Question 4: Multiple Choice

Using our policy intervention dataset, if California implemented a policy (Treatment = "Yes") and saw GDP increase from 3.2 to 3.8, while Oregon had no policy (Treatment = "No") and saw GDP increase from 3.1 to 3.3, what is the naive estimate of the treatment effect?

0.4 0.6 0.5 0.2

Question 5: Multiple Choice

In a Synthetic Control Method, if Nevada contributes 30%, Oregon contributes 40%, and Washington contributes 30% to your synthetic California, what would be the synthetic prediction if these states have unemployment rates of 4.2%, 3.9%, and 4.5% respectively?

4.2% 4.18% 4.0% 3.9%

Question 6: Data Analysis

The following table shows treatment effects for a job training program across different demographic groups:

Demographic Group	Sample Size	Treatment Effect (Income Increase %)	Standard Error	p-value
Women under 30	245	14.2%	2.1%	0.001
Women over 30	278	9.5%	1.8%	0.008
Men under 30	231	6.8%	2.4%	0.015
Men over 30	252	3.2%	1.9%	0.089
Overall Average	1006	8.4%	1.1%	0.003

What does this data tell us about the Conditional Average Treatment Effects (CATEs) of the job training program?

The treatment effects are homogeneous across all demographic groups The treatment has stronger effects for women than for men, with the strongest effect for younger women Only the overall average treatment effect is statistically significant (p < 0.05) The program is equally effective for both genders but age is the primary factor affecting outcomes

Question 7: Chart Analysis

Consider this chart showing the actual outcome for a treated unit (red line) compared with two different counterfactual estimates (blue and green lines):

Based on the data shown in this chart, which of the following statements is most accurate?

The treatment effect estimate would be similar regardless of which counterfactual method is used The treatment had no effect according to the difference-in-differences method The synthetic control method suggests a larger treatment effect than the difference-in-differences method The actual outcome shows that the treatment increased hospital admissions

Question 8: Multiple Choice

If a policy intervention lowered unemployment from 5.1% to 4.8% in treated regions, but would have naturally decreased to 5.0% without intervention (according to synthetic control), what is the actual treatment effect?

-0.3% -0.2% -0.1% -0.5%

Question 9: Multiple Choice

In the context of LATE (Local Average Treatment Effect), what do we mean by "compliers"?

People who always take the treatment regardless of encouragement People who take the treatment only when encouraged to do so People who never take the treatment regardless of encouragement People who comply with all study protocols

Question 10: Chart Comparison

The charts below compare the Synthetic Control Method (SCM) and Difference-in-Differences (DiD) approaches for estimating the effect of a minimum wage increase on employment levels:

Based on these charts, which statement is most accurate regarding the advantages of the Synthetic Control Method compared to Difference-in-Differences?

The SCM produces smaller estimated treatment effects The SCM uses pre-treatment data more efficiently to create a better-matched counterfactual The SCM requires fewer data points to implement The SCM does not rely on the parallel trends assumption

Question 11: Multiple Choice

Using our dataset, if both California (treated) and Washington (untreated) had GDP growth of 0.2% before policy implementation, and after implementation California grew by 0.5% while Washington grew by 0.3%, what is the difference-in-differences estimate?

0.2% 0.3% 0.1% 0.4%

Question 12: SHAP Analysis

The following SHAP (SHapley Additive exPlanations) plot shows the contribution of various features to house price predictions:

Based on the SHAP values displayed in this visualization, which feature contributes the most to increasing the house price prediction above the base value?

Square Footage Location Fireplace Absence of a Pool

Question 13: External Validity Analysis

Below is data from four different studies of the same educational intervention conducted in different settings:

Study	Location	Population	Sample Size	Effect Size (Std. Deviations)	95% Confidence Interval
Study A	Urban USA	Low-income	1,250	0.42	[0.36, 0.48]
Study B	Rural USA	Mixed income	850	0.29	[0.22, 0.36]
Study C	Urban Kenya	Low-income	1,100	0.68	[0.61, 0.75]
Study D	Urban India	Low-income	950	0.71	[0.63, 0.79]

Based on this data, which conclusion about the external validity of this educational intervention is best supported?

The intervention works consistently across all contexts with similar effect sizes The intervention appears more effective in developing countries than in the United States The intervention works best in urban settings regardless of country The intervention has no statistically significant effect in rural settings

Question 14: Synthetic Control Assumptions

Below is a graph showing pre-treatment fit for a Synthetic Control Method analysis of a tax policy implemented in State A:

Based on this pre-treatment fit chart, which key assumption of the Synthetic Control Method is most likely violated in this analysis?

The donor pool must contain at least 30 control units The outcome variable must follow a normal distribution The synthetic control must closely match the treated unit in the pre-treatment period Treatment must be randomly assigned among potential control units

Question 15: True/False

The Synthetic Control Method (SCM) requires a perfect pre-treatment fit between the treated unit and its synthetic counterpart.

True False

Question 16: True/False

The Average Treatment Effect (ATE) and the Treatment Effect on the Treated (TT) are always identical.

True False

Question 17: True/False

SHAP values always sum up to the difference between the model's prediction and the average prediction over the entire dataset.

True False

Question 18: Scenario Question

Consider a job training program that shows the following Conditional Average Treatment Effects (CATE) across different demographic groups:

Based on this CATE diagram, which of the following decisions would be most appropriate if resources are limited?

Target the program equally across all age groups Focus resources primarily on the 25-40 age group, followed by under 25 Focus resources primarily on the over 55 age group as they need the most help Cancel the program as the effects are too heterogeneous

Question 19: Scenario Question

A government implemented a tax credit for electric vehicles and wants to assess its impact on EV adoption rates. The CATE analysis shows:

Based on these CATEs, what can we conclude about the tax credit policy?

The policy is most effective for its intended purpose of increasing EV adoption The policy disproportionately benefits high-income groups who would likely purchase EVs anyway The policy shows uniform benefits across all income groups The tax credit has no effect on EV adoption

Question 21: Multiple Choice

In the context of counterfactual analysis, what does the statement "What would happen to house prices if every home had a fireplace?" represent?

A descriptive analysis of current market conditions A predictive analysis of future market trends A causal intervention analysis examining the impact of a hypothetical policy or change A correlation study between fireplace presence and housing prices

Question 22: Scenario Question

A housing market analysis produced the following counterfactual visualization. The blue histogram shows the current distribution of house prices in a neighborhood, while the orange histogram shows the counterfactual distribution if every house without a fireplace had one installed:

Based on this counterfactual analysis, which of the following conclusions is most accurate?

Adding fireplaces would have no significant effect on the housing market Adding fireplaces would primarily benefit lower-priced homes Adding fireplaces would shift the entire distribution upward, with the median price increasing by approximately $20,000 Adding fireplaces would decrease the number of affordable homes in the market

Causal ML & Meta-Learners Quiz - Stephen Quiz Week 6

Student Information

Question 1: Data Interpretation

Question 2: Multiple Choice

Question 3: Multiple Choice

Question 4: Multiple Choice

Question 5: Multiple Choice

Question 6: Data Analysis

Question 7: Chart Analysis

Question 8: Multiple Choice

Question 9: Multiple Choice

Question 10: Chart Comparison

Question 11: Multiple Choice

Question 12: SHAP Analysis

Question 13: External Validity Analysis

Question 14: Synthetic Control Assumptions

Question 15: True/False

Question 16: True/False

Question 17: True/False

Question 18: Scenario Question

Question 19: Scenario Question

Question 21: Multiple Choice

Question 22: Scenario Question

Quiz Results