Introduction: Geocoding | Installation and Setup | tidygeocoder and other relevant packages in R | Nominatim Server via Docker (Optional; Recommended for Private Data) | Data Privacy in Geocoding Services | One Solution: Nominatim Server | Step 1: Download OSM Map Data | Step 2: Set Up Docker | Step 3: Create Folders for File Storage | Step 4: Start Nominatim Server | Shutting Down Nominatim Server | Deleting All Files (Geocoding No Longer Needed) | Example: Geocoding Addresses | Example: Visualizing Locations | Example: Calculating Distance | Example: Data Augmentation with Census Data | 1. Median Income | 2. Educational Attainment | 3. Race and Ethnicity | Example: CCC District Service Areas | Appendix: R and R Package Versions

Introduction: Program Evaluation in Education | Toy Data Set: Writing Center at a Community College | Based on this exploration, students that enrolled in the writing center course have an overall ENG 1 course success rate of R round(mean(writing_center$Main_Course_SuccessFlag[writing_center$WR_Center==1])*100, 1)%, whereas those that did not enroll in the writing center have an overall ENG 1 course success rate of R round(mean(writing_center$Main_Course_SuccessFlag[writing_center$WR_Center==0])*100, 1)%. On the surface, it appears the writing center is marginally effective at increasing course success. | Estimating Propensity Score Weights Using the twang Package | ``` | plot(ps_fit_att, plots='es') ## plots=3 | ``` | plot(ps_fit_att, plots='boxplot') ## plots=2 | ## plot(ps_fit_att, plots=4) ## p-values of mean differences | ## plot(ps_fit_att, plots=5) ## p-values of KS statistics | summary(ps_fit_att$gbm.obj, n.trees=ps_fit_att$desc$es.mean.ATT$n.trees) | # Visualize | d_demo_summ %>% | arrange(desc(Comparison_Type), desc(std.eff.sz)) %>% | group_by(Variable) %>% | mutate(std.eff.sz0=std.eff.sz[Comparison_Type=='Unadj.']) %>% | ungroup %>% | ggplot(data=., mapping=aes(x=fct_reorder(factor(Variable), std.eff.sz0), y=std.eff.sz, group=Variable)) + | geom_point(mapping=aes(size=ifelse(Comparison_Type=='Unadj.', 1, 2))) + | scale_size_continuous(range = c(1,4)) + | geom_line() + | geom_hline(yintercept=0, lty=2) + | coord_flip() + | ylab('Standardized Difference') + | xlab('') + | ggtitle('Writing Center vs. Not') + | theme_tufte(base_family='Arial Narrow') + | theme(text=element_text(size=14)) + | # theme_ipsum(grid=FALSE) + | # scale_color_manual(values=c('slategrey', 'darkblue'), guide=FALSE) + | theme(legend.position='none') + | labs(caption='Small dots: unweighted difference\nLarge dots: weighted difference') | Statistical Inference Using Propensity Score Weights Using the survey Package | Those that enrolled in the writing center are associated with a R round(exp(fit_unw$coef)[2], 1)-fold increase in the odds of course success. | After incorporating the propensity score weights, the writing center is associated with a R round(exp(fit_att$coef[2]), 1)-fold increase in the odds for course success (95% CI: R round(exp(confint(fit_att))[2,1], 1) -- R round(exp(confint(fit_att))[2,2], 1)). | Exercise: Doubly Robust Estimation | Other Considerations In Evaluation | Appendix: R and R Package Versions

Introduction: Survey Non-response Bias | Toy Data Set: Student Satisfaction at a Community College | Based on this exploration, Email_Masked is the common key that could be used to join the two data sets. The response rate for the survey is R round(mean(survey_response$Responded)*100, 0)%. | Modeling Survey Response with twang | ``` | # Visualize | d_demo_summ_trick %>% | arrange(desc(Comparison_Type), desc(std.eff.sz)) %>% | group_by(Variable) %>% | mutate(std.eff.sz0=std.eff.sz[Comparison_Type=='Unadj.']) %>% | ungroup %>% | # Originally full sample vs. responders, let's reverse it. | mutate(std.eff.sz0=-1 * std.eff.sz0 | , std.eff.sz=-1 * std.eff.sz | ) %>% | ggplot(data=., mapping=aes(x=fct_reorder(factor(Variable), std.eff.sz0), y=std.eff.sz, group=Variable)) + | geom_point(mapping=aes(size=ifelse(Comparison_Type=='Unadj.', 1, 2))) + | scale_size_continuous(range = c(1,4)) + | geom_line() + | geom_hline(yintercept=0, lty=2) + | coord_flip() + | ylab('Standardized Difference') + | xlab('') + | # ggtitle('Responded to Survey vs. Not') + | ggtitle('Responders vs. Full Sample') + | theme_tufte(base_family='Arial Narrow') + | theme(text=element_text(size=14)) + | # theme_ipsum(grid=FALSE) + | # scale_color_manual(values=c('slategrey', ivc_color), guide=FALSE) + | theme(legend.position="none") + | labs(caption='Small dots: unweighted difference\nLarge dots: weighted difference') | ``` | Statistical Inference with Survey Weights | Recap | Appendix: R and R Package Versions

Introduction: Instructor- or Section-level Effects in Program Evaluation | Example: GLM Not Accounting for Instructor-level Effects | After adjusting for student demographic and background information, the odds ratio comparing those enrolled in the writing center to those not in the writing center is estimated to be R round(exp(coef(fit_glm))['WR_Center'], 2) (95% CI: R round(exp(confint(fit_glm, 'WR_Center'))[1], 2) -- R round(exp(confint(fit_glm, 'WR_Center'))[2], 2)). | Example: Bayesian Multilevel Model Accounting for Instructor-level Effects | The above results show the odds ratio estimate and credible intervals for the impact of the writing center accounting for both the adjustment variables and the instructor-level effects. That is, after adjusting for the student background characteristics (those listed in the model) and instructor-level effects, those that enrolled in the writing center are associated with a R round(mean(OR_WR_Center), 2)-fold increase in the odds for course success in ENG 1 compared to those not enrolled in the writing center (95% credible interval: R round(quantile(OR_WR_Center, 0.025), 2) -- R round(quantile(OR_WR_Center, 0.975), 2)). | ``` | library(knitr) | d_results <- tibble(Model=c('GLM', 'Bayesian Multi-level Model') | , Odds Ratio: WR Center vs. Not=c(exp(coef(fit_glm))['WR_Center'], mean(OR_WR_Center)) | , Lower Limit=c(exp(confint(fit_glm, 'WR_Center'))[1], quantile(OR_WR_Center, 0.025)) | , Upper Limit=c(exp(confint(fit_glm, 'WR_Center'))[2], quantile(OR_WR_Center, 0.975)) | ) | kable(d_results) | ``` | Illustration: Range in Instructor Effects Based on Model | ``` | # Plot instructor-level alphas's posterior mean and 95% credible interval | ggplot(data = a_df, aes(x = Rank, y = Varying_Intercept)) + | geom_pointrange(aes(ymin = Q2.5, ymax = Q97.5), position = position_jitter(width = 0.1, height = 0)) + | geom_hline(yintercept = mean(a_df$Varying_Intercept), size = 0.5, col = "red") + | # scale_x_continuous("Rank", breaks = seq(from = 0, to = 80, by = 5)) + | # scale_y_continuous(expression(paste("varying intercept, ", alpha[j]))) + | theme_bw( base_family = "serif") | References and Further Reading | Appendix: R and R Package Versions

Introduction: Webscraping | Installation and Setup | Virtualization (VT-x) in System BIOS | Docker and Selenium Server | SelectorGadget Extension for Chrome | RSelenium in R | Rtools to access the zip command in Windows | Summary of Webscraping Workflow | Example: Webscrape Scorecard Data for All Colleges | Exploration and Strategy | Execution | Example: Webscrape Student Success Metrics Data for All Colleges | Identifying the XPath of Web Elements for Clicking via SelectorGadget | Identifying Web Elements Manually in Chrome (To Fix Some XPath's) | Exercise: Webscrape Data Mart's Transfer Velocity Data | Appendix: R and R Package Versions

Introduction | Load DisImpact and toy data set | Execute di_iterate on a data set | Explore resulting summary data set | Visualization (emulating dashboard features) | What is the difference between group_vars and scenario_repeat_by_vars? | Understanding the default parameters in di_iterate, and overriding them | Passing a summarized data set to data and using weight_var | Suppress non-disaggregated results using include_non_disagg_results | PPG reference groups and other parameters | Proportionality index DI threshold | 80% index reference groups and DI threshold | Multiple PPG or DI parameter scenarios in results | FERPA block / suppression | Suppress / disable "Joining, by" messages | Exclude specific scenarios generated by the use of scenario_repeat_by_vars | Parallel execution | di_iterate_dt: efficient in-memory DI calculations using data.table | di_iterate_sql: DI calculations on a database table using the database's SQL engine, especially useful for larger-than-memory data | Appendix: R and R Package Versions

Background on Multi-Ethnicity Categorization | Multi-Ethnicity Data Format | Multi-Ethnicity Example Data Set | Analysis of Multi-Ethnicity Data Using DisImpact | Visualizing in Dashboard Platform | Appendix: R and R Package Versions

Introduction | Load DisImpact and toy data set | Select relevant rows | Execute di_iterate_on_long on a data set | Second layer of disaggregation / Intersectionality | Custom Reference Groups | Additional Information | Appendix: R and R Package Versions

Introduction | Install Package | Load Packages | Load toy student equity data | Percentage point gap (PPG) method | Proportionality index method | 80% index method | When dealing with a non-success variable like drop-out or probation | Transformations on the fly | Calculate DI for many variables and groups | Appendix: R and R Package Versions

Background on Intersectionality | Intersectionality Using DisImpact | Visualizing in Dashboard Platform | Appendix: R and R Package Versions

Install Package | Load Packages | Setup | Obtain access token | Create interSegmentKey's for each student | Query data from CalPASS | Multiple batches (when there are many interSegmentKey's)