STHDA - Machine learning essentials

A very good guide book on STHDA about machine learning.

Supervised learning

regression analysis– predict continuous variable

Different methods for regression analysis:

• Ordinary least squares (Chapter @ref(linear-regression))
  • Simple linear regression
  • Multiple linear regression
• Model selection methods:
  • Best subsets regression (Chapter @ref(best-subsets-regression))
  • Stepwise regression (Chapter @ref(stepwise-regression))
• Principal component-based methods (Chapter @ref(pcr-and-pls-regression)):
  • Principal component regression (PCR)
  • Partial least squares regression (PLS)
• Penalized regression (Chapter @ref(penalized-regression)):
  • Ridge regression
  • Lasso regression

Regression Analysis

• Linear Regression

• Interaction Effect in Multiple Regression

• Regression with Categorical Variables: Dummy Coding

• Nonlinear Regression: Polynomial and Spline Regression Models

• Simple Linear Regression

• Multiple Linear Regression

• Model Predictions and Confidence Intervals

Regression Model Diagnostics

• Linear Regression Assumptions and Diagnostics

• Multicollinearity and VIF

• Confounding Variable

Regression Model Validation

• Regression Model Accuracy Metrics: R-square, AIC, BIC, Cp and more

• Cross-Validation

• Bootstrap Resampling

Model Selection Methods

• Best Subsets Regression

• Stepwise Regression

• Penalized Regression: Ridge, Lasso & Elastic Net

• Principal Component and Partial Least Squares Regression

Classification– predict class/group variable

• Logistic regression, for binary classification tasks (Chapter @ref(logistic-regression))
• Stepwise and penalized logistic regression for variable selections (Chapter @ref(stepwise-logistic-regression) and @ref(penalized-logistic-regression))
• Logistic regression assumptions and diagnostics (Chapter @ref(logistic-regression-assumptions-and-diagnostics))
• Multinomial logistic regression, an extension of the logistic regression for multiclass classification tasks (Chapter @ref(multinomial-logistic-regression)).
• Discriminant analysis, for binary and multiclass classification problems (Chapter @ref(discriminant-analysis))
• Naive bayes classifier (Chapter @ref(naive-bayes-classifier))
• Support vector machines (Chapter @ref(support-vector-machine))
• Classification model evaluation (Chapter @ref(classification-model-evaluation))

Logistic Regression

• Stepwise Logistic Regression

• Penalized Logistic Regression: Ridge, Lasso and Elastic Net

• Logistic Regression Assumptions and Diagnostics

• Multinomial Logistic Regression

• Discriminant Analysis

• Naive Bayes Classifier

• SVM Model: Support Vector Machine

Evaluation of Classification Model Accuracy

• Building a classification model

• Overall classification accuracy

• Confusion matrix

• Precision, Recall and Specificity

• ROC curve

  • Computing and plotting ROC curve
  • Multiple ROC curves

• Multiclass settings

  ​

Advanced machine learning methods

• KNN: K-Nearest Neighbors

• CART Model: Decision Tree

• Bagging and Random Forest

• Gradient Boosting in R Using XGBOOST

Unsupervised learning

principal component analysis

• Required R Packages for Principal Component Methods

• PCA - Principal Component Analysis Essentials

• PCA in R Using Ade4

• Principal Component Analysis: prcomp vs princomp

• Multidimensional Scaling

• Correspondence Analysis: Theory and Practice

• CA - Correspondence Analysis

• Correspondence Analysis in R: Million Ways

• MCA - Multiple Correspondence Analysis

• HCPC - Hierarchical Clustering on Principal Components

• MFA - Multiple Factor Analysis

• FAMD - Factor Analysis of Mixed Data

Cluster analysis

Part I. Cluster Analysis Basics:

• Data Preparation and Essential R Packages for Cluster Analysis
• Clustering Distance Measures Essentials

Part II. Partitioning Clustering methods:

• K-Means Clustering Essentials
• K-Medoids Essentials: PAM clustering
• CLARA - Clustering Large Applications

Part III. Hierarchical Clustering:

• Agglomerative Clustering
  • Algorithm and steps
  • Verify the cluster tree
  • Cut the dendrogram into different groups
• Divisive Clustering
• Compare Dendrograms
  • Visual comparison of two dendrograms
  • Correlation matrix between a list of dendrograms
• Visualize Dendrograms
  • Case of small data sets
  • Case of dendrogram with large data sets: zoom, sub-tree, PDF
  • Customize dendrograms using dendextend
• Heatmap: Static and Interactive
  • R base heat maps
  • Pretty heat maps
  • Interactive heat maps
  • Complex heatmap
  • Real application: gene expression data

Part IV. Clustering Validation and Evaluation Strategies :

• Assessing Clustering Tendency
• Determining the Optimal Number of Clusters
• Cluster Validation Statistics
• Choosing the Best Clustering Algorithms
• Computing p-value for Hierarchical Clustering

Part V. Advanced Clustering:

• Hierarchical K-means Clustering
• Fuzzy Clustering
• Model-Based Clustering
• DBSCAN: Density-Based Clustering