Note 8 Note 12

SAS – Statistical Analysis System Boxplot with Proc Sgplot

4 main windows:  Boxplot summarizes numeric variable using median, quartiles, and potential outliers
1. Editor: For writing, editing, submitting code  Vertical boxplot created using vbox statement, and a horizontal boxplot is created using hbox statement
2. Log: Displays messages, error, notes about code execution  Category = to compare distributions across groups
3. Explorer: Manage libraries, files, datasets virtually  Diamond in each box is the mean of the group
4. Output/graph: Shows results and output from procedures  SAS computes 1.5x IQR Upper and Lower Fence values, only extends to max and min if no outliers
Three ways to run SAS Codes:  Group=option can show multiple boxplots for comparison
1. Run  Also used to create separate-colored boxplots within each category as a second level of grouping
2. Submit – Running Man Scatter Plots with Proc Sgplot
3. F8  Scatter plot shows relationship between 2 numeric variables
Basic Syntax Rules  Each point represents one observation in datasheet
 Every line should end with a semicolon;  Grouping variable uses different colors for different groups – grouping helps rveal whether relationship differs across categories
 Keywords are not case sensitive, DATA, Data, data all the same  Markerrattrs=option controls appearance of points
 Comments  Common options are symbols = sets marker shape such as circle, circle filled square, triangle
 Color = sets the color of markers such as blue, red or green
 Size = sets marker size in points
Two main components in SAS:  Opacity = controles markers transparency 0(transparent)-1(solid)
1. “Data” Step: Used to load, manipulate, manage data  Weight = adjusts thickness of marker outline
2. Proc Step: Processes data, runs analyses, outputs  Outline color = sets outline color
 Each step beginning with data or proc must end withCombining
run Boxplots and Scatterplots
 Semicolon that ends datalines section should be placed on its own line
 after
Boxplot
the shows
last data
distribution
record of each group, adding scatterplot shows each individual observation
Data Step  Jitter Option spreads points horizontally so they do not overlap
 Read/input data Bubble Plots with Prog sgplot
 Modify and create new variables  Bubble plot extends a scatterplot by using the size (counts) of each market to represent a third numeric variable
 Filter or merge datasets  Useful when we want to show relationship between two variables while showing an additional measure
 Prepare data for analysis – proc Regression Lines
Use Proc to Print  Regression statement fits a simple linear regression model and overlays the fitted line on the scatterplot
 Proc print data = example_data; run;  Useful when relationship is approximately linear
 Proc Print prints data table in output window  Lineattrs =:
 Data=example_data specifies displayed data set  Color sets line color
Reading Data in Data Step  Thickness controles line thickness
 Add two new variables with a data set:  Pattern Controls line thickness
 Pattern changes line style
 Cli Option adds prediction limits for individual predicted values
 Clm option adds confidence limits for mean predicted value
 Degree option fits a polynomial regression curve instead of a straight line (default is 1)
 Cli clm adds prediction and confidence limit
 Confidence limits show range in estimating mean response at each weights value
 Data example_new sets name of new dataset  Prediction limit shows range where we expect individual observations to fall at that weight value
 Data example_data replaces old example with new Polynomial Fitted Lines
 Set example_data reads all observations from data set  Confidence limits form a narrower band around the fitted regression line
 Example_data – set statement bridges between existing and new data Higher degree polynomial may overfit data
Keep and Drop Options LOESS Smoothing
 Keep = and drop = control variables kept or dropped from new data set Locally estimated scatterplot smoothing used to fit a smooth curve that captures nonlinear patterns in data
Rename Options  LOESS fits many local regressions along the x-axis rathe than a single global equation – loess x=wt, y=mpg
 Dataset renamed can replace variables, new dataset  Does not assume a specific polynomial degree, but can serve as a visual guide for choosing a appropriate polynomial degree
Where Option  LOESS has many of same customization as regression, but no cli or clm
 Where = Filters observations read into the data  Smooth = option controles how much the LOESS curve smoothes the data by choosin how much of the data is used for each local fit
 First observations and observations (First obs, obs), specify rows SAS  should
Larger read
smooth
from
= input
value data
meanssetmore smooth curve because it uses more data, and vice versa
 First obs= tells SAS where to start reading observations  .5 default value, usually between .2 and 1.0
 Obs= tells SAS where to stop reading observations  LOESS best when relationship is smooth but not exactly linear
 Obs= max tells SAS to read every observation  Does not extrapolate well beyond data range
Missing Value  Sensitive to outliers
 A period . represents missing numerical value  Not for formal analysis; preliminary
 SAS aligns variables by name  Both reg and loess statement support group =
 If variable exists in one dataset but not the other, SAS creates that variable
 Scatter
forand
the LOESS
missing need
dataset
same
andgroup
fills with
= option
missingfor
values
points and fitted lines to match in color and group definition
Reading Multiple Lines Note 13
 @@ tells SAS to allow multiple rows of data on each line The Proc Reg Step
Importing Data from web in SAS  Proc Reg fits linear regression models and relates a quantifiable response variable to one or more predictor variables
 Use filename with URL option assigning link  Reports overall model fit stats, error variance estimate, r-square value and adjusted r-square
 Use Proc import to read the data into SAS  Computes diagnostic stats. Help detect unusual, outlying, influential observations
 File specifies file reference created by filename  Creates diagnostic plots, like residual and normal probability plots when ODS graphics turned on
 Out names the output dataset Basic Proc sgplot syntax
 Dbms defines file format as csv  Data=tells data to use
Note 9  Ends with run; tells SAS to run the code
Data and Proc Steps  New piece model statement – specifies regression equation
 Proc Import to read external data files into SAS  Response Variable left side = in model
 Proc Print display contents of dataset in table  One or more predictor variables right side = in model statement
Proc Means Step Proc Sgplot Output
 Used to compute descriptive statistics for numeric variables in a dataset
 Proc Reg Produces ANOVA (Analysis of Variance) Table
 Reports number of observations, mean, standard deviation, min/max,  for
Separates
each numeric
variationvariable
in response explained by model and a part left in the errors
 data=option specifies which dataset to analyze, if omitted SAS will use
 most
ANOVA recently
explains
creates
how well regression model explains variability in mpg
Var Statement ANOVA Table
 Selects Specific Numeric Variables to include in Proc Means summary
 F value compares model mean square to error mean square and measures how strong regression relationship is
 Focus on a few variables of interest  Pr > F gives p values for testing null hypothesis that all slope coefficient are zero
The Proc Step  F-test provides overall test of goodness of fit
 Can request specific stats in proc means by listing after procedure statement
 Reports r-square and adjusted r-square as measures of model fit
Class Statement  R-square coefficient of determination, measures proportion of total response variation explained by regression model
 Allows us to compute summary stats for groups defined by a categorical
 Retio
variable
of sum of squares: 1-sserror/sstotal = ssmodel/sstotal
 When Class Statement used, SAS produces separate descriptive stats forAdjusted
each category
r-squared
of class
modifies
groups
r-squared to account for number of predictors – more for comparison across models with different number of predictors, unadjusted r^2 increases with predictors even without improving fit
The By Statement Parameter Estimates
 Also group stats  Show fitted regression coefficients
 Separate table of results for each group  Estimate column gives estimated value of regression coefficient that best fits data in least squares sense
 Useful for separate, clearly separated output for each level of a grouping
 Standard
variableerror column reports estimated standard deviation of sampling distribution of coefficient estimate
 Before using by, must sort dataset by grouping variable  T value column shows test statistics for testing whether true coefficient is equal to zero
 Separate tables, easier to see for each group  Pr > |t| column gives p value for that t test, which measures the evidence against the null hypothesis that the coefficient is zero
Proc Freq Step Diagnostic Plots
 Create frequency tables for categorical variables  Uses residual plots and influence plots to check the assumptions of the regression model
 Proc Freq reports frequency count and % for each level of the variables
 Each small plot focuses on a different model fit aspect, like linearity, variance, normality
Multiple
 Table statement specifies which variables or combinations of variables Linear
to include in Regression
frequency analysis
Two-way Frequency Tables  Output has the same types of tables; anova, diagnostic, regression
 Tables statement in Proc Freq can include 2 variables to produce two-way
 Slope
contingency
for the weight
tableis the expected change in miles per gallon expected for a one unit increase in weight, horsepower fixed
 Helps examine relationship between two categorical variables  Slop for horsepower is the expected change in miles per gallon for a one unit increase in horsepower, weight is fixed
 Table cyl*gear - * symbol makes cross tabulation of two variables  Intercept value of mpg when weight, horsepower equal zero – irrelevant
The Max Dec Step  ANOVA table, R-squared, Adjusted r-squared now are how well predictors together explain miles per gallon
 Maxdec in proc means controls # of decimals  Selection = option in model statement selects predictor variables based on p-values and builds regression model step by step
 Reducing dec makes table easier to read  Selection = forward starts with no predictors, add variables one at a time if p-values below .15
 Max dec placed immediately after proc means statement  Selection = backward starts with all candidate predictors in model, removes variables whose p-values are above .15
Filtering with Where  Selection = stepwise combines both, allows variables to enter when significant (.15 cutoff) and to leave
 Where statement inside proc meansNote 13 the analysis to only observations that meet a condition
restricts
Thelogical
 Where statement must reference existing variable names and use valid Proc conditions
Reg Step
Operators for the where statement  Proc Reg fits linear regression models and relates a quantifiable response variable to one or more predictor variables
 In – checks whether a variable matches any value in a list of values  Reports overall model fit stats, error variance estimate, r-square value and adjusted r-square
 Ne or -= operators means not equal to  Computes diagnostic stats. Help detect unusual, outlying, influential observations
 Or, and  Creates diagnostic plots, like residual and normal probability plots when ODS graphics turned on
 >,>=,<,<= comparison operators Basic Proc sgplot syntax
 Not in excludes specific values: not in (1,2)  Data=tells data to use
 Between operator selects observations within a range of values  Ends with run; tells SAS to run the code
 Contains operator searches for substring within a character variable  New piece model statement – specifies regression equation
 Not contains is the opposite  Response Variable left side = in model
 Like operator performs pattern matching character variables using wildcards
 One or more predictor variables right side = in model statement
Adding Titles and Labels in Summary Tables Proc Sgplot Output
 Title Statement adds custom title in results window  Proc Reg Produces ANOVA (Analysis of Variance) Table
  Separates
Label Statement assigns descriptive labels to variables, making the output more interpretable
variation in response explained by model and a part left in the errors
 Array [5] – 1,3,2,2,1 – inside data  ANOVA explains how well regression model explains variability in mpg
 In SAS, numeric variables cannot be directly converted to characterANOVA
values, soTable
first have to create new character variables to store letters
 Do Loop in SAS plays same role as a for loop in R after removing new  Fcharacter
value compares
variables,
model
we remove
mean square
extra character
to error mean
variables
square
so no
andlonger
measures
needhowonlystrong
letterregression
vector in output
relationship is
 Rand () function for random numbers  Pr > F gives p values for testing null hypothesis that all slope coefficient are zero
 Callstreamit () for seed  F-test provides overall test of goodness of fit
 Reports r-square and adjusted r-square as measures of model fit
 R-square coefficient of determination, measures proportion of total response variation explained by regression model
Note 10  Retio of sum of squares: 1-sserror/sstotal = ssmodel/sstotal
 Adjusted r-squared modifies r-squared to account for number of predictors – more for comparison across models with different number of predictors, unadjusted r^2 increases with predictors even without improving fit
 Proc univariate provides examination of distribution of one or moreParameter Estimates
numeric variables
 Produces descriptive statistics like mean, median, variance, standard deviation
 Show fitted regression coefficients
 Provides measures of distribution and graphical displays to assess shape,
 Estimate
center, spread,
columnpotential
gives estimated
outliersvalue of regression coefficient that best fits data in least squares sense
 Used to explore data preliminarily  Standard error column reports estimated standard deviation of sampling distribution of coefficient estimate
 Statements/styles from proc means apply here mostly  T value column shows test statistics for testing whether true coefficient is equal to zero
 More detailed summary than proc means  Pr > |t| column gives p value for that t test, which measures the evidence against the null hypothesis that the coefficient is zero
 Produces Moments Table: Diagnostic Plots
 Uses residual plots and influence plots to check the assumptions of the regression model
 Each small plot focuses on a different model fit aspect, like linearity, variance, normality
Multiple Linear Regression
 Output has the same types of tables; anova, diagnostic, regression
Skewness and Kurtosis  Slope for the weight is the expected change in miles per gallon expected for a one unit increase in weight, horsepower fixed
 Skewness measures symmetry of distribution  Slop for horsepower is the expected change in miles per gallon for a one unit increase in horsepower, weight is fixed
 Near 0 means symmetric (roughly)  Intercept value of mpg when weight, horsepower equal zero – irrelevant
 Positive value longer right tail meaning few large outlier pull right  ANOVA table, R-squared, Adjusted r-squared now are how well predictors together explain miles per gallon
  Selection = option in model statement selects predictor variables based on p-values and builds regression model step by step
Negative value means longer left tail, unusually small outliers pulling lef
 Kurtosis measures how peaked or flat the distribution is compared to aSelection
normal distribution
= forward starts with no predictors, add variables one at a time if p-values below .15
 Selection = backward starts with all candidate predictors in model, removes variables whose p-values are above .15
 Selection = stepwise combines both, allows variables to enter when significant (.15 cutoff) and to leave
Note 11
o Values of skewness or kurtosis between -1 and 1 are considered reasonably close to 0
 Other items in moments table:  Proc sgplot is the main SAS procedure to create most graphs
 Uncorrected SS – sum of squared values around mean Histogram with Proc sgplot
 Corrected SS – sum squared deviations from mean  Sgplot more histogram options than uni
 Coeff variation measures variability relative to the mean: STD/Meanx SAS
100%shows histogram with percentages on y-axis, while r shows frequency counts
 Standard error mean measures standard deviation of sample mean: standard
 Scale deviation
= count changes
over square
y-axis
root
to frequency
of n count
Basic Statistical Measures Table o Or can do scale = proportion
 Summaries central tendency, variability of the variable  Density statement adds a density curve to histogram
Test for Location  Grouped histogram can be added with group = option
 Table shows 3 hyptohesis tests, evaluate whether population mean or median
Class does
is equal
not work
to a specified
to makevariable
multiple plots
Quantile Table  Can use by statement or proc sgpanel
 Several Quantiles listed, describe how data values are spread across adistribution
Rowlattice and column lattice place each plot in their own row and column
Extreme Observations Table  Nbins = sets number of bars show in histogram
 5 lowest and highest mpg values, with corresponding dataset observation
 Binwidth
numbers = sets width on x-axis
 Not necessarily outlier Bar Chart with Proc Sgplot
The Class Statement  Summarizes categorical variables, categories instead of numeric variables
 Separate summaries for groups, like in proc means  Vbar vertical bar charts, hbar horizontal
 When class variable included, proc univariate produces output tablesforDefault
each group
y-axis is frequency
Normality Assessment in Proc Univariate  Y-axis can display % instead of counts with stat=percent
  Bar
Proc Univariate provides numerical tests and a graphical representations / tools
chart to
summarizes
assess howcategories,
closely a variable
not a numeric
followsscale
a normal distribution
 Normal option in proc univariate adds normality tests  Can also be displayed as multiple plots for clearer group comparisons
  By = p value assess evidence against normality
All tests evaluate null hypothesis that data comes from a normal distribution,
 Shapiro-Wilk most commonly used test, preferred for small to moderate Categoryorder=
sample sizes option controles order of categories
 Rely on Shapiro when others are conflicting  Category order – resp asc orders bars by response statistics (bar height) ascending
Histogram  Category order = resp dec orders by response statistics descending
 Can add it to with forward slash /  Category order = data orders categories by the order the appear in the data set
  Category order = ascending orders cateogries alphabetically or numerically ascending or vice versa for category order = descending
Histogram mpg / normal kernel – adds kernel density curve on histogram
  Data labels option shows values on each bar
Histogram mpg . normal kernel – adds kernel density curve on histogram
  Barwidth adjusts white space between bars
Kernel density smooth estimate of distribution, does not assume normality
Normal Q-Q plot Clustered bar chart with Proc Sgplot
 Ggplot requests normal qqplot  Places bars for each group side by side
 If points close to reference line, roughly normal  Easy to compare groups within each category
 Points away suggest curve, non-normality  Stacked bar chart places bars for each group on top of each other show how each category is composed by group
The Plot Option  Can be controlled by group display
 Simple Histogram and qqplot added with plot option in proc univariate

The Proc Sgplot Step