Data Analysis and Interpretation @jrodrige-blog - Tumblr Blog

Posts

Task 4: Testing a Potential Moderator

In this post I’m generating the anova test for two variables of my dataset with a potential moderator.

I’m using the dataset nesarc to response my research question:

are dystymia symtons associated with the gender?

According with my research question, my variables are:

AGE

SEX

RACE

S1Q1C (Hispanic or latin origin)

S1Q1D1 (American indian or Alaska native)

S1Q1D2 (Asian)

S1Q1D3 (Black or African American)

S1Q1D4 (Native Hawaiian or other pacific islander)

S1Q1D5 (White)

S4CQ5 (Age at onset of first episode)

S4CQ6A (Number of episodes)

sub1 = data[['AGE', 'SEX', 'S1Q1C', 'S1Q1D1','S1Q1D2','S1Q1D3', 'S1Q1D4', 'S1Q1D5', 'S4CQ5', 'S4CQ6A']]

After checking my Code Book, I need to replace the values 69 and 99 in my variables “S4CQ5″ and “S4CQ6A” by NaN, but after to apply the filters that response my research question I realized that I didn’t have this values into my dataset.

# setting missing data

sub1["S4CQ5"] = sub1["S4CQ5"].replace(69, numpy.nan)

sub1["S4CQ5"] = sub1["S4CQ5"].replace(99, numpy.nan)

sub1["S4CQ6A"] = sub1["S4CQ6A"].replace(99, numpy.nan)

I made a new column into my dataset, this column represent the sex of the person but represented with as letters F and M:

recode1 = {1: 'M', 2:'F'}

sub1["SEX"] = sub1["SEX"].map(recode1)

After, I create a new column to unified the ethnicity of each person in a single column and eliminate missing data.

sub1["RACE"] = sub1.apply (lambda row: setRace(row),axis=1)

sub1["RACE"] = sub1["RACE"].replace(7, numpy.nan)

Then, create the sub2 with only the columns that it will be used in the analysis

sub2 = sub1[['AGE', 'SEX', 'RACE', 'S4CQ5', 'S4CQ6A']].dropna()

The Anova Test was performed without a moderator and these were the results.

I used the variable “RACE” as a moderator to analyze the relationship between “SEX” and “S4CQ6A, in order to answer my reseach question:

Lating people

# using ols function for calculating the F-statistic and associated p value sub3 = sub2[sub2['RACE'] == 1].dropna() model2 = smf.ols(formula='S4CQ6A ~ C(SEX)', data=sub3) results2 = model2.fit() print (results2.summary())

No latin people

# using ols function for calculating the F-statistic and associated p value sub4 = sub2[sub2['RACE'] != 1].dropna() model3 = smf.ols(formula='S4CQ6A ~ C(SEX)', data=sub4) results3 = model3.fit() print (results3.summary())print ('means for sex') m3 = sub4.groupby('SEX').mean() print(m3)

Summary

The execution of the Anova test show us that there isn’t a signitifcant relation between the selected variables because the p-value is high.

Using the variable RACE, the result show the same. In both cases, when RACE is equal to 1, which means latin people, and is not. The p-value high, so besides there is not exist significant relationship between my variables, RACE is not a varliable moderator for this relationship.

Code

""" Created on Sun Oct 25 15:40:25 2015

@author: Jorge Rodriguez E. <[email protected]> """ import numpy import pandas import statsmodels.formula.api as smf import statsmodels.stats.multicomp as multi import scipy.stats import matplotlib.pyplot as plt import seaborn

def frequencyTable(colName, data): count = data[colName].value_counts(sort=False) percent = data[colName].value_counts(sort=False, normalize=True) print("counts for ", colName) # age at onset of first episode print(count)

print("percent for ", colName) print(percent)

#creating 7 level race variable def setRace (row): if row['S1Q1C'] == 1 : return 1 # Hispanic or latin origin elif row['S1Q1D1'] == 1 : return 2 # AMERICAN INDIAN OR ALASKA NATIVE elif row['S1Q1D2'] == 1 : return 3 # ASIAN elif row['S1Q1D3'] == 1 : return 4 # BLACK OR AFRICAN AMERICAN elif row['S1Q1D4'] == 1 : return 5 # NATIVE HAWAIIAN OR OTHER PACIFIC ISLANDER elif row['S1Q1D5'] == 1 : return 6 # WHITE else : return 7

""" analysis of variance. Analysis of variance assesses whether the means of two or more groups are statistically different from each other. """

# Import the entire dataset to memory data = pandas.read_csv("nesarc.csv", low_memory=False) #Set PANDAS to show all columns in DataFrame pandas.set_option('display.max_columns', None) #Set PANDAS to show all rows in DataFrame pandas.set_option('display.max_rows', None) # Upper case all DataFrame column names data.columns = map(str.upper, data.columns) # bug fix for display formats to avoid run time errors pandas.set_option('display.float_format', lambda x:'%f'%x)

#setting variables I will be working with to numeric data["AGE"] = data["AGE"].convert_objects(convert_numeric=True) # AGE data["SEX"] = data["SEX"].convert_objects(convert_numeric=True) # SEX # Hispanic or latin origin data["S1Q1C"] = data["S1Q1C"].convert_objects(convert_numeric=True) # AMERICAN INDIAN OR ALASKA NATIVE data["S1Q1D1"] = data["S1Q1D1"].convert_objects(convert_numeric=True) # ASIAN data["S1Q1D2"] = data["S1Q1D2"].convert_objects(convert_numeric=True) # BLACK OR AFRICAN AMERICAN data["S1Q1D3"] = data["S1Q1D3"].convert_objects(convert_numeric=True) # NATIVE HAWAIIAN OR OTHER PACIFIC ISLANDER data["S1Q1D4"] = data["S1Q1D4"].convert_objects(convert_numeric=True) # WHITE data["S1Q1D5"] = data["S1Q1D5"].convert_objects(convert_numeric=True) # age at onset of first episode data["S4CQ5"] = data["S4CQ5" ].convert_objects(convert_numeric=True) # number of episodes data["S4CQ6A"] = data["S4CQ6A"].convert_objects(convert_numeric=True)

# Reserch question: # the association between the age when someone has dystymia symtons and the gender

# Refined question: # Are dystymia symtons and the gender associated with young adults whose # suffer the first episode in their adolescence

# subdata to young adults with age between 19 and 35 who suffer the first # episodo between 12 and 18 years

sub1 = data[['AGE', 'SEX', 'S1Q1C', 'S1Q1D1','S1Q1D2','S1Q1D3', 'S1Q1D4', 'S1Q1D5', 'S4CQ5', 'S4CQ6A']]

# setting missing data sub1["S4CQ5"] = sub1["S4CQ5"].replace(69, numpy.nan) sub1["S4CQ5"] = sub1["S4CQ5"].replace(99, numpy.nan) sub1["S4CQ6A"] = sub1["S4CQ6A"].replace(99, numpy.nan)

recode1 = {1: 'M', 2:'F'} sub1["SEX"] = sub1["SEX"].map(recode1)

sub1["RACE"] = sub1.apply (lambda row: setRace(row),axis=1) sub1["RACE"] = sub1["RACE"].replace(7, numpy.nan)

sub2 = sub1[['AGE', 'SEX', 'RACE', 'S4CQ5', 'S4CQ6A']].dropna()

# en los latinos # SEX --> S4CQ6A Sexo determina el número de episodios # using ols function for calculating the F-statistic and associated p value model1 = smf.ols(formula='S4CQ6A ~ C(SEX)', data=sub2) results1 = model1.fit() print (results1.summary())

# En los latinos # SEX --> S4CQ6A Sexo determina el número de episodios # using ols function for calculating the F-statistic and associated p value sub3 = sub2[sub2['RACE'] == 1].dropna() model2 = smf.ols(formula='S4CQ6A ~ C(SEX)', data=sub3) results2 = model2.fit() print (results2.summary())

print ('means for sex') m2 = sub3.groupby('SEX').mean() print(m2)

# En los no latinos # SEX --> S4CQ6A Sexo determina el número de episodios # using ols function for calculating the F-statistic and associated p value sub4 = sub2[sub2['RACE'] != 1].dropna() model3 = smf.ols(formula='S4CQ6A ~ C(SEX)', data=sub4) results3 = model3.fit() print (results3.summary())

print ('means for sex') m3 = sub4.groupby('SEX').mean() print(m3)

•18+ Adults Only

Watch Anya Live on Cam

Anya is live and ready to show you everything. Watch her strip, dance, and perform exclusive shows just for you. Interact in real-time and make your fantasies come true.

✓ Live Streaming✓ Interactive Chat✓ Private Shows✓ HD Quality✓ Free Actions

Free to watch • No registration required • HD streaming

Task 3 - Generating Correlation Coefficient

In this post I’m generating the correlation coefficeint for two variables of my dataset.

I’m using the dataset nesarc to response my research question:

are dystymia symtons and the gender in latin people associated with young adults whose suffer the first episode in their adolescence?

Correlation coefficient

In order to response my reseach question, my variables are categorical that’s why I’m use the followin hypothesis with quantitive variables.

Then my hythesis are:

Ho: there is no relationship bewteen the age of persons with dystymia sympstomps with number of episodes Ha: there is a relationship bewteen the age of persons with dystymia sympstomps with number of episodes

Summary

I use the variables “age” and “number of episode” to answer my hypothesis by generating the correlation coefficient.

After execute the test, I obteined:

correlation coefficient: 0.07

p-value: 0.001817121882

In my case, my association between the age and number of episodes is slightly possitive but it is not so strong and there is not linear relation. This is not mean that these are independent variables.

Code

# -*- coding: utf-8 -*- """ Created on Sun Nov 29 19:48:21 2015

@author: Jorge Rodriguez E. [email protected] """ import numpy import pandas import seaborn import scipy import matplotlib.pyplot as plt

# Import the entire dataset to memory data = pandas.read_csv("nesarc.csv", low_memory=False)

#Set PANDAS to show all columns in DataFrame pandas.set_option('display.max_columns', None)

#Set PANDAS to show all rows in DataFrame pandas.set_option('display.max_rows', None)

# Upper case all DataFrame column names data.columns = map(str.upper, data.columns)

# bug fix for display formats to avoid run time errors pandas.set_option('display.float_format', lambda x:'%f'%x) #setting variables I will be working with to numeric # AGE data["AGE"] = data["AGE"].convert_objects(convert_numeric=True) # number of episodes data["S4CQ6A"] = data["S4CQ6A"].convert_objects(convert_numeric=True) data["S4CQ5"] = data["S4CQ5"].convert_objects(convert_numeric=True)

data['S4CQ6A'] = data['S4CQ6A'].replace('99', numpy.nan) data['S4CQ6A'] = data['S4CQ6A'].replace(' ', numpy.nan)

data['S4CQ5'] = data['S4CQ5'].replace('99', numpy.nan) data['S4CQ5'] = data['S4CQ5'].replace(' ', numpy.nan)

scat1 = seaborn.regplot(x="AGE", y="S4CQ6A", fit_reg=True, data=data) plt.xlabel('Age') plt.ylabel('Number of episodes') plt.title('Scatterplot for the Association Between \n Age and Number of Dystymia Episodes')

#scat1 = seaborn.regplot(x="S4CQ6A", y="S4CQ5", fit_reg=True, data=data) #plt.xlabel('Age First Episode') #plt.ylabel('Number of episodes') #plt.title('Scatterplot for the Association Between Age and Number of Dystymia Episodes')

data_clean = data[['AGE', 'S4CQ6A', 'S4CQ5']].dropna() print ('Association Between Age and Number of Dystymia Episodes') print (scipy.stats.pearsonr(data_clean['AGE'], data_clean['S4CQ6A']))

#print ('Association Between Age and Number of Dystymia Episodes') #print (scipy.stats.pearsonr(data_clean['S4CQ5'], data_clean['S4CQ6A']))

estdatosespol-blog

Deber numero Dos de Estructuras de Datos - Paralelo 3

En este post voy a explicar como hice el deber numero dos de la clase de Estructuras de Datos del Ing. Jorge Rodriguez.

Para empezar tenemos que completar el sgte. codigo para que corra:

public class Programa {

public static void main(String[] args){ LinkedList<Coordenada> conjunto1 = new LinkedList<>(); LinkedList<Coordenada> conjunto2 = new LinkedList<>(); LinkedList<Coordenada> union, interseccion, resta; Programa.cargarC1(conjunto1); Programa.cargarC2(conjunto2); union = Conjunto.union(conjunto1,conjunto2); interseccion = Conjunto.interseccion(conjunto1,conjunto2); resta = Conjunto.diferencia(conjunto1,conjunto2); Conjunto.imprimir(union); Conjunto.imprimir(interseccion); Conjunto.imprimir(resta); } private static void cargarC1(LinkedList<Coordenada> conjunto){ conjunto.add(new Coordenada(10, 10, “Ciudad 1”)); conjunto.add(new Coordenada(20, 20, “Ciudad 2”)); conjunto.add(new Coordenada(30, 30, “Ciudad 3”)); conjunto.add(new Coordenada(40, 40, “Ciudad 4”)); } private static void cargarC2(LinkedList<Coordenada> conjunto){ conjunto.add(new Coordenada(50, 50, “Ciudad 5”)); conjunto.add(new Coordenada(20, 20, “Ciudad 2”)); conjunto.add(new Coordenada(60, 60, “Ciudad 6”)); conjunto.add(new Coordenada(40, 40, “Ciudad 4”)); } }

Este programa representauna serie de conjuntos con unas listas enlazadas. Los contenidos de estas listas son cargados por los metodos cargarC1 y cargarC2.

Para que este programa pueda correr, necesitamos algunas cosas:

Una Clase llamada Conjunto

Cuatro metodos estaticos dentro de esa clase: union, interseccion , diferencia e imprimir.

Dos metodos estaticos en la clase Programa: cargar ( la cual lee un archivo de texto y usa la informacion dentro para crear una LinkedList) y un metodo guardar ( el cual guarda los contenidos de la LinkedList en un archivo de texto).

Primero me concentre en crear los metodos de la clase conjunto ya que son los mas importantes para el funcionamiento del programa.

El primer metodo que cree fue el metodo union:

public static LinkedList<Coordenada> union(LinkedList conjunto1, LinkedList conjunto2){

LinkedList<Coordenada> resultado = new LinkedList<>(); LinkedList<Coordenada> conjuntoGrande; LinkedList<Coordenada> conjuntoPequeño;

if( conjunto1.size() < conjunto2.size() ){ conjuntoGrande = conjunto2; conjuntoPequeño = conjunto1; } else{ conjuntoGrande = conjunto1; conjuntoPequeño = conjunto2; }

resultado.addAll(conjuntoGrande);

Iterator iter = conjuntoPequeño.iterator();

Coordenada tmp; Coordenada tmp2;

while( iter.hasNext() ){ tmp = (Coordenada)iter.next();

for(int i =0; i < resultado.size(); i++){ tmp2 = (Coordenada) resultado.get(i); if(tmp2.equals(tmp)){ break; } else{ resultado.addLast(tmp); } }

}

En este metodo, obtenemos dos listas enlazadas que entran por parametro. Usamos un condicional para escoger la mas lista con mas contenidos que se guarda en la variable conjuntoGrande, mientras que la mas pequeña se guarda en la variable conjuntoPequeño. Despues declaramos una variable de tipo LinkedList y le agregamos todos los contenidos de conjuntoGrande. A continuacion use un iterador para recorrer todos los contenidos del conjunto pequeño y copararlos con los contenidos del conjunto resultado. Si el contenido del conjuntoPequeño era igual (aqui se uso el metodo equal() ) a uno de los elementos de resultado, no se lo agregaba. Sin embargo esto creo el primer error del programa, ya que la forma en que lo implemente me dio la sgte. salida:

El primer grupo de Coordenadas deberia solo tener 6 elementos que correspondan a las ciudades 1, 2, 3, 4, 5 y 6. Sin embargo se puede ver que hay 21 Coordenadas. Deje el arreglo de este metodo para despues.

Me enfoque mas en el segundo metodo que es la interseccion:

public static LinkedList<Coordenada> interseccion(LinkedList conjunto1, LinkedList conjunto2){

LinkedList<Coordenada> resultado = new LinkedList<>();

Iterator iter = conjunto1.iterator();

Coordenada tmp; Coordenada tmp2;

while( iter.hasNext() ){ tmp = (Coordenada)iter.next();

for(int i =0; i < conjunto2.size(); i++){ tmp2 = (Coordenada) conjunto2.get(i); if(tmp2.equals(tmp)){ resultado.addLast(tmp); break; }

}

return resultado; }

Aqui se recorre la primera lista con un iterador y se usa un lazo para comparar cada elemento de esta lista con la de la segunda. si se esncuentra un elemento de la segunda lista que es igual al elemento que se esta viendo actualmente en el iterador, se agrega ese elemento a la lista resultado y se hace un break en el lazo, lo cual noslleva al sgte. elemento del iterador. al final se retorna la lista resultado.

El tercer metodo es diferencia:

public static LinkedList<Coordenada> diferencia(LinkedList conjunto1, LinkedList conjunto2){

LinkedList<Coordenada> interseccion = interseccion(conjunto1, conjunto2); LinkedList<Coordenada> resultado = new LinkedList<>();

resultado.addAll(conjunto1);

Iterator iter = interseccion.iterator();

Coordenada tmp;

while(iter.hasNext()){

tmp = (Coordenada) iter.next();

for(int i = 0; i<resultado.size(); i++){ if( tmp.equals(resultado.get(i)) ){ resultado.remove(i); }

}

return resultado;

}

Aqui se crea una lista llamada interseccion a la cual se le asigna la interseccion de las dos listas que entran por parametro y una lista llamada resultado a la cual se le agrega los elementos de la orimera lista por parametro. Despues se usa un iterador en la lista interseccion y un lazo en la lista resultado con la cual se chequea igualdad de elementos. Todo elemnto que se encuentra en las dos listas se lo remueve de la lista resultado y se retorna esa lista.

Aqui regrese al primer metodo donde simplemente lo reescribi en terminos de intersecciones y diferencia. Basicamente el metodo diferencia se deshace de los elementos repetidos, y el metodo interseccion se encarga de guardar una copia de cada elemento repetido. Asi que para la union simplement cree cuatro listas (las dos primeras son para asignar las listas de parametro a listas locales):

LinkedList<Coordenada> con1 = conjunto1; LinkedList<Coordenada> con2 = conjunto2; LinkedList<Coordenada> resta1Menos2 = diferencia(con1, con2); LinkedList<Coordenada> resta2Menos1 = diferencia(con2, con1); LinkedList<Coordenada> resultado = new LinkedList<> LinkedList<Coordenada> interseccion = interseccion(conjunto1,conjunto2);

BAsicamente la union(a,b) va a tener los mismos elementos de una lista que tenga los elementos de la diferencia(a,b), diferencia(b,a) e interseccion(a,b). Asi que solo agregue los elementos de esas operaciones a la lista resultado que se la retorna al final. Esto me dio el sgte output:

El cual es correcto.

Para el metodo de imprimir solo use un FOR EACH para imprimir por pantalla cada Coordenada.

Para el metodo cargar en la clase Programa simplemente reuse el mismo metodo que esta en la clase Coordenada que usamos en clase.

If it ain´t broke, don´t fix it.

En el sgte. metodo, guardar, use un FileWriter y un BufferedWriter para escoger la representacion de String de las coordenadas y escribirlas en un archivo.

private static void guardar(LinkedList<Coordenada> conjunto, String archivo){

LinkedList<Coordenada> lista = conjunto;

FileWriter fw; BufferedWriter bw; Iterator iter = lista.iterator(); String linea;

try{ fw = new FileWriter(archivo); bw = new BufferedWriter(fw);

while(iter.hasNext()){ linea = iter.next().toString(); bw.write(linea); bw.newLine();

} bw.close();

}catch(IOException ex){ System.out.println(“Error de escritura!! >:0(”); }

}

Use un iterador para recorrer la lista que se entrega por parametro, guerde su representacion en String en una variable llamada linea:

while(iter.hasNext()){ linea = iter.next().toString(); bw.write(linea); bw.newLine();

Y al final se cierra el archivo con el comando bw.close();.

En el metodo main de Programa, use el metodo cargar para darle los valores a conjunto1 y conjunto2.

jrodrige-blog

Excelente!! Muy buen trabajo.

Task: Running a Chi-Square Test of Independence

I’m using the dataset nesarc to response my research question:

are dystymia symtons and the gender in latin people associated with young adults whose suffer the first episode in their adolescence?

Chi-Square Test of Independence

Now I will make the Chi-Square Test of Independence to examining the association between range of age of persons with dystymia symtomps (categorical explanatory) and sex (categorical response).

Then my hythesis are:

Ho: there is no relationship bewteen the range of age of persons with dystymia sympstomps with their sex Ha: there is a relationship bewteen the range of age of persons with dystymia sympstomps with their sex

Summary

A chi-square test of independence revealed that there no is relationship bewteen the range of age of persons with dystymia symptomps with their sex, X2 =6.73, 1 df, p=0.08. Then according with the results, I accept the null hypothesys.

The df or degree of freedom we record is the number of levels of the explanatory variable -1. Here the df is 1 nicotine dependence which has 2 levels (df 2-1=1). tworky�d.g

Post hoc test

If I had rejected the null hypotesis, a Chi Square test of independence had relevated that there is a relation between the sex and range of age.

Is neccesary to execute a Post hoc comparison by pairs of range of age.

CODE.py

# -*- coding: utf-8 -*- """ Created on Sun Nov 1 22:27:59 2015

@author: Jorge Rodriguez E. <[email protected]> """

import pandas import numpy import scipy.stats import seaborn import matplotlib.pyplot as plt

def ChiSquaredTest(cat_responseVariable, cat_explanatoryVariable, data): # contingency table of observed counts ct = pandas.crosstab(data[cat_responseVariable], data[cat_explanatoryVariable], dropna=True) print (ct)

# column percentages colsum = ct.sum(axis=0) colpct = ct/colsum print(colpct)

print ('chi-square value, p value, expected counts') cs = scipy.stats.chi2_contingency(ct) print (cs)

data = pandas.read_csv("nesarc.csv", low_memory=False) # Import the entire dataset to memory pandas.set_option('display.max_columns', None)#Set PANDAS to show all columns in DataFrame pandas.set_option('display.max_rows', None)#Set PANDAS to show all rows in DataFrame data.columns = map(str.upper, data.columns)# Upper case all DataFrame column names pandas.set_option('display.float_format', lambda x:'%f'%x)# bug fix for display formats to avoid run time errors

# Ensure each of these columns are numeric data["AGE"] = data["AGE"].convert_objects(convert_numeric=True) # AGE data["SEX"] = data["SEX"].convert_objects(convert_numeric=True) # SEX data["S1Q1C"] = data["S1Q1C"].convert_objects(convert_numeric=True) # Hispanic or latin origin data["S4CQ5"] = data["S4CQ5"].convert_objects(convert_numeric=True) # age at onset of first episode

# management data "S4CQ5" data["S4CQ5"] = data["S4CQ5"].replace(69, numpy.nan) data["S4CQ5"] = data["S4CQ5"].replace(99, numpy.nan)

# Reserch question: the association between the age when someone has dystymia symtons and the gender in latin people

# Refined question: are dystymia symtons and the gender in latin people associated # with young adults whose suffer the first episode in their adolescence

# subdata to Latin young adults with age between 19 and 35 who suffer the first episodo between 12 and 18 years sub1 = data[(data['S1Q1C'] == 1) & (data['AGE'] >= 19) & (data['AGE'] <=35 ) & (data['S4CQ5'].fillna(0) >= 12) & (data['S4CQ5'].fillna(0) <= 18)]

#make a copy of my new subsetted data sub2 = sub1.copy()

# quartile split (use qcut function & ask for 4 groups) sub2['AGE_GROUP4'] = pandas.qcut(sub2.AGE, 4, labels=["1 = 25%","2 = 50%","3 = 75%","4 = 100%"]) #AGE - categories by quartiles sub2['AGE_GROUP4_2'] = pandas.cut(sub2.AGE, [18,22,26,30,35]) #AGE - categories by ranges: 19-22, 23-26, 27-30, 31-35

# subset only with my variables sub3 = sub2[['AGE', 'SEX', 'S1Q1C', 'S4CQ5', 'AGE_GROUP4', 'AGE_GROUP4_2']]

# recoding values for AGE_GROUP4_2 into a new variable, AGE_GROUP4_2 recode2 = {'(18, 22]': 1, '(22, 26]':2, '(26, 30]': 3, '(30, 35]': 4} sub3["AGE_GROUP4_2"] = sub3["AGE_GROUP4_2"].map(recode2)

# contingency table of observed counts ChiSquaredTest(cat_responseVariable = 'SEX', cat_explanatoryVariable = 'AGE_GROUP4_2', data = sub3)

sub3["AGE_GROUP4_2"] = sub3["AGE_GROUP4_2"].astype('category') # set variable types sub3['SEX1'] = sub3['SEX'].convert_objects(convert_numeric=True) # new code for setting variables to numeric:

# graph percent with nicotine dependence within each smoking frequency group seaborn.factorplot(x="AGE_GROUP4_2", y="SEX", data=sub3, kind="bar", ci=None) plt.xlabel('Age') plt.ylabel('Sex')

#ChiSquaredPostHocTest recode3 = {1: 1, 2: 2} sub3['COMP1v2']= sub2['AGE_GROUP4_2'].map(recode3) ChiSquaredTest(cat_responseVariable = 'SEX', cat_explanatoryVariable = 'COMP1v2', data = sub3)

recode4 = {1: 1, 3: 3} sub3['COMP1v3']= sub2['AGE_GROUP4_2'].map(recode4) ChiSquaredTest(cat_responseVariable = 'SEX', cat_explanatoryVariable = 'COMP1v3', data = sub3)

recode5 = {1: 1, 4: 4} sub3['COMP1v4']= sub2['AGE_GROUP4_2'].map(recode5) ChiSquaredTest(cat_responseVariable = 'SEX', cat_explanatoryVariable = 'COMP1v4', data = sub3)

recode6 = {2: 2, 3: 3} sub3['COMP2v3']= sub2['AGE_GROUP4_2'].map(recode6) ChiSquaredTest(cat_responseVariable = 'SEX', cat_explanatoryVariable = 'COMP2v3', data = sub3)

recode7 = {2: 2, 4: 4} sub3['COMP2v4']= sub2['AGE_GROUP4_2'].map(recode7) ChiSquaredTest(cat_responseVariable = 'SEX', cat_explanatoryVariable = 'COMP2v4', data = sub3)

recode8 = {3: 3, 4: 4} sub3['COMP3v4']= sub2['AGE_GROUP4_2'].map(recode8) ChiSquaredTest(cat_responseVariable = 'SEX', cat_explanatoryVariable = 'COMP3v4', data = sub3)

#python #data analysis tools #Chi Squared

Task: Running an analysis of variance

I’m using the dataset nesarc to response my research question:

are dystymia symtons and the gender associated with young adults whose suffer the first episode in their adolescence?

According with my research question, I made the data management:

Analysis of variance (ANOVA)

Now I will make the analysis of variance to assesses whether the means of dystymia episode by sex are statistically different. In this analysis I want to compare the number of dystymia episodes (quantitative variable) by sex (categorical variable).

Then my hythesis are:

Ho: there is no difference in the mean of the dystymia across sex Ha: there is difference in the mean of the dystymia across sex

After to do the analysis of variance, I obtein this results:

Summary

According with the results, I accept the null hypothesys and these means are statistically equals and that there is no association between the sex and dystymia episodes in young adults.

Post hoc test

To examining the association between the race and the number of dystymia episode, I’m going to use the Turkey HSDT

These are the differences in dystymia episode for each ethnic group pair.

Summary

According with the results, I accept the null hypotesys and these means are statitically equals and thar there is no difference between the number of dystymia episodes and the differents ethnicithy.

CODE

# -*- coding: utf-8 -*- """ Created on Sun Oct 25 15:40:25 2015

@author: Jorge Rodriguez E. <[email protected]> """ import numpy import pandas import statsmodels.formula.api as smf import statsmodels.stats.multicomp as multi

print("percent for ", colName) print(percent)

""" The assignments for this course start where the Data Management and Visualization course assignments left off. Now that you have selected a data set and research question, managed your variables of interest and visualized their relationship graphically, we are ready to test those relationships statistically. We have included the codebooks and data sets from Data Management and Visualization for your convenience.

The first assignment deals with analysis of variance. Analysis of variance assesses whether the means of two or more groups are statistically different from each other.

This analysis is appropriate whenever you want to compare the means (quantitative variables) of groups (categorical variables).

The null hypothesis is that there is no difference in the mean of the quantitative variable across groups (categorical variable), while the alternative is that there is a difference.

Note that if your research question does not include one quantitative variable, you can use one from your data set just to get some practice with the tool.

If your research question does not include a categorical variable, you can categorize one that is quantitative. """

# Import the entire dataset to memory data = pandas.read_csv("nesarc.csv", low_memory=False)

#Set PANDAS to show all columns in DataFrame pandas.set_option('display.max_columns', None)

#Set PANDAS to show all rows in DataFrame pandas.set_option('display.max_rows', None)

# Upper case all DataFrame column names data.columns = map(str.upper, data.columns)

# bug fix for display formats to avoid run time errors pandas.set_option('display.float_format', lambda x:'%f'%x)

# Reserch question: # the association between the age when someone has dystymia symtons and the gender

# Refined question: # Are dystymia symtons and the gender associated with young adults whose # suffer the first episode in their adolescence

# subdata to young adults with age between 19 and 35 who suffer the first # episodo between 12 and 18 years sub1 = data[(data['AGE'] >= 19) & (data['AGE'] <=35 ) & (data['S4CQ5'].fillna(0) >= 12) & (data['S4CQ5'].fillna(0) <= 18)]

# setting missing data sub1["S4CQ5"] = sub1["S4CQ5"].replace(69, numpy.nan) sub1["S4CQ5"] = sub1["S4CQ5"].replace(99, numpy.nan) sub1["S4CQ6A"] = sub1["S4CQ6A"].replace(99, numpy.nan)

recode1 = {1: 'M', 2:'F'} sub1["SEX1"] = sub1["SEX"].map(recode1)

sub1["RACE"] = sub1.apply (lambda row: setRace(row),axis=1) sub1["RACE"] = sub1["RACE"].replace(7, numpy.nan) recode2 = {1: 'Hispanic or latin origin', 2:'American indian or Alaska native', 3: 'Asian', 4: 'Black or African American', 5: 'Native Hawaiian or othe pacific islander', 6: 'White'} sub1["RACE1"] = sub1["RACE"].map(recode2)

#Ho: there is no difference in the mean of the dystymia across sex #Ha: there is difference in the mean of the dystymia across sex

sub2 = sub1[['SEX1', 'S4CQ6A']].dropna()

# using ols function for calculating the F-statistic and associated p value model1 = smf.ols(formula='S4CQ6A ~ C(SEX1)', data=sub2) results1 = model1.fit() print (results1.summary())

print ('means for sex') m1 = sub2.groupby('SEX1').mean() print(m1)

print ('standard deviations for sex') sd1 = sub2.groupby('SEX1').std() print(sd1)

# accept the null hypothesys and these means are statistically equals # and that there is no association between the sex and dystymia episodes # in young adults

# subset only with my variables sub3 = sub1[['RACE1', 'S4CQ6A']].dropna() mc2 = multi.MultiComparison(sub3['S4CQ6A'], sub3['RACE1']) res2 = mc2.tukeyhsd() print(res2.summary())

#anova #data analysis tools #post hoc test #python

•18+ Adults Only

Watch Anya Live on Cam

Anya is live and ready to show you everything. Watch her strip, dance, and perform exclusive shows just for you. Interact in real-time and make your fantasies come true.

✓ Live Streaming✓ Interactive Chat✓ Private Shows✓ HD Quality✓ Free Actions

Free to watch • No registration required • HD streaming

Creating graphs for your data (Task 4)

I’m using the dataset nesarc_pds to response my research question:

are dystymia symtons and the gender in latin people associated with young adults whose suffer the first episode in their adolescence?

According with my research question, I created the following graphs for my variables:

Sex (SEX1). This univariate graph lets us to know how many males and females suffer the first dystymia episode in their adolescence

SUMMARY

The graph above show us the distribution of the latin young adult with dystymia symptoms in their dolescense by sex.

AGE_GROUP_2, this variable is based in the variable “AGE” and represent categories by age ranges. In this univariate graph we can identify that the new generation of adults suffered any dystymia episode.

SUMMARY

The graph above is bimodal, show us the distribution with a concentration of values in the ranges (18,22] and (30,35].

To explore the relation between the gender and the age first dystymia episode, I made the following graph to females:

To explore the relation between the gender and the age first dystymia episode, I made the following graph to males:

SUMMARY

The both graph above are uniform. We cannot see a value around which the observations are concentrated, the values by age and sex are independent, that is, the distribution for males and females are similar. The bar chart don’t show a different between the gender and dystymia symptoms to lating young adult.

Week 4 Task.py

# -*- coding: utf-8 -*- """ Created on Sun Oct 4 16:12:53 2015

@author: Jorge Rodríguez """ import pandas import numpy import seaborn import matplotlib.pyplot as plt

print("percent for ", colName) print(percent)

# Import the entire dataset to memory data = pandas.read_csv("nesarc_pds.csv", low_memory=False)

#Set PANDAS to show all columns in DataFrame pandas.set_option('display.max_columns', None)

#Set PANDAS to show all rows in DataFrame pandas.set_option('display.max_rows', None)

# Upper case all DataFrame column names data.columns = map(str.upper, data.columns)

# bug fix for display formats to avoid run time errors pandas.set_option('display.float_format', lambda x:'%f'%x)

# Ensure each of these columns are numeric data["AGE"] = data["AGE"].convert_objects(convert_numeric=True) # AGE data["SEX"] = data["SEX"].convert_objects(convert_numeric=True) # SEX data["S1Q1C"] = data["S1Q1C"].convert_objects(convert_numeric=True) # Hispanic or latin origin data["S4CQ5"] = data["S4CQ5" ].convert_objects(convert_numeric=True) # age at onset of first episode

# management data "S4CQ5" data["S4CQ5"] = data["S4CQ5"].replace(69, numpy.nan) data["S4CQ5"] = data["S4CQ5"].replace(99, numpy.nan)

# Frequecy table for "age at onset of first episode" frequencyTable("S4CQ5", data)

# Reserch question: the association between the age when someone has dystymia symtons and the gender in latin people

# Refined question: are dystymia symtons and the gender in latin people associated # with young adults whose suffer the first episode in their adolescence

#make a copy of my new subsetted data sub2 = sub1.copy()

# Frequecy table for "age" frequencyTable("AGE", sub2)

# Frequecy table for "sex" frequencyTable("SEX", sub2)

# Frequecy table for "hispanic or latin origin, yes=1" frequencyTable("S1Q1C", sub2)

# Frequecy table for "S4CQ5 - age at onset of first episode" frequencyTable("S4CQ5", sub2)

# quartile split (use qcut function & ask for 4 groups) print("AGE - categories by quartiles") sub2['AGE_GROUP4'] = pandas.qcut(sub2.AGE, 4, labels=["1 = 25%","2 = 50%","3 = 75%","4 = 100%"]) C5 = sub2['AGE_GROUP4'].value_counts(sort=False, dropna=True) print(C5)

print("AGE - categories by ranges: 19-22, 23-26, 27-30, 31-35") sub2['AGE_GROUP4_2'] = pandas.cut(sub2.AGE, [18,22,26,30,35]) C6 = sub2['AGE_GROUP4_2'].value_counts(sort=False, dropna=True) print(C6)

# subset only with my variables sub3 = sub2[['AGE', 'SEX', 'S1Q1C', 'S4CQ5', 'AGE_GROUP4', 'AGE_GROUP4_2']] print(sub3)

# create a new variable with the equivalent values to SEX recode1 = {1: 'M', 2:'F'} sub3["SEX1"] = sub3["SEX"].map(recode1)

print(pandas.crosstab(sub3['AGE_GROUP4_2'], sub3['AGE'])) print(pandas.crosstab(sub3['AGE_GROUP4_2'], sub3['SEX1']))

# Frequecy table for "AGE_GROUP4_2" frequencyTable("AGE_GROUP4_2", sub3)

#univariate bar graph for categorical variables # First hange format from numeric to categorical sub3["AGE_GROUP4_2"] = sub3["AGE_GROUP4_2"].astype('category')

seaborn.countplot(x="AGE_GROUP4_2", data=sub3) plt.xlabel('Age Latin Young Adult') plt.title('Latin Young Adult whose suffer the first dystymia episodo\n in their adolescence by Age (NESARC Study)')

print ('describe AGE_GROUP4_2') desc2 = sub3['AGE_GROUP4_2'].describe() print (desc2)

# Frequecy table for "SEX1" frequencyTable("SEX1", sub3)

#univariate bar graph for categorical variables # First hange format from numeric to categorical sub3["SEX1"] = sub3["SEX1"].astype('category')

seaborn.countplot(x="SEX1", data=sub3) plt.xlabel('Sex') plt.title('Latin Young Adult whose suffer the first dystymia episodo\nin their adolescence by Sex (NESARC Study)')

print ('describe SEX1') desc1 = sub3['SEX1'].describe() print (desc1)

# bivariate bar graph C->C sub4 = sub3[(sub3['SEX1'] == "F")] seaborn.factorplot(x='AGE_GROUP4_2', y='S4CQ5', data=sub4, kind="bar", ci=None) plt.xlabel('Age') plt.ylabel('Count') plt.title('Latin Young Adult Females whose suffer the first dystymia episodo\nin their adolescence by Sex (NESARC Study)')

# bivariate bar graph C->C sub5 = sub3[(sub3['SEX1'] == "M")] seaborn.factorplot(x='AGE_GROUP4_2', y='S4CQ5', data=sub5, kind="bar", ci=None) plt.xlabel('Age') plt.ylabel('Count') plt.title('Latin Young Adult Males whose suffer the first dystymia episodo\nin their adolescence by Sex (NESARC Study)')

Making Data Management Decisions (Task 3)

I’m using the dataset nesarc_pds to response my research question:

are dystymia symtons and the gender in latin people associated with young adults whose suffer the first episode in their adolescence?

According with my research question, my variables are:

AGE

SEX

S1Q1C (Hispanic or latin origin)

S4CQ5 (Age at onset of first episode)

After checking my Code Book, I need to replace the values 69 and 99 in my variable “S4CQ5″ by NaN, but after to apply the filters that response my research question I realized that I didn’t have this values into my dataset.

data["S4CQ5"] = data["S4CQ5"].replace(69, numpy.nan)

data["S4CQ5"] = data["S4CQ5"].replace(99, numpy.nan)

I made a new column into my dataset, this column represent the sex of the person but represented with as letters F and M:

recode1 = {1: 'M', 2:'F'}

sub3["SEX1"] = sub3["SEX"].map(recode1)

I made a new variable “AGE_GROUP_2″ into the dataset, this variable is based in the variable “AGE” and represent categories by age ranges:

print("AGE - categories by ranges: 19-22, 23-26, 27-30, 31-35")

sub2['AGE_GROUP4_2'] = pandas.cut(sub2.AGE, [18,22,26,30,35])

C6 = sub2['AGE_GROUP4_2'].value_counts(sort=False, dropna=True)

print(C6)

I crossed my variables AGE_GROUP4_2 and SEX to determine the gender distribution by age category:

print(pandas.crosstab(sub3['AGE_GROUP4_2'], sub3['SEX1']))

I crossed my variables AGE_GROUP4_2 and AGE to determine the age distribution by age category:

print(pandas.crosstab(sub3['AGE_GROUP4_2'], sub3['AGE']))

Week3.py

# -*- coding: utf-8 -*- """ Created on Sun Oct 4 16:12:53 2015

@author: Jorge Rodríguez """ import pandas import numpy

# Import the entire dataset to memory data = pandas.read_csv("nesarc_pds.csv", low_memory=False)

# Upper case all DataFrame column names data.columns = map(str.upper, data.columns)

# bug fix for display formats to avoid run time errors pandas.set_option('display.float_format', lambda x:'%f'%x)

# Ensure each of these columns are numeric data["AGE"] = data["AGE"].convert_objects(convert_numeric=True) # AGE data["SEX"] = data["SEX"].convert_objects(convert_numeric=True) # SEX data["S1Q1C"] = data["S1Q1C"].convert_objects(convert_numeric=True) # Hispanic or latin origin data["S4CQ5"] = data["S4CQ5" ].convert_objects(convert_numeric=True) # age at onset of first episode

print("counts for original S4CQ5") # age at onset of first episode C1 = data["S4CQ5"].value_counts(sort=False) print(C1)

data["S4CQ5"] = data["S4CQ5"].replace(69, numpy.nan) data["S4CQ5"] = data["S4CQ5"].replace(99, numpy.nan)

print("counts for S4CQ5 with 69 and 99 set to NAN") # age at onset of first episode C1 = data["S4CQ5"].value_counts(sort=False) print(C1)

# Reserch question: the association between the age when someone has dystymia symtons and the gender in latin people

# Refined question: are dystymia symtons and the gender in latin people associated # with young adults whose suffer the first episode in their adolescence

#make a copy of my new subsetted data sub2 = sub1.copy()

print("counts for AGE") C1 = sub2["AGE"].value_counts(sort=False) print(C1)

print("percent for AGE") P1 = sub2["AGE"].value_counts(sort=False, normalize=True) print(P1)

print("counts for SEX") C2 = sub2["SEX"].value_counts(sort=False) print(C2)

print("percent for SEX") P2 = sub2["SEX"].value_counts(sort=False, normalize=True) print(P2)

print("counts for S1Q1C - hispanic or latin origin, yes=1") C3 = sub2["S1Q1C"].value_counts(sort=False) print(C3)

print("percent for S1Q1C - hispanic or latin origin, yes=1") P3 = sub2["S1Q1C"].value_counts(sort=False, normalize=True) print(P3)

print("counts for S4CQ5 - age at onset of first episode") C4 = sub2["S4CQ5"].value_counts(sort=False) print(C4)

print("percent for S4CQ5 - age at onset of first episode") P4 = sub2["S4CQ5"].value_counts(sort=False, normalize=True) print(P4)

print("AGE - categories by ranges: 19-22, 23-26, 27-30, 31-35") sub2['AGE_GROUP4_2'] = pandas.cut(sub2.AGE, [18,22,26,30,35]) C6 = sub2['AGE_GROUP4_2'].value_counts(sort=False, dropna=True) print(C6)

# subset only with my variables sub3 = sub2[['AGE', 'SEX', 'S1Q1C', 'S4CQ5', 'AGE_GROUP4', 'AGE_GROUP4_2']] print(sub3)

# create a new variable with the equivalent values to SEX recode1 = {1: 'M', 2:'F'} sub3["SEX1"] = sub3["SEX"].map(recode1)

print(pandas.crosstab(sub3['AGE_GROUP4_2'], sub3['AGE'])) print(pandas.crosstab(sub3['AGE_GROUP4_2'], sub3['SEX1']))

print("counts for AGE_GROUP4_2") C5 = sub3["AGE_GROUP4_2"].value_counts(sort=False) print(C5)

print("percent for AGE_GROUP4_2") P5 = sub3["AGE_GROUP4_2"].value_counts(sort=False, normalize=True) print(P5)

print("counts for SEX1") C6 = sub3["SEX1"].value_counts(sort=False) print(C6)

print("percent for SEX1") P6 = sub3["SEX1"].value_counts(sort=False, normalize=True) print(P6)

OUTPUT RESULT

counts for original S4CQ5 5.000000 43 6.000000 10 7.000000 9 8.000000 16 9.000000 7 10.000000 20 11.000000 17 12.000000 49 13.000000 41 14.000000 53 15.000000 66 16.000000 81 17.000000 38 18.000000 50 19.000000 54 20.000000 44 21.000000 40 22.000000 50 23.000000 49 24.000000 56 25.000000 61 26.000000 37 27.000000 51 28.000000 58 29.000000 32 30.000000 75 31.000000 49 32.000000 47 33.000000 49 34.000000 34 .. 55.000000 25 56.000000 10 57.000000 13 58.000000 9 59.000000 8 60.000000 15 61.000000 4 62.000000 6 63.000000 5 64.000000 7 65.000000 22 66.000000 6 67.000000 7 68.000000 2 69.000000 6 70.000000 8 71.000000 1 72.000000 6 73.000000 4 74.000000 4 75.000000 4 76.000000 2 77.000000 2 78.000000 1 79.000000 6 80.000000 9 81.000000 3 82.000000 3 84.000000 1 99.000000 69 dtype: int64 counts for S4CQ5 with 69 and 99 set to NAN 5.000000 43 6.000000 10 7.000000 9 8.000000 16 9.000000 7 10.000000 20 11.000000 17 12.000000 49 13.000000 41 14.000000 53 15.000000 66 16.000000 81 17.000000 38 18.000000 50 19.000000 54 20.000000 44 21.000000 40 22.000000 50 23.000000 49 24.000000 56 25.000000 61 26.000000 37 27.000000 51 28.000000 58 29.000000 32 30.000000 75 31.000000 49 32.000000 47 33.000000 49 34.000000 34 .. 53.000000 18 54.000000 15 55.000000 25 56.000000 10 57.000000 13 58.000000 9 59.000000 8 60.000000 15 61.000000 4 62.000000 6 63.000000 5 64.000000 7 65.000000 22 66.000000 6 67.000000 7 68.000000 2 70.000000 8 71.000000 1 72.000000 6 73.000000 4 74.000000 4 75.000000 4 76.000000 2 77.000000 2 78.000000 1 79.000000 6 80.000000 9 81.000000 3 82.000000 3 84.000000 1 dtype: int64 counts for AGE 19 3 20 6 21 5 22 4 23 2 24 5 25 3 27 1 31 2 32 2 33 1 34 1 35 1 dtype: int64 percent for AGE 19 0.083333 20 0.166667 21 0.138889 22 0.111111 23 0.055556 24 0.138889 25 0.083333 27 0.027778 31 0.055556 32 0.055556 33 0.027778 34 0.027778 35 0.027778 dtype: float64 counts for SEX 1 15 2 21 dtype: int64 percent for SEX 1 0.416667 2 0.583333 dtype: float64 counts for S1Q1C - hispanic or latin origin, yes=1 1 36 dtype: int64 percent for S1Q1C - hispanic or latin origin, yes=1 1 1.000000 dtype: float64 counts for S4CQ5 - age at onset of first episode 16.000000 9 17.000000 7 18.000000 4 12.000000 2 13.000000 2 14.000000 4 15.000000 8 dtype: int64 percent for S4CQ5 - age at onset of first episode 16.000000 0.250000 17.000000 0.194444 18.000000 0.111111 12.000000 0.055556 13.000000 0.055556 14.000000 0.111111 15.000000 0.222222 dtype: float64 AGE - categories by quartiles 1 = 25% 9 2 = 50% 9 3 = 75% 10 4 = 100% 8 dtype: int64 AGE - categories by ranges: 19-22, 23-26, 27-30, 31-35 (18, 22] 18 (22, 26] 10 (26, 30] 1 (30, 35] 7 dtype: int64 AGE SEX S1Q1C S4CQ5 AGE_GROUP4 AGE_GROUP4_2 336 20 1 1 12.000000 1 = 25% (18, 22] 681 23 2 1 12.000000 3 = 75% (22, 26] 2024 21 2 1 16.000000 2 = 50% (18, 22] 3500 20 1 1 16.000000 1 = 25% (18, 22] 5004 21 1 1 13.000000 2 = 50% (18, 22] 8198 22 1 1 17.000000 2 = 50% (18, 22] 9025 32 1 1 15.000000 4 = 100% (30, 35] 12514 20 2 1 15.000000 1 = 25% (18, 22] 12529 19 1 1 16.000000 1 = 25% (18, 22] 12690 21 2 1 17.000000 2 = 50% (18, 22] 15571 33 1 1 17.000000 4 = 100% (30, 35] 15779 24 2 1 14.000000 3 = 75% (22, 26] 18548 25 2 1 17.000000 3 = 75% (22, 26] 18743 31 1 1 18.000000 4 = 100% (30, 35] 18768 22 2 1 15.000000 2 = 50% (18, 22] 19079 25 2 1 16.000000 3 = 75% (22, 26] 21166 22 2 1 18.000000 2 = 50% (18, 22] 21885 19 1 1 15.000000 1 = 25% (18, 22] 22577 24 2 1 14.000000 3 = 75% (22, 26] 26651 19 2 1 17.000000 1 = 25% (18, 22] 26877 21 2 1 15.000000 2 = 50% (18, 22] 26960 23 1 1 17.000000 3 = 75% (22, 26] 27751 35 2 1 16.000000 4 = 100% (30, 35] 28029 27 1 1 15.000000 4 = 100% (26, 30] 28269 32 2 1 14.000000 4 = 100% (30, 35] 28705 25 2 1 16.000000 3 = 75% (22, 26] 29124 20 2 1 17.000000 1 = 25% (18, 22] 30312 21 1 1 16.000000 2 = 50% (18, 22] 33397 22 1 1 16.000000 2 = 50% (18, 22] 36182 24 2 1 15.000000 3 = 75% (22, 26] 37348 31 1 1 16.000000 4 = 100% (30, 35] 37982 24 2 1 13.000000 3 = 75% (22, 26] 38738 34 2 1 18.000000 4 = 100% (30, 35] 39637 24 2 1 14.000000 3 = 75% (22, 26] 39799 20 1 1 18.000000 1 = 25% (18, 22] 41896 20 2 1 15.000000 1 = 25% (18, 22] AGE 19 20 21 22 23 24 25 27 31 32 33 34 35 AGE_GROUP4_2 (18, 22] 3 6 5 4 0 0 0 0 0 0 0 0 0 (22, 26] 0 0 0 0 2 5 3 0 0 0 0 0 0 (26, 30] 0 0 0 0 0 0 0 1 0 0 0 0 0 (30, 35] 0 0 0 0 0 0 0 0 2 2 1 1 1 SEX1 F M AGE_GROUP4_2 (18, 22] 9 9 (22, 26] 9 1 (26, 30] 0 1 (30, 35] 3 4 counts for AGE_GROUP4_2 (18, 22] 18 (22, 26] 10 (26, 30] 1 (30, 35] 7 dtype: int64 percent for AGE_GROUP4_2 (18, 22] 0.500000 (22, 26] 0.277778 (26, 30] 0.027778 (30, 35] 0.194444 dtype: float64 counts for SEX1 F 21 M 15 dtype: int64 percent for SEX1 F 0.583333 M 0.416667 dtype: float64

#python #pandas #data analysis #data manipulation

Running My First Program (Task 2)

My research question originally said "the association between the age when someone has dystymia symtons and the gender in latin people”.

After looking the data, my refined research question is: are dystymia symtons and the gender in latin people associated with young adults whose suffer the first episode in their adolescence?

According with my research question, my variables are:

AGE

SEX

S1Q1C (Hispanic or latin origin)

S4CQ5 (Age at onset of first episode)

Week 2 Task.py

import pandas import numpy

# Import the entire dataset to memory data = pandas.read_csv("nesarc_pds.csv", low_memory=False)

# Upper case all DataFrame column names data.columns = map(str.upper, data.columns)

# bug fix for display formats to avoid run time errors pandas.set_option('display.float_format', lambda x:'%f'%x)

# Ensure each of these columns are numeric data["AGE"] = data["AGE"].convert_objects(convert_numeric=True) # AGE data["SEX"] = data["SEX"].convert_objects(convert_numeric=True) # SEX data["S1Q1C"] = data["S1Q1C"].convert_objects(convert_numeric=True) # Hispanic or latin origin data["S4CQ5"] = data["S4CQ5" ].convert_objects(convert_numeric=True) # age at onset of first episode

print("counts for AGE") C1 = data["AGE"].value_counts(sort=False) print(C1)

print("percent for AGE") P1 = data["AGE"].value_counts(sort=False, normalize=True) print(P1)

print("counts for SEX") C2 = data["SEX"].value_counts(sort=False) print(C2)

print("percent for SEX") P2 = data["SEX"].value_counts(sort=False, normalize=True) print(P2)

print("counts for S1Q1C - hispanic or latin origin, yes=1") C3 = data["S1Q1C"].value_counts(sort=False) print(C3)

print("percent for S1Q1C - hispanic or latin origin, yes=1") P3 = data["S1Q1C"].value_counts(sort=False, normalize=True) print(P3)

print("counts for S4CQ5 - age at onset of first episode") C4 = data["S4CQ5"].value_counts(sort=False) print(C4)

print("percent for S4CQ5 - age at onset of first episode") P4 = data["S4CQ5"].value_counts(sort=False, normalize=True) print(P4)

sub1 = data[(data['S1Q1C'] == 1)]# latin people print("Latin people") print(len(sub1)) # Number of latin people (rows)

sub2 = sub1[(sub1['AGE'] >= 19) & (sub1['AGE'] <=35 )] # young adults print("Latin people with age between 19 and 35") print(len(sub2)) # Number of Latin young adults with age between 19 and 35 (rows)

# the variable S4CQ5 has nan values and cannot be filtered at least first you replace this value with a numeric valid value, it can be do with fillna function.

sub3 = sub2[(sub2['S4CQ5'].fillna(0) >= 12) & (sub2['S4CQ5'].fillna(0) <= 18)] print("Latin young adults whose suffer the first episode in adolescense") print(len(sub3))

sub4 = sub3[(sub3['SEX'] == 1)] print("Latin young males whose suffer the first episode in adolescense") print(len(sub4))

sub5 = sub3[(sub3['SEX'] == 2)] print("Latin young females whose suffer the first episode in adolescense") print(len(sub5))

Frequency tabla for S4CQ5

counts for S4CQ5 - age at onset of first episode 5.000000 43 6.000000 10 7.000000 9 8.000000 16 9.000000 7 10.000000 20 11.000000 17 12.000000 49 13.000000 41 14.000000 53 15.000000 66 16.000000 81 17.000000 38 18.000000 50 19.000000 54 20.000000 44 21.000000 40 22.000000 50 23.000000 49 24.000000 56 25.000000 61 26.000000 37 27.000000 51 28.000000 58 29.000000 32 30.000000 75 31.000000 49 32.000000 47 33.000000 49 34.000000 34 .. 55.000000 25 56.000000 10 57.000000 13 58.000000 9 59.000000 8 60.000000 15 61.000000 4 62.000000 6 63.000000 5 64.000000 7 65.000000 22 66.000000 6 67.000000 7 68.000000 2 69.000000 6 70.000000 8 71.000000 1 72.000000 6 73.000000 4 74.000000 4 75.000000 4 76.000000 2 77.000000 2 78.000000 1 79.000000 6 80.000000 9 81.000000 3 82.000000 3 84.000000 1 99.000000 69 dtype: int64 percent for S4CQ5 - age at onset of first episode 5.000000 0.000998 6.000000 0.000232 7.000000 0.000209 8.000000 0.000371 9.000000 0.000162 10.000000 0.000464 11.000000 0.000394 12.000000 0.001137 13.000000 0.000951 14.000000 0.001230 15.000000 0.001532 16.000000 0.001880 17.000000 0.000882 18.000000 0.001160 19.000000 0.001253 20.000000 0.001021 21.000000 0.000928 22.000000 0.001160 23.000000 0.001137 24.000000 0.001300 25.000000 0.001416 26.000000 0.000859 27.000000 0.001183 28.000000 0.001346 29.000000 0.000743 30.000000 0.001740 31.000000 0.001137 32.000000 0.001091 33.000000 0.001137 34.000000 0.000789 ... 55.000000 0.000580 56.000000 0.000232 57.000000 0.000302 58.000000 0.000209 59.000000 0.000186 60.000000 0.000348 61.000000 0.000093 62.000000 0.000139 63.000000 0.000116 64.000000 0.000162 65.000000 0.000511 66.000000 0.000139 67.000000 0.000162 68.000000 0.000046 69.000000 0.000139 70.000000 0.000186 71.000000 0.000023 72.000000 0.000139 73.000000 0.000093 74.000000 0.000093 75.000000 0.000093 76.000000 0.000046 77.000000 0.000046 78.000000 0.000023 79.000000 0.000139 80.000000 0.000209 81.000000 0.000070 82.000000 0.000070 84.000000 0.000023 99.000000 0.001601 dtype: float64

Frequency tabla for SEX

counts for SEX 1 18518 2 24575 dtype: int64 percent for SEX 1 0.429722 2 0.570278 dtype: float64

Frequency tabla for S1Q1C

counts for S1Q1C - hispanic or latin origin, yes=1 1 8308 2 34785 dtype: int64 percent for S1Q1C - hispanic or latin origin, yes=1 1 0.192792 2 0.807208 dtype: float64

SUB DATASET

Latin people 8308 Latin people with age between 19 and 35 3519 Latin young adults whose suffer the first episode in adolescense 36 Latin young males whose suffer the first episode in adolescense 15 Latin young females whose suffer the first episode in adolescense 21

#python #data analysis #pandas #numpy

Dataset chosen

After looking the codebook NESARC, I’m interested in the topic related about Dysthymia, maybe diagnosis.

I decide that I am most interested in exploring the association between the age when someone has dystymia symtons and the gender in latin people. I add to my codebook variables reflecting dysthymia (e.g. age at onset of first episode, frequency, number of episodes, ages when the most recdnt episode began).

My codebook: https://goo.gl/iQPrG6

References

León, C. A., & León, A. (1993). The clinical profile of dysthymia in a group of Latin American women. European Psychiatry.

Moskalewicz, J., Wciórka, J., Stokwiszewski, J., Rabczenko, D., & Kessler, R. C. (2015). The prevalence of common mental disorders in the population of adult Poles by sex and age structure–an EZOP Poland study. Psychiatr. Pol,49(1), 15-27.

van den Ban, E. F., Souverein, P. C., van Engeland, H., Swaab, H., Egberts, T. C., & Heerdink, E. R. (2015). Differences in ADHD medication usage patterns in children and adolescents from different cultural backgrounds in the Netherlands. Social Psychiatry and Psychiatric Epidemiology, 1-10.

Peters, A. T., Shankman, S. A., Deckersbach, T., & West, A. E. (2015). Predictors of first-episode unipolar major depression in individuals with and without sub-threshold depressive symptoms: A prospective, population-based study. Psychiatry research.

Trending Blogs

Last Seen Blogs

Data Analysis and Interpretation