Did our posner experiment work? lets look at plotting the data from a single participant

There are many different libraries for data analysis in Python. But today we will mention just a few:

numpy : Fast multi-dimensional arrays, basic numeric operations

pandas : Very similar to numpy but supporting named columns

matplotlib : Visualization

scipy : Many basic scientific functions (including inferential stats)

*A more detailed version of these slides can be found at*

This library is at the peak of the ecosystem. Everything depends on this. It includes an implementation of multi-dimensional arrays with different data-types.

This is a very simple concept, but also a very powerful one!

It also contains some basic mathematical functions to operate on these arrays

To create an array, you may pass a sequence of elements into the numpy.array function:

```
import numpy as np
my_array = np.array([1,2,3])
```

This object has a shape attribute, that reports on its size:

```
print(my_array.shape)
```

And a dtype attribute, that reports on its data-type:

```
print(my_array.dtype)
```

This doesn’t seem very interesting, but consider a slightly more complicated array:

```
my_array = np.array([[1, 2, 3], [4, 5, 6]])
print(my_array.shape)
```

A 2d array can be considered a matrix.

You can automatically create an array with a range of numbers:

```
first_way = np.arange(0, 10.5, 0.5)
another_way = np.linspace(0, 10, 21)
print(first_way)
print(another_way)
```

And you can determine the shape you want it to have:

```
ten_by_ten = np.arange(100).reshape((10, 10))
print(ten_by_ten)
```

Exercise: Using numpy 1. make a list ranging 0 to 1 with 90 steps 2. print the shape of the array 3. reshape the array to be 3 rows of 30 4. print only the final row from the matrix

There are many functions in the numpy name-space to do operations on arrays:

```
print(np.mean(ten_by_ten))
print(np.mean(ten_by_ten, axis=0))
print(np.sqrt(ten_by_ten))
```

Matplotlib is very general and customizable. There are several different interfaces to the objects and functions implemented in MPL:

matplotlib - raw access to the plotting library. useful for extending matplotlib or doing very custom things

pylab - Matlab-like interface to matplotlib

pyplot - Object-oriented interface to matplotlib => use this one!

To create a figure and start plotting data:

```
import matplotlib.pyplot as plt
fig, ax = plt.subplots(1)
```

The two objects returned from this call are a `matplotlib.figure.Figure`

and `matplotlib.axes.AxesSubplot`

. They each have multiple methods that
can now be used.

For example:

```
t = np.linspace(-6*np.pi, 6*np.pi, 100)
ax.plot(t, np.sin(t)/t)
plt.show()
```

Use the object’s ‘setter’ functions, to set various attrbutes of the arrays. For example:

```
ax.set_xlabel('Time')
ax.set_ylabel('Amplitude')
```

The subplots command can return an array of axes:

```
fig, ax = plt.subplots(2,2)
```

We can plot different kinds of plots in each one:

```
x = np.arange(0, 100)
y = np.random.rand(100) # 100 random numbers
ax[0, 0].hist(y)
ax[0, 1].scatter(x, y)
ax[1, 0].boxplot(y)
ax[1, 1].loglog(x, y)
```

To show both figures call plt.show() at the end of the script.

Take a look at the Matplotlib gallery for many examples.

Each thumb-nail in the gallery contains a link to a page that has the source code that created that image. Use the code as a starting point for your own visualization.

There are two simple ways to find files. You could search for all files in a folder using a module called `glob`

:

```
import glob
filenames = glob.glob("data/*.csv")
```

Or you could open a file dialog using PsychoPy:

```
from psychopy import gui
filenames = gui.fileOpenDlg(allowed="*.csv")
```

Either way you get a list of file paths a list (which might be empty).

It’s also useful to have something to manipulate filenames. We’ll use `os.path`

for that:

```
from os import path
from psychopy import gui
filenames = gui.fileOpenDlg(allowed="*.csv")
for thisFilename in filenames:
thisPath, thisFullName = path.split(thisFilename)
fileNoExt, fileExt = path.splitext(thisFilename)
print('thisPath:', thisPath, 'thisFullName:', thisFullName)
print('this fileNoExt:', fileNoExt, 'this fileExt:', fileExt)
```

Use a similar trick to the last one but we’re going to use another library called pandas (http://pandas.pydata.org/)

```
from os import path
from psychopy import misc, gui
import pandas as pd
filenames = gui.fileOpenDlg(allowed="*.csv")
for thisFilename in filenames:
print(thisFilename)
thisDat = pd.read_csv(thisFilename)
print(thisDat)
```

Boom! We’ve got our data, as easily as that!

But, really, how easily can we *use* that data? Let’s try to pull out just the reaction times:

```
for thisFilename in filenames:
print(thisFilename)
thisDat = pd.read_csv(thisFilename)
print(thisDat['rt'])
```

We can also select parts of the data that fulfill certain criteria. Let’s get rid of trials where rt>1.0 (not ready?) and corr==0:

```
for thisFilename in filenames:
print(thisFilename)
thisDat = pd.read_csv(thisFilename)
#filter out bad data
filtered = thisDat[ thisDat['rt']<=1.0 ]
filtered = filtered[ filtered['corr']==1 ]
print(filtered['rt'])
```

OK, from our filtered data we need the mean and std.dev. reaction time for each condition:

```
import scipy
from scipy import stats
...
conflict = filtered[filtered.descr == 'conflict']
congruent = filtered[filtered.descr != 'conflict']
#get mean/std.dev
meanConfl = scipy.mean(conflict['rt'])
sdConfl = scipy.std(conflict['rt'], ddof=1) # ddof=1 means /sqrt(N-1)
meanCongr = scipy.mean(congruent['rt'])
sdCongr = scipy.std(congruent['rt'], ddof=1)
print("Conflict = %.3f (sd=%.3f)" %(meanConfl, sdConfl))
print("Congruent = %.3f (sd=%.3f)" %(meanCongr, sdCongr))
```

Let’s plot our data from the posner experiment:

```
import matplotlib.pyplot as plt
fig, ax = plt.subplots(1)
ax.bar(['Conflict', 'Congruent'], [meanConfl, meanCongr], yerr=[sdConfl, sdCongr])
ax.set_ylabel('RT (mean)')
plt.show()
```

If you want to save the Figure:

```
fig.savefig('my_figure.png')
fig.savefig('my_figure.pdf')
```

**Exercise** Create a three panel figure, something like this:

Solution:

```
fig, ax = plt.subplots(3)
ax[0].bar(['conflict', 'congruent'], [meanConfl, meanCongr], yerr=[sdConfl, sdCongr])
ax[0].set_ylabel('RT (mean)')
ax[1].bar(['conflict', 'congruent'], [AccConfl, AccCongr])
ax[1].set_ylabel('Accuracy (proportion)')
ax[2].scatter(conflict['MainBlock.thisN'], conflict['rt'],
c = 'r', label = 'conflict')
ax[2].scatter(congruent['MainBlock.thisN'], congruent['rt'],
c = 'b', label = 'congruent')
ax[2].set_ylabel('RT (mean)')
ax[2].set_xlabel('trial N')
ax[2].legend(loc = 'upper right')
```

**Exercise** Now collect another set of posner data, and try to plot for each participant in turn

This contains an implementation of many of the core computational routines you need in scientific computing:

scipy.cluster : Vector quantization / Kmeans

scipy.constants : Physical and mathematical constants

scipy.fftpack : Fourier transform

scipy.integrate : Integration routines

scipy.interpolate : Interpolation

scipy.io : Data input and output

scipy.linalg : Linear algebra routines

scipy.ndimage : n-dimensional image package

scipy.odr : Orthogonal distance regression

scipy.optimize : Optimization

scipy.signal : Signal processing

scipy.sparse : Sparse matrices

scipy.spatial : Spatial data structures and algorithms

scipy.special : Any special mathematical functions

scipy.stats : Statistics

Is the difference in RT between congruent and conflict responses significant (at the group level)?

Make sure you have multiple files to select and before your loop create an empty list where you will store each participants mean RT:

```
Confl_mean_rts=[]
Congr_mean_rts=[]
```

Then, once you have calculated each participants mean, append the list:

```
#in your loop
Confl_mean_rts.append(meanConfl)
Congr_mean_rts.append(meanCongr)
```

Finally, test congruent and incongruent RTs using a paired samples t-test:

```
t, p = stats.ttest_rel(Confl_mean_rts, Congr_mean_rts)
print("Related samples t-test: t=%.3f, p=%.4f"%(t, p))
```

*Note* we must have even observations (and more than one observation) for this to run.

Realistically, we would rarely perform inferential statistics at the single subject level. At the group level, we would have one value per condition per participant (so even observations).

Hopefully today you have learnt how to:

Create an experiment using only python code in PsychoPy

Optimise the experiment

Visualise the results (and move forward with analysing them)

What else? Python syntax and objects

Back to top