# 量化分析师的Python日记【第5天：数据处理的瑞士军刀pandas】

## 一、Pandas介绍

`pandas`不同的版本之间存在一些不兼容性，为此，我们需要清楚使用的是哪一个版本的`pandas`。现在我们就查看一下量化实验室的`pandas`版本：

``````import pandas as pd
pd.__version__

'0.14.1'
``````

`pandas`主要的两个数据结构是`Series``DataFrame`，随后两节将介绍如何由其他类型的数据结构得到这两种数据结构，或者自行创建这两种数据结构，我们先导入它们以及相关模块：

``````import numpy as np
from pandas import Series, DataFrame
``````

## 二、Pandas数据结构：`Series`

### 2.1 创建`Series`

``````a = np.random.randn(5)
print "a is an array:"
print a
s = Series(a)
print "s is a Series:"
print s

a is an array:
[-1.24962807 -0.85316907  0.13032511 -0.19088881  0.40475505]
s is a Series:
0   -1.249628
1   -0.853169
2    0.130325
3   -0.190889
4    0.404755
dtype: float64
``````

``````s = Series(np.random.randn(5), index=['a', 'b', 'c', 'd', 'e'])
print s
s.index

a    0.509906
b   -0.764549
c    0.919338
d   -0.084712
e    1.896407
dtype: float64
Index([u'a', u'b', u'c', u'd', u'e'], dtype='object')
``````

``````s = Series(np.random.randn(5), index=['a', 'b', 'c', 'd', 'e'], name='my_series')
print s
print s.name

a   -1.898245
b    0.172835
c    0.779262
d    0.289468
e   -0.947995
Name: my_series, dtype: float64
my_series
``````

`Series`还可以从字典（`dict`）创建：

``````d = {'a': 0., 'b': 1, 'c': 2}
print "d is a dict:"
print d
s = Series(d)
print "s is a Series:"
print s

d is a dict:
{'a': 0.0, 'c': 2, 'b': 1}
s is a Series:
a    0
b    1
c    2
dtype: float64
``````

``````Series(d, index=['b', 'c', 'd', 'a'])

b     1
c     2
d   NaN
a     0
dtype: float64
``````

``````Series(4., index=['a', 'b', 'c', 'd', 'e'])

a    4
b    4
c    4
d    4
e    4
dtype: float64
``````

### 2.2 `Series`数据的访问

``````s = Series(np.random.randn(10),index=['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j'])
s[0]

1.4328106520571824
``````
``````s[:2]

a    1.432811
b    0.120681
dtype: float64
``````
``````s[[2,0,4]]

c    0.578146
a    1.432811
e    1.327594
dtype: float64
``````
``````s[['e', 'i']]

e    1.327594
i   -0.634347
dtype: float64
``````
``````s[s > 0.5]

a    1.432811
c    0.578146
e    1.327594
g    1.850783
dtype: float64
``````
``````'e' in s

True
``````

## 三、Pandas数据结构：`DataFrame`

### 3.1 创建`DataFrame`

``````d = {'one': Series([1., 2., 3.], index=['a', 'b', 'c']), 'two': Series([1., 2., 3., 4.], index=['a', 'b', 'c', 'd'])}
df = DataFrame(d)
print df

one  two
a    1    1
b    2    2
c    3    3
d  NaN    4
``````

``````df = DataFrame(d, index=['r', 'd', 'a'], columns=['two', 'three'])
print df

two three
r  NaN   NaN
d    4   NaN
a    1   NaN
``````

``````print "DataFrame index:"
print df.index
print "DataFrame columns:"
print df.columns
print "DataFrame values:"
print df.values

DataFrame index:
Index([u'alpha', u'beta', u'gamma', u'delta', u'eta'], dtype='object')
DataFrame columns:
Index([u'a', u'b', u'c', u'd', u'e'], dtype='object')
DataFrame values:
[[  0.   0.   0.   0.   0.]
[  1.   2.   3.   4.   5.]
[  2.   4.   6.   8.  10.]
[  3.   6.   9.  12.  15.]
[  4.   8.  12.  16.  20.]]
``````

`DataFrame`也可以从值是数组的字典创建，但是各个数组的长度需要相同：

``````d = {'one': [1., 2., 3., 4.], 'two': [4., 3., 2., 1.]}
df = DataFrame(d, index=['a', 'b', 'c', 'd'])
print df

one  two
a    1    4
b    2    3
c    3    2
d    4    1
``````

``````d= [{'a': 1.6, 'b': 2}, {'a': 3, 'b': 6, 'c': 9}]
df = DataFrame(d)
print df

a  b   c
0  1.6  2 NaN
1  3.0  6   9
``````

``````df = DataFrame()
print df

Empty DataFrame
Columns: []
Index: []
``````

``````a = Series(range(5))
b = Series(np.linspace(4, 20, 5))
df = pd.concat([a, b], axis=1)
print df

0   1
0  0   4
1  1   8
2  2  12
3  3  16
4  4  20
``````

``````df = DataFrame()
index = ['alpha', 'beta', 'gamma', 'delta', 'eta']
for i in range(5):
a = DataFrame([np.linspace(i, 5*i, 5)], index=[index[i]])
df = pd.concat([df, a], axis=0)
print df

0  1   2   3   4
alpha  0  0   0   0   0
beta   1  2   3   4   5
gamma  2  4   6   8  10
delta  3  6   9  12  15
eta    4  8  12  16  20
``````

### 3.2 `DataFrame`数据的访问

``````print df[1]
print type(df[1])
df.columns = ['a', 'b', 'c', 'd', 'e']
print df['b']
print type(df['b'])
print df.b
print type(df.b)
print df[['a', 'd']]
print type(df[['a', 'd']])

alpha    0
beta     2
gamma    4
delta    6
eta      8
Name: 1, dtype: float64
<class 'pandas.core.series.Series'>
alpha    0
beta     2
gamma    4
delta    6
eta      8
Name: b, dtype: float64
<class 'pandas.core.series.Series'>
alpha    0
beta     2
gamma    4
delta    6
eta      8
Name: b, dtype: float64
<class 'pandas.core.series.Series'>
a   d
alpha  0   0
beta   1   4
gamma  2   8
delta  3  12
eta    4  16
<class 'pandas.core.frame.DataFrame'>
``````

``````print df['b'][2]
print df['b']['gamma']

4.0
4.0
``````

``````print df.iloc[1]
print df.loc['beta']

a    1
b    2
c    3
d    4
e    5
Name: beta, dtype: float64
a    1
b    2
c    3
d    4
e    5
Name: beta, dtype: float64
``````

``````print "Selecting by slices:"
print df[1:3]
bool_vec = [True, False, True, True, False]
print "Selecting by boolean vector:"
print df[bool_vec]

Selecting by slices:
a  b  c  d   e
beta   1  2  3  4   5
gamma  2  4  6  8  10
Selecting by boolean vector:
a  b  c   d   e
alpha  0  0  0   0   0
gamma  2  4  6   8  10
delta  3  6  9  12  15
``````

``````print df[['b', 'd']].iloc[[1, 3]]
print df.iloc[[1, 3]][['b', 'd']]
print df[['b', 'd']].loc[['beta', 'delta']]
print df.loc[['beta', 'delta']][['b', 'd']]

b   d
beta   2   4
delta  6  12
b   d
beta   2   4
delta  6  12
b   d
beta   2   4
delta  6  12
b   d
beta   2   4
delta  6  12
``````

``````print df.iat[2, 3]
print df.at['gamma', 'd']

8.0
8.0
``````

`dataframe.ix`可以混合使用索引和下标进行访问，唯一需要注意的地方是行列内部需要一致，不可以同时使用索引和标签访问行或者列，不然的话，将会得到意外的结果：

``````print df.ix['gamma', 4]
print df.ix[['delta', 'gamma'], [1, 4]]
print df.ix[[1, 2], ['b', 'e']]
print "Unwanted result:"
print df.ix[['beta', 2], ['b', 'e']]
print df.ix[[1, 2], ['b', 4]]

10.0
b   e
delta  6  15
gamma  4  10
b   e
beta   2   5
gamma  4  10
Unwanted result:
b   e
beta   2   5
2    NaN NaN
b   4
beta   2 NaN
gamma  4 NaN
``````