argmax, argmin argsort and quick sort | 快速排序

This Saturday class we went over indexing and ordering a group of items by their sorted indices. For those who are more advanced, please go over the section on quick sort.

For example,

>> import numpy as np
>>> packpack =np.array([‘snack’,’book’,’pen’,’eraser’,’apple’])
# Position of the biggest word (alphabetically)
>>> np.argmax(packpack)

[out]: 0
# Position of the smallest word (alphabetically)
>>> np.argmin(packpack)

[out]: 4

# Position of the words if we are to sort them alphabetically
>>> np.argsort(packpack)

[out]: array([4, 1, 3, 2, 0], dtype=int64)

Now let us sort them:
>>> packpack[np.argsort(packpack)]

[out]: array([‘apple’, ‘book’, ‘eraser’, ‘pen’, ‘snack’], dtype='<U6′)

Then we tried sorting numbers:

numbers = np.array([2,3,5,7,1,4,6,15,5,2,7,9,10,15,9,17,12])
>>> numbers[np.argsort(numbers)]

[out]: array([ 1, 2, 2, 3, 4, 5, 5, 6, 7, 7, 9, 9, 10, 12, 15, 15, 17])


Finally we dig deeper: how do you really sort things fast systematically? Using quick sort!

 

Logarithm | 对数

As we had explored in previous classes, division is subtraction again and again and again, multiplication is adding again and again.  Exponentiation is multiply again and again and again— They are all inventions to simplify repeated computation.

So is the invention of logarithm: taking log is division again and again and again.   They were invented by John Napier who was a Scottish mathematician, physicist, and astronomer  in 1614 as a means to simplify calculations.

🙂 Today’s  Python numpy class summary:

Log10 means how many times divide by 10 will return you to 1. log10(100) will give you 2 because 100 divide by 10 twice returns us to one.
>>> np.log10(100)
One trillion divide by 10 twelve times returns it to 1.
>>> np.log10(1000000000000)

>>> np.linspace(0.0, 3.0, num=4)
Out: array([0., 1., 2., 3.])

>>> np.logspace(0.0, 3.0, num=4)
Out: array([   1.,   10.,  100., 1000.])

>>> np.linspace(0.0, 12.0, num=13)
Out: array([ 0.,  1.,  2.,  3.,  4.,  5.,  6.,  7.,  8.,  9., 10., 11., 12.])
>>> np.logspace(0.0, 12.0, num=13)
Out: array([1.e+00, 1.e+01, 1.e+02, 1.e+03, 1.e+04, 1.e+05, 1.e+06, 1.e+07, 1.e+08, 1.e+09, 1.e+10, 1.e+11, 1.e+12])

Bonus:  Did you know that Engineers and scientists used to use a tool called “slide rule” (计算尺) to do logarithmic computations until 1970s when electronic computer and calculators came into use.  You should go and check it out if any of your grandparents have one of these.

Find if something is also somewhere else (contd) | 找一找那里是不是也有

Today’s class we continued the game of finding matches.  We expanded from numbers to names.

Let’s pretend that there is a room out there that has the following famous people:

room1 = pd.Series([‘Grace Hopper’, ‘Albert Einstein’,’Michael Faraday’,’Emmy Noether’,’Ada Lovelace’])

And another room with these famous people:

room2 = pd.Series([‘Isaac Newton’, ‘Thomas Edison’,’Mary Somerville’,’Matilda’,’Ada Lovelace’])

These two rooms are in building:

building = pd.concat([room1, room2], axis=1)

building.columns= [‘room1′,’room2’]

             room1            room2
0     Grace Hopper     Isaac Newton
1  Albert Einstein    Thomas Edison
2  Michael Faraday  Mary Somerville
3     Emmy Noether          Matilda
4     Ada Lovelace     Ada Lovelace

Now we want to find a list of people “who” in the building and the rooms.

who = np.array([‘Albert Einstein’,’Michael Faraday’,’Ada Lovelace’])

Are they in room 1?

np.isin(who, room1)
Out: array([ True,  True,  True])

Are they in room2?

np.isin(who, room2)

Out: array([False, False,  True])

Are they in the building?

np.isin(who, building)
Out: array([ True,  True,  True])

We are constantly comparing things.  How to compare is a very tricky and interesting subject.  You should look up the source code of the function in1d and see how it does it.

Find if something is also somewhere else | 找一找那里是不是也有

In today’s class we played a game: finding the numbers in a group that are also in another group. As usual, we started simple using numpy (np).   Here is the summary:

score1 = np.array([0, 1, 3, 5, 10,3])

score2 = np.array([2,3])

We use the inld function from numpy to do the matching and counting:
np.in1d(score1,score2)

score1[np.in1d(score1,score2)]

Our result is:

array([3, 3])

There is another way to do this:

np.array([item in score2 for item in score1])

array([False, False,  True, False, False,  True])

score1[np.array([item in score2 for item in score1])]

will give you exactly the same answer.   Try it.

For very large groups of numbers or words, what we’ve just learned will work just as well.

Please practice and turn in the homework.

Count non-zeros using numpy.count_nonzero | 数非零数

Today our class practiced making the computer count number of non-zero numbers using the numpy library from Python.  This can be useful if you have a ton of numbers.

import numpy as np; import pandas as pd

some_array = np.array([[0,1,7,0,0],[3,0,0,2,19]])

array([[ 0,  1,  7,  0,  0],
[ 3,  0,  0,  2, 19]])

np.count_nonzero(some_array)

5

np.count_nonzero(some_array,  axis=0)  Count across the rows, i.e. count along the column

array([1, 1, 1, 1, 1], dtype=int64)

np.count_nonzero(some_array,  axis=1)  Count across the columns, i.e. count along the row

array([2, 3], dtype=int64)

We talked about this example:

d = {'Basket1': [3, 0], 'Basket2': [3, 4]}
df = pd.DataFrame(data=d, index=['Apple','Chips'])

# Count the number of non-zeros across the rows
pd.Series(np.count_nonzero(df, axis=0), index=df.columns.tolist())

This was the result we got.

Basket1    1
Basket2    2
dtype: int64

That was a very tiny data. If we have a dataset with a million rows and columns, we should definitely do this!

Parents and students in China – impressions from our summer trip | 中国家长和学生- 暑期中国行印象

We always knew that Chinese parents and teachers are dedicated to education.   Even so, we were still amazed by how Chinese teachers and parents are working hand-in-hand pushing the boundaries of education.

We don’t necessarily agree with everything they do, such as the bias towards solving test problems in classes.  But there are a great deal more things we agree or appreciate, such as relentless hard work and practice than ones we don’t appreciate.

Below is a snapshot of a first-day summer class for seven year old children.   Just look at how intently the parents are.   Some parents sat through the entire class to take notes, and some sat in the lounge in the corridor chatting about schools or doing various things while waiting for their kids.    The temperature was about 100 degrees outside.

Magic Math Mandarin

The level of dedication is astonishing. 

From time investment perspective, at least one family member, whether mother, father or a grandparent, spends as much as 20 hours and up each week on their children’s education.   

From the money side, summer/winter break, weekend, or after-hour schoolings are privately run, which aren’t cheap.   Some well-to-do ones have spent hundreds of thousands (measured in US dollars) before high school.  Some bold ones even send their kids, sometimes as young as eight or nine years old, to overseas private boarding schools in exclusive locations in Switzerland, the UK and other places.    Poorer families still pay for various lessons to make sure their kids are as nourished as possible in education.   For those very constraint in resources, such as those parents who must work 7 days a week, we saw their kids studying with video lessons on mobile phones in cram corners instead of hanging out loose.  

The primary motivation is the quality of survival for the next generation: to get into top middle schools, top high schools, top universities, great social network and ultimately great jobs in adulthood.   Parents start count down of the number of days till college entrance exam even at primary schools.   Kids routinely study until mid-night since third grade, and don’t get a day off until winter/summer break.

On the contrary, in United States, students and parents are heard asking for less homework and more free time to play.    Over the last two decades, the quality of education, as measured by test scores, have steadily declined in the US.   Less work is a key factor.  However, a deeper question is: why do American parents and kids want less work from schools?   These questions need to be answered by representative data, not ideaologies.