Our cofounder’s new book release

We are really proud of our co-founder(联合创始人 lián hé chuàng shǐ rén), Sarah Chen, for her new book release, which is now public.   Congratulations!

Her book is available in various outlets:

Here is her pre-release announcement on LinkedIn in July this year.

 

 

argmax, argmin argsort and quick sort | 快速排序

This Saturday class we went over indexing and ordering a group of items by their sorted indices. For those who are more advanced, please go over the section on quick sort.

For example,

>> import numpy as np
>>> packpack =np.array([‘snack’,’book’,’pen’,’eraser’,’apple’])
# Position of the biggest word (alphabetically)
>>> np.argmax(packpack)

[out]: 0
# Position of the smallest word (alphabetically)
>>> np.argmin(packpack)

[out]: 4

# Position of the words if we are to sort them alphabetically
>>> np.argsort(packpack)

[out]: array([4, 1, 3, 2, 0], dtype=int64)

Now let us sort them:
>>> packpack[np.argsort(packpack)]

[out]: array([‘apple’, ‘book’, ‘eraser’, ‘pen’, ‘snack’], dtype='<U6′)

Then we tried sorting numbers:

numbers = np.array([2,3,5,7,1,4,6,15,5,2,7,9,10,15,9,17,12])
>>> numbers[np.argsort(numbers)]

[out]: array([ 1, 2, 2, 3, 4, 5, 5, 6, 7, 7, 9, 9, 10, 12, 15, 15, 17])


Finally we dig deeper: how do you really sort things fast systematically? Using quick sort!

 

Saving Ipython script history

%save sessionName linesToKeep

This will save script in line numbers you specify in linesToKeep,for example, 1-20 34-50 64 into a file called “sessionName.py” in your current working directory.

If you are not so specific on which lines you want to keep, you can save everything.

%save sessionName ~0/

This saves everything from the current session, denoted as “~0″ into file “sessionName.py” in your cwd (current working directory).

%save pastSession ~1/

This saves everything from the past session, denoted as “~1” into file “pastSession.py” in your cwd.

tensorflow upgrade and testing | tensorflow 升级和测试

It has been over a year since I last used tensorflow. Not only library versions have changed, but also the syntax. My old jupyter notebook was throwing errors all over the place this morning. It is time to update everything.

For tensorflow, I am using Python 3.6.3 (I used to use Python 3.5), and numpy 1.16.1. TensorFlow has a few dependencies. numpy is one of them. Note the mnist dataset has 70,000 images.

python -c ‘import tensorflow as tf; print(tf.__version__)

More

An great collection of Python notebooks | Python 笔记本集

Here is a really great collection of Python notebooks with lots and lots of links.  We start with some appetizers:

But there are so many and so much more!  You can find them from this page:

Mathematics

    • Linear algebra with Cython. A tutorial that styles the notebook differently to show that you can produce high-quality typography online with the Notebook. By Carl Vogel.

More

Python Data Types | Python 数据类型

Python type NumPy type pandas dtype Usage
str string_, unicode_ object Text
int int_, int8, int16, int32, int64, uint8, uint16, uint32, uint64 int64 Integer numbers
float float_, float16, float32, float64 float64 Floating point numbers
bool bool_ bool True/False values
datetime64[ns] datetime64 Date and time values
timedelta[ns] Differences between two datetimes
category Finite list of text values

Note that what is “str” in Python is called “object” in pandas. This is a potential source of confusion (isn’t everything in Python an object?).

TensorFlow and many ways to use it | TensorFlow人工智能系统

What is TensorFlow?    Since the best way to learn anything is to first draw upon what we already know, and then to practice.   Let’s start with what it looks like to things we already know, and then what is unique about it.

Relating to the old (or something we already knew): from its design, TensorFlow has something in common with kids’ programming environment Scratch and data mining tool such as SAS Enterprise Miner: all of them use graph to represent a process, and nodes/modules to represent specific blocks of functions, without having to write a ton of code.

SCRATCH

SCRATCH

SAS Enterprise Miner

SAS Enterprise Miner Project Flow Graph

Although not a drag-and-drop as Scratch, SAS EM,  TensorFlow does operate in similar way in that you define the graph of the process before any numerical computation is done.  According to Google (GCP), “With TensorFlow, you don’t need to be knowledgeable about the advanced math models and optimization algorithms needed to implement deep neural networks. Just download the sample code and read the tutorials and you can get started in no time. The library lowers the barrier to entry for machine learning significantly.” This has been what Scratch, SAS Enterprise Miner, Alteryx and other GUI have done for their specializations.

TensorFlow graph for a simple linear regression from my computer

TensorFlow graph for a simple linear regression from my computer

Why it is so special

There are many other great libraries that help facilitate deep learning as well.  TensorFlow is special because it is developed and maintained by Google, and it has more pros than other libraries.   Google gives it away because Google believes it will have more value if more people uses it.

One of many cool things showcased at Google TF 2017 Developer Summit is a project that detects eye disease based on the same technology that identifies cats from pictures.  What is special is that TF allows the project to focus more on solving the problem than figuring out all the code.

tensorflow detect eye disease

TensorFlow-based project that detects eye disease from images

On Github TensorFlow is one of the most starred repos, where there are many exciting resources and examples getting developed each day.

https://github.com/sherrym/tf-tutorial/blob/master/1_linear_regression_model.ipynb

https://github.com/sherrym/tf-tutorial/blob/master/2_mnist.ipynb

  • Repository of models: https://github.com/tensorflow/models

Before watching the tutorial or reading example code, it would be better to have TensorFlow installed so that you can learn by doing instead of just watching.

copyright Magic Math Mandarin

I need to start the virtual enviroment, by typing “activate universe” at the command prompt. Then type “tensorboard –logdir=graphs”. Then follow the address provided by tensorboard on the command prompt, which says “http://IBM-THINK:6006”, not “http://localhost:6006”. This was successful.

tensorboard on Magic Math Mandarin

When I ran a different model but with the same graph folder, then I need to delete all contents in graph before running the different model.  Otherwise the graph stays in the old one while information in some other tabs of TensorBoard may change.

tensorboard on Magic Math Mandarin

Python for fun 2: graphics

This Saturday we had fun playing with Python using TK library.  The computer screen is made up of lots of tiny pixels.  Using coordinate math, or, identifying each pixel by 2 numbers (horizontal and vertical) from a point (say the upper left corner), we can make the computer draw any shape.
More