Saving Ipython script history

%save sessionName linesToKeep

This will save script in line numbers you specify in linesToKeep,for example, 1-20 34-50 64 into a file called “” in your current working directory.

If you are not so specific on which lines you want to keep, you can save everything.

%save sessionName ~0/

This saves everything from the current session, denoted as “~0″ into file “” in your cwd (current working directory).

%save pastSession ~1/

This saves everything from the past session, denoted as “~1” into file “” in your cwd.

tensorflow upgrade and testing | tensorflow 升级和测试

It has been over a year since I last used tensorflow. Not only library versions have changed, but also the syntax. My old jupyter notebook was throwing errors all over the place this morning. It is time to update everything.

For tensorflow, I am using Python 3.6.3 (I used to use Python 3.5), and numpy 1.16.1. TensorFlow has a few dependencies. numpy is one of them. Note the mnist dataset has 70,000 images.

python -c ‘import tensorflow as tf; print(tf.__version__)


An great collection of Python notebooks | Python 笔记本集

Here is a really great collection of Python notebooks with lots and lots of links.  We start with some appetizers:

But there are so many and so much more!! Okay, I am just going to copy (fork) most of this page:


  • Linear algebra with Cython. A tutorial that styles the notebook differently to show that you can produce high-quality typography online with the Notebook. By Carl Vogel.
  • More

Women, men and money

From a dataset on restaurant tip included in the Python seaborn library, we investigate using Python with Jupyter Notebook how much money women and men relatively make, and why.

Most people don’t tip more or less because of server is a man or woman, but we do tip according to tip amount (for equal services).

I will tell you base on data analysis the real reason women make less is mostly because:
1. women work mostly lunch shifts
2. lunch bills are smaller than dinner ones.

Indeed, in every industry, including food services like in the restaurants, women have shouldered more share of the most difficult job in the world: parenting, and dedicated their time and energy to their families.

As a result, they have earned less money than men on average. Let’s give our appreciation to women for their roles in taking care of families!

There is a profound reason why we say “Mother nature”!

Should women work more dinner shifts and leave the important job of taking care of children to …?

That is a difficult question… Take a look at this Jupyter Notebook for the detailed data analysis.

Python Data Types | Python 数据类型

Python type NumPy type pandas dtype Usage
str string_, unicode_ object Text
int int_, int8, int16, int32, int64, uint8, uint16, uint32, uint64 int64 Integer numbers
float float_, float16, float32, float64 float64 Floating point numbers
bool bool_ bool True/False values
datetime64[ns] datetime64 Date and time values
timedelta[ns] Differences between two datetimes
category Finite list of text values

Note that what is “str” in Python is called “object” in pandas. This is a potential source of confusion (isn’t everything in Python an object?).

TensorFlow and many ways to use it | TensorFlow人工智能系统

What is TensorFlow?    Since the best way to learn anything is to first draw upon what we already know, and then to practice.   Let’s start with what it looks like to things we already know, and then what is unique about it.

Relating to the old (or something we already knew): from its design, TensorFlow has something in common with kids’ programming environment Scratch and data mining tool such as SAS Enterprise Miner: all of them use graph to represent a process, and nodes/modules to represent specific blocks of functions, without having to write a ton of code.



SAS Enterprise Miner

SAS Enterprise Miner Project Flow Graph

Although not a drag-and-drop as Scratch, SAS EM,  TensorFlow does operate in similar way in that you define the graph of the process before any numerical computation is done.  According to Google (GCP), “With TensorFlow, you don’t need to be knowledgeable about the advanced math models and optimization algorithms needed to implement deep neural networks. Just download the sample code and read the tutorials and you can get started in no time. The library lowers the barrier to entry for machine learning significantly.” This has been what Scratch, SAS Enterprise Miner, Alteryx and other GUI have done for their specializations.

TensorFlow graph for a simple linear regression from my computer

TensorFlow graph for a simple linear regression from my computer

Why it is so special

There are many other great libraries that help facilitate deep learning as well.  TensorFlow is special because it is developed and maintained by Google, and it has more pros than other libraries.   Google gives it away because Google believes it will have more value if more people uses it.

One of many cool things showcased at Google TF 2017 Developer Summit is a project that detects eye disease based on the same technology that identifies cats from pictures.  What is special is that TF allows the project to focus more on solving the problem than figuring out all the code.

tensorflow detect eye disease

TensorFlow-based project that detects eye disease from images

On Github TensorFlow is one of the most starred repos, where there are many exciting resources and examples getting developed each day.

  • Repository of models:

Before watching the tutorial or reading example code, it would be better to have TensorFlow installed so that you can learn by doing instead of just watching.

copyright Magic Math Mandarin

I need to start the virtual enviroment, by typing “activate universe” at the command prompt. Then type “tensorboard –logdir=graphs”. Then follow the address provided by tensorboard on the command prompt, which says “http://IBM-THINK:6006”, not “http://localhost:6006”. This was successful.

tensorboard on Magic Math Mandarin

When I ran a different model but with the same graph folder, then I need to delete all contents in graph before running the different model.  Otherwise the graph stays in the old one while information in some other tabs of TensorBoard may change.

tensorboard on Magic Math Mandarin

Python for fun 2: graphics

This Saturday we had fun playing with Python using TK library.  The computer screen is made up of lots of tiny pixels.  Using coordinate math, or, identifying each pixel by 2 numbers (horizontal and vertical) from a point (say the upper left corner), we can make the computer draw any shape.