Decision trees | 决策树 jué cè shù

Today, we focused our the class on decision tree. Decision tree is a way to organize data.   You can look at it this way: you ask a bunch of questions and make a bunch of decisions, and organize data based on these decisions.

For example, if our data consists of colors and shapes of 3 pieces of fruits. We have 1 yellow apple, 1 red apple and a yellow banana. We have two features: shape and color.

By organizing our data, we can identify types of fruit. We go through our data on shape and color one by one. If we first organize our data by color, we know that will incorrectly group the yellow apple with the yellow banana. But if we first organize our data by shape, that will right away group apples and banana separately. So we organize this data by shape, and then by color (if we want to make a distinction between yellow apple and red apple).
Magic Math Mandarin
The way to organize data may be (highly likely) different for another dataset of fruits. But you get the point: we organize data to best group things. In each step of the way, our data gets more organized. The “energy distribution” has become lower entropy.

That was a classification tree model.

When we have lots of decision trees for different random parts of a larger data, we have the so-called “random forest” 随机森林 model, originated by Leo Breiman.

We showed in class how to code a decision tree from scratch.  Here is a shorter version using Python sklearn library.

 

Entropy | 熵 shāng

Nothing is lost, nothing is created, everything is transformed.
― Antoine Lavoisier (August 1743 – 8 May 1794)

Unlike before, we started the class today with a quote. This is because it is really difficult to talk about entropy, and we made many analogies (such as water flows from high to low, a mirror broken never, or almost never, returns to whole again) to bring our attention to how things work in daily life that we have taken for granted.

Some theory/hypothesis says that the universe started with Big Bang, a state with very low entropy. There are many states of high entropy than low entropy (imagine 10…000 to 1). So we will have to cycle through lots of high entropy states before it is low again. Well, we only have barely touched the topic. Whereas our true goal is to talk about the so-called decision tree model, which we will cover tomorrow.

To help you remember the word “entropy” and its meaning (as if we knew!), “en” comes from “energy”.  “tropy” means “transfom”, and comes from Latin.

Entropy is a measure of the number of possible ways energy can be distributed in a system.

By the way, Lavoisier 拉瓦锡 was a great chemist.

zero, one and two | 零,一,二

It is not easy for a young child to comprehend multiplication by 1, as how they are taught in school is often the robotic multiplication table.   She or he can very quickly answer mutiplications by 2, or 3.    Because of this, questions like “what is the product of 1,2, 3, 4” (i.e. 4 factorial) can get a wide range of answers because the number “1” confuses the young mind.

Pychologist says that an infant learns the number 2 before the number 1.   And we can see why: with 2, there is something to compare against, like two fingers.  If there is only one finger, there is no variation, it is confusing.

When we teach multiplication, don’t forget to show that math is an integral part of the real world around us.   It is invented to simplify addition.  Multiply by 1 means just the thing itself.  Multiply by 2 means adding two of this thing together.  Multiply by 3 means adding three of the thing together.  The thing can be a bag of candies or the footage of a home.

Finally, we should show children how to use computers (not calculators) to do computations.   While a question like “give me the sum from 1 to 199” can be solved within seconds with math tricks, a slightly different question “give me the product from 1 to 199” won’t work with the same trick.  But if you know how to make the computer do the job, you can still answer it within seconds.

 

An great collection of Python notebooks | Python 笔记本集

Here is a really great collection of Python notebooks with lots and lots of links.  We start with some appetizers:

But there are so many and so much more!  You can find them from this page:

Mathematics

    • Linear algebra with Cython. A tutorial that styles the notebook differently to show that you can produce high-quality typography online with the Notebook. By Carl Vogel.

More

Math olympia medal count analysis | 奥数奖牌分析

The International Mathematical Olympiad (IMO) is an annual six-problem mathematical olympiad for pre-college students younger than 20. The first IMO was held in Romania in 1959. As we will see, eastern Europeans were top performers in the IMO in the earlier years. You can find the summary data analysis in our Jupyter Notebook on GitHub.

It has since been held annually, except in 1980 (what happened in 1980?). More than 100 countries, representing over 90% of the world’s population, send teams of up to six students (under 20 years old) to compete.

Problems cover extremely difficult algebra, pre-calculus, and branches of mathematics not conventionally covered at school and often not at university level either, such as
– projective and complex geometry
– functional equations
– combinatorics
– number theory (where extensive knowledge of theorems is required).

No calculus is required. Supporters of not requiring calculus claim that this allows “more universality and creates an incentive to find elegant, deceptively simple-looking problems which nevertheless require a certain level of ingenuity”.

Rank Country Appearance Gold Silver Bronze Honorable_Mentions
0 1 China 32 147 33 6 0
1 2 United States 43 119 111 29 1
2 3 Russia 26 92 52 12 0
3 4 Hungary 57 81 160 95 10
4 5 Soviet Union 29 77 67 45 0
5 6 Romania 58 75 141 100 4
6 7 South Korea 30 70 67 27 7
7 8 Vietnam 41 59 109 70 1
8 9 Bulgaria 58 53 111 107 10
9 10 Germany 40 49 98 75 11

Debt | 债务

Exactly two years ago, we wrote about national debt.  It was close to $20 trillion at that time.  Now it is $22 trillion.


We are  presenting  very large numbers.

But large is only a relative term, depending on the unit we are using, and relative to what.

According to the Institute of International Finance, global debt, as of 3Q2018, is close to $244 trillion.
About one third of the debt was added in the last ten years or so. So that means that over the last ten years the total global debt grew by a half.

You can see it from the Global Debt Monitor January 2019 Report.

This probably does not mean much to you or me, unless we have some comparisons.

Visuals can help you see the numbers, but it stops short of helping us to understand the number, since money in dollars is just money in dollars unless we compare it with something.

How about we compare it with GDP (gross domestic product)? GDP in dollar is the value of all the things people produce or service for a period of time in dollar.

So debt to GDP ratio is like the amount of money you owe at the end of the year relative to the amount of money you have made over the year.   When the ratio is over 1, it means what we owe is more than what we have made in a year.

Now hopefully we can understand the ratio a little bit.

For a great narrative of history of US debt to gdp ratio, see “The Long Story of U.S. Debt, From 1790 to 2011, in 1 Little Chart” from The Atlantic by Matt Phillips.

The article was written on Nov 13, 2012. But history does not go away.

You can connect the dots to the following chart, which you can find from Federal Reserve Bank of St. Louis.  It seems that we have debt to GDP ratio getting close to historical highest level.

That was right after World War II.

So what is in the US debt?

The total US debt now is about $22 trillion.

The U.S. debt to China is $1.138 trillion as of October 2018. That’s 29 percent of the $3.9 trillion in Treasury bills, notes, and bonds held by foreign countries.

The rest of the $22 trillion national debt is owned by either the American people or by the U.S. government itself. China has the greatest amount of U.S. debt held by a foreign country.

Domestically, the total US household debt as of 4Q2018 is at $13.54 trillion (New York Fed). For a fantastic and fascinating visual account of the numbers, see the report by New York Fed.

You can find the numbers and reports easily from different federal reserve banks and government office such as the Congressional Budget Office, and the US Treasury.

These numbers, ratios and time series by component are a lot more interesting and tell a whole lot more than everyday noisy news.

Herstory of money-1 | 钱的历史

This is the first installment of a series of post about money, cryptocurrency and credit scoring, accompanied by Python Jupyter Notebook in our GitHub repo on credit scoring.

In this post we talk about paper money 纸币.  The reason why we keep it in the practical math category is because the herstory of money is also the herstory of math.  In God we trust and in math we trust.  God made the universe with beautiful math.

Did you know that paper money 纸币 was first used in ancient China around the 11th century 北宋朝?

Paper money was used broadly during those days due to shortage of copper and the convenience of paper money. However, the convenience combined with the unlimited power of the government to print money lead to inflation, subsequently the loss of credibility of the government, and its eventual downfall. So, even though the Northern Song dynasty had an advance monetary system, its credit failed due to long and costly wars.

Did you know that the Chinese Southern Song 南宋 dynasty government printed money in no less than six ink colors to prevent counterfeiting?

They printed notes with intricate designs and sometimes even with mixture of unique fiber in the paper to avoid counterfeiting. That was in 1107!

Backed by gold or silver too?

Isn’t it amazing that their nationwide standard currency of paper money was backed by gold or silver?! That was in between 1265 and 1274.

In the 13th century, Chinese paper money of Mongol Yuan 元 became known in Europe through the accounts of travelers, such as Marco Polo

“All these pieces of paper are, issued with as much solemnity and authority as if they were of pure gold or silver… with these pieces of paper, made as I have described, Kublai Khan causes all payments on his own account to be made; and he makes them to pass current universally over all his kingdoms and provinces and territories, and whithersoever his power and sovereignty extends… and indeed everybody takes them readily, for wheresoever a person may go throughout the Great Khan’s dominions he shall find these pieces of paper current, and shall be able to transact all sales and purchases of goods by means of them just as well as if they were coins of pure gold”
— Marco Polo, The Travels of Marco Polo

Python for fun 4


Today we work on animation with hundreds of triangles, balls, squares and vickies (you can call the shapes anything you like).   Shapes are defined by coordinates (x,y).   A huge improvement over last time is that we use a loop to add shape to lists of shapes.
More

Pineapple math

This Saturday morning, Victoria went on an adventure of counting how many “eyes” a pineapple has.  She used a marker to mark the ones she has counted: 10, 15,

It is not Fibonacci sequence as we expected. Why?

Geometry and machine learning

Geometry helps us see clearly without cluttering our minds with jargon and notations.  We will begin with something everyone in machine learning does.

Linear Regression= ” linear regression is a linear approach for modeling the relationship between a scalar dependent variable y and one or more explanatory variables (or independent variables) denoted X.”

What does it mean? Let’s think about it from geometry.  If we don’t know how tall Emma is (dependent variable y, or “label”) since we haven’t met her. But we can guess her height by other numbers, such as the length of her shadow (independent variable X, or “feature”). Our guess can be quite accurate if we are in same city where she lives and we are measuring everything at the same time of day.

That was easy.   Now we are going to guess the height of many different people, based on the the length of their shadows.  Again, our guess can be quite accurate if we all live in same city  and we are measuring everything at the same time of day.   But this is usually not the case.    Say if we live in northeast America and the measures are taken during early afternoon in autumn, our predictions may not be perfect but are still in the ball park.  Otherwise we really cannot make a good prediction.

What are we to do with measures that are not taken under perfect conditions? If we know the location and time when the measures are taken, then we can make adjustments to the numbers before using them. But there still can be lots of problems if we made mistakes in taking down the numbers or locations or time. Like a trades person, we have different tools to fix problems. One is called “least squares method”. Some other methods are “weighted least squares”, “maximum likelihood”, “gradient descent” and etc.