# More Python Fun

Continuing to get acquainted with Python, I looked at the book „Introduction to Data Science“ by Joel Grus. I have it in German as an electronic book, which is unfortunately no fun to read, although the content is solid – with restrictions. The most important problem is the formatting of the Python code, which does not even work in landscape mode. The printed book uses a smaller font and lines are completely in one line. You can and should download the examples. There are also some misprints. I found one in the introduction to the Bayes formula, where the fraction is reversed. But most importantly, I have my doubts if the book is really a help for the starting data scientist. The presentation is very short. You will need a good background in math and programming, or have someone to ask if you do not understand the theory or the code. The code gets pretty advance pretty fast.

However, this is not a review of the book. But I take an example of the book to show how Python can be used from within Euler Math Toolbox. The following code reads the bible and prints the 10 most common words.

infile = open("bible.txt","r")

from collections import Counter

num_words = 10

counter = Counter(word.lower()
for line in infile
for word in line.strip().split()
if word)

for word, count in counter.most_common(num_words):
print(str(count)+"\t"+word)

infile.close()

I downloaded the file „bible.txt“ from the net and put it into the folder, where the IPython notebook resides. The output is the following.

64183	the
51379	and
34746	of
13643	to
12799	that
12559	in
10263	he
9840	shall
8987	unto
8835	for

Now, let us do this programming exercise in Euler Math Toolbox via Python.

>>> from collections import Counter
>function python testbible (filename) ...
$""" 10 most common words in the bible$"""
$infile = open(filename,"r")$num_words = 10
$counter = Counter(word.lower()$    for line in infile
$for word in line.strip().split()$    if word)
$res = ""$for word, count in counter.most_common(num_words):
$res = res+str(count)+"\t"+word+"\n"$infile.close()
$return res$endfunction
>testbible("bible.txt")
64183   the
51379   and
34746   of
13643   to
12799   that
12559   in
10263   he
9840    shall
8987    unto
8835    for

>>> help(testbible)
Help on function testbible in module __main__:

testbible(filename)
10 most common words in the bible


The import was done with a direct Python command. The function returns a string with the answer. But it could just as well use „eumat.dump(„….“)“ for a direct output. Note, that you do not need to indent the Python definition. That is done automatically by EMT.

The „$“-signs in the function definition allow copying and pasting the code into an EMT notebook. They are not visible in the EMT window. Here is another example, done in IPython. def sieve(N): v = [True for _ in range(N+1)] for k in range(2,N+1): if k*k>N: break if v[k]: m=2*k while m<=N: v[m]=False m=m+k return [k for k in range(N+1) if v[k] and k>1] s=sieve(100_000_000) from math import log x=[k for k in range(1000,len(s),1000)] y=[(s[k]/(k*log(k))-1-(log(log(k))-1)/log(k))*log(k)**2 for k in range(1000,len(s),1000)] import matplotlib as mpl import matplotlib.pyplot as plt fig, ax = plt.subplots(figsize=(8,8)) ax.plot(x,y)  This code computes the primes up to 100 Million and plots the actual size of the k-th prime, divided by a known asymptotics for the size of the k-th prime. We only use Python to do the sieve. For the plot, we use Euler Math Toolbox. >function python sieve (N) ...$v = [True for _ in range(N+1)]
$for k in range(2,N+1):$    if k*k>N:
$break$    if v[k]:
$m=2*k$         while m<=N:
$v[m]=False$             m=m+k
$return [k for k in range(N+1) if v[k] and k>1]$endfunction
>tic; s = sieve(100 000 000); toc;
Used 29.407 seconds
>n = length(s)
5761455
>k = 1000:1000:n;
>y = (s[k]/(k*log(k))-1-(log(log(k))-1)/log(k))*log(k)^2;
>plot2d(k,y):


As you see, the sieve takes about 30 seconds on my computer (Core I7, 4.0GHz). Of course, it is much faster to do that in C.

>function tinyc csieve (N) ...
$ARG_DOUBLE(N);$int n=(int)N;
$CHECK(n>10,"Need a positive integer.");$char *v = (char *)getram((n+1)*sizeof(char));
$CHECK(v,"Out of heap space.");$
$for (int i=0; i<n+1; i++) v[i]=1;$for (int k=2; k<n+1; k++)
${$   if (k*k>n) break;
$if (v[k])$   {
$int m=2*k;$       while (m<=n)
${$           v[m]=0; m+=k;
$}$   }
$}$int count=0;
$for (int k=2; k<n+1; k++) count+=v[k];$
$header *out=new_matrix(1,count);$CHECK(out,"Out of heap space");
$double *res=matrixof(out);$
$int j=0;$for (int k=2; k<n+1; k++)
$if (v[k])$       res[j++]=k;
moveresults(out);
\$endfunction
>s=csieve(100)
[2,  3,  5,  7,  11,  13,  17,  19,  23,  29,  31,  37,  41,  43,  47,
53,  59,  61,  67,  71,  73,  79,  83,  89,  97]
>tic; s=csieve(100 000 000); toc;
Used 1.459 seconds


The procedures to communicate from EMT to Tiny C are a bit involved. But the result is 15 times faster. The basic code is a direct translation of the Python code.

Depending on your heap space of Euler Math Toolbox, you can do more primes. If you compress the sieve using bits, you could to a lot more. But for this code and the default size, 100 millions are the limit. Increasing the stack size to 2048 MB, I could do all primes up to 1 billion in 15 seconds.

Diese Website verwendet Akismet, um Spam zu reduzieren. Erfahre mehr darüber, wie deine Kommentardaten verarbeitet werden.