Comments on: The future can be written in RPython now

By: Billy

Billy — Wed, 13 Apr 2011 23:05:21 +0000

Christian,

Eric Floehr in his pycon 2011 presentation states that pypy runs pyevolve code 3-4 times faster than cpython. But I’ve downloaded his code and ran it myself, and cpython runs the code 3 times faster than pypy (the exact opposite of what Eric proclaims, but I’m running python 2.6 (and 2.7) and pypy 1.4.1, so that might make some of the difference).

I’ve ran several of the examples in the trunk version of pyevolve, and for quick examples, cpython wins, and for longer jobs pypy usually wins (which is expected). I’ve also tried different population sizes for GPs to see if that would make a difference since memory usage would be different, and that does not affect the results when comparing cpython and pypy.

I’ve only ran Eric’s code on one machine, but I’ll try others and see if I get different results. If I can’t reproduce Eric’s results, I might send him an email and see if he can give me some details.

Again, I want to thank you for pyevolve, I’ve really enjoyed using it.

Billy

By: Christian S. Perone

Christian S. Perone — Wed, 13 Apr 2011 12:16:26 +0000

Hello Billy, this is surprise to me too, I haven’t tested the GP core using PyPy yet, but it should not so slow as you reported. Have you tried to compare the CPython vs yours PyPy version with some Genetic Algorithm ? I’ll probably need to profile it in PyPy and check why is it so slow when compared to CPython. Have you checked the memory footprint between two interpreters ? What is the size of your population ?

By: Billy

Billy — Wed, 13 Apr 2011 01:55:54 +0000

To let you know, I’m running pyevolve 0.6, Multiprocessing off. A sample run of 10 generations of a GP (depth = 5) takes about 7 minutes on pypy, and only 2 minutes on python 2.7. pypy is jit enabled. Pysco is not being used.

I’ll try running a few other apps through pypy and python and see if the performance is about the same or if pypy is faster.

Any info you can provide will be helpful!

By the way, pyevolve is awesome!

By: Billy

Billy — Tue, 12 Apr 2011 22:41:02 +0000

Christian,

I’m using Pypy 1.4.1, and I’ve tried running a pyevolve GP on 3 different machines with different processors (AMD, Intel, 64 and 32 bit), and each run is much slower than regular python. The version of PyPy I’m running is the JIT version compiled version you can download on the pypy web site.

Are there special command line parameters I should be passing to pypy?
What command line arguments did you use to start and run your pypy scripts?

When I run a pystone test, pypy out performs the python version. So this really surprises me.

Thanks
Billy

By: Stefan Behnel

Stefan Behnel — Fri, 28 May 2010 06:36:18 +0000

I agree that the comparison wouldn’t be fair, but a Cython implementation would still be the easiest way to get a good base line for how close to C speed your PyPy benchmark results are. Cython is the language that many Python users write numeric calculations in, so the example you give would have been predestined for a Cython implementation – unless your real fear is the competition.

By: Carl Friedrich Bolz

Carl Friedrich Bolz — Wed, 26 May 2010 15:10:43 +0000

Cool. No hurries though

By: Christian S. Perone

Christian S. Perone — Wed, 26 May 2010 12:20:14 +0000

In my opinion, the start-up time makes part of the whole run, would be nice to have it cited, but it’s part of the whole run since you’ll have to lose this time anyway (specially in optimization problems for instance). If you run a test with the other interpreters and with Jython, you’ll see that the startup time is amortized in the long run time, what makes Jython so slow in the benchmarks isn’t the the start-up time, but the overall performance, even when the server mode is enabled, you can note this in the course of the evolution, the time required to evolve 500 generations for example, is very high when compared with other interpreters (and even after the jvm has performed JIT optimizations like inlining, etc…).

By: Serhiy

Serhiy — Wed, 26 May 2010 08:11:01 +0000

This is kinda weird to see Jython performing so badly. Why didn’t you include warmup rounds for Jython? It’s known that startup time of JVM is pretty big and warmups are required to have adequate benchmarks(let HotSpot do it’s job in JVM).

By: Brendan Wood

Brendan Wood — Wed, 26 May 2010 03:48:46 +0000

“all testes were performed over 10k generations”

Ah, there’s the key. I missed that in your post and assumed you were running the optimization until it hit the global minimum, rather than for a set number of generations. So yes, I agree with you now.

By: fijal

fijal — Tue, 25 May 2010 18:21:47 +0000

Hm.

I’m getting increasingly annoyed by cython PR. Let’s face it – Cython is *not* a python runtime and as such doesn’t have to be included in every python runtimes comparison whether developers like it or not. It can be included, but it’s not mandatory, especially if module X does not compile under cython