Reducing Python processes resource usage

Repository of web frameworks and applications for Microsoft IIS.
User avatar
Posts: 56
Joined: 01 Nov 2013, 01:50

Reducing Python processes resource usage

28 May 2014, 22:04

Hi,

Our Python processes take too much resources, around 90 MB / 10% - 20% CPU per each, while on Linux the usage is much lower.

Any idea what parameters we can tune or change to lower this usage?

Thanks.

User avatar
Posts: 110
Joined: 07 Mar 2012, 10:22

Re: Reducing Python processes resource usage

29 May 2014, 05:03

Hello,

1. What is an application you running in Python?
2. What is the bitness of python.exe and bitness of OS? Is it virtual machine?
3. Did you measure response time from app on Linux and Windows? What is the difference?

User avatar
Posts: 56
Joined: 01 Nov 2013, 01:50

Re: Reducing Python processes resource usage

01 Jun 2014, 10:48

1) Django 1.4
2) Both 64-bit, VMWare guest.
3) Yes, Linux has 2-3 times faster performance, I will post exact measures soon.

User avatar
Posts: 110
Joined: 07 Mar 2012, 10:22

Re: Reducing Python processes resource usage

02 Jun 2014, 07:13

1. Are applications running on Windows and Linux box identical?
2. Do the application use a database? What is database type? What the driver?

User avatar
Posts: 56
Joined: 01 Nov 2013, 01:50

Re: Reducing Python processes resource usage

02 Jun 2014, 15:58

Ruslan wrote:1. Are applications running on Windows and Linux box identical?
2. Do the application use a database? What is database type? What the driver?


1) Yes, it's exactly the same code.

2) MSSQL used for Windows (with SQL Client 2012, pyodbc and pyodbc-azure), Oracle 11 used for Linux (with cxOracle and oracle-instantclient 11).

User avatar
Posts: 110
Joined: 07 Mar 2012, 10:22

Re: Reducing Python processes resource usage

03 Jun 2014, 06:27

You have quite different environments on Windows and Linux, so it's difficult to compare them correctly.
By default Zoo installs 32-bit Python even for 64-bit machines. But our experience shows that using 64-bit Python may lead to increased performance of web apps.
So, please consider installing 64-bit Python (on dev machine first). And notice that you will need to have all python native libs for 64-bit version too.

User avatar
Posts: 56
Joined: 01 Nov 2013, 01:50

Re: Reducing Python processes resource usage

05 Jun 2014, 08:47

Hi,

If you recall other post on this matter, this exactly what we found as the reason for unbearable slowdowns, hence all our deployments are 64-bit now.

That said, even now the Python process works slower than on Linux / Apache, and take much more resources.

User avatar
Posts: 110
Joined: 07 Mar 2012, 10:22

Re: Reducing Python processes resource usage

06 Jun 2014, 05:59

Lets create simple django view without db access in your code and measure request time on Windows and Linux.

Example view:

Code: Select all
def hello_world(request):
    return HttpResponse('Hello World!')


Measure with ab:

Code: Select all
ab -c 10 -n 10000 http://<url to test view>


Please show output of ab for Windows and Linux servers.

Return to Helicon Zoo

Who is online

Users browsing this forum: No registered users and 9 guests