Personal tools

funkload

Mar 26, 2013

Funkload: a simple buildout to spread and run your test!

Load testing can be one of the key for a successful website deployment. Moreover it can help to solve many situations. Let's see a way to spread your tests!

Recently we faced a performance issue on a Plone site, which has been reported to be slower than expected.

The application, after a migration to a new hardware, has shown to be a bit lazy compared to the expectations.

Because of the weird behaviour and confident on our Plone installation, we assumed a hardware misconfiguration.
But without evidence (numbers in this case), a claim is merely an opinion.

So, to get ride of the problem, we decided to perform multiple load test on different servers, and then compare the results.

 

read more

Mar 13, 2012

Fix funkload report generation problems by upgrading gnuplot

Filed Under:

If your funkload tests are succesful, but you get the error "Report generation failed for ..." check your gnuplot version. Older versions are in conflict with our benchmarking tool. This is my way to fix the problem

After running funkload tests [1] on CentOS 5.7 I ran into a problem: I was not able to generate the reports.

Digging into the funkload internals, I realized that the problem was related to a syntax error raised by gnuplot, that was silently swallowed by the report generator command.

The funkload generated plot commands were in conflict with the gnuplot version installed by default on the system (4.0 patchlevel 0, a very old one!).

On my shiny kubuntu I had no problems with gnuplot version 4.4.4, so I solved this problem by adding an updated gnuplot version to my buildout.

Here is how I made it work:

[buildout]
extends = base.cfg
parts+= 
    gnuplot
   
[gnuplot]
recipe = zc.recipe.cmmi
url = http://sourceforge.net/projects/gnuplot/files/gnuplot/4.4.4/gnuplot-4.4.4.tar.gz/download 
configure-options =
    --bindir=${buildout:directory}/bin

 

For this trick to work properly, the bin directory inside the buildout folder should be in your PATH environment variable. If you are using virtualenv properly, you will probably stumble upon this.

[1] Version 1.11.0, the one that, to my knowledge, works best with collective.recipe.funkload as of version 0.3.1

May 04, 2010

New collective.funkload releases

I have recently released a new version of collective.funkload and collective.recipe.funkload. There is one major improvment - funkload recorder is now working properly with collective.funkload scripts.

Here @RedTurtle we are heavily using funkload for acceptance and benchmarking tests. We have started using it in 2008 just after Bristol Performance Sprint. We have found even more useful with buildout recipe. The only missing part was the recorder. It's built-in Funkload itself, but it was not enabled in the recipe. Well - now it is ;-)

/ If you want to know how to include funkload in buildout project - check my previous blog /

Starting a recorder is quite easy:

$ ./bin/funkload record -p 8080 MyTest


for full usage call:

$ ./bin/funkload record --help


This will start proxy on 8080 port and save all your browser requests to MyTest funkload scenario. Now open your browser, change proxy configuration to localhost:8080 and click-through test case. When you finish - stop the proxy with Ctrl-C. Funkload will generate a test_MyTest.py file and notify you where you can find it. It should be now collective.funkload compatible - you can lunch:

$ ./bin/funkload test /path/to/test_MyTest.py


To test it.

Another small improvement is a PloneFLTestCase. It extends default funkload test case, implementing two additional methods helping with Plone content creation. Right now instead of using this approach:

folder_portal_factory = self._browse(server_url + "/coreloadtests/Members/" + self.user_id +"/createObject?type_name=Folder",
                                             method='get', 
                                             follow_redirect=False,
                                             description = 'Get folder portal factory')

folder_edit_url = folder_portal_factory.headers.get('Location')        
folder_id = folder_edit_url.split('/')[-2]

folder_created = self.post(folder_edit_url, params=[
    ['id', folder_id],
    ['title', 'folder'],
    ['description', ''],
    ['description_text_format', 'text/plain'],
    ['subject_existing_keywords:default:list', ''],
    ['last_referer', 'http://localhost:8080/coreloadtests/Members/' + self.user_id + '/view'],
    ['form_submit', 'Save']],
    description="Post /coreloadtests/Members/user...280843853/atct_edit")

new_folder_id = folder_created.url.split('/')[-2]

 

you can just do:

new_folder_id = self.addContent(
        server_url, 
        portal_type='Folder', 
        params=[['id', 'id'],
                ['title', 'testing title'],
                ['description', 'testing description'],
                ['description_text_format', 'text/plain'],
                ['form.submitted', '1'],
                ['last_referer', ''],
                ['form_submit', 'Save']], 
        description='Create folder')

 

It doesn't do much, but for sure it helps you keep you test case readable.

Nov 13, 2009

How to write funkload test in few minutes

After PloneConf2009 I noticed interesting changes in collective.recipe.funkload. Right now writing a simple test for funkload is just few steps.

After Plone Conference 2009 I noticed interesting changes in collective.recipe.funkload and in collective.funkload itself. Right now writing a simple stress test for funkload is much easier - you can do it in few steps.

Let's start with creating a simple package called mytest:

$ ls ./mytest
boostrap.py buildout.cfg setup.py src

$ ls ./mytest/src/mytest
__init__.py tests

$ ls ./mytest/src/mytest/tests
__init__.py test_Simple.py Simple.conf

 

buildout.cfg is also simple:

[buildout]
develop = .
parts = funkload    

[funkload]
recipe = collective.recipe.funkload
url = http://www.plone.org
eggs = mytest


where www.plone.org is the url to the site you want to test (it doesn't need to be on you localhost)

Now the funkload test configuration file (Simple.conf):

[main]
title=Simple
label=Simple test for blog.redturtle.it
description=Simple test scenario

[test_Simple]
description=Simple test scenario

[ftest]
log_to = console file
log_path = Simple-test.log
result_path = Simple-test.xml

[bench]
cycles =  5:15:30
duration = 100
startup_delay = 2
sleep_time = 2
cycle_time = 2
log_to = file
log_path = Simple-bench.log
result_path = Simple-bench.xml
sleep_time_min = 2
sleep_time_max = 2


In this example most important bench configurations:

  • cycles = 5:15:30  - contains 3 cycles: for 5, 15 and 30 concurrent users
  • duration = 100 - duration of one cycle in seconds
  • startup_delay = 2 - time to wait between starting-up threads in seconds
  • sleep_time = 2 - time to wait between test in seconds
  • cycle_time = 2 - time to wait between cycle in seconds

And the funkload test scenario (test_Simple.py)

import unittest
from collective.funkload import testcase

class Simple(testcase.FLTestCase):
    """Simple load test scenario."""

    def setUp(self):
        """Setting up test."""
        self.logd("setUp")
        self.label = 'Simple test'
        self.server_url = self.conf_get('main', 'url')

    def test_ReadOnly(self):

        self.get(self.server_url + "/",
             description="Get /")

        self.get(self.server_url + "/login_form",
             description="Get /login_form")

        self.post(self.server_url + "/login_form", params=[
            ['came_from', self.server_url],
            ['form.submitted', '1'],
            ['js_enabled', '0'],
            ['cookies_enabled', ''],
            ['login_name', ''],
            ['pwd_empty', '0'],
            ['__ac_name', 'admin'],
            ['__ac_password', 'admin'],
            ['submit', 'Login']],
            description="Post for login a user /login_form")

        self.get(self.server_url + "/folder/document1",
            description="Get /folder/document1")

        self.get(self.server_url + "/logout",
            description="Get /logout")

    def tearDown(self):
        """Setting up test."""
        self.logd("tearDown.\n")

def test_suite():
    return unittest.makeSuite(Simple)

additional_tests = test_suite

if __name__ == '__main__':
    unittest.main(defaultTest='test_suite')

 

It's a very simple scenario in which we are:

  • opening plone site (get /)
  • opening login_form (get /login_form)
  • sending a post with login credentials (post /login_form)
  • opening a document (get /folder/document1)
  • logout (get /logout).

Now you need to bootstrap the buildout:

$ cd mytest
$ python bootstrap.py


lunch the buildout:

$ ./bin/buildout


and lunch the bench:

$ ./bin/funkload bench -t mytest.tests.test_Simple


final report you can find in your buildout directory:

$ ls ./var/funkload/reports/2009-11-11-19-40-20
test_Simple-20091111T194020

$ ls ./var/funkload/reports/2009-11-11-19-40-20/test_Simple-20091111T194020
funkload.css  index.html  index.rst  tests.data  tests.gplot  tests.png


For more information please check: