Personal tools

Andrew Mleczko

May 04, 2010

New collective.funkload releases

I have recently released a new version of collective.funkload and collective.recipe.funkload. There is one major improvment - funkload recorder is now working properly with collective.funkload scripts.

Here @RedTurtle we are heavily using funkload for acceptance and benchmarking tests. We have started using it in 2008 just after Bristol Performance Sprint. We have found even more useful with buildout recipe. The only missing part was the recorder. It's built-in Funkload itself, but it was not enabled in the recipe. Well - now it is ;-)

/ If you want to know how to include funkload in buildout project - check my previous blog /

Starting a recorder is quite easy:

$ ./bin/funkload record -p 8080 MyTest


for full usage call:

$ ./bin/funkload record --help


This will start proxy on 8080 port and save all your browser requests to MyTest funkload scenario. Now open your browser, change proxy configuration to localhost:8080 and click-through test case. When you finish - stop the proxy with Ctrl-C. Funkload will generate a test_MyTest.py file and notify you where you can find it. It should be now collective.funkload compatible - you can lunch:

$ ./bin/funkload test /path/to/test_MyTest.py


To test it.

Another small improvement is a PloneFLTestCase. It extends default funkload test case, implementing two additional methods helping with Plone content creation. Right now instead of using this approach:

folder_portal_factory = self._browse(server_url + "/coreloadtests/Members/" + self.user_id +"/createObject?type_name=Folder",
                                             method='get', 
                                             follow_redirect=False,
                                             description = 'Get folder portal factory')

folder_edit_url = folder_portal_factory.headers.get('Location')        
folder_id = folder_edit_url.split('/')[-2]

folder_created = self.post(folder_edit_url, params=[
    ['id', folder_id],
    ['title', 'folder'],
    ['description', ''],
    ['description_text_format', 'text/plain'],
    ['subject_existing_keywords:default:list', ''],
    ['last_referer', 'http://localhost:8080/coreloadtests/Members/' + self.user_id + '/view'],
    ['form_submit', 'Save']],
    description="Post /coreloadtests/Members/user...280843853/atct_edit")

new_folder_id = folder_created.url.split('/')[-2]

 

you can just do:

new_folder_id = self.addContent(
        server_url, 
        portal_type='Folder', 
        params=[['id', 'id'],
                ['title', 'testing title'],
                ['description', 'testing description'],
                ['description_text_format', 'text/plain'],
                ['form.submitted', '1'],
                ['last_referer', ''],
                ['form_submit', 'Save']], 
        description='Create folder')

 

It doesn't do much, but for sure it helps you keep you test case readable.

Apr 28, 2010

Where is my FSS file?

Filed Under:

Sometimes you need to know where is your FSS file on disk. There is an easy way to get the full path

Following example returns full path to ATFile 'file' field on disk:

>>> field = self.context.getField('file')
>>> fss_info = field.storage.getFSSInfo('file',self.context)
>>> fss_info
<FSSFileInfo at test-file.pdf/>
>>> fss_strategy = field.storage.getStorageStrategy('file',self.context)
>>> fss_strategy
<iw.fss.strategy.DirectoryStorageStrategy instance at 0x21332c68>
>>> fss_props = field.storage.getStorageStrategyProperties('file',self.context,fss_info)
>>> path = fss_strategy.getValueFilePath(**fss_props)
>>> path
'/opt/buildout.plone/var/fss_storage_global/5b/5b09/5b09c322ae211207649e1803fe012cca_file'

Mar 26, 2010

"Future is bright, future is Plone"

Sometimes when you are doing a lot of Plone development and integrations you could miss the big picture: Plone is not just a CMS. It's a damn good CMS with almost unlimited possibilities of integration. However its 'unlimity' has started to be one of its biggest limitations.

I'm working right now on a project that uses ore.contentmirror to serialize Plone content data to postgresql which can be later reused in repoze.bfg. I have found several similar deployments in the Plone community which were quite inspiring.

Plone integrationsThe scenario is almost always the same:

  • to use Plone as a CMS (backend)
  • to have fast framework (frontend) to serve my backend data
  • to include in frontend layer Web2.0 functionality and some other dynamic stuff
  • to make benefit of external indexing server for full text search, for both frontend and backend

 

You can of course use solr with collective.solr for search engine but thanks to tsearch2 some of the search/ranking functionalities can be taken directly from postgresql (making the whole stack much smaller and easier to maintain).

Serializing Plone data to SQL opens many possibilities. But with Plone you can do much more.

Plone integration world

You can integrate other CMS's like Alfresco or Sharepoint using CMIS. You can connect your Plone instance to Salesforce. Instead of using tsearch2 or solr as an indexing engine - you can use Google Search Appliance inside your Plone instance. If you want to start online document collaboration you can even connect it to Google Docs.

External services and applications are not the end of the story. There is of course WSGI with collective.xdv and Deliverance and bunch of other interesting middlewares and we shouldn't forget about PAS plugins.

It's possible that I missed something interesting. Plone community is very creative, thought. It's hard to be always up to date. That's why I'm sure that I miss a place where integrators and developers can share they successful stories and discuss potential use cases. A place which will make Plone future bright.

 

Nov 13, 2009

How to write funkload test in few minutes

After PloneConf2009 I noticed interesting changes in collective.recipe.funkload. Right now writing a simple test for funkload is just few steps.

After Plone Conference 2009 I noticed interesting changes in collective.recipe.funkload and in collective.funkload itself. Right now writing a simple stress test for funkload is much easier - you can do it in few steps.

Let's start with creating a simple package called mytest:

$ ls ./mytest
boostrap.py buildout.cfg setup.py src

$ ls ./mytest/src/mytest
__init__.py tests

$ ls ./mytest/src/mytest/tests
__init__.py test_Simple.py Simple.conf

 

buildout.cfg is also simple:

[buildout]
develop = .
parts = funkload    

[funkload]
recipe = collective.recipe.funkload
url = http://www.plone.org
eggs = mytest


where www.plone.org is the url to the site you want to test (it doesn't need to be on you localhost)

Now the funkload test configuration file (Simple.conf):

[main]
title=Simple
label=Simple test for blog.redturtle.it
description=Simple test scenario

[test_Simple]
description=Simple test scenario

[ftest]
log_to = console file
log_path = Simple-test.log
result_path = Simple-test.xml

[bench]
cycles =  5:15:30
duration = 100
startup_delay = 2
sleep_time = 2
cycle_time = 2
log_to = file
log_path = Simple-bench.log
result_path = Simple-bench.xml
sleep_time_min = 2
sleep_time_max = 2


In this example most important bench configurations:

  • cycles = 5:15:30  - contains 3 cycles: for 5, 15 and 30 concurrent users
  • duration = 100 - duration of one cycle in seconds
  • startup_delay = 2 - time to wait between starting-up threads in seconds
  • sleep_time = 2 - time to wait between test in seconds
  • cycle_time = 2 - time to wait between cycle in seconds

And the funkload test scenario (test_Simple.py)

import unittest
from collective.funkload import testcase

class Simple(testcase.FLTestCase):
    """Simple load test scenario."""

    def setUp(self):
        """Setting up test."""
        self.logd("setUp")
        self.label = 'Simple test'
        self.server_url = self.conf_get('main', 'url')

    def test_ReadOnly(self):

        self.get(self.server_url + "/",
             description="Get /")

        self.get(self.server_url + "/login_form",
             description="Get /login_form")

        self.post(self.server_url + "/login_form", params=[
            ['came_from', self.server_url],
            ['form.submitted', '1'],
            ['js_enabled', '0'],
            ['cookies_enabled', ''],
            ['login_name', ''],
            ['pwd_empty', '0'],
            ['__ac_name', 'admin'],
            ['__ac_password', 'admin'],
            ['submit', 'Login']],
            description="Post for login a user /login_form")

        self.get(self.server_url + "/folder/document1",
            description="Get /folder/document1")

        self.get(self.server_url + "/logout",
            description="Get /logout")

    def tearDown(self):
        """Setting up test."""
        self.logd("tearDown.\n")

def test_suite():
    return unittest.makeSuite(Simple)

additional_tests = test_suite

if __name__ == '__main__':
    unittest.main(defaultTest='test_suite')

 

It's a very simple scenario in which we are:

  • opening plone site (get /)
  • opening login_form (get /login_form)
  • sending a post with login credentials (post /login_form)
  • opening a document (get /folder/document1)
  • logout (get /logout).

Now you need to bootstrap the buildout:

$ cd mytest
$ python bootstrap.py


lunch the buildout:

$ ./bin/buildout


and lunch the bench:

$ ./bin/funkload bench -t mytest.tests.test_Simple


final report you can find in your buildout directory:

$ ls ./var/funkload/reports/2009-11-11-19-40-20
test_Simple-20091111T194020

$ ls ./var/funkload/reports/2009-11-11-19-40-20/test_Simple-20091111T194020
funkload.css  index.html  index.rst  tests.data  tests.gplot  tests.png


For more information please check:

Nov 07, 2009

RuPy conference

Short summary about RuPy - strongly dynamic conference. The philosophy of RuPy is to put togheter Python & Ruby experts with young programmers and to support a good communication channel for East-West exchange of prospective ideas.

I have been delegated as a RedTurtle member to join RuPy conference that has been held in Poznan, my hometown city.

The idea behind the conference is to try to animate central- and east-european Python and Ruby communities. By inviting experts we want to broaden our knowledge and undersanding of presented concepts. The conference is meant to serve the exchange of ideas on the scientific ground, that is use of mentioned languages for certain scientific problems like, for example, text parsing, and on the buisness ground, like use of Django or Rails for creating web applications.

http://rupy.eu/about/


The state of Plone

short talk presented by Maciej Dziergwa about current Plone situation - based on Matt Hamilton's PloneConf2009 talk. After the talk we had a short discussion about current situation of Polish Plone Community. Plone is still not popular in Poland as it is in Italy...

Compiled Websites with Plone, Django and SSI

very interesting talk by Łukasz Langa and Wojciech Lichota about Plone, Django and SSI deployment. Using stxnext.staticdeployment you can deploy Plone site to static files. You can add dynamic stuff (like commenting) using Django. Finally you can glue everything together using Apache SSI mod_include. If you need search engine you can use and integrate, everything what I've mentioned before, with Xapian (or Solr).
You can find slides here.

Caching techniques in Python

some interesting caching techniques explained by Michał Domański and Michał Chruszcz. In Plone world we already know plone.memoize - a decorator for caching the values of functions and methods. You can use it of course outside Zope/Plone environment. If you need something more - check out i.e. pylibmc. At the end they've covered also reverse proxy with Varnish.