I spent last weekend (22-23 October) in Kiev promoting Plone and Pyramid at PyCon UA. It was an amazing opportunity to spread the latest news about Plone4 and its enhancements. 

The Kiev event was fruitful experience from two points of view: the audience was not aware of what has been happening in Plone since 3.x; they were mostly Django developers. I think I did my best to explain what are the key values of using Plone and Pyramid together and what are the benefits comparing to other frameworks like Django. I met several fantastic guys deeply interested in what we are doing in both Plone and Pyramid community. Hope to see you all on another Python event.

Apart of Plone talk there were other invited speakers. Let me mention few of them.

Opening talk done by Tarek was about Packaging. He explained current situation of packaging in Python and the future of it. I hope we will have setup.cfg to rull-them-all in near future and that the community will start to use proper versioning soon.

Then there was Armin with Basket of Random Python Snippets. I don't like the idea of showing ONLY code while doing your presentation but I strongly encourage you to read them online. Some of them are really great tips!

Last talk I attended was done by Alexander and I must admit - I have enjoyed it a lot. He was showing "SQLAlchemy: a better ORM". I was never a big fun of Django ORM but now I'm strongly convinced that Pyramid + SQLAlchemy + pyramid_formalchemy is a way much easier to play with.

Bottom line: I must agree with Yury that it would be more interesting event if all of the speakers present slides in English (like on RuPy2011 last October). Nevertheless I enjoy this weekend a lot. I love the city and the atmosphere. The topics and invited speakers were properly chosen so everybody could find a subject to discuss about. Thanks again and hopefully see you next year.

And tomorrow it is time for Plone Conference 2011 which will end this autumn's conference marathon.

Last weekend (14-16 Oct 2011) I've attended RuPy (Ruby + Python), a strongly dynamic conference held in Poznań (Poland) and organized by GIK Association. I had a great opportunity to share RedTurtle's "Pyramid and Plone" integration use case with wider audience. It was truly an open source event and a great opportunity to meet geeks from the Ruby community. 

It was RuPy's 3rd 4th edition and apart it was the biggest (ca. 300 attendees) I must say - it was the best (I was attending all previous editions). The GIK have chosen Poznań International Fairs as the conference venue which was a damn good idea. During 3 days of the conference there was ca. 30 events (talks, workshops and sprints). Most of them was related to Ruby which is a problem each year. Getting more support from Python community is highly appreciated! 

What I think is notable is the speaker feedback system. Each room had 3 containers marked: :-) :-| and :-(. After each session the speaker was receiving his anonymous feedback from the audience. From the speaker point of view (so also mine) it's surprisingly easy and gives you great opportunity to check if what you've just said wasn't a completely bullshit. What I would suggest is to make bigger effort in promoting it after each talk.

Interesting talks

my subjective list of notable talks

Programmer Anarchy

by Fred George

Writing your own programming language

by José Valim

Tradeoffs and Choices: Why Ruby Isn't Python

by Yehuda Katz

I didn't find the slides but I hope RuPy team will publish them soon. In the mean time - short abstract:

"When a Pythonista first dives into Ruby, he is confronted with a strange and unusual world. Multiple kinds of functions, implicitness everywhere, violations of the Zen of Python galore! In this talk, Yehuda will talk about the tradeoffs in Ruby's language design: why, in many ways, Ruby couldn't be more like Python even if it wanted to."

Summary

It was an amazing weekend with some interesting discussions with people from Python and Ruby world. Thanks RuPy, hope to see you next year!

Last but not least the organization committee followed best practices and shared their website source code to wide public.

Pyramid CRUD sprint is over. It was an amazing event thanks to Gaël Pasgrimaud and Patrick Gerken. We have been sprinting in Redturtle's office to improve pyramid_formalchemy and fa.jquery. We have archived most of the sprint goals!

Patrick was working on the first two tasks:

PASTER TEMPLATE

It can be used to add a skeleton to an existing project or to create a new project. If you create a new project, you must first install pyramid_formalchemy in your python environment, either with pip:

$ pip install pyramid_formalchemy

or with easy_install:

$ easy_install pyramid_formalchemy

Only after that, the paster template becomes available. The template was made with the idea that it can be used to extend existing applications. It does not create an app for you. The provided template works well with pyramid_alchemy, pyramid_routesalchemy and akhet. To bootstrap an application, call paster like that:

$ paster create -t akhet -t pyramid_fa myapp

The application is created by akhet, akhet does not know about pyramid_formalchemy, and pyramid_formalchemy cannot modify the app configuration. So you have to do this by hand. First, you must add the install dependency like explained earlier. Second, you must add the following line in the main method that returns the wsgi app, directly after Configurator has been created (The example assumes that the Configurator instance is stored under the name “config”):

...
config.include(myapp.fainit)
...

To add the minimum configuration to an existing application, you should be able to run:

$ paster create -t pyramid_fa myapp

All files that paster creates are prefixed with fa, and should not interfere with existing code. 

FANSTATIC

Patrick's second task was fanstatic integration. Fanstatic is a small but powerful framework for the automatic publication of resources on a web page. Think Javascript and CSS. It just serves static content, but it does it really well. Integrating it with fa.jquery was a time consuming task but we managed to get a working proof of concept. I will blog more details when we publish something stable.

Next tasks were handled by Gaël:

>>> from pyramid_formalchemy.views import ModelView
>>> from pyramid_formalchemy import actions
>>> class MyView(ModelView):
...     actions_categories = ('buttons', 'custom_actions')
...     defaults_actions = actions.defaults_actions.copy()
...     defaults_actions.update(edit_custom_actions=Actions())

>>> from sqlalchemy import Column, Integer
>>> from sqlalchemy.ext.declarative import declarative_base
>>> Base = declarative_base()
>>> class MyArticle(Base):
...     __tablename__ = 'myarticles'
...     edit_buttons = Actions()
...     id = Column(Integer, primary_key=True)

>>> from pyramid_formalchemy.views import ModelView
>>> from pyramid_formalchemy import actions
>>> class MyView(ModelView):
...     actions.action()
...     def extra(self):
...         # do your stuff
...         return self.render(**kw)

[app:pyramid]
...
default_locale_name = en
available_languages = fr en

I was working on the first three tasks: 

View customization

This was one of the main sprint tasks - to have a possibility to customize CRUD views per model.  You can register them simply like that:

config.formalchemy_model_view('admin',
                              model='pyramidapp.models.Foo',
                              context='pyramid_formalchemy.resources.ModelListing',
                              renderer='templates/foolisting.pt',
                              attr='listing',
                              request_method='GET',
                              permission='view')

and per Model:

config.formalchemy_model_view('admin',
                              model='pyramidapp.models.Foo',
                              context='pyramid_formalchemy.resources.Model',
                              name='',
                              renderer='templates/fooshow.pt',
                              attr='show',
                              request_method='GET',
                              permission='view')

formalchemy_model_view is an extension for config.add_view so you can use all view specific kwargs.

Widgets

We were able to finish simple implementation of autocomplete widget. It's not yet fully documented but it will be released in pyramid_formalchemy 0.4.  Here is how you use it:

from pyramid_formalchemy.renderers import pyramid_autocomplete
User.group.set(renderer=pyramid_autocomplete(filter_by='name'))

where User is your fieldset and group is your relation field; filter_by parameter is the SQLAlchemy column you want to filter (autocomplete) on. 

Events hooks

We have provided four events: 

• IBeforeValidateEvent
• IAfterSyncEvent
• IBeforeDeleteEvent
• IBeforeRenderEvent

There are also two more specific render evnts: 

• IBeforeShowRenderEvent
• IBeforeEditRenderEvent

You can use pyramid_formalchemy.events.subscriber decorator to use them:

from pyramid_formalchemy import events
from pyramidapp.models import Foo

@events.subscriber([Foo, events.IBeforeValidateEvent])
def before_foo_validate(context, event):
   #do your stuff here

Summary

It was a very productive four days. Thanks again for all your help. We should release new versions of pyramid_formalchemy and fa.jquery in the following days. If you don't want to wait - grab the development versions on github.

You can find more sprint photos here - http://www.flickr.com/photos/tags/crudsprint2011

For one of the projects we are using a very aggressive Varnish configuration. We are caching also html (not for long but still) for anonymous users. It could be a problem if you have some dynamical features - like rating. 

But not any more. Using this simple vcl add-on you can purge article view after making a vote:

sub vcl_recv {
...
   if (req.http.Cookie && req.http.Cookie ~ "statusmessages=") { 
       purge("req.url ~ " req.url);
       pass;
   }
...
}

It simply checks if you have a statusmessages cookie in request (after rating an article Plone is displaying confirmation status message) and purge current url from cache. Simple, clean and it works!

After few months of discussions it is finally here: Pyramid CRUD sprint. It will be held in Ferrara just before EuroPython (June 16-19). Everyone is welcome! We will focus on two main goals: 

• make the CRUD more flexible then it is right now (using ZCA for view and subscribers registrations), increase test coverage and make it production ready
• add missing widgets and fix existing one

So if you want to taste classical Italian kitchen, visit UNESCO World Heritage Site and improve Pyramid - join us!

For more details check www.coactivate.org/projects/pyramid-crud-sprint

In my previous post I was trying to see what could be the benefit for Pylons Community to have a crud. We have a working prototype based on bfg. Now we would like to rewrite it for Pyramid. I've received interesting feedback, but the most promising was pyramid_formalchemy posted by Gael. 

What I miss in pyramid_formalchemy

I will start with description of our use case. At the moment we are using our CRUD to handle most of the form generation and CRUD operations. We are using it in several projects, one of them with ca. 100 models. The part of the application that is exposed to anonymous user (like registration) or it's more complex (like wizard multip-step forms) we are handling with formish. For those 100 models we are using heavily ZCA for customizing View/List templates. As far I can see in the pyramid_formalchemy source code:

def formalchemy_admin(config, route_name,
                      factory='pyramid_formalchemy.resources.Models',
                      view='pyramid_formalchemy.views.ModelView',
                      package=None, models=None, forms=None, session_factory=None):
...
config.add_view(context='pyramid_formalchemy.resources.ModelListing',
                    renderer='pyramid_formalchemy:templates/admin/listing.pt',
                    attr='listing',
                    request_method='GET',
                    permission='view',
                    **kw)

you can only pass one ModelView for config registration and practically you can't customize any of the templates per model.
What we would actually like to have is a ZCA registration, like:

  <view
      for=".interfaces.IModel"
      view=".listing"
      renderer="templates/my_model_listing.pt"
      permission="list"
  />

ZCML is of course optional...

Other benefit of this approach is a possibility to register other non-crud views (not only list/add/view/edit/delete) only for specific model (can be helpful for some of the widgets implementation, i.e. autocomplete).

This is our main concern. 

At the end we miss some widgets (which should be part of fa.jquery project and can be extended easily). One of the most interesting one is relation widget (with several variations),  which right now is missing :-)

This year, EuroPython is in Florence, Italy, which is ca. 140km from where I live now. It couldn't get easier to attend and propose a talk:

Web development is a complexity challenge nowadays. Growing number of functionalities results in customer expectations increase which makes project design more difficult. Using proper tools that suite your customer needs is essential.

In this talk I would like to present two successful stories using closely together Pyramid and Plone. Basing on these examples I wished to highlight the main reasons for using Plone as a CMS only and letting Pyramid do the rest (vertical application). Moreover, I will underscore good and bad practices during integration process and how to make farsighted architectural decisions in a right moment.

This year, everyone who has bought EuroPython ticket can vote. So If you have one and you'd like to see this talk, vote for it directly from abstract page.

I had recently problems with one of munin plugins - it disappear from munin graphs. In my munin-node.log I have a lot of:

[21134] Service 'temperature' timed out.

How to change munin timeout value? There is nothing about it in munin documentation. But if you look into source code you will find that it exists. Adding:

timeout 60

to munin-node.conf or your plugin specific configuration does the tick!

What we have is a working version of a traversal CRUD. General concept is similar to Sergey Volobuev Kelpie. Everything is based on repoze.bfg 1.2 (though it should work smoothly on 1.3).

Model definition is done using SQLAlchemy declarative approach.  This choice provoke us to use  FormAlchemy as a form generator (with fa.jquery widget support). It's easy to start with but difficult to extend later.  That's why it's the first thing we would like to change in the future.  Everything is glued together with bfg hybrid traversal approach. 

What we are planing to do (probably not exactly in that order):

• check if somebody else didn't start similar project and maybe share concepts or do it together
• check possibility of integrate existing solutions with Pyramid
• reuse what can be reused from our bfg crud and move it to Pyramid
• organize a python sprint that will work on these topics

 

With the newest version of munin.zope you have 4 different plugins:

• ZServer threads
• ZODB activity
• Zope cache parameters
• Zope memory usage
  
  

Using aggregation you can end like this:

What it does? It takes data from multiple zeoclients (in this case every project from A-E have 4 zeoclients) and renders only total amount per project.
Here is munin.conf which does this trick:

[Server1;projectA]
  address 127.0.0.1
[Server1;projectB]
  address 127.0.0.2
[Server1;projectC]
  address 127.0.0.3
[Server1;projectD]
  address 127.0.0.4
[Server1;projectE]
  address 127.0.0.5

[Server1;Aggregated]
  update no
  total_memory.update no
  total_memory.graph_category Zope
  total_memory.graph_title Aggregated Zope memory
  total_memory.graph_order \
    projectA \
    projectB \
    projectC \
    projectD \
    projectE

  total_memory.projectA.sum \
    projectA:projecta_zopememory_instance_1.VmSize \
    projectA:projecta_zopememory_instance_2.VmSize \
    projectA:projecta_zopememory_instance_3.VmSize \
    projectA:projecta_zopememory_instance_4.VmSize
  total_memory.projectA.label project A

  total_memory.projectB.sum \
    projectB:projectb_zopememory_instance_1.VmSize \
    projectB:projectb_zopememory_instance_2.VmSize \
    projectB:projectb_zopememory_instance_3.VmSize \
    projectB:projectb_zopememory_instance_4.VmSize
  total_memory.projectB.label project B

  total_memory.projectC.sum \
    projectC:projectc_zopememory_instance_1.VmSize \
    projectC:projectc_zopememory_instance_2.VmSize \
    projectC:projectc_zopememory_instance_3.VmSize \
    projectC:projectc_zopememory_instance_4.VmSize
  total_memory.projectC.label project C

  total_memory.projectD.sum \
    projectD:projectd_zopememory_instance_1.VmSize \
    projectD:projectd_zopememory_instance_2.VmSize \
    projectD:projectd_zopememory_instance_3.VmSize \
    projectD:projectd_zopememory_instance_4.VmSize
  total_memory.projectD.label project D

  total_memory.projectE.sum \
    projectE:projecte_zopememory_instance_1.VmSize \
    projectE:projecte_zopememory_instance_2.VmSize \
    projectE:projecte_zopememory_instance_3.VmSize \
    projectE:projecte_zopememory_instance_4.VmSize
  total_memory.projectE.label project E

For more information please check:
http://munin-monitoring.org/wiki/aggregate_examples
http://munin-monitoring.org/wiki/PercentGraphHowto
http://munin-monitoring.org/wiki/stack_examples