Pyramid CRUD sprint is over. It was an amazing event thanks to Gaël Pasgrimaud and Patrick Gerken. We have been sprinting in Redturtle's office to improve pyramid_formalchemy and fa.jquery. We have archived most of the sprint goals!

Patrick was working on the first two tasks:

PASTER TEMPLATE

It can be used to add a skeleton to an existing project or to create a new project. If you create a new project, you must first install pyramid_formalchemy in your python environment, either with pip:

$ pip install pyramid_formalchemy

or with easy_install:

$ easy_install pyramid_formalchemy

Only after that, the paster template becomes available. The template was made with the idea that it can be used to extend existing applications. It does not create an app for you. The provided template works well with pyramid_alchemy, pyramid_routesalchemy and akhet. To bootstrap an application, call paster like that:

$ paster create -t akhet -t pyramid_fa myapp

The application is created by akhet, akhet does not know about pyramid_formalchemy, and pyramid_formalchemy cannot modify the app configuration. So you have to do this by hand. First, you must add the install dependency like explained earlier. Second, you must add the following line in the main method that returns the wsgi app, directly after Configurator has been created (The example assumes that the Configurator instance is stored under the name “config”):

...
config.include(myapp.fainit)
...

To add the minimum configuration to an existing application, you should be able to run:

$ paster create -t pyramid_fa myapp

All files that paster creates are prefixed with fa, and should not interfere with existing code. 

FANSTATIC

Patrick's second task was fanstatic integration. Fanstatic is a small but powerful framework for the automatic publication of resources on a web page. Think Javascript and CSS. It just serves static content, but it does it really well. Integrating it with fa.jquery was a time consuming task but we managed to get a working proof of concept. I will blog more details when we publish something stable.

Next tasks were handled by Gaël:

>>> from pyramid_formalchemy.views import ModelView
>>> from pyramid_formalchemy import actions
>>> class MyView(ModelView):
...     actions_categories = ('buttons', 'custom_actions')
...     defaults_actions = actions.defaults_actions.copy()
...     defaults_actions.update(edit_custom_actions=Actions())

>>> from sqlalchemy import Column, Integer
>>> from sqlalchemy.ext.declarative import declarative_base
>>> Base = declarative_base()
>>> class MyArticle(Base):
...     __tablename__ = 'myarticles'
...     edit_buttons = Actions()
...     id = Column(Integer, primary_key=True)

>>> from pyramid_formalchemy.views import ModelView
>>> from pyramid_formalchemy import actions
>>> class MyView(ModelView):
...     actions.action()
...     def extra(self):
...         # do your stuff
...         return self.render(**kw)

[app:pyramid]
...
default_locale_name = en
available_languages = fr en

I was working on the first three tasks: 

View customization

This was one of the main sprint tasks - to have a possibility to customize CRUD views per model.  You can register them simply like that:

config.formalchemy_model_view('admin',
                              model='pyramidapp.models.Foo',
                              context='pyramid_formalchemy.resources.ModelListing',
                              renderer='templates/foolisting.pt',
                              attr='listing',
                              request_method='GET',
                              permission='view')

and per Model:

config.formalchemy_model_view('admin',
                              model='pyramidapp.models.Foo',
                              context='pyramid_formalchemy.resources.Model',
                              name='',
                              renderer='templates/fooshow.pt',
                              attr='show',
                              request_method='GET',
                              permission='view')

formalchemy_model_view is an extension for config.add_view so you can use all view specific kwargs.

Widgets

We were able to finish simple implementation of autocomplete widget. It's not yet fully documented but it will be released in pyramid_formalchemy 0.4.  Here is how you use it:

from pyramid_formalchemy.renderers import pyramid_autocomplete
User.group.set(renderer=pyramid_autocomplete(filter_by='name'))

where User is your fieldset and group is your relation field; filter_by parameter is the SQLAlchemy column you want to filter (autocomplete) on. 

Events hooks

We have provided four events: 

• IBeforeValidateEvent
• IAfterSyncEvent
• IBeforeDeleteEvent
• IBeforeRenderEvent

There are also two more specific render evnts: 

• IBeforeShowRenderEvent
• IBeforeEditRenderEvent

You can use pyramid_formalchemy.events.subscriber decorator to use them:

from pyramid_formalchemy import events
from pyramidapp.models import Foo

@events.subscriber([Foo, events.IBeforeValidateEvent])
def before_foo_validate(context, event):
   #do your stuff here

Summary

It was a very productive four days. Thanks again for all your help. We should release new versions of pyramid_formalchemy and fa.jquery in the following days. If you don't want to wait - grab the development versions on github.

You can find more sprint photos here - http://www.flickr.com/photos/tags/crudsprint2011

Location e memorie

Nella località balneare in cui Hitchcock aveva ambientato il thriller “Gli Uccelli” (Bodega Bay, California), Steve McMahon ha recentemente organizzato il Cioppino Sprint, uno dei più significativi Sprint degli ultimi anni, che ha portato molti interessanti risultati, ed ha offerto anche l'occasione di ricordare la figura di Dorneles Treméa, una delle figure di maggior rilievo nella comunità Plone sudamericana, prematuramente scomparso in un incidente stradale (l'Associazione Python Brasiliana sta raccogliendo fondi per la famiglia).

Focus: Plone.org

A questo fine, secondo i suggerimenti di Alex Limi la principale area di intervento è stata il portale istituzionale Plone.org, oggetto di importanti interventi strutturali e di migliorie alle interfacce utente, risolvendo le anomalie rilevate negli ultimi mesi, e consolidando il PloneOrg add-on (l'add-on specifico del portale ufficiale). Non si è neppure trascurata qualche miglioria estetica (nella maschera di log-in, nei CSS del portale), e sono stati risolti sia i problemi dei certificati SSL di plone.org, sia quelli di plone.net: questa vetrina delle aziende Plone, ora migliorata nell'accesso alle immagini, a tutti gli effetti è stata incorporata nella sezione “support” di plone.org, che ora dispone anche di una sezione “follow” con collegamenti a Facebook e a Twitter.

Risultati

Tra gli altri risultati tecnici, è stata diminuita la dipendenza di Plone dalla Python Image Library (sempre necessaria, ma la sua assenza non è più bloccante).  A margine, c'è stato il tempo di proporre il logo dell'evento, simpatico e fuori dai canoni.

Dopo la conclusione ufficiale dello Sprint, le pagine di PloneSoftwareCenter, l'elenco delle soluzioni disponibili, sono state corredate di un sistema di rating a due pulsanti, “pollice su” e “pollice verso”, sono proseguite le migliorie stilistico-funzionali alle sezioni “documentation” e “support” ed è stata annunciata l'imminente messa in linea su plone.org degli ultimi video mancanti con gli interventi della Plone Conference 2008.

Un duro lavoro, insomma: ma, come sempre negli eventi tecnici Plone, un clima piacevole e rilassato, anche grazie alla previdenza di Ross Patterson, il cui “bar da viaggio” ha garantito l'adeguato ristoro di tutti i partecipanti.

First of all: the inherited problems

Version 1.0 of collective.amberjack.core is a simple Plone 4 compatibility version of the old 0.9 released.

When we released this we already know that the Javascript code inside was a mess, but before beginning some kind of refactoring procedure we need to know how the project will be changed in future.

The product has a core got from the Amberjack library, in last years many different programmers take part to the develop, meanwhile the product itself (and also Plone) was changing.

Amberjack original library is framework independent while Plone (that is a smart guy) rely on jQuery. Also we must not forget that Amberjack has not exactly the same target of collective.amberjack...

Wow... how many problems!

Not all those problems will be fixed in the 1.1 version, but I managed to simplify a little the code of two of the main method of amberjackPlone.js source.

How the old amberjackPlone.js was done

Saying "the code is ugly but works" is not a valid excuse... amberjackPlone.js contains two important methods:

• highlightStep is the method that highlight all the clickable/usable elements in the page for all steps.
  
• doStep is the method called to reproduce the user action on the page

What a user can do in a Plone site is a long list of different actions... list that become something like this for highlightStep:

...
if(type_obj=="checkbox" || type_obj=="radio"){
	obj.parent().addClass(theAJClass);
	obj.addClass(theAJClassBehaviour);
}
else if (type_obj=="select"){
	var highlightThis = jq(obj + " option[value="+ AjSteps[num].getValue() +"]");
	highlightThis.addClass(theAJClass);
	obj.addClass(theAJClassBehaviour);
}
else if (type_obj=="multiple_select") {
...
...going on and on...
...
else{
	obj.addClass(theAJClass);
	obj.addClass(theAJClassBehaviour);
}
...

...and like this for doStep:

...
if (type_obj == 'link') {
	AmberjackPlone.setAmberjackCookies();
	location.href = obj.attr('href');
}
else if (type_obj == 'button')
	obj.click();
else if (type_obj == 'collapsible') {
...
...going on and on and on and on...
...

Ugly enough? Do you think that the AntiIfCampaign hate us? Yes... probably...

So the first step is to remove this mess and try to port in Javascript the concept of adapter (unluckily not so flexible as Zope ones).

Even worst: many of the if statement in both doStep and highlightStep was testing the same expressions (like the "select")!

All this code has been removed.

Now both doStep and highlightStep are simply calling a sort of adapter built in the stepAdapters.js file:

AmberjackPlone.stepAdapters = {
	
	link: {
		highlight: null,
		checkStep: null,
		step: function(obj, type_obj, jq_obj, value) {
			AmberjackPlone.setAmberjackCookies();
			location.href = obj.attr('href');
		}
	},
	button: {
		highlight: null,
		checkStep: null,
		step: function(obj, type_obj, jq_obj, value) {
			obj.click();
		}
	},
	collapsible: {
...
... going on and on, but more readable!
...

For every possible action know, highlightStep and doStep will check if an handler is present in this structure (in this example the code is from the doStep):

if (AmberjackPlone.stepAdapters[type_obj] && AmberjackPlone.stepAdapters[type_obj].step)
	AmberjackPlone.stepAdapters[type_obj].step(obj, type_obj, jq_obj, value);

The highlightStep will check for highlight function.

If not if found, perform the old final else statement.

What is the checkStep section? In this sprint another group (Mirco and Simone Orsi) take a look to the feature that test if a user has completed all the steps before can go to the next page. Again, for some possible action the tour can try to test if you have finished your work.

In the old approach this will lead us to a new big and monolithic function. Now is only a new tiny method that call an handler.

What's next?

Adding some documentation, for sure...

Also other part of the code can be changed to become more clear...

...to be continued!

Symposium

We have started collective.amberjack presence in Sorrento with Massimo's presentation at the European Plone Symposium. 

Sprint

Then we had two sprinting days. We have started day one with brainstorming. We had three objectives/questions:

• collective.amberjack sandbox
• refactoring the code - how to simplify collective.amberjack tour definition and registration
• translations - how to manage tour/steps translations

thanks to a fruitful discussion (garbas, dukebody, gborelli, miziodel, shywolf9982, sorry if I forgot your name ;-) we have right now answers to all of our dilemmas.

SanDbox

First idea of having a sandbox for collective.amberjack came up at last Plone Conference in Budapest. We have collected a lot of ideas for different use cases. We have had also some internal brainstorms in RedTurtle about it. Finally after confronting all the concepts - during Sorrento brainstorming - we will have a simple solution which should satisfy most of the users:

Code refactoring

We have started refactoring collective.amberjack. There are 3 areas of interests: javascript, tour definition and tour registration. Luca was doing a great job trying to simplify javascript part. He also prepare a good startup for future enhancements. I was refactoring tour part. We have decided to use a 

configuration baseD tour definition,

which looks like that:

[amberjack]
steps = 
        0_create-a-new-folder
        1_fill-out-the-fields
        2_publish-the-folder
        3_all-done
title = Add and publish a Folder

[0_create-a-new-folder]
blueprint = collective.amberjack.blueprints.step
title = Create a new folder
url = /
text = Folders are one of ....
validators = 
        python: isManager
        python: isNotFolderCreated
microsteps = 
        0_0_microstep
        0_1_microstep
        0_2_microstep

[0_0_microstep]
blueprint = collective.amberjack.blueprints.microstep
description = If you don't want to perform the ...
automatically done by your browser.

[0_1_microstep]
blueprint = collective.amberjack.blueprints.microstep
idstep = menu_add-new
description = Click the [Add new...] drop-down menu.

[0_2_microstep]
blueprint = collective.amberjack.blueprints.microstep
idstep = new_folder
description = Select [Folder] from the menu.

This concept is well known across the community. I hope it will be also more human readable comparing to python dictionary ;-). We have a working version in the trunk. For new tours we have also 

new tour registration 

- comparing to old approach (registration only using ZCML) you can now use also TTW registration. Other important change is that tour doesn't need to be a python package any more. Because it's a simple .cfg file right now you can simply zip it into an archive (zip, tar) and register it. We have a working implementation for web source and local file system registration (which you can find in the trunk). Archive file can be shipped with multiple tours and with

translationS

which right now are still just a concept that need to be implemented. We would like to make a use of i18ndude tool but because it was designed for ZPT this could be a wrong approach. Overall idea is to keep all the files (tour and the translations) together, something like:

- mytours.zip
  |- tour1.cfg
  |- tour1_en.po
  |- tour1_it.po
  |- tour1.pot
  |- tour2.cfg
  |- tour2_en.po
  ...

Fixing bugs

Mirco and Federica were working together and did an enormous job fixing most of the 1.1a release bugs. Thanks to Rob we have solved also some TinyMCE problems that we have had. We have still some open issues, like 'new portlet creation' tour that need some attention so we still need your help!

What's next?

The current development focus is on 1.1 release, which includes:

• new tour definition (python dictionary based tour will still work until 1.2)
• new tour registration (old registration will still work until 1.2)
• finish translation implementation
• fix last bugs
• deploy sandbox probably on demo.plone.org
• collective.amberjack.windmill - which should become default interface for creating and editing tours; we have right now something more then just proof of concept:

Non so esattamente quale sia la percezione dall'esterno, ma ogni anno la conference rappresenta, per me, la possibilità di ritrovare vecchi amici che hanno una passione in comune.

Si parte!

Già alla cena pre-conference l'accoglienza è stata incredibile.
Aaron aveva un posto buono a fianco a lui e subito i gruppi si sono mescolati: italiani, americani, polacchi, francesi, tedeschi, olandesi.

Tutti li' a parlare di tutto. C'era quello mi chiedeva di amberjack, quello che si ricordava di un piatto italiano e voleva la ricetta, quello che ti racconta che si è finalmente preso una spider e ne è così orgoglioso.

Tutto come se fossimo vecchi compagni di scuola che hanno condiviso qualcosa di particolare e prezioso.

Spesso si dice:

plone il cms, plone il framework, ma soprattutto plone la comunità

La solita retorica, trita retorica, dell'open source, vero? beh, magari no.

Se ne sei fuori, o meglio se ne vuoi stare fuori, sei libero di vederla così, ma così non è. Frequento e ho frequentato in passato altre comunità (i partner ibm, i lug, l'xpug) e nonostante abbia incotnrato alti livelli di eccellenza, questa community è quella più coinvolgente, appassionante, e professionalmente energizzante.

ZEA

Il gruppo ZEA si è riunito dopo diverso tempo in una riunione live. Ho potuto conoscere di persona e di scambiare opinioni con Matt, Russ, Kit e Jean Paul, che conoscevo solo in modo marginale, ma anche di rivedere vecchi amici come Xavier e Godefroid. E' stato utile, credo che il gruppo sia oggi più compatto e propositivo.

Plone Italia

Il riaggregarsi del gruppo italiano è stato allo stesso modo entusiasmante.

Ognuno aveva la sua idea, il suo modo di vedere le cose. E si è discusso, preso in considerazione le varie tesi.

Quello che ne è venuto fuori lo potete o lo avete letto sulla ML, ma il punto interessante, ancora una volta, non è stato solo il risultato finale.

Il confronto, il discutere, il mettere in dubbio quello che è il tuo piccolo mondo di fronte agli altri è un modo estremamente appagante di crescere. E ti rendi conto che dopo 12 anni di lavoro e non so quanti di informatica, non sei ancora arrivato. C'è sempre qualcuno che ha qualcosa di nuovo da raccontarti un punto di vista diverso, e cresci.

collective.amberjack

Poi c'è stata la mia personalissima ciliegina sulla torta: collective.amberjack.

Ho potuto presentarlo in un talk ufficiale, ho "guidato" uno sprint e raccontato a tutti gli sprinter gli avanzamenti del lavoro fatto.
Messa giù così ha un che di vagamente autocelebrativo e presuntuoso. E sicuramente una parte di me è decisamente soddisfatta dell'interesse che ne è uscito. Il punto però è un altro.

Perdersi nella comunità

Ho sempre visto tutti coloro che contribuivano come una specie di esseri con capacità tecniche, qualità personali di livello superiore.

Il che in parte è anche vero. Quello che è, però, ancora più vero non sono le capacità tecniche, ma la voglia di proporsi e di perseverare.

In fondo quello che è cambiato in me rispetto alle precedenti conference è stato il voler mettere sul piatto quello che so fare, tanto o poco non importa. La comunità ha apprezzato ed evidenziato. Poi, magari, amberjack sarà stata anche un buona idea, ma in ogni caso è stato l'impegno che ha fatto la differenza.

E' la differenza che passa tra lo stare nell'ombra e sfruttare l'open source, come molti purtroppo fanno, e il fondersi nella comunità: più dai, più ti impegni, più avrai in cambio in termini di credibilità, riconoscenza, amicizia!

Ritorno a casa

Alla fine del viaggio la lezione imparata è che tutta la retorica dell'open source, del fatto che tutti possono contribuire, che è la community che segna la strada è niente di più che la realtà.

Non ci si può, però, limitare a raccontarla (alle conferenze, ai linux day, in post sui blog), occorre declinarla in azioni, in codice, in attività, in quello che sai fare.

Allora? che pensi di proporre alla prossima ploneconf?

We have organized a small amberjack sprint. There was a lot of things to-do and only 5 sprinters: vincentfretin (online from France), fdelia, massimo, mirna and amleczko (all directly from Ferrara).

Beneath you can find summary of the sprint:

This is a report of the work made today
we just refer to the issue that were in today's worklist

put tours in a pipeline:
 - Add a link to the first tour in the first page of the second one, so a user
   who selects the second one directly will be able to takethe first one first,
   since you need to create the MyFolder folder first.
 - The whole Amberjack should manage branches like "if you are not logged in,
   then, log in".
 - At the end of a tour, add a link to the next tutorial.

We decided to have preconditions that has to added to a step (macrostep).
For example:
isAnonymous
isAuthenticated
etc.

there is already a initial implementation (amleczko).
completed - there are two basic validators: isAnonymous and isAuthenticated
when a precondition is not met a warning "the step is not visible"
is displayed. next, we should disable the step and all the followers.

we need to choose and implement a good strategy to define dependencies between
tours


 - Check the entered texts: check if the texts entered in the fields are the
   ones we expected. If not we may have problems with ids (in the url) and with
   the xcontent. Otherwise, let both url and xcontent be function of what the
   user entered.
  
we decided that before submitting we have a ajax call that checks if
all the mandatory things has been done by the user.
to be done.


 - collective.jqueryui. actually we have a js file in core/skins. it contains
   both jquery.ui and jquery.ui.draggable.
   we could remove the jquery.ui stuff and leave only a file ui.draggable.js.
   unluckly, collective.js.jquery + ui.draggable.js doesn't work (raise a js error)
  
Vincent Fretin, both complete this ticket and created two different profiles
for plone3.2 and plone>=3.3
  
 - Write tests for meta directives, tour_manager...

Andrew Mleczko and federica d'elia wrote a couple of tests and also refactored the
code see:
http://svn.plone.org/svn/collective/collective.amberjack.core/trunk/docs/HISTORY.txt


 - Add an option to the Unified Installer
 - Add a run-profile option to plone.recipe.unifiedinstaller to run additional
   profiles when the Plone Site is created
  
to be done
  
 - Tours: (massimo azzolini e mirna bagnolatti)
  Tour 5: Format a page using the visual editor
  Tour 6.1: Create internal links
  Tour 6.2: Create external links
  to be completed

  Tour 7: Upload an image
  completed, but cannot show the last "all done" step since the url is
  function of the name of the image uploaded. we need a fix [1]

  Tour 8: Insert image on a page
  completed, but cannot correctly save. when submitting the form the stuff
  inside kupu is not saved. [2]
 
  Tour 9: Upload and link to a File
  completed, with the same problems as above:
  1. the file obj's url is function of the name of the file itself. [1]
  2. cannot save the kupu stuff [2]
 
  Tour 10: Using the Contents tab
  the main problem is that we have the same page, but we want to move on
  many different tour steps. see [1]
 
  Tour 12: Create a static text portlet
  almost finished. problems:
  1. click on Save button show the alert "are you sure you want to leave..." [2]
 
 
Some open issues

 - [1] Provide a better way to manage the progression of the steps: too often
   path + xpath/xcontent do not suffice.
   Use the step position number 1/5, 2/5 etc.
to be done
  
 - [2] all the submit buttons (form.button.save & co) have problems if just
   "submitting", maybe we have to say that we are clicking on that button.
   we cannot either click() on it since it opens the alert window
   "are you sure you want to leave..."

to be investigated