Per lo sviluppo di un'applicazione mi sono trovata a dover generare un pdf il cui contenuto consiste in un modulo, ad esempio il modulo del censimento, compilato con i dati degli utenti dell'applicazione presi run-time dal sistema.

Avrei impiegato troppo tempo a disegnare il modulo insieme ai dati dell'utente, in più, per certi moduli (come il modulo censimento) sono già forniti i pdf.

Pensare di caricare il modulo come immagine di sfondo non è fattibile. Prima di tutto per la qualità scarsa del pdf finale e poi perché le dimensioni del file di output aumentano molto a causa dell'immagine usata come sfondo.

Il file da ottenere è un modulo già fornito in versione pdf, ma a cui ho bisogno di aggiungere testo.

La soluzione, a mio avviso migliore, è stata quella di usare ReportLab per disegnare un pdf fatto unicamente dei dati dell'utente per poi fare un merge del primo con il pdf contente il modulo, usando PyPDF.

L'unica difficoltà in questo approccio è creare il pdf contenente i dati di riempimento. Infatti questi dati devono essere nella posizione giusta, prioprio come se stessimo compilando il modulo.

Per superare questo ostacolo agilmente ho usato uno screen ruler. Io ho preferito "Free Ruler" per Mac OS X tra i vari righelli che ho trovato in rete, soprattutto per il suo tick mark interattivo che permette di fare una misura abbastanza accurata.

Ho aperto il pdf con il modulo da compilare, e poi ho misurato con il righello i punti in cui inserire i dati. Per creare il pdf con i dati di riempimento ho usato ReportLab che consente di scrivere stringhe di testo in un punto esatto, utilizzando le coordinate x e y.

Ecco la MIA ricetta

Passo 1

Aprire il pdf che rappresenta la base del nostro impasto, ovvero  il modulo da compilare, usando la classe PdfFileReader di pyPdf.

import codecs
from pyPdf import PdfFileWriter, PdfFileReader

input1 = PdfFileReader(codecs.open(path_modello_censimento, "rb"))

Passo 2

Preparare il pdf con i nostri canditi: i dati di riempimento. Qui entra in gioco ReportLab.

import StringIO
from reportlab.pdfgen import canvas
from reportlab.lib.units import cm

packet = StringIO.StringIO()
can = canvas.Canvas(packet)
...
textobject = can.beginText()
textobject.setTextOrigin( 11.94*cm, 20.12*cm)
textobject.textLine('hello world')
...
can.drawText(textobject)
can.showPage()
can.save()
packet.seek(0)

Come unità ho usato i centrimenti, sia nel fissare l'origine di ogni striga di testo inserita, sia quando ho misurato con il righello. 11.94 e 20.12 sono due esempi di coordinate.

Tenere presente che l'orgine si trova in basso a sinistra. Ma con l'istruzione seguente possiamo spostare l'orgine in alto.

can.translate(cm, cm)

Passo 3

Aprire il pdf creato sopra con PdfFileReader.

input2 = PdfFileReader(packet)

Passo 4

Aprire il pdf finale, che conterrà il merge dei due pdf precedenti con la classe PdfFileWriter di pyPdf.

output = PdfFileWriter()

Passo 5

Adesso la parte più divertente: "Impastare".

page0 = input1.getPage(0)
page0.mergePage(input2.getPage(0))
output.addPage(page0)

Dopo aver unito i due pdf pagina per pagina si arriva finalment al punto 6..

Passo 6

Adesso abbiamo il nostra risultato finale "output" che può essere memorizzato ovunque si voglia, io lo scrivo in un file temporaneo per completare l'esempio:

import tempfile
temp_file_name = tempfile.mktemp()
outputStream = codecs.open(temp_file_name, mode="wb")
output.write(outputStream)
outputStream.close()
os.unlink(temp_file_name)

L'elaborazione richiede un po' di tempo, ma il risultato finale è buono. Buon appetito!

I spent last weekend (22-23 October) in Kiev promoting Plone and Pyramid at PyCon UA. It was an amazing opportunity to spread the latest news about Plone4 and its enhancements. 

The Kiev event was fruitful experience from two points of view: the audience was not aware of what has been happening in Plone since 3.x; they were mostly Django developers. I think I did my best to explain what are the key values of using Plone and Pyramid together and what are the benefits comparing to other frameworks like Django. I met several fantastic guys deeply interested in what we are doing in both Plone and Pyramid community. Hope to see you all on another Python event.

Apart of Plone talk there were other invited speakers. Let me mention few of them.

Opening talk done by Tarek was about Packaging. He explained current situation of packaging in Python and the future of it. I hope we will have setup.cfg to rull-them-all in near future and that the community will start to use proper versioning soon.

Then there was Armin with Basket of Random Python Snippets. I don't like the idea of showing ONLY code while doing your presentation but I strongly encourage you to read them online. Some of them are really great tips!

Last talk I attended was done by Alexander and I must admit - I have enjoyed it a lot. He was showing "SQLAlchemy: a better ORM". I was never a big fun of Django ORM but now I'm strongly convinced that Pyramid + SQLAlchemy + pyramid_formalchemy is a way much easier to play with.

Bottom line: I must agree with Yury that it would be more interesting event if all of the speakers present slides in English (like on RuPy2011 last October). Nevertheless I enjoy this weekend a lot. I love the city and the atmosphere. The topics and invited speakers were properly chosen so everybody could find a subject to discuss about. Thanks again and hopefully see you next year.

And tomorrow it is time for Plone Conference 2011 which will end this autumn's conference marathon.

Last weekend (14-16 Oct 2011) I've attended RuPy (Ruby + Python), a strongly dynamic conference held in Poznań (Poland) and organized by GIK Association. I had a great opportunity to share RedTurtle's "Pyramid and Plone" integration use case with wider audience. It was truly an open source event and a great opportunity to meet geeks from the Ruby community. 

It was RuPy's 3rd 4th edition and apart it was the biggest (ca. 300 attendees) I must say - it was the best (I was attending all previous editions). The GIK have chosen Poznań International Fairs as the conference venue which was a damn good idea. During 3 days of the conference there was ca. 30 events (talks, workshops and sprints). Most of them was related to Ruby which is a problem each year. Getting more support from Python community is highly appreciated! 

What I think is notable is the speaker feedback system. Each room had 3 containers marked: :-) :-| and :-(. After each session the speaker was receiving his anonymous feedback from the audience. From the speaker point of view (so also mine) it's surprisingly easy and gives you great opportunity to check if what you've just said wasn't a completely bullshit. What I would suggest is to make bigger effort in promoting it after each talk.

Interesting talks

my subjective list of notable talks

Programmer Anarchy

by Fred George

Writing your own programming language

by José Valim

Tradeoffs and Choices: Why Ruby Isn't Python

by Yehuda Katz

I didn't find the slides but I hope RuPy team will publish them soon. In the mean time - short abstract:

"When a Pythonista first dives into Ruby, he is confronted with a strange and unusual world. Multiple kinds of functions, implicitness everywhere, violations of the Zen of Python galore! In this talk, Yehuda will talk about the tradeoffs in Ruby's language design: why, in many ways, Ruby couldn't be more like Python even if it wanted to."

Summary

It was an amazing weekend with some interesting discussions with people from Python and Ruby world. Thanks RuPy, hope to see you next year!

Last but not least the organization committee followed best practices and shared their website source code to wide public.

It has been a tiring but exciting week in Florence.

After four years of national PyCon Italy, the team has been excellent and set up an event that clearly shows the level of maturity of the community.

Over the years since the first EuroPython conference, the big change is that we don't have to justify our language choice anymore.

Actually, I would not even be a programmer it where was no Python. I was a system administrator, more like a plumber than an internal decorator.
Python has taken me by hand, teaching and letting me write things I would not have attempted in other languages available at the time.

As programmers, we make trade-offs and take compromises several times per day, while we design and write applications.
Very often we take a different road than we did the day before, because we've seen what lays ahead and want to avoid it, or the customer has changed his mind, or we have a better understanding of the whole.

As a matter of fact the right answer to a programming question is often "it depends", because the architectures, markets and business models are getting more and more complex, and we as software craftsmen have none of the certainties than mathematicians or computer science people take for granted.

Fortunately for us and our customers, software can be easily changed, and nothing prevents experimenting with different approaches if time allows.
But once in a while - typically at the start of a development cycle - we have bigger choices, the decisions that make room for the following ones: the basic architecture.

• What language(s) are we going to use? (that's easy, Python. but it depends)
• Light framework or a heavy, full featured one?

• Relational or non-relational database? And if non-relational, which of the many?
• Is our application process or data driven, or both? (this is vague. it's usually both)
• Browser or server side processing?
• How will it scale to many concurrent users?

and so on, and on.

At this point, if we pick the wrong road, the application could still be delivered, hopefully in time and the customer will be happy (at first), but the program itself might be a lot more complicated than it needs to be, and hard to change. The price will be paid over time, as new needs come and maintenance is required.

Unfortunately for us, some of these are hard questions, also subject of fashions and marker pressure, and it is very hard to have a comprehensive picture just by following technical blogs.

That's why I long for conferences like EuroPython, where the scope of the talks is as wide as can be, while still remaining highly technical.
Where the authors are ready to discuss about it over the lunch, and everybody is smart for having chosen Python :-)

Best talks.. ever :-)

A few of the talks tackling old problems in light of new tools and experiences:

Relate or !Relate (Mark Ramm-Christensen)

Mark explains the difference between the forest of NoSQL databases, and shows that a good SQL DBMS can scale surprisingly well.
This is a dear topic to me, because direct approaches in one side require complex contraptions on the other one, and vice-versa.
As a side note, lately the MySQL - Postgres debate seems to be over. The latter one has sharply won, if not the market, the minds of the programmers.

How to build complex web applications having fun? (Andrew Mleczko)

If the user wants both, give them both. Instead of fitting every requirement in a single architecture, try deploying a full-featured
CMS along with a light, flexible framework and divide the tasks -- a little like loading a bike on your camper.
You can go anywhere and they integrate well.
This is easier today (in the post-WSGI world) thanks to Pyramid, a highly refined foundation for web applications that lets a lot
of freedom: should the architecture be modeled after Zope, Plone, Pylons, Turbogears, Rails or Django? With Pyramid, "it depends" is the main concept :)

Other worth mentioning

• Challenges in developing a large Django site (Simon Willison)
• An iPhone-Python love affair: Building APIs for mobile (Anna Callahan - Nate Aune)

I also had fun with Raymond Hettinger, the only developer I've known that adds a +1 charisma bonus within a 10mt radius.

In his training track, he tore apart core concepts of Python (classes, decorators, properties, descriptors, dictionaries) and as he spoke rebuilt them using lists only. MacGyver couldn't do better!

But really, I could list half of the talks in the schedule, they are that good, and they kept a good balance of more and less advanced matters.

and the plone-ish stuff?

Sure, I would have liked a few more talks about Zope, Plone, BlueBream or Grok, especially highlighting how they have changed in the latest years, becoming more powerful, easier to extend and deploy, and there should be some effort communicating that to new developers.

They are coming to the conference to look for directions, and might go after some fashionable, but less capable tools.

Pyramid CRUD sprint is over. It was an amazing event thanks to Gaël Pasgrimaud and Patrick Gerken. We have been sprinting in Redturtle's office to improve pyramid_formalchemy and fa.jquery. We have archived most of the sprint goals!

Patrick was working on the first two tasks:

PASTER TEMPLATE

It can be used to add a skeleton to an existing project or to create a new project. If you create a new project, you must first install pyramid_formalchemy in your python environment, either with pip:

$ pip install pyramid_formalchemy

or with easy_install:

$ easy_install pyramid_formalchemy

Only after that, the paster template becomes available. The template was made with the idea that it can be used to extend existing applications. It does not create an app for you. The provided template works well with pyramid_alchemy, pyramid_routesalchemy and akhet. To bootstrap an application, call paster like that:

$ paster create -t akhet -t pyramid_fa myapp

The application is created by akhet, akhet does not know about pyramid_formalchemy, and pyramid_formalchemy cannot modify the app configuration. So you have to do this by hand. First, you must add the install dependency like explained earlier. Second, you must add the following line in the main method that returns the wsgi app, directly after Configurator has been created (The example assumes that the Configurator instance is stored under the name “config”):

...
config.include(myapp.fainit)
...

To add the minimum configuration to an existing application, you should be able to run:

$ paster create -t pyramid_fa myapp

All files that paster creates are prefixed with fa, and should not interfere with existing code. 

FANSTATIC

Patrick's second task was fanstatic integration. Fanstatic is a small but powerful framework for the automatic publication of resources on a web page. Think Javascript and CSS. It just serves static content, but it does it really well. Integrating it with fa.jquery was a time consuming task but we managed to get a working proof of concept. I will blog more details when we publish something stable.

Next tasks were handled by Gaël:

>>> from pyramid_formalchemy.views import ModelView
>>> from pyramid_formalchemy import actions
>>> class MyView(ModelView):
...     actions_categories = ('buttons', 'custom_actions')
...     defaults_actions = actions.defaults_actions.copy()
...     defaults_actions.update(edit_custom_actions=Actions())

>>> from sqlalchemy import Column, Integer
>>> from sqlalchemy.ext.declarative import declarative_base
>>> Base = declarative_base()
>>> class MyArticle(Base):
...     __tablename__ = 'myarticles'
...     edit_buttons = Actions()
...     id = Column(Integer, primary_key=True)

>>> from pyramid_formalchemy.views import ModelView
>>> from pyramid_formalchemy import actions
>>> class MyView(ModelView):
...     actions.action()
...     def extra(self):
...         # do your stuff
...         return self.render(**kw)

[app:pyramid]
...
default_locale_name = en
available_languages = fr en

I was working on the first three tasks: 

View customization

This was one of the main sprint tasks - to have a possibility to customize CRUD views per model.  You can register them simply like that:

config.formalchemy_model_view('admin',
                              model='pyramidapp.models.Foo',
                              context='pyramid_formalchemy.resources.ModelListing',
                              renderer='templates/foolisting.pt',
                              attr='listing',
                              request_method='GET',
                              permission='view')

and per Model:

config.formalchemy_model_view('admin',
                              model='pyramidapp.models.Foo',
                              context='pyramid_formalchemy.resources.Model',
                              name='',
                              renderer='templates/fooshow.pt',
                              attr='show',
                              request_method='GET',
                              permission='view')

formalchemy_model_view is an extension for config.add_view so you can use all view specific kwargs.

Widgets

We were able to finish simple implementation of autocomplete widget. It's not yet fully documented but it will be released in pyramid_formalchemy 0.4.  Here is how you use it:

from pyramid_formalchemy.renderers import pyramid_autocomplete
User.group.set(renderer=pyramid_autocomplete(filter_by='name'))

where User is your fieldset and group is your relation field; filter_by parameter is the SQLAlchemy column you want to filter (autocomplete) on. 

Events hooks

We have provided four events: 

• IBeforeValidateEvent
• IAfterSyncEvent
• IBeforeDeleteEvent
• IBeforeRenderEvent

There are also two more specific render evnts: 

• IBeforeShowRenderEvent
• IBeforeEditRenderEvent

You can use pyramid_formalchemy.events.subscriber decorator to use them:

from pyramid_formalchemy import events
from pyramidapp.models import Foo

@events.subscriber([Foo, events.IBeforeValidateEvent])
def before_foo_validate(context, event):
   #do your stuff here

Summary

It was a very productive four days. Thanks again for all your help. We should release new versions of pyramid_formalchemy and fa.jquery in the following days. If you don't want to wait - grab the development versions on github.

You can find more sprint photos here - http://www.flickr.com/photos/tags/crudsprint2011

After few months of discussions it is finally here: Pyramid CRUD sprint. It will be held in Ferrara just before EuroPython (June 16-19). Everyone is welcome! We will focus on two main goals: 

• make the CRUD more flexible then it is right now (using ZCA for view and subscribers registrations), increase test coverage and make it production ready
• add missing widgets and fix existing one

So if you want to taste classical Italian kitchen, visit UNESCO World Heritage Site and improve Pyramid - join us!

For more details check www.coactivate.org/projects/pyramid-crud-sprint

In my previous post I was trying to see what could be the benefit for Pylons Community to have a crud. We have a working prototype based on bfg. Now we would like to rewrite it for Pyramid. I've received interesting feedback, but the most promising was pyramid_formalchemy posted by Gael. 

What I miss in pyramid_formalchemy

I will start with description of our use case. At the moment we are using our CRUD to handle most of the form generation and CRUD operations. We are using it in several projects, one of them with ca. 100 models. The part of the application that is exposed to anonymous user (like registration) or it's more complex (like wizard multip-step forms) we are handling with formish. For those 100 models we are using heavily ZCA for customizing View/List templates. As far I can see in the pyramid_formalchemy source code:

def formalchemy_admin(config, route_name,
                      factory='pyramid_formalchemy.resources.Models',
                      view='pyramid_formalchemy.views.ModelView',
                      package=None, models=None, forms=None, session_factory=None):
...
config.add_view(context='pyramid_formalchemy.resources.ModelListing',
                    renderer='pyramid_formalchemy:templates/admin/listing.pt',
                    attr='listing',
                    request_method='GET',
                    permission='view',
                    **kw)

you can only pass one ModelView for config registration and practically you can't customize any of the templates per model.
What we would actually like to have is a ZCA registration, like:

  <view
      for=".interfaces.IModel"
      view=".listing"
      renderer="templates/my_model_listing.pt"
      permission="list"
  />

ZCML is of course optional...

Other benefit of this approach is a possibility to register other non-crud views (not only list/add/view/edit/delete) only for specific model (can be helpful for some of the widgets implementation, i.e. autocomplete).

This is our main concern. 

At the end we miss some widgets (which should be part of fa.jquery project and can be extended easily). One of the most interesting one is relation widget (with several variations),  which right now is missing :-)

In questi giorni c'è un discreto dibattito sulla proposta di sostituzione della mailing list [plone-users] con StackOverflow. 

L'idea è di sperimentare questo servizio per un 2-3 mesi e quindi di decidere il da farsi. Quello che su cui si vuol far leva sono diversi aspetti:

• Semplificare: Per molti, le mailing list sono uno strumento un po' ostico e Nabble non è il massimo come integrazione. Stack Overflow non necessita di particolari setup per poter cominciare a "chiedere".
  
• Valutare la reputazione: Una mailing list va frequentata per capire chi sa dare le risposte corrette e chi invece no. Stackoverflow ha un allegro sistema di valutazione dei partecipanti.
• Non chiedere 100 volte la stessa cosa: nessuno legge gli archivi di una mailing list, davvero nessuno. StackOverflow è facilmente "cercabile".
• Visibilità verso il mondo: Occorre che la comunità esca dal suo guscio dorato. Ebbene si, siamo maledetti snob che giocano con il miglior CMS sul mercato. Il confronto non può che far bene a noi stessi e anche agli altri: per Giove, non possono continuare ad utilizzare quegli antiquati CMS :)
• Notifiche: Se c'è un'argomento che vuoi seguire, beh è bene che tu venga avvertito: una ML, tantomeno Nabble, è in grado di farlo.
  
• Tonnellate di email vs RSS: questo potrebbe essere vantaggioso, ma è un punto alquanto controverso. Gli RSS di Stackoverflow mostrano le domande, non le risposte. E c'è chi preferisce il proprio client di posta a l'ennesimo portale da tenere sotto controllo.

Personalmente credo che sia un cambiamento interessante, in generale sono per i cambiamenti, ma questo lo è in modo particolare.

Non si tratta di avere uno strumento nuovo da utilizzare, si tratta di cambiare l'approccio con il resto del mondo.

Significa davvero uscire dalle nostre conferenze, dai nostri tool, dalle nostre mailing list e aprirci al confronto. Il confronto è bello! Si, si, subisci spesso delle critiche, ma ne esci sempre migliore di prima.

E voi che ne pensate?

Intanto troviamoci tutti su StackOverflow:

http://stackoverflow.com/questions/tagged/plone

Location e memorie

Nella località balneare in cui Hitchcock aveva ambientato il thriller “Gli Uccelli” (Bodega Bay, California), Steve McMahon ha recentemente organizzato il Cioppino Sprint, uno dei più significativi Sprint degli ultimi anni, che ha portato molti interessanti risultati, ed ha offerto anche l'occasione di ricordare la figura di Dorneles Treméa, una delle figure di maggior rilievo nella comunità Plone sudamericana, prematuramente scomparso in un incidente stradale (l'Associazione Python Brasiliana sta raccogliendo fondi per la famiglia).

Focus: Plone.org

A questo fine, secondo i suggerimenti di Alex Limi la principale area di intervento è stata il portale istituzionale Plone.org, oggetto di importanti interventi strutturali e di migliorie alle interfacce utente, risolvendo le anomalie rilevate negli ultimi mesi, e consolidando il PloneOrg add-on (l'add-on specifico del portale ufficiale). Non si è neppure trascurata qualche miglioria estetica (nella maschera di log-in, nei CSS del portale), e sono stati risolti sia i problemi dei certificati SSL di plone.org, sia quelli di plone.net: questa vetrina delle aziende Plone, ora migliorata nell'accesso alle immagini, a tutti gli effetti è stata incorporata nella sezione “support” di plone.org, che ora dispone anche di una sezione “follow” con collegamenti a Facebook e a Twitter.

Risultati

Tra gli altri risultati tecnici, è stata diminuita la dipendenza di Plone dalla Python Image Library (sempre necessaria, ma la sua assenza non è più bloccante).  A margine, c'è stato il tempo di proporre il logo dell'evento, simpatico e fuori dai canoni.

Dopo la conclusione ufficiale dello Sprint, le pagine di PloneSoftwareCenter, l'elenco delle soluzioni disponibili, sono state corredate di un sistema di rating a due pulsanti, “pollice su” e “pollice verso”, sono proseguite le migliorie stilistico-funzionali alle sezioni “documentation” e “support” ed è stata annunciata l'imminente messa in linea su plone.org degli ultimi video mancanti con gli interventi della Plone Conference 2008.

Un duro lavoro, insomma: ma, come sempre negli eventi tecnici Plone, un clima piacevole e rilassato, anche grazie alla previdenza di Ross Patterson, il cui “bar da viaggio” ha garantito l'adeguato ristoro di tutti i partecipanti.

What we have is a working version of a traversal CRUD. General concept is similar to Sergey Volobuev Kelpie. Everything is based on repoze.bfg 1.2 (though it should work smoothly on 1.3).

Model definition is done using SQLAlchemy declarative approach.  This choice provoke us to use  FormAlchemy as a form generator (with fa.jquery widget support). It's easy to start with but difficult to extend later.  That's why it's the first thing we would like to change in the future.  Everything is glued together with bfg hybrid traversal approach. 

What we are planing to do (probably not exactly in that order):

• check if somebody else didn't start similar project and maybe share concepts or do it together
• check possibility of integrate existing solutions with Pyramid
• reuse what can be reused from our bfg crud and move it to Pyramid
• organize a python sprint that will work on these topics