I'm working right now on a project that uses ore.contentmirror to serialize Plone content data to postgresql which can be later reused in repoze.bfg. I have found several similar deployments in the Plone community which were quite inspiring.

The scenario is almost always the same:

• to use Plone as a CMS (backend)

• to have fast framework (frontend) to serve my backend data

• to include in frontend layer Web2.0 functionality and some other dynamic stuff

• to make benefit of external indexing server for full text search, for both frontend and backend

You can of course use solr with collective.solr for search engine but thanks to tsearch2 some of the search/ranking functionalities can be taken directly from postgresql (making the whole stack much smaller and easier to maintain).

Serializing Plone data to SQL opens many possibilities. But with Plone you can do much more.

You can integrate other CMS's like Alfresco or Sharepoint using CMIS. You can connect your Plone instance to Salesforce. Instead of using tsearch2 or solr as an indexing engine - you can use Google Search Appliance inside your Plone instance. If you want to start online document collaboration you can even connect it to Google Docs.

External services and applications are not the end of the story. There is of course WSGI with collective.xdv and Deliverance and bunch of other interesting middlewares and we shouldn't forget about PAS plugins.

It's possible that I missed something interesting. Plone community is very creative, thought. It's hard to be always up to date. That's why I'm sure that I miss a place where integrators and developers can share they successful stories and discuss potential use cases. A place which will make Plone future bright.

What's up?

The Plone site I'm describing here is quite normal, but customer ask us to give some special additional permissions to users that reach the Plone site from an internal domain.

In facts the wanna be still anonymous (forcing no-one to authenticate) but be able to see some documents in a special "Published internally" state.

How tho give this permission to anonymous users?

AutoRole?

I never used AutoRole before, but its clear that the idea behind is what we need. AutoRole is an interesting PAS plugin provide additional roles automatically using the IP of the client that is not what we really wanna there.
It also works well with anonymous users making some magic inside the plugin!

AutoRoleFromHost?

You can find on the Plone SVN our first attempt to use the AutoRole idea for our needs. Changing some lines of codes here and there we changed roles provided relying on HTTP_HOST used.

Problem

The HTTP_HOST works only when the client reach directly the Zope server (not exacly, but we have no controls on the Apache of that company)... and we wanna put Varnish in front of it.

Limit

What if tomorrow I need to give somewhere an additional role to users that use a specific browser, or something else? I can't spend all of my live developing AutoRoleFromSomething products!

AutoRoleFromHostHeader!

The best choice we found is to look at HTTP Header in general, making what header and what value completely configurable.

We developed and released AutoRoleFromHostHeader. Similar to AutoRole, but  you can configure it like this:

HTTP Header;regexp;role,[role,]

To make it the most general as possible, the value of the header is used as a regular expression.

Using this you can reproduce some of the AutoRole features, but you can also make something like this:

HTTP_X_FORWARDED_HOST;special\.hostname\.it;SpecialAnonymous
HTTP_USER_AGENT;(MSIE|Internet\ Explorer);BrowserlessVisitor

Disclaimer: Tested only on linux and mac

Symptoms: python readline does not work at all in the pdb shell.

If you find out that inside a pdb after pressing some keys, e.g. the arrow keys, some escape symbols appear on the line...

(Pdb) test^[[D^[[D^[[D

...and you cannot move the cursor to the desired position, probably some package screwed up your pdb!

This is very annoying to me because some times I have to spend a lot of time in the pdb to heal my buggy code: a misfunctional readline makes me lose too much time!

This probably happened because somewhere in the code the standard output (sys.stdout) has been redefined and this screws up the pdb. 

Check it out in the console:

(Pdb) import sys
(Pdb) sys.stdout

 If the output is not something like

<open file '<stdout>', mode 'w' at 0xb783d070>

than you can go on reading to find out how to fix it. 

Don't ask me why pdb does not like it. If you are able to identify the nasty piece of code that modifies the standard output and replicate it in a python console, this problem does not appear.

Therapy: fix the stdout!

In most of the cases this will give you back a fully functional console:

>>> import sys;sys.stdout=file('/dev/stdout','w')

Describing the problem

I'm near to release a new version of collective.flowplayer_toolbar (so it's only on collective SVN right now).

The first version of the product (see the old blog post about it) give a new Javascript controlsbar to Flowplayer, disabling the native Flash ones.

In a production environment we received some comments about this choice because when disabling the Flash controls you are not able anymore to use fullscreen mode... bad!
The only solution is to keep both control systems because is not possible (due to security issue) to control fullscreen from Javascript.

Have both controls give us the best freedom (features and accessibility)... but to be honest I find it really ugly!

So, the best way is to give some configuration for switch from:

• using only the Javascript controls
• using Javascript and Flash controls

The collective.flowplayer choice

The same kind of problem has been overcome by collective.flowplayer.
Some of the configuration you can manage in the flowplayer_properties leave to changes in the client-side Javascript behaviour.

I'm not sure that this is the only motivation, but this problem is solved there providing the Javascript code not using a static file (so not giving a Javascript text source) but using a zope 3 view.
This view (collective.flowplayer.js) return a text/javascript content type:

    def __call__(self, request=None, response=None):
        """ Returns global configuration of the Flowplayer taken from portal_properties """
        self.update()
        self.request.response.setHeader("Content-type", "text/javascript")
        ...

The problem with this approach is that you must work with Javascript code written inside Python string... not really comfortable.

Of course, the problem is only for the developer. Client side the Web browser will not feel any difference.

Experimental approach

As far as Flowplayer works well with jQuery (and jQuery Tools, that is a requirement of collective.flowplayer using plone.app.jquerytools) I'm trying a second approach to the problem. This because:

• I don't like to write Javascript inside strings
• I really like the power of jQuery plugins support, and this has become more clear to me after having read recently a book on this.

Following the server side collective.flowplayer approach I added a new property to the portal_properties tool (it's only a single property, so I added it to the same flowplayer_properties sheet).

This boolean property (toolbar_flash_controlsbar) must only controls this single line of Javascript code:

this.getPlugin("controls").hide();

I need a way to execute this only when the flag is True.

The solution is simple: execute this line of code only when a Javascript variable is evaluated true/false:

if (!show_flash_controlsbar)
	this.getPlugin("controls").hide();

Now we only need a way to controls from the server this client variable, but first it's better to provide a default initialization.

As far as this control is executed inside a Flowplayer event, we can add the initialization before or after this piece of code, directly in the flow of page execution (so the order matters not):

var show_flash_controlsbar = false;

So the default is also the default of the toolbar_flash_controlsbar property (False) and we will not see that Flash controls.

Now we really need a way to react to a True value server-side.

The way used here is to include another (minimal) Javascript piece of code, this time after the ones above (so, if included, it must be after the initialization of the show_flash_controlsbar).

show_flash_controlsbar = true;

To keep controls on the inclusion of this we need to put this into a separate Javascript file and include it conditionally after the ones above.

The Generic Setup inclusion take this form:

 <javascript cacheable="True"
             compression="safe"
             cookable="True"
             enabled="True"
             id="++resource++flowplayer_toolbar_js/flash_controlsbar.js"
             insert-after="++resource++flowplayer_toolbar_js/flowplayer.accessible-controls.js"
             expression="portal/portal_properties/flowplayer_properties/toolbar_flash_controlsbar|nothing"
             inline="False"/>

In the expression TAL we check and evaluate the server side property (and for security we also put a |nothing, to prevent all sort of errors if, for example, the property has been deleted).

The portal_javascript tool take care of providing this Javascript script or not, even when the it's merged with other and cached.

Don't you talked of jQuery somewhere above?

The only problem with this approach is the namespace pollution. We must be sure that the name of this variable is not used/taken from other Javascript code somewhere in Plone.

jQuery and its plugins technology help in this, and also help to keep things more clear and ordered.

With the syntax below we can make this definition available only inside the jQuery environment, preventing that non-jQuery Javascript code change/use this variable.

jQuery.show_flash_controlsbar = false;

Note the use of jQuery name, not $ or jq: this is how plugin development guidelines recommend to define and use jQuery object when developing plugins.

In our code we can continue testing the condition in this way:

if (!jq.show_flash_controlsbar)
	this.getPlugin("controls").hide();

This is better than before, but we can make it more robust.

This approach can fails only if another jQuery plugin define and use the same name for an internal variable of function (ok, this is paranoyc); if this become true we can get errors or unexpected results.

The best jQuery way is to define an internal namespace, making this problem more and more difficult to arise.

So finally we define the variable in this way:

jQuery.flowplayer_toolbar = {
	/**
	 * Some other script can put this to true to enable also the native controlsbar plugin
	 * This is the only way to get features like the fullscreen.
	 */
	show_flash_controlsbar: false
};

And we test it like this:

if (!jq.flowplayer_toolbar.show_flash_controlsbar)
	this.getPlugin("controls").hide();

The last benefit of using an internal jQuery namespace is to keep similar variables or function together. May be that some other variables (or global functions?) can be needed in future version of this product, so we can add them inside the same namespace.

Going back: is this approach better or not?

Aswer: I don't know. This works well with browser cache but as you can see it needs some more work than providing a dynamic Javascript source from the server using a view.

The most annoying things is the registration of an additional entry inside the portal_javascript tool only for put an additional line of Javascript code... this is not a problem for one script, but can you imagine this repeated for all collective.flowplayer configuration (or for other products too)?
The numbers of entries will explode!

The new release of PloneboardNotify doesn't give us many new features but is not this the real focus of this article.

This add-on was already available and used in production environment, and one of the first effort was to keep Plone 2.5 compatibility.
The same is for the new 0.4 release (and maybe Plone 2.5 compatibility will continue until all of our customer will drop this version).

So? What is the problem?

Thanks for testing

Sometimes when you develop something that is useful for a single site, it became magically useful for other... but sometimes you are too stupid (and lazy) to understand it.
Let's go back to Ploneboard: after all the notification e-mail after a new forum discussion/response is a wanted feature (worst... an expected feature), not available in Ploneboard (that, even if it isn't perfect, is the only mature choice available right now on Plone to have a forum).

PloneboardNotify is a good example of a bad way to start a product... developed fast but with no eye on good OOP, without thinking about make it extensible... not tested at all. It was only a stupid event script that send e-mail (luckily this old version isn't available or any public SVN so the Story will not judge me).

First public released versions were only more "user friendly", making it usable in any context where you have Plone and Ploneboard. Maybe they are good release but was clean the need of refactoring: but refactoring is someway a dangerous task (the code is working... why I must spend my time making it better only to get the same features and be forced to test all feature and behaviour from scratch? I'm sure to remember what I need to test?)!

The problem there was (and is also now) the "core" (please... don't look at it!). Tests at every release were done by me (and my name is not Funkload) manually playing with the browser, repeating the same operation on Plone 2.5 and 3. Time after time the core grow as new feature is added... Tests are every time more expensive and boring...

What is changed now? Nothing! PloneboardNotify is still a mess... but now it's a tested mess!

Future

The primary effort of this release was adding a complete functional tests coverage of all features (for both Plone 2.5 and 3 versions). After that, thanks to the help of Nicolas Laurance, I also added the HTML e-mail feature.

Now I have no fear to make changes to the code core... also in this release I removed some stupid piece of code (after completing tests!) and introduce adapters. Having tests available now will speed up dramatically the develop process of new features.

Next steps will be to reach a better OOP and making the product more customizable by other developer in their sites (like customize the e-mail template in some simple way).

Lessons learned

• Don't be lazy. You can make this code more reusable. Always.
  
• Don't be lazy. Write tests! Better: you will became lazy if you don't write tests because testing manually is very boring and you'll begin skipping some action!
  
• Don't be stupid. You can't make deep refactoring and expect that all will work after you've finished, so...
  make tests!

In these days it happened to me to work on a couple of custom Plone edit forms using ajax calls to simplify user data entry.

At a first glance, the user experience was great, but under the hood the whole machinery was hiding potential problems, the reason being that in both cases the ajax calls used in the form were relative to the document path, even when if the document was in the factory!

This means that every time the ajax call was invoked, additional security checks were performed and in the case of the documents inside the factory other this lead to an unwanted creation of temporary documents!

The security checks and the portal factory abuse increased the response time and could potentially raise conflict errors.

During the development phase the problem is hardly noticeable, but on a production server it will probably make the form unusable, especially if you complex security checks are needed.

The solution was simply to call the views with another context (in those cases the container). 

Lesson learned: be careful to the context of your ajax call

Feedly fondamentalmente è un aggregatore di feed rss. bello eh?

Ok, se la metto giù così non vale neanche la pena di perderci tempo e di scrivere una pagina in questo allegro blog.

In effetti ha alcune simpatiche funzionalità che lo rendono decisamente gradevole e che lo rendono superiore ad altri sistemi.

Ovviamente è una web application e si integra con google: usi lo ss utente e sfrutti i feed che hai già configurato su Reader. Ovviamente li puoi riaggregare come preferisci e ne puoi aggiungere altri molto rapidamente.

L'effetto immediato è di ottenere una versione magazine dei tuoi feed. Dal feed stesso ottiene immagini e li riorganizza mettendone alcuni in evidenza.

Ma la cosa che lo distingue dagli altri è il fatto che, a partire dai tuoi feed, ottenga altre informazioni da altre fonti: twitter, flickr, youtube, amazon.

Per ogni tua aggregazione puoi definire una view predefinita.

Interessante la lettura del feed. Te lo presenta come popup in maniera estesa e già da li lo puoi ri-sharare verso i nostri cari vecchi social network, ma lo puoi anche inviare via gmail.

Lo installi come estensione di Firefox, di Chrome o di Safari, IE... non so :), altrimenti non va (questo non è per niente cool)

Interessante il feedly mini, li appoggiato in basso a destra nella pagina. Mentre navighi analizza il contenuto della pagina e ti propone i migliori post tra i tuoi feed relativi al contesto.

Puoi anche salvare la pagina piuttosto che postarla su twitter/facebook o inviarla via mail.

Ogni tanto ti ritrovi con qualcuno che ti racconta le ultime meraviglie della realtà aumentata [wikipedia]. 

Fondamentalmente (e semplificando moltissimo la cosa..) si tratta di sovrapporre livelli informativi alla realtà che stiamo vivendo.

Lo possono fare sfruttando in vari modi oggettini che sono già presenti: dal famoso caschetto da realtà virtuale all'ormai famigerato iPhone.

Sicuramente l'uso di device mobili è l'opzione più attraente e più consumer.

I signori di wikitude si sono inventati:

• una community dove ognuno può aggiungere i suoi punti di interesse (POI): wikitude.me
• un browser che gira su Android, iPhone, e Symbian (Nokia N97 e N97 mini). iPhone 3GS. Il mio ovviamente non va :(
  
• volevamo farci mancare delle API? ovviamente no! Certo che in Java...

l'effetto finale è notevole:

The product plone.app.discussion is becoming the standard way to add comments in Plone. If both products plone.app.discussion and Products.PloneGazette are installed the creation of any PloneGazette content type fails, because PloneGazette content types not provide the adapter for "IConversation":

Traceback (innermost last):
  ...
  Module plone.app.discussion.catalog, line 29, in total_comments
TypeError: ('Could not adapt', <NewsletterTheme at /ausl/newslettertheme.2010-01-26.2440135204>, <InterfaceClass plone.app.discussion.interfaces.IConversation>)

To solve the problem we set an adapter for the types defined in Products.PloneGazette. This adapter provides the interface "IConversation" of plone.app.discussion. The adapter provides the attribute "total_comments" and the method "enabled", which returns False (means that the commeting is disable).

<adapter
      for="Products.PloneGazette.interfaces.INewsletterTheme"
      factory=".newsletter.NewsletterConversation"
      provides="plone.app.discussion.interfaces.IConversation"
      />

So you can create objects Newsletter without any problem.

Unfortunately PloneGazette does not define an interface for each content types, so through zcml we say that these content type implements a particular interfaces.Then we give the adapter for those interfaces.

The new product collective.discussionintegration.plonegazette makes possible the creation of the PloneGazette content types if there is an installation of plone.app.discussion in your instance.

http://svn.plone.org/svn/collective/collective.discussionintegration.plonegazette/