J5’s Blog

Tomeu finished up his review this weekend and gave me the green light to merge my long awaited invoke-rewrite branch. Besides being twice as fast as the previous invoke implementation, this branch cleaned up the code into discreet layers and modules which should make it safer and easier to debug as well as profile and add more speed improvements. I have also beefed up memory cleanup in the case that a call fails and the parameters have yet to be marshalled to python objects.

This rewrite is the basis for the future PyGObject 3 release. PyGObject 3 will permanently remove the static bindings but be parallel installable with PyGObject 2 provided that introspection is turned off in the PyGObject 2 build. While this may cause a small amount of pain for packagers, this will provide a smooth upgrade path for developers with little to no expected change to their current PyGObject Introspection projects.

As soon as we can make the two parallel installable we will do a release, update the JHBuild module sets and get the word out that unstable distros should switch to using the new setup. We are committed to stabilizing PyGObject 3 for the GNOME 3.2 release.

I figured instead of just complaining I would offer some practical fixes which would avoid the issue of disfranchising in the future.

First let’s examine why this is is an issue.  In order to become a member of the Foundation one must prove they have worked on significant pieces of GNOME and have two references to back them up.  A lot of people I know balk at even going through this first step because as far as they can see there is little benefit to becoming a member.  I try to remind them that voting is an important benefit.  During my tenure as a Foundation Board member we saw this as an issue and tried to promote Foundation participation by those in the GNOME Community.  Among other things we required people receiving funding from the foundation to apply for membership at some point in the future.  However the problem still remains that being a member constitutes a voting ballot once a year (1 and a half years during my tenure due to realigning  election years) and little else (there is the occasional smaller ballot).  A foundation member also has to renew their membership once every two years.  So if a member has to think about Foundation process once a year and membership process once every two years it is easy for membership to laps until a member finds themselves in the quandary I now find myself in.  To further exacerbate the situation it is a known issue by the membership and election committees that spam filters eat their e-mails and that the only reminders sent out are the exact same e-mails.

So how do we fix this?

Long Term Solutions

1.) Make registration an in person event to compliment the reminder e-mails – one of the things Foundation Members tend to do every year is go to one of the many events the Foundation sponsors.  If there is a registration system their names should be flagged and they given the opportunity to  renew their membership if it is set to expire in less than a year.  This should be an offline form that is processed at a later date as we all know how bad conference networking can be.  This can be a stipulation to organizers for receiving funding from the Board.

2.) Remind people when they are interacting with the Foundation process – Since voting happens every year, why not warn people when they vote that they will not be eligible for the next years vote unless they reregister at some point during the year?

Short Term Solution

All those who were denied a vote and who voted in last years elections should be allowed to reregister for this vote and given a ballot.

Having not received a ballot for this years elections I went into my spam folder to see if it had been caught there.  Instead I found a renewal notification for my foundation membership.  The policy for members to renew their membership every two years is not a bad one but I was just informed that I can not be renewed until after the elections and as such can not cast any vote in this cycle.  That is a bad policy.  For someone who is in good standing with the Foundation, having worked on its behalf and even have been a board member myself, I feel I have a right to vote in this election.

The policy to not allow renewal at the ballot box has in effect disenfranchised me.  Most bodies of democracy I know allow registering at the ballot box for those who have a record of having been previously registered and can prove they are who they say they are.  We aren’t talking about someone voting who has not participated in a vote before and would require excessive vetting before gaining that right.

What really gets me steamed about the situation is that due to being caught in a spam filter the only other way I could know that I was removed from the Foundation’s member list was to realize that I never received a voting ballot in which case I am not even give a chance to re-register before voting cuts it off.

UPDATE: I have been notified by the membership committee that many people failed to re-register and are unable to vote this cycle.  If this isn’t a sign of a broken system then people are ignoring the issue.  I could see if this was an isolated incident but since there is a small number of people who are eligible to vote I wonder what the ratio is of people who were denied a vote due to this procedural issue.  I bet it is statistically significant.  How can a vote be legitimate when a significant portion of the community is denied their vote?  It is grounds for a challenge during the ratification stage.

I just filed a bug to request a review for merging my invoke-rewrite branch. This is a huge overhaul of internals of the introspection layer of PyGObject which I feel both speeds up calls as well as makes it easier to fix bugs and add features.

Right now the invoke-rewrite branch is stable and complete enough to be merged to master. It passes all the tests and even cleans up after itself when an error is raised while processing input parameters (something the current code doesn’t do).

A brief list of the advantages of this branch

• over 2 times faster running thought the test suite than the current implementation.
• more robust – handles a couple of corner cases which the current code does not such as multiple lists pointing to the same aux length value, having argument 0 be a length value and correctly blocking all NULL values which are not marked as allow-none
• correctly handles argument parameter counting and reduces the amount of complicated math used to determine argument counts and indexes
• splits code into logical layers such as caching, invoking, marshalling and cleanup layers
• splits code into manageable units so, for example, if there is an error in list processing it is easier to set break points and focus on the list code instead of wading through large switch statements
• ripe for even more optimizations and better layout for profiling

Things that are yet to be done

• Memory management still leaks and some types such as gclosures are not yet cleaned
• Some of the earlier code like struct marshalling can be split up further
• signals and closures still use the old marshalling paths
• remove the FFI/GArgument layer and marshal directly to FFI

Why merge now?

Code rots and getting more people to use the new codepath makes it easier to fix bugs. It is already feature complete with the old code and simply trades off some new bugs while fixing a slew of other bugs. Because of the caching layer the code is somewhat more complex but follows a much more consistent pattern. In my eyes it is easier to understand once learned but developers need ramp up time since the structure is different. This is the future, there is no need to keep bandaging the old invoke code. As we get to more and more corner cases it gets harder to patch the old code without breaking other pieces or causing memory leaks that are hard to track down and plug (see garray hack we added to handle C Array vs GArray failures).

A brief overrview of the design

New modules

• pygi-invoke-ng – the new invoke API handles setup/invoking/teardown at a high level. Replaces pygi-invoke and does not attempt to handle memory it did not itself create (see garray handling for example)
• pygi-invoke-state-structure – the state struct used to pass around changing state during the caching, marshalling, invoking and cleanup stages
• pygi-cache – sets up the cache which normalizes the information and support methods such as marshalling and cleanup of each interface when called. This allows us to process method calls in a more efficent manner as well as makes it easier to debug
• pygi-marshal – marshalling functions for each parameter type. Replaces pygi-arguments
• pygi-marshal-cleanup – cleanup functions for each parameter type.  Includes cleanup for parameters which were not processed due to an error.

Basic flow

invoke is called on an GICallableInfo object
  |
  |-> if GICallableInfo does not have a cache we loop over the interface
  |    creating the PyGICallableCache and ArgCaches for the interface
  |    and its arguments
  |
  |----> use the cache to marshal the parameters into the state struct
  |
  |---------> using the marshalled values invoke the interface using the
  |           GI FFI calls
  |
  |----> use the cache to cleanup any in values that need cleaning
  |
  |---------> use the cache to marshal the out values to python objects
  |
  |----> use the cache to cleanup any out values that need cleaning
  |
  |-> cleanup state
  |
return out values to python app

Standard paterns

All marshalling and cleanup methods for types are added to the argument cache as function pointers. A PyGIArgCache has these four function pointers no matter what the type we are caching for:

• in_marshaller – the function which marshals in values for this argument
• out_marshaller – the function which marshals out values for this argument
• in_cleanup – the function that cleans up in values for this argument
• out_cleanup – the function that cleans up out and in/out values for this argument

All type marshallers and cleanup methods follow this naming pattern:

• in marshallers – _pygi_marshal_in_<type> (e.g. _pygi_marshal_in_utf8 or _pygi_marshal_in_interface_enum)
• out marshallers – _pygi_marshal_out_<type> (e.g. _pygi_marshal_out_utf8 or _pygi_marshal_out_interface_enum)
• in_cleanup – _pygi_marshal_cleanup_in_<type> (e.g. _pygi_marshal_cleanup_in_utf8)
• out_cleanup – _pygi_marshal_cleanup_out_<type> (e.g. _pygi_marshal_cleanup_out_utf8)

These are all setup during the caching phase in functions named similarly (e.g. _arg_cache_in_utf8_setup)

This is just a reminder to those who are coming to the launch party. The dinner reservations has be moved to 8:00 at The Blue Room in Kendall Square. We also have the two back pool tables reserved at Flat Top Johnny’s at 9:30. See you there.

Hey all. We are winding down the release and ramping up to getting our party on! The Boston/Cambridge GNOME 3.0 Launch Party will be happening this Sunday down in Kendall Square, Cambridge, MA. We will be meeting for dinner around 7 at The Blue Room and then going to our tradition hang out spot, Flat Top Johnny’s, for drinks and a couple of rounds of pool. If you are planning on coming please add your name to the wiki ASAP so I can make reservations. I plan on making them by tomorrow night. Hope to see you there!

Hey all you Northeast GNOME developers and users. We are having a GNOME release party this Sunday, April 10th sponsored by Red Hat. We are still scouting for a good location but the idea is to go to a bar with either a bowling lane or pool tables where drinks, games and food will be sponsored. Please head over to the wiki and let us know you will be coming so we can figure out the best venue to hold it.

Come on down and have some fun while we celebrate the hard work that went into this historic release for the GNOME Project.

I am pleased to announce version 2.28.3 of the Python bindings for GObject. There was an issue with the patch that went out with 2.28.2 and all users should upgrade from 2.28.2 to this release. This is a minor bugfix release of the stable 2.28 branch.

The new release is available from ftp.gnome.org:

http://download.gnome.org/sources/pygobject/2.28/

What’s new since PyGObject 2.28.1?

• fixed an ABI break in the static bindings when setting string gvalues
  • e.g. passing an int to a Gtk.ListStore column which expects a string automatically converts the int to a string instead of throwing an error
• This release fixes a bug that the 2.28.2 release introduced

What’s new since PyGObject 2.28.0?

• pygi-convert.sh now supports webkit conversions and favors using GObject over gobject
• Raw closures can now be passed from a signal/vfunc callback to a method
• Revert linking to the python libs because the python runtime statically links it in
• TreeModel column marshalling is now more robust (supports GObject Python Object storing)
• Gtk.MessageDialog now respects the MessageType
• You can now send None in for the signature of GDBus messages that have no parameters
• TreeViewColumn.set_cell_data_func can take None for the func_data
• Fix syntax error so we can run in Python 2.5
• Add pickers and menu demos

Blurb:

GObject is a object system library used by GTK+ and GStreamer.

PyGObject provides a convenient wrapper for the GObject library for use in Python programs, and takes care of many of the boring details such as managing memory and type casting. When combined with PyGTK, PyORBit and gnome-python, it can be used to write full featured Gnome 2 applications.

Like the GObject library itself PyGObject is licensed under the GNU LGPL, so is suitable for use in both free software and proprietary applications. It is already in use in many applications ranging from small single purpose scripts up to large full featured applications.

PyGObject requires glib >= 2.22.4 and Python >= 2.5.1 to build.
GIO bindings require glib >= 2.22.4.

The Introspection module is the next generation Python GObject library bindings. Instead of statically wrapping every GObject based library we can now dynamically accesses any of those libraries using GObject Introspection. It replaces the need for separate modules such as PyGTK, GIO and python-gnome to build a full GNOME 3.0 application. Once new functionality is added to gobject library it is instantly available as a Python API without the need for an intermediate Python module.

Introspection/Python 2 bindings requires gobject-introspection >= 0.9.5 and pycairo >=1.0.2 or py2cairo >=1.8.10
Introspection/Python 3 bindings requires gobject-introspection >= 0.9.5, pycairo >=1.8.10 and Python >= 3.1

–
John (J5) Palmieri
GNOME Foundation member
johnp@redhat.com

UPDATE: Please use the 2.28.3 release

I am pleased to announce version 2.28.2 of the Python bindings for GObject.  This is a minor bugfix release of the stable 2.28 series.

The new release is available from ftp.gnome.org:

http://download.gnome.org/sources/pygobject/2.28/

What’s new since PyGObject 2.28.1?

• fixed an ABI break in the static bindings when setting string gvalues
  • e.g. passing an int to a Gtk.ListStore column which expects a string automatically converts the int to a string instead of throwing an error

What’s new since PyGObject 2.28.0?

• pygi-convert.sh now supports webkit conversions and favors using GObject over gobject
• Raw closures can now be passed from a signal/vfunc callback to a method
• Revert linking to the python libs because the python runtime statically links it in
• TreeModel column marshalling is now more robust (supports GObject Python Object storing)
• Gtk.MessageDialog now respects the MessageType
• You can now send None in for the signature of GDBus messages that have no parameters
• TreeViewColumn.set_cell_data_func can take None for the func_data
• Fix syntax error so we can run in Python 2.5
• Add pickers and menu demos

Blurb:

GObject is a object system library used by GTK+ and GStreamer.

PyGObject provides a convenient wrapper for the GObject library for use in Python programs, and takes care of many of the boring details such as managing memory and type casting.  When combined with PyGTK, PyORBit and gnome-python, it can be used to write full featured Gnome 2 applications.

Like the GObject library itself PyGObject is licensed under the GNU LGPL, so is suitable for use in both free software and proprietary applications. It is already in use in many applications ranging from small single purpose scripts up to large full featured applications.

PyGObject requires glib >= 2.22.4 and Python >= 2.5.1 to build.
GIO bindings require glib >= 2.22.4.

The Introspection module is the next generation Python GObject library bindings. Instead of statically wrapping every GObject based library we can now dynamically accesses any of those libraries using GObject Introspection. It replaces the need for separate modules such as PyGTK, GIO and python-gnome to build a full GNOME 3.0 application. Once new functionality is added to gobject library it is instantly available as a Python API without the need for an intermediate Python module.

Introspection/Python 2 bindings requires gobject-introspection >= 0.9.5 and pycairo >=1.0.2 or py2cairo >=1.8.10
Introspection/Python 3 bindings requires gobject-introspection >= 0.9.5, pycairo >=1.8.10 and Python >= 3.1

–
John (J5) Palmieri
GNOME Foundation member
johnp@redhat.com

I am pleased to announce version 2.28.1 of the Python bindings for GObject.  This is a minor bugfix release of the stable 2.28 series.

The new release is available from ftp.gnome.org:

http://download.gnome.org/sources/pygobject/2.28/

What’s new since PyGObject 2.28.0?

• pygi-convert.sh now supports webkit conversions and favors using GObject over gobject
• Raw closures can now be passed from a signal/vfunc callback to a method
• Revert linking to the python libs because the python runtime statically links it in
• TreeModel column marshalling is now more robust (supports GObject Python Object storing)
• Gtk.MessageDialog now respects the MessageType
• You can now send None in for the signature of GDBus messages that have no parameters
• TreeViewColumn.set_cell_data_func can take None for the func_data
• Fix syntax error so we can run in Python 2.5
• Add pickers and menu demos

Blurb:

GObject is a object system library used by GTK+ and GStreamer.

PyGObject provides a convenient wrapper for the GObject library for use in Python programs, and takes care of many of the boring details such as managing memory and type casting.  When combined with PyGTK, PyORBit and gnome-python, it can be used to write full featured Gnome 2 applications.

Like the GObject library itself PyGObject is licensed under the GNU LGPL, so is suitable for use in both free software and proprietary applications. It is already in use in many applications ranging from small single purpose scripts up to large full featured applications.

PyGObject requires glib >= 2.22.4 and Python >= 2.5.1 to build.
GIO bindings require glib >= 2.22.4.

The Introspection module is the next generation Python GObject library bindings. Instead of statically wrapping every GObject based library we can now dynamically accesses any of those libraries using GObject Introspection. It replaces the need for separate modules such as PyGTK, GIO and python-gnome to build a full GNOME 3.0 application. Once new functionality is added to gobject library it is instantly available as a Python API without the need for an intermediate Python module.

Introspection/Python 2 bindings requires gobject-introspection >= 0.9.5 and pycairo >=1.0.2 or py2cairo >=1.8.10
Introspection/Python 3 bindings requires gobject-introspection >= 0.9.5, pycairo >=1.8.10 and Python >= 3.1

–
John (J5) Palmieri
GNOME Foundation member
johnp@redhat.com