J5’s Blog

UPDATE: Please use the 2.28.3 release

I am pleased to announce version 2.28.2 of the Python bindings for GObject.  This is a minor bugfix release of the stable 2.28 series.

The new release is available from ftp.gnome.org:

http://download.gnome.org/sources/pygobject/2.28/

What’s new since PyGObject 2.28.1?

• fixed an ABI break in the static bindings when setting string gvalues
  • e.g. passing an int to a Gtk.ListStore column which expects a string automatically converts the int to a string instead of throwing an error

What’s new since PyGObject 2.28.0?

• pygi-convert.sh now supports webkit conversions and favors using GObject over gobject
• Raw closures can now be passed from a signal/vfunc callback to a method
• Revert linking to the python libs because the python runtime statically links it in
• TreeModel column marshalling is now more robust (supports GObject Python Object storing)
• Gtk.MessageDialog now respects the MessageType
• You can now send None in for the signature of GDBus messages that have no parameters
• TreeViewColumn.set_cell_data_func can take None for the func_data
• Fix syntax error so we can run in Python 2.5
• Add pickers and menu demos

Blurb:

GObject is a object system library used by GTK+ and GStreamer.

PyGObject provides a convenient wrapper for the GObject library for use in Python programs, and takes care of many of the boring details such as managing memory and type casting.  When combined with PyGTK, PyORBit and gnome-python, it can be used to write full featured Gnome 2 applications.

Like the GObject library itself PyGObject is licensed under the GNU LGPL, so is suitable for use in both free software and proprietary applications. It is already in use in many applications ranging from small single purpose scripts up to large full featured applications.

PyGObject requires glib >= 2.22.4 and Python >= 2.5.1 to build.
GIO bindings require glib >= 2.22.4.

The Introspection module is the next generation Python GObject library bindings. Instead of statically wrapping every GObject based library we can now dynamically accesses any of those libraries using GObject Introspection. It replaces the need for separate modules such as PyGTK, GIO and python-gnome to build a full GNOME 3.0 application. Once new functionality is added to gobject library it is instantly available as a Python API without the need for an intermediate Python module.

Introspection/Python 2 bindings requires gobject-introspection >= 0.9.5 and pycairo >=1.0.2 or py2cairo >=1.8.10
Introspection/Python 3 bindings requires gobject-introspection >= 0.9.5, pycairo >=1.8.10 and Python >= 3.1

–
John (J5) Palmieri
GNOME Foundation member
johnp@redhat.com

I am pleased to announce version 2.28.1 of the Python bindings for GObject.  This is a minor bugfix release of the stable 2.28 series.

The new release is available from ftp.gnome.org:

http://download.gnome.org/sources/pygobject/2.28/

What’s new since PyGObject 2.28.0?

• pygi-convert.sh now supports webkit conversions and favors using GObject over gobject
• Raw closures can now be passed from a signal/vfunc callback to a method
• Revert linking to the python libs because the python runtime statically links it in
• TreeModel column marshalling is now more robust (supports GObject Python Object storing)
• Gtk.MessageDialog now respects the MessageType
• You can now send None in for the signature of GDBus messages that have no parameters
• TreeViewColumn.set_cell_data_func can take None for the func_data
• Fix syntax error so we can run in Python 2.5
• Add pickers and menu demos

Blurb:

GObject is a object system library used by GTK+ and GStreamer.

PyGObject provides a convenient wrapper for the GObject library for use in Python programs, and takes care of many of the boring details such as managing memory and type casting.  When combined with PyGTK, PyORBit and gnome-python, it can be used to write full featured Gnome 2 applications.

Like the GObject library itself PyGObject is licensed under the GNU LGPL, so is suitable for use in both free software and proprietary applications. It is already in use in many applications ranging from small single purpose scripts up to large full featured applications.

PyGObject requires glib >= 2.22.4 and Python >= 2.5.1 to build.
GIO bindings require glib >= 2.22.4.

The Introspection module is the next generation Python GObject library bindings. Instead of statically wrapping every GObject based library we can now dynamically accesses any of those libraries using GObject Introspection. It replaces the need for separate modules such as PyGTK, GIO and python-gnome to build a full GNOME 3.0 application. Once new functionality is added to gobject library it is instantly available as a Python API without the need for an intermediate Python module.

Introspection/Python 2 bindings requires gobject-introspection >= 0.9.5 and pycairo >=1.0.2 or py2cairo >=1.8.10
Introspection/Python 3 bindings requires gobject-introspection >= 0.9.5, pycairo >=1.8.10 and Python >= 3.1

–
John (J5) Palmieri
GNOME Foundation member
johnp@redhat.com

I am pleased to announce version 2.28.0 of the Python bindings for GObject.

Including the stable improvements in the base pygobject modules this is the first stable release of the Introspection binding.  PyGObject Introspection is compatible with both the Python 2 and Python 3 runtimes.

This marks the second major development project I have had the pleasure to help bring to fruition.  Like for D-Bus, I am merely the public face of a rather vibrant development community.  While I hope I don’t leave anyone out I would like to thank the following people who dedicated their valuable time to the future of the Python platform for GNOME.

• Johan Dahlin – creator of PyBank, the predecessor to PyGObject Introspection, for giving us direction, advice and helping to review patches
• Tomeu Vizoso – co-maintainer, my good friend, the reason why we decided to push for Introspection support and probably the hardest worker on our team
• Steve Frécinaux – for all of his leak hunting and bug fixing while testing against libpeas and gedit plugins
• Ignacio Casal Quinterio – for fixing bugs while porting the gedit plugins to the new bindings
• Laszlo Pandy – for fixing some of the tougher bugs and winning the “why are you awake at this hour, hacking on PyGObject while on irc?” award
• Martin Pitti – who worked on GVarients and making GDBus easy to use from PyGObject
• Simon van der Linden – who started this journey with us and has been helping us along when he finds the time
• Simon Schampijer – our newcomer from OLPC/Sugar development who jumped right in to help at the hackfest
• Sebastian Pölsterl – another hackfest alum who ported GNOME DVB Daemon’s GUI
• John Ehresman – creator of the initial Python 3 support code
• Dave Malcolm – some of the initial Python 3 bits and my sounding board for any Python issues I ran into
• The entire GObject Introspection team – especially the tireless Colin Walters who fixed our many GI bugs quickly while also putting up with my complaining both at work and whenever we happened to grab a beer in one of the local watering holes
• The Python team – for making an excellent language for us to bind to
• And many more, too numerous to list, who have contributed to the continuing maintenance of PyGObject throughout the years

The new release is available from ftp.gnome.org:

http://download.gnome.org/sources/pygobject/2.28/

What’s new since PyGObject 2.27.92?

• fix sinking of floating objects
• fix leaks when setting properties
• add basic icon view demo
• add search entry demo
• override Gdk.RGBA so you can construct it like Gdk.RGBA(1.0, 1.0, 1.0, 1.0)
• handle unichar gvalues in TreeModels
• check for _thread module when configuring threading
• package config file now contains overridesdir variable for 3rd party overrides
• on windows set bdist_wininst user-access-control property when installing
• Gtk.stock_lookup return None on failure instead of a success value
• Python 2.5 fixes
• Python 3 fixes

Blurb:

GObject is a object system library used by GTK+ and GStreamer.

PyGObject provides a convenient wrapper for the GObject library for use in Python programs, and takes care of many of the boring details such as managing memory and type casting.  When combined with PyGTK, PyORBit and gnome-python, it can be used to write full featured Gnome 2 applications in Python 2.

Like the GObject library itself PyGObject is licensed under the GNU LGPL, so is suitable for use in both free software and proprietary applications. It is already in use in many applications ranging from small single purpose scripts up to large full featured applications.

PyGObject requires glib >= 2.22.4 and Python >= 2.5.1 to build.
GIO bindings require glib >= 2.22.4.

The Introspection module is the next generation Python GObject library bindings. Instead of statically wrapping every GObject based library we can now dynamically accesses any of those libraries using GObject Introspection. It replaces the need for separate modules such as PyGTK, GIO and python-gnome to build a full GNOME 3.0 application in Python 2 or 3. Once new functionality is added to gobject library it is instantly available as a Python API without the need for an intermediate Python module.

Introspection/Python 2 bindings requires gobject-introspection >= 0.9.5 and pycairo >=1.0.2 or py2cairo >=1.8.10
Introspection/Python 3 bindings requires gobject-introspection >= 0.9.5, pycairo >=1.8.10 and Python >= 3.1

–
John (J5) Palmieri
GNOME Foundation member

I submitted a patch for the last known blocker bug which is waiting on review.  If no other bugs are added to the blocker list, I will be releasing a stable 2.28.0 PyGObject on Monday.  Please add your bug to the blocker list if you want to see it get into the stable release.  They will be the only bugs I will be looking at so if it isn’t there as far as I am concerned, it doesn’t exist until the next development cycle.  If I deem it to not be a blocker I will take it off the list and punt it for the next release.  If it breaks dynamic ABI, except in a few cases, it will be punted to the next major release.  If it touches any of the static binding code in order to fix a GI issue, it will be punted to the next major release.  Most bugs, unless they absolutely positively have to be fixed will be punted to the next release.

I am pleased to announce version 2.27.91 of the Python bindings for GObject.  We found some major blockers from the last release so this is a second pre-release for the upcoming 2.28.0 stable release.  There is currently one more crasher bug on the blocker list. Once that is fixed and if there are no more blockers we will be releasing the stable version.

Fixes in this release include the following:

• ignore ginterfaces when doing method resolutions to avoid diamond inheritance issues
• Fix union inheritance issues for Gdk.Event (e.g. Gdk.EventButton structure can be passed to interfaces which expect a Gdk.Event)
• Override all Gdk.Event union members to export Gdk.Event methods
• Leak instead of crashing if marshalling a C array and an error occurs – future releases will fix the leak
• When setting rows in a TreeModel you can now pass None to indicate you wish to ignore a column
• Fix setting types not represented in Python such as GObject.TYPE_UINT, when setting a TreeModel column
• Flag and enum behaviour fixes
• Typelibs now loaded at import time instead of lazy loading to fix name resolution issues from factories which return objects where the typelib is not yet loaded (e.g. GStreamer plugins) – this requires that application authors call the gi.require_version API BEFORE importing a module from gi.repository if they wish to specify a typelib version (e.g. gi.require_verison(’Gtk’, ‘3.0′))
• don’t load the gi module when using the static bindings
• Gtk.Menu.popup fixed
• Gtk.Cursor now API equivalent to PyGtk
• Signal callbacks now convert foreign types (e.g. you get a PyCairo compatible context object in a ‘draw’ signal callback)
• Gdk.Rectangle aliases cairo.RectangleInt correctly
• fix the demos in gtk-demo.py
• For a complete list of changes please read the NEWS file

The new release is available from ftp.gnome.org and its mirrors:
http://download.gnome.org/sources/pygobject/2.27/

Blurb:
GObject is a object system library used by GTK+ and GStreamer.

PyGObject provides a convenient wrapper for the GObject library for use in Python programs, and takes care of many of the boring details such as managing memory and type casting. When combined with PyGTK, PyORBit and gnome-python, it can be used to write full featured GNOME 2.0 applications.

Like the GObject library itself PyGObject is licensed under the GNU LGPL, so is suitable for use in both free software and proprietary applications. It is already in use in many applications ranging from small single purpose scripts up to large full featured applications.

PyGObject requires glib >= 2.22.4 and Python >= 2.5.1 to build.
GIO bindings require glib >= 2.22.4.

The Introspection module is the next generation Python GObject library bindings. Instead of statically wrapping every GObject based library we can now dynamically accesses any of those libraries using GObject Introspection. It replaces the need for separate modules such as PyGTK, GIO and python-gnome to build a full GNOME 3.0 application. Once new functionality is added to gobject library it is instantly available as a Python API without the need for an intermediate Python module.

Introspection/Python 2 bindings requires gobject-introspection >= 0.9.5 and pycairo >=1.0.2 or py2cairo >=1.8.10
Introspection/Python 3 bindings requires gobject-introspection >= 0.9.5, pycairo >=1.8.10 and Python >= 3.1

I am pleased to announce version 2.27.90 of the Python bindings for GObject. This is a pre-release for the upcoming 2.28.0 stable release which should come in about a week if no major bugs are found. Note to maintainers and packagers, the 2.2x series will be the last series to support the static bindings. Going forward we will moving towards PyGObject 3.0 which will drop support for the static bindings and be able to clean up the code by breaking the current static binding ABI. The 2.28 and 3.0 branches will both be parallel installable and minimal support for the static bindings will continue in the 2.28 branch.

A lot of fixes went in under the hood getting us ready for GNOME 3.0. A number of apps have already been ported and the last major enhancement we are waiting for is support for introspected signal parameters so that the draw signal in Gtk can pass a Cairo Context that works with PyCairo. Some higlight in this release include:

• faster handling of virtual methods when constructing objects
• enhanced refcounting fixes when dealing with sink refs
• ABI stability for the static bindings
• enhanced gdbus and gvarient handling
• you can now override signals like you do vfuncs
• enhanced drag and drop support added
• return GErrors instead of generic runtime errors where appropriate
• enhanced GtkTextBuffer support
• enhanced pygi-convert.sh script for automating PyGtk to PyGObject Introspection migration
• Gio.FileEnumerator now supports the Python iter protocol
• Gdk.EventType 2BUTTON_PRESS and 3BUTTON_PRESS now prepended with an underscore
• Gtk.Adjustment now support positional or keyword arguments in constructor
• Properties now handle unicode correctly
• Fix enum handling
• dir() now returns GI attributes
• Numerous overrides to replicate PyGtk and other interfaces
• Python 3 fixes so we can compile under Python 3.2
• For a complete list of changes please read the NEWS file

The new release is available from ftp.gnome.org and its mirrors:
http://download.gnome.org/sources/pygobject/2.27/

Blurb:
GObject is a object system library used by GTK+ and GStreamer.

PyGObject provides a convenient wrapper for the GObject library for use in Python programs, and takes care of many of the boring details such as managing memory and type casting. When combined with PyGTK, PyORBit and gnome-python, it can be used to write full featured GNOME 2.0 applications.

Like the GObject library itself PyGObject is licensed under the GNU LGPL, so is suitable for use in both free software and proprietary applications. It is already in use in many applications ranging from small single purpose scripts up to large full featured applications.

PyGObject requires glib >= 2.22.4 and Python >= 2.5.1 to build.
GIO bindings require glib >= 2.22.4.

The Introspection module is the next generation Python GObject library bindings. Instead of statically wrapping every GObject based library we can now dynamically accesses any of those libraries using GObject Introspection. It replaces the need for separate modules such as PyGTK, GIO and python-gnome to build a full GNOME 3.0 application. Once new functionality is added to gobject library it is instantly available as a Python API without the need for an intermediate Python module.

Introspection/Python 2 bindings requires gobject-introspection >= 0.9.5 and pycairo >=1.0.2 or py2cairo >=1.8.10
Introspection/Python 3 bindings requires gobject-introspection >= 0.9.5, pycairo >=1.8.10 and Python >= 3.1

The hackfest is over and a lot of things got fixed but unfortunately Python3 support was broken.  I just committed a couple of patches to fix this.  Here are some things PyGObject developers should look out for. When I note to only use something in tests it means that there is a compat module that is only available to the tests. Actual code should either add the workaround to the top of their module or try not to have a distinction between things such as unicode and longs which no longer exist in Python 3.

• use range instead of xrange – loss of performance in Python 2 but Python 3 treats range similarly to python 2’s xrange
• use dict.items() instead of dict.iteritems() – same as the xrange issue
• callable does not exist in 3.x, use hasattr(obj, ‘__call__’) or

            if sys.version_info > (3, 0):
                def callable(obj):
                    return hasattr(obj, '__call__')
      

• using unicode in tests is tricky, you can’t use u” even in a versioned conditional as python3’s parser chokes on it. Do this in tests (and only i
  in tests):

            from compathelper import _unicode
            unicode_string = _unicode('this is a unicode string')
      

• exception caching changed in 2.7, instead of except Exception, e we now use except Exception as e. Do this to be compatible with older versions:

            except Exception:
                etype, e = sys.exc_info()[:2]
      

• Unbound methods with an im_func attribute no longer exits in 3.x. Unbound methods are now just functions so class.method in 3.x is equivalent to class.method.im_func in 2.x. If you have to go this low level do this:

            func = class.method
            if sys.version_info < (3,0):
                func = func.im_func
      

  Update: It was pointed out to me that this can be expressed as a simple getattr triple:

            getattr(class.method, 'im_func', class.method)
      

• all numbers are long in 3.x so 42L is invalid in 3.x. In tests (and only in tests) do this:

            from compathelper import _long
            l = _long(42)
      

  [read this post in: ar de es fr it ja ko pt ru zh-CN ]
   

It is all one big blur

We’ve been busy at the brmlab hacker space for a couple of days now hacking away at PyGObject Introspection.  A lot of bugs have been fixed and some of the biggest GEdit plugins now works quite well thanks to Ignacio.  Martin Pit has been working on porting system-config-printer which touches a lot of the Gtk+ API which he fixes as he runs into issues.  Right now he is working on making GVariants easier to use.  Nud has been hard at work tracking down reference counting issues which no one else wants to touch due to having to deal with two ref systems and a garbage collector.  Simon Schampijer is working on Gdk issues and is readying a patch to fix places where we don’t quite get constructors correct. Sebastian is porting gnome-dvb and Tomeu is unbreaking Cairo in PyGI after the Gtk+ guys decided to wrap Cairo structs in full boxed types.   Lazlo is adding annotations to Gtk+ and reviewing some of the annotations in Pavel’s gobject-introspection branch. I’ve been working on the new PyGI invoke branch which has been going really well but most likely won’t be ready to swap over until after the hackfest.

One of the greatest things about the hackfest is getting the PyGI developers familiar with debugging some of the harder areas of the library.  After this is over in a few more days everyone will go home with a greater knowledge of the different bits that make up the PyGI infrastructure.  This means we can start to accelerate porting and bug fixing of a few apps up to the release of GNOME 3 and then be in a good place to move everything over in the next cycle.

Thanks goes to Collabora for funding our beers which puts us to sleep so we can wake up refreshed the next morning.  The Collabora thank you dinner will be happening tomorrow night.   Also, thanks to the Foundation for giving us the infrastructure for organising such events.  I plan on writing something less formal than the current guidelines so other people can use as another reference to organizing similar events.  BTW, any hackers near Prague should check out the hackspace.  We need one of these in every city there is FOSS developers.

I got to a bit of a milestone today on the new PyGObject Introspection invoke code I have been working on. I can now run a handful of tests that marshal in’s out’s and returns. It is mostly just basic types right now but it works. Of significant importance is that I got the first torture test to run. We call a simple interface with a couple of in and out parameters 10000 times in a loop. Here is the output from the old implementation:

test_torture_profile (test_everything.TestTortureProfile) ...
        torture test 1 (10000 iterations): 0.240000 secs

and now from my new implementation:

test_torture_profile (test_cache.TestTortureProfile) ...
	torture test 1 (10000 iterations): 0.070000 secs

That is more than a 3x speedup for a simple case. Of course there is still a lot of work to do to handle more complex types and all of the edge cases but again, progress. I’m probably losing some speed gains due to moving to function calls instead of a big switch statement but one of the benefits of splitting everything up is when issues occur I know exactly where it is happening instead of having to scroll up the code to see if I am decoding or encoding and what type is causing the issues.

The hackfest is next week in Prague. Note to those going, because of the small amounts involved in your travel costs, I will be handling reimbursement in Euros or CZK. We will figure it all out when I get there.

I’ve been so busy I also forgot to thank Collabora who is sponsoring the hotel, a thank you dinner, beer and a coffee machine for the hackspace which is donating rooms for us to hack in. I’m looking forward to having a great time and getting some work done. Hopefully this blizzard that is hitting us tonight won’t effect my travel.

I’m sitting here at my parents house (I wanted to beat the snow for the holidays) working on the document that will guide the development of the new invoke module for PyGI. To start I attacked it from a very high level, idealistic standpoint where we could both modularize and optimize the code all in one sweeping iteration over the GI function arguments.

Of course I knew there would be edge cases and a few places where the model needed to be tweaked. As I dived more and more into the details (I’m designing the caching layer right now) I realized there isn’t enough information in GI to completely get rid of processing my own metadata, nor are the args guaranteed to be able to be processed linearly. For instance most library init functions take argc (array length) before argv (the array). Of course this isn’t to say that GI and the typelibs are not complete. They are more than complete, just that we haven’t normalized the state machine.

This all necessitates a metadata sorting loop which begs the question, why don’t we just combine that with the cache and state initialization layers and then just loop over the in args during the in validation and marshalling layer? We could for a majority of in arguments, process them in the metadata loop but for the few edge cases that break the model, it would mean separate code paths and increased complexity.

The truth is even though paper is entitled Speeding Up Invoke, my real goals are to fully understand what was going on under the hood while explaining it to others and to modularize the code so it is easier to understand (and optimize) going forward. In any case almost anything we do will be an improvement over the multiple times we call into the GI library for the same data in one invoke call. By producing a roadmap and breaking the code into smaller chunks, more people will be able to look at the code and provide optimization, feature and stability fixes. Actually the way things are laid out in the paper we will already fix some of the limitations such as simplified argument counting which allow optional user_data values, multiple callback support, and multiple lists/arrays referencing one length in value (see clutter_actor_animatev for an example).

The good news is once I fully flesh out the paper implementing it should take a couple of days. Most of it is just rearranging code. I hope to get to it before the hackfest in the middle of January.