Welcome our new Twisted Fellow: Tom Prince

Twisted is an open, community-based project that aims for the highest quality and technical standards. As a community-based open source project, Twisted is for the most part run by volunteers, with code contributed by users and developers. As part of our ongoing goal to produce reliable, maintainable software we have strict requirements for coding standards, unit testing and code reviews. Unfortunately, sometimes these goals work at cross purposes. For example, code reviews have become a bottleneck for our development process; they're hard thankless work. There's a reason they'll get you the most points in our high scores.

The Twisted Maintainer Fellowship is our attempt to bridge these two goals: to remove bottlenecks and get necessary work done, thus enabling more contributions from the community. By paying for the Fellow's efforts we can ensure an ongoing focus on the maintenance needs of Twisted, most importantly code reviews.

I am happy to announce that we have signed a contract with Tom Prince to be our new Fellow. Tom Prince is a core developer of Buildbot, who became a Twisted contributor last year via his involvement with the Buildbot project. As a result, he has been maintaining Twisted's buildbot deployment on a volunteer basis for the last eight months.  He also maintains the buildbots of other open source projects.

As you can see from the progress report below, Tom is already hard at work! I encourage you to donate to Twisted via the Software Freedom Conservancy (a 501(c)3 non-profit). By paying for Tom's time you can help support our efforts to maintain and improve Twisted.

---

Tom Prince's First Progress Report

I'm pleased to announce that I have been accepted as a Twisted fellow and full-time maintainer for 2013. I've completed my first two weeks of development. I spent the majority of my time clearing the review queue, from 32 tickets when I started to a low of 4 tickets.

All told, 42 tickets got some attention that they would not have received without the sponsored development. The result was 10 closed tickets and 29 other tickets unblocked for other developers to resume work on.

This work is made possible by the sponsorship of individuals and organizations which have donated to the Twisted project, part of the Software Freedom Conservancy, a not-for-profit organization that helps promote, improve, and develop open source software. Thanks!

The tickets I reviewed, that have now been closed:

• #2915 (Provide way of incrementing patch number with date-based versioning)
• #5601 (Add a howto for "Deploying Twisted with SystemD")
• #5925 (Deprecate twisted.python.runtime.Platform.isWinNT)
• #5937 (Check current Cooperator status)
• #6026 (MultiService.stopService doesn't handle failures in startService)
• #6208 (Admin scripts don't tweak sys.path)
• #6265 (SafeRepr.test_brokenReprIncludesID)
• #6282 (Trial tracebacks are not properly trimmed since 12.3.0)
• #6288 (OpenSSLCertificateOptions contains asserts)
• #6309 (getProcessOutput doc string should use proper epydoc instead of *this*)

Other tickets I have reviewed:

• #585 (Include sendfile(2) support into defaultreactor)
• #2611 (Protocols in twisted.protocols.basic use O(n**2) complexity)
• #4151 (Remove deprecated IDomain.startMessage)
• #4152 (Remove deprecated DomainSMTP and DomainESMTP)
• #4173 (WebSocket server support)
• #4398 (PBServerFactory should support plain password credentials)
• #4623 (Missing interface IQueue)
• #4685 (IResolver doesn't document its argument types or return values)
• #4696 (client endpoint: process)
• #4847 (Improve integration of SerialPort (via an endpoint))
• #5270 (Add API to endpoints for connecting a protocol instance)
• #5386 (Get rid of references and code specific to Python 2.3)
• #5411 (ftp_NLST and ftp_LIST should only send data as "str")
• #5787 (trial –order for selecting an order for running test cases)
• #5797 (constants cannot be attributes of another class)
• #5852 (Deprecating tlsMode)
• #5864 (Remove a sha1 import from < 2.5)
• #5953 (Split up and improve a couple of printing tests for twisted.python.failure)
• #5981 (Clean up twisted.python.threadpool implementation using context managers)
• #5989 (Standardize twisted names examples to use task.react)
• #6042 (twisted.protocols.test.test_basic doesn't following coding standard)
• #6118 (twisted.web.http.Request.noLongerQueued has no test coverage)
• #6120 (twisted.web.http.Request.__repr__ is untested)
• #6165 (Docstring for twisted.trial.unittest.SynchronousTestCase.mktemp not informative enough)
• #6249 (twisted.mail.pop3client.__all__ should be empty, so API docs list those objects in correct module)
• #6256 (Long-lived KnownHostsFile instance will clobber changes made by others if its `save` method is used)
• #6259 (epoll reactor continually errors when given a delayed call with an excessive timeout)
• #6284 (FTP.ftp_STOR() should handle IsADirectoryError error)
• #6302 (FlagConstant objects should be iterable)

Other tickets I have worked on:

• #6290 (TimerService.stopService should wait for a running call to finish.)
• #6314 (There are failing tests with tip of python 2.7 branch)
• #6044 (Deprecate `deferredGenerator')
  • #6315 (twisted/words/test/test_service.py uses `deferredGenerator`)

Twisted, Python 3, and You

As we described in our last announcement, Twisted 12.3.0 contains partial support for Python 3.3.  But what exactly does "partial support" mean?  Unfortunately, It doesn't mean that Twisted is ready for you to use on Python 3, unless all you need is the core reactor functionality.

What it really means is that it's time for you, Twisted users who care about Python 3, to contribute patches that port specific modules to Python 3.  Our development infrastructure is now set up to efficiently deal with such patches, and to keep modules working once they're ported.

• We have a buildbot set up to run a subset of the tests on Python 3.
• There is a list of supported modules that can be updated as new modules are ported, to ensure they keep working.  Most of the modules which are particularly tricky have now been ported, so contributors should now just need to do the tedious work of examining each module for small things that need to be addressed.
• There's a tool that the buildbot runs, which all Twisted developers (including you) can run to verify that everything is working properly on 3.x if you have a functioning local Python 3.3 installation.  (This tool is an interim step until Trial itself is fully ported to 3.x.)
• There's a milestone you can look at containing a list of tasks for python 3.x, although this list is not exhaustive.  If you come up with some work that needs to be done to port a module to python 3, put the ticket in that milestone.

So, if you're interested in using Twisted for a project that runs on Python 3, it should be pretty quick to port the functionality you need and submit a patch.

Twisted 12.3.0 released

On behalf of Twisted Matrix Laboratories, I am pleased to announce, in extremis, the release of Twisted 12.3.
161 tickets are closed by this release, with the following highlights:

• Support for local parallel testing in trial
• A new "react" function to easily run a single asynchronous function in a script with the reactor.
• Partial support for Python 3.3.

Twisted 12.3 requires zope.interface 3.6.0 or later, so you may need to upgrade.  Also, on Windows, Python 2.7 or later is now required.

For more information, see the NEWS file here: NEWS.txt
Download it now from: Twisted-12.3.0.tar.bz2 or Twisted-12.3.0.win32-py2.7.msi
Thanks to the supporters of Twisted via the Software Freedom Conservancy and to the many contributors for this release.

Second pre-release of Twisted 12.3.0

As it turns out, making a 4th release in 2012 wasn't going to be easy! Thanks to the testing of David Reid, we found and fixed an issue with pip which was making installation difficult when zope.interface was missing. And we got another fix with the help of Eric Mangold, fixing the compilation on OpenBSD.

The tarballs of the second pre-release can be found here: http://twistedmatrix.com/~therve/12.3.0pre2/

Thank you for testing it again!

Seeking a Motivated Maintainer for Twisted

The Twisted project, via the Software Freedom Conservancy, is seeking to recruit a full-time paid maintainer. We've got a fellowship proposal published on the wiki, but here are some highlights:

• You will be working on an open source project and your contributions will be consumed by companies such as Google, Canonical, Lucasfilm/Disney, Rackspace, and Justin.TV to name a few.
• We want to hire a full-time maintainer who will assist with the continued growth of the Twisted project.
• Thanks to the meticulous work of your predecessors, we have a clear view of what your average week would look like in terms of tasks and responsibilities (more below).

Desired qualifications in a maintainer:

• The ability to review code for suitability, stability, and clarity; to point out meritable work, and provide actionable feedback on how to improve the code when necessary. 
• The ability to implement features in the Twisted library. 
• A positive presence in Twisted's online community on IRC and the mailing list. 
• Have familiarity with the infrastructure of the Twisted project (including trac and buildbot). 
• Work to provide education or outreach on Twisted in other forums such as workshops, conferences, and user groups. 

We will strongly consider someone with experience maintaining other open source projects, but with just a little Twisted experience, as long as such an applicant is passionate about and committed to becoming a Twisted expert.

During an average work week, the maintainer will distribute their work along these lines:

• Code review (55%) Review code, apply patches, and set and enforce standards for code quality. Improve documentation and code test coverage. 
• Mailing list and IRC participation (10%) Encourage and participate in discussions on Twisted's mailing lists and IRC channels. 
• Bug report triage (5%) Supervise Twisted's ticket workflow to ensure that bug tickets are addressed and closed. 
• New feature development (30%) Serve as architect for the development and implementation of new features. 

Furthermore, the Maintainer will provide a forthnightly written report of their progress to the Conservancy and the Twisted community.

Information on whom to contact in the event of your interest is provided at the bottom of the page.

We look forward to hearing from you!

First pre-release of Twisted 12.3.0

Yes, you're reading well. For the first time ever (since 2008), we're actually pushing not 2, not 3, but 4 releases in a calendar year! Just to be able to use the fancy 12.3 version number.

But, apart from this nice label, this release also closes 158 tickets, among them:

• Support for local parallel testing in trial.
• A new "react" function to easily run a single asynchronous function in a script with the reactor.
• Partial support for Python 3.3.

Note that only Python 2.7 is supported on Windows now. We also require zope.interface 3.6.0 or newer.

You can find the tarballs of the pre-release here: http://twistedmatrix.com/~therve/12.3.0pre1/

For more information, see the NEWS file.

Please download the tarballs and test them!

Quick Fix: Upgrading Zope Interface so you can work on Twisted trunk post-12.2 release

Immediately after the recent 12.2 release, Twisted has upgraded its required version of Zope Interface.  Some platforms ship with a system-installed version of Zope Interface that is slightly too old for this change.  You may have noticed an unhelpful error message if you happened to svn up before learning about this change.

Upgrading system-installed Python packages without doing harm can be a challenge, so in this post we'll provide simple instructions for users who:

• have a system-installed version of Zope Interface
• are on a Unix-y platform, like Linux, BSD, or OS X (if you're on Windows, just upgrade to the latest on your system installation)
• are not using a tool like virtualenv or Combinator to manage Zope Interface (if you are, you should already know what to do!)
• don't want to upgrade anything system-wide, so as to avoid disturbing other users or software on their system
• have easy_install already available in their environment.  (While Pip is generally better, it is pre-installed less often, and Pip doesn't seem to deal as well as easy_install with the case of parallel installation of user and system installed dependencies; it assumes you are using virtualenv for that. Update: as of the most recent release,  pip deals just fine with parallel user/system installation; so, if you've got pip installed, "pip install zope.interface==4.0.2" – possibly with a --user option if that's appropriate to your environment – will work just fine.  Thanks, Pip maintainers!)

This set of attributes describes many Linux and OS X developers who are likely to encounter this version skew issue, so off we go.

Since easy_install can't be coerced to do the right thing by command-line options, you'll need to give it a configuration file that it can read.  The easiest way to do this is to create a temporary directory, cd into it, and create a file called setup.cfg.  For Linux and BSD, setup.cfg looks like this:

[install]
prefix=~/.local

For OS X, the file looks like this:

[install]
prefix=~/.local
install_lib=~/Library/Python/$py_version_short/lib/python/site-packages

Once you've created that file, just  easy_install --upgrade zope.interface from that directory and you should be ready to start developing with Twisted again!  All the older versions of Twisted that I've tried still work with the newest Zope Interface so this shouldn't disturb your ability to test your code against older versions of Twisted as well, if you're managing it with something like Combinator or virtualenv.