Google Summer of Code and Outreach Program for Women

Twisted is participating in Google's Summer of Code this year. If you're a student interested in working on Twisted as part of a paid internship, please visit our Google SoC page. We use best practices like in-depth code reviews and full coverage unit tests, so this is a great way to improve your technical skills whether you're a beginner or an expert programmer.

In addition, Twisted and its non-profit home the Software Freedom Conservancy have partnered with the GNOME Outreach Program for Women to fund an internship for one woman to spend the summer participating in and contributing to Twisted, while being mentored by Jessica McKellar. Jessica is a long-time Twisted contributor as well as a software engineer and an organiser, among other things, of the Boston Python Workshops for women.

Unlike Google's program, the outreach program is not restricted to students; if you qualify, we do encourage you to apply to both. This internship is appropriate for any level of open source experience. If you have worked on an open source project before, great! If not, we'll help you learn the development and communication tools we use as part of the internship. Some Python experience is a prerequisite, and a small initial contribution to Twisted is a part of applying (if this sounds intimidating, don't worry, we'll help you pick a task to complete and you'll have lots of support as you work through submitting your first patch.) Please check out the full project description and apply today!

Announcing Twisted 12.0

Thanks to Thomas Hervé, we are proud to announce the release of Twisted 12.0.

47 tickets are closed by this release, among them:

• A fix to the GTK2 reactor preventing unnecessary wake-ups 
• Preliminary support of IPV6 on the server side
• Several fixes to the new protocol-based TLS implementation
• Improved core documentation's main page 

Twisted no longer supports Python 2.4, the latest supported version is 2.5.

For more information, see the NEWS file here: http://twistedmatrix.com/Releases/Twisted/12.0/NEWS.txt

Download it now from:
http://pypi.python.org/packages/source/T/Twisted/Twisted-12.0.0.tar.bz2 or
http://pypi.python.org/packages/2.5/T/Twisted/Twisted-12.0.0.win32-py2.5.msi or
http://pypi.python.org/packages/2.6/T/Twisted/Twisted-12.0.0.win32-py2.6.msi or
http://pypi.python.org/packages/2.7/T/Twisted/Twisted-12.0.0.win32-py2.7.msi

Thanks to the supporters of Twisted via the Software Freedom Conservancy and to the many contributors for this release.

December Sprint Report

Twisted sprint? Twisted sprint! Here's the final Twisted sprint report of 2011, from our December 10th event at Smarterer in Boston.

David Sturgis:

• was Sprint Host and Food Wrangler
• discussed miscellaneous twisted.web feature

JP Calderone:

• investigated 64 bit Windows 7 IPv6 problems
• finished #5383: Provide a library for simple valued named constants
• finished #5084: Accept IPv6 address literals (with embedded scope ids) in IReactorTCP.listenTCP
• reviewed #3420: twisted.web.client persistent connections
• reviewed #1902: compatibility work-around for commercial SSH 2.0.12 misbehaviours
• reviewed #5400: Change UDP port to have an explicit state machine, and no FileDescriptor dependency
• reviewed #3648: twisted.cred.credentials.UsernameHashedPassword doesn't hash password strings when checkPassword

This was the last sprint for JP as a Bostonian. We will miss you!

Itamar:

• worked on #5427: Improve core documentation index page
• reviewed #5383: Provide a library for simple valued named constants

Alex Levy:

• made headway on several website and documentation improvements

Glyph:

• discussed and then worked on #1956: Make a less sucky producer/consumer API

I (Jessica McKellar):

• reviewed #5427: Improve core documentation index page
• reviewed #5429: Documentation index
• reviewed #5422: pbgtk2.py example is excessively complex

Thank you David for organizing this, and Smarterer for hosting.

Thank you to everyone who closed out 2011 with contributions to Twisted!

November Sprint Report

November saw another successful Sprint at the Smarterer.com offices, thanks to a smaller but extremely busy group of developers:

Reviewmeister(ess) Jessica McKellar secured second place in the Twisted High Score list with an impressive number of reviews, including:

• #3037: "loseWriteConnection breaks loseConnection" (now resolved)
• #4262: "Multicast documentation is misleading" (now resolved)
• #538: "Website-template.tpl is out of date"

She also resolved #1247: ("kqreactor timeout error (traceback)"), and opened #5385, #5386, and #5387, while resolving #5060: ("Correct python requirement in twisted/topfiles/setup.py").

Not to mention all the tickets reviewed at recent Sprints which have since been resolved! (Though, to mention them anyway, #1247, #2115, #2447, #2498, #2507, and #5370)

Eternal High Score Overlord JP Calderone made a fair showing as well, finishing #5371 ("In the developer guide, forbid new global reactor unit tests"), and splitting #4671 ("sometimes I want to have some constant values that are part of a set") into more managable tickets:

• #5382: "Provide a library for valueless named constants"
• #5383: "Provide a library for simple valued named constants"
• #5384: "Provide a library for bitvector-like valued named constants"

Of which #5382 and #5383 saw immediate progress.

Returning Champion Sprinter Paul Swartz reviewed and otherwise worked on several tickets, including:

• #2997: "Conch and getPeer"
• #5352: "Do not register overlapping old DH key exchange messages"
• #1902: "Compatibility work-around for commercial SSH 2.0.12 misbehaviours"
• #3242: "Use python 2.5 'spwd' module instead of z3p secret 'shadow' module when available"

One-word Twisted celebrity Glyph was also on hand to assist with the reviews of #3242, #5382, and #1918

With considerably more words in his name, Itamar Turner-Trauring started porting a Nevow branch with IRI support (internationalized URLs) to Twisted as per ticket #5388, as well as helping resolve #3037 with Jessica.

Finally, Glyph, Itamar, and Dave Sturgis conferred on #288, and the related cluster of web-resource-model issues it involves; It is hoped that this will eventually result in a new Web Server URL Traversal API, or WSURLTA, not to be confused with the nordic kraken-headed demigod of lore.

Twisted 11.1.0 has been released!

Twisted Matrix Laboratories is proud to announce the release of Twisted 11.1, thanks to the efforts of our release manager Thomas Hervé.

Highlights of the 185 tickets closed include:

• The poll() reactor as default where applicable, instead of select() everywhere.
• A new SSL implementation only relying on OpenSSL for cryptography, (not I/O) making it more robust.
• Several improvements to the fresh HTTP/1.1 client implementation, including proxy and cookie support.
• Thomas' personal favorite: a new howto has been published on test-driven development with Twisted.
• A new abortConnection() method for immediately closing TCP and SSL connections, the oldest ticket closed by this release.

For more information, see the NEWS file. Note that this is the last release supporting Python 2.4 (the support on Windows stopped with 11.0).

Thanks to the supporters who have donated to Twisted via the Software Freedom Conservancy, and to the many contributors for this release! Download it now:

• Twisted-11.1.0.tar.bz2
• Twisted-11.1.0.win32-py2.5.msi
• Twisted-11.1.0.win32-py2.6.msi
• Twisted-11.1.0.win32-py2.7.msi

Sponsored Development, October 2011

I'm happy to report that I've just completed another two weeks of sponsored Twisted development.  As usual, the issue tracker directed most of my work.  I spent time on new development aimed at resolving tickets and I reviewed changes proposed by other developers with that aim.

All told, 59 tickets got some attention that they would not have received without the sponsored development.  The result was 36 closed tickets and 17 other tickets unblocked for other developers to resume work on.  Over the coming days or weeks a few more tickets will probably be resolved as a result of this work, as developers (including myself) respond to review feedback.

This work is made possible by the sponsorship of individuals and organizations which have donated to the Twisted project, part of the Software Freedom Conservancy, a not-for-profit organization that helps promote, improve, and develop open source software.  Thanks!

Apart from working to resolve tickets in the issue tracker, I also did some work on infrastructure.  I:

• cleaned up our BuildBot master configuration and published it
• implemented improved error reporting for our API documentation build automation
• converted the subversion repository for the Twisted website to bzr and made it public
• moved some Windows build infrastructure to a new host with better network connectivity to try to reduce spurious build failures

For more details about a ticket, see <http://tm.tl/NNNN>.

The tickets I reviewed, with tickets now closed in bold:

#78 - abortConnection() method for transports

#2674 - t.i.defer.inlineCallbacks documentation issues

#2675 - Test timeout policy should be decided by runner

#3077 - twistd --help plugin list isn't alphabetized

#3078 - Dynamic ZSH tab completion for any commands using t.p.usage

#3350 - make each peer register as a producer in portforward

#3453 - twisted.web._auth.digest mostly ignores the value of the uri field

#3896 - Passing a unicode object to request.write corrupts the entire response

#3926 - twisted.positioning -- a better positioning framework

#4159 - Add pre-exec hook to spawnProcess

#4519 - Delay parsing of request body until Request.args is accessed

#4751 - t.c.telnet.ITelnetProtocol.unhandledSubnegotiation doesn't reflect reality

#4849 - Add stream compression to twisted.words.jabber

#5024 - MSI buildbot patches version to avoid breaking distutils MSI builder, but does it in the wrong place

#5040 - twisted.web.template is lacking patterns

#5044 - Client documentation should use explicit buildProtocol()s rather than magic "protocol = MyProtocol" on factories

#5085 - Accept IPv6 address literals (with embedded scope ids) in IReactorTCP.connectTCP

#5139 - Replace usage of os.path.walk in favor of twisted.python.filepath.FilePath.walk

#5156 - no ambiguous antecedents (plugins documentation edition)

#5192 - 100-continue support for twisted.web.client.Agent

#5194 - log.msg can fail if someone removes an observer at the wrong time

#5250 - finger15.tac patch

#5252 - Manhole should support CTRL-A and CTRL-E for home/end

#5253 - Add support for input history persistence in twisted.conch.manhole

#5263 - add multipart/form-data support to twisted.web.client.Agent for easy form posts

#5267 - allow twisted.internet.ssl.[Private]Certificate (and friends) to be backed with TLSLite rather than pyOpenSSL

#5271 - trial test method timeout support is fragile

#5275 - t.w.template does not always properly escape comments

#5279 - pass mode to FilePath.createDirectory()

#5282 - ILogObserver, log.msg, log.err, and logging.html are insufficiently stern about log observer thread safety

#5283 - opt_user unused in manhole twistd plugin

#5285 - win32eventreactor misses disconnect events that happen too soon

#5286 - twisted.web.template documentation doesn't cover very basic use-case of rendering more than one of something

#5288 - twisted.web.template.Tag.fillSlots documentation doesn't mention it returns self

#5301 - test_tcp_internals.PlatformAssumptionsTestCase fails when we leak fds

#5312 - Create some release automation for building Sphinx documentation

And tickets I developed:

#581 - CopiedFailure.getTraceback should include the result of CopiedFailure.getErrorMessage even if unsafeTracebacks is not set

#745 - writeSequence not well-tested

#1946 - ErrorHolder is basically untested

#2838 - _dumbwin32proc.Process does not implement IProcessTransport completely

#4603 - names should handle unknown records

#4666 - t.i.task.Cooperator scheduler objects must return an IDelayedCall

#4671 - sometimes I want to have some constant values that are part of a set

#5040 - twisted.web.template is lacking patterns

#5062 - ProtocolWrapper masks the name of the wrapped protocol in the logs

#5075 - IntNStringReceiver copies too much data in dataReceived, causing AMP to be slow

#5140 - Deprecate twisted.scripts.tkunzip

#5158 - Installing on Pypy fails because of CPython-specific extension modules

#5233 - win32eventreactor doesn't notice TCP connection is lost in certain (hopefully rare) cases

#5237 - Add SFTP example

#5273 - Improve the API documentation for MultiService.addService to clarify its use in comparison to Service.setServiceParent

#5278 - When connecting with endpoints, the application factory used is neither started nor stopped, and irrelevant start/stop messages are logged instead

#5285 - win32eventreactor misses disconnect events that happen too soon

#5291 - twisted.trial TestCase should support assertDictEqual from Python 2.7 unittest

#5292 - Listening ports used with wrapping factories could log better start messages

#5293 - Remove mktap and corresponding twistd options (mktap is broken in trunk)

#5299 - udp.Port schedules connectionLost call in the global reactor

#5303 - Default transport implementation doesn't write out bytes immediately

#5304 - Win32Reactor.spawnProcess is redundant

#5308 - Make UDP ports identify themselves when stopping

#5316 - Incorrect API link in twisted.web.proxy.Proxy docstring

October Sprint Report

After a brief rest, the Boston Twisted crew is back on the sprinting treadmill. Dave Sturgis came through for us once again for the venue, working with Smarterer to let us use their office space and eat their snacks last Saturday afternoon. We also took advantage of the Python Software Foundation's sprint funding program to buy all the sprinters dinner.

Eight sprinters turned out to fix bugs, add features, and review changes.

Chris (radix) (triumphantly returned from the inhospitable southwest) worked on adding a history-tracking mode to Deferreds to aid in debugging. This feature will let a developer see everything that has happened to a Deferred - what callbacks it has run, what values have passed through it, and what other Deferreds it has been associated with.

Itamar picked up a ticket relating to HTTP client support for persistent connections. He also finished up the work on adding a new transport method, abortConnection, fixed a logging bug in the new TLS implementation, and did some ticket triage and a review.

Allister (amacleod) heroically tackled the problem of our insufficient Windows testing infrastructure, getting introduced to our build farm and setting up a new Windows 7 virtual machine on it.

Jessica (jesstess) did a ton of follow-up on old tickets (1247 2115 2447 2498 2507 2513 2861), trying to learn if they are still valid and elicit further information from the original reporters. She also managed to close one which was no longer valid (but she also filed a new one for a documentation bug, in case you were worried we might run out of tickets).

Glyph knuckled down on a review for our conversion to Sphinx. We're so close I can practically taste it.

Ying (cyli) took a look at improving the authentication options for some of Twisted's built-in servers. She started with the FTP server and made great headway.

As for myself, I reviewed that abortConnection feature as well as a ticket for HTML5 support in twisted.web.template and the TLS logging bug that Itamar fixed.

Thanks to all the sprinters and to Smarterer and the PSF for their support of the sprint! We're going to do another one of these real soon, watch out for an announcement!