April mid-month Fellowship report

During the first half of the month, 43 tickets got some attention that they would not have received without the sponsored development. The result was 17 closed tickets and 24 other tickets unblocked for other developers to resume work on.

I set up an official git mirror that contains all the branches, including release branches and tags on github and documented my workflow on the wiki.

I've documened how our current infrastructure (cube), and with the help of Jonathan Stoppani, I've been working on scripts to automate the deployment of its successor (dornkirk).

This work is made possible by the sponsorship of individuals and organizations which have donated to the Twisted project, part of the Software Freedom Conservancy, a not-for-profit organization that helps promote, improve, and develop open source software. Thanks!

The tickets I reviewed, that have now been closed:

• #3844 (Parse mIRC format codes)
• #4696 (client endpoint: process)
• #5386 (Get rid of references and code specific to Python 2.3)
• #5411 (ftp_NLST and ftp_LIST should only send data as "str")
• #5952 (Improve docstring for test_notTrapped in twisted.test.test_failure)
• #5955 (Move inlineCallbacks test from twisted.test.test_failure to somewhere better (test_defer?))
• #5989 (Standardize twisted names examples to use task.react)
• #6120 (twisted.web.http.Request.__repr__ is untested)
• #6240 (Merge `twisted/internet/_utilspy3.py` into `twisted/internet/utils.py`)
• #6284 (FTP.ftp_STOR() should handle IsADirectoryError error)
• #6350: Add a few unittest2 assertions to TestCase
• #6391 (Improve the format of http.Request.__repr__ to include eg class and object identity)
• #6392: Typo in test_irc
• #6400 (trial.test.test_directoryNotPackage is incorrect)
• #6405 (os.geteuid and os.getegid are called unnecessarily in SSHPublicKeyDatabase.checkKey)
• #6430 (test_loader.FileTest.test_filenameNotPython should create its test data)
• #6439 (Spelling mistakes in pb-intro.xhtml)

Other tickets I have reviewed:

• #2061: _sslverify.py: Wrong use of param caCerts in OpenSSLCertificateOptions
• #3701: Add a checker for ISSHPrivateKey which parameterizes the location of the keys
• #4891: twisted.web.http does not throttle pipelined requests
• #5228: twisted.test.test_paths.FilePathTestCase.test_getPermissions_Windows creates undeleteable file
• #5270: Add API to endpoints for connecting a protocol instance
• #5329: Replace deprecated string functions with str methods in t.w.p.irc
• #5435: twisted.web.client.RedirectAgent doesn't inform caller of final URI
• #5531: Add anonymous support to Conch SSHFactory
• #5532: AMP Command specification format
• #5787: trial –order for selecting an order for running test cases
• #5795: Failure's startDebugMode should allow using a debugger other than pdb
• #585: Include sendfile(2) support into defaultreactor
• #5951: Documentation for the best way to get a reference to a reactor reads as contradictory
• #6115: twisted.web.http.parseContentRange has no unit tests
• #6119: twisted.web.http.Request.parseCookies has incomplete test coverage
• #6166: Deprecate pickling support (__getstate__ and __setstate__) in twisted.internet._sslverfiy
• #6341: Replace usage of twisted.python.text and deprecate it
• #6342: Deprecate twisted.python.hashlib
• #6390: Using trial -j can result in wrong paths in sys.path
• #6393: Utility function to pretty-format list of things
• #6395: Delete twisted.words.protocols.msn
• #6402: twisted.words setup metadata is not up to date anymore
• #6426: test_mail.py failure
• #6437: twisted.trial.unittest.TestCase.flushWarnings may fail if run without .py files (with only .pyc or .pyo files)

Other tickets I have worked on:

• #6251 (Add a helper function for getting the body of an HTTP request.)
• #6348 (trial doesn't show `DeprecationWarnings` by default on 2.7)

Join Twisted for Google Summer of Code and the Outreach Program for Women

Twisted is once again participating in Google's Summer of Code this year, as well as the GNOME Outreach Program for Women. If you're a student interested in working on Twisted as part of a paid internship, please visit our Google SoC page. We use best practices like in-depth code reviews and full coverage unit tests, so this is a great way to improve your technical skills whether you're a beginner or an expert programmer.

In addition, Twisted is once again participating the GNOME Outreach Program for Women, with the generous support of Mozilla, providing a paid internship for one woman to spend the summer participating in and contributing to Twisted. Unlike Google's program, the outreach program is not restricted to students; if you qualify, we do encourage you to apply to both. This internship is appropriate for any level of open source experience.

If you have worked on an open source project before, great! If not, we'll help you learn the development and communication tools we use as part of the internship. Some Python experience is a prerequisite, and a small initial contribution to Twisted is a part of applying (if this sounds intimidating, don't worry, we'll help you pick a task to complete and you'll have lots of support as you work through submitting your first patch.) Please check out the full OPW and GSoC descriptions and apply today!

Twisted 13.0.0 released

On behalf of Twisted Matrix Laboratories, I am pleased to announce the release of Twisted 13.0.

Among the 70 tickets closed, we can see:

• A new Introduction to Deferreds  document.
• A fix in twisted.web.template where attributes were not quoted properly, risking HTML injection. 
• Support for unicode domain names in twisted.names SRVConnector and Name classes, after a 12.3 regression
• A workaround for platform limitations when trying to schedule events far in the future.

For more information, see the NEWS file here: NEWS.txt

Download it now from Twisted-13.0.0.tar.bz2 or Twisted-13.0.0.win32-py2.7.msi

Thanks to the supporters of Twisted via the Software Freedom Conservancy and to the many contributors for this release.

Report the Third

During the past three weeks, 23 tickets got some attention that they would not have received without the sponsored development. The result was 9 closed tickets and 8 other tickets unblocked for other developers to resume work on.

I also spent some time working on getting newly donated slaves setup, as well as documenting the setup of the main twistedmatrix.com server, in preparation for automating the redeployment of services to new hardware.

This work is made possible by the sponsorship of individuals and organizations which have donated to the Twisted project, part of the Software Freedom Conservancy, a not-for-profit organization that helps promote, improve, and develop open source software. Thanks!

The tickets I reviewed, that have now been closed:

• #1333: Survey existing FTP servers and clients to figure out how LIST works
• #3908: Code using twisted.names.client.Resolver is hard to unit test
• #6240: Merge `twisted/internet/_utilspy3.py` into `twisted/internet/utils.py`
• #6337: OpenSSLCertificateOptions doesn’t disallow SSLv2
• #6354: Main NEWS file is not generated properly anymore
• #6377: print traceback in successResultOf error reporting
• #6380: Add optional expected exception types to failureResultOf
• #6392: Typo in test_irc

Other tickets I have reviewed:

• #5126: Add http cache support to Agent
• #5270: Add API to endpoints for connecting a protocol instance
• #5386: Get rid of references and code specific to Python 2.3
• #5411: ftp_NLST and ftp_LIST should only send data as "str"
• #5955: Move inlineCallbacks test from twisted.test.test_failure to somewhere better (test_defer?)
• #6286: SSL server endpoints string syntax results in creation of DefaultOpenSSLContextFactory
• #6341: Replace usage of twisted.python.text and deprecate it
• #6350: Add a few unittest2 assertions to TestCase

Tickets I have closed:

• #6316: Minor cleanup of pam tests in `twisted.test.test_newcred`.

Other tickets I have worked on:

• #1009: Document inlineCallbacks
• #6251: Add a helper function for getting the body of an HTTP request.
• #6284: FTP.ftp_STOR() should handle IsADirectoryError error
• #6286: SSL server endpoints string syntax results in creation of DefaultOpenSSLContextFactory
• #6363: Make twisted.trial.unittest._iterateTests public.

Second Fellowship Report

I've completed my second two weeks of development. I again spent a large amount of my time reviewing tickets. In addition, I did some development work, as well as buildbot improvements.

All told, 42 tickets got some attention that they would not have received without the sponsored development. The result was 23 closed tickets and 14 other tickets unblocked for other developers to resume work on.

Buildbot improvements

• Added a pyflakes builder.
• Improved lint-like builders to only show new warnings/errors introduced in the branch, rather than relative to the tip of trunk
• Added a form to the build results page to allow switching branches and forcing builds.
• Added an easy way to look at the history of the unsupported builders.
• Removed links to some useless views.

This work is made possible by the sponsorship of individuals and organizations which have donated to the Twisted project, part of the Software Freedom Conservancy, a not-for-profit organization that helps promote, improve, and develop open source software. Thanks!

The tickets I reviewed, that have now been closed:

• #1333 (Survey existing FTP servers and clients to figure out how LIST works)
• #4685 (IResolver doesn't document its argument types or return values)
• #5797 (constants cannot be attributes of another class)
• #5852 (Deprecating tlsMode)
• #5918 (t.p.b.StatefulStringProtocol has undocumented .state instance attribute)
• #5953 (Split up and improve a couple of printing tests for twisted.python.failure)
• #5989 (Standardize twisted names examples to use task.react)
• #6218 (AbstractMaildirDomain implements IAliasableDomain but doesn't advertise that fact.)
• #6256 (Long-lived KnownHostsFile instance will clobber changes made by others if its `save` method is used)
• #6259 (epoll reactor continually errors when given a delayed call with an excessive timeout)
• #6297 (Remove usage of twisted.python.compat.set and frozenset)
• #6322 (Missing tests for NotImplementedError in t.protocols.basic)
• #6332 (fake twisted.trial._dist.test.test_workertrial.MainTestCase.fdopen returns None sometimes)
• #6338 (IResolver docstrings contain broken references)
• #6349 (MemoryReactor incompletely implements IReactorSocket)
• #6337 (OpenSSLCertificateOptions doesn’t disallow SSLv2)
• #6354 (Main NEWS file is not generated properly anymore)

Other tickets I have reviewed:

• #84 (We need tests for example code)
• #585 (Include sendfile(2) support into defaultreactor)
• #4859 (client endpoint: super-smart name-based TCP connection algorithm)
• #5386 (Get rid of references and code specific to Python 2.3)
• #5411 (ftp_NLST and ftp_LIST should only send data as "str")
• #5513 (Detailed documentation on Proxy by adding docstrings to many of the methods.)
• #5952 (Improve docstring for test_notTrapped in twisted.test.test_failure)
• #5955 (Move inlineCallbacks test from twisted.test.test_failure to somewhere better (test_defer?))
• #6220 (Replace Deprecated Assertion Methods in twisted.conch)
• #6240 (Merge `twisted/internet/<sub>utilspy3</sub>.py` into `twisted/internet/utils.py`)
• #6286 (SSL server endpoints string syntax results in creation of DefaultOpenSSLContextFactory)

Tickets I have closed:

• #6290 (TimerService.stopService should wait for a running call to finish.)
• #6323 (twisted.python.test.test_reflect emits DeprecationWarning)
• #6324 (twisted/python/test/test_util.py emits DeprecationWarning)
• #6326 (web/test_template emits DeprecationWarning)

Other tickets I have worked on:

• #601 (ThrottlingFactory doesn't throttle static web resource)
  • #6339 (Make twisted.protocol.tls._PullToPush public.)
• #3231 (twisted.trial.test.test_testvisitor emits deprecation warnings)
• #5513 (Detailed documentation on Proxy by adding docstrings to many of the methods.)
• #5921 (Trial fails flushing reporter stream with ENOSPC on Windows)
• #6291 (SyncTestCase.assertNoResult should not eat any eventual result.)
• #6314 (There are failing tests with tip of python 2.7 branch)
• #6333 (Deprecate DryRunVisitor.)

Welcome our new Twisted Fellow: Tom Prince

Twisted is an open, community-based project that aims for the highest quality and technical standards. As a community-based open source project, Twisted is for the most part run by volunteers, with code contributed by users and developers. As part of our ongoing goal to produce reliable, maintainable software we have strict requirements for coding standards, unit testing and code reviews. Unfortunately, sometimes these goals work at cross purposes. For example, code reviews have become a bottleneck for our development process; they're hard thankless work. There's a reason they'll get you the most points in our high scores.

The Twisted Maintainer Fellowship is our attempt to bridge these two goals: to remove bottlenecks and get necessary work done, thus enabling more contributions from the community. By paying for the Fellow's efforts we can ensure an ongoing focus on the maintenance needs of Twisted, most importantly code reviews.

I am happy to announce that we have signed a contract with Tom Prince to be our new Fellow. Tom Prince is a core developer of Buildbot, who became a Twisted contributor last year via his involvement with the Buildbot project. As a result, he has been maintaining Twisted's buildbot deployment on a volunteer basis for the last eight months.  He also maintains the buildbots of other open source projects.

As you can see from the progress report below, Tom is already hard at work! I encourage you to donate to Twisted via the Software Freedom Conservancy (a 501(c)3 non-profit). By paying for Tom's time you can help support our efforts to maintain and improve Twisted.

---

Tom Prince's First Progress Report

I'm pleased to announce that I have been accepted as a Twisted fellow and full-time maintainer for 2013. I've completed my first two weeks of development. I spent the majority of my time clearing the review queue, from 32 tickets when I started to a low of 4 tickets.

All told, 42 tickets got some attention that they would not have received without the sponsored development. The result was 10 closed tickets and 29 other tickets unblocked for other developers to resume work on.

This work is made possible by the sponsorship of individuals and organizations which have donated to the Twisted project, part of the Software Freedom Conservancy, a not-for-profit organization that helps promote, improve, and develop open source software. Thanks!

The tickets I reviewed, that have now been closed:

• #2915 (Provide way of incrementing patch number with date-based versioning)
• #5601 (Add a howto for "Deploying Twisted with SystemD")
• #5925 (Deprecate twisted.python.runtime.Platform.isWinNT)
• #5937 (Check current Cooperator status)
• #6026 (MultiService.stopService doesn't handle failures in startService)
• #6208 (Admin scripts don't tweak sys.path)
• #6265 (SafeRepr.test_brokenReprIncludesID)
• #6282 (Trial tracebacks are not properly trimmed since 12.3.0)
• #6288 (OpenSSLCertificateOptions contains asserts)
• #6309 (getProcessOutput doc string should use proper epydoc instead of *this*)

Other tickets I have reviewed:

• #585 (Include sendfile(2) support into defaultreactor)
• #2611 (Protocols in twisted.protocols.basic use O(n**2) complexity)
• #4151 (Remove deprecated IDomain.startMessage)
• #4152 (Remove deprecated DomainSMTP and DomainESMTP)
• #4173 (WebSocket server support)
• #4398 (PBServerFactory should support plain password credentials)
• #4623 (Missing interface IQueue)
• #4685 (IResolver doesn't document its argument types or return values)
• #4696 (client endpoint: process)
• #4847 (Improve integration of SerialPort (via an endpoint))
• #5270 (Add API to endpoints for connecting a protocol instance)
• #5386 (Get rid of references and code specific to Python 2.3)
• #5411 (ftp_NLST and ftp_LIST should only send data as "str")
• #5787 (trial –order for selecting an order for running test cases)
• #5797 (constants cannot be attributes of another class)
• #5852 (Deprecating tlsMode)
• #5864 (Remove a sha1 import from < 2.5)
• #5953 (Split up and improve a couple of printing tests for twisted.python.failure)
• #5981 (Clean up twisted.python.threadpool implementation using context managers)
• #5989 (Standardize twisted names examples to use task.react)
• #6042 (twisted.protocols.test.test_basic doesn't following coding standard)
• #6118 (twisted.web.http.Request.noLongerQueued has no test coverage)
• #6120 (twisted.web.http.Request.__repr__ is untested)
• #6165 (Docstring for twisted.trial.unittest.SynchronousTestCase.mktemp not informative enough)
• #6249 (twisted.mail.pop3client.__all__ should be empty, so API docs list those objects in correct module)
• #6256 (Long-lived KnownHostsFile instance will clobber changes made by others if its `save` method is used)
• #6259 (epoll reactor continually errors when given a delayed call with an excessive timeout)
• #6284 (FTP.ftp_STOR() should handle IsADirectoryError error)
• #6302 (FlagConstant objects should be iterable)

Other tickets I have worked on:

• #6290 (TimerService.stopService should wait for a running call to finish.)
• #6314 (There are failing tests with tip of python 2.7 branch)
• #6044 (Deprecate `deferredGenerator')
  • #6315 (twisted/words/test/test_service.py uses `deferredGenerator`)

Twisted, Python 3, and You

As we described in our last announcement, Twisted 12.3.0 contains partial support for Python 3.3.  But what exactly does "partial support" mean?  Unfortunately, It doesn't mean that Twisted is ready for you to use on Python 3, unless all you need is the core reactor functionality.

What it really means is that it's time for you, Twisted users who care about Python 3, to contribute patches that port specific modules to Python 3.  Our development infrastructure is now set up to efficiently deal with such patches, and to keep modules working once they're ported.

• We have a buildbot set up to run a subset of the tests on Python 3.
• There is a list of supported modules that can be updated as new modules are ported, to ensure they keep working.  Most of the modules which are particularly tricky have now been ported, so contributors should now just need to do the tedious work of examining each module for small things that need to be addressed.
• There's a tool that the buildbot runs, which all Twisted developers (including you) can run to verify that everything is working properly on 3.x if you have a functioning local Python 3.3 installation.  (This tool is an interim step until Trial itself is fully ported to 3.x.)
• There's a milestone you can look at containing a list of tasks for python 3.x, although this list is not exhaustive.  If you come up with some work that needs to be done to port a module to python 3, put the ticket in that milestone.

So, if you're interested in using Twisted for a project that runs on Python 3, it should be pretty quick to port the functionality you need and submit a patch.