Thursday, July 10, 2008

June/July TSF Sponsored Development

The second (of what should end up being many) round of TSF sponsored development has just wrapped up. Like last time, this was two weeks of work by yours truly (with an afternoon of Glyph's time spent reviewing code to keep things moving). This time around there were fewer tickets waiting to be reviewed when I got started, however there was still plenty of code to look at. Over the full period, I reviewed:

#1144 Documentation: twisted.internet.reactor does not appear in the API docs
#1253 Create index.xhtml files for non-core doc trees
#1255 Update copyrights in the man pages
#1878 twisted.web.monitor traceback_ AttributeError: class IChangeNotified has no attribute '__class__'
#1890 move examples from core into correct sub-packages
#1900 Error in documentation online
#2169 twisted.plugin documentation errors
#2208 Standardize on the Python shebang line
#2438 Get rid of references to maintainer email addresses from code
#2552 broken links in intro.xhtml
#2607 conch.checks.SSHPublicKeyDatabase calls os.seteuid/os.setegid even if it's not necessary
#2716 Eliminate relative imports from twisted.conch
#2815 Update VFS backends to an async interface
#2821 create twistd plugin for vfs
#2845 twisted.internet.thread._putResultInDeferred should be public
#3182 Trivial typo in twisted.internet.protocol
#3257 Rewrite twisted.web.static.File.directoryListing to not use woven
#3269 curses.setupterm must only be called once per process
#3300 twistd should support setting the syslog facility
#3326 Typo in gtkmanhole.py

Other tickets I worked on included:

#1069 log observers that throw exceptions should not be removed
#1152 xmlrpc.html doesn't describe how to return errors to the client
#1291 Expose "process exited" hook on ProcessProtocol_ turning current processEnded into user-overriddable behavior
#1493 static File web module doesn't support byte ranges
#2303 Deprecate setUpClass and tearDownClass and_ if possible_ fix the subclassing behaviour.
#2327 Intermittent failure in PB tests
#2631 Update coding standard to indicate preference for TestCase methods which being with "assert" and which do not have an underscore in their name
#2874 _sslverify.problemsFromTransport should be deprecated
#3029 documentation for twisted.python.deprecate.deprecated is incomplete (and other sundries)
#3059 twisted.internet.tcp.Client.getPeer incorrectly returns hostnames
#3116 Errors in processEnded can cause processes to be eternally reaped
#3159 t.i.utils process functions should have a default cwd of None_ not '.'
#3218 SSL disconnection sometimes hangs indefinitely with pyOpenSSL 0.7
#3255 Trial fails to display the line where the error occured in case of SyntaxError
#3300 twistd should support setting the syslog facility
#3301 superfluous local in AMP.__init__
#3305 CR IAC ignored in TelnetClient
#3306 twisted.test.test_ssl.StolenTCPTestCase has a number of defects
#3339 mailmail raises an exception instead of giving an error message

In particular, I spent a lot of time on #1291 which will make it possible to control child processes more precisely with Twisted by separating out the notification that a child process has actually exited from the notification that all of its file descriptors have been closed.


Thanks to the SFC (<http://conservancy.softwarefreedom.org/>) and all of the sponsors (<http://twistedmatrix.com/trac/wiki/TSF/FoundingSponsors>) who made this possible, as well as to all the other Twisted developers who helped out by writing or reviewing code.

Wednesday, July 2, 2008

Twisted in the News

Everyone's Hiring Twisted Talent

The jobs category made it to the top of the list this time around, with folks like Rackspace hiring Knowers in the Ways of Twist. This automation engineer position puts Twisted on par with .Net and C#, giving it title space. This one is looking for talent working with custom protocols and mobile device products. Fluidinfo is also looking for some Twisted talent; if you're interested, email me and I'll put you in touch with them.

The Rackspace job posting is interesting, because a reliable source said that they are doing some very sexy stuff with Twisted and cloud computing. This would be quite the catch, job hunters...

The Fredericksburg Twist

Zope Corp recently released zc.twist 1.3, "Talking to the ZODB in Twisted Reactor Calls." Their page on PyPI has extensive documentation and some great examples. With their Zope 3 work, Zope Corp has consistently provided some of the best technical documentation I've seen in the Python world (possibly fueled by their excellent use of doctests). This release makes me want to jump back into the ZODB :-)

Related to the ZODB, be sure to read Martijn Faassen's recent post.

Compliments

Terry Jones referred to the Twisted crew nicely in his blog post, saying that the framework is "an extraordinarily good set of asynchronous networking libraries written by a set of extraordinarily young and gifted programmers." Well, we're starting to get grey hairs, bald spots, and love handles... so perhaps we're not quite as young as we used to be ;-) But we'll take the compliment anyway, Terry ;-)

Holy Fervor

Colin Alston had some interesting things to say about "religious" technology debates and managed to nicely support Twisted and Divmod's Nevow at the same time. I'm saddened when people try to debate with us about the "right way" to do network programming (right now, everyone's asking how we compare to Erlang). This isn't a jihad, folks. Colin said it very well:
Why is it though that people become so fanatical about a particular framework or language? At the end of the day our choices of language and framework should be based on merit and not on emotion.
Colin also highlighted a few of the many reasons I like to use Twisted and Nevow:
Once you can break this mould, the endless possibilities of web applications become far more apparent. Twisted and Nevow gives you a clean web server tailored for your application, and you can keep your own state information and event control without needing the client to hit on special pages or abusing AJAX to poll resources.
Scaling

Next up is Glyph's absolutely fantastic essay on scaling with Twisted and Mantissa. JP and I have actually been talking to him about this stuff in great detail lately, due to the messaging work that Glyph is doing. It's a delight that Glyph took the time to put his thoughts and analysis on "paper." There's too much good stuff in there to quote it; you'll have to read the whole thing.

Intermediate to Advanced

Glyph gets the spotlight again with his response to a recent critique of Twisted by a programmer who prefers to write threaded applications. This is a must-read for intermediate to advanced Twisters.

In a similar vein, folks wanting to push their Twisted skills further and do some integration would do well to read this post, which got upmodded at reddit.com last week. It covers some basic advice on creating Twisted-ready async code.

More Blog Bits

JP sketched out a quick prototype of TCP in the browser using Nevow/Athena. The PyPy team has Nevow running on PyPy now! Prasanna Gautam has a quick example of sending binary data over XML-RPC using Twisted. Jack Moffitt of Chesspark fame blogged about their new chat product (based on Twisted) called speeqe. Jack gives us another node in his earlier Firefox GTD blog post. There was a post by Duncan covering some examples of "batching" deferreds in Twisted. Glyph posted an interesting comment on Steve Holden's blog. And finally, Jonathan Lange gives a shout out to the crew on his post about code reviews.