When I started working on P6SGI, I thought, "Hey, I'll just update PSGI to use
Perl 6, take advantage of some async data structures, and be done." That is
not how this process has gone down. First, I learned that I needed to know
more about Perl 6. Then, I found that I need to know more about HTTP/1.1 and
more about PSGI. Most recently, I have been researching HTTP/2, Mojolicious,
WebSockets, Akka, and a whole pile of other things.
So, here's the progress report on thing that have changed in the last week or so
on our way toward a complete P6SGI standard, which is still a ways off.
Standardizing the application life cycle. I count this as the most
significant change this week. An application server is now expected to call the
application as soon as the request headers have been read. That is, it should
not wait until the entire request body has been read from the
incoming socket. This makes it possible for applications to respond to Expect:
100-continue
headers. Furthermore, the application server should send the
response headers back at the earliest opportunity. Therefore, it is not
mandated, but strongly encouraged that application servers always run
concurrently with their applications.
Using Supply for all the things. The p6sgi.input
and p6sgi.errors
streams have been replaced with Supply objects. The input stream comes from the
server as a series of Blob objects while the application and middleware write
Str objects to the error stream.
Also the output stream has been expanded to allow Lists of Pairs so that
trailing headers or multiple-headers (as may occur handling 1xx responses) may
be sent. In PSGI, this was handled just by writing out the headers as part of
the content directly. This is a problem, however, if the application server
wants to implement HTTP/2 or WebSockets as they have special framing
requirements. This way the application needs to know a bit less about the
protocol and can focus on the content more.
Another consequence of input streaming is that buffering is now gone from the
spec. It will come be back as an extension of some kind to handle application
servers that are inherent buffered (like CGI) or those that provide it at the
server for convenience. Details TBD.
Additional promises have been added. I have specified new Promises to be
kept by the server. The p6sgi.ready
Promise is kept when the server has tapped
and is ready to receive the output, just in case the application wants to
deliver a live Supply for some reason. The p6sgix.header.done
Promise is kept
when the server has sent the headers and the p6sgix.body.done
Promise is kept
when the server has finished sending the body.
Extensions have been ported over from PSGI. I have ported the extensions of
PSGI over without much modification or thought. These will likely change. The
p6sgix.io
extension, in particular, is controversial, problematic, and might
not even be necessary. (Partly discussed below.)
Backpressure extensions have been added. I have added a set of
"backpressure" extensions (also providing a Supply) that allow the application
to monitor when the output socket is blocking. This allows an application that
is sending a long stream to a slow connection to pause processing while waiting
packets to clear the intertubes between peers. (Though, implementation of these
will wait until non-blocking I/O is implemented.)
Protocol header extension. I am actively thinking through the first draft of
this issue as I finish this post. One problem with PSGI is that is has a
strong assumption that your application only cares about some flavor of
HTTP/1.1. This is hardly likely in the modern web and a weakness of PSGI. PSGI
provides a basic extension, psgi.io
, that aims to allow the application to
implement more advanced subprotocols. This has at least two major problems:
The details of psgix.io
is implementation-specific. This means that an
application using it is tied to a specific implementation or has to provide
multiple implementations.
Why would you want protocol implementation details in the application? The
reason you use an application server is so that the server takes care of those
details on your behalf. The application should not be responsible for
implementation something like WebSocket or HTTP/2 or Comet or whatever.
To address this issue, I am working on a mechanism that would allow the
application server to specify additional protocols supported and then allow the
application to request that the server negotiate the handshake for these. So
far, this works for protocols like HTTP/2 or WebSocket, where the application
can detect a client request to upgrade a connection for such (by the presence of
an upgrade header). In those cases, the application could check an environment
variable to see which protocols the server supports. If the desired protocol is
supported, the application may tell the server to negotiate the handshake for
that server and take any other actions required by including a special header,
possible P6SGIx-Upgrade
, which specifies the desired upgrade.
It is my hope that we can encourage servers to implement specialized protocols
by providing upgrades like this or other forms of server-application
communication rather than relying on a kludge like psgix.io
.
In conclusion, it is my hope that more details of protocol handling can be kept
to the server and the applications can focus more directly on the details of
message content. It is my hope as well that through better asynchronous
communication and concurrent operation of application and application server, we
can resolve many of the weaknesses of PSGI and gain an overall more flexible and
useful standard for the modern web.
P.S. I will be at the Pittsburgh Perl Workshop in a couple weeks and giving a talk on the P6SGI process and where we're headed with it.