When I started working on P6SGI, I thought, "Hey, I'll just update PSGI to use Perl 6, take advantage of some async data structures, and be done." That is not how this process has gone down. First, I learned that I needed to know more about Perl 6. Then, I found that I need to know more about HTTP/1.1 and more about PSGI. Most recently, I have been researching HTTP/2, Mojolicious, WebSockets, Akka, and a whole pile of other things.

So, here's the progress report on thing that have changed in the last week or so on our way toward a complete P6SGI standard, which is still a ways off.

Standardizing the application life cycle. I count this as the most significant change this week. An application server is now expected to call the application as soon as the request headers have been read. That is, it should not wait until the entire request body has been read from the incoming socket. This makes it possible for applications to respond to Expect: 100-continue headers. Furthermore, the application server should send the response headers back at the earliest opportunity. Therefore, it is not mandated, but strongly encouraged that application servers always run concurrently with their applications.

Using Supply for all the things. The p6sgi.input and p6sgi.errors streams have been replaced with Supply objects. The input stream comes from the server as a series of Blob objects while the application and middleware write Str objects to the error stream.

Also the output stream has been expanded to allow Lists of Pairs so that trailing headers or multiple-headers (as may occur handling 1xx responses) may be sent. In PSGI, this was handled just by writing out the headers as part of the content directly. This is a problem, however, if the application server wants to implement HTTP/2 or WebSockets as they have special framing requirements. This way the application needs to know a bit less about the protocol and can focus on the content more.

Another consequence of input streaming is that buffering is now gone from the spec. It will come be back as an extension of some kind to handle application servers that are inherent buffered (like CGI) or those that provide it at the server for convenience. Details TBD.

Additional promises have been added. I have specified new Promises to be kept by the server. The p6sgi.ready Promise is kept when the server has tapped and is ready to receive the output, just in case the application wants to deliver a live Supply for some reason. The p6sgix.header.done Promise is kept when the server has sent the headers and the p6sgix.body.done Promise is kept when the server has finished sending the body.

Extensions have been ported over from PSGI. I have ported the extensions of PSGI over without much modification or thought. These will likely change. The p6sgix.io extension, in particular, is controversial, problematic, and might not even be necessary. (Partly discussed below.)

Backpressure extensions have been added. I have added a set of "backpressure" extensions (also providing a Supply) that allow the application to monitor when the output socket is blocking. This allows an application that is sending a long stream to a slow connection to pause processing while waiting packets to clear the intertubes between peers. (Though, implementation of these will wait until non-blocking I/O is implemented.)

Protocol header extension. I am actively thinking through the first draft of this issue as I finish this post. One problem with PSGI is that is has a strong assumption that your application only cares about some flavor of HTTP/1.1. This is hardly likely in the modern web and a weakness of PSGI. PSGI provides a basic extension, psgi.io, that aims to allow the application to implement more advanced subprotocols. This has at least two major problems:

1. The details of psgix.io is implementation-specific. This means that an application using it is tied to a specific implementation or has to provide multiple implementations.

2. Why would you want protocol implementation details in the application? The reason you use an application server is so that the server takes care of those details on your behalf. The application should not be responsible for implementation something like WebSocket or HTTP/2 or Comet or whatever.

To address this issue, I am working on a mechanism that would allow the application server to specify additional protocols supported and then allow the application to request that the server negotiate the handshake for these. So far, this works for protocols like HTTP/2 or WebSocket, where the application can detect a client request to upgrade a connection for such (by the presence of an upgrade header). In those cases, the application could check an environment variable to see which protocols the server supports. If the desired protocol is supported, the application may tell the server to negotiate the handshake for that server and take any other actions required by including a special header, possible P6SGIx-Upgrade, which specifies the desired upgrade.

It is my hope that we can encourage servers to implement specialized protocols by providing upgrades like this or other forms of server-application communication rather than relying on a kludge like psgix.io.

In conclusion, it is my hope that more details of protocol handling can be kept to the server and the applications can focus more directly on the details of message content. It is my hope as well that through better asynchronous communication and concurrent operation of application and application server, we can resolve many of the weaknesses of PSGI and gain an overall more flexible and useful standard for the modern web.

P.S. I will be at the Pittsburgh Perl Workshop in a couple weeks and giving a talk on the P6SGI process and where we're headed with it.

So my last couple of posts have brought up some grousing from people who do not like PSGI in Perl 5. I am sorry to say that some of this grousing against P6SGI has made some assumptions based on PSGI and not based on P6SGI. I am now going to try to dispel some of these myths.

• P6SGI is a direct port of PSGI to Perl 6. If you have started with that assumption, you are wrong. P6SGI is fundamentally different from PSGI in significant ways, mostly in that it is completely asynchronous from the start: literally, your application must be wrapped in a start-block, which joins a thread.

• P6SGI will not do X. This is an early spec. Anything can change right now. Nothing is bolted down for at least 3 to 6 months and nothing is final until 12 to 18 months from now. There is time to make it right. Come contribute. If you want to a commit bit to make your changes, I'm handing them out if you ask nicely.

• There is no need for P6SGI/the approach is dead. I beg to differ. I want a P6SGI implementation for myself. Therefore, even if I end up being the only one, there is at least one user that wants it. I am hoping that by engaging the community, I can find other users interested and other use-cases to support.

• P6SGI is just re-hashing 15 year old technology. I would rather say that it is improving upon 15 year old technology. P6SGI is asynchronous while most of it's predecessors are not. It will provide tools for streaming input in and out asynchronously without requiring raw access to the socket connection (not totally spec'd, but it will be soon). This is not your father's web gateway.

Seriously folks. We have history with PSGI and WSGI and whatever, but we're not stopping there. Let's get it right this time so that Perl 6 has the implementation everyone is copying for the next 15 years.

The P6SGI standard is progressing reasonably well now. There are a number of issues yet to be worked out it is a reasonably good start. However, before we can really be sure of that, we need an implementation that puts the standard to use and helps us find the warts as well as provides a way to get started working with it.

Introducing: Smack.

Perl has Plack. Perl 6 has Smack. So far, the basics of it's own standalone server are built and working and CGI is started. Some of the built-in apps and some standard middleware have been drafted, but there is a large amount of work to be done.

I have a regular job, 3 boys, and limited spare time. If you have any interest in helping work on the next generation of Perl application servers, your help would be most welcome.

Cheers.