Zoffix Znet

This document is the February, 2017 progress report for TPF Standardization, Test Coverage, and Documentation of Perl 6 I/O Routines grant

Timing

I'm currently running slightly behind the schedule outlined in the grant. I expect to complete the Action Plan and have it ratified by other core members by March 18th, which is the date of the 2017.03 compiler release. Then, I'll implement all of the Action Plan (and complete the grant) by the 2017.04 compiler release on April 15th. This is also the release the next Rakudo Star distribution will be based on, and so the regular end users will receive better IO there and then.

Some members of the Core Team voiced concerns over implementing any changes that can break users' code, even if the changes do not break 6.c-errata specification tests. Once the full set of changes is known, they will be reviewed on a case-by-case basis, and some of them may be implemented under 6.d.PREVIEW pragma, to be included in 6.d language version, leaving 6.c language versions untouched. Note that changes that are decided to be 6.d material may delay the completion of this grant due to not fully-fleshed out infrastructure for supporting multiple language versions. The April 15th deadline stated above applies only to changes to 6.c language and new deadline will be ascertained for completion of the 6.d changes.

User Communications

I wrote and disseminated advanced notice of the changes to be made due to this grant, to prepare the users to expect some code to break (some routines were found to be documented, despite being absent entirely from the Specification and not officially part of the language).

The notice can be seen at: http://rakudo.org/2017/02/26/advance-notice-of-significant-changes/

It is possible the Core Team will decide to defer all breaking changes to 6.d language version, to be currently implemented under v6.d.PREVIEW pragma.

Bonus Deliverable

The bonus deliverable—The Map of Perl 6 Routines—is now usable. The code is available in perl6/routine-map repository, and the rendered version is available on map.perl6.party. Its current state is sufficient to serve the intended purpose for this grant, but I'll certainly add improvements to it sometime in the future, such as linking to docs, linking to routines' source code, having an IRC bot looking stuff up in it, etc.

It'll also be fairy easy to use the Map to detect undocumented routines or ones that are documented under the incorrect type.

Identified Issues/Deficiencies with IO Routines

These points, issues, and ideas were identified this month and will be included for consideration in the Action Plan.

• Calling practically any method on a closed IO::Handle results in an LTA (Less Than Awesome) error message that reads <something> requires an object with REPR MVMOSHandle where <something> is sometimes the name of the method called by the user and others is some internal method invoked indirectly. We need better errors for closed file handles; and not something that would require a is-fh-closed() type of conditional called in all the methods, which would be a hefty performance hit.
• Several routines have been identified which in other languages return useful information: number of bytes actually written or current file position, whereas in Perl 6 they just return a Bool (.print, .say, .write) or a Mu type object (.seek). Inconsistently, .printf does appear to return the number of bytes written. It should be possible to make other routines similarly useful, although I suspect some of it may have to wait until 6.d language release.
• The .seek routine takes the seek location as one of three Enum values. Not only are they quite lengthy to type, they're globally available for no good reason and .seek is virtually unique in using this calling convention. I will seek to standardize this routine to take mutually-exclusive named arguments instead, preferably with much shorter names, but those are yet to be bikeshed.
• IO.umask routine simply shells out to umask. This fails terribly on OSes that don't have that command, especially since the code still tries to decode the received input as an octal string, even after the failure. Needs improvement.
• link's implementation and documentation confuses what a "target" is. Luckily (or sadly?) there are exactly zero tests for this routine in the Perl 6 Specification, so we can change it to match the behaviour of ln Linux command and the foo $existing-thing, $new-thing argument order of move, rename, and other similar routines.
• When using run(:out, 'some-non-existant-command').out.slurp-rest it will silently succeed and return an empty string. If possible, this should be changed to return the failure or throw at some point.
• chdir's :test parameter for directory permissions test is taken as a single string parameter. This makes it extremely easy to mistakenly write broken code: for example, "/".IO.chdir: "tmp", :test<r w> succeeds, while "/".IO.chdir: "tmp", :test<w r> fails with a misleading error message saying the directory is not readable/writable. I will propose for :test parameter to be deprecated in favour of using multiple named arguments to indicate desired tests. By extension, similar change will be applied to indir, tmpdir, and homedir routines (if they remain in the language).
• Documentation: several inaccuracies in the documentation were found. I won't be identifying these in my reports/Action Plan, but will simply ensure the documentation matches the implementation once the Action Plan is fully implemented.

Discovered Bugs

The hunt for 6-legged friends has these finds so far:

Will (attempt to) fix as part of the grant

• indir() has a race condition where the actual dir it runs in ends up being wrong. Using indir '/tmp/useless', { qx/rm -fr */ } in one thread and backing up your precious data in another has the potential to result in some spectacular failurage.
• perl6 -ne '@ = lines' crashes after first iteration, crying about MVMOSHandle REPR. I suspect the code is failing to follow iterator protocol somewhere and is attempting to read on an already closed handle. I expect to be able to resolve this and the related RT#128047 as part of the grant.
• .tell incorrectly always returns 0 on files opened in append mode
• link mixes up target and link name in its error message

Don't think I will be able to fix these as part of the grant

• seek() with SeekFromCurrent as location fails to seek correctly if called after .readchars, but only on MoarVM. This appears to occur due to some sort of buffering. I filed this as RT#130843.
• On JVM, .readchars incorrectly assumes all chars are 2 bytes long. This appears to be just a naive substitute for nqp::readcharsfh op. I filed this as RT#130840.

Already Fixed

• While making the Routine Map, I discovered .WHICH and .Str methods on IO::Special were only methods defined only for the :D subtype, resulting in a crash when using, say, infix:<eqv> operator on the type object, instead Mu.WHICH/.Str candidates getting invoked. This bug was easy and I already commited fix in radudo/dd4dfb14d3 and tests to cover it in roast/63370fe054

Auxiliary Bugs

While doing the work for this grant, I also discovered some non-IO related bugs (some of which I fixed):

• .Bool, .so, .not, .hash, and .elems on Baggy:U crash (fixed in e8af8553)
• .sort on reified empty arrays crashes (reported as RT#130866)
• SEGV with Scalar type object in unique() (reported as RT#130852)
• .Bool, .so, .not and possibly others crash on Seq:U; need to evaluate entire codebase to see where only methods are used instead of multi, preventing dispatch to Mu.* candidates. (reported as RT#130867 )
• Incorrect line number reported for wrong routine call when unpacking/heredocs are used (reported as RT#130862)
• Warnings produced by core code marked as "shouldn't happen" (reported as RT#130857)