1. constructor attributes overriding same-named object methods, and
2. constructor attributes overriding object proxy methods

I asked, because this is the most complicated behavior of my OO code, and would be a key issue in porting the code to Moose. I also wanted to know if there would be other benefits to using Moose for this app.

Unlike my earlier OO-related posts, no comments appeared.

After some days, Aristotle replied that he believes my question was Warnock'ed (ignored) because the description lacked specifics.

Here is my attempt to provide them. Since I'm not sure exactly what to include and what to exclude, I'm documenting the process of generating the signal routing specification for a run of the Ecasound audio engine.

Say I have a project with three tracks: sax, piano, drums. I want to record only drums, a mono signal, from the soundcard channel 1. I set drums to REC. I set piano and sax to OFF. Something has to happen before I can press START.

Anatomy of a Track object

Here are relevant attributes of the Track object:

name: drums
class: Track
n: 3                   # unique integer index
group: Main            # bus affiliation
rw: REC                # could be REC, MON or OFF
source_id: 1           # input device ID, here the soundcard channel
source_type: soundcard # input device type
send_id: ~             # aux output device ID
send_type: ~           # aux output device type
width: 1               # number of channels in the input signal

Routing considerations

The 'drums' track belongs to Bus 'Main'. All tracks in Main automatically provide a signal to the 'Master' track. Master connects to the soundcard by default. It serves as a mixer, providing effects and a fader.

Each track has a raw (prefader) input signal as well as a "cooked" (postfader) signal that has undergone effects processing. Mono input signals get copied to allow for pan control and stereo monitoring.

To accomplish the routing and recording, we generate a file called a 'chain setup' that is used to configure Ecasound, the audio engine.

The desired output - an Ecasound chain setup

Here is the output we want:

# audio inputs

-a:1 -i:loop,Master_in
-a:3,R3 -i:alsa,default

# post-input processing

-a:3  -chcopy:1,2

# audio outputs

-a:1 -o:alsa,default
-a:3 -o:loop,Master_in
-a:R3 -f:s16_le,1,44100,i -o:/home/jroth/nama/test0329/.wav/drums_1.wav

In brief, each of the '-a' statements names one or more chains. Each chain has one input and one output (specified by -i and -o). Loop devices such as 'loop,Master_in' allow one chain to connect to another, one chain to have multiple inputs our outputs.

Track objects in Nama map to Ecasound chains. For the chain setup, we use Track indices rather than names as arguments to '-a'.

Generating the routing graph

To generate a chain setup, we first create a graph based on the following:

* the state of Tracks
* the state of Buses
* various global variables
* a specific user command

Here is the graph for this example:

  x     soundcard_in            # endpoint (signal input)
  x     |          | 
  x   drums     drums_rec_file  # Tracks
  x     |          |
  x     |        wav_out        # endpoint (signal output)  
  x     |      
  x   Master_in                 # endpoint (loop device)
  x     |
  x   Master                    # Track
  x     |
  x   soundcard_out             # endpoint (signal output)

Two observations:

We have added a temporary track 'drumsrecfile' (an alias to 'drums') to provide a chain for recording the signal.

We have inserted a loop device 'Master_in' to be able to connect 'drums' to 'Master'.

When we create the graph, we may supply attributes to the nodes or edges to control behavior of the back end.

For example, the chain_id for the edge (chain) corresponding the temporary track 'drums_rec_file' is set to 'R3', overriding the default value (which would be the track index).

As another example, when a Bus routes a signal, it sets the 'send_type' and 'send_id' attributes in the graph because it doesn't care about the track's own values for these attributes.

Processing the Graph

Now we are ready to process our graph. We want to use each edge of the graph to generate the lines of our chain setup.

We start with a dispatch() routine that creates an IO object for each edge of the graph. The object then generates either an input clause (-i) or an output clause (-o).

The following output lists the class of each object generated from the above graph, and the corresponding attributes that dispatch() passes to the object constructor.

Class: IO::from_soundcard_device
---
chain_id: 3
endpoint: soundcard_in
track: drums

Class: IO::to_loop
---
chain_id: 3
device_id: loop,Master_in
endpoint: Master_in
track: drums

Class: IO::from_loop
---
chain_id: 1
device_id: loop,Master_in
endpoint: Master_in
track: Master

Class: IO::to_soundcard_device
---
chain_id: 1
endpoint: soundcard_out
track: Master

Class: IO::from_soundcard_device
---
chain_id: R3
endpoint: soundcard_in
mono_to_stereo: ''
track: drums_rec_file

Class: IO::to_wav
---
chain_id: R3
endpoint: wav_out
mono_to_stereo: ''
track: drums_rec_file

Discussion:

The 'endpoint' maps to the class, and the class provides methods to generate the output.

The track name is provided to enable the class to find other attributes, such as signal width, or the name of the output file.

Generally the input classes convert a mono signal to stereo; we need to suppress behavior when we are recording a signal. Hence we pass a 'mono_to_stereo' attribute that overrides the output of eponymous object method.

As I write this, I can see that some override complexity could be reduced by specifying whether the signal is a 'raw' (prefader) signal or 'cooked' (postfader) signal.

There is flexibility from being able to pass attributes that override the result of any object method. I could potentially modify the methods to provide this behavior rather than use AUTOLOAD.

I am told that finding my own solutions to get the behavior I want may lead to poor (dirty, unreliable) code when a better solution could come through a framework such as Moose.

I rolled my own perl OO for Nama, a multitrack audio recording app I've written. However, I'm ready to consider introducing an OO framework for the sake of learning, code quality, testing and maintenance. Provided that I can find satisfactory ways to solve my problems.

Here's a laundry list of (just two) OO features that I'd need or want to implement in a Mooselike framework. One is short and simple to describe, the other will take several paragraphs.

Inheriting field definitions

Track objects in Nama have 34 attributes, and are the biggest class by far. Track subclasses serve special purposes that somewhat evident in their names: SimpleTrack, CacheRecTrack, MixDownTrack, EditTrack, VersionTrack, MixTrack. I've found it convenient to write the base class so Track subclasses inherit attribute names from ancestor classes.

Update: I see this is default behavior for Moose

Object proxy and attribute overriding

I'd read about AUTOLOAD for years, and wondered what the fuss was about. With my home-grown OO, I found I needed to use it.

Here is what I did.

A lot of the complexity in multitrack audio production, relates to routing, that is, creating signal processing networks. Signals travel

1. from sources such as audio files, soundcards or programs
2. through various components such as tracks and buses,
3. into sinks such as audio files, soundcards or programs

Each recording or playback run requires an appropriate network. Nama doesn't perform audio processing itself; it invokes the Ecasound audio engine, expressing the signal-processing network as an Ecasound chain setup file.

Chain setups are made up of one or more signal chains. Each chain has one input and one output. Intermediate nodes called loop devices allow for summing or branching. Whenever needed or commanded by the user, Nama iterates over all tracks and generates a routing graph. The graph is implemented as a Graph object whose nodes and edges may contain attributes.

Nama traverses this graph to generate a set of IO objects, each corresponding to the input or output clause of an Ecasound chain. IO subclasses correspond to each variety of input or output. To give one example, an IO::wav_out object is used for writing a signal to a WAV file. It creates a line such as:

 -a:2 -f:s16_le,2,44100 -o:Mixdown_2.wav

where "2" is the chain's ID, "s16_le,2,44100" is the signal format, and "Mixdown_2.wav" is the output file.

Iterating over the IO objects generates the entire Ecasound chain setup.

Generally attributes of the Graph node or edge will override the methods of the corresponding IO object, but in some cases methods have priority.

If an IO object doesn't have a particular attribute that one of its methods needs, it looks to the corresponding Track object (proxy relationship.)

AUTOLOAD implementation described

The way I currently implement this is as follows: When constructing IO objects, each key gets an appended underscore ("chain_id" becomes "chain_id_".)

Within the class definition, methods can be prefixed by an underscore to allow them to be overridden.

package IO::wav_out;
sub _format_template { $raw_to_disk_format }

When resolving $io_object->format_template, since format_template is not defined, the call is picked up by the AUTOLOAD subroutine.

AUTOLOAD returns the object attribute $io_object->{format_template_} if that key exists.

AUTOLOAD returns $io_object->_format_template if that method exists.

AUTOLOAD returns $track->format_template if $track has that method.

Otherwise AUTOLOAD throws an exception.

If the object has defined a method "format_template" (no underscore) the call is resolved immediately. AUTOLOAD is never invoked, giving the method definition the highest priority.

Discussion

While this was an achievement to figure out and to debug, I arrived at a clear design that meets my requirements.

I cannot say whether my solution is sufficiently general, sufficiently robust, or represents appropriate use of an appropriate hammer.

That is one reason I am writing this! I would like to know if I'm missing a better solution in the OO framework world.

Code

Here is code for the base class (with a few cosmetic changes)

package IO;

our $AUTOLOAD;


# The template code below gets a comment-stripped list
# of attributes from the source file "io_fields" and appends
# an underscore to each key

use Object qw([% join " ",map{$_."_" }split " ", qx(./strip_all ./io_fields) %]);

# IO objects for writing Ecasound chain setup file
#
# Object attributes can come from three sources:
#
# 1. As arguments to the constructor new() while walking the
#    routing graph:
#      + assigned by dispatch: chain_id, loop_id, track, etc.
#      + override by graph node (higher priority)
#      + override by graph edge (highest priority)
# 2. methods called as $object->method_name
#      + defined as _method_name (access via AUTOLOAD, overrideable by constructor)
#      + defined as method_name  (not overrideable)
# 3. AUTOLOAD
#      + any other method calls are passed to the the associated track
#      + illegal track method call generates an exception

sub new {
    my $class = shift;
    my %vals = @_;
    my @args = map{$_."_", $vals{$_}} keys %vals; # add underscore to key

    # note that we won't check for illegal fields
    # to allow for AUTOLOAD resolution

    bless {@args}, $class
}
sub AUTOLOAD {
    my $self = shift;
    # get last part of method call
    my ($call) = $AUTOLOAD =~ /([^:]+)$/;
    my $field = "$call\_";
    my $method = "_$call";
    return $self->{$field} if exists $self->{$field};
    return $self->$method if $self->can($method);
    if ( my $track = $Track::by_name{$self->{track_}} ){
        return $track->$call if $track->can($call)
        # ->can is reliable here because Track has no AUTOLOAD
    }
    print $self->dump;
    croak "Autoload fell through. Object type: ", (ref $self), ", illegal method call: $call\n";
}

He says he won't include even one example because "it really doesn't send the right message. My revisions to perlobj will explain how to implement accessors using the object as a hash reference."

I was shocked, because I find I need to visualize my object's data structures. Also, in several cases, reading or writing attributes directly allows me to implement some needed functionality.

I gave one example of this in a previous post..

Dave suggested the reason I needed to break encapsulation this way is that

1. I haven't designed my objects properly or

2. I haven't taken advantage of a full OO framework.

He concludes that I'm using the same coarse hammer to solve all my programming problems, because I'm simply not aware of better solutions.

Those are interesting criticisms.

The second point is absolutely right: I began my OO experiment with Object::Tiny, adding a setter and other facilities as I needed them. In fact, I am proud of this incremental approach, admittedly chosen for my own learning.

Regarding the first point, "design smarts" for me is an extremely limited resource. So I substitute an iterative process of problem-solving and refactoring.

Is it acceptable to use standard Perl tools for this, or should I be reaching higher, tapping more advanced tools? Suppose I had used Moose. Could I have solved my problems easier, more elegantly, with less stumbling?

Well, for one I would have had to ask for help in the Moose community, whereas with hand-rolled OO, I can understand the entire codebase, even if it doesn't meet certain standards of tidiness, design or even hygiene.

On the other hand, I do want my code to be beautiful, for various aesthic reasons, for ease of testing and maintenance, and also to have a code base that will be sufficiently attractive to other people to hack on.

With the aim of exploring whether I can achieve something better using Moose, Mouse, Moo or other advanced framework, in my next post, I'll give an example of some of my more advanced needs, and the naive, if not neolithic ways I implemented them.

$self->{foo} = "Look Ma! No encapsulation"; # don't do this, they say

In general that is true, but not always, We should avoid absolutist language, especially in tutorials. I'll come to an example of that.

For Nama, I generally use a setter with this syntax:

$self->set(foo => "bar");

The set method (inherited from a parent class based on Object::Tiny) makes sure the key is a legal one.

Because it looks distinct, I'm not likely to use it unless I want really write access to that attribute.

This simple approach allows me to manage object attributes culturally, i.e. without specifying them as read-only or read-write. In an app of 13k lines, the 'set' method appears just 110 times.

But it's still possible to directly modify an object in other ways:

my $self = Object->new( foo => [qw(this is a pen)] );
my $array_ref = $self->foo;
$array_ref->[3] = 'lobster'; 
print $self->as_string # "this is a lobster"

I am living with that.

The other point is that I believe there can be legitimate reasons to violate encapsulation. I recently found a good example. Nama allows you to create a new track object that refers to WAV files with a basename other than the track's own name.

For an ordinary track, the name matches the WAV file:

my $Mixdown = Track->new( name => 'Mixdown'); # associates with Mixdown.wav

Here is a track created by the 'link_track' command, that also associates with Mixdown.wav.

my $song = Track->new( name => 'song', target => 'Mixdown');

Here is a track created by the 'new_region' command, that indirectly associates with the same file.

my $final_song = Track->new( name => 'final_song', target => 'song' )

Here is the code I use so that $final_song->target returns 'Mixdown':

package Track;

sub target {
    my $self = shift;
    my $parent = $Track::by_index{$self->{target}};
    defined $parent && $parent->target || $self->{target};
}

Now there could be some discussion about my using the track name (as opposed to a Track object) as the value for the 'target' field. In short, this design decision has made it easy to serialize and to debug by dumping objects as YAML.

Regarding encapsulation, the point is that accessing $self->{target} is essential for this code to work.

Accessing the underlying hash provides the behavior I want.

I think it's simplistic to assume that every new user could or should find other ways to achieve this behavior without violating encapsulation.

Our tutorials should respect readers' intelligence. I think we can tell them what is good practice, without prescribing an idealistic straitjacket.

Even if I'm missing an easy solution that doesn't access the hash, why should I be required to find one?

For example, I could do it this way:

package Track;

sub _target{ $_[0]->{target} }

sub target {
    my $self = shift;
    my $parent = $Track::by_index{$self->_target};
    defined $parent && $parent->target || $self->_target;
}

But the code is no clearer and the first line still violates encapsulation.

I can see the importance of respecting encapsulation in large projects using others' OO libraries, but not all use cases fall into that category.

I haven't seen much about the step-by-step process of dividing and modularizing code. Here how I am approching it.

The app already uses OO, but suffers from some 260 global variables and about 6k lines of code in the main namespace.

I broke that code into about 30 different modules, testing after creating each new module, with the help of git for source control, and a couple scripts showing which files and in which subs each variable appears

Most modules still occupy the same namespace, however have access to only the minimum necessary subset of global variables. I did this using a structure like this:

[Audio_engine_setup.pm]

 package main;
 our ( $sampling_frequency,.... )
 sub configure_freq { say "configuring soundcard at $sampling_frequency" }

All subroutines are still in the main namespace, so they work unchanged.

The 'our' declaration tells me exactly what global variables the module touches. So I can look at them and think: Do they need to be in the main namespace? Can they be lexicalized? Can they be passed as arguments?

A next step is to provide more separation by introducing a separate namespace.

  package main;
  our ( $sampling_frequency,....)
  package Audio_engine_setup;
  sub report_freq { say "soundcard is running at $sampling_frequency" }

These subroutines (and those in the main namespace they invoke) will have to be called with the full package affiliation, or be imported into the desired namespace via Exporter and 'use', or be converted to use some kind of OO.

Physically reorganizing the code provides an opportunity to partition functions, to define interfaces and to review provisional design decisions made years ago. Of course greater separation of concerns helps the software to be robust and to be readable. And having the code divided up helps anyone who wants to browse/hack.

How much separation is desirable practical is an open question. Most global variables are indices or configuration variables that are set once and read-accessed. Yes, anywhere it's possible to set them by accident. On the other hand, this is a single app, not a toolkit that will be (ab)used by other programs in an adverse environment.