Ed J

If you do scientific and/or numerical computing, you probably use or at least know about BLAS, and LAPACK. If you've done much programming with them, you may have passed in data that it cannot handle, and then you'll probably know about xerbla_.

xerbla_ is the error-handler in those two libraries, and by default it terminates the whole process. To do otherwise you have to provide a replacement routine, and have the linker for your program insert that replacement, which the library will then use instead. This is increasingly hard or impossible to do, including on macOS, Windows, and static libraries. There is now an open pull-request in the reference versions of those libraries that will permanently solve this, by allowing client programs to just pass in a new handler, which will then be used. No linker stuff involved.

Implementing this involved figuring out how to achieve this in Fortran. As a test-bed, I made a directory with this simple, no-nonsense Makefile:

FC = gfortran
FFLAGS = -g

.f90.o:
        $(FC) $(FFLAGS) -c $< -o $@
.f90:
        $(FC) $(FFLAGS) $< -o $@ $(LDFLAGS)
.SUFFIXES:
.SUFFIXES: .f90 .o

all: callbacks

The first try was using somewhat modern Fortran, with a module:

module xerbla_callbacks
  implicit none
  private
  public :: active_callback, xerbla_interface
  procedure(xerbla_interface), pointer :: active_callback => null()
  interface
    subroutine xerbla_interface(srname, info)
      character*(*), intent(in) :: srname
      integer, intent(in) :: info
    end subroutine
  end interface
end module xerbla_callbacks

subroutine set_xerbla(cb)
  use xerbla_callbacks
  implicit none
  procedure() :: cb
  active_callback => cb
end subroutine set_xerbla

subroutine get_xerbla(cb_ret)
  use xerbla_callbacks
  procedure(xerbla_interface), pointer :: cb_ret
  cb_ret => active_callback
end subroutine get_xerbla

subroutine xerbla(srname, info)
  use xerbla_callbacks
  character*(*), intent(in) :: srname
  integer, intent(in) :: info
  if (associated(active_callback)) then
    call active_callback(srname, info)
  else
    print *, 'I am the main xerbla'
  end if
end subroutine

subroutine xerbla_replacement(srname, info)
  character*(*), intent(in) :: srname
  integer, intent(in) :: info
  print *, 'I am the replacement xerbla'
end subroutine

program hello
  use xerbla_callbacks
  implicit none
  external xerbla_replacement, set_xerbla, xerbla
  procedure(), pointer :: already_cb
  interface
    subroutine get_xerbla(cb_ret)
      procedure(), pointer :: cb_ret
    end subroutine
  end interface
  call xerbla('Hello, World!', 5)
  call get_xerbla(already_cb)
  call set_xerbla(xerbla_replacement)
  call xerbla('Hello, World!', 5)
  call set_xerbla(already_cb)
  call xerbla('Hello, World!', 5)
end program hello

This didn't work in LAPACK because it has an automatic translation facility to turn its source files into versions that suffix routines with _64 that cannot handle modules well. Using an older idiom that allows stored, persistent data was necessary, using the ENTRY keyword.

subroutine xerbla(srname, info)
  character*(*), intent(in) :: srname
  integer, intent(in) :: info
  procedure(xerbla_interface), pointer :: active_callback => null(), cb_ret
  procedure(xerbla_interface) :: cb
  abstract interface
    subroutine xerbla_interface(srname, info)
      character*(*), intent(in) :: srname
      integer, intent(in) :: info
    end subroutine
  end interface
  if (associated(active_callback)) then
    call active_callback(srname, info)
  else
    print *, 'I am the main xerbla'
  end if
  return
  entry set_xerbla(cb)
    active_callback => cb
  return
  entry get_xerbla(cb_ret)
    cb_ret => active_callback
  return
end subroutine

subroutine xerbla_replacement(srname, info)
  character*(*), intent(in) :: srname
  integer, intent(in) :: info
  print *, 'I am the replacement xerbla'
end subroutine

program hello
  implicit none
  external xerbla_replacement, set_xerbla, xerbla
  procedure(), pointer :: already_cb
  interface
    subroutine get_xerbla(cb_ret)
      procedure(), pointer :: cb_ret
    end subroutine
  end interface
  call xerbla('Hello, World!', 5)
  call get_xerbla(already_cb)
  call set_xerbla(xerbla_replacement)
  call xerbla('Hello, World!', 5)
  call set_xerbla(already_cb)
  call xerbla('Hello, World!', 5)
end program hello

Amazingly, this actually all works, and when the C interfaces for these things are figured out, this facility will become available. That will solve a lot of tricky problems, especially for those who make dynamic-language bindings for BLAS etc, but also for those using it on macOS, Windows, etc.

There have been a couple of developments in PDL since the last announcement on here I could find, from 2013. To hypersummarise: 64-bit indexing, native complex number support, automatic pthreading using all available CPU cores, faster installation thanks to parallel-building, memory-mapped data, repository hosted on GitHub, easy to use "with" Inline. Returning you to the announcement:

PDL 2.063_01 has just been released. Notable changes since 2.062:

• Various API changes (see below)
  
• Improvements to $MACRONAME() handling including that arguments …

GraphQL is the new, shiny way to do APIs. It minimises number of round-trips for clients to query what need. But what about real-time updates? How can we cut down the time needed for clients to get new information? Are we forcing them to constantly poll? That seems expensive and also slow.

GraphQL's official standard now includes "subscriptions". The obvious transport for that is WebSockets, and the de facto standard for that is Apollo GraphQL's subscriptions-transport-ws.

Hopefully, the sesquipedalian polysyllabalisation of the title will have made your eyes glaze over. Now to wake you up: MASSIVE PERFORMANCE GAINZ!

Memoising, as any fule kno, is storing answers after they're calculated, against each set of inputs, so you don't have to keep re-calculating the same thing. If the calculating process is expensive, or even just more expensive than normalising (see below) and a lookup, this can make your programs run faster. Possibly much faster!

This is a thing that works great for computing factorials or Fibonacci numbers. It's a terrible idea for calculating the lengths of simple lists, since normalising would take longer than simply recalculating, and worse would be very unlikely to give repeated hits.

A hybrid answer for that case would be to only store the lengths, but that would still not add value in that specific case because if the normalised input is the answer, this guarantees that normalise+lookup will be slower than normalise=calculate. It does however give a hint that calculating(ha!) whether to normalise or not depends in part on the relationship between normalising and calculating, together with the likelihood of cache hits in that domain.

The memoising process works like so:

• normalise the inputs: getting the right thing to store answers against
• look to see if an answer for those inputs exists, and return it if so
• calculate the answer
• store that
• return it

The problem here, such as it is, is that while the Perl module obviously has a facility for normalising inputs, I would say it's most orientated towards functions, rather than methods. My claim is that what's needed here is a convention, or even just a module (probably based heavily on Attribute::Memoize, to make this easy. It would probably just default to a method in the existing package called normalise.

A remaining problem would be what I am currently thinking of as "partial memoisation". This is inspired by considering SQL::Abstract's select method. This call:

$sql->select('tickets', '*', { id => 3 });

returns:

("SELECT * FROM tickets WHERE ( id = ? )", 3)

There is no absolutely general way of knowing from the outside that one should normalise based on the keys of the third parameter, then include the memoised output (here, the SQL) as part of the returns, together with the remaining stuff. Here, one would:

• split the internals of the SQL-generator out into a memoisable method
• make a _fractionate_where method that could be used by the above, plus the public select as below, to return in suitable order array-refs with the normalisable keys, and the values
• have the normalise method use its knowledge of the relevant object state (e.g. quote_char) plus the "fractionated" keys of \%where to produce a normalised input
• lookup / calculate the SQL using normal memoisation as already discussed
• have the select method use that, then return the SQL plus the "fractionated" values

"Someone should" (which means I will) make an Attribute::Memoize::Defaults (or a better name - weigh in!) that implements the above, then pull-request its use onto SQL::Abstract.

Perl community, your thoughts are welcome!

Version 0.01 of JSON::Transform is now on CPAN. It lets you express transformations of JSON-able data (i.e. data that is only hashes, arrays, simple scalars plus booleans) concisely and declaratively, without writing any other code.

Locations within the data structure are expressed using JSON Pointer (RFC 6901), and there are both user and automatically-system-set variables available that can be interpolated to make such locations be computed.

What is this for?

Be…