NetAddr::IP 4.080_08: another trial, please smoke test

NetAddr::IP 4.080_08 is on CPAN as a trial release. If you use NetAddr::IP, install it and run your code against it before 4.080 goes stable.

The last post here covered 4.080_05. Since then, 4.080_06, 4.080_07 and 4.080_08 changed constructor, parsing, comparison and method behaviour, and they added new4() and NetAddr::IP::Count. In almost every case, the behaviour changes handle bad or ambiguous input better: they reject it, return undef, or croak with a clear error instead of producing a wrong address. The 4.080 series is also relicensed to the MIT license. This post lists the API changes you may need to check in your code.

For every detail, see the Changes file on MetaCPAN.

New API

  • new4() (GH#237, 4.080_07). Reads input like new() and returns the object only when the result is IPv4. Use it when your code should accept IPv4 input (including IPv4-mapped text that carries an IPv4 address) and reject IPv6 input. IPv4-mapped text with a prefix of 96 bits or longer returns the IPv4 address it carries; other IPv6 results return undef. It can rebuild an IPv4 object from numeric() when the mask is above 128; a numeric mask of 0 is still prefix length 0.
    use NetAddr::IP;
    
    my $ip     = NetAddr::IP->new4('192.0.2.1');            # 192.0.2.1/32
    my $mapped = NetAddr::IP->new4('::ffff:192.0.2.1');  # 192.0.2.1/32
    my $v6     = NetAddr::IP->new4('2001:db8::1');          # undef
  • NetAddr::IP::Count (GH#147, 4.080_07). A new implementation of the interface of NetAddr-IP-Count 2.01 by Luis E. Muñoz (valcount contributed by Alvaro Carvajal). An address now counts once in every group whose subnet contains it. Bad argument types croak before any total changes. The cache size is still accepted and reported, but it no longer affects counting.

Breaking changes and deprecations

Almost every breaking change below corrects behaviour found through fuzzed input testing. In most cases the fix is about bad input: input that used to parse wrongly, silently change meaning, or produce an invalid object is now rejected or handled in a safer, clearer way.

  • split, splitref, rsplit and rsplitref croak on an unaligned plan (GH#259, 4.080_08). A plan that places a piece at an address not aligned to that piece's size now croaks. For example, split(30, 29) on 192.0.2.0/28 used to produce 192.0.2.4/29, which is not a /29 network. Plans ordered from largest to smallest still work.
  • Importing :old_nth or :old_storable warns (deprecated) (GH#261, 4.080_08). The tags still do what they did, but the warning is in the deprecated category. Code that runs with fatal warnings will die at import. $NetAddr::IP::Lite::Old_nth is deprecated but does not warn. Both tags are planned for removal in version 5.
  • fake_AF_INET6 is deprecated (GH#165, 4.080_07). It returns 0 and will be removed in version 5. AF_INET6 and inet_ntop come from core Socket; IPv6 names go through getaddrinfo. Socket6 is no longer recommended.
  • IPv4 text parsing is stricter (4.080_06). No leading zero is accepted in any IPv4 part, and the whole string is read as the address in the InetBase and Util text functions. Trailing newlines and some whitespace forms are rejected where they were previously accepted (\z replaces $ in the relevant patterns); inet_aton and naip_gethostbyname return undef for text with whitespace without a resolver lookup. Constructors still strip surrounding whitespace from the address and now also strip it from a separate mask. Under :aton, ends are kept only for 4-byte or 16-byte strings. Prefix-length leading zeros still work (192.0.2.0/024). new_no() still reads leading-zero parts as decimal. Several inputs that used to parse now return undef or a different result; see the 4.080_06 Changes entry for the before/now table.
  • canon() for IPv4-mapped IPv6 addresses changed (GH#98, 4.080_07). It returns RFC 5952 section 5 mixed form, such as ::ffff:192.0.2.1. IPv4-compatible, IPv4-translated and NAT64 prefixes stay hex. short(), addr() and stringification are unchanged, so canon() and short() may differ for mapped addresses, and re()/re6() do not match the canon() form.

Constructors and parsing

Most of the remaining API changes in 4.080_07 fix how constructors and parsers read numbers, masks, ranges and host names. These changes mainly make bad or ambiguous input fail clearly instead of being read as the wrong address.

  • Whole numbers, masks and keywords in the IPv6 constructors (GH#237, GH#219, 4.080_07). The IPv6 constructors now read whole numbers as 128-bit IPv6 values. The IPv4 constructor still treats numeric entries from 0 to 2**32 - 1 as IPv4, so use an IPv6 constructor when you need the IPv6 value. An IPv6 constructor can rebuild an IPv6 object from the values returned by numeric(), including ::/0 and ::1/0.
    CallBeforeNow
    new('5')0.0.0.5/320.0.0.5/32
    new6('5')wrong 128-bit placement in older 4.080 behaviour0:0:0:0:0:0:0:5/128
    new6('5', 8)0:0:0:0:0:0:500:0/1040:0:0:0:0:0:0:5/8
    new6('127/8')0:0:0:0:0:0:7F00:0/1040:0:0:0:0:0:0:7F/8
    new6('127.0/8')0:0:0:0:0:0:7F00:0/1040:0:0:0:0:0:7F00:0/104
    new6('-1')32-bit complement formundef
    new($v6->numeric) where $v6 is ::1could not round-trip the IPv6 value as IPv6 through new()0.0.0.1/32
    new6($v6->numeric) where $v6 is ::1could not rebuild the IPv6 object0:0:0:0:0:0:0:1/128

    Dotted text keeps its IPv4 prefix length; a plain whole number uses the given prefix. A keyword uses the caller's mask on the keyword's IPv6 address. A negative number or invalid mask gives undef.

    NetAddr::IP->new6('127.0/8');          # 0:0:0:0:0:0:7F00:0/104
    NetAddr::IP->new6('127/8');            # 0:0:0:0:0:0:0:7F/8
    NetAddr::IP->new6('loopback', 64);     # 0:0:0:0:0:0:0:1/64
    NetAddr::IP->new6('loopback/64');      # 0:0:0:0:0:0:0:1/64
    NetAddr::IP->new6('loopback', undef);  # 0:0:0:0:0:0:0:1/128
  • Mapped and Cisco IPv6 constructors (GH#237, GH#234, 4.080_07). The mapped and Cisco IPv6 constructors use the same numeric reading as the IPv6 constructor. The mapped constructor maps only IPv4 addresses; IPv6 addresses and keywords return the same result as the IPv6 constructor.
    NetAddr::IP->new6FFFF('192.0.2.1');    # 0:0:0:0:0:FFFF:C000:201/128
    NetAddr::IP->new6FFFF('2001:db8::1');  # 2001:DB8:0:0:0:0:0:1/128
    NetAddr::IP->new6FFFF('::192.0.2.1');  # 0:0:0:0:0:0:C000:201/128
    NetAddr::IP->new6FFFF('loopback');     # 0:0:0:0:0:0:0:1/128
  • Numeric masks and address values (GH#235, GH#236, 4.080_07). A small number with a whole-number mask is read as the address value when the mask comes from a numeric form such as numeric(). A numeric mask of 0 is still prefix length 0. With a prefix length, the number is still the first octet. An IPv4 address with a mask written as a whole number wider than 32 bits needs the 96 leading ones; otherwise the IPv4 constructor returns undef.
    NetAddr::IP->new(5, 4294967292);                                             # 0.0.0.5/30
    NetAddr::IP->new('10', 8);                                                   # 10.0.0.0/8
    NetAddr::IP->new('192.0.2.4', '340277174624079928635746076935438991360');  # undef
  • Mask and constructor edge cases (GH#201, GH#203, GH#211, 4.080_07). An IPv6-form mask of 1 to 95 ones on an IPv4 address makes the IPv4 constructor return undef (it used to produce a negative prefix length). A bracketed IPv6 literal with the mask as a separate argument now gets the same mask as without brackets. The ipv6 mask on default, any, loopback or unspecified gives the IPv6 address.
    NetAddr::IP->new('192.0.2.4', 'ffff::');  # undef
    NetAddr::IP->new('[2001:db8::1]', 64);     # 2001:DB8:0:0:0:0:0:1/64
    NetAddr::IP->new('loopback', 'ipv6');      # 0:0:0:0:0:0:0:1/128
  • Ranges (GH#241, GH#221, 4.080_07). A short range returns undef unless it covers exactly one subnet. Aligned ranges are unchanged. The IPv6 constructor uses the same reading. A range with a space or tab on only one side of the dash parses the same as with spaces on both sides or none.
    NetAddr::IP->new('192.0.2.5-6');             # undef
    NetAddr::IP->new('192.0.2.4-7');             # 192.0.2.4/30
    NetAddr::IP->new('192.0.2.0 -192.0.2.255');  # 192.0.2.0/24
  • Host name resolution (GH#170, 4.080_07). A host name passed to a constructor is resolved with one call to getaddrinfo. The IPv4 constructor uses an IPv4 answer when there is one. The IPv6 constructor uses an IPv6 answer when there is one, unless that answer is IPv4-mapped or IPv4-compatible and an IPv4 answer exists. Short IPv4 text is packed without a lookup. Under :noSock6, names still go to gethostbyname. A name whose only answer is an IPv6 address in ::/96 now returns that address from both constructors. Text ending in a number or 0x hex label is refused without a lookup.
    # short IPv4 text: packed without a lookup
    NetAddr::IP->new('127.1');
    
    # refused without a lookup (number/0x label at the end)
    NetAddr::IP->new('host.123');

Comparisons and operators

Sorting and comparisons are now more consistent. Bad operands, such as strings, numbers or undef where an address object was expected, are handled predictably instead of making comparisons die.

  • IPv4 vs same 128-bit IPv6 sort order (GH#252, 4.080_08). An IPv4 address sorts before the IPv6 address that holds the same 128 bits (192.0.2.4/24 before ::192.0.2.4/120). <=> returns 0 only when == is true, and cmp only when eq is true.
  • Comparisons with non-address values (GH#228, GH#231, 4.080_07). Comparing an address with a string, number or undef no longer dies. <=>, <, >, <= and >= return undef or false (as == does), and cmp, lt, gt, le and ge compare text (as eq does). Comparing with the string NetAddr::IP or NetAddr::IP::Lite, or passing either to within() or contains(), answers as for any other string.

Methods

Aggregation and splitting methods also changed where the old behaviour was wrong or unsafe, often when given bad plans, class-name calls, or mixed/duplicate input.

  • split family (GH#259, 4.080_08; GH#215, 4.080_07). split, splitref, rsplit and rsplitref croak when a plan puts a piece at an address not aligned to that piece's size. Plans ordered from largest to smallest still work. IPv6 splits with a mask 54 or more bits longer than the subnet now return the correct pieces.
  • compactref (GH#233, 4.080_07). Calling compactref() on an object with no list returns that object's network, as $ip->compact() does.
  • coalesce and Coalesce (GH#232, 4.080_07; GH#39, 4.080_06). Each address is counted once; a subnet listed twice or inside another subnet is not double-counted. Mixed IPv4 and IPv6 lists are coalesced separately with the same mask length applied to each family, and IPv4 results come before IPv6 results.
  • Class and cross-type calls (GH#229, 4.080_07). coalesce, compact or compactref called on the class name croak with a clearer message. coalesce on a NetAddr::IP::Lite object, with Lite objects in the list, and compactref on one, work where they previously croaked or died.

Regex, formatting and predicates

  • re6() and re() (GH#222, 4.080_07). The IPv6 regex no longer matches an address inside a longer one.
  • canon() (GH#98, 4.080_07). IPv4-mapped addresses are returned in RFC 5952 section 5 mixed form (see breaking changes above).
  • is_rfc1918() (GH#253, 4.080_08). An RFC 1918 address held in an IPv6 object as a mapped address, such as ::ffff:10.0.0.1, is now recognised as RFC 1918.

Arithmetic and indexing

Large integers and out-of-range values are bad input for these operations; they now fail in clearer, safer ways.

  • Subtraction (GH#200, 4.080_07). Subtraction accepts every constant that addition accepts. Constants from 2**63 + 1 to 2**64 - 1 no longer croak, and a Math::BigInt of magnitude 2**64 or more croaks instead of wrapping.
  • Math::BigInt inputs (GH#214, 4.080_07). A Math::BigInt address or mask keeps its sign (Math::BigInt->new(-5) gives 255.255.255.251/32). Math::BigInt NaN or infinity returns undef.
  • nth() (GH#202, 4.080_07). An index of 2**64 - 1 or more no longer makes nth() croak. An index inside the subnet returns its address, and one past the end returns undef.

NetAddr::IP::InetBase and NetAddr::IP::Util text and packed functions

4.080_06 also tightened low-level handling of bad packed and text arguments, with clearer error reporting, in these modules:

  • Packed wide characters refused (GH#160). InetBase functions that measure packed arguments croak with Wide character in <function> at the caller's line when an argument contains a character above 0xFF. A UTF-8-upgraded string is read as its bytes where that path is valid.
  • ipv6_ntoa length checks. Length errors are reported by ipv6_ntoa at the caller's line on every host. A 4-byte IPv4 address is still refused.
  • isNewIPv4, isIPv4 and isAnyIPv4 (GH#156). isNewIPv4 croaks on an argument that is not 16 bytes, naming itself. POD for all three is aligned.
  • inet_ntop and inet_pton without Socket6. Non-numeric families are refused before comparison (no numeric warning), and a number followed by text such as 2abc is refused. Padded, signed, decimal and exponent numeric forms are unaffected.
  • Error reporting. InetBase errors name the caller's line; functions croak where they used to die (message text before at is unchanged).

Build notes

After checking the API changes, please test how the build behaves on your platform. 4.080_08 fixes configure and build selection: the configure step tests the compiler and linker the build will use (CC=/LD= given to Makefile.PL, else Perl's own), and rerunning Makefile.PL in an XS build directory cleans generated Util.c/Util.o (GH#268, GH#263). Test both modes if you can:

perl -MNetAddr::IP::Util -e 'print NetAddr::IP::Util::mode(), "\n"'

It prints CC XS or Pure Perl. If you can, run your tests once with XS and once with perl Makefile.PL -noxs.

Try it

Install the trial from CPAN:

cpanm DJZORT/NetAddr-IP-4.080_08.tar.gz

A plain cpanm NetAddr::IP still installs the last stable release, so use the tarball. Or clone and test from source:

git clone https://github.com/cpan-authors/perl-NetAddr-IP.git
cd perl-NetAddr-IP
dzil test

If something breaks, please open an issue with your Perl version, OS, whether you built XS or pure Perl, and the output of mode(). Smoke reports now help turn this trial into a stable 4.080.

3 Comments

Dean!
I'm finally taking the time to sign in here to tell you that I've tried each trial release you announced here and haven't run into a single problem.
Most of all, I'd like to thank you for doing this! It's much appreciated!

Leave a comment

About Dean

user-pic I blog about Perl. I am back in Sydney