My module Astro::SpaceTrack retrieves orbital data from the Space Track web site. To track this change, I intend shortly to release a new version of Astro::SpaceTrack which uses the REST interface by default. My understanding is that the old, screen-scraping interface will be available for an indefinite (but finite) time. For details, click on the post title for the extended version of the post.
]]> For some time now, the Space Track web site, which is the official source for satellite orbital elements, has been working on an upgrade to a REST interface. This interface is scheduled to go live on February 20.I have been tracking this work in the Astro::SpaceTrack package, which retrieves data from the Space Track web site. This module uses the space_track_version attribute to determine which interface to use to access the Space Track web site: A value of 1 (the default) gets you the old, screen scraping interface, and a value of 2 gets you the new REST interface.
I intend to release, shortly, a production version of Astro::SpaceTrack which makes a single change: the default value of the space_track_version interface will become 2. The screen-scraping interface will still be available by explicitly setting the value of the space_track_version attribute to 1. I understand that this interface will remain available once the REST interface goes live, for an indefinite (but finite) period. Because I have no control over the life of the screen-scraping interface, users would do well to move to the REST interface as soon as they reasonably can.
Once the REST interface becomes live, I intend to make another production release, whose sole change is to access the REST interface through the production URL rather than the beta URL. This release will also deprecate the screen-scraping interface.
Some time down the road the screen-scraping interface will go away, both on the Space Track web site and (as soon as I notice) in Astro::SpaceTrack.
]]>"Ah, bon soir, mon ami," said Dupin, "and what brings you beside the Seine on this crisp evening?"
"Something has been puzzling me," I replied, "and I thought the walk would give my thoughts an opportunity to put themselves in order."
"It is a day of puzzlements, no? Will you not share your puzzlement with me?"
"Well," I demurred, "it is a small thing -- probably something I did to myself. But I do not wish to intrude on your time."
"Ah, but it is no intrusion for a friend."
"Well," I said as we turned across the Pont Neuf, "as you know I dabble in Perl, and I like to keep a variety of systems available for testing purposes. I upgraded one of these and was re-installing Perl, when one single module failed to install -- indeed, failed to unpack."
"Perhaps the author had the misfortune to make a bad package."
"No, I don't think so. The same package unpacked and installed correctly on other systems. I suspected something OS-specific, but Perl Testers reports no such problems with the package. Then I suspected my cpan client, but I deleted everything from the .cpan directory but the configuration, to no avail."
"Most unusual. You say this distribution failed to unpack. Can you be specific?"
"Yes," I replied as we turned onto the Quai de Conti, "it was odd. When I used the cpan look command, I found myself in a directory named after the author of the package. That directory contained the actual package in a subdirectory, along with some other stuff."
"In a subdirectory you say? Was it a good copy?"
"Yes. I was able to install from it by hand, the only problem being that there was a dependency to take care of."
"And this so-called 'other stuff'," Dupin pursued, "of what did it consist?"
"It was a directory -- PaxHeaders.11295 or some such. I did not look into it."
"Ah, that is very interesting. Tell me also under what operating system did you experience this problem?"
"OpenBSD. I had no problems under FreeBSD or Linux."
"Yes, that follows. But to complete the picture we need one final piece. Tell me, mon ami, how have you set up cpan to unpack your distributions?"
"I don't know. The default I suppose, whatever that is. Does it make a difference?"
"It does indeed, mon ami," Dupin replied. The cpan client uses setting prefer_external_tar to determine whether it uses an external executable or Perl modules to unpack the distribution. Unless you explicitly set this, it tends to be 1, which uses an external tar, or undef, which leaves it to discretion of the CPAN::Tarzip code.
"I still don't see the point," I exclaimed, "tar is tar, is it not?"
"By no means!" Dupin asserted. "This pax that gives you no peace was an archiver introduced to try to unify tar and cpio. It never really caught on, but the format was enshrined in a POSIX standard, and some tar programs add pax metadata in response to --format=posix. Linux uses Gnu tar, which understands this sort of thing. FreeBSD use bsdtar, which can also cope with the pax metadata. OpenBSD, however, does not."
"So I have to replace the system tar with Gnu tar?"
"Indeed, that is what CPAN::Tarzip suggests. But on a new installation such as yours, or any installation with a modern Archive::Tar, it suffices to set prefer_external_tar to 0."
"Just that? I am tempted to misquote the Bard of Avon: 'The fault, dear Brutus, is not in our selves but in our tar.'"
With apologies to Shakespeare as well as Poe.
This was a problem I ran into recently. With no Dupin to guide me I went to even greater lengths than Dupin's anonymous friend to try to sort the problem out. Eventually I convinced myself that the system tar itself gave rise to the extra file, which confused CPAN::Tarzip mightily. Yes, I could have painstakingly characterized the various tar programs involved, but it was much easier to turn off prefer_external_tar and see if that fixed the problem. It did, and the rest was, well, not so much history as a blog entry inflicted on long-suffering readers.
The couple of sentences on the origins of pax are derived (perhaps faultily) from the relevant Wikipedia article.
"But there is, mon ami -- now."
"Now? Why now and not before?"
"Because in the interim, M. Garcia-Suarez, the author of the Safe module, has enhanced it to work nicely with Devel::Cover."
"Wonderful," I replied. "It must have been a complicated matter."
"Simplicity itself, mon ami. He simply adds '&Devel::Cover::use_file' to the default share list if Devel::Cover is detected."
"So, Dupin, you know this M. Suarez-Garcia and informed him of the problem?"
"Non, mon ami, this was done without my involvement. Though it would be nice to think that your small note upon the problem helped to bring it to his attention."
My apologies to Rafaël Garcia-Suarez and my readers for being so long about writing this epilog. The change was made in version 2.32 released March 31 2012. His solution could probably have been implemented by hand, but though Dupin is certainly smart enough to figure this out, I was not.
No, I do not consider a blog entry a substitute for an RT ticket, but in this case I was unsure which module to ticket, and then other things in life intervened.
ack hits from the crufty ones. I am ashamed to say how long it took me to think of the following:
function ackx {
if [ -f MANIFEST ];
then ack "$@" `sed 's/[[:space:]].*//' MANIFEST`;
else ack "$@";
fi
}
Yes, this could equally well have been a small Perl script involving ExtUtils::Manifest, as a more portable implementation, and a cleaner way to get rid of any comments.
]]>"A death?" I asked, smiling, remembering the events of a few days previous.
"Non, mon ami, simply a puzzle, and a rather pretty one. It appears that one M. Tueur, in attempting to improve his Perl, has entrapped himself in a web of conflicting requirements from which he can not extricate himself."
"Better him than me," I observed. "How did this come about?"
"Very simply. He found himself in a position where he had a string containing Data::Dumper output, which he needed to load back into his code. Now, normally this requires a stringy eval, but since he was justly cautious about doing this, he decided to use the Safe module instead."
"A wise precaution," I remarked, trying to hold up my end of the conversation.
"But then he wanted to do coverage testing."
"Another wise decision. It is a well-known axiom that untested code is buggy code. But when do the conflicting requirements come in?"
"Ah, they have already done so, mon ami. Devel::Cover, in order not to pollute the name spaces in which it runs, makes fully-qualified calls to itself, which can not be resolved from inside a Safe compartment. So however great his test coverage might have been, by simply trying to measure it he ensures that code below his calls to Safe's reval() method is not executed."
"By blazes! That is a poser. What can he do? I suppose he could change his code to detect the loading of Devel::Cover, and fall back to a stringy eval. But no, you don't want to change production code just to test it."
"Non, non that is never wise," replied Dupin.
"Wait. He could write a mock Safe object, and use that in his testing. His new() would return a blessed reference to an empty hash, and his reval() could simply do a stringy eval on its argument. Other methods could simply return."
"How, then, will he know that his calls to Safe actually function as he expects? It may be that he needs to adjust the operand mask allowed by his Safe compartment. If he uses a mock object in his normal testing, he will not discover this."
"You're right. Whatever he does he gives up something. Do you have any thoughts, Dupin?"
"A small one only, I fear. Your idea of the mock object has merit, but it must be more selective in its application. Suppose M. Tueur had a way to use the mock object only when he is doing coverage testing, but without modifying his production code?"
"Can this thing be done be done?" I asked. "It seems impossible".
"Not impossible. He can simply place his mock object in a directory of its own, mock/cover/ for example, and insert that directory into @INC only when he is doing coverage testing.
"There are a number of ways to do this. He could detect the loading of Devel::Cover in his test modules, and modify @INC there, either directly or by calling lib->import().
"But if he is using Module::Build, a better way is simply to subclass that module, and add the following code to the subclass:
sub ACTION_testcover {
my ( $self, @args ) = @_;
local @INC = ( 'mock/cover', @INC );
return $self->SUPER::ACTION_testcover( @args );
}
"In this way, the change is centralized to the one place where it is
known that a coverage test is being requested."
"Well, I'll be ..." But I was at a loss for words, and the sentence was left dangling in the air of Paris.
I confess to being not entirely satisfied with the solution that I have placed into the mouth of M. Dupin (which is a workaround rather than a real solution), and am left with the feeling that the shade of Edgar Allen Poe, in his capacity as analyst of Maelzel's chess-playing engine, deserves better. But after a day or so of ringing the changes on Safe's share_from() method, a mock object was the best I could do. Maybe there is a simpler solution, but the RT queue for the Safe distribution says that at least one other person is as fuddled as I am. Readers?
"A death. And a rather gruesome one, n'est-ce pas?" he replied, handing me the paper. On it I saw:
$ perl -e '%h = map { "-$_" => 1 } qw{ foo bar };'
Not enough arguments for map at -e line 1, near "} qw{ foo bar }"
syntax error at -e line 1, near "} qw{ foo bar }"
Execution of -e aborted due to compilation errors.
Well, a map takes either a block or an expression followed by a comma, and either must be followed by a list. The block was there, and so was the list. Clearly a neophite had been playing with bleadperl, and paid the price. With a rueful shake of my head, I silently passed the paper back to the detective.
"No, this was a production release of Perl, mon ami."
"But how --" I expostulated.
"-- did I know what you were thinking? Simple observation. When I gave you the paper, your brow contracted in puzzlement. This was succeeded by the abstracted expression of deep thought. Your right thumb played back and forth between the first two fingers, as though considering two alternatives, thus I knew you were thinking of the two forms of map. Your expression of rueful pity for the victim as you returned the paper to me completed the picture."
"But how --" I began again.
"-- can a production version of Perl fail to parse such an obviously correct map? Ah, that is a deeper problem. The solution lies in the fact that in Perl, curly brackets are overloaded, and may be used either as block delimiters or a hash constructor.
"The map takes either an expression or a block as its first argument, and the documentation states plainly that the Perl parser must choose between these alternatives before it analyzes the entire statement, and that, absent clear clues, it guesses." His face assumed an expression of something like revulsion as he pronounced the last word. "Since the parse failure makes no sense with a block, it must be that Perl parsed the curly brackets as a hash constructor, and therefore as an expression. The comma which must follow an expression was not found, lacking which a painful death ensued.
"But how --" I broke in for the third time.
"-- could this tragedy have been prevented? Nothing simpler. The perlref document states that a unary plus sign before the left curly bracket forces it to be interpreted as a hash constructor, and a semicolon after it forces the interpretation to be a block. Two simple strokes could have prevented this horrible death."
And with his pen, he inserted the semicolon which would have spared the victim's life:
$ perl -e '%h = map {; "-$_" => 1 } qw{ foo bar };'
With apologies to Edgar Allen Poe for borrowing his detective (whom I return, I hope, only a little worse for wear), and to Francophones for possible butchering of their language. The one-liner is an abstraction of a piece of code that puzzled me, until I remembered that Perl might not see the map the same way I did.
Under Mac OS X you can get this functionality by wrapping your Perl script in an Apple Script. The following example assumes you want to wrap a Perl script named droplet.PL in an application called PerlDroplet. It works because a Mac OS X application bundle is simply a directory whose contents are known to the system.
/Applications/AppleScript/PerlDroplet.app/Contents/Resources/.Yes, this will work if you move PerlDroplet somewhere else.
The 'do shell script' directive used to get the Perl script to run will actually work on any command that /bin/sh can handle. This means you need a shebang line in your script, or you need to explicitly run Perl. The do shell script is the equivalent of Perl's back tick or qx{} operators; anything the command writes to standard out is returned to the Apple Script.
The environment your script sees is fairly minimal. Specifically, you will not see anything you defined in your .profile or .bash_profile file. You will, however, see anything you define in ~/.MacOSX/environment.plist -- that is, once you log out and log back in again. Restarting the finder is not sufficient.
OK, at long last here is the actual Apple Script. The blogging software truncates it on the right, but it turns out that if you select everything and do a Copy, the un-truncated text ends up in the paste buffer, at least using Firefox 3.6.24 under Mac OS 10.5.8 Leopard.
-- field the drop event
on open argv
-- Make the file names into strings containing quoted POSIX file names, and concatenate them with spaces between.
set shell_argv to ""
repeat with arg in argv
set shell_argv to shell_argv & " " & quoted form of POSIX path of arg
end repeat
-- Our Perl script is droplet.PL in the Resources directory of the application bundle.
set perl_script to quoted form of POSIX path of (path to resource "droplet.PL")
-- Run the Perl script. The file names appear in @ARGV. Its output to STDOUT is captured in result.
set result to do shell script perl_script & shell_argv
-- Display a dialog containing the result.
display dialog result as string
end open
satpass script take advantage of the new lazy_pass_position attribute.
So who uses this distribution anyway? Since it is open source, the only users I find out about are the ones who write me — usually with a problem of some sort. Most of these represent an opportunity to improve the distribution, even if only to try to make the documentation a little clearer.
My impression from the correspondence is that most uses of this package are casual — hobbyists, interested amateurs, and the like. This is not to deprecate those users: I am one myself.
But it appears to be be in serious, day-to-day use in at least four places. These are:
An European Space Agency subcontractor uses it to help his communications antennae acquire and track satellites, and to plan communications. No, this is not a real-time use of Perl; the antennae can track a satellite once it is acquired, but they need to be told where to start, and they need help to prevent their being hijacked by stronger signals.
A web service that provides notification of International Space Station overflights is in the throes of adopting this package to do its predictions.
A radio astronomer uses this package to monitor the positions of certain satellites so that he can plan his observing.
A meteorologist who needs to avoid satellite interference with his Doppler radar uses this package to know when the relevant satellites are above the horizon.
Now, I have not quizzed any of these people over and above whatever their problem at hand was. But my impression is that only the first two are professional programmers. The last two are people in technical jobs of which programming is only a part.
I mention this last because there have been some posts recently about the market and mind share of Perl among professional programmers. But to what extent should our marketing take into account other sorts of users? I do not know the answer, but thought it might be useful to ask the question.
]]>On June 11 2011 I wrote about the changes to the Astro::Coord::ECI::TLE pass() method which were in release 0.039_04. The blog entry detailed those changes and expressed the intent to make another release in about two weeks.
A respondent questioned the performance of the pass() method when the interval attribute was positive. This attribute causes the pass() method to return periodic positions during the pass, as well as the significant events of the pass. Investigation revealed a serious inefficiency in this functionality, and turned up some more “edge case” pass prediction problems.
Because of the further changes to the pass() method, I have made another development release: 0.039_05. I intend to allow another two weeks for evaluation and feedback before making either another development release (if further issues turn up) or a production release (if not).
The specific problems fixed in this release are:
Versions 0.039_03 and 0.039_04 occasionally reported passes which ended shortly before the prediction interval started.
Versions 0.039_03 and 0.039_04 sometimes calculated the wrong max time. Again.
Passes in progress when the prediction interval started were still sometimes being missed if the visible attribute was false.
The calculation of positions periodically during the pass (controlled by the interval attribute) was anywhere from slightly slower to horribly slower than it needed to be. Also, the illumination reported for these positions was sometimes wrong. The re-implementation reports positions starting from the rise of the satellite. The original implementation reported positions starting from the start of the prediction interval, Since this was undocumented I feel justified in changing it, but since this is a development release I also feel justified in changing it back (which I think I can do without re-incurring the performance penalty) if someone is relying on the old functionality.
Added new attribute lazy_pass_position, which gives the pass() method permission to not calculate and report the position of the event. Currently, this only affects the periodic positions calculated in response to setting the interval attribute.
The pass command in the satpass script did not display periodic positions even when the verbose setting was true.
The validation was again partially automated, so all I really know is how the code performs versus 0.039. Please let me know if you have any problems or concerns. I can be reached by electronic mail at wyant at cpan dot org.
Thank you for your time and attention,
Tom Wyant
]]>Recently Jaap Meijers wrote to me about some inconsistencies between the satellite visibility predictions provided by the Astro-satpass package and the predictions of the Heavens Above web site (http://www.heavens-above.com/). This led to the correction of a number of bugs relating to the reporting (or not) of marginal passes by the Astro::Coord::ECI::TLE pass() method. These changes are currently available as release 0.039_04 of the Astro-satpass package.
This is the first time this specific method has been touched in quite a while, and I know that there are people doing real work with this code. So what I would like to do is to detail the changes and how they were tested, and give people an evaluation and feedback period -- a couple weeks, unless I get feedback that more time is needed. If additional problems are found, I will put out another development release, and repeat as needed. I can be reached by electronic mail at 'wyant at cpan dot org'.
The specific problems fixed are:
* Passes in progress at the beginning of the prediction period were not being reported, or being reported with an incorrect rise time.
* Visible passes which began during daylight were not being reported, or were being reported with an incorrect rise time.
* Passes which spent less than a minute above the horizon, or visible passes which spent less than a minute visible, were being sporadically detected.
* Passes which spent less than two minutes above the horizon, or visible passes which spent less than two minutes visible, were having their maximum elevation times incorrectly computed.
Initial testing was done on the passes which Mr. Meijers reported to be in error, and these have been incorporated into the formal test suite. This testing includes predicting appulses with the Moon, since I know the appulse functionality has real-world uses. Unfortunately I do not have permission to distribute TLE data needed for these tests, but the required OIDs and epochs are documented in t/tle_pass.t, and those with Space Track accounts can download the data.
In addition, I downloaded Dr. T. S. Kelso's "100 (or so) Brightest" (which at the time of download consisted of 152 objects), predicted both visible passes and all passes over the world's 40 most populous cities, and accounted for the differences between version 0.039 and 0.039_04.
No, I did not hand-validate the thousands of differences -- instead I used a Perl script to identify the differences and check the data produced by the pass() method for the circumstances that triggered the various bugs. According to this script all differences between 0.039 and 0.039_04 can be traced to bugs that have been fixed. Yes, I found (and fixed) two regressions during the testing; one involving the appulse functionality of the pass() method, and one involving missed passes which occurred near the end of the prediction period.
Please let me know if you have any problems or concerns.
Thank you for your time and attention,
Tom Wyant