This is a really stupid (and kludgy) Stupid Lucene Trick: how do you search the full text of your document + metadata without the full text? Simple - you create a separate field (call it fulltext, for example), then populate it with the text of all of your other fields. Voilà! You can now search against "fulltext" for all of the text in your Lucene document without having to know whether, for example, "physical" is in the document body, the abstract, or only in the keywords.
Note that fulltext should be created dynamically from the fields for the particular document, rather than from a fixed set of named fields so you maximize the amount of text you can search against.
And yes, you are duplicating the text of the all of the other fields in this field :(.
So I don't forget this - my expanded version of something our Larry Wall once said:
"Looking at Perl code, it is easy to tell whether you are looking at the ravings of a lunatic. With Java, you can be several weeks into the project before you realize that you are dealing with the demented discourse of someone who should be trusted with nothing sharper than a spoon."
Today in the Dist-Pen I am going to have a quick Post-ette.
In yesterday’s post I was a little disappointed with Dist::Zilla::Plugin::Test::Legal not working, so for today’s post I was going to have a look at it and see why.
Well to start off I did do a force install yesterday which is never a good way to do things but I was a little short of time. Looking at the code in Dist::Zilla::Plugin::Test::Legal the part I we are interested in is only a few lines long;
I have a piece of Perl containing manifest constants that have the same value, but which signify different things. In my testing, I wanted to make sure I was getting the right one out of my code. That seems to mean changing code to test it, which is anathema to me.
It occurred to me today that if I held my tongue right I could change the values of the constants without touching the code they were defined in or exported from. Holy aspect-oriented programming, Batman!
The trick is based on the fact that what use constant really does is to define a constant subroutine with an empty prototype. Now, Perl will allow me to redefine a subroutine -- grumpily if use warnings qw{redefine} is in effect, and happily if not. So I thought all I had to do was load the module that exported the constant, hammer its symbol table, and voila!
Usually people deal with YAML files from trusted sources. But maybe you want
to load input from a Website as YAML. This can lead to problems, and this
article will talk about what you can do to make the loading safe.
The problems I'll talk about are loading objects, cyclic references and general parsing
problems.
While updating Set::Jaccard::SimilarityCoefficient to v1.6.1, I realized that I should install it on a clean, clean system so I find all missing prerequisites (not just the ones I already knew about). So where do I get a clean, clean system? Aha - a minimal Docker container (like the plain-vanilla Ubuntu 16.04 LTS) would be just the ticket as I am using an Ubuntu 16.04 LTS laptop. As an example, here are the steps I ran through:
Run: "sudo docker run -v /home/mark/src/perllib/Set-Jaccard-SimilarityCoefficient/Set-Jaccard-SimilarityCoefficient-1.6.0:/Set-Jaccard-SimilarityCoefficient-1.6.0 -it ubuntu".
Because this is a plain-vanilla Ubuntu, I need to install some development tools (including missing pieces of Perl) by running: "apt-get update; apt-get install gcc make perl-modules-5.22". Followed by: "cpan -i App::cpanminus"
(If anyone points out in the comments how I could use the direct "curl" install for "cpanm" rather than using "cpan", I will update these directions.)
YAP, Yet another Post-ette here at the Dist-Pen today.
There seem to be a never ending list of the chores that are involved with creating a distribution for CPAN. This is my eight post dealing with these little chores and hopefully the last in this vain. So today I am going to see if Dist::Zilla can help me with the README file.
This file normally contains detailed instructions on on how to install the module, not really requirement for a simple Moose module like mine, but something like DBD::Oracle, it uses XS and an external data driver, the README is essential. Anyway I always think it is a good Idea to have one.
Dist::Zill comes with a '
Readme
' plug-in so lets give that a try first, by adding it into my '.ini' file;
We are delighted to announce The Swiss Perl Workshop 2018. This year the workshop will be held in Bern at Gewerblich-Industrielle Berufsschule, an industrial vocational school. The workshop will be held in English, although of course other languages are welcome.
The workshop will take place on Friday 7th and Saturday 8th of September 2018 and thanks to the generosity of the school the workshop will be free to attend. As the venue is a technical school we hope to attract students and teachers, so this year's workshop may be an ideal place to promote Perl to those who have not yet heard of or used the language.
Please spread the word, register, submit talks, and come enjoy a perl workshop in Switzerland's historic capital city. You will find more details over the coming weeks added to the official workshop site.
Encode is a well known core module in Perl with support for encoding and decoding text from almost any character encoding you can think of. But it's also an old module with a large amount of historical cruft.
With inspiration from #perl on freenode IRC, Encode::Simple is an attempt to at least improve on its interface and usability issues. Rather than an awkward and unintuitive bitmask and the option of clobbering the input data, Encode::Simple exports straightforward encode and decode functions that simply return the encoded or decoded result or throw an exception. For the cases where you want to allow invalid characters or byte sequences, encode_lax and decode_lax are provided.
Encode itself supports many more operations such as partial, mixed, and in-place encoding or decoding, where its options may make more sense, but these uncommon use cases are not supported by Encode::Simple.
As a final note, if you are working strictly with UTF-8, I suggest avoiding Encode entirely and using the excellent Unicode::UTF8, or the related filehandle layer :utf8_strict, for fast and correct encoding and decoding.
It appears that one of the more common uses for the CASE statement in SQL is in a 'GROUP BY' or at least a good 80% of the tutorials for SQL I have looked at have it as an example so let see how well the present Driver::DBI handles it.
To start I just add this test to the '60_order_by.t' test case;
Somebody once said, that the power of a programming language is not what it lets you do, but what it lets you do easily. Might be Larry himself who said it, not sure, but I heard it at the London Perl Workshop. I started learning Perl from an old second hand book I found in a charity shop a few years ago. I can't say I am any good at it. While previously I could find lots of resources available, these are increasingly harder to find, more outdated, and less able to address modern use cases. Some folk even suggest it is terminal decline. It would be a pity. It is a powerful language, and using it for me has given me a daily discovery for some feature that I didn't know, and exposed me to a large number of clever folk who can bring a different angle to a problem you have struggling with. To my mind it reflects the versatility of a language that has more than one way to to anything.
Recently, I came across a somewhat-frantic comment on StackOverflow that describes a 2017.01 change to the type of return value of .sort:
"you just can't be sure what ~~ returns" Ouch. […] .list the result of a
sort is presumably an appropriate work around. But, still, ouch. I don't know
of a blog post or whatever that explains how P6 approaches changes to the
language; and to roast; and to Rakudo. Perhaps someone will write one that also
explains how this aspect of 2017.01 was conceived, considered and applied;
what was right about the change; what was wrong; etc.
Today, I decided to answer that call to write a blog post and reply to all of the
questions posed in the comment, as well as explain how it's possible that such an "ouch"
change made it in.
Expected--> SELECT CASE WHEN Price < ? THEN ? WHEN Price >= ? AND Price <= ? THEN ? WHEN Price > ? AND Price <= ? THEN ? ELSE ? END AS price_group FROM Products
Generated-> SELECT CASE WHEN Price < ? THEN ? WHEN Price >= ?andPrice <= ? THEN ? WHEN Price > ?andPrice <= ? THEN ? ELSE ? END price_group FROM Products
In the above I see at least fourt problems;
No space between my 'AND' conditionals
The conditional is in lower case it should be coming into the DAD in upper case and
There is no 'AS' between the 'END' and the 'Alias' 'price_group'
The first one should be no problem as this little patch fixes that
This article covers scalar styles in YAML 1.1 and
1.2. It mostly works the same in both
versions.
YAML is a data serialization language, and one design goal was that it's
human friendly. It should be easy to read and edit, even if that makes parsing
it harder.
Let's look at strings, specifically.
If you look at JSON, you have only one style to encode strings, and that's
the double quoted style which doesn't allow literal linebreaks.
YAML files are used for many different purposes, and there are many types of
strings, especially multiline strings. For each use case, you can choose the
type of quoting (or no quoting) that makes the string readable and easy to edit.
This gives you lots of freedom, but you also have to learn using it to avoid
mistakes.
