DBIx::Class::FilterColumn: making transformation easier
Over a year ago I was tasked with creating a data warehouse for sports data. Having known absolutely nothing about data warehousing/ETL, my first sport ended up quite the mess; scrapers would extract and transform at the same time then stuff it into a database where it most likely needed additional transformations. At the time, additional transformations meant writing a script to iterate over every row and change whatever column to whatever regex I had constructed. Sometime later after i'd have generated a report i'd find something wrong, often missing data due to a bad transform regex, which meant re-scraping websites (and often times purchasing another membership).