Dear all,
greetings from a former colleague.
Is there any specific reason why the tags releases were not updated anymore?
Specifically
I see there is a jar ETL file and the relative conf but without tag information is not possible to create a “fat” JAR.
Moreover, the repo above is constantly update with new branches and the latest master could not work with the latest reference.conf (for instance 23.06)
Can you please evaluete to start to tag the release again?
Many thanks
Cinzia
Hi Cinzia,
That’s correct, the ETL has gone (and it’s going) through a lot of changes and, at some point along last year (I think around June 2022), we didn’t do any more release tags, the ETL .jar file that is provided within the bucket has the commit hash from which it was built from, as name suffix, just as a reference mechanism or name scoping mechanism.
I’m not sure about the reason for this, I think it may be related to very quick iterations on the ETL while producing the different releases, we’ve had some challenging ones over the last year.
At the backend team we are leveraging this chance to decouple software releases from data releases, because this metadata is put together by the release deployment context in the repository where we define the infrastructure as code.
Regarding the ETL, we want the analysis process to be reproducible, so we’ll be releasing a fat jar as well as the one we already release (built for Dataproc), but we need to fit these changes in the release scheduling.
Thanks,
Manuel
Hi Manuel,
thanks for the clarification.
As a developer I would like to clone the repository and possibly extend the code of a specific release.
In this specific case a fat jar or any jar won’t help me to achieve my goal but tags all the repositories involved can help the user/developer to figure out when the release was done.
However, I get that the reproducibility is the most important point for the release process and for the OT community
Thank you so much again for the clarification.