Information Systems:Managing dependencies in Iptor Integrator - Revision history

Norwinu: Minor fixup

2021-06-02T17:31:44Z

Minor fixup

Norwinu at 17:20, 2 June 2021

2021-06-02T17:20:56Z

Norwinu: /* Solution */

2021-06-02T01:17:36Z

Solution

Norwinu at 01:16, 2 June 2021

2021-06-02T01:16:55Z

Norwinu: Created page with "==Overview== This article discusses the limitations regarding Iptor Integrator and the Java [https://en.wikipedia.org/wiki/Classpath classpath], as well as the solution to the..."

2021-06-02T01:16:42Z

Created page with "==Overview== This article discusses the limitations regarding Iptor Integrator and the Java [https://en.wikipedia.org/wiki/Classpath classpath], as well as the solution to the..."

New page

==Overview==
This article discusses the limitations regarding Iptor Integrator and the Java [https://en.wikipedia.org/wiki/Classpath classpath], as well as the solution to the problem (Maven and pre-packaged project dependencies). This does not apply if you're using Integrator within its confines i.e. use Integrator Studio to develop programs, [[http://owl.unipharm.local/mediawiki/index.php/Information_Systems:Developing_integrations_using_Java_and_Iptor_Integrator#Not_using_Integrator_Studio_as_an_IDE which I don't do]].

==Background==
Being itself built with Java, Iptor Integrator relies on open-source Java libraries to run. An example is the Apache DBCP library for database connection pooling, which Integrator's own Connection Pooling feature is built on (''This could be inaccurate, but the dbcp JAR does exist and I'm just making a point anyway'' -[[User:Norwinu|norwizzle]] ([[User talk:Norwinu|talk]])). These open-source libraries (about 250 JARs) come pre-packaged with the Integrator installation and are found in <code>C:\Integrator\bin\extensions\ibs\eclipse\plugins\IBSExternalPackages</code>. Because this folder is part of the classpath, it makes sense to also put project dependencies in here. However, adding JARs to this folder gives rise to two problems:

# The classpath becomes too long as more JARs are added. There are alreaady 242 dependencies. Empirically, the classpath issue arises at around the 300 JAR mark. Integrator's JRE has some proprietary elements so it is not easy/possible to change the classpath to perhaps use the folder/wildcard notation.
# Version conflict between shared system and project dependencies. i.e. the project requires the same library as Integrator, but needs a newer version which is incompatible with Integrator. This is exacerbated when version conflicts exist within [https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Transitive_Dependencies transitive dependencies]. To give an example, let's say Integrator can only use version 4.1 of a hypothetical "logginglib", but the Backblaze B2 SDK (project dependency) requires v5.6 or higher of the lib. This is known as [[https://en.wikipedia.org/wiki/Dependency_hell|'''dependency hell'''.]]

==Solution==
There were several things tried, but the eventual solution involves 2 things:

# The incorporation of Maven to manage project dependencies and package them into a fat jar (aka uber jar) called uwd.jar''. Now, Integrator loads only uwd.jar from the classpath and there is no longer a risk of this becoming too long. Regarding Maven, IntelliJ IDEA has plugins that can be used to simplify Maven processes (e.g mvn:compile, mvn:install etc.). The pom.xml file is found in the root folder of the project.
# Manual and careful curation of shared dependencies (libs required by both Integrator and the project). This is not an elegant process and can basically be described as follows: if I encounter an error like NoClassFoundError or NoSuchMethodError related to a shared dependency when running project code in Integrator, I check to confirm that the Integrator lib folder contains an older version of the shared dependency (uwd.jar contains the newer one). I then delete this so the new version gets used, and test (by stopping and starting the Integrator service) to see if Integrator complains about having to use the new version. If it doesn't, rejoice. If it does, then I figure something else out (''Like I said, inelegant.'')

==Notes==
* There is a IBSUserPackages folder, but it doesn't seem to get loaded whatsoever by Integrator. This is something worth asking about.
* Related to the previous point, Integrator Studio provides a mechanism so that users can add packages to the Integrator environment. Interestingly, using this feature puts the jar in an adjacent folder (plugins\IBSIntegrator), which is not part of the classpath for Timer and FolderWatcher jobs.

[[Category: Iptor Integrator]]
[[Category: Programming/Development]]

@@ Line 6: / Line 6: @@
 # The classpath becomes too long as more JARs are added. There are alreaady 242 dependencies. Empirically, the classpath issue arises at around the 300 JAR mark. Integrator's JRE has some proprietary elements so it is not easy/possible to change the classpath to perhaps use the folder/wildcard notation.
-# Version conflict between shared system and project dependencies. i.e. the project requires the same library as Integrator, but needs a newer version which is incompatible with Integrator. This is exacerbated when version conflicts exist within [https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Transitive_Dependencies transitive dependencies]. To give an example, let's say Integrator can only use version 4.1 of a hypothetical "logginglib", but the Backblaze B2 SDK (project dependency) requires v5.6 or higher of the lib. This is known as [https://en.wikipedia.org/wiki/Dependency_hell|'''dependency hell'''.]
+# Version conflict between shared system and project dependencies. i.e. the project requires the same library as Integrator, but needs a newer version which is incompatible with Integrator. This is exacerbated when version conflicts exist within [https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Transitive_Dependencies transitive dependencies]. To give an example, let's say Integrator can only use version 4.1 of a hypothetical "logginglib", but the Backblaze B2 SDK (project dependency) requires v5.6 or higher of the lib. This is known as [https://en.wikipedia.org/wiki/Dependency_hell '''dependency hell'''.]
 ==Solution==

@@ Line 1: / Line 1: @@
 ==Overview==
-This article discusses the limitations regarding Iptor Integrator and the Java [https://en.wikipedia.org/wiki/Classpath classpath], as well as the solution to the problem (Maven and pre-packaged project dependencies). This does not apply if you're using Integrator within its confines i.e. use Integrator Studio to develop programs, [[http://owl.unipharm.local/mediawiki/index.php/Information_Systems:Developing_integrations_using_Java_and_Iptor_Integrator#Not_using_Integrator_Studio_as_an_IDE which I don't do]].
+This article discusses the limitations regarding Iptor Integrator, dependencies, and the Java [https://en.wikipedia.org/wiki/Classpath classpath], as well as the solution to these problems (i.e. Maven and packaged project dependencies). This article is not of any conncern if you're using Integrator within its confines i.e. if you use Design Studio to develop programs, [[http://owl.unipharm.local/mediawiki/index.php/Information_Systems:Developing_integrations_using_Java_and_Iptor_Integrator#Not_using_Integrator_Studio_as_an_IDE which I don't do]].
 ==Background==
@@ Line 17: / Line 17: @@
 * There is a IBSUserPackages folder, but it doesn't seem to get loaded whatsoever by Integrator. This is something worth asking about.
 * Related to the previous point, Integrator Studio provides a mechanism so that users can add packages to the Integrator environment. Interestingly, using this feature puts the jar in an adjacent folder (plugins\IBSIntegrator), which is not part of the classpath for Timer and FolderWatcher jobs.
 [[Category: Iptor Integrator]]

@@ Line 11: / Line 11: @@
 There were several things tried, but the eventual solution involves 2 things:
-# The incorporation of Maven to manage project dependencies and package them into a fat jar (aka uber jar) called uwd.jar''. Now, Integrator loads only uwd.jar from the classpath and there is no longer a risk of this becoming too long. Regarding Maven, IntelliJ IDEA has plugins that can be used to simplify Maven processes (e.g mvn:compile, mvn:install etc.). The pom.xml file is found in the root folder of the project.
+# The incorporation of Maven to manage project dependencies and package them into a fat jar (aka uber jar) called ''uwd.jar''. Now, Integrator loads only uwd.jar from the classpath and there is no longer a risk of this becoming too long. Regarding Maven, IntelliJ IDEA has plugins that can be used to simplify Maven processes (e.g mvn:compile, mvn:install etc.). The pom.xml file is found in the root folder of the project.
 # Manual and careful curation of shared dependencies (libs required by both Integrator and the project). This is not an elegant process and can basically be described as follows: if I encounter an error like NoClassFoundError or NoSuchMethodError related to a shared dependency when running project code in Integrator, I check to confirm that the Integrator lib folder contains an older version of the shared dependency (uwd.jar contains the newer one). I then delete this so the new version gets used, and test (by stopping and starting the Integrator service) to see if Integrator complains about having to use the new version. If it doesn't, rejoice. If it does, then I figure something else out (''Like I said, inelegant.'')