View source for Information Systems:Retiring Python/open-source development on IBM i

==Overview==
This page discusses a project/endeavor (specifically, its cancellation) that took place between 2017-2019 to implement a post-RPG software development strategy on IBM i by using a modern programming language as well as open-source software. As a concept, this was (and this day continues to be) [https://ibmi-oss-docs.readthedocs.io/en/latest/README.html an approach strongly advocated by IBM] for users of IBM i.

The following archived articles are related to this endeavor:
:* [[Information Systems:IBM i open-source development conventions]]
:* [[Information Systems:Python development notes - early stages]]
:* [[Information Systems:Setting up IBM i for Python development]]
==Background and History==
* uniPHARM uses IBM i to host its ERP. The ERP and its extensions are programmed in RPG, which is an aging language (officially known as "legacy", along with COBOL).
* IBM i is a highly proprietary operating system but contains a "space" called PASE that emulates a Linux environment, complete with a shell. This is somewhat analogous to [https://en.wikipedia.org/wiki/Windows_Subsystem_for_Linux WSL].
* Starting in 2017, IBM injected mass improvements to PASE, porting over major programming languages and open-source software such as MySQL/MariaDB, Python, Mono (.NET), Node.js, and nginx. They essentially created Linux within IBM i (pedants would be quick to correct this but it's the practical truth).
* Along with bringing over modern programming languages so that they can be run on the i, IBM also provided [https://github.com/IBM/xmlservice XMLSERVICE], which is a system API of sorts to allow these languages to interact with IBM i programs and objects e.g. Python program can call an RPG program.

'''Therefore, the ability to use modern programming languages and an open-source development strategy common, combined with the proximity to the built-in Db2 database and usefuleness of XMLSERVICE to interface with the system, made this a dream situation, on paper.'''

==Decision to abandon==
There were several factors that led to the eventual demise of this endeavor and strategy:
====Issues with ported versions of the software====
Open-source software within PASE exists as '''ports'''. They are forks of the actual packages, verified to work on IBM i and maintained by IBM themselves. It is not possible to run Linux binaries in PASE nor compile programs from source and expect them to work without issue. This led to two main issues:
:* '''Version lag'''. The package ports were most of the time several versions behind the latest release of their Linux counterparts. With software that doesn't change significantly over time e.g. the text-editor '''nano''', this is not an issue. However, a package like python-ibmdb-django became virtually unusable (unless modified) because it was too old and did not support later versions of Django.
:* '''Feature differences'''. Usually, software ports are meant to function as close to the original as possible. With IBM i ports, there were a lot of odd inconsistencies between the Linux and PASE versions. Either features omitted altogether (probably due to complexity of implementation) or just different behavior. OpenSSH for example had issues with IBM group profiles and permissions.

====Newness of technology to a legacy audience====
It became apparent after a year or two of observing the IBM i OSS community that they were in a state described as a [https://en.wikipedia.org/wiki/Hype_cycle "peak of inflated expectations"]. Community discourse mainly pertained to how "cool" the technology was, and conferences/tech-talks would always showcase pedestrian examples, focusing more on the fact that the technology was indeed running on IBM i rather than how to use the tech to solve real-world business challenges. This was found to be a major barrier, as there was no active development community experiencing the same type of issues to consult with. As of 2021, ''IBM i OSS still comes off as a cute pet project for IBM to show how the platform isn't the "AS/400" of old and can be "modern". Nah, not really... -[[User:Norwinu|norwizzle]] ([[User talk:Norwinu|talk]])''

====Performance====
The Power8 is a well-spec'ed machine, but ultimately it has a single-core CPU (i.e. 1/6 activated) and spinning disk storage. While the design of IBM i may be able to skirt this sort of a legacy hardware configuration (in the age of multi-core CPUs and flash storage), these bottlenecks very obviously affected performance in PASE (the emulation taking place may also very have been a factor). The web server (nginx) and app server (gunicorn) would be sluggish compared the built-in IBM i HTTP server which runs the sites built with WebSmart. Running the Python app server (gunicorn) even without any system activity constantly ate 2-3% of the CPU, which perhaps is another indicator of the inefficient performance of PASE applications. By comparison, IBM i HTTP server instances were constantly 0% without activity.

====Database flavor differences====
[https://db-engines.com/en/ranking Db2 has a respectable marketshare] amongst other relational databases, but it exists in many flavors, presumably the most popular being Db2 LUW (Linux, Unix, Windows). The one built in to IBM i is known as Db2 for i, and several issues have already been encountered that are attributed to the flavor differences. Things related row_over(), resource record numbers, and system naming vs SQL naming are just some of the issues that have caused major delays or necessitated hacky workarounds because of their particular implementation.

====System availability====
The IBM i server is shut down nightly between 1AM to 6AM for maintenance processes. And because it runs critical workloads, the server also cannot just be restarted for whatever reason, and extra care must always be taken in terms of where files are written, how much CPU programs are taking up, and the security impact of running programs directly alongside the ERP. For example, opening up another port on the server to host a web app in debug mode (for testing/development) to be accessible on the internet potentially exposes the entire server. It was also deemed impractical or overly complex to find workarounds to limited system availability, such as maintaining a parallel environment on x86 for testing, subscribing to free IBM i public-shared instances, or setting up chroot within PASE.

====Availability of more appropriate alternatives====
Over the span of this endeavor, alternatives started to appear more appropriate as a solution. These include:

* Using the more common strategy of deploying Linux x86 servers and connecting remotely via ODBC to the database.
* Increased feasibility of running and maintaining Linux workloads on-premise. IT has Linux administration skills in-house and a virtual environment with ample resources on which new virtual servers can be provisioned. 
* Using the Java programming language, which is supported on the i and has more "backing". In particular, the IBM Toolbox for Java / JTOpen software is a very mature product that serves the same purpose as XMLSERVICE, but is far superior in every way (e.g. performant, well-supported, intended for remote system access rather than for accessing the i from within PASE etc.)
* Availability and eventual procurement of Iptor Integrator, which is Java-based middleware that provides a structured way of developing/running integrations.
* The revelation that much of DC1 (the upgrade to ASW) uses the Java platform (XT, Netstore, Integrator).

====Evolving philosophy within IT====
In hindsight, part of why open-source on i was approved as an exploratory endeavor is because of the 'i' factor. That is, it may have been too much of a shock to propose open-source on x86 sintead, which is the conventional method of modern development (e.g. Linux or Windows servers). At this time, there were no Linux servers in place, no use of external databases, and no use of other programming languages for development (only RPG, CL, and WebSmart). The fact that IBM i OSS was an official project provided (now false) assurance that if new, open-source development were to be explored, doing so within IBM i was the best choice. It turned out to cause more headaches than provide gains, and it took this experience to gain the philosophy that the i is better left to run RPG/CL and that running integrations externally is no less viable than running on the same system.

==Outcome==
* uniPHARM has continued with modern development using Iptor Integrator, Java, IBM Toolbox for Java, Python, Django, Apache, and MariaDB as the main components.
* Java is the prevailing language to be used for developing back-end integrations. Python is used for Django web app development (InfoNext).
* The IBM i is no longer being used to do any new development (WebSmart being the exception). All the software listed above runs in Windows (Iptor Integrator) or Linux (Apache, Django, MariaDB).
* uniPHARM is no longer working with the 2 consultant firms previously contracted for the Carrier Interface / IBM i OSS project.

[[Category: Programming/Development]]