Information Systems:Exploring transactional backups for our IBM i (Bart)
Overview
It is well understood that the value of our current backup strategy (which consists of nightly full system saves to tape) is severely undermined by the fact that all system/user activity after the backup (i.e. throughout the entire next day) is not backed up and thus prone to complete loss during a system outage. On the other hand, all exploration of potential high availability/disaster recovery (HA/DR) solutions to date have centered around a second, fully operational IBM i, which, gauging merely by the inaction with past endeavors, should indicate that such a solution is outside of the company's interests (primarily because of cost). Thus the goal of this endeavor is to develop a "customized" solution based on third-party software that avoids owning a second machine as a requirement, yet still achieves a lower recovery point objective (RPO) than our current strategy.
Considerations
It is worthy to first note the assumptions of this exploration:
- High-availability i.e. having a second machine ready to fail over to and serve users when the primary system is down, is of less interest to the company because a) the cost required of owning and licensing a second machine and b) the reality that any disaster impairing the system beyond immediate serviceability/repair is likely to equally affect company operation on other fronts (i.e. the case of a natural disaster affecting building infrastructure (warehouse inventory), network connectivity to the outside world, logistics (shipping routes) etc.). In other words, business continuity planning that provides redundancy only for the critical machine but not other parts of the business (i.e. ability to fulfill orders) does not make sense.
- Because of the first point, backup technologies and concepts involving a second machine will be ignored i.e. PowerHA, FlashCopy, LUN-level switching etc. The focus will be on logical replication of data.
- Bandwidth constraints limit any strategies that involve a full system backup over the network every night. Such solutions (i.e. Amazon S3/Glacier for x86) do not seem to be prevalent for the IBM i architecture anyhow. Nevertheless, the focus is on sending incremental data to remote backup target.
- Full system saves to tape and offsite storage is still a viable solution and will be treated as the null hypothesis (accepted by default and built upon unless an alternative is encountered that can replace it at more value provided to the company). Needless to say, the goal is not to replace our current tape-based backup strategy entirely, but rather to augment or revise it.
Remote journaling
Remote journaling is a core concept in all high-availability solutions for IBM i that fit under the theme of logical replication. Journaling is a simple concept - it's a system feature that allows you to copy data writes to a remote target. The specifics of the configuration and differences in setup is where third-party HA solutions vary (thereby giving rise to all the different software products in the HA space), but the core technology, invariably, is remote journaling.
As mentioned, remote journaling is just a system feature that can be used many ways. But the core concept can be described as such:
- Enable journaling on an object (e.g. the G/L file).
- Create a journal receiver and a journal (an endpoint and a channel for writing data).
- Whereas writes before would be directly to the physical file, now they are written as journal entries to the journal receiver.
- Journal entries would accumulate during the day (as users interact with ASW and data is written).
- At some point, the journal needs to be committed back to the physical file, i.e. the journal entries become records in the G/L file.
- Journal is reset at that point, and the G/L is up-to-date.
To summarize: FILE + JOURNAL ENTRIES = CURRENT DATA