Information Systems:Exploring transactional backups for our IBM i (Bart)

From uniWIKI
Jump to navigation Jump to search

Overview

It is well understood that the value of our current backup strategy (which consists of nightly full system saves to tape) is severely undermined by the fact that all system/user activity after the backup (i.e. throughout the entire next day) is vulnerable to complete loss during a system outage. On the other hand, all exploration of potential High-availability/Disaster Recovery (HA/DR) solutions to date have centered around a second, fully operational IBM i, which, gauging by the inaction in the wake of past endeavors, indicates that such a solution is outside of the company's interests (primarily because of cost). Thus the goal of this endeavor is to find a solution (whether turn-key or customized) that avoids having to run a fully-replicated and fully-operational second machine as a requirement, yet still achieves a lower recovery point objective (RPO) and recovery time objective (RTO) than our current strategy.

Considerations

It is worthy to first note the assumptions of this exploration:

  • High-availability i.e. having a second machine ready to fail over to and serve users when the primary system is down, is of least interest to the company because of a) the cost of owning and licensing a second machine and b) the reality that any disaster impairing the system beyond immediate serviceability/repair is likely to equally affect company operation on other fronts (i.e. the case of a natural disaster affecting building infrastructure (warehouse inventory), network connectivity to the outside world, logistics (shipping routes) etc.). In other words, business continuity planning that accounts for the IBM i but not other parts of the business (i.e. ability to fulfill orders) does not make sense.
  • Because of the first point, backup technologies and concepts involving hardware mirroring to a second machine (aka hardware replication) will be ignored i.e. PowerHA, FlashCopy, LUN-level switching etc. The focus will be on logical replication (aka software replication) of data.
  • Bandwidth constraints limit any strategies that involve a full system backup over the network every night. Such solutions (i.e. Amazon S3/Glacier for x86) do not seem to be prevalent for the IBM i architecture anyhow. Nevertheless, the focus is on sending incremental data to a remote backup target.
  • Full system saves to tape and offsite storage is still a viable solution and will be treated as the null hypothesis (accepted by default and built upon unless an entire solution alternative is encountered that can replace it at more value provided to the company). Needless to say, the goal is not to replace our current tape-based backup strategy entirely, but rather to augment or revise it.

Remote journaling

Remote journaling is the core technology underlying all high-availability solutions for IBM i that classify under logical replication. Remote journaling is a simple concept - it's a system feature (i.e. IBM Remote Journaling) that allows you to copy data writes to a remote target. The specifics of the configuration and differences in setup is where third-party HA solutions vary (thereby giving rise to the many different software products in the HA space), but the core technology, invariably, is remote journaling.

The basic function of remote journaling can be described as such:

  1. Journaling is enabled on an object (let's use the G/L file as an example). Note that journaling can be local, and there are actually use cases for journaling that are not at all related to HA/DR (for example, activity auditing).
  2. A journal receiver and a journal are created (i.e. the endpoint/target and the channel for writing data).
  3. Now, writes to the G/L file (which before would be directly to the physical file), are captured as journal entries to the journal receiver.
  4. Journal entries would accumulate during the day (as users interact with ASW and data is written).
  5. At some point, the journal needs to be committed back to the physical file, i.e. the journal entries become records in the G/L file.
  6. Journal is reset at that point, and the G/L is up-to-date.
To summarize: Base file (physical file/table) + Journal entries = CURRENT DATA

Given the basic summary of function above, it makes sense that the journal is on a separate system (although if not for HA/DR use, local journaling has other benefits/use cases). Thus IBM built into the OS remote journaling features. Still journaling, but to a remote receiver.

Potential solutions

Sometimes you'll be researching a topic and just stumble upon something that just jumps out at you as an exceptionally good option. That's what Maxava RTV was in this situation, so the following discussion will be about this offering and then all other ones grouped together. -norwizzle (talk)

Maxava RTV

Maxava is a big player in the IBM i HA space. They offer a wide array of services based on their core HA software (which in turn utilizes IBM Remote Journaling). Many resellers also offer solutions based on Maxava software. Maxava has two cloud-based HA/DR solutions: HA in the Cloud and RTV, or Real-Time Vaulting. Being cloud-based, both avoid having to own and run a second machine, instead substituting that with an LPAR in a multi-tenanted system (hence the 'cloud' aspect). The difference between the two services is that with HA in the Cloud, the provisioned LPAR is of sufficient resources (allocated CPU/CPW, RAM, disk space) to host the production workload in the case of a failover. In RTV, the LPAR is more of a barebones IBM i configuration, intended merely as a journal receiver.

With RTV, the premise of the backup strategy would be that for every file write that occurs after a backup (i.e. throughout the day), a copy is sent to a remote journal. Thus, there would be a local journal (i.e. journaling is enabled), and a remote journal to which the local journal entries are mirrored. The local journal would be the one committed to the file every night, while the remote journal exists solely as an off-site copy. Theoretically, you could rebuild right up to the time of a system failure by using the most recent tape and the remote journal.

As for the recovery process, a tape with the journal changes would be sent back to us within 24 hours. 

I have asked Maxava if there is a way we could remotely apply the journal instead of waiting for the tape. Seeing that it's delta data we're dealing with, it shouldn't be a lot to download. Waiting to hear back. -norwizzle (talk)

Other

Maxava HA-in-the-Cloud

As alluded to in the previous section, HA in the Cloud is the other DRaaS offering from Maxava. I confirmed with them that it's basically the same functionality as in RTV, but with an LPAR of sufficient resources that can support a production workload. I'm assuming this would be much more expensive, but the scalability of this solution is appealing and makes it a worthwhile consideration. To elaborate, if our warehouse were to be struck with a natural disaster that impaired our production machine, it may still be of some value for us to have remote Domino, Apache (Web Orders, IOP) etc. Perhaps we wouldn't be able to pick (i.e. the RF subsystem could be omitted on the backup server), but we would be able to take orders. Because you can scale an LPAR to the exact specs required to support the workload, this may work for us despite the additional cost of actually running a second production-ready server.

Vision Solutions MIMIX Availability

Another solution based on IBM remote-journaling, to be further investigated. It seems though that this wouldn't scale down as much as RTV seems to, but will confirm. At best, it is a 1:1 alternative to Maxava RTV.

Notes

  • It's worth noting that while Maxava and Vision offer solutions, they are in essence software makers, and that likely means the cloud-hosting aspect of their solutions is done through a partner. I've confirmed that for Maxava RTV, the likely provider since we're Canadian is R2I based in Toronto.
  • In talking with Maxava trying to get an understanding of the concept of an LPAR in a multi-tenanted system, I brought up Krengeltech's Litmis Spaces. They immediately knew it was Aaron Bartell's company, and have actually been in talks with them to perhaps be another cloud provider. I love when things just come together :)
Retrieved from "https://owl.unipharm.com/mediawiki/index.php?title=Information_Systems:Exploring_transactional_backups_for_our_IBM_i_(Bart)&oldid=8300"