Optimizing Your Solimar Environments: Best Practices and Pitfalls

Hello. Welcome to this session that discusses the best and worst practices for managing your non-production and production environments. We’ll go over some pointers on how to minimize unplanned downtime, errors, and instability in your production environment to help make you more aware of that topic. Our tech support and Solimar University Online are also great resources that can help you refine your plans with more details. We’ll start with what we mean by an environment. The environment is all of the software, hardware and applications and connectivity configured to execute all the operations in your workflow. You could include hardware such as printers and finishing, but their handling is different than software.

There are a number of different environments that are typically used for production and in support of your production. Production is, of course, the environment used to produce all of your work. It’s a very clean, high performance, and it must be protected from changes that create instability or introduce errors or failures. Things like automatic upgrades are not okay here, and no changes should be made to the system without testing them first in a non-production environment.

A standby system is an exact duplicate of your production system that stands ready to shoulder your production work at any time, in case of a production down situation. It’s basically a production environment, so all the same rules should be applied. Staging systems are a clean mirror of the production system, where you apply things like upgrades and patches, where applications are tested to determine if they work. Once all of the changes, including upgrades and patches, have been tested and passed to regression testing, they can be applied to the production server at the end of this process.

Staging and production should be identical, protected environments. Development test systems are the daily drivers for staff who are charged with maintaining existing jobs or developing new jobs for production. They may have development tools, debugging tools, and other software that can cause instability in performance or otherwise jeopardize production. These are messy systems, so they can’t be relied upon as a perfect stand in for production, which is why you need a staging system.

Recovery is a mirror of the production system that can be stood up or activated for use, as required to handle production while the production system is unavailable to produce work. A recovery system must be kept in perfect sync with the production environment to ensure that when it’s brought up, it will function perfectly in place of the production system for situations like disaster recovery.

This diagram shows the preferred method for making any kind of changes to your production environment. Build and test applications in your Test/Dev environment and apply all the operating system patches and software upgrades here. The staging environment should be kept in sync with the Test/Dev as well. The difference is that it is a clean system without dev tools or other software that could impact production applications. Any software operating system upgrades are pushed to the staging environment where they are extensively tested. Once they’ve been validated in that environment, an image is made of the fully tested staging server environment. Any software operating system upgrades are then made to the production environment to make it match. When migrating Solimar content, you should migrate the entire server rather than just moving an individual application. Following this simple flow can ensure that your production system is stable and up to date, reducing the risk of any downtime because of upgrades and etc. The process is a bit different for SOLitrack and SOLsearcher Enterprise because they use a database backend which is covered in the documentation.

The Solimar support team has been around for a long time, and they’ve seen all sorts of environmental situations, which we’ll go over for some of the best and worst practices they’ve encountered.

Any upgrades made to your production environment without being fully tested in staging creates a risk that something will break. Databases, operating system patches, drivers, networking, really anything has the possibility of disrupting production. Some of these upgrades are very hard to back out, so if the staging is identical to production and it passes testing there, you can safely and confidently push changes into production.

Customers who don’t have a staging environment are forced to push applications directly from Test/Dev into production, which can be risky. Many times, the Test/Dev environments have tools, libraries, and other components that are not available in staging and production, so the developer runs the app and if it works fine there, then they push it into production and it doesn’t work because of a missing or different version of a library install as part of that development environment. Remember these environments can be complicated. Opening apps to a staging environment and feeding it up there allows you to catch issues there and not in production. Finding out the hard way they mean on the shop floor after it’s printed or worse yet, in the customer’s hands.

During an actual disaster, there’s no time to test your disaster recovery process. It must be exercised and practiced on a regular basis to ensure that the people, processes, and technologies are all working. Calling for after hour support could be an awkward and expensive way to get things up and running again. The safe route is to make a full copy of your Solimar servers, and pushing that to a shared storage location that will always be available should you need to stand up a recovery server at another location. Regularly rehearsing their procedures is necessary not just for the technology, but especially for the people involved in the process.

Additional environmental best practices include being diligent about keeping all of your environments in sync, and backing up configurations and resources. Plan for upgrades where you can try to find off peak times, or where you can have planned downtime. Build comprehensive regression testing and automate as much of it as possible.

You may also need additional non-production environments, such as adding quality assurance, user acceptance, and active development where users are onboarding new applications using the matching production versions in parallel to the testing environments. Make sure you have coverage for as many applications or scenarios as possible, including non-regular jobs like quarterly or month end runs. There are many features in the Solimar software, such as alias mapping to help you move configurations from one server to the other environments.

And finally, tribal knowledge is no substitute for good documentation. Documenting and testing all of your IT processes makes bringing in new resources, managing DR and different geographies much easier. You may want to have users record their processes for future staff training, and reminders of how and why things are configured as they are.

That’s it for this session. Thank you very much for your time. If you have any questions or easy to reach and look forward to hearing from you.