Client Background
Our client, a gaming company, is a Japanese multinational consumer electronics and video game company headquartered in Kyoto, Japan. The American branch, established in 1980 and based in Redmond, Wash., is a wholly-owned subsidiary of our client, Ltd. They are committed to delivering best-in-class products and services to the customers and investing in the well-being of their employees.
Challenges
We had DevOps in place, but we still saw duplication of effort. Our goal was to reduce overlap and address the disconnect between Dev and Ops. As a result, the user faces challenges whenever there is critical deployment with several plug-ins involved.
We’ve always approached this from a data management perspective to create tools that parks could use. This started with a focus on managing online gaming data and other digital assets. There were parallel dev projects going on in each of these programs. There were also very similar efforts going on at multiple portfolios. We knew there was no way we would ever get the resources to support each one individually. We had to think of more cost-effective ways to operate. DevOps has helped us think about how to operate at a more enterprise level.
Approach
BrickRed started to connect with the client’s team to initiate a more collaborative approach. Specifically, we had developers in several divisions. We created a blended “dev” team and hosted meetings and presentations for staff to get to know each other.
Since DevOps and other agencies blended teams were there, no manual deployments were there. But there was a lack of automated build process end-to-end, which we have enabled the same. Similarly, maturity needs to be added in automated testing to check the failure and rollback when necessary.
This knowledge should also include the extensive research we performed to educate everyone about the concept, process, and policy changes that would impact current workflows and ways of working.
Historically, the Client’s dev side has been entirely decentralized. On the ops side, users (online gamers) would want the non-disruptive gaming experience globally. They had become accustomed to easy, higher-level, or privileged access. We needed to gain more control, take back some of the management and oversight for security and compliance reasons, and reign in costs.
Because we were also responsible for the data center, we started seeing other geo applications need to be hosted for certain business reasons. Some of them we already had solutions for. We didn’t necessarily stop them once an application was built but seeing this duplication early on helped us realize the magnitude of the issue. We knew if they came to us earlier with their requirements, we could help them avoid the development costs and point them to existing solutions.
Before, all parks would get their virtual machine with the database and platform software and manage it themselves. To change this, we needed to think through how we engaged with stakeholders. We have since consolidated the data centers into a central location to be more effectively managed.
Lessons Learned
Our strategy to start with one or two teams was good for building early wins and some momentum. One of the challenges of this approach was that it creased variations in program adoption and maturity. For example, the processes refined through lessons learned as they spread to other teams- leaving some applications behind. We could have rolled out a more comprehensive solution in the first iteration if we had had more eyes on this initially.
Further, the order in which some of the tools could have been more strategic. If we’d done more planning about the end state, we might have put in some of the front-end pieces differently. For example, we’d have been better off if we’d all used Git instead of Subversion. In another example, we’re getting our security scans in with the container builds now. Ideally, we would have prioritized that with the security tools- earlier in the process.
Outcome
- The biggest benefit has been time savings. This equates to more features being developed. Like others, we have a “never-ending” backlog. Before DevOps, developers stayed up until 2 am to run validation scripts off hours on a Friday night.
- “No or very less downtime” deployments have gone way up
- Having data on increased code coverage. They feel better about putting something out quickly that used to take three months in the aging process.
