Client Background
Our client, a gaming company, is a Japanese multinational consumer electronics and video game company headquartered in Kyoto, Japan. The American branch, established in 1980 and based in Redmond, Wash., is a wholly-owned subsidiary of our client, Ltd. They are committed to delivering best-in-class products and services to the customers and investing in the well-being of their employees.
Challenges
Our overarching goals were to have all next-generation portfolios adhere to Agile methodology. We set out to improve the alliance between product owners, developers, testers, cybersecurity, and infrastructure. We’ve largely reached those goals and have seen the following benefits:
- Increase in embracing DevOps culture to streamline product deployment
- More rapidly delivering new features to production
- Minimizing manual involvement in building and releasing a softwarechange, and
- Increasing visibility into key metrics to determine code promotion to the next environment
Approach
BrickRed started DevOps engagement with a client in 2016. Our approach was grounded in four objectives: make incremental changes, focus on collaboration, remove organizational barriers whenever possible, and continuously improve. These helped shape the specific actions taken in the six completed phases.
Phase 1
We established our initial operating capability by implementing an enterprise software versioning and source code management platform. We used Azure Devops solutions for a build platform that automated the deployment process. In addition, we provided coaching for migrating existing products to the new platform.
Phase 2
We focused on improving build automation by implementing a Build Once Deploy Anywhere (BODA) framework. We defined critical metrics for all development teams and published their results on Power BI dashboards for transparency.
Phase 3
We fully automated testing in all environments. We focused on improving timely bug detection (fast and reusable) and improving test coverage. When Phase 2 was complete, we could run most test plans 24×7 unattended.
Phase 4
We fully automated the deployment process and released it more frequently with repeatable deployments.
Phase 5
We focused on establishing quality gates by continuously testing every release candidate for quality and security. We concentrated on detecting and remediating defects and critical security vulnerabilities earlier during development. We implemented a one-click automated build and deployment process.
Phase 6
Most recently, we have been working on continuous improvements. We are optimizing the platform to capitalize on emerging technologies. We are facilitating and extending enterprise
Lessons Learned
Loosely Coupled Architecture: When we started onboarding teams to the DevOps platform, a Build Once Deploy Anywhere (BODA) framework was defined, ideal for loosely coupled architectures. We quickly learned that teams had to refactor to adhere to the BODA framework, which caused lots of work for teams to refactor their code. We have changed the approach now and work closely with the Enterprise Architecture so that teams starting development can develop non-monolithic applications that can be managed efficiently.
Tools: Earlier, there were manual steps to build a Production-Ready release candidate. We have introduced an automated deployment CICD pipeline for all teams that eliminates manual steps, delays, errors, and rework.
Process: We did not have a set of processes defined when the platform was stood up. We learned that to manage a repeatable build & deploy process effectively; we had to invest in the right processes & procedures upfront. The Automated Deployment Pipeline framework & DevOps Policy & Procedures documents were created to guide SB&D software products teams to utilize DevOps & the Automated Deployment Pipeline fully.
- If an agency has decided to use a CICD pipeline, all new projects must be onboarded from the “get-go” instead of retrofitting them later. Using this approach saves us a lot of rework at a later time. It provides the necessary automation from the start and more comprehensive code coverage.
- Dissolving the organizational silos is critical. All divisions, offices, branches, etc., must be equally tasked to meet DevOps objectives. It can’t just be one organization’s goal and not the others.
Outcome
- Fewer organizational silos make it easier to bring the best available resources together. Increased flexibility and enhanced transparency to manage risks or issues
- Better quality through improving change management and enhanced testing
- Over 10,000 hours of manual/personal work have haven automated by the DevOps pipeline annually.
