TDD project summaries (KIXEYE & EA)

I recently spent two years at KIXEYE, building a test engineering group and integrating automated testing into every game team. Significant improvements in development time, scalability and operational cost/quality were directly attributed to the load-testing work I led, driven by metrics for the engineering, QA, producer and executive teams.
After the initial success wave of KIXEYE, operational costs skyrocketed and server failures during peak revenue hours became commonplace; the infrastructure and development processes simply weren’t scaling. The KIXEYE executive team issued a priority one mandate to implement load testing tools and test-driven processes for all titles.
To staff the testing team, we set up intern programs with several schools and hired the best interns out of each batch into our QE group, where we trained them in load testing, application metrics and how to work with each game team. We then embedded these QE engineers into each game team, working with the game teams to establish priorities and continuing to run daily scrums for mentoring purposes and fine-tuning direction.

This approach overcame each game team’s strong initial resistance to load testing: engineering decisions and launch decisions became largely driven by results from the automated tests and embedded performance metrics probes. We then grew the work into performance testing for mobile and web clients.
A quite interesting project for me was Robbie; a friendly front-end to the Maxis automated testing systems. After the success of automated testing in The Sims Online, building a second-generation system, integrated into all teams, became a top studio priority. We first tackled build stability, performance and finally content regression for the studio’s next major release: The Sims 2.0. By embedding myself into the Sims 2.0 team, we found tremendous opportunities for turning test and measure tools into task accelerators for the content team, and by aggregating performance statistics into an executive level dashboard we were able to help focus engineering resources and provide clear cross-team communication. For example, the studio head had been able to start a non-confrontational, very directed conversation on performance, as shown in this simple email: “I see on the dashboard that your frame rate is not where it should be at this point in your project. How are you planning to address this risk in the remaining time before launch?”


Abstract: Iterative Innovation in Games and Gardens

Games and gardens are immersive environments that are designed for aesthetics and enjoyment: goals that are difficult to quantify and harder to engineer. They take years to complete, and it is hard to evaluate success without the user’s viewpoint from inside the immersive environment.

The hurdles: quickly, constantly and cheaply acquiring user, system and process data; preventing defects and implementation gaps from compromising the data; and continually shifting from rapid prototyping to fielded, highly complex systems over years of development and operations.

A cyclic, incremental evolution strategy that draws metrics from the current cycle to guide the next cycle is discussed. This Measure / Change / Measure model provides better focus on tangible problems than the traditional Guess / Change / Hope model, more predictable progress and manages continual growth.
The integration of several agile development strategies is proposed, ranging from test driven development, short, instrumented sprints, metrics driven development and Kaizen process improvement. The hypothesis is that tightly integrating agile strategies into the core of your project via Automation, Architecture and Analytics produces a hyper-agile software system: one can quickly identify problems and quickly shift both code and process, while also casting meaningful projections of future behaviors from current and historical data.

Iterative Innovation is a force multiplier that augments your entire team. Get more done, in less time, with less blood.