Infinite Undo! RSS


 
 

Hello, I'm Noah Sussman

Jul
18th
Fri
permalink

Please stop trying to sell me on management positions while offering a “QA Engineer” title and salary

Recently I was sent a job posting that embodied what I will call the Pernicious Myth of the QA Automation Engineer. Here’s just one line:

Own automated testing capabilities across Web, mobile and production processes.

Um.

An engineer cannot by definition take responsibility (that’s what “ownership” means — right?) for features that cut across multiple groups. In all but the smallest companies, Web and mobile would be managed by different teams. With different MANAGERS.

Engineers don’t take responsibility for the behavior of MULTIPLE managers across different teams.

This is a Director’s job.

Managing software capabilities across multiple teams is the job of a mid-level Engineering manager such as a director or (in larger organizations) a VP.

Why not? Because decades of computer engineering experience show that it never works.

Management (with all its wondrous hierarchical levels) is responsible for behavior of people within and across teams. Engineers and designers are responsible for the behavior and organization of the product. Not the people. People are a management problem. Especially at scale. Organizations that forget this fail.

Takeaway: do not sign on for a director’s job at an engineer’s salary

I’m not saying no one should take responsibilty for the “tests and testability” of an application or service.

What I am saying is that someone should be explicity responsible for testing across the whole organizaiton and that person should be at the director or executive level. Never at the engineer or team lead level. Ever.

Jul
17th
Thu
permalink

Who owns testing?

There is a widespread misconception that QA should own testing. This is false. Not only is it false but it doesn’t even make sense. You cannot have a discrete group that “owns” testing because testing is integral to the activities of both engineers and designers. Organizations often make a choice to ignore this, but the choice is based on cultural inertia rather than science.

Engineering and Product always owns testing because testing is a design process. If QA performs all testing and devises test plans, then that’s an explicit decision by Engineering and Product groups to try not to own testing.

As an engineer-or-designer, you can try not to own testing, but it doesn’t work. Because testing is part of the engineering and design process. Testing is not an activity in itself. Rather it is an aspect of the larger activity of designing and building software systems.

permalink
There is a very deep disconnect in the software industry with regard to QA and Testing.

There is a very deep disconnect in the software industry with regard to QA and Testing.

Jun
1st
Sun
permalink

The problem with asking where testing/qa “fit in” to devops

The problem with asking where testing/qa “fit in” to devops is that testing/qa is part of dev.

It’s a historical mistake that test/qa was marginalized to the point it’s now seen as a separate discipline.

May
30th
Fri
permalink
permalink
permalink
permalink
permalink
permalink
If you want your functional test suite to run faster, work out how to write smaller tests that achieve the same effect.
— Simon Stewart
permalink
All dependencies of a method should be obvious from looking at the method signature.
— Sebastian Bergmann
permalink
Static analysis is a good replacement for 100% [test] coverage… not that you shouldn’t get to 100% coverage
— Rasmus Lerdorf
permalink

CI systems as a tool for scaling communication between engineers

In the tech world we worry a lot about scaling. Whenever someone comes up with software innovation, one of the first questions they are likely to hear is: nice concept, but will it scale?

But what is it that does not scale?

In growing systems, technology and process may not scale. In the life cycle of a mature / legacy / successful system, it is communication that does not scale. Communication between individuals and between teams, is universally found to be the bottleneck on execution in very large organizations.

Therefore the fact that CI can scale engineer communication almost indefinitely is critically important! It means that CI is a tool for dealing with diseconomies of scale, at least as pertains to an engineering organization.

Any "IT Crisis" then can be re-understood as hitting the steep rightward end of the diseconomies of scale curve (shown below) with costs spiking at the beginning and end of the life of the organization. The spike at the end is due to communication costs, which again: CI mitigates communication costs, at least for an engineering team.

illustration showing the cost curve for a diseconomy of scale: costs steeply dropping on the left, followed by a period of flat per-unit cost, followed at the extreme right edge of the graph by a sharp *increase* in per-unit cost as the organization grows in size

permalink

PHP Static Analysis with SonarQube: everything you need to know to get up and running on CentOS 6

example of a treemap plot of lines-of-code vs. test coverage

SonarQube (formerly known as just “Sonar”) is a tool for finding "interesting" areas of your codebase.

PHP support in SonarQube is relatively new, but I was able to get it set up with a little googling. So I thought it’d be worthwhile to share my notes.

I’ve embedded the Gists containing my configuration steps, below:

Or if you prefer you can view my SonarQube / PHP install instructions on GitHub.

permalink
Flaky red build detector

The diagram above illustrates my preferred algorithm for detecting intermittent test failures in a CI system.

If you are building logic like this, you should first ask why are the tests flaky in the first place?

Here are 8 articles where different authors explore various approaches to flaky tests:

No more flaky tests on the Go team
What are the roots of ‘clowning’ or ‘clowny behavior’ or ‘clowntown’ at Facebook?
How to solve intermittent issues
Your test suite is trying to tell you something
Testing Problems Are Test Results 
Buildbot and Intermittent Tests
Did you ‘try’ it before you committed?
Eradicating Non-Determinism in Tests

Flaky red build detector

The diagram above illustrates my preferred algorithm for detecting intermittent test failures in a CI system.

If you are building logic like this, you should first ask why are the tests flaky in the first place?

Here are 8 articles where different authors explore various approaches to flaky tests:

  1. No more flaky tests on the Go team
  2. What are the roots of ‘clowning’ or ‘clowny behavior’ or ‘clowntown’ at Facebook?
  3. How to solve intermittent issues
  4. Your test suite is trying to tell you something
  5. Testing Problems Are Test Results
  6. Buildbot and Intermittent Tests
  7. Did you ‘try’ it before you committed?
  8. Eradicating Non-Determinism in Tests