DevOps Enterprise Summit 2020 – Notes on Compliance

Last month, I was lucky enough to attend the London edition of the DevOps Enterprise Summit 2020. Like most events this year, I got to watch the presentations from the comfort of my living room even though I was planning to go to London for it. The reason this conference is particularly interesting to me is two-fold; first, because it is addressing DevOps challenges in large enterprise and secondly because it has dedicated track to compliance governance. As someone working on a compliance service in a big company, this was a perfect event for me to have a look at the industry’s trends.

So here are my notes on 3 presentations I attended, with the focus on handling compliance in DevOps.

Bureaucracy and DevOps by Mark Schwartz (AWS)

When hearing the word “bureaucracy” most of us get a bit anxious and irritated. In his presentation, Mark Schwartz highlighted the benefits of setting up rules to be followed by everyone and how DevOps help that these rules don’t get in the way of the actual work that needs to be done.

He then proceeded to make the bold claim,

DevOps is a kind of bureaucracy.

He backed it up, by noting that DevOps principles like automating repetitive tasks, align with bureaucratic benefits like: fairness – the rules are the same for everyone, predictability – if you do what you are required, you get what you need, consistency – everyone uses things the same way.

DevOps also help remove some of the burdens that come with bureaucracy, for example by getting fast feedback on the process, it is easier to adjust the rules. By creating “minimal viable bureaucracy” when automating compliance rules, we remove waste and reduce “red tape” instances.

Bureaucracy is a value stream and the product is compliance.

In summary, his message was to take our bureaucracy, automate it and make it enabling.

Low Context DevOps: 3 Ways to End Knowledge Frustration by Thomas Limoncelli (Stack Overflow)

Thomas Limoncelli framed his presentation around high and low-context cultures, an anthropology concept. You are in a high-context environment when you are supposed to know how things work just because you do, for example in family gatherings. On the other hand, in low-context environments all the rules are explicitly set out for you.

He proposed that a DevOps environment should strive to be low-context and suggested three ways to achieve this.

2020-06-27_11-40-08

The defaults are how you expect most people to work, so defining them and supporting them “out-of-the-box” is something to go for. His second point was to embody and “standardise” the recommended practices for all of our tools, from the CI/CD pipelines to our bug ticketing systems. Or simply put:

The lazy path guides you to the right way.

The final point was to provide information exactly where people need to see it. Adding a URL pointing to a meaningful explanation is cheap, so why not use it? It also implies that information needs to be documented along with development, so that knowledge is shared and the context is reduced.

I believe compliance is a good candidate to lower its context. The defaults should address the company’s main risks. They should be machine readable and easy to check against. And finally, they should be easy to explain and versioned so that they are always up to date.

DevOps and Internal Audit: A Great Partnership by Rusty Lewis & Clarissa Lucas  (Nationwide Insurance)

In their presentation, Clarissa Lucas and Rusty Lewis set out to answer the question that many teams dread to ask:

How can we pass an audit when we are moving to a DevOps operating model?

Using the example of the risks that accompany the change management process, they walked us through how controls such as Approvals and Segregation of Duties can be evaluated from a DevOps perspective.

When thinking about Segregation of Duties what we usually have in mind is one human handing over to another, for example one developer writes the code and a different one reviews it. To shorten the time and efficiency of the control, we can implement some automated checks that we deem necessary for a review and leave only the critical thinking part to the reviewer. And we can go further, by logging any deployed changes and verify if they belong in a pre-approved low-risk list. In both cases, we mitigate the change management risk by automating some of the steps, thus enabling fast deliveries in a compliant way.

They summarised their findings in the slide below:

2020-07-05_13-12-10

For me, the biggest takeaway and the biggest challenge is to enable the collaboration between development and audit teams. Development teams need to understand the risks to the enterprise and find efficient ways to implement controls while the audit teams need to look beyond their existing checklist and be open to evaluate them.

Some conclusions

Whether we like it or not, there are always some rules we use to build our software. Some we choose to follow and some we must. But all three presentations mentioned here made the point that having such rules is not something inherently bad. Automating what makes sense to automate, allows for more time to invest in “custom” quality for a product, which brings even more value to its users. The “DevOps Automated Governance Reference Architecture” paper (access to Forum papers), provides invaluable information on how to achieve that technically. But in the end, it’s up to all of us – developers, auditors or management- to find ways to uphold the rules in a sustainable way. And we can do that only by working closely and honestly together.

Shifting my Quality Perspective: From a Quality to a Product Owner Role

Not having the word “quality” in my job description is something new for me. I went from “Quality Engineer”, to “Quality Product Accountable”, to “Quality Coach”, always being directly involved in the quality of the product or the process. Now as a product owner I am trying to figure out what is my role in supporting the quality of our product.  Here are some things I’ve noted down.

Not directly involved in the test strategy

In the team I am working in, we don’t have a dedicated tester or someone in a quality role. The developers decide on the test strategy. What to test, when and how is entirely up to them. I have a very high level idea of how it looks like, but I don’t have the time to deep dive and understand it properly. I can only experience the outcomes of the strategy in the form of increase of usage of our tool or bugs reported by our users. Even if  I would like to find out how testing is done, I don’t have the time to do so. The quality of the process is no longer my priority, which makes me feel a bit awkward.

Not involved in functional testing

Even if I am tempted to test the inside-outs of any new feature we want to release, I need to hold myself back and simply not do it. If by the time I have a look at new functionality the straightforward scenario paths are not working, we have all done something very wrong. My main testing efforts concern whether what we are building fits the user’s expectations, so mostly I spend my day talking to them to try to figure out what they need. Nevertheless, from time to time I do some exploration on our product, to identify gaps or possible improvements in the workflows.

Acceptance criteria – not as easy as I thought

As a tester I complained (a lot) about not well defined acceptance criteria. Now, as I am the one trying to identify them, I realise what a complicated task this is. Not all users have exactly the same process of doing things. Figuring out what a “correct” flow of actions should look like, especially for a new product, can be daunting at times. I have in mind techniques like example mapping that can help, but I have not figured out how to incorporate it in our Scrum process. Another aspect that confuses me, is how to improve the acceptance criteria during development, as sometimes our original assumptions might be wrong. I try to be involved as much as I can in development, but it doesn’t always work out.  This part is really work in progress for me.

Providing vs receiving information

As a tester within a development team I would test, collect my findings and provide this information to whoever needed it to make a decision. Now, I find myself on the other side, making the decisions based on the input from the team. This is the most complicated aspect for me, as it boils down to me a tester, trusting the information provided by the team, aka the developers. This is definitely a steep learning curve. The instinct of questioning the quality of the information is alive and kicking. But I find that slowly, as we work longer together as a team this gets better. I am positive that a fair amount of scepticism from my side will always be there, but at least it will be directed to the correct things to question and not everything.

I find myself wondering why in Scrum, the PO is not a member of the development team. Wouldn’t be more efficient, from a quality point of view, if POs were more involved in the development process? I try to participate as much as I can in all development discussions, but in the end it might be just a time availability constraint.

Identifying shifts in my perspective of quality, helps me use my testing skills to serve my needs as a PO. I try to focus on that rather than become a PO whose main concern is testing. But it seems that the road is long and bumpy.