Friday, December 9, 2022

Site Reliability Engineering - Part 2

 SRE lifecycle

  1. More involvement in architecture design
  2. Less in Development Stage - inputs on testing
  3. Increase in Limited availability period - Measure reliability and get feedback from users
  4. General availability - full support for support from operations perspective. More reliability inputs got here.
  5. Deprecation stage - Service is abandoned and no more new features added. SRE team continues with support here for old service. When anyone is no longer using it, they will turn the resources for the old services off and help transition to the new service and support for new service.
They provide guidance on
  1. Architecture design
  2. Provide inputs on how to create code that can be reliable
  3. best practices
  4. identify pitfalls
  5. Help define SLA
SRE team should work closely with Development team, DevOps Team and Product Team towards a common goal of reliability.

They should identify the risks, threats and rank them. Monitor and review them.

SRE teams also participate in sprints and focus on Reliability. Key milestones
  1. Production Readiness Review
  2. Launch Meetings
  3. Retrospective meetings
Production changes, outages,

How to measure SRE performance
  1. Impact - Metrics 
  2. Maturity Matrix
  3. Re-evaluate
Meaure
  1. Customer experience - Usability - Server side latency and client side latency
  2. Percentile latency - 
  3. Median latency. Average percentage of times response is valid. Doesn't take slow response into consideration
  4. Long tail latency - includes responses that past the acceptable cut off time as errors
  5. 99 percentile latency - How cut of is decided. based on 99 percentile reliability.
  6. Future Load - Man forecast. How much load might be expected in future. Memory usage, Storage size, bandwidth, queries per second.
  7. Organic growth - Statistical model, based on history
  8. Inorganic growth - This cannot be extrapolated from past data. Ex temporary surge
Service Efficiency
  1. Load Testing should be preformed frequently by SREs.
  2. Should build outage prevention in system so outage of service doesn't occur
  3. Cascading of outages should be prevented
Working Model
  1. Strategic workshop for reliability with Dev and SRE
  2. Service Definition - Consider network architecture, security, reliability, business process etc
  3. Service Design - Dev team create service, SRE focus on infra needs
  4. Testing - Dev, SRE . Involve customer also
  5. Pilot release - Further feedback and get closer to final product

Posted by at