DevOps engineers will collaboratively work with the Development team to deploy and operate the workloads. Their major roles and responsibilities are to help automate and streamline the operations, processes, build and maintain tools for deployment, monitoring, operations, maintenance, and troubleshooting, and resolve issues in Development, Quality assurance, User Acceptance Testing, staging, and production environments. In this article, we will go through and understand what skills does a DevOps engineer need ?
Before the DevOps word was coined in the industry, Our team had already started applying (referred to as “Code deploy and automation” in 2008) DevOps practices and I encountered this new way of thinking for the first time. Exactly 12 years (2008) ago there was no such standard as DevOps, but the people who practiced at that time figured out some related concepts and principles like
- Continuous integration
- Automate the build of a software project
- Automated unit-testing framework and code quality report.
- Every team member is responsible for the software release
- Communicate directly with customers
- Collect and analyse business/application metrics;
- Documentation
Pre-DevOps Era
When I was working (years ago), operation and development were separated from each other. At one end of this gap are hard-working simple developers, and at the other end are system administrators who are like walking dead in the eyes of developers. Sysadmins are not involved in any sort of development work, nor do they communicate with the development team.
They usually get the software packaging from the developers and try to run it in a certain location. Every running attempt is very much painful and if something doesn’t work, reverting back to the previous state was a very frustrating process and more painful. Sometimes system administrators also spend time monitoring the system, reporting all kinds of errors, analyzing database queries, and getting stuck in endless processes.
From my experience many times we have solved the production issues by defining a new variable :), adding a new parameter, placing the correct config, or creating a temp folder with a defined name. I still remember the system administrator will ask what is the name of the product and what language it was written in.
‘DevOps’ jobs a decade ago
Ten years ago, I was working as a software engineer/ Administrator in an R & D team. The company had a flexible mindset. I was given the freedom to raise objections, express my concerns, and my boss (German) encouraged me to come up with a variety of thoughts for solving problems. From that moment on, I also embarked on my journey as a Code Deploy and Automation engineer for straight 1.5 years.
I have proven to the entire team with data that a comprehensive review of the design, development, and operation of the product can indeed bring huge benefits. Also when everyone responsible for the product is aware of the environment in which the software will run, the development process will truly integrate with the production process.
All these were not a culture shock at that time where developers and administrators really joined hands, and everybody understood it cannot be done with a team divided by a traditional process. I have even gone up to playing developer and administrator in a major project. Gone those days in a blink of an eye 🙂
What are the core roles and responsibilities of a DevOps Engineer?
From my perspective, the role of the DevOps can be performed by anyone and no specific requirements for positions or background are required. They could be system administrators, developers, project managers, or even quality assurance personnel. The main goal of DevOps is to bridge the gaps in the continuous integration, delivery, and operation cycles of the software development lifecycle.
If you are from an administrator background, you can easily separate the underlying infrastructure-related factors from patch upgrades, database upgrades, configuration management, or all other distracting or annoying developers. It’s easier for you guys to get started with DevOps.
What skills does a DevOps engineer need ?
What should DevOps engineers know, and what should they do? Below I have covered the most essential skills required for a DevOps engineer. Of course, the list may not be complete and only covers some of the core skills that DevOps engineers need to focus on.
Soft skills and communication
As I stated in my previous post, DevOps is trying to combine operations with development. It’s evident that DevOps jobs require excellent communication skills. Also, you can’t forget empathy. Remember, your empathy shows with a team-first mentality. In any teamwork, conflicts and arguments are very common. At this time empathy is needed, understanding from the perspective of the other party and harmonious cooperation, and quick problem-solving.
When it comes to DevOps skills, we cannot forget integrity. The role of the DevOps engineer is to bring people from various IT departments together. So, you must maintain the highest level of honesty when doing this. Trust among team members is an important quality required to achieve the goals of a DevOps team.
Agile development principles
This is also one of the most important skills in the modern development world (especially in current remote collaborative development scenarios). This includes not only identifying the difference between Kanban and Scrum, but also requires us to be able to communicate smoothly with the team, understand customer value, track time progress, and organize easy-to-understand work logs, independent reports, and the ability to clearly document documents.
Automation
In the DevOps model, Engineers should stop doing manual operations. They should automate all the daily tasks corresponding to automated tools. If you can’t find tools specific to your needs, You can use Jenkins to (CI/CD) Deploy your workloads to respective environments (Dev / QA / UAT / Staging / Prod).
Use programming languages like Go, Python, and bash to write one to automate the manual operations. Say for example, if you need to create a virtual machine image, use Packer. If you need to configure more than 15 hosts, please use Ansible. If you create a Kubernetes cluster in the Google Cloud platform or need to use a CDN (Akamai) on Azure Cloud or Aws Cloud or Google cloud, Please use Terraform to simplify the operation process.
In short, everything from creating infrastructure including compute, storage, and networking resources and deploying container workloads to respective environments need to be automated. In addition, the code you write should be reproducible and idempotent and documented well for better understanding.
Cloud and hybrid architecture
Today, we find most companies distribute their applications among different cloud service providers in order to avoid vendor lock-in issues. Some companies run different parts of their workloads on AWS, Azure, Google with a mix of On-prem data centers. So concentrate on learning to create optimized infrastructure, deploy, test and release the application workloads simply and rudely by the cloud solution.
Always work hard to find the most ideal solution and ensure the service migration between different platforms can be completed within a short span of time. While going through these processes don’t forget about my recommendation on automation mentioned earlier. The higher the degree of automation the lower the difficulty of migration.
Scalability and high availability requirements
The most important thing is to realize what kind of downtime and data loss the enterprise can withstand within a certain period of time. In a real-world scenario, there can be a situation where a sudden spike or dip in the traffic can affect the performance of your application, and sometimes it can cause a server outage (application down) and your application cannot be down for more than 24 hours.
You have to make this clear before finalizing the availability requirement (Explore RTO and RPO). With the help of cloud services and containerization technology, it has become very easy to expand the scale of the system. However, the infrastructure and services themselves also need to be prepared for this flexible scalability.
Monitoring and alerting
In order to make a timely review, forecast, and response, it is of course necessary to collect all available indicators in the system, application, and business. These indicators are like the eyes of the team and cannot be fully achieved with a single monitoring solution. Each cloud service or platform provides its own set of available indicators and alarms, but you also need to use external systems such as New Relic or Datadog in combination with your needs or build custom monitoring services on Prometheus. In short, all choices should be based on a reasonable budget, time, and task requirements.
Security
Security assurance is indeed the core responsibility of a DevOps engineer. However, DevOps security (DevSecOps) has to be built into the process and designed into the application at an earlier stage of the SDLC. This is also called shift-left security, ie Start injecting security in the development lifecycle rather than performing in pre-production or before a delivery process.
The goal is to increase quality, reduce the amount of time required for testing, and, perhaps most important, mitigate the risk of security problems at the end of the cycle when it is much more expensive and time-consuming to make fixes.
Troubleshooting
DevOps engineers are often involved in day-to-day operations, and such senior DevOps professionals are typically expected to provide extensive problem-solving support in a team. Specifically this can include deploying, managing, and troubleshooting container and non-container workloads running in remote or hosted locations. Troubleshooting can also extend to infrastructure such as server, storage and network issues.
Role and responsibilities of a DevOps engineer
So, what do devops engineers do every day? A typical devops engineer responsibilities list looks like this
- Design, build, test and deploy scalable distributed systems to automate development to deployment
- Manage code bases (such as Git, BitBucket, etc.), including code merging and integration, branching and maintenance, remote library management, etc.
- Design database architecture and database objects, and synchronise various environments
- Design and Implement DevOps continuous integration and continuous delivery pipelines
- Research and implement new technologies and practises
- Research and Brainstorm for new ideas and ways to improve development delivery, technologies and practises
- Document processes, systems, and workflows, and update existing processes.
- Create and manage dynamic monitoring and alerting solutions.
- Continuously automate the manually performed tasks
- Create and enhance continuous deployment automation based on Docker and Kubernetes
- IT infrastructure maintenance and management (including hardware, software, networking, storage, virtual and remote assets, and cloud data storage)
- Performance testing and benchmarking.
- Prod Monitoring (Analysing – Logs and finding error patterns)
- Available for on-call rotation 24/7 for production outages and rollouts
Of course, no two jobs are the same, and the role and responsibilities of a devops engineer varies from company to company. The size of the organisation may have specific implications. In a small business, your responsibilities may span the entire devops lifecycle, while in larger teams you may end up focusing on a subset of them.
Conclusion
Depending on the current world use cases of the software or application being developed and used, there may be various SME’s for security, Infrastructure, process and project management, etc will be frequently involved but not necessarily a full-time employee of the DevOps team.
On any given day, DevOps will be the first point of contact for any customer issues and they will be connected to multiple silos in an organisation from Dev,, QA Teams, and Project/Product Management Teams.
