Log aggregation in Kubernetes using Grafana Loki

Well, all of us have been using Grafana for quite a while. It allows you to query, visualize, alert on, and explore your metrics no matter where they are stored. In fact, I personally liked the colorful dashboards that almost support all the time-series database (TSDB) data. Grafana also has rich support for various input sources like Prometheus, InfluxDB, AWS Cloud Watch, Azure Monitor, etc. But what about log aggregation? How do you aggregate your logs from your applications? Using another solution like ELK, EFK, etc. What if I tell you that the…


Effective cost-saving for Apache Spark workloads on EKS with AWS Spot Instances

Apache Spark is a data processing framework that can perform rapid processing tasks on huge datasets and can distribute the data processing tasks across multiple nodes. With the rapid containerization of applications, organizations have also started running spark on a containerized platforms like Kubernetes and have also been using them in production. Ideally, there are two ways of running spark on Kubernetes.

a) Spark Submit: This is a single script used to submit a spark program and launch the Kubernetes cluster application.

Example:

b) Spark Operator: The…


Hey Mohan, Yes I had a similar issue while implementing this. Upon Investigation, I found out that Chaos Mesh only supports the following runtimes. The runtime should either be docker or containerd ( https://github.com/chaos-mesh/chaos-mesh/blob/master/helm/chaos-mesh/values.yaml#L112 ). That is the reason why I have created my cluster using kops with this flag explicitly specified ( --container-runtime docker ). Hope this helps.


Serverless in Kubernetes using Kubeless

Well, how many of you have come across the term serverless at least once in your IT journey? Boom, I guess a lot of you might have heard about this term and most of them are also using these concepts to run their production environments. Do you also have your applications running in Kubernetes? If yes, Serverless and Kubernetes make a deadly combination. I have heard tales from a few of my peers saying that it really makes it harder to integrate Kubernetes with any cloud agnostic serverless component because it might make their migration…


Get free and Automatic SSL certificates using Cert manager and Let’s Encrypt

In today's scenario, SSL certificates are the most important part of Deploying an application to the Internet. It is only the most important attributes that determine whether your websites are safe or not.

Image Credits: Google

This paddle lock symbol conveys to your customer that the website they are visiting is safe, secured, and verified. So how do you actually achieve HTTPS on your website?

HTTP + SSL = HTTPS.

Getting an SSL certificate is not that easy. Moreover, it is expensive too. In today's world where Kubernetes is Predominantly everywhere…


I guess, probably you can create a storage class by adding the following parameter WaitForFirstConsumer. This will delay the binding and provisioning of a PersistentVolume until a Pod using the PersistentVolumeClaim is created. More info here:

https://kubernetes.io/docs/concepts/storage/storage-classes/#volume-binding-mode


Chaos Engineering in Kubernetes using Chaos Mesh

With your applications being migrated to the cloud, the architecture has become really complex. With such complex architecture of your applications in the cloud, it is really difficult to predict the failures. Any such failures in the application can cause an expensive outage to your company. These constant unpredictable outages can defame the companies reputation and might lose customers too. Companies must explore an option to predict such outages rather than fixing them during the next outage. And here comes Chaos Engineering to our rescue. Chaos Engineering is a disciplined approach to Identifying…


Monitoring Kubernetes Workloads with Thanos and Prometheus Operator

Are your applications running on Kubernetes? Is it highly scalable and you are happy with the way it works? Wait a minute, How are you monitoring them? Ahh, Prometheus Right? Cool, Did you ever wonder how scalable and Highly available your Prometheus Cluster Is? Before that, here is a mail from your boss asking you to find out the number of http_requests that your website received last Xmas or Let's make this the Indian Style. Your boss wants to know the number of customers who had visited your website ( total number…


Monitoring Kubernetes Workloads with Thanos and Prometheus Operator

In Part I of this article we have understood the various components of Thanos and its use cases. In this Part II, we will configure Thanos with GCS ( Google Cloud Storage ) and understand how metrics can be retained for longer periods using Thanos. We will also configure Grafana to use the Thanos Query Frontend to visualize graphs from various clusters ( Thanos Queriers ).

Image Credits: Thanos website

What is the entire story all about? (TLDR)

  1. Install Thanos using the Bitnami Helm chart.
  2. Configure Thanos to use GCS as its Object store.

Prerequisites

  1. A Kubernetes cluster ( Can be either On-Prem, AKS, EKS…


Security checks in Kubernetes Cluster Using Kube-bench and Kube-hunter

We are now living in the Kubernetes world. Wherever you go the only word that you hear is Kubernetes. Well, how far is this safe and reliable. Is the Kubernetes cluster production ready? Everyone merely cares about how to create the Kubernetes cluster and how to deploy the application to the Kubernetes cluster, Once the cluster and ready and once the application is deployed they move on. The main aspect that is missing here is Security. Does anyone still remember that securing the Kubernetes cluster is also a part of the…

Pavan Kumar

Cloud DevOps Engineer at Informatica || CKA | CSA | CRO | AWS | ISTIO | AZURE | GCP | DEVOPS Linkedin:https://www.linkedin.com/in/pavankumar1999/

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store