kubernetes event-driven pods autoscaling with keda and rabbitmq

By
admin
-
0
799

hi sobat sekolahlinux, in this article i will tell you about how to implement kubernetes event-driven pods autoscaling with keda and rabbitmq,

RABBITMQ

I assume you already have and run rabbitmq with detail below

rabbitmq_host: 192.168.99.150
rabbitmq_user: admin
rabbitmq_pass: qwerty12345
rabbitmq_queue_name: sekolahlinux

you can generate queue with python script, but you must install pika library first

python3 -m pip install pika --upgrade

and you can use script python send.py below for send or produce message to queue name sekolahlinux

#!/usr/bin/env python
import pika
import sys

parameters = pika.URLParameters('amqp://admin:qwerty12345@192.168.99.150:5672/')
connection = pika.BlockingConnection(parameters)
channel = connection.channel()

channel.queue_declare(queue='sekolahlinux', durable=True)

message = ' '.join(sys.argv[1:]) or "Hello World!"
channel.basic_publish(
    exchange='',
    routing_key='sekolahlinux',
    body=message,
    properties=pika.BasicProperties(
        delivery_mode=pika.spec.PERSISTENT_DELIVERY_MODE
    ))
print(" [x] Sent %r" % message)
connection.close()

after you create send.py you can run with command below, if you run command below once, you can create 1 queue length, and if you run again you have 2 queue length in sekolahlinux queue, repeat for create more queue length for triger hpa base on queue length with keda

python3 send.py test1
python3 send.py test2
python3 send.py test3
python3 send.py test4
python3 send.py test5
python3 send.py test6 ...

after you run send.py you will create queue name sekolahlinux with queue length 6 like image below

KEDA

for setup keda you can use helm, for detail you can visit link below

after you deploy keda on you k8s cluster, now create manifest deployment.yaml like below

apiVersion: apps/v1
kind: Deployment
metadata:
  name: php-apache
spec:
  selector:
    matchLabels:
      run: php-apache
  template:
    metadata:
      labels:
        run: php-apache
    spec:
      containers:
      - name: php-apache
        image: k8s.gcr.io/hpa-example
        ports:
        - containerPort: 80
        resources:
          limits:
            cpu: 500m
          requests:
            cpu: 200m

and before you create manifest keda.yaml you must encode your rabbitmq url connection with base64 like below

echo -n "amqp://user:user@192.168.99.150:5672/"|base64

after you run command above you will get result like below

YW1xcDovL3VzZXI6dXNlckAxOTIuMTY4Ljk5LjE1MDo1NjcyLw==

now you can create manifest keda.yaml like below, and use result from encode base64 above

apiVersion: v1
kind: Secret
metadata:
  name: keda-rabbitmq-secret
data:
  host: YW1xcDovL3VzZXI6dXNlckAxOTIuMTY4Ljk5LjE1MDo1NjcyLw==
---
apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
  name: keda-trigger-auth-rabbitmq-conn
  namespace: default
spec:
  secretTargetRef:
    - parameter: host
      name: keda-rabbitmq-secret
      key: host
---
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: rabbitmq-scaledobject
  namespace: default
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: php-apache
  minReplicaCount: 1
  maxReplicaCount: 4    
  triggers:
  - type: rabbitmq
    metadata:
      protocol: amqp
      queueName: sekolahlinux
      mode: QueueLength
      value: "4"
    authenticationRef:
      name: keda-trigger-auth-rabbitmq-conn

 

now we can apply the manifest with command below

kubectl apply -f deployment.yaml
kubectl apply -f keda.yaml

if you want simulate the pods autoscaler you can run send.py 4x or more for triger pods autoscale

reference: