Kafka / Docker Tips

This page just contains some useful Kafka and Docker information I refer to from time to time.

Kafka Tips

How to get a list of Kafka topics?

docker exec -it kafka /opt/kafka_2.13-2.8.0/bin/kafka-topics.sh --bootstrap-server localhost:9092 --list

How to get info on a Kafka topic?

docker exec -it kafka /opt/kafka_2.13-2.8.0/bin/kafka-topics.sh --bootstrap-server localhost:9092 --describe --topic core-eventingest

How to create a Kafka topic?

docker exec -it kafka /opt/kafka_2.13-2.8.0/bin/kafka-topics.sh --bootstrap-server localhost:9092 --create --topic core-events --partitions 3 --replication-factor 1

How do I get a list of the Kafka bin folder?

docker exec -it kafka ls /opt/kafka_2.13-2.8.0/bin/

How do I consume a topic from the console?

docker exec -it kafka /opt/kafka_2.13-2.8.0/bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic events --formatter kafka.tools.DefaultMessageFormatter --property print.timestamp=true --property print.key=true --property print.value=true

Docker

CommandPurpose
docker-compose up|downbrings docker images up or down as per the yaml file
docker-compose rm -svfremoves stopped service containers
docker imagesdisplays a list of docker images
docker psshows docker images currently running
docker container ls –alllists all docker containers
docker volume lslists all docker volumes
docker container rm idremoves the container with the specified id
docker volume rm idremoves the volume with the specified id
docker stop idshutdown the container with the specified id
docker kill $(docker ps -q)shutdown all running containers
docker ps -q | xargs docker stopanother way to shutdown all running containers
docker exec -ti id cmdexecutes the specified command in the container with the specified id. For example:
docker exec -ti 7717da13fcbc sh -c "echo a && echo b"

Basic ASP.NET 6 DockerFile

The following examples are modified from Les Jackson’s excellent free course, see here.

Add Dockerfile to your project, in the root folder:

FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build-env
WORKDIR /app

COPY *.csproj ./
RUN dotnet restore

COPY . ./
RUN dotnet publish -c Release -o out

FROM mcr.microsoft.com/dotnet/aspnet:6.0
WORKDIR /app
COPY --from=build-env /app/out .
ENTRYPOINT ["dotnet", "<service name>.dll"]

Docker Commands for a Basic ASP.NET 6 Application

Execute the commands from a terminal, from the root folder of the project, where the .csproj file is located. Docker Desktop should be running. You can create a docker hub id here.

docker build -t <docker hub id>/<service name> .

docker run -p 8080:80 -d <docker hub id>/<service name>

docker ps

docker stop <container id>

docker start <container id>

docker login

docker push <docker hub id>/<service name>

Example Kubernetes Deploy file for the Basic Application

From the same course, a sample Kubernetes deploy file named platforms-depl.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: platform-depl
spec:
  replicas: 1
  selector:
    matchLabels:
      app: platformservice
  template:
    metadata:
      labels:
        app: platformservice
    spec:
      containers:
        - name: platformservice
          image: beyondvelocity/platformservice:latest

Example Kubernates NodePort file for the Basic Application

From the same course, a sample Kubernetes NodePort creation file named platforms-np-srv.yaml:

apiVersion: v1
kind: Service
metadata:
  name: platformnpservice-srv
spec:
  type: NodePort
  selector:
    app: platformservice
  ports:
    - name: platformservice
      protocol: TCP
      port: 80
      targetPort: 80

KubeCtl Commands for the Basic Application

With Kubernetes enabled within Docker Desktop, use the following comands:

kubectl version

kubectl apply -f platforms-depl.yaml

kubectl get deployments

kubectl get pods

kubectl delete deployment <name>

kubectl apply -f platforms-np-srv.yaml

kubectl get services

Installing Docker Compose in Ubuntu

There’s plenty of online help for installing Docker, but for Docker-Compose 2.0, help was harder to find, these links were useful:

Install Docker in Ubuntu 21.10, Install Dockercompose on Ubuntu 21.10

How to Install the Lastest Docker Compose on Linux

Host Configuration

Add the following hosts to /etc/hosts:

127.0.0.1 kafka
127.0.0.1 kubernetes.docker.internal
172.17.0.1 host.docker.internal

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s