About me

• Michael Lang
• SAP SE (Walldorf / St.Leon Rot)
• CLM - Cloud Lifecycle Management
• Perl Developer

CLM - Cloud & Lifecycle Management

Developing tools and cloud services to orchestrate the application and content lifecycle of SAP solutions

• 1000+ employees

CLM - Cloud & Lifecycle Management

Service Provider Cockpit (SPC)

The platform at SAP that covers all aspects of cloud management.

• 200+ developers

CLM - Cloud Lifecycle Management

Vega

CLM - Cloud & Lifecycle Management

Vega

• started as an adapter for SPC

• reduce complexity / easier onboarding (REST vs. SOAP)

• reduce traffic to SPC (e.g. deduplication)

• status cache

• 2 developers

CLM - Cloud Lifecycle Management

Vega

Testing with Travis CI

language: perl
dist: bionic
before_install:
    - sudo apt-get update
    - sudo apt-get install libssl-dev -y
    - git submodule update --init --recursive
perl:
  - '5.22'
  - '5.30'
  - '5.36'
env:
  matrix:
    - TEST="build/prove_all.pl"
install:
  - build/update_cpan_dependencies.pl 
script: $TEST
...

Travis CI - Sunset Annoncement

Dear Travis CI users,

As per our earlier email on Dec 20, 2022, we informed that we plan to sunset Travis CI by May 2023. As a first step, we plan to start the migration from travis-tools@cloud.sap.corp to Github Actions. The Tools team has prepared a detailed step by step guide to migrate to Github Actions: Migration Guide from Travis CI to GHA

…

• sunset Travis CI by May 2023
• migration to Github Actions

Github Actions - Marketplace

https://github.com/marketplace/actions/setup-perl-environment

Github Actions - Marketplace

️ shogo82148/actions-setup-perl

Github Actions - Marketplace

shogo82148/actions-setup-perl

Github Actions - Self hosted runners

Migration Guide

The runner VMs do not have a lot of software installed apart from docker. Also during native workflow execution, there is no root access to the VMs for installing additional software. Therefore, the best way to run builds is via docker containers that have all of the required build tools. If your build tools themselves need to run docker containers, it is possible to map the docker socket of the host VM into a container thus avoiding docker-in-docker.

• slim VM -- no additional software installable
• docker is the way to go

Requirements

requires "Mojolicious";
requires "CryptX";
requires "Net::SSLeay";

use Mojo::Base -strict;
use Mojolicious;
use Net::SSLeay;
use CryptX;
my $imagename = shift;
say "[$imagename]";
say '-> ' . join( ' - ',
  "Perl: $^V",
  "CryptX: $CryptX::VERSION",
  "Mojolicious: $Mojolicious::VERSION",
  "Net::SSLeay: $Net::SSLeay::VERSION\n" 
);

testscript to build and test the images

...
if [[ "$COMMAND" == build ]]; then
  $DOCKER build -f docker/$image.Dockerfile -t $image:$TAG . 
elif [[ "$COMMAND" == test ]]; then
  $DOCKER run --rm -it $image:$TAG perl test.pl $imagename
fi
...

Dockerfile - 01-docker-perl-bookworm

FROM perl:5.38.2-bookworm
WORKDIR /src
COPY src/cpanfile .
RUN curl -fsSL https://raw.githubusercontent.com/skaji/cpm/main/cpm > cpm
RUN chmod +x cpm
RUN ./cpm install -g --show-build-log-on-failure 
COPY src/*.pl .

• Official Perl Docker Image
• based on Debian 12 (bookworm)

https://github.com/Perl/docker-perl

Dockerfile - 01-docker-perl-bookworm

michel@x1c7th:/media/michel/gpw2024$ ./run -i1 build
STEP 1: FROM perl:5.38.2-bookworm
STEP 2: WORKDIR /src
--> Using cache ffc072450cce987d7e4a58138e8ef258e7ad10824d08f34966fa05ed0385ab9e
STEP 3: COPY src/cpanfile .
--> Using cache 2c49cfbf499cb300912f464316ec1839e6020c3e1f2ffc789bb7f4fb8aaa82d9
STEP 4: RUN curl -fsSL https://raw.githubusercontent.com/skaji/cpm/main/cpm > cpm
--> Using cache 1b688394de3f87e1b2cb5c889a567a62a65dec617754ce9ccdd033e05640dfbd
STEP 5: RUN chmod +x cpm
--> Using cache 28dfdc2a644eba2270fcd2ef6b24bc24a773b81738f5135f99add2b70c0e32fa
STEP 6: RUN ./cpm install -g --show-build-log-on-failure 
--> Using cache 46febec70add6fd76026e35d077f826eef2164152ce47b784e7cc16b053cf330
STEP 7: COPY src/*.pl .
STEP 8: COMMIT 01-docker-perl-bookworm:test
--> 92afb30ad0b
92afb30ad0b71cd5cbd269e722b7c520509a04714b681c90402782844c84c2ba

Dockerfile - 01-docker-perl-bookworm

michel@x1c7th:/media/michel/gpw2024$ ./run -i1 test
[01-docker-perl-bookworm]
-> Perl: v5.38.2 - CryptX: 0.080 - Mojolicious: 9.36 - Net::SSLeay: 1.94

michel@x1c7th:/media/michel/gpw2024$ docker image ls
REPOSITORY                                        SIZE
localhost/01-docker-perl-bookworm                 1.07 GB

• test ok
• image size 1.07 GB

Dockerfile - 02-docker-perl-bookworm-slim

FROM perl:5.38.2-slim-bookworm
WORKDIR /src
RUN apt-get update 
RUN apt-get install -y curl build-essential
RUN curl -fsSL https://raw.githubusercontent.com/skaji/cpm/main/cpm > cpm
RUN chmod +x cpm
COPY src/cpanfile .
RUN ./cpm install -g --show-build-log-on-failure 
RUN apt-get purge -y curl build-essential
RUN apt-get autoremove -y 
COPY src/*.pl .

• based on Debian 12 (bookworm-slim)
• compiler toolchain needs to be installed (build-essentials)

Dockerfile - 02-docker-perl-bookworm-slim

michel@x1c7th:/media/michel/gpw2024$ ./run -i2 test
[02-docker-perl-bookworm-slim]
-> Perl: v5.38.2 - CryptX: 0.080 - Mojolicious: 9.36 - Net::SSLeay: 1.94

michel@x1c7th:/media/michel/gpw2024$ docker image ls
REPOSITORY                                        SIZE
localhost/02-docker-perl-bookworm-slim            545 MB
localhost/01-docker-perl-bookworm                1.07 GB

• test ok
• size 545 MB is still too much

Dockerfile - 03-docker-perl-bookworm-slim-optimized

FROM perl:5.38.2-slim-bookworm
WORKDIR /src
COPY src/cpanfile .
RUN    apt-get update \ 
    && apt-get install -y curl build-essential \
    && curl -fsSL https://raw.githubusercontent.com/skaji/cpm/main/cpm > cpm \
    && chmod +x cpm \
    && ./cpm install -g --show-build-log-on-failure \ 
    && apt-get purge -y curl build-essential \
    && apt-get autoremove -y 
COPY src/*.pl .

• same as before but optimized
• all RUN ... commands in one layer

https://docs.docker.com/build/guide/layers/

Dockerfile - 03-docker-perl-bookworm-slim-optimized

michel@x1c7th:/media/michel/gpw2024$ ./run -i3 test
[03-docker-perl-bookworm-slim-optimized]
-> Perl: v5.38.2 - CryptX: 0.080 - Mojolicious: 9.36 - Net::SSLeay: 1.94

michel@x1c7th:/media/michel/gpw2024$ docker image ls
REPOSITORY                                        SIZE
localhost/03-docker-perl-bookworm-slim-optimized  260 MB
localhost/02-docker-perl-bookworm-slim            545 MB
localhost/01-docker-perl-bookworm                1.07 GB

• test ok
• much better (size 260 MB) - but let's try another approach: alpine linux

Dockerfile - 04-alpine-native

FROM alpine:3.19
WORKDIR /src
COPY src/cpanfile .

RUN apk add --update --no-cache perl perl-dev curl build-base openssl openssl-dev zlib-dev
RUN curl -fsSL https://raw.githubusercontent.com/skaji/cpm/main/cpm > cpm
RUN chmod +x cpm
RUN ./cpm install -g --show-build-log-on-failure 
RUN apk del perl-dev curl build-base openssl openssl-dev zlib-dev

COPY src/*.pl .

• based on alpine linux
• "secure and lightweight" - base image is approx. 8MB

https://www.alpinelinux.org/

Dockerfile - 04-alpine-native

michel@x1c7th:/media/michel/gpw2024$ ./run -i4 test
[04-alpine-native]
-> Perl: v5.38.2 - CryptX: 0.080 - Mojolicious: 9.36 - Net::SSLeay: 1.94

michel@x1c7th:/media/michel/gpw2024$ docker image ls
REPOSITORY                                        SIZE
localhost/04-alpine-native                        359 MB
localhost/03-docker-perl-bookworm-slim-optimized  260 MB
localhost/02-docker-perl-bookworm-slim            545 MB
localhost/01-docker-perl-bookworm                1.07 GB

• test ok
• 359 MB -- let's optimize

Dockerfile - 05-alpine-native-optimized

FROM alpine:3.19
WORKDIR /src
COPY src/cpanfile .

RUN apk add --update --no-cache perl perl-dev curl build-base openssl openssl-dev zlib-dev \
    && curl -fsSL https://raw.githubusercontent.com/skaji/cpm/main/cpm > cpm \
    && chmod +x cpm \
    && ./cpm install -g --show-build-log-on-failure \
    && apk del perl-dev curl build-base openssl openssl-dev zlib-dev 

COPY src/*.pl .

Dockerfile - 05-alpine-native-optimized

michel@x1c7th:/media/michel/gpw2024$ ./run -i5 test
[05-alpine-native-optimized]
-> Perl: v5.38.2 - CryptX: 0.080 - Mojolicious: 9.36 - Net::SSLeay: 1.94

michel@x1c7th:/media/michel/gpw2024$ docker image ls
REPOSITORY                                        SIZE
localhost/05-alpine-native-optimized              115 MB
localhost/04-alpine-native                        359 MB
localhost/03-docker-perl-bookworm-slim-optimized  260 MB
localhost/02-docker-perl-bookworm-slim            545 MB
localhost/01-docker-perl-bookworm                1.07 GB

• test ok
• 115 MB seems to be ok -- but ...

Docker Image - What's inside?

https://github.com/wagoodman/dive

Dockerfile - 06-alpine-native-optimized-optimized

FROM alpine:3.19
WORKDIR /src
COPY src/cpanfile .
RUN apk add --update --no-cache perl perl-dev curl build-base openssl openssl-dev zlib-dev \
    && curl -fsSL https://raw.githubusercontent.com/skaji/cpm/main/cpm > cpm \
    && chmod +x cpm \
    && ./cpm install -g --show-build-log-on-failure \
    && apk del perl-dev curl build-base openssl openssl-dev zlib-dev \
    && rm -R /root/.perl-cpm
COPY src/*.pl .

• new: rm -R /root/.perl-cpm

Dockerfile - 06-alpine-native-optimized-optimized

michel@x1c7th:/media/michel/gpw2024$ ./run -i6 test
[06-alpine-native-optimized-optimized]
-> Perl: v5.38.2 - CryptX: 0.080 - Mojolicious: 9.36 - Net::SSLeay: 1.94

michel@x1c7th:/media/michel/gpw2024$ docker image ls
REPOSITORY                                        SIZE
localhost/06-alpine-native-optimized-optimized     53 MB
localhost/05-alpine-native-optimized              115 MB
localhost/04-alpine-native                        359 MB
localhost/03-docker-perl-bookworm-slim-optimized  260 MB
localhost/02-docker-perl-bookworm-slim            545 MB
localhost/01-docker-perl-bookworm                1.07 GB

• test ok
• 53 MB
• perl versions 5.22 / 5.30 -- no apk packages available! -- let's compile

Dockerfile - 07-alpine-5.22

FROM alpine:3.19
ENV PATH="$PATH:/opt/perl/bin"
WORKDIR /src/perl

RUN apk add --update --no-cache curl build-base openssl openssl-dev zlib-dev 

ENV PERL_SOURCE_FILE=perl-5.22.4.tar.gz
ENV PERL_PATCH_FILE=522.patch
ENV PERL_SOURCE_HASH=ba9ef57c2b709f2dad9c5f6acf3111d9dfac309c484801e0152edbca89ed61fa

COPY src/perl/${PERL_SOURCE_FILE} .
COPY src/perl/${PERL_PATCH_FILE} .

RUN echo "${PERL_SOURCE_HASH} *${PERL_SOURCE_FILE}" | sha256sum -c -
RUN tar --strip-components=1 -xaf ${PERL_SOURCE_FILE} -C /src/perl
RUN cat ${PERL_PATCH_FILE} | patch -p1 
RUN ./Configure -Darchname=amd64 -Duse64bitall -Duseshrplib -Dvendorprefix=/opt/perl -Dprefix=/opt/perl -Dman1dir=none -Dman3dir=none -des
RUN make -j$(nproc) && make install

WORKDIR /src
COPY src/cpanfile .
RUN curl -fsSL https://raw.githubusercontent.com/skaji/cpm/main/cpm > cpm && chmod +x cpm
RUN ./cpm install -g --show-build-log-on-failure 
COPY src/*.pl .

• compile perl 5.22 from source

https://github.com/Perl/docker-perl/tree/master/eol/5.022.004-main-buster

Dockerfile - 07-alpine-5.22

michel@x1c7th:/media/michel/gpw2024$ ./run -i7 test
[07-alpine-5.22]
-> Perl: v5.22.4 - CryptX: 0.080 - Mojolicious: 9.36 - Net::SSLeay: 1.94

michel@x1c7th:/media/michel/gpw2024$ docker image ls
REPOSITORY                                        SIZE
localhost/07-alpine-5.22                          533 MB
localhost/06-alpine-native-optimized-optimized     53 MB
localhost/05-alpine-native-optimized              115 MB
localhost/04-alpine-native                        359 MB
localhost/03-docker-perl-bookworm-slim-optimized  260 MB
localhost/02-docker-perl-bookworm-slim            545 MB
localhost/01-docker-perl-bookworm                1.07 GB

• 533 MB -- not optimized yet

Dockerfile - 08-alpine-5.22-optimized

FROM alpine:3.19 as builder
ENV PATH="$PATH:/opt/perl/bin"
WORKDIR /src/perl
RUN apk add --update --no-cache curl build-base openssl openssl-dev zlib-dev 
ENV PERL_SOURCE_FILE=perl-5.22.4.tar.gz \
    PERL_PATCH_FILE=522.patch \
    PERL_SOURCE_HASH=ba9ef57c2b709f2dad9c5f6acf3111d9dfac309c484801e0152edbca89ed61fa
COPY src/perl/${PERL_SOURCE_FILE} .
COPY src/perl/${PERL_PATCH_FILE} .
RUN echo "${PERL_SOURCE_HASH} *${PERL_SOURCE_FILE}" | sha256sum -c - && \
    tar --strip-components=1 -xaf ${PERL_SOURCE_FILE} -C /src/perl && \
    cat ${PERL_PATCH_FILE} | patch -p1 && \ 
    ./Configure -Darchname=amd64 -Duse64bitall -Duseshrplib -Dvendorprefix=/opt/perl -Dprefix=/opt/perl -des && \
    make -j$(nproc) && make install
WORKDIR /src
COPY src/cpanfile .
RUN curl -fsSL https://raw.githubusercontent.com/skaji/cpm/main/cpm > cpm && chmod +x cpm && \
    ./cpm install -g --show-build-log-on-failure 
RUN find /opt/perl/lib -type f -iname "*.pod" -exec rm {} \;

FROM alpine:3.19
WORKDIR /src
ENV PATH="$PATH:/opt/perl/bin"
COPY src/*.pl .
COPY src/cpanfile .
COPY --from=builder /opt/perl /opt/perl

Dockerfile - 08-alpine-5.22-optimized

michel@x1c7th:/media/michel/gpw2024$ ./run -i8 test
[08-alpine-5.22-optimized]
-> Perl: v5.22.4 - CryptX: 0.080 - Mojolicious: 9.36 - Net::SSLeay: 1.94

michel@x1c7th:/media/michel/gpw2024$ docker image ls
REPOSITORY                                        SIZE
localhost/08-alpine-5.22-optimized                 51.4 MB
localhost/07-alpine-5.22                          533   MB
localhost/06-alpine-native-optimized-optimized     53   MB
localhost/05-alpine-native-optimized              115   MB
localhost/04-alpine-native                        359   MB
localhost/03-docker-perl-bookworm-slim-optimized  260   MB
localhost/02-docker-perl-bookworm-slim            545   MB
localhost/01-docker-perl-bookworm                1.07   GB

• test ok
• ~ 51 MB

Dockerfile - 09-alpine-5.xy-optimized

ARG PERL=5.38
FROM alpine:3.19 as base
ENV PATH="$PATH:/opt/perl/bin"
WORKDIR /src/perl
RUN apk add --update --no-cache curl build-base openssl openssl-dev zlib-dev 

FROM base as base-5.22
ENV PERL_SOURCE_FILE=perl-5.22.4.tar.gz \
    PERL_PATCH_FILE=522.patch \
    PERL_SOURCE_HASH=ba9ef57c2b709f2dad9c5f6acf3111d9dfac309c484801e0152edbca89ed61fa

FROM base as base-5.30
ENV PERL_SOURCE_FILE=perl-5.30.3.tar.gz \
    PERL_PATCH_FILE=530.patch \
    PERL_SOURCE_HASH=32e04c8bb7b1aecb2742a7f7ac0eabac100f38247352a73ad7fa104e39e7406f

FROM base as base-5.38
ENV PERL_SOURCE_FILE=perl-5.38.2.tar.gz \
    PERL_PATCH_FILE=no.patch \
    PERL_SOURCE_HASH=a0a31534451eb7b83c7d6594a497543a54d488bc90ca00f5e34762577f40655e

FROM base-${PERL} as builder
COPY src/perl/${PERL_SOURCE_FILE} .
COPY src/perl/${PERL_PATCH_FILE} .
RUN echo "${PERL_SOURCE_HASH} *${PERL_SOURCE_FILE}" | sha256sum -c - && \
    tar --strip-components=1 -xaf ${PERL_SOURCE_FILE} -C /src/perl && \
    if [[ "${PERL_PATCH_FILE}" != 'no.patch' ]]; then cat ${PERL_PATCH_FILE} | patch -p1; fi && \ 
    ./Configure -Darchname=amd64 -Duse64bitall -Duseshrplib -Dvendorprefix=/opt/perl -Dprefix=/opt/perl -des && \
    make -j$(nproc) && make install
WORKDIR /src
COPY src/cpanfile .
RUN curl -fsSL https://raw.githubusercontent.com/skaji/cpm/main/cpm > cpm && chmod +x cpm && \
    ./cpm install -g --show-build-log-on-failure 
RUN find /opt/perl/lib -type f -iname "*.pod" -exec rm {} \;

FROM alpine:3.19
WORKDIR /src
ENV PATH="$PATH:/opt/perl/bin"
COPY src/*.pl .
COPY src/cpanfile .
COPY --from=builder /opt/perl /opt/perl

Dockerfile - 09-alpine-5.xy-optimized

michel@x1c7th:/media/michel/gpw2024$ ./run -i9 test
[09-alpine-5.xy-optimized-5.22]
-> Perl: v5.22.4 - CryptX: 0.080 - Mojolicious: 9.36 - Net::SSLeay: 1.94

[09-alpine-5.xy-optimized-5.30]
-> Perl: v5.30.3 - CryptX: 0.080 - Mojolicious: 9.36 - Net::SSLeay: 1.94

[09-alpine-5.xy-optimized-5.38]
-> Perl: v5.38.2 - CryptX: 0.080 - Mojolicious: 9.36 - Net::SSLeay: 1.94

• test ok

Dockerfile - 09-alpine-5.xy-optimized

michel@x1c7th:/media/michel/gpw2024$ docker image ls
REPOSITORY                                        SIZE
localhost/09-alpine-5.xy-optimized-5.38            61.9 MB
localhost/09-alpine-5.xy-optimized-5.30            57.5 MB
localhost/09-alpine-5.xy-optimized-5.22            51.4 MB
localhost/08-alpine-5.22-optimized                 51.4 MB
localhost/07-alpine-5.22                          533   MB
localhost/06-alpine-native-optimized-optimized     53   MB
localhost/05-alpine-native-optimized              115   MB
localhost/04-alpine-native                        359   MB
localhost/03-docker-perl-bookworm-slim-optimized  260   MB
localhost/02-docker-perl-bookworm-slim            545   MB
localhost/01-docker-perl-bookworm                1.07   GB

• Perl Versions 5.22, 5.30 and 5.38
• size 51 MB - 62 MB