How to Install OpenLambda on Alibaba Cloud

By Sai Sarath Chandra, Alibaba Cloud Community Blog author

OpenLambda is an open-source platform for building web services in the serverless model. The rapid innovation of software and hardware especially more recently through all of IT has led to a great change in how we develop, build, deploy, as well as even maintain applications and services. In many ways, there has been a paradigm shift in terms of how we used to run the applications to how we are now accustom to running applications and services on virtual machine in the cloud.

You could argue that Virtualization is simple as grouping all the services you've got into one single machine. However, sometimes the machines will be underutilized. To address this problem, containerization technology was introduced. Container technology can be understood as a bunch of UNIX processes, which restructure an application in such a way that each feature is packaged into a single unit and deployed over the containers.

Even though containers showed great improvement over previous technologies, they are still susceptible to underutilization. A serverless computing framework can help with tackling this problem by abstracting the effort of managing the runtime, including the OS and Hardware. But there is also a downside of serverless architectures in terms of the increased complexity of managing clusters. Of course, there are several frameworks out there building solutions that revolve around this specific use case. One of them is OpenLambda.

In this article, we will discuss how we can use OpenLambda on Alibaba Cloud for your serverless solution to issues underutilization. For this, we will be setting up OpenLambda on an ECS Instance.

Setting up Your ECS Instance

When it comes to creating the ECS instance, there are several basic steps which you should pay attention to:

Basic Configurations:

1.Billing method

Make sure you choose Pay-As-You-Go as the billing method if you are just prototyping the application.

2.Image

You need to select the Ubuntu 18.04 (64-bit)

Networking

1.Make sure you select the proper security group here. Of course, though, we can open the ports while we are working on this.

System Configurations

1.For this tutorial, I chose password-based authentication.

Grouping
Preview

1.The complete information for the selected configuration that you choose. Note that you can also save it as a template so that in the future when creating an ECS instance you can do so in a single click.

2.If you wish to release the instance at the later point in time, select the Auto-Release schedule option.

Getting Your ECS Instance All Ready

Now, let's get into the nitty gritty when it comes to getting your ECS instance ready for the later steps of this tutorial. So let's get started. Open your command line interface and let's do some coding! First you'll want to connect to the instance using SSH, as shown below:

ssh root@<IP ADDRESS>
The authenticity of host 'AAA.CCC.BBB.DDD' can't be established.
ECDSA key fingerprint is SHA256:/il4fRYvgAake/ENoBgJSZvdO8DxY7H99oeIuye8AEo.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added ' AAA.CCC.BBB.DDD ' (ECDSA) to the list of known hosts.

root@<IP ADDRESS>'s password: 
Welcome to Ubuntu 18.04 LTS (GNU/Linux 4.15.0-23-generic x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

Welcome to Alibaba Cloud Elastic Compute Service !

You can update the package entries by running the following command.

sudo apt-get update

Once that's done, we will upgrade any obsolete packages. Next, we will know what packages are to be updated and removed by running the below command:

sudo apt-get upgrade

You will also see a similar message as the one below, where it asks for your consent to upgrade:

The following packages were automatically installed and are no longer required:
  libopts25 sntp
Use 'sudo apt autoremove' to remove them.
The following packages have been kept back:
  linux-generic linux-headers-generic linux-image-generic
The following packages will be upgraded:
  amd64-microcode apparmor apt apt-utils base-files bind9-host binutils binutils-common binutils-x86-64-linux-gnu
  bsdutils chrony console-setup console-setup-linux cpp cpp-7 cups-bsd cups-client cups-common curl
  distro-info-data dnsutils dpkg dpkg-dev fdisk friendly-recovery g++ g++-7 gcc gcc-7 gcc-7-base gcc-8-base
  grub-common grub-pc grub-pc-bin grub2-common guile-2.0-libs initramfs-tools initramfs-tools-bin
  initramfs-tools-core intel-microcode iperf keyboard-configuration kmod language-pack-en language-pack-en-base
  language-pack-gnome-en language-pack-gnome-en-base libapparmor1 libapt-inst2.0 libapt-pkg5.0
  libarchive-zip-perl libarchive13 libasan4 libatomic1 libavahi-client3 libavahi-common-data libavahi-common3
  libbind9-160 libbinutils libblkid1 libcc1-0 libcilkrts5 libcryptsetup12 libcups2 libcupsimage2 libcurl4
  libdns-export1100 libdns1100 libdpkg-perl libfdisk1 libgcc-7-dev libgcc1 libglib2.0-0 libglib2.0-data libgomp1
  libirs160 libisc-export169 libisc169 libisccc160 libisccfg160 libitm1 libjpeg-turbo8 libkmod2 liblsan0
  liblwres160 libmount1 libmpx2 libmysqlclient20 libnss-systemd libpam-systemd libparted2 libperl5.26
  libplymouth4 libpng16-16 libpolkit-gobject-1-0 libpython3-stdlib libpython3.6 libpython3.6-minimal
  libpython3.6-stdlib libquadmath0 librados2 librbd1 libsmartcols1 libstdc++-7-dev libstdc++6 libsystemd0
  libtsan0 libubsan0 libudev1 libuuid1 libx11-6 libx11-data libxml2 linux-libc-dev lshw man-db mount netplan.io
  networkd-dispatcher nplan openssh-client openssh-server openssh-sftp-server parted perl perl-base
  perl-modules-5.26 plymouth plymouth-theme-ubuntu-text python-apt-common python3 python3-apt python3-distupgrade
  python3-minimal python3-requests python3-update-manager python3.6 python3.6-minimal sntp ssh systemd
  systemd-sysv tzdata ubuntu-release-upgrader-core udev update-manager-core util-linux uuid-runtime
148 upgraded, 0 newly installed, 0 to remove and 3 not upgraded.
Need to get 79.6 MB of archives.
After this operation, 312 kB of additional disk space will be used.
Do you want to continue? [Y/n] Y

Input Y and also if you want to remove the unused packages, removal can be done by running below command:

sudo apt autoremove

This command will also give the similar prompt asking for the same thing as above. The upgrade will take some time. There might be some warning you might while it is installing, like the one I gave below:

perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
  LANGUAGE = (unset),
  LC_ALL = (unset),
  LC_CTYPE = "UTF-8",
  LANG = "en_US.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to a fallback locale ("en_US.UTF-8").

One fix to the warning given above is to do the following. First you can run the locale command. You will see the below output.

locale
LANGUAGE=
LC_CTYPE=C.UTF-8
LC_NUMERIC="C.UTF-8"
LC_TIME="C.UTF-8"
LC_COLLATE="C.UTF-8"
LC_MONETARY="C.UTF-8"
LC_MESSAGES="C.UTF-8"
LC_PAPER="C.UTF-8"
LC_NAME="C.UTF-8"
LC_ADDRESS="C.UTF-8"
LC_TELEPHONE="C.UTF-8"
LC_MEASUREMENT="C.UTF-8"
LC_IDENTIFICATION="C.UTF-8"
LC_ALL=

From this you can figure out that you need to set the LC_ALL and Language settings by running the following commands.

export LANGUAGE="en_US.UTF-8"
export LC_ALL="en_US.UTF-8"

Before we proceed with anything else, we need to point the pip registry to the global site, this can be done by running the following command:

~root@iZa2d4spx9n9e90jofcyxoZ:~# sudo nano /root/.pip/pip.conf

Next, you'll want to replace the output with the below text:

## Note, this file is written by cloud-init on first boot of an instance
## modifications made here will not survive a re-bundle.
###[m
[global]
## index-url=http://ap-south-1.mirrors.cloud.aliyuncs.com/pypi/simple/
index-url=https://pypi.python.org/simple/
[install]
## trusted-host=ap-south-1.mirrors.cloud.aliyuncs.com
trusted-host=pypi.python.org

We need to install git to the machine. You can do so with this command:

root@OPENLAMBDA:~# apt install git

Setting up OpenLambda

After everything's installed, and you've done all those configurations to your ECS instance. It's time to get down with doing things with OpenLambda now.

First, we need to clone the open Lambda repository. We can do this with this command.

git clone https://github.com/open-lambda/open-lambda.git
Cloning into 'open-lambda'...
.
.
.
Resolving deltas: 100% (4716/4716), done.
Checking connectivity... done.

OpenLambda team has simplified the process of installation by grouping everything into the single script file, but these scripts require root permissions. You can give the root permission by running the Sudo -s command. Then, once root permissions are provided, change to the openLambda directory with this command:

root@OPENLAMBDA:~# cd open-lambda/

Now, run the following dependency installation script:

root@OPENLAMBDA:~/open-lambda# ./quickstart/deps.sh

You'll get the following output:

Resolving storage.googleapis.com (storage.googleapis.com)... 74.125.24.128, 2404:6800:4003:c04::80
Connecting to storage.googleapis.com (storage.googleapis.com)|74.125.24.128|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 84237339 (80M) [application/x-gzip]
Saving to: '/tmp/go1.7.6.tar.gz'
(14.1 MB/s) - '/tmp/go1.7.6.tar.gz' saved [84237339/84237339]

The script will take a while, but you can see all the dependencies are downloaded and installed. You can test all the setup using the below command.

root@OPENLAMBDA:~/open-lambda# make test-all
-------- OUTPUT --------
cd hack/go/src/github.com/open-lambda/open-lambda/worker/admin && /root/open-lambda/hack/go.sh install
Using GOPATH=/root/open-lambda/hack/go
+ go install
mkdir -p bin
cp hack/go/bin/admin ./bin
make -C lambda
make[1]: Entering directory '/root/open-lambda/lambda'
gcc -O2 -o spin spin.c
gcc -Wall -O2 ol-init.c -o ol-init
cd nsmodule && python setup.py build_ext --inplace
running build_ext
building 'ns' extension
creating build
creating build/temp.linux-x86_64-2.7
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/include/python2.7 -c nsmodule.c -o build/temp.linux-x86_64-2.7/nsmodule.o
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wl,-Bsymbolic-functions -Wl,-z,relro -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/nsmodule.o -o /root/open-lambda/lambda/nsmodule/ns.so
mv nsmodule/ns.so .
make[1]: Leaving directory '/root/open-lambda/lambda'
docker build -t lambda lambda

You will see all the commands are being triggered with the root privileges and all the go scripts are being executed. You might see errors, as shown below:

tarting worker...
./bin/admin setconf -cluster=testing/test-cluster  '{"sandbox": "sock", "handler_cache_size": 0, "import_cache_size": 0, "cg_pool_size": 10}'
./bin/admin workers -cluster=testing/test-cluster
Started worker: pid 31057, port 8080, log at /root/open-lambda/testing/test-cluster/logs/worker-0.out

Waiting for worker to initialize...
At least one worker failed the status check
At least one worker failed the status check
At least one worker failed the status check
At least one worker failed the status check
Worker ready. Requesting lambdas...
curl -XPOST localhost:8080/runLambda/echo -d '{}'
fork/exec /usr/local/bin/sock-init: no such file or directory
.
.
.
.
At least one worker failed the status check
At least one worker failed the status check
^CMakefile:114: recipe for target 'test-sock-import' failed
make: *** [test-sock-import] Interrupt

Even though OpenLambda should work on both Ubuntu 16.04 and 14.04 without fail, the above error is prevalent even on those versions of Ubuntu. This error doesn't mean your installation is unsuccessful. I have personally tested it with multiple machines, this error is still the same even if you installed it successfully.

Create a OpenLambda Cluster

Next, we can create a cluster using the below command:

root@OPENLAMBDA:~/open-lambda# ./bin/admin new -cluster my-cluster
Cluster Directory: /root/open-lambda/my-cluster

Worker Defaults: 
{
  "registry": "local",
  "sandbox": "docker",
  "registry_dir": "/root/open-lambda/my-cluster/registry",
  "registry_server": "",
  "registry_access_key": "",
  "registry_secret_key": "",
  "cluster_name": "/root/open-lambda/my-cluster",
  "pip_mirror": "",
  "Pkgs_dir": "/root/open-lambda/my-cluster/packages",
  "max_runners": 0,
  "handler_cache_size": 0,
  "import_cache_size": 0,
  "worker_dir": "/root/open-lambda/my-cluster/workers/default",
  "SOCK_base_path": "",
  "worker_port": "?",
  "cg_pool_size": 0,
  "Skip_pull_existing": false,
  "sandbox_config": {
    "processes": 10
  },
  "benchmark_log": "",
  "timing": false,
  "startup_pkgs": null
}

You may now start a cluster using the "workers" command

We need to start the cluster using the workers command:

root@OPENLAMBDA:~/open-lambda# ./bin/admin workers -cluster=my-cluster
Started worker: pid 1471, port 8080, log at /root/open-lambda/my-cluster/logs/worker-0.out

Next, you can check the status of the cluster using the following command. This checks whether the previous operation was successfully performed or not.

root@OPENLAMBDA:~/open-lambda# ./bin/admin status -cluster=my-cluster
Worker Pings:
  http://localhost:8080/status => ready
 [200 OK]

All the functions in OpenLambda are treated as handlers. You can create one using the below command.

root@OPENLAMBDA:~/open-lambda# cp -r ./quickstart/handlers/hello ./my-cluster/registry/hello

Once that's finished, your function will be up and running. You can test by invoking the function from the command line as below

 root@OPENLAMBDA:~/open-lambda# curl -X POST localhost:8080/runLambda/hello -d '{"name": "Sai Sarath Chandra"}'

"Hello, Sai Sarath Chandra!"

We can edit the function and test it by just modifying the file on the server:

root@OPENLAMBDA:~/open-lambda/my-cluster/registry#  sudo nano lambda_func.py

Now, replace the function with the below code:

def handler(event):
    try:
        return "I am , %s! & I just created a serverless function named %s on openLambda" %(event['name'],event['funcDef'])
    except Exception as e:
        return {'error': str(e)}

Next, trigger the function using a curl command again:

root@OPENLAMBDA:~/open-lambda# curl -X POST localhost:8080/runLambda/hello -d '{"name": "Sai Sarath Chandra"}'
"I am , Sai Sarath Chandra! & I just created a serverless function named myLambda on openLambda"

Conclusion

By following this tutorial, you have now installed OpenLambda on Alibaba Cloud. This is only scratching the surface of what is possible with OpenLambda! There are so many other configurations that you could make with OpenLambda installed on Alibaba Cloud. Keep a look out for more tutorials in the future, where I can discuss more things you can do with Openlambda.

Community

How to Install OpenLambda on Alibaba Cloud

Setting up Your ECS Instance

Getting Your ECS Instance All Ready

Setting up OpenLambda

Create a OpenLambda Cluster

Conclusion

Read previous post:

Read next post:

Alibaba Clouder

You may also like

Comments

Alibaba Clouder

Related Products

ECS(Elastic Compute Service)

Function Compute