Multi Cloud

Multi-cloud Local Cluster Installation and Testing

Here are the steps to install SODA Multi Cloud service in a containerized local environment .

Pre-config (Ubuntu 16.04/Ubuntu 18.04)

The installation is tested on Ubuntu 16.04 and Ubuntu 18.04, please make sure to use the right one. Root user is suggested for the installation.

  • packages

Install following packages:

apt-get install -y libltdl7 libseccomp2 git curl wget make
  • docker

Install docker:

dpkg -i docker-ce_18.03.1~ce-0~ubuntu_amd64.deb 
  • golang

Check golang version information: SODA multi-cloud needs go version 1.13.x


root@proxy:~# go version
go version go1.13.9 linux/amd64

You can install golang by executing commands below: [Using 1.13.9 here for example]

tar -C /usr/local -xzf go1.13.9.linux-amd64.tar.gz
echo 'export PATH=$PATH:/usr/local/go/bin' >> /etc/profile
echo 'export GOPATH=$HOME/gopath' >> /etc/profile
source /etc/profile
  • docker-compose

Download and install the latest version of Docker Compose:

curl -L "$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose

# docker-compose --version
docker-compose version 1.22.0, build 1719ceb

Install and start SODA Multi-cloud Server

Download the source code:

mkdir -p /root/gopath/src/
git clone
cd /root/gopath/src/

Make and build the docker image

make docker

Execute docker-compose up -d to start multi-cloud local cluster, the successful result is supposed to be as follows:

# docker-compose up -d

Creating redis                                ... done
Creating multi-cloud_block_1_87f15fb3fcf4     ... done
Creating tidb                                 ... done
Creating multi-cloud_zookeeper_1_77bbf2cb55a6 ... done
Creating multi-cloud_datastore_1_2341f44ebbba ... done
Creating multi-cloud_file_1_4885d7d3334d      ... done
Creating multi-cloud_backend_1_5ab6fd6feed4   ... done
Creating multi-cloud_s3api_1_395c90ae45c3     ... done
Creating multi-cloud_api_1_437a019a9278       ... done
Creating multi-cloud_s3_1_5cf4e4514669        ... done
Creating multi-cloud_kafka_1_ff762483f0b6     ... done
Creating multi-cloud_dataflow_1_592ca1c6f730  ... done
Creating multi-cloud_datamover_1_390d343d0e7c ... done

To check the installation status run the command docker ps

# docker ps

CONTAINER ID        IMAGE                                  COMMAND                  CREATED             STATUS              PORTS                                                NAMES
bbe74a5359eb        sodafoundation/multi-cloud-dataflow    "/dataflow"              40 seconds ago      Up 39 seconds                                                            multi-cloud_dataflow_1_8ce4de44c4a7
e18133da2d9f        sodafoundation/multi-cloud-datamover   "/datamover"             40 seconds ago      Up 39 seconds                                                            multi-cloud_datamover_1_1ae5af80e61f
ec8052d3db15        wurstmeister/kafka:2.11-2.0.1          ""         41 seconds ago      Up 39 seconds>9092/tcp                               multi-cloud_kafka_1_74c5e392fd7d
5fb7560159b2        sodafoundation/multi-cloud-s3          "/"             41 seconds ago      Up 40 seconds                                                            multi-cloud_s3_1_59cd2512722b
b28213c1ddec        sodafoundation/multi-cloud-backend     "/backend"               42 seconds ago      Up 39 seconds                                                            multi-cloud_backend_1_a67a449c92a5
a47ec4ef770b        sodafoundation/multi-cloud-file        "/file"                  42 seconds ago      Up 40 seconds                                                            multi-cloud_file_1_cd84c1c9dad3
be2b170b2d68        sodafoundation/multi-cloud-api         "/api"                   42 seconds ago      Up 41 seconds>8090/tcp                               multi-cloud_s3api_1_4698613dfd2e
f0f2419b98db        sodafoundation/multi-cloud-api         "/api"                   42 seconds ago      Up 41 seconds>8089/tcp                               multi-cloud_api_1_aacf7b9a5141
ad7a85f51343        wurstmeister/zookeeper                 "/bin/sh -c '/usr/sb…"   42 seconds ago      Up 41 seconds       22/tcp, 2888/tcp, 3888/tcp,>2181/tcp   multi-cloud_zookeeper_1_68df196d327a
3539d1a91298        mongo                                  "docker-entrypoint.s…"   42 seconds ago      Up 41 seconds>27017/tcp                             multi-cloud_datastore_1_a2f8fb0deb9d
0722d5174b8f        pingcap/tidb:v2.1.16                   "/tidb-server --stor…"   42 seconds ago      Up 41 seconds>4000/tcp,>10080/tcp     tidb
e03e03d88dcb        sodafoundation/multi-cloud-block       "/block"                 42 seconds ago      Up 40 seconds                                                            multi-cloud_block_1_7adafa5c5b8d
edc2036f5e1f        redis                                  "docker-entrypoint.s…"   42 seconds ago      Up 41 seconds>6379/tcp                               redis