[docs] move one level higher
This commit is contained in:
		
					parent
					
						
							
								26d5c91625
							
						
					
				
			
			
				commit
				
					
						bd03f95e99
					
				
			
		
					 17 changed files with 1 additions and 1 deletions
				
			
		
							
								
								
									
										0
									
								
								docs/source/__init__.py
									
										
									
									
									
										Normal file
									
								
							
							
						
						
									
										0
									
								
								docs/source/__init__.py
									
										
									
									
									
										Normal file
									
								
							
							
								
								
									
										158
									
								
								docs/source/admin-guide
									
										
									
									
									
										Normal file
									
								
							
							
						
						
									
										158
									
								
								docs/source/admin-guide
									
										
									
									
									
										Normal file
									
								
							|  | @ -0,0 +1,158 @@ | |||
| .. _admin-guide: | ||||
| 
 | ||||
| 
 | ||||
| Usage Guide For Administrators | ||||
| ============================== | ||||
| 
 | ||||
| Start API | ||||
| ---------- | ||||
| 
 | ||||
| .. code-block:: sh | ||||
| 
 | ||||
|     ucloud api | ||||
| 
 | ||||
| Host Creation | ||||
| ------------- | ||||
| 
 | ||||
| Currently, we don't have any host (that runs virtual machines). | ||||
| So, we need to create it by executing the following command | ||||
| 
 | ||||
| .. code-block:: sh | ||||
| 
 | ||||
|     ucloud-cli host create --hostname ungleich.ch --cpu 32 --ram '32GB' --os-ssd '32GB' | ||||
| 
 | ||||
| You should see something like the following | ||||
| 
 | ||||
| .. code-block:: json | ||||
| 
 | ||||
|     { | ||||
|         "message": "Host Created" | ||||
|     } | ||||
| 
 | ||||
| Start Scheduler | ||||
| --------------- | ||||
| Scheduler is responsible for scheduling VMs on appropriate host. | ||||
| 
 | ||||
| .. code-block:: sh | ||||
| 
 | ||||
|     ucloud scheduler | ||||
| 
 | ||||
| Start Host | ||||
| ---------- | ||||
| Host is responsible for handling the following actions | ||||
| 
 | ||||
| * Start VM. | ||||
| * Stop VM. | ||||
| * Create VM. | ||||
| * Delete VM. | ||||
| * Migrate VM. | ||||
| * Manage Network Resources needed by VMs. | ||||
| 
 | ||||
| It uses a hypervisor such as QEMU to perform these actions. | ||||
| 
 | ||||
| To start host we created earlier, execute the following command | ||||
| 
 | ||||
| .. code-block:: sh | ||||
| 
 | ||||
|     ucloud host ungleich.ch | ||||
| 
 | ||||
| Create OS Image | ||||
| --------------- | ||||
| 
 | ||||
| Create ucloud-init ready OS image (Optional) | ||||
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||||
| This step is optional if you just want to test ucloud. However, sooner or later | ||||
| you want to create OS images with ucloud-init to properly | ||||
| contexualize VMs. | ||||
| 
 | ||||
| 1. Start a VM with OS image on which you want to install ucloud-init | ||||
| 2. Execute the following command on the started VM | ||||
| 
 | ||||
|    .. code-block:: sh | ||||
| 
 | ||||
|     apk add git | ||||
|     git clone https://code.ungleich.ch/ucloud/ucloud-init.git | ||||
|     cd ucloud-init | ||||
|     sh ./install.sh | ||||
| 3. Congratulations. Your image is now ucloud-init ready. | ||||
| 
 | ||||
| 
 | ||||
| Upload Sample OS Image | ||||
| ~~~~~~~~~~~~~~~~~~~~~~ | ||||
| Execute the following to get the sample OS image file. | ||||
| 
 | ||||
| .. code-block:: sh | ||||
| 
 | ||||
|     mkdir /var/www/admin | ||||
|     (cd /var/www/admin && wget https://cloud.ungleich.ch/s/qTb5dFYW5ii8KsD/download) | ||||
| 
 | ||||
| Run File Scanner and Image Scanner | ||||
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||||
| Currently, our uploaded file *alpine-untouched.qcow2* is not tracked by ucloud. We can only make | ||||
| images from tracked files. So, we need to track the file by running File Scanner | ||||
| 
 | ||||
| .. code-block:: sh | ||||
| 
 | ||||
|     ucloud filescanner | ||||
| 
 | ||||
| File Scanner would run, scan your uploaded image and track it. You can check whether your image | ||||
| is successfully tracked by executing the :code:`ucloud-cli user files`, It will return something like the following | ||||
| 
 | ||||
| .. _list-user-files: | ||||
| 
 | ||||
| .. code-block:: json | ||||
| 
 | ||||
|     { | ||||
|         "message": [ | ||||
|             { | ||||
|                 "filename": "alpine-untouched.qcow2", | ||||
|                 "uuid": "3f75bd20-45d6-4013-89c4-7fceaedc8dda" | ||||
|             } | ||||
|         ] | ||||
|     } | ||||
| 
 | ||||
| Our file is now being tracked by ucloud. Lets create an OS image using the uploaded file. | ||||
| 
 | ||||
| An image belongs to an image store. There are two types of store | ||||
| 
 | ||||
| * Public Image Store | ||||
| * Private Image Store (Not Implemented Yet) | ||||
| 
 | ||||
| .. note:: | ||||
|     **Quick Quiz** Have we created an image store yet? | ||||
| 
 | ||||
| The answer is **No, we haven't**. Creating a sample image store is very easy. | ||||
| Just execute the following command | ||||
| 
 | ||||
| .. code-block:: sh | ||||
| 
 | ||||
|     (cd ~/ucloud && pipenv run python api/create_image_store.py) | ||||
| 
 | ||||
| An image store (with name = "images") would be created. Now, we are fully ready for creating our | ||||
| very own image. Executing the following command to create image using the file uploaded earlier | ||||
| 
 | ||||
| .. code-block:: sh | ||||
| 
 | ||||
|     ucloud-cli image create-from-file --name alpine --uuid 3f75bd20-45d6-4013-89c4-7fceaedc8dda --image-store-name images | ||||
| 
 | ||||
| Please note that your **uuid** would be different. See :ref:`List of user files <list-user-files>`. | ||||
| 
 | ||||
| Now, ucloud have received our request to create an image from file. We have to run Image Scanner to make the image. | ||||
| 
 | ||||
| .. code-block:: sh | ||||
| 
 | ||||
|     ucloud imagescanner | ||||
| 
 | ||||
| To make sure, that our image is create run :code:`ucloud-cli image list --public`. You would get | ||||
| output something like the following | ||||
| 
 | ||||
| .. code-block:: json | ||||
| 
 | ||||
|     { | ||||
|         "images": [ | ||||
|             { | ||||
|                 "name": "images:alpine", | ||||
|                 "status": "CREATED" | ||||
|             } | ||||
|         ] | ||||
|     } | ||||
							
								
								
									
										53
									
								
								docs/source/conf.py
									
										
									
									
									
										Normal file
									
								
							
							
						
						
									
										53
									
								
								docs/source/conf.py
									
										
									
									
									
										Normal file
									
								
							|  | @ -0,0 +1,53 @@ | |||
| # Configuration file for the Sphinx documentation builder. | ||||
| # | ||||
| # This file only contains a selection of the most common options. For a full | ||||
| # list see the documentation: | ||||
| # https://www.sphinx-doc.org/en/master/usage/configuration.html | ||||
| 
 | ||||
| # -- Path setup -------------------------------------------------------------- | ||||
| 
 | ||||
| # If extensions (or modules to document with autodoc) are in another directory, | ||||
| # add these directories to sys.path here. If the directory is relative to the | ||||
| # documentation root, use os.path.abspath to make it absolute, like shown here. | ||||
| # | ||||
| # import os | ||||
| # import sys | ||||
| # sys.path.insert(0, os.path.abspath('.')) | ||||
| 
 | ||||
| 
 | ||||
| # -- Project information ----------------------------------------------------- | ||||
| 
 | ||||
| project = "uncloud" | ||||
| copyright = "2019, ungleich" | ||||
| author = "ungleich" | ||||
| 
 | ||||
| # -- General configuration --------------------------------------------------- | ||||
| 
 | ||||
| # Add any Sphinx extension module names here, as strings. They can be | ||||
| # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom | ||||
| # ones. | ||||
| extensions = [ | ||||
|     "sphinx.ext.autodoc", | ||||
|     "sphinx_rtd_theme", | ||||
| ] | ||||
| 
 | ||||
| # Add any paths that contain templates here, relative to this directory. | ||||
| templates_path = ["_templates"] | ||||
| 
 | ||||
| # List of patterns, relative to source directory, that match files and | ||||
| # directories to ignore when looking for source files. | ||||
| # This pattern also affects html_static_path and html_extra_path. | ||||
| exclude_patterns = [] | ||||
| 
 | ||||
| # -- Options for HTML output ------------------------------------------------- | ||||
| 
 | ||||
| # The theme to use for HTML and HTML Help pages.  See the documentation for | ||||
| # a list of builtin themes. | ||||
| # | ||||
| 
 | ||||
| html_theme = "sphinx_rtd_theme" | ||||
| 
 | ||||
| # Add any paths that contain custom static files (such as style sheets) here, | ||||
| # relative to this directory. They are copied after the builtin static files, | ||||
| # so a file named "default.css" will overwrite the builtin "default.css". | ||||
| html_static_path = ["_static"] | ||||
							
								
								
									
										44
									
								
								docs/source/diagram-code/ucloud
									
										
									
									
									
										Normal file
									
								
							
							
						
						
									
										44
									
								
								docs/source/diagram-code/ucloud
									
										
									
									
									
										Normal file
									
								
							|  | @ -0,0 +1,44 @@ | |||
| graph LR | ||||
|     style ucloud fill:#FFD2FC | ||||
|     style cron fill:#FFF696 | ||||
|     style infrastructure fill:#BDF0FF | ||||
|     subgraph ucloud[ucloud] | ||||
|         ucloud-cli[CLI]-->ucloud-api[API] | ||||
|         ucloud-api-->ucloud-scheduler[Scheduler] | ||||
|         ucloud-api-->ucloud-imagescanner[Image Scanner] | ||||
|         ucloud-api-->ucloud-host[Host] | ||||
|         ucloud-scheduler-->ucloud-host | ||||
| 
 | ||||
|         ucloud-host-->need-networking{VM need Networking} | ||||
|         need-networking-->|Yes| networking-scripts | ||||
|         need-networking-->|No| VM[Virtual Machine] | ||||
|         need-networking-->|SLAAC?| radvd | ||||
|         networking-scripts-->VM | ||||
|         networking-scripts--Create Networks Devices-->networking-scripts | ||||
|         subgraph cron[Cron Jobs] | ||||
|             ucloud-imagescanner | ||||
|             ucloud-filescanner[File Scanner] | ||||
|             ucloud-filescanner--Track User files-->ucloud-filescanner | ||||
|         end | ||||
|  subgraph infrastructure[Infrastructure] | ||||
|     radvd | ||||
|     etcd | ||||
|     networking-scripts[Networking Scripts] | ||||
|     ucloud-imagescanner-->image-store | ||||
|     image-store{Image Store} | ||||
|     image-store-->|CEPH| ceph | ||||
|     image-store-->|FILE| file-system | ||||
|     ceph[CEPH] | ||||
|     file-system[File System] | ||||
|     end | ||||
| subgraph virtual-machine[Virtual Machine] | ||||
|   VM | ||||
|   VM-->ucloud-init | ||||
| 
 | ||||
| end | ||||
| 
 | ||||
| subgraph metadata-group[Metadata Server] | ||||
| metadata-->ucloud-init | ||||
| ucloud-init<-->metadata | ||||
| end | ||||
| end | ||||
							
								
								
									
										25
									
								
								docs/source/hacking.rst
									
										
									
									
									
										Normal file
									
								
							
							
						
						
									
										25
									
								
								docs/source/hacking.rst
									
										
									
									
									
										Normal file
									
								
							|  | @ -0,0 +1,25 @@ | |||
| Hacking | ||||
| ======= | ||||
| Using uncloud in hacking (aka development) mode. | ||||
| 
 | ||||
| 
 | ||||
| Get the code | ||||
| ------------ | ||||
| .. code-block:: sh | ||||
|     :linenos: | ||||
| 
 | ||||
|      git clone https://code.ungleich.ch/uncloud/uncloud.git | ||||
| 
 | ||||
| 
 | ||||
| 
 | ||||
| Install python requirements | ||||
| --------------------------- | ||||
| You need to have python3 installed. | ||||
| 
 | ||||
| .. code-block:: sh | ||||
|     :linenos: | ||||
| 
 | ||||
|     cd uncloud! | ||||
|     python -m venv venv | ||||
|     . ./venv/bin/activate | ||||
|     ./bin/uncloud-run-reinstall | ||||
							
								
								
									
										494
									
								
								docs/source/images/ucloud.svg
									
										
									
									
									
										Normal file
									
								
							
							
						
						
									
										494
									
								
								docs/source/images/ucloud.svg
									
										
									
									
									
										Normal file
									
								
							
										
											
												File diff suppressed because one or more lines are too long
											
										
									
								
							| After Width: | Height: | Size: 37 KiB | 
							
								
								
									
										27
									
								
								docs/source/index.rst
									
										
									
									
									
										Normal file
									
								
							
							
						
						
									
										27
									
								
								docs/source/index.rst
									
										
									
									
									
										Normal file
									
								
							|  | @ -0,0 +1,27 @@ | |||
| .. ucloud documentation master file, created by | ||||
|    sphinx-quickstart on Mon Nov 11 19:08:16 2019. | ||||
|    You can adapt this file completely to your liking, but it should at least | ||||
|    contain the root `toctree` directive. | ||||
| 
 | ||||
| Welcome to ucloud's documentation! | ||||
| ================================== | ||||
| 
 | ||||
| .. toctree:: | ||||
|    :maxdepth: 2 | ||||
|    :caption: Contents: | ||||
| 
 | ||||
|    introduction | ||||
|    user-guide | ||||
|    setup-install | ||||
|    admin-guide | ||||
|    user-guide/how-to-create-an-os-image-for-ucloud | ||||
|    troubleshooting | ||||
|    hacking | ||||
| 
 | ||||
| 
 | ||||
| Indices and tables | ||||
| ================== | ||||
| 
 | ||||
| * :ref:`genindex` | ||||
| * :ref:`modindex` | ||||
| * :ref:`search` | ||||
							
								
								
									
										58
									
								
								docs/source/introduction.rst
									
										
									
									
									
										Normal file
									
								
							
							
						
						
									
										58
									
								
								docs/source/introduction.rst
									
										
									
									
									
										Normal file
									
								
							|  | @ -0,0 +1,58 @@ | |||
| Introduction | ||||
| ============ | ||||
| 
 | ||||
| ucloud is a modern, IPv6 first virtual machine management system. | ||||
| It is an alternative to `OpenNebula <https://opennebula.org/>`_, | ||||
| `OpenStack <https://www.openstack.org/>`_ or | ||||
| `Cloudstack <https://cloudstack.apache.org/>`_. | ||||
| 
 | ||||
| ucloud is the first cloud management system that puts IPv6 | ||||
| first. ucloud also has an integral ordering process that we missed in | ||||
| existing solutions. | ||||
| 
 | ||||
| The ucloud documentation is separated into various sections for the | ||||
| different use cases: | ||||
| 
 | ||||
| * :ref:`The user guide <user-guide>` describes how to use an existing | ||||
|   ucloud installation | ||||
| * There are :ref:`setup instructions <setup-install>` which describe on how to setup a new | ||||
|   ucloud instance | ||||
| * :ref:`The admin guide <admin-guide>` describe on how to | ||||
|   administrate ucloud | ||||
| 
 | ||||
| 
 | ||||
| Architecture | ||||
| ------------ | ||||
| We try to reuse existing components for ucloud. Generally speaking, | ||||
| ucloud consist of a variety of daemons who handle specific tasks and | ||||
| connect to a shared database. | ||||
| 
 | ||||
| All interactions with the clients are done through an API. | ||||
| 
 | ||||
| ucloud consists of the following components: | ||||
| 
 | ||||
| * API | ||||
| * Scheduler | ||||
| * Host | ||||
| * File Scanner | ||||
| * Image Scanner | ||||
| * Metadata Server | ||||
| * VM Init Scripts (dubbed as ucloud-init)How does ucloud work? | ||||
| 
 | ||||
| 
 | ||||
| Tech Stack | ||||
| ---------- | ||||
| The following technologies are utilised: | ||||
| 
 | ||||
| * Python 3 | ||||
| * Flask | ||||
| * QEMU as hypervisor | ||||
| * etcd (key/value store) | ||||
| * radvd for Router Advertisement | ||||
| 
 | ||||
| 
 | ||||
| Optional components: | ||||
| 
 | ||||
| * CEPH for distributed image storage | ||||
| * uotp for user authentication | ||||
| * netbox for IPAM | ||||
							
								
								
									
										32
									
								
								docs/source/misc/todo.rst
									
										
									
									
									
										Normal file
									
								
							
							
						
						
									
										32
									
								
								docs/source/misc/todo.rst
									
										
									
									
									
										Normal file
									
								
							|  | @ -0,0 +1,32 @@ | |||
| TODO | ||||
| ==== | ||||
| 
 | ||||
| Security | ||||
| -------- | ||||
| 
 | ||||
| * **Check Authentication:** Nico reported that some endpoints | ||||
|   even work without providing token. (e.g ListUserVM) | ||||
| 
 | ||||
| Refactoring/Feature | ||||
| ------------------- | ||||
| 
 | ||||
| * Put overrides for **IMAGE_BASE**, **VM_BASE** in **ImageStorageHandler**. | ||||
| * Expose more details in ListUserFiles. | ||||
| * Throw KeyError instead of returning None when some key is not found in etcd. | ||||
| * Create Network Manager | ||||
|     * That would handle tasks like up/down an interface | ||||
|     * Create VXLANs, Bridges, TAPs. | ||||
|     * Remove them when they are no longer used. | ||||
| 
 | ||||
| Reliability | ||||
| ----------- | ||||
| 
 | ||||
| * What to do if some command hangs forever? e.g CEPH commands | ||||
|   :code:`rbd ls ssd` etc. hangs forever if CEPH isn't running | ||||
|   or not responding. | ||||
| * What to do if etcd goes down? | ||||
| 
 | ||||
| Misc. | ||||
| ----- | ||||
| 
 | ||||
| * Put "Always use only one StorageHandler" | ||||
							
								
								
									
										323
									
								
								docs/source/setup-install.rst
									
										
									
									
									
										Normal file
									
								
							
							
						
						
									
										323
									
								
								docs/source/setup-install.rst
									
										
									
									
									
										Normal file
									
								
							|  | @ -0,0 +1,323 @@ | |||
| .. _setup-install: | ||||
| 
 | ||||
| Installation of ucloud | ||||
| ====================== | ||||
| To install ucloud, you will first need to install the requirements and | ||||
| then ucloud itself. | ||||
| 
 | ||||
| We describe the installation in x sections: | ||||
| 
 | ||||
| * Installation overview | ||||
| * Requirements on Alpine | ||||
| * Installation on Arch Linux | ||||
| 
 | ||||
| 
 | ||||
| Installation overview | ||||
| --------------------- | ||||
| 
 | ||||
| ucloud requires the following components to run: | ||||
| 
 | ||||
| * python3 | ||||
| * an etcd cluster | ||||
| 
 | ||||
| 
 | ||||
| Installation on Arch Linux | ||||
| -------------------------- | ||||
| 
 | ||||
| In Arch Linux, some packages can be installed from the regular | ||||
| repositories, some packages need to be installed from AUR. | ||||
| 
 | ||||
| 
 | ||||
| System packages | ||||
| ~~~~~~~~~~~~~~~ | ||||
| 
 | ||||
| .. code-block:: sh | ||||
|     :linenos: | ||||
| 
 | ||||
|     pacman -Syu qemu | ||||
| 
 | ||||
| 
 | ||||
| AUR packages | ||||
| ~~~~~~~~~~~~ | ||||
| Use your favorite AUR manager to install the following packages: | ||||
| 
 | ||||
| * etcd | ||||
| 
 | ||||
| 
 | ||||
| Alpine | ||||
| ------ | ||||
| 
 | ||||
| .. note:: | ||||
|     Python Wheel (Binary) Packages does not support Alpine Linux as it is | ||||
|     using musl libc instead of glibc. Therefore, expect longer installation | ||||
|     times than other linux distributions. | ||||
| 
 | ||||
| Enable Edge Repos, Update and Upgrade | ||||
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||||
| 
 | ||||
| .. warning:: | ||||
|     The below commands would overwrite your repositories sources and | ||||
|     upgrade all packages and their dependencies to match those available | ||||
|     in edge repos. **So, be warned** | ||||
| 
 | ||||
| .. code-block:: sh | ||||
|     :linenos: | ||||
| 
 | ||||
|     cat > /etc/apk/repositories << EOF | ||||
|     http://dl-cdn.alpinelinux.org/alpine/edge/main | ||||
|     http://dl-cdn.alpinelinux.org/alpine/edge/community | ||||
|     http://dl-cdn.alpinelinux.org/alpine/edge/testing | ||||
|     EOF | ||||
| 
 | ||||
|     apk update | ||||
|     apk upgrade | ||||
| 
 | ||||
|     reboot | ||||
| 
 | ||||
| 
 | ||||
| Install Dependencies | ||||
| ~~~~~~~~~~~~~~~~~~~~ | ||||
| .. note:: | ||||
|     The installation and configuration of a production grade etcd cluster | ||||
|     is out of scope of this manual. So, we will install etcd with default | ||||
|     configuration. | ||||
| 
 | ||||
| .. code-block:: sh | ||||
|     :linenos: | ||||
| 
 | ||||
|     apk add git python3 alpine-sdk python3-dev etcd etcd-ctl openntpd \ | ||||
|             libffi-dev openssl-dev make py3-protobuf py3-tempita chrony | ||||
| 
 | ||||
|     pip3 install pipenv | ||||
| 
 | ||||
| 
 | ||||
| **Install QEMU (For Filesystem based Installation)** | ||||
| 
 | ||||
| .. code-block:: sh | ||||
| 
 | ||||
|     apk add qemu qemu-system-x86_64 qemu-img | ||||
| 
 | ||||
| **Install QEMU/CEPH/radvd (For CEPH based Installation)** | ||||
| 
 | ||||
| .. code-block:: sh | ||||
| 
 | ||||
|     $(git clone https://code.ungleich.ch/ahmedbilal/qemu-with-rbd-alpine.git && cd qemu-with-rbd-alpine && apk add apks/*.apk --allow-untrusted) | ||||
|     apk add ceph radvd | ||||
| 
 | ||||
| Syncronize Date/Time | ||||
| ~~~~~~~~~~~~~~~~~~~~ | ||||
| 
 | ||||
| .. code-block:: sh | ||||
|     :linenos: | ||||
| 
 | ||||
|     service chronyd start | ||||
|     rc-update add chronyd | ||||
| 
 | ||||
| 
 | ||||
| Start etcd and enable it | ||||
| ~~~~~~~~~~~~~~~~~~~~~~~~ | ||||
| 
 | ||||
| .. note:: | ||||
|     The following :command:`curl` statement shouldn't be run once | ||||
|     etcd is fixed in alpine repos. | ||||
| 
 | ||||
| .. code-block:: sh | ||||
|     :linenos: | ||||
| 
 | ||||
|     curl https://raw.githubusercontent.com/etcd-io/etcd/release-3.4/etcd.conf.yml.sample -o /etc/etcd/conf.yml | ||||
|     service etcd start | ||||
|     rc-update add etcd | ||||
| 
 | ||||
| 
 | ||||
| Install uotp | ||||
| ~~~~~~~~~~~~ | ||||
| 
 | ||||
| .. code-block:: sh | ||||
|     :linenos: | ||||
| 
 | ||||
|     git clone https://code.ungleich.ch/ungleich-public/uotp.git | ||||
|     cd uotp | ||||
|     mv .env.sample .env | ||||
| 
 | ||||
|     pipenv --three --site-packages | ||||
|     pipenv install | ||||
|     pipenv run python app.py | ||||
| 
 | ||||
| Run :code:`$(cd scripts && pipenv run python get-admin.py)` to get | ||||
| admin seed. A sample output | ||||
| 
 | ||||
| .. code-block:: json | ||||
| 
 | ||||
|     { | ||||
|         "seed": "FYTVQ72A2CJJ4TB4", | ||||
|         "realm": ["ungleich-admin"] | ||||
|     } | ||||
| 
 | ||||
| Now, run :code:`pipenv run python scripts/create-auth.py FYTVQ72A2CJJ4TB4` | ||||
| (Replace **FYTVQ72A2CJJ4TB4** with your admin seed obtained in previous step). | ||||
| A sample output is as below. It shows seed of auth. | ||||
| 
 | ||||
| .. code-block:: json | ||||
| 
 | ||||
|     { | ||||
|         "message": "Account Created", | ||||
|         "name": "auth", | ||||
|         "realm": ["ungleich-auth"], | ||||
|         "seed": "XZLTUMX26TRAZOXC" | ||||
|     } | ||||
| 
 | ||||
| .. note:: | ||||
|     Please note both **admin** and **auth** seeds as we would need them in setting up ucloud. | ||||
| 
 | ||||
| 
 | ||||
| Install and configure ucloud | ||||
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||||
| 
 | ||||
| .. code-block:: sh | ||||
|     :linenos: | ||||
| 
 | ||||
|     git clone https://code.ungleich.ch/ucloud/ucloud.git | ||||
|     cd ucloud | ||||
| 
 | ||||
|     pipenv --three --site-packages | ||||
|     pipenv install | ||||
| 
 | ||||
| **Filesystem based Installation** | ||||
| 
 | ||||
| You just need to update **AUTH_SEED** in the below code to match your auth's seed. | ||||
| 
 | ||||
| .. code-block:: sh | ||||
|     :linenos: | ||||
| 
 | ||||
|     mkdir /etc/ucloud | ||||
| 
 | ||||
|     cat > /etc/ucloud/ucloud.conf << EOF | ||||
|     AUTH_NAME=auth | ||||
|     AUTH_SEED=XZLTUMX26TRAZOXC | ||||
|     AUTH_REALM=ungleich-auth | ||||
| 
 | ||||
|     REALM_ALLOWED = ["ungleich-admin", "ungleich-user"] | ||||
| 
 | ||||
|     OTP_SERVER="http://127.0.0.1:8000/" | ||||
| 
 | ||||
|     ETCD_URL=localhost | ||||
| 
 | ||||
|     STORAGE_BACKEND=filesystem | ||||
| 
 | ||||
|     BASE_DIR=/var/www | ||||
|     IMAGE_DIR=/var/image | ||||
|     VM_DIR=/var/vm | ||||
| 
 | ||||
|     VM_PREFIX=/v1/vm/ | ||||
|     HOST_PREFIX=/v1/host/ | ||||
|     REQUEST_PREFIX=/v1/request/ | ||||
|     FILE_PREFIX=/v1/file/ | ||||
|     IMAGE_PREFIX=/v1/image/ | ||||
|     IMAGE_STORE_PREFIX=/v1/image_store/ | ||||
|     USER_PREFIX=/v1/user/ | ||||
|     NETWORK_PREFIX=/v1/network/ | ||||
| 
 | ||||
|     ssh_username=meow | ||||
|     ssh_pkey="~/.ssh/id_rsa" | ||||
| 
 | ||||
|     VXLAN_PHY_DEV="eth0" | ||||
| 
 | ||||
|     EOF | ||||
| 
 | ||||
| 
 | ||||
| 
 | ||||
| **CEPH based Installation** | ||||
| You need to update the following | ||||
| 
 | ||||
| * **AUTH_SEED** | ||||
| * **NETBOX_URL** | ||||
| * **NETBOX_TOKEN** | ||||
| * **PREFIX** | ||||
| * **PREFIX_LENGTH** | ||||
| 
 | ||||
| 
 | ||||
| .. code-block:: sh | ||||
|     :linenos: | ||||
| 
 | ||||
|     mkdir /etc/ucloud | ||||
| 
 | ||||
|     cat > /etc/ucloud/ucloud.conf << EOF | ||||
|     AUTH_NAME=auth | ||||
|     AUTH_SEED=XZLTUMX26TRAZOXC | ||||
|     AUTH_REALM=ungleich-auth | ||||
| 
 | ||||
|     REALM_ALLOWED = ["ungleich-admin", "ungleich-user"] | ||||
| 
 | ||||
|     OTP_SERVER="http://127.0.0.1:8000/" | ||||
| 
 | ||||
|     ETCD_URL=localhost | ||||
| 
 | ||||
|     STORAGE_BACKEND=ceph | ||||
| 
 | ||||
|     BASE_DIR=/var/www | ||||
|     IMAGE_DIR=/var/image | ||||
|     VM_DIR=/var/vm | ||||
| 
 | ||||
|     VM_PREFIX=/v1/vm/ | ||||
|     HOST_PREFIX=/v1/host/ | ||||
|     REQUEST_PREFIX=/v1/request/ | ||||
|     FILE_PREFIX=/v1/file/ | ||||
|     IMAGE_PREFIX=/v1/image/ | ||||
|     IMAGE_STORE_PREFIX=/v1/image_store/ | ||||
|     USER_PREFIX=/v1/user/ | ||||
|     NETWORK_PREFIX=/v1/network/ | ||||
| 
 | ||||
|     ssh_username=meow | ||||
|     ssh_pkey="~/.ssh/id_rsa" | ||||
| 
 | ||||
|     VXLAN_PHY_DEV="eth0" | ||||
| 
 | ||||
|     NETBOX_URL="<url-for-your-netbox-installation>" | ||||
|     NETBOX_TOKEN="netbox-token" | ||||
|     PREFIX="your-prefix" | ||||
|     PREFIX_LENGTH="64" | ||||
|     EOF | ||||
| 
 | ||||
| 
 | ||||
| Install and configure ucloud-cli | ||||
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||||
| 
 | ||||
| .. code-block:: sh | ||||
|     :linenos: | ||||
| 
 | ||||
|     git clone https://code.ungleich.ch/ucloud/ucloud-cli.git | ||||
|     cd ucloud-cli | ||||
|     pipenv --three --site-packages | ||||
|     pipenv install | ||||
| 
 | ||||
|     cat > ~/.ucloud.conf << EOF | ||||
|     UCLOUD_API_SERVER=http://localhost:5000 | ||||
|     EOF | ||||
| 
 | ||||
|     mkdir /var/www/ | ||||
| 
 | ||||
| **Only for Filesystem Based Installation** | ||||
| 
 | ||||
| .. code-block:: sh | ||||
| 
 | ||||
|     mkdir /var/image/ | ||||
|     mkdir /var/vm/ | ||||
| 
 | ||||
| 
 | ||||
| Environment Variables and aliases | ||||
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||||
| 
 | ||||
| To ease usage of ucloud and its various components put the following in | ||||
| your shell profile e.g *~/.profile* | ||||
| 
 | ||||
| .. code-block:: sh | ||||
| 
 | ||||
|     export OTP_NAME=admin | ||||
|     export OTP_REALM=ungleich-admin | ||||
|     export OTP_SEED=FYTVQ72A2CJJ4TB4 | ||||
| 
 | ||||
|     alias ucloud='cd /root/ucloud/ && pipenv run python ucloud.py' | ||||
|     alias ucloud-cli='cd /root/ucloud-cli/ && pipenv run python ucloud-cli.py' | ||||
|     alias uotp='cd /root/uotp/ && pipenv run python app.py' | ||||
| 
 | ||||
| and run :code:`source ~/.profile` | ||||
							
								
								
									
										98
									
								
								docs/source/theory/summary.rst
									
										
									
									
									
										Normal file
									
								
							
							
						
						
									
										98
									
								
								docs/source/theory/summary.rst
									
										
									
									
									
										Normal file
									
								
							|  | @ -0,0 +1,98 @@ | |||
| Summary | ||||
| ======= | ||||
| 
 | ||||
| .. image:: /images/ucloud.svg | ||||
| 
 | ||||
| .. code-block:: | ||||
| 
 | ||||
|                               <cli> | ||||
|                                 | | ||||
|                                 | | ||||
|                                 | | ||||
|     +-------------------------<api> | ||||
|     |                           | | ||||
|     |           |```````````````|```````````````| | ||||
|     |           |               |               | | ||||
|     |     <file_scanner>    <scheduler>   <image_scanner> | ||||
|     |                           | | ||||
|     |                           | | ||||
|     +-------------------------<host> | ||||
|                                 | | ||||
|                                 | | ||||
|                                 | | ||||
|                          Virtual Machine------<init>------<metadata> | ||||
| 
 | ||||
| 
 | ||||
| 
 | ||||
| **ucloud-cli** interact with **ucloud-api** to do the following operations: | ||||
| 
 | ||||
| - Create/Delete/Start/Stop/Migrate/Probe (Status of) Virtual Machines | ||||
| - Create/Delete Networks | ||||
| - Add/Get/Delete SSH Keys | ||||
| - Create OS Image out of a file (tracked by file_scanner) | ||||
| - List User's files/networks/vms | ||||
| - Add Host | ||||
| 
 | ||||
| ucloud can currently stores OS-Images on | ||||
| 
 | ||||
| * File System | ||||
| * `CEPH <https://ceph.io/>`_ | ||||
| 
 | ||||
| 
 | ||||
| **ucloud-api** in turns creates appropriate Requests which are taken | ||||
| by suitable components of ucloud. For Example, if user uses ucloud-cli | ||||
| to create a VM, **ucloud-api** would create a **ScheduleVMRequest** containing | ||||
| things like pointer to VM's entry which have specs, networking | ||||
| configuration of VMs. | ||||
| 
 | ||||
| **ucloud-scheduler** accepts requests for VM's scheduling and | ||||
| migration. It finds a host from a list of available host on which | ||||
| the incoming VM can run and schedules it on that host. | ||||
| 
 | ||||
| **ucloud-host** runs on host servers i.e servers that | ||||
| actually runs virtual machines, accepts requests | ||||
| intended only for them. It creates/delete/start/stop/migrate | ||||
| virtual machines. It also arrange network resources needed for the | ||||
| incoming VM. | ||||
| 
 | ||||
| **ucloud-filescanner** keep tracks of user's files which would be needed | ||||
| later for creating OS Images. | ||||
| 
 | ||||
| **ucloud-imagescanner** converts images files from qcow2 format to raw | ||||
| format which would then be imported into image store. | ||||
| 
 | ||||
| * In case of **File System**, the converted image would be copied to | ||||
|   :file:`/var/image/` or the path referred by :envvar:`IMAGE_PATH` | ||||
|   environement variable mentioned in :file:`/etc/ucloud/ucloud.conf`. | ||||
| 
 | ||||
| * In case of **CEPH**, the converted image would be imported into | ||||
|   specific pool (it depends on the image store in which the image | ||||
|   belongs) of CEPH Block Storage. | ||||
| 
 | ||||
| **ucloud-metadata** provides metadata which is used to contextualize | ||||
| VMs. When, the VM is created, it is just clone (duplicate) of OS | ||||
| image from which it is created. So, to differentiate between my | ||||
| VM and your VM, the VM need to be contextualized. This works | ||||
| like the following | ||||
| 
 | ||||
| .. note:: | ||||
|     Actually, ucloud-init makes the GET request. You can also try it | ||||
|     yourself using curl but ucloud-init does that for yourself. | ||||
| 
 | ||||
| * VM make a GET requests http://metadata which resolves to actual | ||||
|   address of metadata server. The metadata server looks at the IPv6 | ||||
|   Address of the requester and extracts the MAC Address which is possible | ||||
|   because the IPv6 address is | ||||
|   `IPv6 EUI-64 <https://community.cisco.com/t5/networking-documents/understanding-ipv6-eui-64-bit-address/ta-p/3116953>`_. | ||||
|   Metadata use this MAC address to find the actual VM to which it belongs | ||||
|   and its owner, ssh-keys and much more. Then, metadata return these | ||||
|   details back to the calling VM in JSON format. These details are | ||||
|   then used be the **ucloud-init** which is explained next. | ||||
| 
 | ||||
| **ucloud-init** gets the metadata from **ucloud-metadata** to contextualize | ||||
| the VM. Specifically, it gets owner's ssh keys (or any other keys the | ||||
| owner of VM added to authorized keys for this VM) and put them to ssh | ||||
| server's (installed on VM) authorized keys so that owner can access | ||||
| the VM using ssh. It also install softwares that are needed for correct | ||||
| behavior of VM e.g rdnssd (needed for `SLAAC <https://en.wikipedia.org/wiki/IPv6#Stateless_address_autoconfiguration_(SLAAC)>`_). | ||||
| 
 | ||||
							
								
								
									
										24
									
								
								docs/source/troubleshooting.rst
									
										
									
									
									
										Normal file
									
								
							
							
						
						
									
										24
									
								
								docs/source/troubleshooting.rst
									
										
									
									
									
										Normal file
									
								
							|  | @ -0,0 +1,24 @@ | |||
| Installation Troubleshooting | ||||
| ============================ | ||||
| 
 | ||||
| etcd doesn't start | ||||
| ------------------ | ||||
| 
 | ||||
| .. code-block:: sh | ||||
| 
 | ||||
|     [root@archlinux ~]# systemctl start etcd | ||||
|     Job for etcd.service failed because the control process exited with error code. | ||||
|     See "systemctl status etcd.service" and "journalctl -xe" for details | ||||
| 
 | ||||
| possible solution | ||||
| ~~~~~~~~~~~~~~~~~ | ||||
| Try :code:`cat /etc/hosts` if its output contain the following | ||||
| 
 | ||||
| .. code-block:: sh | ||||
| 
 | ||||
|     127.0.0.1    localhost.localdomain localhost | ||||
|     ::1        localhost localhost.localdomain | ||||
| 
 | ||||
| 
 | ||||
| then unfortunately, we can't help you. But, if it doesn't contain the | ||||
| above you can put the above in :file:`/etc/hosts` to fix the issue. | ||||
							
								
								
									
										121
									
								
								docs/source/user-guide.rst
									
										
									
									
									
										Normal file
									
								
							
							
						
						
									
										121
									
								
								docs/source/user-guide.rst
									
										
									
									
									
										Normal file
									
								
							|  | @ -0,0 +1,121 @@ | |||
| .. _user-guide: | ||||
| 
 | ||||
| User Guide | ||||
| ========== | ||||
| 
 | ||||
| Create VM | ||||
| --------- | ||||
| 
 | ||||
| The following command would create a Virtual Machine (name: meow) | ||||
| with following specs | ||||
| 
 | ||||
| * CPU: 1 | ||||
| * RAM: 1GB | ||||
| * OS-SSD: 4GB | ||||
| * OS: Alpine Linux | ||||
| 
 | ||||
| .. code-block:: sh | ||||
| 
 | ||||
|     ucloud-cli vm create --vm-name meow --cpu 1 --ram '1gb' --os-ssd '4gb' --image images:alpine | ||||
| 
 | ||||
| 
 | ||||
| .. _how-to-check-vm-status: | ||||
| 
 | ||||
| Check VM Status | ||||
| --------------- | ||||
| 
 | ||||
| .. code-block:: sh | ||||
| 
 | ||||
|     ucloud-cli vm status --vm-name meow | ||||
| 
 | ||||
| .. code-block:: json | ||||
| 
 | ||||
|     { | ||||
|         "hostname": "/v1/host/74c21c332f664972bf5078e8de080eea", | ||||
|         "image_uuid": "3f75bd20-45d6-4013-89c4-7fceaedc8dda", | ||||
|         "in_migration": null, | ||||
|         "log": [ | ||||
|             "2019-11-12T09:11:09.800798 - Started successfully" | ||||
|         ], | ||||
|         "metadata": { | ||||
|             "ssh-keys": [] | ||||
|         }, | ||||
|         "name": "meow", | ||||
|         "network": [], | ||||
|         "owner": "admin", | ||||
|         "owner_realm": "ungleich-admin", | ||||
|         "specs": { | ||||
|             "cpu": 1, | ||||
|             "hdd": [], | ||||
|             "os-ssd": "4.0 GB", | ||||
|             "ram": "1.0 GB" | ||||
|         }, | ||||
|         "status": "RUNNING", | ||||
|         "vnc_socket": "/tmp/tmpj1k6sdo_" | ||||
|     } | ||||
| 
 | ||||
| 
 | ||||
| Connect to VM using VNC | ||||
| ----------------------- | ||||
| 
 | ||||
| We would need **socat** utility and a remote desktop client | ||||
| e.g Remmina, KRDC etc. We can get the vnc socket path by getting | ||||
| its status, see :ref:`how-to-check-vm-status`. | ||||
| 
 | ||||
| 
 | ||||
| .. code-block:: sh | ||||
| 
 | ||||
|     socat TCP-LISTEN:1234,reuseaddr,fork UNIX-CLIENT:/tmp/tmpj1k6sdo_ | ||||
| 
 | ||||
| 
 | ||||
| Then, launch your remote desktop client and connect to vnc://localhost:1234. | ||||
| 
 | ||||
| Create Network | ||||
| -------------- | ||||
| 
 | ||||
| Layer 2 Network with sample IPv6 range fd00::/64 (without IPAM and routing) | ||||
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||||
| .. code-block:: sh | ||||
| 
 | ||||
|     ucloud-cli network create --network-name mynet --network-type vxlan | ||||
| 
 | ||||
| 
 | ||||
| Layer 2 Network with /64 network with automatic IPAM | ||||
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||||
| .. code-block:: sh | ||||
| 
 | ||||
|     ucloud-cli network create --network-name mynet --network-type vxlan --user True | ||||
| 
 | ||||
| Attach Network to VM | ||||
| -------------------- | ||||
| 
 | ||||
| Currently, user can only attach network to his/her VM at | ||||
| the time of creation. A sample command to create VM with | ||||
| a network is as follow | ||||
| 
 | ||||
| .. code-block:: sh | ||||
| 
 | ||||
|     ucloud-cli vm create --vm-name meow2 --cpu 1 --ram '1gb' --os-ssd '4gb' --image images:alpine --network mynet | ||||
| 
 | ||||
| .. _get-list-of-hosts: | ||||
| 
 | ||||
| Get List of Hosts | ||||
| ----------------- | ||||
| 
 | ||||
| .. code-block:: sh | ||||
| 
 | ||||
|     ucloud-cli host list | ||||
| 
 | ||||
| 
 | ||||
| Migrate VM | ||||
| ---------- | ||||
| 
 | ||||
| .. code-block:: sh | ||||
| 
 | ||||
|     ucloud-cli vm migrate --vm-name meow --destination server1.place10 | ||||
| 
 | ||||
| 
 | ||||
| .. option:: --destination | ||||
| 
 | ||||
|    The name of destination host. You can find a list of host | ||||
|    using :ref:`get-list-of-hosts` | ||||
|  | @ -0,0 +1,53 @@ | |||
| How to create VM images for ucloud | ||||
| ================================== | ||||
| 
 | ||||
| Overview | ||||
| --------- | ||||
| 
 | ||||
| ucloud tries to be least invasise towards VMs and only require | ||||
| strictly necessary changes for running in a virtualised | ||||
| environment. This includes configurations for: | ||||
| 
 | ||||
| * Configuring the network | ||||
| * Managing access via ssh keys | ||||
| * Resizing the attached disk(s) | ||||
| 
 | ||||
| 
 | ||||
| Network configuration | ||||
| --------------------- | ||||
| All VMs in ucloud are required to support IPv6. The primary network | ||||
| configuration is always done using SLAAC. A VM thus needs only to be | ||||
| configured to | ||||
| 
 | ||||
| * accept router advertisements on all network interfaces | ||||
| * use the router advertisements to configure the network interfaces | ||||
| * accept the DNS entries from the router advertisements | ||||
| 
 | ||||
| 
 | ||||
| Configuring SSH keys | ||||
| -------------------- | ||||
| 
 | ||||
| To be able to access the VM, ucloud support provisioning SSH keys. | ||||
| 
 | ||||
| To accept ssh keys in your VM, request the URL | ||||
| *http://metadata/ssh_keys*. Add the content to the appropriate user's | ||||
| **authorized_keys** file. Below you find sample code to accomplish | ||||
| this task: | ||||
| 
 | ||||
| .. code-block:: sh | ||||
| 
 | ||||
|     tmp=$(mktemp) | ||||
|     curl -s http://metadata/ssk_keys > "$tmp" | ||||
|     touch ~/.ssh/authorized_keys # ensure it exists | ||||
|     cat ~/.ssh/authorized_keys >> "$tmp" | ||||
|     sort "$tmp" | uniq > ~/.ssh/authorized_keys | ||||
| 
 | ||||
| 
 | ||||
| Disk resize | ||||
| ----------- | ||||
| In virtualised environments, the disk sizes might grow. The operating | ||||
| system should detect disks that are bigger than the existing partition | ||||
| table and resize accordingly. This task is os specific. | ||||
| 
 | ||||
| ucloud does not support shrinking disks due to the complexity and | ||||
| intra OS dependencies. | ||||
		Loading…
	
	Add table
		Add a link
		
	
		Reference in a new issue