search

    Hive OS deploy PXE - deploy hundreds of rigs

    Introduction

    If you own a large farm with hundreds or thousands of GPU rigs and you are planning to migrate to Hive OS, Hive OS Deploy PXE can significantly reduce migration time and gain complete control over the deployment process. Hive OS Deploy uses PXE Network Boot technology to deploy Hive OS to your rigs.

    System and hardware requirements:

    • Any computer running Ubuntu 16.04 (tested)
    • Dual-core processor
    • Static IP address over cable Ethernet
    • At least 4 Gb of RAM
    • At least 6 Gb of free disk space

    You must set up a DHCP server by adding next-server option to CLONEDEPLOYSERVERIP (static IP address of host where you install Hive OS Deploy PXE). Read more in “Network infrastructure configuration” section.

    Network infrastructure configuration

    Depending on the configuration of your local network, the clients in your network receive IP addresses and other necessary information either dynamically using DHCP or configured statically on each client.

    LAN with dynamic IP addresses

    This is the default case or when you plan to setup a DHCP server anywhere in your network. Please note you should provide a set of dhcp-options:

    • dhcp-server ip address. Example: 192.168.1.1
    • dhcp network address. Example: 192.168.1.0
    • dhcp addresses range OR network mask. Example: 192.168.1.10-192.168.1.200 OR 192.168.1.0/24 (NOTE: different tools and hardware (routers) has it own syntax)
    • dhcp broadcast address. Example: 192.168.1.255
    • gateway (also known as router in dnsmasq). Example: 192.168.1.1
    • tftp-root (your tftp folder for storing kernels, initrds, pxe configs and binaries). Example: /tftp
    • dhcp-boot (PXE binary file PATH relative to tftp-root). Must be ,,CLONEDEPLOY_SERVER_IP
    • dns-server
    • next-server (dhcp option 66)

    NOTE: you must specify next-server with your CloneDeployServer IP address. It’s the only option that you must define if you have a DHCP-server already running. By default, most routers have predefined dhcp options and the ability to change them, so you must specify next-server option only.

    LAN with static IP addresses

    In case of STATIC network: you must configure dhcp-server and tftp-server anywhere in your network to use PXE technology. For example, you could use dnsmasq - lightweight DNS, DHCP and TFTP server.

    DHCP configuration examples

    See examples of configuration files in the examples/ folder

    • dnsmasq: example/dnsmasq.conf
    • dnsmasq : examples/same-server.dnsmasq.conf - all-in-one config example dhcp и deploy server on the same computer

    Installation of deploy server

    Installation procedure

    All installation commands and scripts in shell must be executed with root credentials by using sudo command.

    1. Download the installation package apt update && apt install wget zip unzip -y cd /opt && wget https://download.hiveos.farm/hivedeploy.zip && unzip hivedeploy.zip && cd cd_package
    pxe image
    1. Run the installation script ./install-ubuntu.sh -f

    It will start the installation procedure by downloading and installing the necessary packages. During the installation, you should answer some questions. Follow these steps:

    3.1. Enter the Deploy server’s IP address in prompt window pxe image

    Let’s assume that our host has IP address 192.168.1.2 then you will see: pxe image

    pxe image

    3.2. Prompt default DB connection string with pressing “Enter” pxe image

    Default settings will be used if no custom DB is given. pxe image

    3.3. Wait until mysql-server sets up, then prompt default root password by empty string (Press Enter). pxe image This screen can appear several times during this step.

    3.4. Enter SMB passwords for read-only and read-write roles: password pxe image

    pxe image

    3.5. Download and update Hive OS image The latest stable Hive OS image will be downloaded and extracted by default. pxe image

    3.6. Wait until the installation finishes the setup and then confirm reboot in the prompt window by entering “y”.

    1. The following steps are done through the web interface.

    Browse to http://<serverip>/clonedeploy for example if your server has ip address 192.168.1.2 then type in browser with http://192.168.1.2/clonedeploy and you will see the login screen: <img src="https://lbd.hiveos.farm/kbase/images/forum/32a0de42f8fcbaa912e36c5ebf59805a594528822_444x375.png" alt="pxe image">

    Login with default credentials:

    Login: admin

    Password: admin

    1. Go to Admin → PXE and press Update PXE settings pxe image
    pxe image pxe image
    1. Go to Admin → “Boot menu” and press “Create Boot Files”
    pxe image pxe image
    1. Go to Groups → deploy → Computer Properties.

    Change the following fields:

    pxe image

    pxe image pxe image

    pxe image
    1. Custom Attribute 2 - NETWORK. Enter your network with netmask, for example 192.168.1.0/24
    2. Custom Attribute 3 - GATEWAY. Enter the gateway, for example 192.168.1.1
    3. Custom Attribute 4 - HIVEOS_API. Enter hiveos-api url, for example http://api.hiveos.farm
    4. Custom Attribute 5 - FARM_HASH. Enter your farm hash, for example 0a188b553e98efba264e7bc61dee

    Note: Make sure the checkboxes near these fields are enabled. pxe image

    and click on the “Update Properties” button pxe image

    Now the server part installation is over.

    Checking installation and services

    Deploy server contains several packages including dhcp proxy, apache2, mysql and tftpd-hpa. Apache2 and MySQL - used for the web interface and tftpd-hpa used for TFTP server. So if you want to check the installation and make sure these services are running properly, use the following commands:

    for DHCP Proxy: service cd_proxy status

    for Apache2: service apache2 status

    for MySQL server: service mysql status

    for TFTP server: service tftpd-hpa status

    Deploy Rigs

    Prepare to Deploy

    You can organize your network according to certain rules.

    1. Assign rig name with MAC address
    2. Assign rig name with MAC address and IP address

    You can import a prepared rigs list. Go to “Computers” pxe image

    and then click the “Upload” button pxe image

    File format is: rig-001 08:00:27:4A:29:BB 192.168.1.19

    Check imported rigs pxe image

    Deploy Task

    Click “Task” pxe image

    Then “Click Computer Task” -> Start Computer Task -> select workers for Deploy pxe image

    and press “Deploy Selected”.

    Deploy process rig side pxe image

    Progress can be followed through the browser pxe image

    After downloading the necessary files, Hive OS Deploy will flash the Hive image to the storage device pxe image

    After the flashing process, the rig will reboot automatically followed by a boot menu with default target “Boot To Local Machine” which will boot your rig after a 5 second delay pxe image

    When the boot process is done, we’ll be able to see our rig on the Hive Dashboard pxe image

    Appendix

    • install-ubuntu.sh - script for install Hive OS Deploy PXE script options pxe image
    • download.sh - script for upload new images to Hive OS Deploy PXE pxe image
    • Hive OS Deploy PXE repository

    Quick Nav