This commit is contained in:
2024-02-23 06:07:41 +03:00
commit a83f4cd5e8
1963 changed files with 56869 additions and 0 deletions

View File

@@ -0,0 +1,181 @@
<h1> 1. Create a Farm </h1>
<h2> Table of Contents </h2>
- [Introduction](#introduction)
- [1) Creating a Polkadot.js account](#1-creating-a-polkadotjs-account)
- [2) Creating the Farm on the ThreeFold Grid](#2-creating-the-farm-on-the-threefold-grid)
- [Optional: Add Public IP Addresses](#optional-add-public-ip-addresses)
- [How to configure public IP addresses for a ThreeFold farm or a 3Node](#how-to-configure-public-ip-addresses-for-a-threefold-farm-or-a-3node)
- [The Difference Between IPs Assigned to Nodes Versus a Farm](#the-difference-between-ips-assigned-to-nodes-versus-a-farm)
- [Farm Creation Note](#farm-creation-note)
***
## Introduction
The following section will show you how to create a farm and a farm ID. In doing so, we will also learn how to create a ThreeFold Twin and how to use the polkadot.js extension on Chrome or Firefox to create an account on the blockchain. You can also [create a ThreeFold Farm with the ThreeFold Connect App](#create-a-farm).
***
## 1) Creating a Polkadot.js account
https://polkadot.js.org/extension/
Go on this link and download the extension based on your browser.
![Farming_Create_Farm_1](./img/farming_createfarm_1.png)
Choose Chrome or Firefox.
https://addons.mozilla.org/en-US/firefox/addon/polkadot-js-extension/
In this case, we use Firefox. Clicking on Download for Firefox leads us to this URL.
![Farming_Create_Farm_1](./img/farming_createfarm_2.png)
Click Add to Firefox.
![Farming_Create_Farm_3](./img/farming_createfarm_3.png)
Click Add.
![Farming_Create_Farm_4](./img/farming_createfarm_4.png)
This message tells you that the extension was added. Select Allow this extension to run in Private Windows if you want, then click Okay.
***
![Farming_Create_Farm_5](./img/farming_createfarm_5.png)
At first, you have no account. Click on the + sign to create an account.
![Farming_Create_Farm_6](./img/farming_createfarm_6.png)
Write down your 12-word mnemonic seed and select ***I have saved my mnemonic seed safely***, once youve done so!
![Farming_Create_Farm_7](./img/farming_createfarm_7.png)
Choose a descriptive name for your account. This name is not relevant. If you remove your account from the browser, you will need to enter your 12-word mnemonic seed and choose a new descriptive name.
You will need the password when accepting transactions. Choose it wisely.
![Farming_Create_Farm_8](./img/farming_createfarm_8.png)
This is what you see when your account is created.
***
## 2) Creating the Farm on the ThreeFold Grid
Go on the ThreeFold Dashboard.
```
https://dashboard.grid.tf/
```
As youve created and logged in your account, you will see your account in the list of connected accounts. Click on your account.
![Farming_Create_Farm_9](./img/tf_dashboard_2023_1.png)
You will be asked to read and accept the `Terms and Conditions`.
![Farming_Create_Farm_10](./img/tf_dashboard_2023_2.png)
You will be asked your password in order to confirm the transaction.
![Farming_Create_Farm_11](./img/tf_dashboard_2023_4.png)
***
Once you've confirmed the transaction, you want to create a relay address. The relay address should be the following:
```
relay.grid.tf
```
Make sure that `relay.grid.tf` is selected, then click "Create".
![Farming_Create_Farm_12](./img/tf_dashboard_2023_5.png)
You will once again be asked to confirm the transaction.
Then, click on the "Farms" button in the Portal menu to your left:
![Farming_Create_Farm_13](./img/tf_dashboard_2023_6.png)
On the "Farms" page, click on "Create Farm".
![Farming_Create_Farm_14](./img/tf_dashboard_2023_7.png)
Choose a farm name and click "Submit".
![Farming_Create_Farm_15](./img/tf_dashboard_2023_8.png)
Once the farm is created, click on the down arrow to see your farm details.
![Farming_Create_Farm_16](./img/tf_dashboard_2023_9.png)
In this section, click on "Add V2 Address" to add your farming rewards address.
![Farming_Create_Farm_17](./img/tf_dashboard_2023_10.png)
Write a valid Stellar address and click "Submit".
![Farming_Create_Farm_17](./img/tf_dashboard_2023_11.png)
You can, for example, use your ThreeFold Connect App* address or your hardware wallet address (Ledger, Trezor).
*ThreeFold Connect App Hyperlinks:
* Link for [Android](https://play.google.com/store/apps/details?id=org.jimber.threebotlogin&hl=en&gl=US)
* Link for [iOS](https://apps.apple.com/us/app/threefold-connect/id1459845885)
***
## Optional: Add Public IP Addresses
***
> Note: Assuming you are a farmer operating from your home, this field can usually be left blank. You do not have to fill in any details. The add IP option is for farmers that have a block of IP addresses routed to their router (in data centers mostly) and want to present “dedicated IP” addresses for deployments.
The add IP option is for farmers that have a block of IP addresses routed to their router. This is mainly for data centers, but some ISP can provide this service for residential locations. With such public IP addresses, your TF farm and/or your 3Nodes can present “dedicated IP” addresses for deployments.
Unless you are paying for a block of IPs, you only have one static IP which is needed for you router
The [ThreeFold simulator](https://simulator.grid.tf/) is referring to this IP address. Only check the Public IP option in the simulator if you have such block of IP addresses.
![farming_001](img/farming_001.png)
***
### How to configure public IP addresses for a ThreeFold farm or a 3Node
***
To configure public IP addresses for your 3Node farm, select the *Farms* section of the Portal menu in the [ThreeFold Dashboard](https://dashboard.grid.tf/), expand your farm by clicking on the down arrow on the left, then click on the down arrow to the right of the subsection *Public IPs*.
![Farming_Create_Farm_18](./img/tf_dashboard_2023_12.png)
Click on "Add IP".
![Farming_Create_Farm_19](./img/tf_dashboard_2023_13.png)
Specify the IP address using CIDR format, add the gateway and then click "Save".
![Farming_Create_Farm_20](./img/tf_dashboard_2023_14.png)
The same process can be done for individual 3Nodes. This is available in the *Nodes* section, just below the *Farms* section of the "Farms" page.
To configure public IP addresses to a specific 3Node, click on the Actions *Add a public config* (represented with a Earth icon).
![Farming_Create_Farm_21](./img/tf_dashboard_2023_15.png)
Fill in the necessary information and click save. Only the IPv4 address and gateway are necessary. The IPv6 address and the Domain are optional.
![Farming_Create_Farm_22](./img/tf_dashboard_2023_16.png)
***
### The Difference Between IPs Assigned to Nodes Versus a Farm
***
IPs assigned to a farm are available to be rented by workloads. They can be assigned to virtual machines for example. IPs assigned to nodes enable each node to become a gateway.
***
## Farm Creation Note
Please note that it is currently possible for anyone to create a node on any farm, by simply entering the farm ID on the [Zero-OS Boot Generator Assistant](https://bootstrap.grid.tf/). This is a known issue that we are working on.

View File

@@ -0,0 +1,169 @@
<h1> 2. Create a Zero-OS Bootstrap Image </h1>
<h2> Table of Contents </h2>
- [Introduction](#introduction)
- [Download the Zero-OS Bootstrap Image](#download-the-zero-os-bootstrap-image)
- [Burn the Zero-OS Bootstrap Image](#burn-the-zero-os-bootstrap-image)
- [CD/DVD BIOS](#cddvd-bios)
- [USB Key BIOS+UEFI](#usb-key-biosuefi)
- [Bootstrap Image for Certified 3Node](#bootstrap-image-for-certified-3node)
- [Expert Mode](#expert-mode)
- [Use a Specific Kernel](#use-a-specific-kernel)
- [Disable GPU](#disable-gpu)
- [Bootstrap Image URL](#bootstrap-image-url)
- [Zeros-OS Bootstrapping](#zeros-os-bootstrapping)
- [Zeros-OS Expert Bootstrap](#zeros-os-expert-bootstrap)
***
## Introduction
We will now learn how to create a Zero-OS bootstrap image in order to boot a DIY 3Node.
Note: For certified 3Nodes, read [this section](#bootstrap-image-for-certified-3node).
## Download the Zero-OS Bootstrap Image
Go on the [ThreeFold Zero-OS Bootstrap Link](https://v3.bootstrap.grid.tf):
```
https://v3.bootstrap.grid.tf
```
![Farming_Create_Farm_21](./img/farming_createfarm_21.png)
This is the Zero-OS v3 Bootstrapping page.
![Farming_Create_Farm_22](./img/farming_createfarm_22.png)
Write your farm ID and choose production mode.
![Farming_Create_Farm_23](./img/farming_createfarm_23.png)
If your system is new, you might be able to run the bootstrap in UEFI mode.
![Farming_Create_Farm_24](./img/farming_createfarm_24.png)
For older systems, run the bootstrap in BIOS mode. For BIOS CD/DVD, choose **ISO**. For BIOS USB, choose **USB**
Download the bootstrap image. Next, we will burn the bootstrap image.
## Burn the Zero-OS Bootstrap Image
### CD/DVD BIOS
For the BIOS **ISO** image, download the file and burn it on a DVD.
### USB Key BIOS+UEFI
For **MAC**, **Linux** and **Windows**, you can use [BalenaEtcher](https://www.balena.io/etcher/) to load/flash the image on a USB stick. This program also formats the USB in the process. This will work for the option **EFI IMG** for UEFI boot, and with the option **USB** for BIOS boot. Simply follow the steps presented to you and make sure you select the bootstrap image file you downloaded previously.
General Steps:
1. Download BalenaEtcher
2. Open BalenaEtcher
3. Select **Flash from file**
4. Find and select the bootstrap image (with your correct farm ID)
5. Select **Target** (your USB key)
6. Select **Flash**
That's it. Now you have a bootstrap image on Zero-OS as a bootable removable media device.
For the BIOS **USB** and the UEFI **EFI IMG** images, with Linux, you will want to do:
sudo dd status=progress if=FILELOCATION.ISO(or .IMG) of=/dev/sd*.
Here the * is to indicate that you must adjust according to your disk. To see your disks, write lsblk in the command window. Make sure you select the proper disk!
*If you USB key is not new, make sure that you format it before burning the Zero-OS image.
For Windows, if you are using the "dd" able image, instead of writing command line, you can use the free USB flashing program called [Rufus](https://sourceforge.net/projects/rufus.mirror/) and it will automatically do this with no need for Linux or command line. Rufus also formats the boot media in the process.
## Bootstrap Image for Certified 3Node
After creating a farm, to enable the certified farming status on your 3Node, you need to provide your farm ID and your farm name to the ThreeFold Support. You can do this by either using the [ThreeFold live chat](https://threefoldfaq.crisp.help/en/) or by sending an email at [info@threefold.io](mailto:info@threefold.io). The ThreeFold team will then send you a bootstrap image for your certified 3Node. You will need to replace the existing bootstrap image present on the USB key attached to the 3Node with this new bootstrap image for certified 3Node. Simply follow the steps below:
* Download the zip file containing your new bootstrap image.
* When you unzip the file, youll see a file named `BOOTX64.EFI` within the `BOOT/EFI` directory.
* Simply copy this file onto the USB key within the same directories that exist there, to overwrite the existing `BOOTX64.EFI` file.
* Insert the USB key in the 3Node and boot it again.
Your 3Node has now a bootstrap image for certified 3Node.
## Expert Mode
You can use the [expert mode](https://v3.bootstrap.grid.tf/expert) to generate specific Zero-OS bootstrap images.
Along the basic options of the normal bootstrap mode, the expert mode allows farmers to add extra kernel arguments and decide which kernel to use from a vast list of Zero-OS kernels.
### Use a Specific Kernel
You can use the expert mode to choose a specific kernel. Simply set the information you normally use and then select the proper kernel you need in the **Kernel** drop-down list.
![](./img/bootstrap_kernel_list.png)
### Disable GPU
You can use the expert mode to disable GPU on your 3Node.
![](./img/bootstrap_disable-gpu.png)
In the expert mode of the Zero-OS Bootstrap generator, fill in the following information:
- Farmer ID
- Your current farm ID
- Network
- The network of your farm
- Extra kernel arguments
- ```
disable-gpu
```
- Kernel
- Leave the default kernel
- Format
- Choose a bootstrap image format
- Click on **Generate**
- Click on **Download**
## Bootstrap Image URL
In both normal and expert mode, you can use the generated URL to quickly download a Zero-OS bootstrap image based on your farm specific setup.
Using URLs can be a very quick and efficient way to create new bootstrap images once your familiar with the Zero-OS bootstrap URL template and some potential varations.
```
https://<grid_version>.bootstrap.grid.tf/<image_format>/<network>/<farm_ID>/<arg1>/<arg2>/.../<kernel>
```
Note that the arguments and the kernel are optional.
The following content will provide some examples.
### Zeros-OS Bootstrapping
On the [main page](https://v3.bootstrap.grid.tf/), once you've written your farm ID and selected a network, you can copy the generated URL of any given image format.
For example, the following URL is a download link to an **EFI IMG** of the Zero-OS bootstrap image of farm 1 on the main TFGrid v3 network:
```
https://v3.bootstrap.grid.tf/uefimg/prod/1
```
### Zeros-OS Expert Bootstrap
You can use the generated sublink at the **Generate step** of the expert mode to get a quick URL to download your bootstrap image.
- After setting the parameters and arguments, click on **Generate**
- Add the **Target** content to the following URL `https://v3.bootstrap.grid.tf`
- For example, the following URL sets an **ipxe** script of the Zero-OS bootstrap of farm 1 on the main TFGrid v3 network, with the **disable-gpu** function enabled as an extra kernel argument and a specific kernel:
- ```
https://v3.bootstrap.grid.tf/ipxe/test/1/disable-gpu/zero-os-development-zos-v3-generic-b8706d390d.efi
```

View File

@@ -0,0 +1,397 @@
<h1> 3. Set the Hardware </h1>
<h2> Table of Contents </h2>
- [Introduction](#introduction)
- [Hardware Requirements](#hardware-requirements)
- [3Node Requirements Summary](#3node-requirements-summary)
- [Bandwidth Requirements](#bandwidth-requirements)
- [Link to Share Farming Setup](#link-to-share-farming-setup)
- [List of Common DIY 3Nodes](#list-of-common-diy-3nodes)
- [The Mini PC DIY 3Node](#the-mini-pc-diy-3node)
- [The Desktop PC DIY 3Node](#the-desktop-pc-diy-3node)
- [The Mini Server DIY 3Node](#the-mini-server-diy-3node)
- [The Tower Server DIY 3Node](#the-tower-server-diy-3node)
- [The Rack Server DIY 3Node](#the-rack-server-diy-3node)
- [Powering the 3Node](#powering-the-3node)
- [Surge Protector](#surge-protector)
- [Power Distribution Unit (PDU)](#power-distribution-unit-pdu)
- [Uninterrupted Power Supply (UPS)](#uninterrupted-power-supply-ups)
- [Generator](#generator)
- [Connecting the 3Node to the Internet](#connecting-the-3node-to-the-internet)
- [Z-OS and Switches](#z-os-and-switches)
- [Using Onboard Storage (3Node Servers)](#using-onboard-storage-3node-servers)
- [Upgrading a DIY 3Node](#upgrading-a-diy-3node)
***
## Introduction
In this section of the ThreeFold Farmers book, we cover the essential farming requirements when it comes to ThreeFold 3Node hardware.
***
## Hardware Requirements
You need a theoretical minimum of 500 GB of SSD and 2 GB of RAM on a mini pc, desktop or server. In short, for peak optimization, aim for 100 GB of SSD and 8GB of RAM per thread. (Thread is equivalent to virtual core or logical core.)
Also, TFDAO might implement a farming parameter based on [passmark](https://www.cpubenchmark.net/cpu_list.php). From the ongoing discussion on the Forum, you should aim at a CPU mark of 1000 and above per core.
> 3Node optimal farming hardware ratio -> 100 GB of SSD + 8 GB of RAM per Virtual Core
Note that you can run Zero-OS on a Virtual Machine (VM), but you won't farm any TFT from this process. To farm TFT, Zero-OS needs to be on bare metal.
Also, note that ThreeFold runs its own OS, which is Zero-OS. You thus need to start with completely wiped disks. You cannot farm TFT with Windows, Linux or MAC OS installed on your disks. If you need to use such OS temporarily, boot it in Try mode with a removable media (USB key).
Note: Once you have the necessary hardware, you need to [create a farm](./1_create_farm.md), [create a Zero-OS bootstrap image](./2_bootstrap_image.md), [wipe your disks](./4_wipe_all_disks.md) and [set the BIOS/UEFI](./5_set_bios_uefi.md) . Then you can [boot your 3Node](./6_boot_3node.md). If you are planning in building a farm in data center, [read this section](../advanced_networking/advanced_networking_toc.md).
***
### 3Node Requirements Summary
***
Any computer with the following specifications can be used as a DIY 3Node.
- Any 64-bit hardware with an Intel or AMD processor chip.
- Servers, desktops and mini computers type hardware are compatible.
- A minimum of 500 GB of SSD and a bare minimum of 2 GB of RAM is required.
- A ratio of 100GB of SSD and 8GB of RAM per thread is recommended.
- A wired ethernet connection is highly recommended to maximize reliability and the ability to farm TFT.
- A [passmark](https://www.passmark.com/) of 1000 per core is recommended and will probably be a minimum requirement in the future.
*A passmark of 1000 per core is recommend and will be a minimum requirement in the future. This is not yet an official requirement. A 3Node with less than 1000 passmark per core of CPU would not be penalized if it is registered before the DAO settles the [Passmark Question](https://forum.threefold.io/t/cpu-benchmarking-for-reward-calculations/2479).
***
## Bandwidth Requirements
<!---
This section should be checked and validated with the TF Team. We can change the constant (here it's 10) if needed. Or use another equation if this one is deemed suboptimal. This equation is an attempt at a synthesis of the discussions we had on the TF Forum.
-->
A 3Node connects to the ThreeFold Grid and transfers information, whether it is in the form of compute, storage or network units (CU, SU, NU respectively). The more resources your 3Nodes offer to the Grid, the more bandwidth will be needed to transfer the additional information. In this section, we cover general guidelines to make sure you have enough bandwidth on the ThreeFold Grid when utilization will be happening.
Note that the TFDAO will need to discuss and settle on clearer guidelines in the near future. For now, we propose those general guidelines. Being aware of these numbers as you build and scale your ThreeFold farm will set you in the proper direction.
> **The strict minimum for one Titan is 1 mbps of bandwidth**.
If you want to expand your ThreeFold farm, you should check the following to make sure your bandwidth will be sufficient when there will be Grid utilization.
***
**Bandwidth per 3Node Equation**
> min Bandwidth per 3Node (mbps) = 10 * max((Total SSD TB / 1 TB),(Total Threads / 8 Threads),(Total GB / 64 GB)) + 10 * (Total HDD TB / 2)
This equation means that for each TB of HDD you need 5 mbps of bandwidth, and for each TB of SSD, 8 Threads and 64GB of RAM (whichever is higher), you need 10 mbps of bandwidth.
This means a proper bandwidth for a Titan would be 10 mbps. As stated, 1 mbps is the strict minimum for one Titan.
***
## Link to Share Farming Setup
If you want ideas and suggestions when it comes to building DIY 3Nodes, a good place to start is by checking what other farmers have built. [This post on the Forum](https://forum.threefold.io/t/lets-share-our-farming-setup/286) is a great start. The following section also contains great DIY 3Node ideas.
***
## List of Common DIY 3Nodes
From the data collected by the ThreeFold DIY Farming Community, we can list some common 3Node builds. Note that these builds can be used as starting inspiration to build your own unique 3Node! That's the beauty of *the TF 3Node DIY Adventure*.
Building a 3Node depends a lot on the hardware accessibility in your specific location. Do your own research and check what are the best deals when it comes to buying computers and parts where you live.
We regroup the 3Node builds in 5 main categories:
1. [Mini PC](#the-mini-pc-diy-3node)
2. [Desktop](#the-desktop-pc-diy-3node)
3. [Mini server](#the-mini-server-diy-3node)
4. [Tower Server](#the-tower-server-diy-3node)
5. [Rack Server](#the-rack-server-diy-3node)
***
### The Mini PC DIY 3Node
Many farmers start with a Titan plug n play and then explore the world of Mini PC DIY 3Nodes.
***
**Lenovo M900 Tiny**
- Lenovo M900 Tiny
- Intel Core i7-6700T CPU (8 Threads)
- CPU Passmark per Core: 1817.75
- 1TB SSD
- Crucial 64GB RAM CT2K32G4SFD8266 (2x32GB) DDR4 2666 MHz CL19 Laptop Memory
A common Mini PC is the Lenovo M900 tiny. Equipped with an i7-6700T CPU (or i7-6700), this 3Node can have up to 64GB of RAM. Note that for a i7-6500 and below, the maximum is 32GB of RAM.
For this DIY 3Node, you usually need to buy the RAM and SSD separately to get the best price. Here's an [amazing TF video](https://www.youtube.com/watch?v=-AmDJYWWS5M) to get you started on this 3Node DIY journey.
***
### The Desktop PC DIY 3Node
***
**DELL OptiPlex 7020**
- DELL OptiPlex 7020
- i5-4570
- CPU Passmark per Core: 1300
- 32GB Ram
- 1x 512GB SSD
This 3Node is a great way to start your ThreeFold adventure: it requires no upgrade or any hardware tweaking.
It has the minimum SSD requirement as well as 32GB of RAM. This SSD/RAM ratio optimizes the farming reward. In short, for around 240$, you can farm around 257 TFT per month. With the current TFT price of 8 cents, this gives you a return on investment of less than 12 months. Building the New Internet isn't that hard after all!
***
**HP EliteDesk 800 G1 Desktop**
- HP EliteDesk 800 G1 Desktop
- Intel Core i7-6700
- CPU Passmark per Core: 2019
- 32GB DDR3 RAM
- 1TB SSD Hard Drive
- Zero-OS boot mode: UEFI
This model has a nice CPU Passmark ratio and has the same basic specs as a Titan. A great advantage of this model is that you can buy it as is, wipe the disk and plug in your bootstrap image and that's it, you're farming TFT! It's thus an excellent first DIY 3Node.
This build can be upgraded to 64GB of RAM, but it requires SODIMM RAM, which can be expenseive. DYOR.
***
### The Mini Server DIY 3Node
***
**Supermicro Pizza Box Chassis**
- AMD 5700G - 8 Core 16 Thread CPU
- CPU Passmark per Core: 3066.875
- Asrock Rack X470 Motherboard
- 128GB RAM
- 2x 2TB Kingston NVME SSDs
- 1x 8TB Enterprise SATA HDD
This 3Node build is very effective and has a good ratio investment/TFT rewards if you can get the parts online. This takes a little bit of DIY working as you need to assemble the parts. The CPU mark is also excellent.
***
### The Tower Server DIY 3Node
***
**HP Z800**
- HP Z800
- 2x Xeon 5650(16 Cores / 32 Threads)
- 192 GB RAM (12x 16GB DDR3 PC3L-8500R)
- CPU Passmark per Core: 960.83
- 3TB SSD (1x1TB + 1x2TB)
- Zero-OS boot mode: UEFI
***
**Dell Precision T5810**
- Dell Precision T5810
- Intel Xeon E5-2698 V3 (16 Cores / 32 Threads)
- CPU Passmark per Core: 1237.125
- 256GB SK Hynix DDR4 ECC RAM
- 4TB Crucial MX500 SSD
***
A Tower Server DIY 3Node is a good way to get major TFT rewards while also having an ergonomic and quiet 3Node. Note that the Rack Server DIY 3Node are usually louder, due to the fans spinning.
Building tower servers as DIY 3Nodes is also has a clear advantage: when GPU will be available on TF Grid, you will have ample room to add a decent GPU. This is thus future-proof in that regard.
***
### The Rack Server DIY 3Node
***
**DELL R720**
- Dell R720 refurbished (125 USD)
- 2 x Intel Xeon E5-2695 V2 (123 USD)
- CPU Passmark per Core: 1126.75
- 48 Threads
- 12 x 32GB (384GB) LR Ram ddr3 ebay (458 USD)
- 2 x 2TB ssd (360 USD)
- 1 x 1TB ssd (80 USD)
- cabels, caddies, etc: (15 USD)
- Zero-OS boot mode: BIOS
Total : 1161 USD
- 3122.5 TFT per month
- 175 Watt
- 126 kWh/month without any cultivation.
Estimated ROI: around 6 months
The prices are subjected to change depending on where you buy them. DYOR.
This 3Node build by @Valentine969 is quite amazing. You get great TFT rewards in a 2U rack server.
Note that 2U servers are quieter than 1U. The reason is simple, 2U servers have larger fans, thus they can move more volume of air with a slower speed than would 1U fans. This is a general statement and it depends on the specific hardware.
***
**DELL R620**
- Dell R620
- 2x E5 2640 V2
- CPU Passmark per Core: 967.625
- 32 Threads
- 256 GB RAM
- 16x16GB RAM DDR3 (256 GB)
- 2x 2TB SSD NVME with PCIe 3.0 X 4 Adaptor Card
- Zero-OS boot mode: BIOS
You can usually build the same 3Node build with a R620 as with a R720. The difference is that the R620 is a 1U server so it can generate more noise. The smaller form factor is a plus though. The HP Proliant
***
**DELL R630**
- E5-2630 v3 - 8 core / 16 thread CPU (32 threads total)
- CPU Passmark per Core: 1298.25
- 256GB DDR4 RAM
- Inland Premium 2TB NVME
- Intel DC P4500 2TB NVME
This server has the same specs as the R620 above, but it is a newer generation of server.
***
**HP Proliant DL 580 G8**
- 4 x Intel Xeon E7-4880 V2 15 core / 30 threads (60/120 total)
- 960 GB DDR3
- CPU Passmark per Core: 3229
- 5 TB SSD (1TB Samsung 860 EVO + 4TB Samsung 870 QVO)
- 3,3 TB HDD (few SAS Samsung / HP)
- Power Consumption (Idle): 283 W
This server is an amazing 3Node with a great CPU passmark ratio. The HDD won't make much difference in terms of farming rewards, but it's alway a nice addition if you have some spare disks to add for your 3Node!
***
Those were examples of what you can do as a DIY 3Node build. The possibilities are huge. Explore and build the New Internet!
***
## Powering the 3Node
### Surge Protector
A surge protector is highly recommended for your farm and your 3Nodes. This ensures your 3Nodes will not overcharge if a power surge happens. Whole-house surge protectors are also an option.
***
### Power Distribution Unit (PDU)
A PDU (power distribution unit) is useful in big server settings in order to manage your wattage and keep track of your power consumption.
***
### Uninterrupted Power Supply (UPS)
A UPS (uninterrupted power supply) is great for a 3Node if your power goes on and off frequently for short periods of time. This ensures your 3Node does not need to constantly reboot. If your electricity provider is very reliable, a UPS might not be needed, as the small downtime resulting from rare power outages with not exceed the DIY downtime limit*. (95% uptime, 5% downtime = 36 hours per month.) Of course, for greater Grid utilization experience, considering adding a UPS to your ThreeFold farm can be highly beneficial.
Note: Make sure to have AC Recovery Power set properly so your 3Node goes back online if power shutdowns momentarily. UPS are generally used in data center to make sure people have enough time to do a "graceful" shutdown of the units when power goes off. In the case of 3Nodes, they do not need graceful shutdowns as Zero-OS cannot lose data while functioning. The only way to power down a 3Node is simply to turn it off directly on the machine.
***
### Generator
A generator will be needed for very large installation with or without an unsteady main power supply.
***
## Connecting the 3Node to the Internet
As a general consideration, to connect a 3Node to the Internet, you must use an Ethernet cable and set DHCP as a network management protocol. Note that WiFi is not supported with ThreeFold farming.
The general route from the 3Node to the Internet is the following:
> 3Node -> Switch (optional) -> Router -> Modem
Note that most home routers come with a built-in switch to provide multiple Ethernet ports. Using a stand-alone switch is optional, but can come quite handy when farmers have many 3Nodes.
***
### Z-OS and Switches
Switches can be managed or unmanaged. Managed switches come with managed features made available to the user (typically more of such features on premium models).
Z-OS can work with both types of switches. As long as there's a router reachable on the other end offering DHCP and a route to the public internet, it's not important what's in between. Generally speaking, switches are more like cables, just part of the pipes that connect devices in a network.
We present a general overview of the two types of switches.
**Unmanaged Switches**
Unmanaged are the most common type and if someone just says "switch" this is probably what they mean. These switches just forward traffic along to its destination in a plug and play manner with no configuration. When a switch is connected to a router, you can think of the additional free ports on the switch as essentially just extra ports on the router. It's a way to expand the available ports and sometimes also avoid running multiple long cables. My nodes are far from my router, so I run a single long ethernet cable to a switch next to the nodes and then use multiple shorter cables to connect from the switch to the nodes.
**Managed Switches**
Managed switches have more capabilities than unmanaged switches and they are not very common in home settings (at least not as standalone units). Some of our farmers do use managed switches. These switches offer much more control and also require configuration. They can enable advanced features like virtual LANs to segment the network.
***
## Using Onboard Storage (3Node Servers)
If your 3Node is based on a server, you can either use PCIe slots and PCIe-NVME adapter to install SSD NVME disk, or you can use the onboard storage.
Usually, servers use RAID technology for onboard storage. RAID is a technology that has brought resilience and security to the IT industry. But it has some limitations that ThreeFold did not want to get stuck with. ThreeFold developed a different and more efficient way to [store data reliably](https://library.threefold.me/info/threefold#/cloud/threefold__cloud_products?id=storage-quantum-safe-filesystem). This Quantum Safe Storage overcomes some of the shortfalls of RAID and is able to work over multiple nodes geographically spread on the TF Grid. This means that there is no RAID controller in between data storage and the TF Grid.
For your 3Nodes, you want to bypass RAID in order for Zero-OS to have bare metals on the system.
To use onboard storage on a server without RAID, you can
1. [Re-flash](https://fohdeesha.com/docs/perc.html) the RAID card
2. Turn on HBA/non-RAID mode
3. Install a different card.
For HP servers, you simply turn on the HBA mode (Host Bus Adapter).
For Dell servers, you can either cross, or [re-flash](https://fohdeesha.com/docs/perc.html), the RAID controller with an “IT-mode-Firmware” (see this [video](https://www.youtube.com/watch?v=h5nb09VksYw)) or get a DELL H310-controller (which has the non-RAID option). Otherwise, you can install a NVME SSD with a PCIe adaptor, and turn off the RAID controller.
***
Once the disks are wiped, you can shutdown your 3Node and remove the Linux Bootstrap Image (USB key). Usually, there will be a message telling you when to do so.
***
## Upgrading a DIY 3Node
***
As we've seen in the [List of Common DIY 3Nodes](#list-of-common-diy-3nodes), it is sometimes necessary, and often useful, to upgrade your hardware.
**Type of upgrades possible**
- Add TBs of SSD/HDD
- Add RAM
- Change CPU
- Change BIOS battery
- Change fans
For some DIY 3Node, no upgrades are required and this constitutes a good start if you want to explore DIY building without going into too much additional steps.
For in-depth videos on how to upgrade mini-pc and rack servers, watch these great [DIY videos](https://www.youtube.com/user/floridanelson).
For a DIY picture guide on how to build a 3Node desktop computer, read [this documentation](../complete_diy_guides/3node_diy_desktop/3node_diy_desktop.md).
For a DIY picture guide on how to build a 3Node rack server, read [this documentation](../complete_diy_guides/3node_diy_rack_server/3node_diy_rack_server.md).

View File

@@ -0,0 +1,16 @@
![Farming_Title](./img/farming_30.png)
<h1> Building a DIY 3Node </h1>
This section of the ThreeFold Farmers book presents the necessary and basic steps to build a DIY 3Node. Note that we also include the information needed to set a certified 3Node.
For advanced farming information, such as GPU farming and room parameters, refer to the section [Farming Optimization](../farming_optimization/farming_optimization.md).
<h2> Table of Contents </h2>
- [1. Create a Farm](./1_create_farm.md)
- [2. Create a Zero-OS Bootstrap Image](./2_bootstrap_image.md)
- [3. Set the Hardware](./3_set_hardware.md)
- [4. Wipe All the Disks](./4_wipe_all_disks.md)
- [5. Set the BIOS/UEFI](./5_set_bios_uefi.md)
- [6. Boot the 3Node](./6_boot_3node.md)

View File

@@ -0,0 +1,104 @@
<h1> 4. Wipe All the Disks </h1>
<h2> Table of Contents </h2>
- [Introduction](#introduction)
- [Main Steps](#main-steps)
- [1. Create a Linux Bootstrap Image](#1-create-a-linux-bootstrap-image)
- [2. Boot Linux in *Try Mode*](#2-boot-linux-in-try-mode)
- [3. Use wipefs to Wipe All the Disks](#3-use-wipefs-to-wipe-all-the-disks)
- [Troubleshooting](#troubleshooting)
***
## Introduction
In this section of the ThreeFold Farmers book, we explain how to wipe all the disks of your 3Node.
## Main Steps
1. Create a Linux Bootstrap Image
2. Boot Linux in *Try Mode*
3. Wipe All the Disks
ThreeFold runs its own OS, which is Zero-OS. You thus need to start with completely wiped disks. Note that ALL disks must be wiped. Otherwise, Zero-OS won't boot.
An easy method is to simply download a Linux distribution and wipe the disk with the proper command line in the Terminal.
We will show how to do this with Ubuntu 20.04. LTS. This distribution is easy to use and it is thus a good introduction for Linux, in case you haven't yet explored this great operating system.
## 1. Create a Linux Bootstrap Image
Download the Ubuntu 20.04 ISO file [here](https://releases.ubuntu.com/20.04/) and burn the ISO image on a USB key. Make sure you have enough space on your USB key. You can also use other Linux Distro such as [GRML](https://grml.org/download/), if you want a ligther ISO image.
The process here is the same as in section [Burning the Bootstrap Image](./2_bootstrap_image.md#burn-the-zero-os-bootstrap-image), but with the Linux ISO instead of the Zero-OS ISO. [BalenaEtcher](https://www.balena.io/etcher/) is recommended as it formats your USB in the process, and it is available for MAC, Windows and Linux.
## 2. Boot Linux in *Try Mode*
When you boot the Linux ISO image, make sure to choose *Try Mode*. Otherwise, it will install Linux on your computer. You do not want this.
## 3. Use wipefs to Wipe All the Disks
When you use wipefs, you are removing all the data on your disk. Make sure you have no important data on your disks, or make sure you have copies of your disks before doing this operation, if needed.
Once Linux is booted, go into the terminal and write the following command lines.
First, you can check the available disks by writing in a terminal or in a shell:
```
lsblk
```
To see what disks are connected, write this command:
```
fdisk -l
```
If you want to wipe one specific disk, here we use *sda* as an example, write this command:
```
sudo wipefs -a /dev/sda
```
And replace the "a" in sda by the letter of your disk, as shown when you did *lsblk*. The term *sudo* gives you the correct permission to do this.
To wipe all the disks in your 3Node, write the command:
```
sudo for i in /dev/sd*; do wipefs -a $i; done
```
If you have any `fdisk` entries that look like `/dev/nvme`, you'll need to adjust the command line.
For a nvme disk, here we use *nvme0* as an example, write:
```
sudo wipefs -a /dev/nvme0
```
And replace the "0" in nvme0 by the number corresponding to your disk, as shown when you did *lsblk*.
To wipe all the nvme disks, write this command line:
```
sudo for i in /dev/nvme*; do wipefs -a $i; done
```
## Troubleshooting
If you're having issues wiping the disks, you might need to use **--force** or **-f** with wipefs (e.g. **sudo wipefs -af /dev/sda**).
If you're having trouble getting your disks recognized by Zero-OS, some farmers have had success enabling AHCI mode for SATA in their BIOS.
If you are using a server with onboard storage, you might need to [re-flash the RAID card](/faq/faq.md#is-there-a-way-to-bypass-raid-in-order-for-zero-os-to-have-bare-metals-on-the-system-no-raid-controller-in-between-storage-and-the-grid).

View File

@@ -0,0 +1,665 @@
<h1> 5. Set the BIOS/UEFI </h1>
<h2> Table of Contents </h2>
- [Introduction](#introduction)
- [Z-OS and DHCP](#z-os-and-dhcp)
- [Regular Computer and 3Node Network Differences](#regular-computer-and-3node-network-differences)
- [Static IP Addresses](#static-ip-addresses)
- [The Essential Features of BIOS/UEFI for a 3Node](#the-essential-features-of-biosuefi-for-a-3node)
- [Setting the Remote Management of a Server (Static IP Address)](#setting-the-remote-management-of-a-server-static-ip-address)
- [Update the BIOS/UEFI firmware](#update-the-biosuefi-firmware)
- [Check the BIOS/UEFI version on Windows](#check-the-biosuefi-version-on-windows)
- [Check the BIOS/UEFI version on Linux](#check-the-biosuefi-version-on-linux)
- [Update the BIOS firmware](#update-the-bios-firmware)
- [BIOS/UEFI and Zero-OS Bootstrap Image Combinations](#biosuefi-and-zero-os-bootstrap-image-combinations)
- [Example of BIOS Settings - Rack Server - Dell R720](#example-of-bios-settings---rack-server---dell-r720)
- [Example of UEFI Settings - Desktop Computer - Dell Optiplex 7010](#example-of-uefi-settings---desktop-computer---dell-optiplex-7010)
***
## Introduction
In this section of the ThreeFold Farmers book, we explain how to properly set the BIOS/UEFI of your 3Node.
Note that the BIOS mode is usually needed for older hardware while the UEFI mode is usually needed for newer hardware, when it comes to booting properly Zero-OS on your DIY 3Node.
If it doubt, start with UEFI and if it doesn't work as expected, try with BIOS.
Before diving into the BIOS/UEFI settings, we will present some general considerations on Z-OS and DHCP.
## Z-OS and DHCP
The operating system running on the 3Nodes is called Zero-OS (Z-OS). When it comes to setting the proper network for your 3Node farm, you must use DHCP since Z-OS is going to request an IP from the DHCP server if there's one present, and it won't get network connectivity if there's no DHCP.
The Z-OS philosophy is to minimize configuration wherever possible, so there's nowhere to supply a static config when setting your 3Node network. Instead, the farmer is expected to provide DHCP.
While it is possible to set fixed IP addresses with the DHCP for the 3Nodes, it is recommended to avoid this and just set the DHCP normally without fixed IP addresses.
By setting DHCP in BIOS/UEFI, an IP address is automatically assigned by your router to your 3Node every time you boot it.
### Regular Computer and 3Node Network Differences
For a regular computer (not a 3Node), if you want to use a static IP in a network with DHCP, you'd first turn off DHCP and then set the static IP to an IP address outside the DHCP range. That being said, with Z-OS, there's no option to turn off DHCP and there's nowhere to set a static IP, besides public config and remote management. In brief, the farmer must provide DHCP, either on a private or a public range, for the 3Node to boot.
### Static IP Addresses
In the ThreeFold ecosystem, there are only two situations where you would work with static IP addresses: to set a public config to a 3Node or a farm, and to remotely manage your 3Nodes.
**Static IP and Public Config**
You can [set a static IP for the public config of a 3Node or a farm](./1_create_farm.md#optional-add-public-ip-addresses). In thise case, the 3Node takes information from TF Chain and uses it to set a static configuration on a NIC (or on a virtual NIC in the case of single NIC systems).
**Static IP and Remote Management**
You can [set a static IP address to remotely manage a 3Node](#setting-the-remote-management-of-a-server-static-ip-address).
***
## The Essential Features of BIOS/UEFI for a 3Node
There are certain things that you should make sure are set properly on your 3Node.
As a general advice, you can Load Defaults (Settings) on your BIOS, then make sure the options below are set properly.
* Choose the correct combination of BIOS/UEFI and bootstrap image on [https://bootstrap.grid.tf/](https://bootstrap.grid.tf/)
* Newer system will use UEFI
* Older system will use BIOS
* Hint: If your 3Node boot stops at *Initializing Network Devices*, try the other method (BIOS or UEFI)
* Set Multi-Processor and Hyperthreading at Enabled
* Sometimes, it will be written Virtual Cores, or Logical Cores.
* Set Virtualization at Enabled
* On Intel, it is denoted as CPU virtualization and on ASUS, it is denoted as SVM.
* Make sure virtualization is enabled and look for the precise terms in your specific BIOS/UEFI.
* Set AC Recovery at Last Power State
* This will make sure your 3Node restarts after losing power momentarily.
* Select the proper Boot Sequence for the 3Node to boot Zero-OS from your bootstrap image
* e.g., if you have a USB key as a bootstrap image, select it in Boot Sequence
* Set Server Lookup Method (or the equivalent) at DNS. Only use Static IP if you know what you are doing.
* Your router will assign a dynamic IP address to your 3Node when it connects to Internet.
* Set Client Address Method (or the equivalent) at DHCP. Only use Static IP if you know what you are doing.
* Your router will assign a dynamic IP address to your 3Node when it connects to Internet.
* Secure Boot should be left at disabled
* Enable it if you know what you are doing. Otherwise, it can be set at disabled.
***
## Setting the Remote Management of a Server (Static IP Address)
***
Note from the list above that by enabling the DHCP and DNS in BIOS, dynamic IP addresses will be assigned to 3Nodes. This way, you do not need any specific port configuration when booting a 3Node.
As long as the 3Node is connected to the Internet via an ethernet cable (WiFi is not supported), Zero-OS will be able to boot. By setting DHCP in BIOS, an IP address is automatically assigned to your 3Node every time you boot it. This section concerns 3Node servers with remote management functions and interfaces.
You can set up a node through static routing at the router without DHCP by assigning the MAC address of the NIC to a IP address within your private subnet. This will give a static IP address to your 3Node.
With a static IP address, you can then configure remote management on servers. For Dell, [iDRAC](https://www.dell.com/support/kbdoc/en-us/000134243/how-to-setup-and-manage-your-idrac-or-cmc-for-dell-poweredge-servers-and-blades) is used, and for HP, [ILO](https://support.hpe.com/hpesc/public/docDisplay?docId=a00045463en_us&docLocale=en_US) is used.
***
## Update the BIOS/UEFI firmware
***
Updating the BIOS firmware is not always necessary, but to do so can help prevent future errors and troubleshootings. Making sure the Date and Time are set correctly can also help the booting process.
Note: updating the BIOS/UEFI firmware is optional, but recommended.
***
### Check the BIOS/UEFI version on Windows
***
Hit *Start*, type in *cmd* in the search box and click on *Command Prompt*. Write the line
> wmic bios get smbiosbiosversion
This will give you the BIOS or UEFI firmware of your PC.
***
### Check the BIOS/UEFI version on Linux
***
Simply type the following command
> sudo dmidecode | less
or this line:
> sudo dmidecode -s bios-version
***
### Update the BIOS firmware
***
1. On the manufacturer's website, download the latest BIOS/UEFI firmware
2. Put the file on a USB flash drive (+unzip if necessary)
3. Restart your hardware and enter the BIOS/UEFI settings
4. Navigate the menus to update the BIOS/UEFI
***
## BIOS/UEFI and Zero-OS Bootstrap Image Combinations
***
To boot your 3Node, you need to create your own boot device for your system. This might sound like a big project to accomplish, but it's actually pretty simple. As simple as can be, you need a removable media (USB key for example) where you will burn/install the Zero-OS bootstrap image on it. The type of file you burn on the removable media will change depending if your system is old or recent. Older system uses BIOS and newer system uses UEFI.
Whether you use BIOS or UEFI, 3Nodes are also self-healing, so once you set up your 3Node, maintenance is minimal.
To properly boot the Zero-OS image, you can either use an image made for a BIOS system or a UEFI system, this depends on your system.
BIOS is older technology. It means *Basic Input/Output System*.
UEFI is newer technology. It means *Unified Extensible Firmware Interface*. BIOS/UEFI is, in a way, the link between the hardware and the software of your computer.
In general, setting a 3Node is similar whether it is with a BIOS or UEFI system. The important is to choose the correct combination of boot media + boot mode (BIOS/UEFI).
The bootstrap images are available [here](https://bootstrap.grid.tf/).
The choices are:
1. EFI IMG - UEFI
2. EFI FILE - UEFI
3. iPXE - Boot from network
4. ISO - BIOS
5. USB - BIOS
6. LKRN - Boot from network
Choices 1 and 2 are for UEFI (newer models).
Choices 4 and 5 are for BIOS (newer models).
Choices 3 and 6 are mainly for network boot.
For information on how to boot Zero-OS with iPXE, read [this section](./6_boot_3node.md#advanced-booting-option---network-booting-pxe).
You might have to try UEFI first and if it doesn't work, try BIOS. Usually when this is the case (UEFI doesn't work with your current computer), the following message will be shown:
> Initializing Network Devices...
And then... nothing. This means that you are still in the BIOS of the hardware and boot is not even started yet. When this happens, try the BIOS mode of your computer.
***
**Additional information :** One of the great features of Zero-OS is that it can be completely run within the cache of your 3Node. Indeed, the booting device, that contains your farm ID, will connect to the ThreeFold Grid and download everything needed to run smoothly. There are many benefits in terms of security and protection of data that comes with this.
***
Let's go into BIOS mode now!
## Example of BIOS Settings - Rack Server - Dell R720
***
The following covers proper settings for the BIOS mode of a DIY 3Node. Note that it is also called the normal BIOS or legacy BIOS.
There might be other ways to configure those settings, depending on the specific computer. Depending on your hardware (disks connections, RAID controller, etc.), you might need to change some settings to fit the specific model. Most of the time, if you follow these lines, your 3Node should work 100% with the Grid. Check the [Troubleshooting and Error Messages](/faq/faq.md#troubleshooting-and-error-messages) section if you have trouble on your way to booting Zero-OS.
It is possible that your 3Node needs to be booted with the UEFI mode. In this case, refer to the [next section](#example-of-uefi-settings---desktop-computer---dell-optiplex-7010).
***
Note that the settings are similar for UEFI and non-UEFI. But some details are important.
To boot Zero-OS, you need to use the bootstrap image [here](https://bootstrap.grid.tf/).
Old servers/desktops will need to use the 4th and 5th options (named: ISO and USB). Newer servers/desktops can use UEFI (1st and 2nd option).
The BIOS setting in BIOS (non-UEFI) mode presented here is mostly based on a Dell R720. We also add information to cover more broadly the subject. There may be differences with your specific 3Node hardware. The following should give you a good overview of how to set it up and you can adjust the details. For newer servers, UEFI mode might be the preferred option.
***
Please note that you can set your iDRAC with remote access. In this section, we set up the 3Node directly with a monitor and keyboard. Once this BIOS settings are set and the 3Node is running, you will not need to do much maintenance on the server's BIOS or iDRAC settings, thanks to the Self-Healing properly of ThreeFold's Zero-OS.
***
SYSTEM SETUP
***
To enter into system setup, you need to press a certain key when the computer is powered on. With Dell server R720, hit F2 to get into System Setup.
To set the 3Node running, you will need to configure the following sections:
- System BIOS
- iDRAC Settings
First, let's go into System BIOS.
***
**System BIOS**
***
MEMORY SETTINGS
***
In `System BIOS`, go into `Memory Settings` and make sure the Memory parameters are correct (Size, Type, Speed, Voltage). If you have less GB of RAM than expected, perhaps the RAM sticks are not well seated.
You will have the choice of enabling or disabling the function `System Memory Testing`. You can leave this at `disabled` in general. You will want it at `enabled` when you want to tests the condition and integrity of the 3Node's memory.
The options for `Memory Operating Mode` are the following, with **Optimizer Mode** being the option by default, which you can leave as such:
- Optimizer Mode
- Advanced ECC Mode
- Mirror Mode
- Spare Mode
- Dell Fault Resilient Mode
You will have the choice of enabling or disabling the function `Node Interleaving`.
Usually your system will run faster with this option `disabled`.
When `enabled`, Node Interleaving will interleave the memory addresses across the memory in each processor. In some situation and for certain workload, this can improve the performance.
***
PROCESSOR SETTINGS
***
Next, we will adjust the proper settings for the processors in `Processor Settings`.
| # | Settings | Option |
|---|---|---|
| 1 | Logical Processors | **Enabled** |
| 2 | QPI Speed | Maximum datarate |
| 3| Alternate RTID | Disabled |
| 4 | Virtualization Technology | Enabled |
| 5 | Adjacent Cache Line Prefetch | Enabled |
| 6 | Hardware Prefetcher | Enabled |
| 7 | DCU Streamer Prefetcher | Enabled |
| 8 | DCU IP Prefetcher | Enabled |
| 9 | Execute Disable | Enabled |
| 10 | Logical Processor Idling | Disabled |
| 11 | Dell Controlled Turbo | Disabled |
| 12 | Number of Cores per Processor | **All** |
Setting Logical Processor at Enabled is particularly important. If you're supposed to have `x` vcores, and the Explorer says you only have `x/2` (only half of expected vcores), chances are you need to enable this setting. Also make sure the number of cores is set to **All**.
Next in the section Processor Settings, you will see the information of your processor(s). The number of cores per processor will also be shown. In short, make sure you have the expected processors and number of cores.
***
SATA SETTINGS
***
Next, we will adjust the proper software settings for the SATA connections and hardware in `SATA Settings`.
| # | Settings | Option |
|---|---|---|
| 1 | Embedded SATA | **AHCI**/RAID/ATA/Off |
| 2 | Port A (First port) | Auto |
| 3| ... | ... |
| 4 | Port Z (Last port) | Auto |
If you are using SATA disks, you probably want #1 set to AHCI. Depending on your disks configurations, you might need to chance this parameter.
The rest of the settings are for the Ports. You can always check in the User's Guide of your server to check which ports are for which hardware. For example, on Dell R720, Port E is for the optical drive (CD/DVD).
***
BOOT SETTINGS
***
Next, we will adjust the proper settings for the processors `BOOT Settings`.
| # | Settings | Option |
|---|---|---|
| 1 | BOOT Mode | **BIOS** / UEFI |
| 2 | Boot Sequence Retry | **Enabled** / Disabled |
| 3| BIOS Boot Settings | ... |
| 4 | BIOS Boot Settings | ... |
Since we are doing a BIOS boot, and not a UEFI boot, for the 3Node, make sure you have BIOS mode instead of UEFI selected. Otherwise it won't boot properly. For newer servers, work with UEFI instead.
You can have Boot Sequence Retry Enabled, but your server should boot fine without. When enabled, the system will reattempt the boot sequence 30 seconds after a failed booting sequence.
Then you need to save these changes and come back to access BIOS Boot Settings (if you were previously set to UEFI mode).
***
BIOS Boot Settings
***
Next, we will adjust the proper settings for the processors `BIOS Boot Settings`. Note that USB keys will often be called **Hard drive C:**. Click on **Boot Sequence** and choose the proper order for your 3Node configuration. Here we choose **Hard drive C:** as we will be booting from USB key. If you use another method than a USB key, make sure you select it.
- BOOT Sequence
- **Hard drive C:**
- SATA Port
- Integrated NIC
- ...
- Boot Option Enable/Disable
- [x] Hard drive C:
- [ ] SATA Port:
- [ ] ...
In Boot Option Enable/Disable, you can check the option you will be using for your booting device. Here we check `Hard drive C:`, as we are booting Zero-OS from a USB key.
***
INTEGRATED DEVICES
***
Next, we will adjust the proper settings for the `Integrated Devices`.
This will change depending on your hardware configuration.
| # | Settings | Option |
|---|---|---|
| 1 | Integrated RAID Controller | Enabled / **Disabled** |
| 2 | User Accesible USB Ports | **All Ports On** / Only Back Ports / All Ports Off |
| 3 | Internal USB | **On** / Off |
| 4 | Integrated Network Card 1 | **Enabled** / Disabled |
| 5 | OS Watchdog Timer | Enabled / **Disabled** |
| 6 | Embedded Video Controller | **Enabled** / Disabled |
| 7 | SR-IOV Global Enable | **Enabled** /*Disabled |
| 8 | Memory Mapped I/O above 4 GB | **Enabled** / Disabled |
| 9 | Slot Disablement | **Enabled** / Disabled |
1. The RAID controller can be disabled if you are plugging your disks with a PCIe adaptors or the SATA connection of the optical drive. If you are using the front panel disks, you will need to deal with RAID configuration. You can also use RAID 0. But in general, 3Nodes do not use RAID.
You can use the onboard storage on a server without RAID. You can [re-flash](https://fohdeesha.com/docs/perc.html) the RAID card, turn on HBA/non-RAID mode, or install a different card. There is no need for RAID on a 3Node.
It's usually easy to set servers such as a HP Proliant with the HBA mode. For Dell servers, you can either cross-flash the RAID controller with an “IT-mode-Firmware” (see this [video](https://www.youtube.com/watch?v=h5nb09VksYw)) or get a DELL H310-controller (which has the non-RAID option). Otherwise, you can install a NVME SSD with a PCIe adaptor as stated above.
Concerning RAID technology, here is what Weynand Kuijpers, one of ThreeFold's co-founders, has to say:
> RAID is a technology that has brought resilience and security to the IT industry. But it has some limitations that we at ThreeFold did not want to get stuck in. We developed a different (and more efficient way to store data reliably). Please have a look [here](https://library.threefold.me/info/threefold#/cloud/threefold__cloud_products?id=storage-quantum-safe-filesystem).
>
> This Quantum Safe Storage overcomes some of the shortfalls of RAID and is able to work over multiple nodes geographically spread on the TF Grid.
2. Usually you can leave all USB ports accessible.
3. You can choose to leave your booting device inside the server (this is recommended). You might sometimes have to upgrade the booting device, but minor upgrades are always done via the ThreeFold Grid so this would not happen often.
4. It is important to enable your Network Card so your 3Node can connect to the ThreeFold Grid.
5. OS Timer is not necessary as Zero-OS is self-healing.
6. By default, this parameter is enabled. The video controller is used to check the 3Node Zero-OS status. Note that it is possible to run a 3Node headless, and/or without a GPU connected.
7. This is for virtualization devices.
8. Memory Mapped is to perit PCIe devices to uses a lot of memory. Useful for NVME SSD PCIe adaptor 3Node build.
9. To enable PCIe slots. You can leave them all enabled, and disabled them for specific use cases.
| # | Settings | Option |
|---|---|---|
| 1 | First Slot (1) | **AHCI**/RAID/ATA/Off |
| 3| ... | ... |
| 4 | Last Slot (7) | **Enabled** / Disabled / Boot Driver Disabled |
All slots should be set at Enabled in Slot Disablement. You might want to disable unused slots depending on your spectific configuration.
***
SERIAL COMMUNICATION
***
Next, we will adjust the proper settings for the `Serial Communication`.
This can change depending on your hardware configuration.
| # | Settings | Option |
|---|---|---|
| 1 | Serial Communication | **On without Console Redirection** |
| 2| Serial Port Address | **Serial Device 1=COM2, Serial Device 2=COM1** |
| 3 | External Serial Connector | **Serial Device1** |
| 4 | Failsafe Baud Rate | **115200** |
| 5| Remote Terminal Type | **VTT100/VT220** / ANSI |
| 6 | Redirection After Boot | **Enabled** / Disabled |
In general, this section can be left with the default parameters. It can be useful when you have specific connections and want to use remote access device.
1. To select serial communication devices.
2. This can be used to set the port address for serial devices.
***
SYSTEM PROFILE
***
Next, we will adjust the proper settings for the `System Profile`.
This can change depending on your hardware configuration.
| # | Settings | Option |
|---|---|---|
| 1 | System Profile | **Performance Per Watt (DAPC)** |
| 2| CPU Power Management | **System DBPM (DAPC)** |
| 3 | Memory Frequency | **Maximum Performance** |
| 4 | Turbo Boost | **Enabled** |
| 5| C1E | **Enabled**|
| 6| C States | **Enabled** |
| 7| Monitor/Mwait | **Enabled** |
| 8| Memory Patrol Scrub | **Standard** |
| 9| Memory Refresh Rate | **1x** |
| 10| Memory Operating Voltage | **Auto** |
| 11| Collaborative CPU Performance Control | **Disabled** |
System Profile Settings can be at Performance Per Watt. In this case, all the parameters are locked to default. This is aimed at being efficient and consume less power.
You can also use custom parameters. Make sure you know what you are doing. Many things can be done such as changing the frequency and the DIMM voltage selection. This can, in certain situations, increase the 3Node's performance.
***
SYSTEM SECURITY
***
Next, we will adjust the proper settings for the `System Security`. Here you can set a password for the BIOS and also set the AC Power Recovery with a set detay.
| # | Settings | Option |
|---|---|---|
| 1 | Password Status | **Unlocked** / Locked |
| 2 | ... | ... |
| 3 | Power Button | **Enabled** / Disabled |
| 4 | ... | ... |
| 5 | AC Power Recovery | **Last** / On / Off |
| 6 | AC Power Recovery Delay | **User Defined*** |
| 7 | ... | ... |
*In System Security, it can be very useful to set AC Power Recovery at **Last**. If your 3Node is powered on and you have an electric outage, your 3Node will power back on when the power outage is over. You can also set it to **power on** after X seconds (from 0 to 240 with Dell servers). Imagine if you have 15 power outage in the same 30 minutes, your serves would constantly power back on, and this could damage the unit if this is done too frequently. For this reason, it is good to set the power on after 30 seconds or more.
To reset your Trusted Platform Module (TPM), use TPM Clear.
***
MISCELLANEOUS SETTINGS
***
Next, we will adjust the proper settings for the `Miscellaneous Settings`.
This can change depending on your hardware configuration.
| # | Settings | Option |
|---|---|---|
| 1 | System Time | HH:MM:SS PM |
| 2 | System Date | DD/MM/YYYY |
| 3 | ... | ... |
| 4 | F1/F2 Prompt on Error | **Enabled** / Disabled |
| 5 | ... | ... |
The settings in this section can be left at default. It is good to verify that the date and time are correct. Some farmers have reported that their 3Nodes were not booting easily when they were out of sync.
***
**iDRAC Settings**
***
The next section covers the basic to set a iDRAC that will run properly with the 3Node and help ease the farming maintenance. It is possible to set the iDRAC completely with a remote access. This will be partly covered here, but it is not the focus. Once your 3Node is booted and connected to the TF Grid, minimal maintenances are required.
***
SYSTEM SUMMARY
***
In System Summary, you can see your computer informations. It's a good thing to make sure you have the latest iDRAC Firmware. Here you can make sure IPv4 and DHCP are Enabled.
To set the iDRAC remotely, you can look at the iDRAC MAC Address.
***
SYSTEM EVENT LOG
***
In System Event Log, you can Clear Records. This can be a good thing to do when you buy a used server.
In Display System Event Log, you can check the history of your 3Node. This can be good for specific troubleshootings.
***
NETWORK
***
In Network, make sure to set Enabled at Enable NIC and Auto Dedicated NIC. You will also need to enable DHCP and IPV4 (and IPV6 if needed). For remote access, you will need to Enabled Register DRAC on DNS and Use DHCP to obtain DNS server addresses.
***
FRONT PANEL SECURITY
***
In Front Panel Security, you want to make sure Disable Power Button is set at No. Front Panel Access can be set to Full Capabilities as this can be useful to check your server's parameters.
***
VIRTUAL MEDIA
***
Virtual Media setting can be set at Auto attach. This is for remote usage of software image files (ISO-files). This is used to update servers and install operating systems.
***
vFlash Media
***
Enable vFlash can be set at Disabled in the vFlash Media section. This SD card can be used as a storage device to boot Zero-OS. It can also be used as a storage backup when replacing the motherboard.
***
LIFECYCLE CONTROLLER
***
It's good to set Enabled at Lifecycle Controller and Collect System Inventory on Restart. This checks if anything new was added to the server between usages. Also Lifecycle Controller can be used to set iDRAC remotely.
***
POWER CONFIGURATION
***
You can disable Power Cap Policy and set Power Supply Options - Redundant Policy at Not Redundant. If you want to put a power limit on the system, you can set Power Cap Policy at enabled.
You can enable Power Factor Correction to improve the server's efficiency and reduce current, but this is not necessary.
Make sure **Hot Spare is Enabled**. This feature makes sure the server will function properly if one of the two PSU (power supply unit) is damaged or removed during use.
***
SMART CARD
***
The Smart Card option can be Disabled. This can be used to connect to iDRAC with a Two Factor Authetication (2FA).
***
THERMAL
***
In the section Thermal, you can set Thermal Base Algorithm at Auto and the Cooling Options at Default.
***
COMMUNICATION PERMISSIONS
***
Communication Permissions can be disabled. When enabled, you can launch the browswer in the host OS, but this does not really apply to a 3Node.
***
USER CONFIGURATION
***
In User Configuration, you want to enable Enable User. You can set a password.
***
REMOTE ENABLEMENT
***
To use iDRAC remotely with Dell OpenManage Server Administrator (OMSA), you will need to set Enable Auto-Discovery at Enabled. Otherwise, you can set it at Disabled.
***
SYSTEM LOCATION
***
System Location can be useful if you are in a data center. You don't need to fill in any information.
***
RESET iDRAC
***
If ever needed, you can do a Factory Reset of the iDRAC configuration.
***
We have been through the BIOS settings and th iDRAC settings of a typical Dell R720 Server. With these notions in mind, you should be able to properly set your DIY 3Node. Some changes may be needed depending on your hardware and desired configuration.
## Example of UEFI Settings - Desktop Computer - Dell Optiplex 7010
***
The UEFI mode is usually needed for newer hardware.
***
There might be other ways to configure those settings, depending on the specific computer. Depending on your hardware (disks connections, RAID controller, etc.), you might need to change some settings to fit your specific computer model. Most of the time, if you follow these lines, your 3Node should work 100% with the ThreeFold Grid. Check the [Troubleshooting and Error Messages](/faq/faq.md#troubleshooting-and-error-messages) section if you have trouble on your way to booting Zero-OS.
It is possible that your 3Node needs to be booted with the BIOS mode. In this case, refer to the [previous section](#example-of-bios-settings---rack-server---dell-r720).
***
To boot Zero-OS, you need to use the bootstrap image [here](https://bootstrap.grid.tf/).
Old servers/desktops will need to use the non-UEFI mode, which are the 4th and 5th options (named: ISO and USB). Newer servers/desktops can use UEFI (1st and 2nd option).
We will now present the BIOS/UEFI settings that work with ThreeFold Zero-OS.
Note that the settings are similar for UEFI and non-UEFI. But some details are important.
The UEFI settings are from a HP G1 EliteDesk Desktop. These should give you a good overview of how to set it up.
***
**UEFI mode with HP EliteDesk.**
***
The following section will propose UEFI settings to boot a 3Node properly on the TF Grid. Note that the settings might need some modifications in your specific situations. These guidelines will help you set your 3Node properly.
Start your computer and hit Escape to get into the startup Menu. For some other computers, it can be another key. It's usually shown at the start of the boot.
Hit F10 or click on Computer Setup.
***
It can be a good idea to first check in System Information that your setup is as expected.
Here you can see your computer model and manufacturer. You can check that the processors are properly recognized and that you have the correct quantity of RAM. It's a good idea to check the BIOS and Firmware Versions. Updating the UEFI can help to boot the 3Node.
***
In Storage -> Device Configuration, you can see if all your disks are well connected (SATA, USB, etc.). Usually, to be recognized, you need to plug your removable media before booting the 3Node.
***
In Storage -> Storage Options, select AHCI as SATA Emulation and make sure Removable Media Boot is Enabled. For some specific configurations, you will want to use another paramter than AHCI.
***
In Storage -> Boot Order, you can choose the UEFI Boot Sources. If you are using a removable media, make sure it is the first in the list. Options can be: USB Hard Drive, USB Floppy/CD, CD/DVD Drive and more.
If you are booting from a USB key, you'll want to select USB Hard Drive first for the Boot Order.
***
In the section Security -> Device Security, you can see that your devices are available.
- Netowrk Controller
- SATA0
- ...
- SATA2
- ...
In the section Device Security, make sure all are set to Device Available.
***
In Security -> USB Security, make sure the port you want to use is Enabled. The easy way is to make them all Enabled.
***
In Security -> Slot Security, you can enable the PCI Express (PCIe) slots. This can be useful when you have SSD NVME disk to use via a PCIe adapter.
***
Network boot can be set at Disabled if you are booting with a removable media (USB key, CD/DVD). If you are using PXE mode, you will need to set the Network Boot at Enabled.
***
In Security -> Master Boot Record Security, you can se it at Disabled. This is used to successfully boot from a disk while accessing the data it contains. In the case of Zero-OS, the OS is on the removable media.
***
In Security -> System Security, you can enable Data Execution Prevention and OS management of Embedded Security Device. This is the place to set the Virtualization Technology (VTx). It is needed with Zero-OS in order for users to deploy virtual machines on the 3Nodes.
***
In Security -> Secure Boot Configuration, make sure you have Legacy Support set at Enabled (without this, your 3Node can't boot without a monitor). You should also Disabled the Secure Boot. Sometimes Zero-OS won't boot otherwise. There is no security problem when doing so to a 3Node running on Zero-OS.
Enabling Secure Boot can be helpful when setting a 3Node and you want to be sure no one can access the BIOS/UEFI and change the settings.
***
In Power -> OS Power Management, it can be set to Enabled Extended and Disabled.
***
In Power -> Hardware Power Management, both SATA Power Management and PCI Express Power Mangement can be set at Enabled. It can be set to Enabled Extended and Disabled. Here are the settings for Hard Power Management. S5 Maximum Power Savings can be set at Disabled. This function, when the computer is off, will turn off the power of all nonessential hardware.
***
In Advanted -> Power-On Options, it is a good idea to have After Power Loss set at Previous State. You can select a proper POST Delay (in seconds). This means if your servers are running and you get a power outage, they will turn back on when power comes back. As 3Nodes don't have "graceful shutdown", if you lose electricity momentarily, the 3Node will symply reboot and continue its work. No data can be lost when a 3Node is shutdown.
If you have different 3Nodes on the same breaker, it can be a good idea to set different POST delay timer. This will ensure that you do not have a sudden high power demand when they power on.
***
In Advanced -> BIOS Power-On, you can set all the BIOS Power-On Option at Disabled.
***
In Advanced -> Bus Options, you can set PCI SERR# Generation at Enabled, but it is simply concerning ill-behaved PCI add-in cards. PCI VGA Palette Snooping concerns graphics and can be set at Disabled.
***
In Advanced -> Device Options, you can Enabled Turbo Mode. It is very important to have **Multi-Processor and Hyperthreading at Enabled.** Sometimes, it will be written Virtual Cores, or Logical Cores.
***
In Advanced -> Management Operations, you can set the Active Management Technology (AMT). This is for remote access and is not necessary when it comes to booting and running a 3Node.
***
In Advanced -> Option ROM Launch Policy, you can all all the parameters at Legacy Only. In Legacy mode, the computer will be able to boot from all bootage devices.
***
In Advanced -> Update BIOS via Network, you can set automatic BIOS Update and enabled Update via Network. You can either leave it Enabled or Disabled. To upgrade the BIOS, you can also use a booting removable device (USB key) containing the newer firmware file.
***
In Advanced -> Ethernet Connection -> NIC Configuration, make sure your Link Status is at Connected.
***
That's it. You went through the whole Computer Setup and this should get your 3Node working good.
In the next section, we will boot the 3Node and connect to the ThreeFold Grid. How exciting!

View File

@@ -0,0 +1,197 @@
<h1> 6. Boot the 3Node </h1>
<h2> Table of Contents </h2>
- [Introduction](#introduction)
- [1. Booting the 3Node with the Zero-OS bootstrap image](#1-booting-the-3node-with-the-zero-os-bootstrap-image)
- [2. Check the 3Node Status on the TF Explorer](#2-check-the-3node-status-on-the-tf-explorer)
- [3. Receive the Farming Rewards](#3-receive-the-farming-rewards)
- [Advanced Booting Option - Network Booting (PXE)](#advanced-booting-option---network-booting-pxe)
- [PXE Booting with OPNsense](#pxe-booting-with-opnsense)
- [PXE Booting with pfSense](#pxe-booting-with-pfsense)
- [Booting Issues](#booting-issues)
- [Multiple nodes can run with the same node ID](#multiple-nodes-can-run-with-the-same-node-id)
***
## Introduction
In this section of the ThreeFold Farmers book, we explain how to boot the 3Node.
***
## 1. Booting the 3Node with the Zero-OS bootstrap image
To boot Zero-OS, insert your Zero-OS Bootstrap Image, power on your computer and choose the right booting sequence and parameters ([BIOS or UEFI](./5_set_bios_uefi.md)) in your BIOS/UEFI settings. Then, restart the 3Node. Zero-OS should boot automatically.
Note that you need an ethernet cable connected to your router or switch. You cannot farm on the ThreeFold Grid with Wifi.
The first time you boot a 3Node, it will be written: “This node is not registered (farmer ***: NameOfFarm). This is normal. The Grid will create a node ID and you will be able to see it on screen. This can take a couple of minutes.
If time passes (an hour and more) and the node does not get registered, in many cases, [wiping the disks](./4_wipe_all_disks.md) all over again and trying another reboot usually resolves this issue.
Once you have your node ID, you can also go on the ThreeFold Explorer to see your 3Node and verify that your 3Node is online.
***
## 2. Check the 3Node Status on the TF Explorer
In your browser, go on the [3Nodes Explorer section](https://dashboard.grid.tf/explorer/nodes) of the ThreeFold Dashboard on Main Net.
![dashboard_1](./img/dashboard_1.png)
Make sure to select the filter "Node ID".
![dashboard_2](./img/dashboard_2.png)
In "NODE ID", write your node ID. We use the node ID 25 as an example.
![dashboard_6](./img/dashboard_6.png)
If you node is online, you should see "up" written under the column "Status".
![dashboard_5](./img/dashboard_5.png)
To see offline nodes, unselect the option "Online (Only").
![dashboard_4](./img/dashboard_4.png)
This example is done with the ThreeFold Dashboard on Main Net. To access the ThreeFold Dashboard on different networks, select those links:
* [ThreeFold Dev Net Dashboard](https://dashboard.dev.grid.tf/)
* [ThreeFold Test Net Dashboard](https://dashboard.test.grid.tf/)
* [ThreeFold QA Net Dashboard](https://dashboard.qa.grid.tf/)
***
## 3. Receive the Farming Rewards
The farming reward will be sent once per month at the address you gave when you set up your farm. You can review this process in the section [Creating the Farm on ThreeFold Grid](./1_create_farm.md). If you created a TF farm with the TF Connect App, the address will be your TF Connect Wallet address.
For the TF V3, rewards are locked for a 24 months period of after 30% of farm utilization. As of now, this is not implemented. This means that you can withdraw your farming rewards anytime you want. For more details on this, please refer to the section [ThreeFold Token (TFT)](../../threefold_token/threefold_token.md). Details of this locking system implementation will most likely be discussed by the upcoming DAO. Stay tuned.
***
That's it. You've now completed the necessary steps to build a DIY 3Node and to connect it to the Grid. In the next section, we will give you more details on Farming Optimization and Planification. This will maximize your farming efficiency as well as help you plan the economics of farming.
***
## Advanced Booting Option - Network Booting (PXE)
### PXE Booting with OPNsense
***
> This documentation comes from the [amazing Network Booting Guide](https://forum.ThreeFold.io/t/network-booting-tutorial/2688) by @Fnelson on the ThreeFold Forum.
Network booting ditches your standard boot USB with a local server. This TFTP server delivers your boot files to your 3 nodes. This can be useful in bigger home farms, but is all but mandatory in a data center setup.
Network boot setup is quite easy and is centered about configuring a TFTP server. There are essentially 2 options for this, a small dedicated server such as a raspberry pi, or piggybacking on your pfsense or opnsense router. I would recommend the latter as it eliminates another piece of equipment and is probably more reliable.
**Setting Up Your Router to Allow Network Booting**
These steps are for OPNsense, PFsense may differ. These set are required regardless of where you have your TFTP server.
> Services>DHCPv4>LAN>Network Booting
Check “Enable Network Booting”
Enter the IP address of your TFTP server under “Set next-server IP”. This may be the routers IP or whatever device you are booting from.
Enter “pxelinux.0” under Set default bios filename.
Ignore the TFTP Server settings.
**TFTP server setup on a debian machine such as Ubuntu or Raspberry Pi**
> apt-get update
>
> apt-get install tftpd-hpa
>
> cd /srv/tftp/
>
> wget http://ftp.nl.debian.org/debian/dists/buster/main/installer-amd64/current/images/netboot/netboot.tar.gz
>
> wget http://ftp.nl.debian.org/debian/dists/buster/main/installer-amd64/current/images/netboot/pxelinux.0
>
> wget https://bootstrap.grid.tf/krn/prod/<FARMID> --no-check-certificate
>
> mv <FARMID> ipxe-prod.lkrn
>
> tar -xvzf netboot.tar.gz
>
> rm version.info netboot.tar.gz
>
> rm pxelinux.cfg/default
>
> chmod 777 /srv/tftp/pxelinux.cfg (optional if next step fails)
>
> echo 'default ipxe-prod.lkrn' >> pxelinux.cfg/default
**TFTP Server on a OPNsense router**
> Note: When using PFsense instead of OPNsense, steps are probably similar, but the directory or other small things may differ.
The first step is to download the TFTP server plugin. Go to system>firmware>Status and check for updates, follow prompts to install. Then click the Plugins tab and search for tftp, install os-tftp. Once that is installed go to Services>TFTP (you may need to refresh page). Check the Enable box and input your router ip (normally 192.168.1.1). Click save.
Turn on ssh for your router. In OPNsense it is System>Settings>Administration. Then check the Enable, root login, and password login. Hop over to Putty and connect to your router, normally 192.168.1.1. Login as root and input your password. Hit 8 to enter the shell.
In OPNsense the tftp directory is /usr/local/tftp
> cd /usr/local
>
> mkdir tftp
>
> cd ./tftp
>
> fetch http://ftp.nl.debian.org/debian/dists/buster/main/installer-amd64/current/images/netboot/netboot.tar.gz
>
> fetch http://ftp.nl.debian.org/debian/dists/buster/main/installer-amd64/current/images/netboot/pxelinux.0
>
> fetch https://bootstrap.grid.tf/krn/prod/<FARMID>
>
> mv <FARMID> ipxe-prod.lkrn
>
> tar -xvzf netboot.tar.gz
>
> rm version.info netboot.tar.gz
>
> rm pxelinux.cfg/default
>
> echo 'default ipxe-prod.lkrn' >> pxelinux.cfg/default
You can get out of shell by entering exit or just closing the window.
**3Node Setup**
Set the server to BIOS boot and put PXE or network boot as the first choice. At least on Dell machines, make sure you have the network cable in plug 1 or it wont boot.
***
### PXE Booting with pfSense
> This documentation comes from the [amazing Network Booting Guide](https://forum.threefold.io/t/network-booting-tutorial/2688/7) by @TheCaptain on the ThreeFold Forum.
These are the steps required to enable PXE booting on pfSense. This guide assumes youll be using the router as your PXE server; pfSense allows boot file uploads directly from its web GUI.
* Log into your pfSense instance
* Go to System>Package Manager
* Search and add tftpd package under Available Packages tab
* Go to Services>TFTP Server
* Under Settings tab check enable and enter the router IP in TFTP Server Bind IP field
* Switch to Files tab under Services>TFTP Server and upload your ipxe-prod.efi file acquired from https://v3.bootstrap.grid.tf/ (second option labeled EFI Kernel)
* Go to Services>DHCP Server
* Under Other Options section click Display Advance next to TFTP and enter router IP
* Click Display Advance next to Network Booting
* Check enable, enter router IP in Next Server field
* Enter ipxe-prod.efi in Default BIOS file name field
That's it! Youll want to ensure your clients are configured with boot priority set as IPv4 in first spot. You might need to disable secure boot and enable legacy boot within BIOS.
***
## Booting Issues
### Multiple nodes can run with the same node ID
This is a [known issue](https://github.com/threefoldtech/info_grid/issues/122) and will be resolved once the TPM effort gets finalized.
***

View File

@@ -0,0 +1,82 @@
<h1>GPU Farming</h1>
Welcome to the *GPU Farming* section of the ThreeFold Manual!
In this guide, we delve into the realm of GPU farming, shedding light on the significance of Graphics Processing Units (GPUs) and how they can be seamlessly integrated into the ThreeFold ecosystem.
<h2>Table of Contents</h2>
- [Understanding GPUs](#understanding-gpus)
- [Get Started](#get-started)
- [Install the GPU](#install-the-gpu)
- [GPU Node and the Farmerbot](#gpu-node-and-the-farmerbot)
- [Set a Price for the GPU Node](#set-a-price-for-the-gpu-node)
- [Check the GPU Node on the Explorer](#check-the-gpu-node-on-the-explorer)
- [Reserving the GPU Node](#reserving-the-gpu-node)
- [Questions and Feedback](#questions-and-feedback)
***
## Understanding GPUs
A Graphics Processing Unit, or GPU, is a specialized electronic circuit designed to accelerate the rendering of images and videos. Originally developed for graphics-intensive tasks in gaming and multimedia applications, GPUs have evolved into powerful parallel processors with the ability to handle complex computations, such as 3D rendering, AI and machine learning.
In the context of ThreeFold, GPU farming involves harnessing the computational power of Graphics Processing Units to contribute to the decentralized grid. This empowers users to participate in the network's mission of creating a more equitable and efficient internet infrastructure.
## Get Started
In this guide, we focus on the integration of GPUs with a 3Node, the fundamental building block of the ThreeFold Grid. The process involves adding a GPU to enhance the capabilities of your node, providing increased processing power and versatility for a wide range of tasks. Note that any Nvidia or AMD graphics card should work as long as it's supported by the system.
## Install the GPU
We cover the basic steps to install the GPU on your 3Node.
* Find a proper GPU model for your specific 3Node hardware
* Install the GPU on the server
* Note: You might need to move or remove some pieces of your server to make room for the GPU
* Here is a quick example with a Dell R720 rack server and an AMD Radeon GPU
<div class="youtubeVideoWrapper">
<iframe width="560" height="315" src="https://www.youtube.com/embed/o10Ef963vsY?si=-64Tu-xx_VIX9JAM" title="ThreeFold GPU Farming" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe>
</div>
* (Optional) Boot the 3Node with a Linux distro (e.g. Ubuntu) and use the terminal to check if the GPU is recognized by the system
* ```
sudo lshw -C Display
```
* Output example with an AMD Radeon (on the line `product: ...`)
![gpu_farming](./img/cli_display_gpu.png)
* Boot the 3Node with the ZOS bootstrap image
## GPU Node and the Farmerbot
If you are using the Farmerbot, it might be a good idea to first boot the GPU node without the Farmerbot (i.e. to remove the node in the config file and restart the Farmerbot). Once you've confirmed that the GPU is properly detected by TFChain, you can then put back the GPU node in the config file and restart the Farmerbot. While this is not necessary, it can be an effective way to test the GPU node separately.
## Set a Price for the GPU Node
You can [set additional fees](../farming_optimization/set_additional_fees.md) for your GPU dedicated node on the [TF Dashboard](https://dashboard.grid.tf/).
* On the Dashboard, go to **Portal -> Farms**
* Under the section **Your Farm Nodes**, locate the GPU node and click **Set Additional Fees** under **Actions**
* Set a monthly fee (in USD) and click **Set**
When a user reserves your 3Node as a dedicated node, you will receive TFT payments once every 24 hours. These TFT payments will be sent to the TFChain account of your farm's twin.
## Check the GPU Node on the Explorer
You can use the Explorer on the [TF Dashboard](https://dashboard.grid.tf/) to verify that the node is displayed as having a GPU.
* On the Dashboard, go to **Explorer -> Nodes**
* Under **Node ID**, write the node ID of the GPU node
* Once the results are displayed, you should see **1** under **GPU**
* If you are using the Status bot, you might need to change the node status under **Select Nodes Status** (e.g. **Down**, **Standby**) to see the node's information
> Note: It can take some time for the GPU parameter to be displayed.
## Reserving the GPU Node
Now, users can reserve the node in the **Dedicated Nodes** section of the Dashboard and then deploy workloads using the GPU. For more information on how to deploy workloads with a GPU node, read [this documentation](../../dashboard/portal/dashboard_portal_dedicated_nodes.md#gpu-support).
## Questions and Feedback
If you have any questions or feedback, we invite you to discuss with the ThreeFold community on the [ThreeFold Forum](https://forum.threefold.io/) or on the [ThreeFold Farmer chat](https://t.me/threefoldfarmers) on Telegram.

Binary file not shown.

After

Width:  |  Height:  |  Size: 10 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 138 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 40 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 5.3 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 25 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 10 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 14 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 3.6 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.4 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 238 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 33 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 95 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 81 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 124 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 119 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 22 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 10 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 45 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 35 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 27 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 13 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 35 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 79 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 21 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 57 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 25 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 30 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 46 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 57 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 120 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 118 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 36 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 30 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 54 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 23 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 34 KiB

View File

@@ -0,0 +1,67 @@
<h1>Minting Receipts</h1>
<h2>Table of Contents</h2>
- [Introduction](#introduction)
- [Download Minting Receipts of Your Farm](#download-minting-receipts-of-your-farm)
- [Download Minting Receipts of a 3Node](#download-minting-receipts-of-a-3node)
- [Minting Receipts Information](#minting-receipts-information)
- [Alpha Minting Tool](#alpha-minting-tool)
***
## Introduction
You can easily consult minting receipts of all your 3Nodes on the ThreeFold Dashboard to get essential minting information of your 3Nodes and your ThreeFold farm. With your minting receipt hash, you can then query the ThreeFold Alpha minting tool for further minting information.
## Download Minting Receipts of Your Farm
You can download minting receipts of your whole farm directly on the ThreeFold Dashboard.
- On the [ThreeFold Dashboard](https://dashboard.grid.tf/), go to **Portal** -> **Farms**
- In the section **Your Farms**, on the left of your **Farm ID**, click on the down arrow button
- Click on **Download Minting Receipts**
## Download Minting Receipts of a 3Node
You can download minting receipts of a single 3Node directly on the ThreeFold Dashboard.
- On the [ThreeFold Dashboard](https://dashboard.grid.tf/), go to **Portal** -> **Farms**
- In the section **Your Farm Nodes**, on the left of your **Node ID**, click on the down arrow button
- Click on **Node Statistics**
- Click on **Download Node Receipt**
## Minting Receipts Information
The minting receipts contain the following information:
- Minting: <minting_receipt_hash>
- start: <start of minting period>
- end: <end of minting period>
- TFT: <TFT minted by the 3Node>
- Cloud Units: <3Node Resources>
## Alpha Minting Tool
You can query additional minting information by using the [Dashboard Alpha Minting tool](https://dashboard.grid.tf/other/minting).
- Download the minting receipts of your farm or of a single 3Node
- Copy a minting receipt hash
- Open the ThreeFold Alpha Minting tool by clicking on **Minting** on the left-side [ThreeFold Dashboard](https://dashboard.grid.tf/) menu
- Paste the minting receipt hash
The ThreeFold Alpha minting tool will present the following information for each minting receipt hash:
- Node ID
- Farm Name
- Measured Uptime
- Node Resources
- CU
- SU
- NU
- CRU
- MRU
- SRU
- HRU
- TFT Farmed
- Payout Address

View File

@@ -0,0 +1,13 @@
<h1> Advanced Networking </h1>
Welcome to the *Advanced Networking* section of the ThreeFold Manual.
In this section, we provide advanced networking tips for farms with public IPs and in data centers (DC). We also cover the differences between IPv4 and IPv6 networking.
<h2>Table of Contents</h2>
- [Networking Overview](./networking_overview.md)
- [Network Considerations](./network_considerations.md)
- [Network Setup](./network_setup.md)
> Note: This documentation does not constitute a complete set of knowledge on setting farms with public IP addresses in a data center. Please make sure to do your own research and communicate with your data center and your Internet service provider for any additional information.

View File

@@ -0,0 +1,120 @@
<h1>Network Considerations</h1>
<h2>Table of Contents</h2>
- [Introduction](#introduction)
- [Running ZOS (v2) at Home](#running-zos-v2-at-home)
- [Running ZOS (v2) in a Multi-Node Farm in a DC](#running-zos-v2-in-a-multi-node-farm-in-a-dc)
- [Necessities](#necessities)
- [IPv6](#ipv6)
- [Routing/Firewalling](#routingfirewalling)
- [Multi-NIC Nodes](#multi-nic-nodes)
- [Farmers and the TFGrid](#farmers-and-the-tfgrid)
***
## Introduction
Running ZOS on a node is just a matter of booting it with a USB stick, or with a dhcp/bootp/tftp server with the right configuration so that the node can start the OS.
Once it starts booting, the OS detects the NICs, and starts the network configuration. A Node can only continue it's boot process till the end when it effectively has received an IP address and a route to the Internet. Without that, the Node will retry indefinitely to obtain Internet access and not finish it's startup.
So a Node needs to be connected to a __wired__ network, providing a dhcp server and a default gateway to the Internet, be it NATed or plainly on the public network, where any route to the Internet, be it IPv4 or IPv6 or both is sufficient.
For a node to have that ability to host user networks, we **strongly** advise to have a working IPv6 setup, as that is the primary IP stack we're using for the User Network's Mesh to function.
## Running ZOS (v2) at Home
Running a ZOS Node at home is plain simple. Connect it to your router, plug it in the network, insert the preconfigured USB stick containing the bootloader and the `farmer_id`, power it on.
## Running ZOS (v2) in a Multi-Node Farm in a DC
Multi-Node Farms, where a farmer wants to host the nodes in a data centre, have basically the same simplicity, but the nodes can boot from a boot server that provides for DHCP, and also delivers the iPXE image to load, without the need for a USB stick in every Node.
A boot server is not really necessary, but it helps! That server has a list of the MAC addresses of the nodes, and delivers the bootloader over PXE. The farmer is responsible to set-up the network, and configure the boot server.
### Necessities
The Farmer needs to:
- Obtain an IPv6 prefix allocation from the provider. A `/64` will do, that is publicly reachable, but a `/48` is advisable if the farmer wants to provide IPv6 transit for User Networks
- If IPv6 is not an option, obtain an IPv4 subnet from the provider. At least one IPv4 address per node is needed, where all IP addresses are publicly reachable.
- Have the Nodes connected on that public network with a switch so that all Nodes are publicly reachable.
- In case of multiple NICS, also make sure his farm is properly registered in BCDB, so that the Node's public IP Addresses are registered.
- Properly list the MAC addresses of the Nodes, and configure the DHCP server to provide for an IP address, and in case of multiple NICs also provide for private IP addresses over DHCP per Node.
- Make sure that after first boot, the Nodes are reachable.
### IPv6
IPv6, although already a real protocol since '98, has seen reluctant adoption over the time it exists. That mostly because ISPs and Carriers were reluctant to deploy it, and not seeing the need since the advent of NAT and private IP space, giving the false impression of security.
But this month (10/2019), RIPE sent a mail to all it's LIRs that the last consecutive /22 in IPv4 has been allocated. Needless to say, but that makes the transition to IPv6 in 2019 of utmost importance and necessity.
Hence, ZOS starts with IPv6, and IPv4 is merely an afterthought ;-)
So in a nutshell: we greatly encourage Farmers to have IPv6 on the Node's network.
### Routing/Firewalling
Basically, the Nodes are self-protecting, in the sense that they provide no means at all to be accessed through listening processes at all. No service is active on the node itself, and User Networks function solely on an overlay.
That also means that there is no need for a Farm admin to protect the Nodes from exterior access, albeit some DDoS protection might be a good idea.
In the first phase we will still allow the Host OS (ZOS) to reply on ICMP ping requests, but that 'feature' might as well be blocked in the future, as once a Node is able to register itself, there is no real need to ever want to try to reach it.
### Multi-NIC Nodes
Nodes that Farmers deploy are typically multi-NIC Nodes, where one (typically a 1GBit NIC) can be used for getting a proper DHCP server running from where the Nodes can boot, and one other NIC (1Gbit or even 10GBit), that then is used for transfers of User Data, so that there is a clean separation, and possible injections bogus data is not possible.
That means that there would be two networks, either by different physical switches, or by port-based VLANs in the switch (if there is only one).
- Management NICs
The Management NIC will be used by ZOS to boot, and register itself to the GRID. Also, all communications from the Node to the Grid happens from there.
- Public NICs
### Farmers and the TFGrid
A Node, being part of the Grid, has no concept of 'Farmer'. The only relationship for a Node with a Farmer is the fact that that is registered 'somewhere (TM)', and that a such workloads on a Node will be remunerated with Tokens. For the rest, a Node is a wholly stand-alone thing that participates in the Grid.
```text
172.16.1.0/24
2a02:1807:1100:10::/64
+--------------------------------------+
| +--------------+ | +-----------------------+
| |Node ZOS | +-------+ | |
| | +-------------+1GBit +--------------------+ 1GBit switch |
| | | br-zos +-------+ | |
| | | | | |
| | | | | |
| | | | +------------------+----+
| +--------------+ | | +-----------+
| | OOB Network | | |
| | +----------+ ROUTER |
| | | |
| | | |
| | | |
| +------------+ | +----------+ |
| | Public | | | | |
| | container | | | +-----+-----+
| | | | | |
| | | | | |
| +---+--------+ | +-------------------+--------+ |
| | | | 10GBit Switch | |
| br-pub| +-------+ | | |
| +-----+10GBit +-------------------+ | +---------->
| +-------+ | | Internet
| | | |
| | +----------------------------+
+--------------------------------------+
185.69.167.128/26 Public network
2a02:1807:1100:0::/64
```
Where the underlay part of the wireguard interfaces get instantiated in the Public container (namespace), and once created these wireguard interfaces get sent into the User Network (Network Resource), where a user can then configure the interface a he sees fit.
The router of the farmer fulfills 2 roles:
- NAT everything in the OOB network to the outside, so that nodes can start and register themselves, as well get tasks to execute from the BCDB.
- Route the assigned IPv4 subnet and IPv6 public prefix on the public segment, to which the public container is connected.
As such, in case that the farmer wants to provide IPv4 public access for grid proxies, the node will need at least one (1) IPv4 address. It's free to the farmer to assign IPv4 addresses to only a part of the Nodes.
On the other hand, it is quite important to have a proper IPv6 setup, because things will work out better.
It's the Farmer's task to set up the Router and the switches.
In a simpler setup (small number of nodes for instance), the farmer could setup a single switch and make 2 port-based VLANs to separate OOB and Public, or even wit single-nic nodes, just put them directly on the public segment, but then he will have to provide a DHCP server on the Public network.

View File

@@ -0,0 +1,86 @@
<h1>Network Setup</h1>
<h2>Table of Contents</h2>
- [Introduction](#introduction)
- [Network Setup for Farmers](#network-setup-for-farmers)
- [Step 1. Testing for IPv6 Availability in Your Location](#step-1-testing-for-ipv6-availability-in-your-location)
- [Step 2. Choosing the Setup to Connect Your Nodes](#step-2-choosing-the-setup-to-connect-your-nodes)
- [2.1 Home Setup](#21-home-setup)
- [2.2 Data Center/Advanced Setup](#22-data-centeradvanced-setup)
- [General Notes](#general-notes)
***
# Introduction
0-OS nodes participating in the Threefold grid, need connectivity of course. They need to be able to communicate over
the Internet with each-other in order to do various things:
- download its OS modules
- perform OS module upgrades
- register itself to the grid, and send regular updates about it's status
- query the grid for tasks to execute
- build and run the Overlay Network
- download flists and the effective files to cache
The nodes themselves can have connectivity in a few different ways:
- Only have RFC1918 private addresses, connected to the Internet through NAT, NO IPv6
Mostly, these are single-NIC (Network card) machines that can host some workloads through the Overlay Network, but
cant't expose services directly. These are HIDDEN nodes, and are mostly booted with an USB stick from
bootstrap.grid.tf .
- Dual-stacked: having RFC1918 private IPv4 and public IPv6 , where the IPv6 addresses are received from a home router,
but firewalled for outgoing traffic only. These nodes are effectively also HIDDEN
- Nodes with 2 NICs, one that has effectively a NIC connected to a segment that has real public
addresses (IPv4 and/or IPv6) and one NIC that is used for booting and local
management. (OOB) (like in the drawing for farmer setup)
For Farmers, we need to have Nodes to be reachable over IPv6, so that the nodes can:
- expose services to be proxied into containers/vms
- act as aggregating nodes for Overlay Networks for HIDDEN Nodes
Some Nodes in Farms should also have a publicly reachable IPv4, to make sure that clients that only have IPv4 can
effectively reach exposed services.
But we need to stress the importance of IPv6 availability when you're running a multi-node farm in a datacentre: as the
grid is boldly claiming to be a new Internet, we should make sure we adhere to the new protocols that are future-proof.
Hence: IPv6 is the base, and IPv4 is just there to accomodate the transition.
Nowadays, RIPE can't even hand out consecutive /22 IPv4 blocks any more for new LIRs, so you'll be bound to market to
get IPv4, mostly at rates of 10-15 Euro per IP. Things tend to get costly that way.
So anyway, IPv6 is not an afterthought in 0-OS, we're starting with it.
# Network Setup for Farmers
This is a quick manual to what is needed for connecting a node with zero-OS V2.0
## Step 1. Testing for IPv6 Availability in Your Location
As descibed above the network in which the node is instaleld has to be IPv6 enabled. This is not an afterthought as we are building a new internet it has to ba based on the new and forward looking IP addressing scheme. This is something you have to investigate, negotiate with you connectivity provider. Many (but not all home connectivity products and certainly most datacenters can provide you with IPv6. There are many sources of infromation on how to test and check whether your connection is IPv6 enabled, [here is a starting point](http://www.ipv6enabled.org/ipv6_enabled/ipv6_enable.php)
## Step 2. Choosing the Setup to Connect Your Nodes
Once you have established that you have IPv6 enabled on the network you are about to deploy, you have to make sure that there is an IPv6 DHCP facility available. Zero-OS does not work with static IPv6 addresses (at this point in time). So you have choose and create one of the following setups:
### 2.1 Home Setup
Use your (home) ISP router Ipv6 DHCP capabilities to provide (private) IPv6 addresses. The principle will work the same as for IPv4 home connections, everything happens enabled by Network Adress Translation (just like anything else that uses internet connectivity). This should be relatively straightforward if you have established that your conenction has IPv6 enabled.
### 2.2 Data Center/Advanced Setup
In this situation there are many options on how to setup you node. This requires you as the expert to make a few decisions on how to connect what what the best setup is that you can support for the operaitonal time of your farm. The same basics principles apply:
- You have to have a block of (public) IPv6 routed to you router, or you have to have your router setup to provide Network Address Translation (NAT)
- You have to have a DHCP server in your network that manages and controls IPV6 ip adress leases. Depending on your specific setup you have this DHCP server manage a public IPv6y range which makes all nodes directly connected to the public internet or you have this DHCP server manage a private block og IPv6 addresses which makes all you nodes connect to the internet through NAT.
As a farmer you are in charge of selecting and creating the appropriate network setup for your farm.
# General Notes
The above setup will allows your node(s) to appear in explorer on the TFGrid and will allowd you to earn farming tokens. At stated in the introduction ThreeFold is creating next generation internet capacity and therefore has IPv6 as it's base building block. Connecting to the current (dominant) IPv4 network happens for IT workloads through so called webgateways. As the word sais these are gateways that provide connectivity between the currenct leading IPv4 adressing scheme and IPv6.
We have started a forum where people share their experiences and configurations. This will be work in progress and forever growing.
**IMPORTANT**: You as a farmer do not need access to IPV4 to be able to rent capacity for IT workloads that need to be visible on IPV4, this is something that can happen elsewhere on the TFGrid.

View File

@@ -0,0 +1,94 @@
<h1> Networking Overview </h1>
<h2> Table of Contents </h2>
- [Introduction](#introduction)
- [Possible Configurations](#possible-configurations)
- [Overall Requirements](#overall-requirements)
- [Notes and Warnings](#notes-and-warnings)
- [Management Interfaces](#management-interfaces)
- [Data Center Cable Management](#data-center-cable-management)
- [Static IP Uplink](#static-ip-uplink)
- [Testing the Setup](#testing-the-setup)
- [Questions and Feedback](#questions-and-feedback)
***
## Introduction
In this section, we provide advanced networking tips for farms with public IPs and in data centers (DC). The information available in this section is a combination of documentation from ThreeFold and tips and advice from community members who experienced first-hand the creation of ThreeFold farms that make use of public IPs block in data centers, personal data centers and home farms. A special thank you to those who contributed to improving the TFGrid and its knowledge base documentation.
## Possible Configurations
For farmers who have public IPs, extra considerations are needed in setting up the network of the farm. We will go through the main considerations in this section.
First, we must acknowledge that by the open-source and design of ThreeFold farming, a farm can range from a simple [single 3Node](../3node_building/3node_building.md) setup, to a multi-rack farm hosted in a typical data center, and everything in-between, from the farmer experiencing with public IP blocks, to the entrepreneur who builds their own data center at home.
There are thus many types of farms and each will have varying configurations. The simplest way to set up a farm has been extensively discussed in the first steps of creating a farm. But what are the other more complex configurations possible? Let's go through some of those:
- Network link
- DC provides a network link into the farmer's rack
- Router and switch
- The farmer provider their own router and switch
- DC provides a router and/or switch in the rack
- Gateway IP and public IP
- Gateway IP provided is in the same range as the public IPs
- Gateway IP is in a different range than the public IPs
- Segmenting
- Farmer segments the OOB ("Zos"/private) interfaces and the public interfaces into
- separate VLANs, OR;
- uses separate switches altogether
- No segmenting is actually necessary, farmer connects all interfaces to one switch
## Overall Requirements
There are overall requirements for any 3Node farm using IP address blocks in a data centere or at home:
- There must be at least one interface that provide DHCP to each node
- Public IPs must be routable from at least one interface
Note that redundancy can help in avoiding single point of failure [(SPOF)](https://en.wikipedia.org/wiki/Single_point_of_failure).
## Notes and Warnings
### Management Interfaces
You should make sure to never expose management interfaces to the public internet.
### Data Center Cable Management
It's important to have a good cable management, especially if you are in a data center. Proper cable management will improve the cooling streams of your farm. There shouldn't be any cable in front of the fans. This way, your servers will last longer. If you want to patch a rack, you have to have all lenght of patch cables from 30cm to 3m. Also, try to keep the cables as short as possible. Arrange the cables in bundles of eight and lead them to the sides of the rack as much as possible for optimal airflow.
<!--
## Configuring Node Interfaces
Add info on this:
There's a program floating around that allows farmers to configure the interfaces on their nodes. This is done over RMB, and I also developed a script for this purpose. Both might be deprecated with the 3.9 release and new RMB. There should be a documented way of doing this that's relatively accessible.
-->
<!--
### NIC
QUESTION: Does it matter which NICs are used? -->
### Static IP Uplink
If your DC uplink is established by simple static IP (which is the case in most DCs), there is a simple setup possible. Note that if you have to use PPPoE or pptp/L2TP (like a consumer internet connection at most homes), this would not work.
If your WAN is established by static IP, you can simply attach the WAN uplink provided by the DC to one of the switches (and not to the WAN-side of your own router). Then, the WAN-side of the router needs to be attached to the switch too. By doing so, your nodes will be able to connect directly to the DC gateway, in the same way that the router is connecting its WAN-side to the gateway, without the public IP traffic being routed/bridged through the router (bypassing).
With a network configured like this, it is absolutely not important on which ports you connect which NIC of your nodes. You can just randomly plug them anywhere. But there is one restriction: the DC uplink must use a static IP. Dynamic IP would also not work because you would then have two DHCP servers in the same physical network (the one from the DC and your own router).
## Testing the Setup
Manual and automatic validation of the network of a farm are possible. More information on automatic validation will be added in the future.
You can test the network of your farm manually by deploying a workload on your 3Nodes with either a gateway or a public IP reserved.
## Questions and Feedback
If you have any questions, you can ask the ThreeFold community for help on the [ThreeFold Forum](http://forum.threefold.io/) or on the [ThreeFold Farmer Chat](https://t.me/threefoldfarmers) on Telegram.

View File

@@ -0,0 +1,566 @@
![Farming_Title](./img/farming_30.png)
<h1>Building a DIY 3Node: Desktop Computer</h1>
In the following 3Node DIY guide, you will learn how to turn a Dell Optiplex 7020 into a 3Node farming on the ThreeFold Grid.
Note that the process is similar for other desktop computers.
<div style="break-after:page"></div>
<h2>Table of Contents</h2>
- [Prerequisite](#prerequisite)
- [DIY 3Node Computer Requirements](#diy-3node-computer-requirements)
- [DIY 3Node Material List](#diy-3node-material-list)
- [1. Create a Farm with ThreeFold Connect App](#1-create-a-farm-with-threefold-connect-app)
- [ThreeFold Connect App Installation](#threefold-connect-app-installation)
- [Verify Your Identity (KYC) by Email](#verify-your-identity-kyc-by-email)
- [Create a Wallet](#create-a-wallet)
- [Create a Farm](#create-a-farm)
- [2. Create a Zero-OS Bootstrap Image](#2-create-a-zero-os-bootstrap-image)
- [Download the Zero-OS Boostrap Image](#download-the-zero-os-boostrap-image)
- [Burn the Zero-OS Bootstrap Image](#burn-the-zero-os-bootstrap-image)
- [3. Set the Hardware](#3-set-the-hardware)
- [4. Wipe All the Disks](#4-wipe-all-the-disks)
- [1. Create a Linux Boostrap Image](#1-create-a-linux-boostrap-image)
- [2. Boot Linux in Try Mode](#2-boot-linux-in-try-mode)
- [3. Use wipefs to Wipe All Disks](#3-use-wipefs-to-wipe-all-disks)
- [5. Set the BIOS/UEFI](#5-set-the-biosuefi)
- [The Essential Features of BIOS/UEFI for a 3Node](#the-essential-features-of-biosuefi-for-a-3node)
- [Set the BIOS/UEFI on a Dell Optiplex 7020](#set-the-biosuefi-on-a-dell-optiplex-7020)
- [6. Boot the 3Node](#6-boot-the-3node)
- [Check the 3Node's Status on the TF Explorer](#check-the-3nodes-status-on-the-tf-explorer)
- [Farming Rewards Distribution](#farming-rewards-distribution)
- [Additional Information](#additional-information)
***
<div style="break-after:page"></div>
# Prerequisite
## DIY 3Node Computer Requirements
Any computer with the following specifications can be used as a DIY 3Node.
- Any 64-bit hardware with an Intel or AMD processor chip.
- Servers, desktops and mini computers type hardware are compatible.
- A minimum of 500 GB of SSD and a bare minimum of 2 GB of RAM is required.
- A ratio of 100GB of SSD and 8GB of RAM per thread is recommended.
- A wired ethernet connection is highly recommended to maximize reliability and the ability to farm TFT.
- A [passmark](https://www.passmark.com/) of 1000 per core is recommended and will be a minimum requirement in the future.
In this guide, we are using a Dell Optiplex 7020. It constitutes a perfect affordable entry DIY 3Node as it can be bought refurbished with the proper ratio of 100GB of SSD and 8GB of RAM per thread, and this, without any need of upgrades or updates.
## DIY 3Node Material List
* Any computer respecting the DIY 3Node Computer Requirements stated above
* Ethernet cable
* Router + Modem
* Surge Protector
* 2x USB key 4 Go
* Android/iOS Phone
* Computer monitor and cable, keyboard and mouse
* MAC/Linux/Windows Computer
<div style="break-after:page"></div>
# 1. Create a Farm with ThreeFold Connect App
## ThreeFold Connect App Installation
The ThreeFold Connect App is available for [Android](https://play.google.com/store/apps/details?id=org.jimber.threebotlogin&hl=en&gl=US) and [iOS](https://apps.apple.com/us/app/threefold-connect/id1459845885).
- Note that for Android phones, you need at minimum Android Nougat, the 8.0 software version.
- Note that for iOS phones, you need at minimum iOS 14.5. It will be soon available to iOS 13.
Either use the links above, or search for the ThreeFold Connect App on the Apple Store or the Google Play store. Then install and open the app. If you want to leave a 5 star review of the App, no one here will stop you!
![farming_tf_wallet_1](./img/farming_tf_wallet_1.png)
![farming_tf_wallet_2](./img/farming_tf_wallet_2.png)
Once you are in the application, you will see some introduction pages to help you familiarize with the TF Connect App. You will also be asked to read and accept ThreeFold's Terms and conditions.
![farming_tf_wallet_4](./img/farming_tf_wallet_4.png)
You will then be asked to either *SIGN UP* or *RECOVER ACCOUNT*. For now, we will show how to sign up. Later in the guide, we will show you how to recover an account.
![farming_tf_wallet_5](./img/farming_tf_wallet_5.png)
You will then be asked to choose a *ThreeFold Connect Id*. This ID will be used, as well as the seed phrase, when you want to recover an account. Choose wisely. And do not forget it! Here we will use TFExample, as an example.
![farming_tf_wallet_6](./img/farming_tf_wallet_6.png)
Next, you need to add a valid email address. This will be used as a broad KYC. You will need to access your email and confirm the validation email from ThreeFold to use properly the TF Connect App Wallet.
![farming_tf_wallet_7](./img/farming_tf_wallet_7.png)
Then, the next step is crucial! Make sure no one is around looking at your screen. You will be shown your seed phrase. Keep this in a secure and offline place. You will need the 3bot ID and the seed phrase to recover your account. This seed phrase is of utmost important. Do not lose it nor give it to anyone.
![farming_tf_wallet_8](./img/farming_tf_wallet_8.png)
Once you've hit Next, you will be asked to write down 3 random words of your seed phrase. This is a necessary step to ensure you have taken the time to write down your seed phrase.
![farming_tf_wallet_9](./img/farming_tf_wallet_9.png)
Then, you'll be asked to confirm your TF 3bot name and the associated email.
![farming_tf_wallet_10](./img/farming_tf_wallet_10.png)
Finally, you will be asked to choose a 4-digit pin. This will be needed to use the ThreeFold Connect App. If you ever forget this 4-digit pin, you will need to recover your account from your 3bot name and your seed phrase. You will need to confirm the new pin in the next step.
![farming_tf_wallet_11](./img/farming_tf_wallet_11.png)
That's it! You've created your ThreeFold Connect account. You can press the hamburger menu on the top left to explore the ThreeFold Connect App.
![farming_tf_wallet_12](./img/farming_tf_wallet_12.png)
<div style="break-after:page"></div>
### Verify Your Identity (KYC) by Email
Once you've created your account, an email will be sent to the email address you've chosen in the account creation process.
To verify your email, go on your email account and open the email sent by *info@openkyc.live* with the subject *Verify your email address*.
In this email, click on the link *Verify my email address*. This will lead you to a *login.threefold.me* link. The process should be automatic. Once this is done, you will receive a confirmation on screen, as well as on your phone.
![farming_tf_wallet_39](./img/farming_tf_wallet_39.png)
![farming_tf_wallet_40](./img/farming_tf_wallet_40.png)
![farming_tf_wallet_41](./img/farming_tf_wallet_41.png)
If for some reason, you did not receive the verification email, simply click on *Verify* and another email will be sent.
![farming_tf_wallet_42](./img/farming_tf_wallet_42.png)
![farming_tf_wallet_43](./img/farming_tf_wallet_43.png)
<div style="break-after:page"></div>
## Create a Wallet
To create a wallet, click on the ThreeFold Connect App menu. This is what you see. Choose *Wallet*.
![farming_tf_wallet_13](./img/farming_tf_wallet_13.png)
Once you are in the section *Wallet*, click on *Create Initial Wallet*.
![farming_tf_wallet_14](./img/farming_tf_wallet_14.png)
This is what you see when the TF Grid is initializing your wallet.
![farming_tf_wallet_15](./img/farming_tf_wallet_15.png)
Once your wallet is initialized, you will see *No blanace found for this wallet*. You can click on this button to enter the wallet.
![farming_tf_wallet_16](./img/farming_tf_wallet_16.png)
Once inside your wallet, this is what you see.
![farming_tf_wallet_17](./img/farming_tf_wallet_17.png)
<div style="break-after:page"></div>
## Create a Farm
To create a farm, follow those steps.
Click on the hamburger menu on the top left of your screen and select *Farmer Migration*. This title is somehow misleading, as you can not only migrate farms, but create farms too!
![farming_tf_wallet_24](./img/farming_tf_wallet_24.png)
To create a farm, either click on the *plus* sign at the top right, or click on *Click to create ->*.
![farming_tf_wallet_25](./img/farming_tf_wallet_25.png)
To create a farm, choose the wallet associated with the farm. Also, choose a Farm Name then read and accept the Terms and Conditions. Finally, click *submit*.
![farming_tf_wallet_26](./img/farming_tf_wallet_26.png)
The TF Connect App will give you a confirmation message if the farm was created successfully. If it doesn't work the first time, make sure you have a good Internet connection and try again. If you click on the down arrow, you will be able to see your *Farm Information*.
![farming_tf_wallet_27](./img/farming_tf_wallet_27.png)
You can see on this screen the address were your farming rewards will be sent to. It is the Stellar address associated with the ThreeFold Wallet used to create the farm.
![farming_tf_wallet_28](./img/farming_tf_wallet_28.png)
We can see on the picture above that the Farm ID here is 1821. The farm ID will be used to create the Zero-OS Bootstrap Image in the next section.
<div style="break-after:page"></div>
# 2. Create a Zero-OS Bootstrap Image
## Download the Zero-OS Boostrap Image
We will now learn how to create a Zero-OS Bootstrap Image in order to boot a DIY 3Node.
https://v3.bootstrap.grid.tf
Go on the [ThreeFold Zero-OS Bootstrap Link](https://v3.bootstrap.grid.tf) as shown above.
![Farming_Create_Farm_21](./img/farming_createfarm_21.png)
This is the Zero-OS v3 Bootstrapping page.
![Farming_Create_Farm_22](./img/farming_createfarm_22.png)
Write your farm ID and choose production mode.
![Farming_Create_Farm_23](./img/farming_createfarm_23.png)
Download the bootstrap image. Next, we will burn the bootstrap image.
<div style="break-after:page"></div>
## Burn the Zero-OS Bootstrap Image
For **MAC**, **Linux** and **Windows**, you can use [BalenaEtcher](https://www.balena.io/etcher/) to load/flash the image on a USB stick. This program also formats the USB in the process. This will work for the option **EFI IMG** for UEFI boot, and with the option **USB** for BIOS boot. Simply follow the steps presented to you and make sure you select the correct bootstrap image file you downloaded previously.
General Steps:
1. Download BalenaEtcher at [https://balena.io/etcher](https://balena.io/etcher)
![3node_diy_desktop_42.png](img/3node_diy_desktop_42.png)
![3node_diy_desktop_43.png](img/3node_diy_desktop_43.png)
![3node_diy_desktop_44.png](img/3node_diy_desktop_44.png)
![3node_diy_desktop_45.png](img/3node_diy_desktop_45.png)
![3node_diy_desktop_48.png](img/3node_diy_desktop_48.png)
![3node_diy_desktop_49.png](img/3node_diy_desktop_49.png)
2. Open BalenaEtcher
![3node_diy_desktop_50.png](img/3node_diy_desktop_50.png)
3. Select **Flash from file**
![3node_diy_desktop_52.png](img/3node_diy_desktop_52.png)
1. Find and select the bootstrap image in your computer
2. Select **Target** (your USB key)
![3node_diy_desktop_53.png](img/3node_diy_desktop_53.png)
![3node_diy_desktop_54.png](img/3node_diy_desktop_54.png)
6. Select **Flash**
![3node_diy_desktop_55.png](img/3node_diy_desktop_55.png)
![3node_diy_desktop_56.png](img/3node_diy_desktop_56.png)
That's it. Now you have a bootstrap image on Zero-OS as a bootable removable media device.
<div style="break-after:page"></div>
# 3. Set the Hardware
Setting the hardware of this DIY 3node is very easy as there are no updates or upgrades needed. Simply unbox the computer and plug everything.
![3node_diy_desktop_40.png](img/3node_diy_desktop_40.jpeg)
![3node_diy_desktop_38.png](img/3node_diy_desktop_38.jpeg)
![3node_diy_desktop_30.png](img/3node_diy_desktop_30.jpeg)
![3node_diy_desktop_29.png](img/3node_diy_desktop_29.jpeg)
Plug the computer cable in the surge protector
![3node_diy_desktop_6.png](img/3node_diy_desktop_6.png)
Connect the computer cable, the ethernet cable, the mouse and keyboard cable and the monitor cable.
![3node_diy_desktop_13.png](img/3node_diy_desktop_13.jpeg)
Plug the ethernet cable in the router (or the switch)
![3node_diy_desktop_6.png](img/3node_diy_desktop_3.png)
<div style="break-after:page"></div>
# 4. Wipe All the Disks
In this section, we will learn how to create a Linux bootstrap image, boot it in Try mode and then wipe all the disks in your 3Node. To create a Linux boostrap image, follow the same process as when we burnt the Zero-OS Boostrap Image.
## 1. Create a Linux Boostrap Image
1. Download the Ubuntu 20.04 ISO file [here](https://releases.ubuntu.com/20.04/)
2. Burn the ISO image on a USB key with Balena Etcher
## 2. Boot Linux in Try Mode
1. Insert your Linux boostrap image USB key in your computer and boot it
2. During boot, press F12 to enter into Settings
3. Select your booting device, here it is: *UEFI: USB DISK 2.0*
![3node_diy_desktop_107.png](img/3node_diy_desktop_107.jpeg)
4. Select Try or install Ubuntu
![3node_diy_desktop_106.png](img/3node_diy_desktop_106.jpeg)
5. Select Try Ubuntu
![3node_diy_desktop_105.png](img/3node_diy_desktop_105.jpeg)
## 3. Use wipefs to Wipe All Disks
Once Ubuntu is booted, you will land on the main page.
![3node_diy_desktop_67.png](img/3node_diy_desktop_67.png)
At the bottom left of the screen, click on Applications.
![3node_diy_desktop_68.png](img/3node_diy_desktop_68.png)
In Applications, select Terminal.
![3node_diy_desktop_69.png](img/3node_diy_desktop_69.png)
If you don't see it, write terminal in the search box.
![3node_diy_desktop_70.png](img/3node_diy_desktop_70.png)
You will land in the Ubuntu Terminal.
![3node_diy_desktop_71.png](img/3node_diy_desktop_71.png)
Write the command line *lsblk* as shown below. You will then see the disks in your computer. You want to wipe the main disk, but not the USB key we are using, named *sdb* here. We can see here that the SSD disk, *sda*, has 3 partitions: *sda1*, *sda2*, *sda3*. Note that when wiping the disks, we want no partition.
In this case, the disk we want to wipe is *sda*.
![3node_diy_desktop_72.png](img/3node_diy_desktop_72.png)
Write the command line *sudo wipefs -a /dev/sda*. This will wipe the disk *sda*.
![3node_diy_desktop_73.png](img/3node_diy_desktop_73.png)
If you write the command line *lsblk* once more, you should see that your SSD disk has no more partition. The disk has been properly wiped.
![3node_diy_desktop_74.png](img/3node_diy_desktop_74.png)
Power off the computer by selecting *Power Off* after having clicked on the button at the top right of the screen as shown below.
![3node_diy_desktop_75.png](img/3node_diy_desktop_75.png)
That's it! The disks are all wiped. All that is left now is to set the BIOS/UEFI settings and then boot the 3Node!
<div style="break-after:page"></div>
# 5. Set the BIOS/UEFI
Before booting the main operating system, in our case Zero-OS, a computer will boot in either BIOS or UEFI mode. Older systems use BIOS and newer systems uses UEFI. Both BIOS and UEFI are low-lewel softwares needed to interact between the hardware and the main OS of the computer. Note that BIOS is also called Legacy BIOS.
## The Essential Features of BIOS/UEFI for a 3Node
There are certain things that you should make sure are set properly on your 3Node.
As a general advice, you can Load Defaults (Settings) on your BIOS, then make sure the options below are set properly.
* Choose the correct combination of BIOS/UEFI and bootstrap image on [https://bootstrap.grid.tf/](https://bootstrap.grid.tf/)
* Newer system will use UEFI --> the Dell Optiplex 7020 uses UEFI
* Bootstrap image: *EFI IMG* and *EFI FILE*
* Older system will use Legacy BIOS
* Bootstrap image: *ISO* and *USB*
* Set *Multi-Processor* and *Hyperthreading* at Enabled
* Sometimes, it will be written *Virtual Cores*, or *Logical Cores*.
* Set *Virtualization* at Enabled
* On Intel, it is denoted as *CPU virtualization* and on ASUS, it is denoted as *SVM*.
* Make sure virtualization is enabled and look for the precise terms in your specific BIOS/UEFI.
* Enable *Network Stack* (sometimes called *Network Boot*)
* Set *AC Recovery* at *Last Power State*
* This will make sure your 3Node restarts after losing power momentarily.
* Select the proper *Boot Sequence* for the 3Node to boot Zero-OS from your bootstrap image
* e.g., if you have a USB key as a bootstrap image, select it in *Boot Sequence*
* Set *Server Lookup Method* (or the equivalent) at *DNS*.
* Only use Static IP if you know what you are doing.
* Your router will automatically assign a dynamic IP address to your 3Node when it connects to Internet.
* Set *Client Address Method* (or the equivalent) at *DHCP*. Only use Static IP if you know what you are doing.
* Your router will automatically assign a dynamic IP address to your 3Node when it connects to Internet.
* *Secure Boot* should be left at disabled
* Enable it if you know what you are doing. Otherwise, it can be set at disabled.
<div style="break-after:page"></div>
## Set the BIOS/UEFI on a Dell Optiplex 7020
1. Insert your Zero-OS boostrap image USB key in your computer and boot it.
2. During boot, press F12 to enter into *Settings* then choose *BIOS Setup*.
![3node_diy_desktop_104.jpeg](img/3node_diy_desktop_109.png)
3. In BIOS Setup, click on Load Default and confirm by clicking on *OK*
![3node_diy_desktop_115.png](img/3node_diy_desktop_115.png)
4. Leave the BIOS Setup (Exit) and re-enter. This will set the default settings.
5. Go through each page and make sure you are following the guidelines set in the section Essential Features as shown in the following pictures.
![3node_diy_desktop_.png](img/3node_diy_desktop_116.png)
![3node_diy_desktop_.png](img/3node_diy_desktop_117.png)
![3node_diy_desktop_.png](img/3node_diy_desktop_118.png)
![3node_diy_desktop_.png](img/3node_diy_desktop_114.png)
![3node_diy_desktop_.png](img/3node_diy_desktop_127.png)
![3node_diy_desktop_.png](img/3node_diy_desktop_120.png)
![3node_diy_desktop_.png](img/3node_diy_desktop_128.png)
![3node_diy_desktop_.png](img/3node_diy_desktop_122.png)
![3node_diy_desktop_.png](img/3node_diy_desktop_123.png)
![3node_diy_desktop_.png](img/3node_diy_desktop_129.png)
![3node_diy_desktop_.png](img/3node_diy_desktop_125.png)
6. Once you are done, click on *Exit* and then click *Yes* to save your changes. The 3node will now boot Zero-OS.
![3node_diy_desktop_126.png](img/3node_diy_desktop_126.png)
<div style="break-after:page"></div>
# 6. Boot the 3Node
If your BIOS/UEFI settings are set properly and you have the Zero-OS bootstrap image USB key plugged in, your 3node should automatically boot Zero-OS every time that it is turned on.
1. Power on the 3Node with the Zero-OS boostrap image USB key
2. Let the 3Node load Zero-OS
* The first time it boots, the 3node will register to the TF Grid
3. Verify the 3Node's status on ThreeFold Explorer
The first time you boot a 3Node, it will be written: “This node is not registered (farmer : NameOfFarm). This is normal. The Grid will create a node ID and you will be able to see it on screen. This can take a couple of minutes.
This is the final screen you should see when your 3Node is connected to the ThreeFold Grid. Note that it is normal if it is written *no public config* next to *PUB* as we did not set any public IP address.
Naturally, your node ID as well as your farm ID and name will be shown.
![3node_diy_desktop_76.png](img/3node_diy_desktop_130.png)
Once you have your node ID, you can also go on the ThreeFold Explorer to see your 3Node and verify that your 3Node is online.
<div style="break-after:page"></div>
## Check the 3Node's Status on the TF Explorer
In a browser, go on the [ThreeFold Explorer](https://dashboard.grid.tf/explorer/nodes).
Select **Grid V3**.
![Farming_Create_Farm_3](./img/farming_25.png)
Then select **Mainnet**.
![Farming_Create_Farm_3](./img/farming_26.png)
In this example, we will look for the farm 100. Use your farm ID.
We can see here that 5 x 3Nodes of the Farm 100 are online.
![Farming_Create_Farm_3](./img/farming_27.png)
On the top right of this section, you can click on "Show offline nodes".
![Farming_Create_Farm_3](./img/farming_28.png)
Now we can see the 3Nodes that are offline.
![Farming_Create_Farm_3](./img/farming_29.png)
## Farming Rewards Distribution
The farming reward will be sent once per month directly in your ThreeFold Connect App wallet. Farming rewards are usually distributed around the 5th of each month.
# Additional Information
Congratulations, you now have built your first ThreeFold 3Node server!
If you have any questions, you can ask the ThreeFold community for help on the [ThreeFold Forum](https://forum.threefold.io/) or on the [ThreeFold Telegram Farmer Group](https://t.me/threefoldfarmers).

Binary file not shown.

After

Width:  |  Height:  |  Size: 247 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 23 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 27 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 34 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 10 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 229 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 68 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.3 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 285 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 726 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 490 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 778 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 798 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 691 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 599 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 788 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 758 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 713 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 796 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 753 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 819 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 670 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 728 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 682 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 709 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 67 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 901 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 229 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 382 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 399 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 352 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 408 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 42 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 56 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 146 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 55 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 340 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 25 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 39 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 9.6 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 45 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 13 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 13 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 10 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 52 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 11 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 16 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 13 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 23 KiB

Some files were not shown because too many files have changed in this diff Show More