How to Set Up and Use Flux

As a data provider on Flux, you play a crucial role in supplying the computational tasks that power our distributed network. This guide will walk you through the process of creating model parameters, uploading datasets, and managing your computational tasks.

Step 1: Create a Model Parameter

To begin processing with Flux, you need to define your model's configuration. This configuration includes details about the model's architecture, such as layers, activation functions, and other parameters essential for its operation.

• Model Configuration Example:

  jsonCopy code{
      "module": "keras",
      "class_name": "Sequential",
      "config": {
          "name": "sequential",
          "trainable": true,
          "dtype": "float32",
          "layers": [
              {
                  "module": "keras.layers",
                  "class_name": "InputLayer",
                  "config": {
                      "batch_shape": [null, 14],
                      "dtype": "float32",
                      "name": "input_layer"
                  }
              },
              ...
              {
                  "module": "keras.layers",
                  "class_name": "Dense",
                  "config": {
                      "name": "dense_2",
                      "trainable": true,
                      "dtype": "float32",
                      "units": 1,
                      "activation": "sigmoid"
                  }
              }
          ]
      },
      "compile_config": {
          "optimizer": "adam",
          "loss": "binary_crossentropy",
          "metrics": ["accuracy"]
      }
  }

  In this configuration, you can define the name, description, and the actual model parameters including layers and optimizer settings.

Step 2: Upload a Dataset

Once your model parameter is set, you can upload your dataset for processing.

• Dataset Configuration:

  1. Naming and Description: Provide a clear name and description for your dataset.

  2. Model Parameter Selection: Select a previously created model parameter from a dropdown menu to associate with your dataset.

  3. Categorical Features: Define the categorical columns in your data, which are crucial for the preprocessing steps.

     jsonCopy code{
         "categorical_features": ["Workclass", "Education", "Marital Status", "Occupation", "Relationship", "Race", "Sex", "Country"]
     }

  4. Training and Verification Sets: Upload your training set and a verification set. The training set is used for building the model, while the verification set is used to evaluate its performance.

Step 3: Energy Estimation and Processing

• Microtask Processing: After uploading, Flux performs initial microtasks on segments of your data to measure the energy consumption. This helps in providing an accurate estimation of the computational cost.

• Energy Estimation: Based on the initial processing, Flux provides an energy estimate. You can then decide whether to proceed with the full processing based on the cost and resources available.

Step 4: Retrieving the Trained Model

• Model Output: Once processing is complete, the trained model is available for download in JSON format, including the model weights. This allows you to easily integrate the model into your systems or further enhance it if necessary.

Best Practices

• Data Privacy: All models and datasets are encrypted by default to ensure data privacy.

• Efficient Use of Resources: Be mindful of the energy estimation and adjust your model or dataset accordingly to optimize processing time and costs.

By following these steps, you can effectively use Flux to process your machine learning tasks, leveraging the power of distributed computing to achieve faster results and reduce operational costs. If you need any assistance or have further questions, please refer to our support resources or contact our helpdesk.