FGLab: Machine Learning Dashboard

Overview

FGLab is a machine learning dashboard, designed to make prototyping experiments easier. Experiment details and results are sent to a database, which allows analytics to be performed after their completion. The server is FGLab, and the clients are FGMachines.

After creating projects on FGLab, you write corresponding project implementations on FGMachine. This is done through the use of different JSON schemas. Experiments are submitted via FGLab, and an available FGMachine client will spawn your machine learning code that will conduct the experiment.

Source code, documentation and examples for both FGLab and FGMachine can be found in their corresponding repositories:

• FGLab
• FGMachine

Quickstart

1. Install: FGLab and FGMachine
2. Code: Adjust your machine learning code, e.g. code.py, to take command-line inputs, e.g. x, y
3. Project schema: Create a project schema, e.g. proj.json, to match the command-line inputs, e.g.
   

     {
       "x": {
         "type": "int",
         "default": -1
       },
       "y": {
         "type": "float",
         "default": 2.5
       }
     }

4. Create project: Go to FGLab at http://localhost:5080. Under New Project, choose the project schema, e.g. proj.json, to upload, then create the project.
5. Create project implementation: In FGLab, click the add button to set up a template in FGMachine's projects.json. In the FGMachine folder, adjust projects.json with the following format:
   

     "<project_id>": {
       "cwd": "<working directory (e.g. .)>",
       "command": "<program (e.g. python)>",
       "args": "<first command line options (e.g. code.py)>",
       "options": "<command line options style for options (e.g. double-dash)>",
       "boolean": "<optional: only pass flag if true, mandatory: pass flag and true/false argument>",
       "capacity": "<machine capacity needed (as a fraction) (e.g. 0.2)>",
       "results": "<results directory (without experiment ID) (e.g. ./results)>"
     }

6. Run experiment: In FGLab, click New Experiment and Submit.

You've just set up FGLab, one instance of FGMachine, and run an experiment! To see what else you can do, check out the full docs.

Workflow

FGLab suits a different kind of workflow to Jupyter notebooks/CodaLab worksheets. The latter are ideal for presenting a single working experiment along with data, live code, explanatory text and visualisations. On the other hand, FGLab decouples code from the dashboard, with the results of a series of experiments with different hyperparameters being stored in a database. Prototyping experiments often involves small code changes that are not worth committing - these can qualitatively be captured in the notes interface that accompanies each experiment, much like with a physical lab notebook.

Screenshots

Optimisation

Experiments

Experiment