Command Line Utility¶
The command line utility for datmo is to be used in tandem with the SDK and will typically be your first contact with the datmo system. If using Python, see Python SDK.
If you are working within a repository already, you will want to run the datmo init within your
repository in order to create your datmo project.
From there, you can create snapshots or run tasks using either the SDK or the CLI. At any given point you
can find out more about all of your snapshots using the datmo snapshot ls command and see the status
of any of your tasks with the datmo task ls command.
Sessions are a way for you to group together tasks and snapshots, but are completely optional. For example, if you want to run a set of hyperparameter experiments modifying some subset of hyperparameters you might want to do them in a designated session. Then you might try another set of hyperparameter sweeps which you would like to group into another session. By default, you will always be in the “default” session unless otherwise specified.
You can delve through more of the commands and each of their parameters below to learn more about each entity and how you can create different versions of them. You can also look through the Getting Started section in the README.
usage: datmo [-h]
{init,version,status,cleanup,dashboard,configure,notebook,jupyterlab,terminal,rstudio,run,ls,stop,delete,rerun,environment,snapshot}
...
commands¶
| command | Possible choices: init, version, status, cleanup, dashboard, configure, notebook, jupyterlab, terminal, rstudio, run, ls, stop, delete, rerun, environment, snapshot |
Sub-commands:¶
init¶
initialize project
datmo init [-h] [--name NAME] [--description DESCRIPTION] [--force]
Named Arguments¶
| --name | |
| --description | |
| --force, -f, --no-prompt | |
boolean if you want to run init without prompts Default: False | |
notebook¶
To run jupyter notebook
datmo notebook [-h] [--gpu] [--environment-id ENVIRONMENT_ID]
[--environment-paths ENVIRONMENT_PATHS] [--mem-limit MEM_LIMIT]
[--data DATA]
Named Arguments¶
| --gpu | boolean if you want to run using GPUs Default: False |
| --environment-id | |
| environment id from environment object | |
| --environment-paths | |
| list of absolute or relative filepaths and/or dirpaths to collect; can specify destination names with ‘>’ (e.g. /path/to/file>hello, /path/to/file2, /path/to/dir>newdir) | |
| --mem-limit, -m | |
| maximum amount of memory the notebook environment can use (these options take a positive integer, followed by a suffix of b, k, m, g, to indicate bytes, kilobytes, megabytes, or gigabytes) | |
| --data | list of absolute or relative filepath and/or dirpaths for data; can specify destination names with ‘>’ (e.g. /path/to/dir, /path/to/dir>newdir, /path/to/file) |
jupyterlab¶
To run jupyterlab
datmo jupyterlab [-h] [--gpu] [--environment-id ENVIRONMENT_ID]
[--environment-paths ENVIRONMENT_PATHS]
[--mem-limit MEM_LIMIT] [--data DATA]
Named Arguments¶
| --gpu | boolean if you want to run using GPUs Default: False |
| --environment-id | |
| environment id from environment object | |
| --environment-paths | |
| list of absolute or relative filepaths and/or dirpaths to collect; can specify destination names with ‘>’ (e.g. /path/to/file>hello, /path/to/file2, /path/to/dir>newdir) | |
| --mem-limit, -m | |
| maximum amount of memory the jupyterlab environment can use (these options take a positive integer, followed by a suffix of b, k, m, g, to indicate bytes, kilobytes, megabytes, or gigabytes) | |
| --data | list of absolute or relative filepath and/or dirpaths for data; can specify destination names with ‘>’ (e.g. /path/to/dir, /path/to/dir>newdir, /path/to/file) |
terminal¶
To run terminal
datmo terminal [-h] [--gpu] [--environment-id ENVIRONMENT_ID]
[--environment-paths ENVIRONMENT_PATHS] [--mem-limit MEM_LIMIT]
[--data DATA] [--ports PORTS]
Named Arguments¶
| --gpu | boolean if you want to run using GPUs Default: False |
| --environment-id | |
| environment id from environment object | |
| --environment-paths | |
| list of absolute or relative filepaths and/or dirpaths to collect; can specify destination names with ‘>’ (e.g. /path/to/file>hello, /path/to/file2, /path/to/dir>newdir) | |
| --mem-limit, -m | |
| maximum amount of memory the terminal environment can use (these options take a positive integer, followed by a suffix of b, k, m, g, to indicate bytes, kilobytes, megabytes, or gigabytes) | |
| --data | list of absolute or relative filepath and/or dirpaths for data; can specify destination names with ‘>’ (e.g. /path/to/dir, /path/to/dir>newdir, /path/to/file) |
| --ports, -p |
|
rstudio¶
To run Rstudio workspace
datmo rstudio [-h] [--environment-id ENVIRONMENT_ID]
[--environment-paths ENVIRONMENT_PATHS] [--mem-limit MEM_LIMIT]
[--data DATA]
Named Arguments¶
| --environment-id | |
| environment id from environment object | |
| --environment-paths | |
| list of absolute or relative filepaths and/or dirpaths to collect; can specify destination names with ‘>’ (e.g. /path/to/file>hello, /path/to/file2, /path/to/dir>newdir) | |
| --mem-limit, -m | |
| maximum amount of memory the rstudio environment can use (these options take a positive integer, followed by a suffix of b, k, m, g, to indicate bytes, kilobytes, megabytes, or gigabytes) | |
| --data | list of absolute or relative filepath and/or dirpaths for data; can specify destination names with ‘>’ (e.g. /path/to/dir, /path/to/dir>newdir, /path/to/file) |
run¶
run module
datmo run [-h] [--gpu] [--ports PORTS] [--environment-id ENVIRONMENT_ID]
[--environment-paths ENVIRONMENT_PATHS] [--mem-limit MEM_LIMIT]
[--interactive] [--data DATA]
[cmd]
Positional Arguments¶
| cmd | command to run within environment |
Named Arguments¶
| --gpu | boolean if you want to run using GPUs Default: False |
| --ports, -p |
|
| --environment-id | |
| environment id from environment object | |
| --environment-paths | |
| list of absolute or relative filepaths and/or dirpaths to collect; can specify destination names with ‘>’ (e.g. /path/to/file>hello, /path/to/file2, /path/to/dir>newdir) | |
| --mem-limit, -m | |
| maximum amount of memory the task environment can use (these options take a positive integer, followed by a suffix of b, k, m, g, to indicate bytes, kilobytes, megabytes, or gigabytes. e.g. 4g) | |
| --interactive | run the environment in interactive mode (keeps STDIN open) Default: False |
| --data | list of absolute or relative filepath and/or dirpaths for data; can specify destination names with ‘>’ (e.g. /path/to/dir, /path/to/dir>newdir, /path/to/file) |
ls¶
list module
datmo ls [-h] [--format FORMAT] [--download] [--download-path DOWNLOAD_PATH]
Named Arguments¶
| --format | output format [‘table’, ‘csv’] Default: “table” |
| --download | boolean is true if user would like to download. use –download-path to specify a path Default: False |
| --download-path | |
| checked only if download is specified. saves output to location specified | |
stop¶
stop runs
datmo stop [-h] [--id ID] [--all]
Named Arguments¶
| --id | run id to stop |
| --all, -a | stop all datmo runs Default: False |
environment¶
environment module
datmo environment [-h] {setup,create,update,delete,ls} ...
subcommands¶
| subcommand | Possible choices: setup, create, update, delete, ls |
Sub-commands:¶
setup¶
setup environment adds a predefined supported environment into your project environment directory
datmo environment setup [-h] [--name NAME] [--type TYPE]
[--framework FRAMEWORK] [--language LANGUAGE]
Named Arguments¶
| --name | name of environment to be used for environment (e.g. my-new-environment). if none is given, a prompt will present the supported names |
| --type | type of environment to be used for environment (e.g. cpu). if none is given, a prompt will present the supported type |
| --framework | framework (and relevant libraries) to be used for environment (e.g. data-analytics). if none is given, a prompt will present the supported names |
| --language | language of environment to be used for environment (e.g. py27). if none is given, a prompt will present the supported language for the name and type |
create¶
create environment using the definition paths given, if not looks in your project environment directory, or creates a default
datmo environment create [-h] [--paths PATHS] [--name NAME]
[--description DESCRIPTION]
Named Arguments¶
| --paths | list of absolute or relative filepaths and/or dirpaths to collect; can specify destination names with ‘>’ (e.g. /path/to/file>hello, /path/to/file2, /path/to/dir>newdir) |
| --name, -n | name given to the environment |
| --description, -d | |
| description of environment | |
update¶
update an environment by id
datmo environment update [-h] [--name NAME] [--description DESCRIPTION] id
Positional Arguments¶
| id | environment id to update |
Named Arguments¶
| --name | new name for the environment |
| --description | new description for the environment |
delete¶
delete a environment by id
datmo environment delete [-h] id
Positional Arguments¶
| id | id of environment to delete |
ls¶
list environments
datmo environment ls [-h] [--format FORMAT] [--download]
[--download-path DOWNLOAD_PATH]
Named Arguments¶
| --format | output format [‘table’, ‘csv’] Default: “table” |
| --download | boolean is true if user would like to download. use –download-path to specify a path Default: False |
| --download-path | |
| checked only if download is specified. saves output to location specified | |
snapshot¶
Datmo snapshots allow you to save the state of your model and experiments by keeping track of your source code, environment, configuration, metrics and large files.
datmo snapshot [-h] {create,update,delete,ls,checkout,diff,inspect} ...
subcommands¶
| subcommand | Possible choices: create, update, delete, ls, checkout, diff, inspect |
Sub-commands:¶
create¶
Run snapshot create any time you want to save the results of your experiments. You can then view all snapshots with the snapshot ls command.
datmo snapshot create [-h] [--message MESSAGE] [--label LABEL]
[--session-id SESSION_ID] [--run-id RUN_ID]
[--environment-id ENVIRONMENT_ID]
[--environment-paths ENVIRONMENT_PATHS] [--paths PATHS]
[--config-filename CONFIG_FILENAME]
[--config-filepath CONFIG_FILEPATH] [--config CONFIG]
[--stats-filename STATS_FILENAME]
[--stats-filepath STATS_FILEPATH] [--stats STATS]
Named Arguments¶
| --message, -m | message to describe snapshot |
| --label, -l | label snapshots with a category (e.g. best) |
| --session-id | user given session id |
| --run-id | specify run id to pull information from |
| --environment-id | |
| environment id from environment object | |
| --environment-paths | |
| list of absolute or relative filepaths and/or dirpaths to collect; can specify destination names with ‘>’ (e.g. /path/to/file>hello, /path/to/file2, /path/to/dir>newdir) | |
| --paths | list of absolute or relative filepaths and/or dirpaths to collect; can specify destination names with ‘>’ (e.g. /path/to/file>hello, /path/to/file2, /path/to/dir>newdir) |
| --config-filename | |
| filename to use to search for configuration JSON | |
| --config-filepath | |
| absolute filepath to use to search for configuration JSON | |
| --config, -c |
|
| --stats-filename | |
| filename to use to search for metrics JSON | |
| --stats-filepath | |
| absolute filepath to use to search for metrics JSON | |
| --stats, -s |
|
update¶
update a snapshot by id
datmo snapshot update [-h] [--config CONFIG] [--stats STATS]
[--message MESSAGE] [--label LABEL]
id
Positional Arguments¶
| id | snapshot id to update |
Named Arguments¶
| --config, -c |
|
| --stats, -s |
|
| --message | new message for the snapshot |
| --label | new label for the snapshot |
delete¶
delete a snapshot by id
datmo snapshot delete [-h] id
Positional Arguments¶
| id | snapshot id to delete |
ls¶
list snapshots
datmo snapshot ls [-h] [--session-id SESSION_ID] [--details] [--all]
[--format FORMAT] [--download]
[--download-path DOWNLOAD_PATH]
Named Arguments¶
| --session-id | session id to filter |
| --details | show detailed snapshot information Default: False |
| --all | show all visible and hidden snapshots Default: False |
| --format | output format [‘table’, ‘csv’] Default: “table” |
| --download | boolean is true if user would like to download. use –download-path to specify a path Default: False |
| --download-path | |
| checked only if download is specified. saves output to location specified | |
checkout¶
checkout a snapshot by id
datmo snapshot checkout [-h] id
Positional Arguments¶
| id | snapshot id to checkout |