Create SIF images for COMPS
To use (with uv):
uvx comps_sif_constructor create -d lolcow.defThis will launch the image creation on COMPS and leave behind a sif.id for the jobs that need the image.
comps_sif_constructor --helpThe CLI has four main commands: create, launch, gather, and login.
Create a Apptainer/Singularity image file on COMPS with create:
comps_sif_constructor create --help
usage: comps_sif_constructor create [--help] [--definition_file DEFINITION_FILE] [--output_id OUTPUT_ID]
[--work_item_name WORK_ITEM_NAME] [--requirements REQUIREMENTS]
options:
--help show this help message and exit
--definition_file DEFINITION_FILE, -d DEFINITION_FILE
Path to the Singularity definition file (default: apptainer.def)
--output_id OUTPUT_ID, -o OUTPUT_ID
Name out Asset id file (default: sif.id)
--work_item_name WORK_ITEM_NAME, -w WORK_ITEM_NAME
Name of the work item (default: Singularity Build)
--requirements REQUIREMENTS, -r REQUIREMENTS
Path to the requirements fileExample:
comps_sif_constructor create \
-d <path_to_definition_file> \
-o <output_id> \
-w <work_item_name> \
[-r <requirements_file>]Launch a COMPS experiment with specified parameters:
comps_sif_constructor launch --help
usage: comps_sif_constructor launch [--help] [--name NAME] [--threads THREADS]
[--priority PRIORITY] [--node-group NODE_GROUP]
--file FILE [--sif-id-file SIF_ID_FILE]
[--experiment-id-file EXPERIMENT_ID_FILE]
options:
--help show this help message and exit
--name NAME, -n NAME Name of the experiment (default: comps-sif-constructor experiment)
--threads THREADS, -t THREADS
Number of threads to use (default: 1)
--priority PRIORITY, -p PRIORITY
Priority level for the experiment (default: AboveNormal)
--node-group NODE_GROUP, -g NODE_GROUP
Node group to use (default: idm_48cores)
--file FILE, -f FILE Path to the trials.jsonl file (REQUIRED)
--sif-id-file SIF_ID_FILE, -i SIF_ID_FILE
Path to the asset ID file (default: sif.id)
--experiment-id-file EXPERIMENT_ID_FILE, -o EXPERIMENT_ID_FILE
Path to the output id file (default: experiment.id)Example:
comps_sif_constructor launch \
-n "My Experiment" \
-t 4 \
-p AboveNormal \
-g idm_48cores \
-f trials.jsonl \
-i sif.id \
-o experiment.idGather data from a COMPS experiment:
comps_sif_constructor gather --help
usage: comps_sif_constructor gather [--help] [--experiment-id EXPERIMENT_ID] [--output OUTPUT]
options:
--help show this help message and exit
--experiment-id EXPERIMENT_ID, -e EXPERIMENT_ID
Experiment id (default: experiment.id)
--output OUTPUT, -o OUTPUT
Output file (default: data_brick.json)Example:
comps_sif_constructor gather \
-e experiment.id \
-o data_brick.jsonSave your COMPS credentials using the login command:
comps_sif_constructor login --help
usage: comps_sif_constructor login [--help] [--comps_url COMPS_URL] [--username USERNAME] [--password PASSWORD]
options:
--help show this help message and exit
--comps_url COMPS_URL
COMPS URL (default: https://comps.idmod.org)
--username USERNAME, -u USERNAME
Username (can also use COMPS_USERNAME environment variable)
--password PASSWORD, -p PASSWORD
Password (can also use COMPS_PASSWORD environment variable)Example:
comps_sif_constructor login \
-u your_username \
-p 'your_password'You can also set COMPS_USERNAME and COMPS_PASSWORD as environment variables to avoid entering credentials each time.
- Learn about definition files