Metadata-Version: 2.1
Name: PyProfQueue
Version: 0.1.1
Summary: A python package to inject profiling initialisation into bash scripts, translate queue options and submit jobs
Home-page: https://github.com/Marcus-Keil/PyProfQueue
Author: Marcus Keil
Author-email: marcusk050291@gmail.com
License: MIT License
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: BSD License
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3.10
Description-Content-Type: text/markdown
Requires-Dist: numpy
Requires-Dist: pytz
Requires-Dist: promql_http_api==0.3.3
Requires-Dist: pandas<=2.2.1

# PyProfQueue
The package PyProfQueue provides the Script class, which takes the path and name of
the bash script to be submitted and profiled in order to read in the queue options
and add any needed initialisations in order to profile using prometheus and likwid.

The Script class has a few methods that can be called, these are:

- **add_likwid**(likwid_req: list, likwid_output: str ='./')
  - Adds necessary initiation to use likwid to create a roof-line model
  - likwid_req is a list that should contain the necessary lines for the system in use to be able to use liquid. 
For example, loading the likwid module:
```
likwid_req = ['module load oneAPI_comp', 'module load likwid']
```
- **add_prometheus**(prometheus_req: list, prometheus_output: str ='./')
  - Adds necessary initiation to use likwid to create a roof-line model
  - prometheus_req is a list that should contain the necessary lines for the system in use to be able to use
prometheus.
It is necessary to at least add the following two entries:
``` 
prometheus_req = [
    'export PROMETHEUS_SOFTWARE=<Path to Prometheus software>',
    'export PROMPYTHON=<Path to Python file for read_prometheus.py>'
]
```
- **change_options**(queue_options: dict)
  - Allows for options to be changed post initiation of a Script object, in case the options given in the 
initialisation are no longer desired.

Beyond these three, it is also possible to call the method 'create_profilefile' if one wants to create the
bash files. This method is called automatically by the submit option and is provided as an optional method
to validate the content of the files to be submitted.

## Requirements
### Python Packages

For the sake of PyProfQueue, the required python version is at least 3.10, as this package utilises the match 
functionality. In order to run *read_prometheus.py*, the path to a python implementation/environment needs to 
be provided. This implementation needs to have the following packages:
- promql_http_api   
- numpy             
- pytz              
- pandas            
- datetime          

This python implementation does not need to be the same as the one being used to submit the scripts, but does
need to be present on the system on which the job is being submitted.

### Non python requirements

In addition to the python requirements listed above, PyProfQueue also needs to have the following software on 
the system to which the job will be submitted:
- [Prometheus](https://prometheus.io/)
- [likwid](https://github.com/RRZE-HPC/likwid)

For prometheus, it is enough to download the software as long as prometheus can be launched by the user without 
sudo rights. For the sake of likwid, it needs to be installed or loaded in such a way that a user could run the 
following command without sudo rights:
```
likwid-perfctr -g MEM_DP -f <executable>
```

## Adding new Queue system compatibility

In order to add new queue system compatibility refer to the block comments in the Script class in script.py. 
Each of the four section that needs changes is marked with "Queue System specifics" followed by a {1}, {2}, 
{3} or {4}.
