Metadata-Version: 2.1
Name: python-grid5000
Version: 0.0.4
Summary: UNKNOWN
Home-page: https://gitlab.inria.fr/msimonin/python-grid5000
Author: Matthieu Simonin
Author-email: matthieu.simonin@inria.fr
License: UNKNOWN
Keywords: REST,evaluation,reproducible research,Grid5000
Platform: UNKNOWN
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: System Administrators
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: GNU General Public License v3 (GPLv3)
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Requires-Dist: requests (<2.22,>=2.21)
Requires-Dist: pyyaml (<4,>=3.13)

===============
python-grid5000
===============


``python-grid5000`` is a python package wrapping the Grid’5000 REST API.

.. warning::

    The code is currently being developed heavily. Jump to the contributing section
    if you want to be involved.

1 Thanks
--------

The core code is borrowed from `python-gitlab <https://github.com/python-gitlab/python-gitlab>`_ with small adaptations to
conform with the Grid5000 API models (with an ’s’!)

2 Contributing
--------------

- To contribute, you can drop me an email or open an issue for a bug report, or feature request.

- There are many areas where this can be improved some of them are listed here:

  - The complete coverage of the API isn’t finished (yet) but this should be fairly easy to reach.
    Most of the logic go in ```grid5000.objects`` <https://gitlab.inria.fr/msimonin/python-grid5000/blob/master/grid5000/objects.py>`_. And to be honnest I only
    implemented the feature that I needed the most.

  - Returned `status code <https://www.grid5000.fr/mediawiki/index.php/API#Status_Codes>`_ aren’t yet well treated.

3 Comparison with ...
---------------------

- `RESTfully <https://api.grid5000.fr/doc/4.0/tools/restfully.html>`_: 
  It consumes REST API following the `HATEOAS <https://en.m.wikipedia.org/wiki/HATEOAS>`_ principles. This allows the client
  to fully discover the resources and actions available. Most of the G5K API
  follow theses principles but, for instance the `Storage API <https://www.grid5000.fr/mediawiki/index.php/Storage_Manager>`_ don’t. Thus
  RESTfully isn’t compatible with all the features offered by the Grid’5000 API.
  It’s a ruby library. Python-grid5000 borrows the friendly syntax for resource
  browsing, but in python.

- `Execo <http://execo.gforge.inria.fr>`_: 
  Written in Python. The api module gathers a lot of utils functions leveraging
  the Grid’5000 API. Resources aren’t exposed in a syntax friendly manner,
  instead functions for some classical operations are exposed (mainly getters).
  It has a convenient way of caching the reference API. Python-grid5000 is a
  wrapper around the Grid’5000 that seeks 100% coverage. Python-grid5000 is
  resource oriented.

- `Raw requests <http://docs.python-requests.org>`_: 
  **The** reference for HTTP library in python. Good for prototyping but low-level.
   python-grid5000 encapsulates this library.

4 Installation and examples
---------------------------

- Please refer to `https://api.grid5000.fr/doc/4.0/reference/spec.html <https://api.grid5000.fr/doc/4.0/reference/spec.html>`_ for 
  the complete specification.

- All the examples are exported in the examples subdirectory so you can
  easily test and adapt them.

- The configuration is read from a configuration file located in the home
  directory (should be compatible with the restfully one). 
  It can be created with the following:

::

    echo '
    username: MYLOGIN
    password: MYPASSWORD
    ' > ~/.python-grid5000.yaml

.. info::

    Inside Grid’5000, you’l probably need to set ``verify_ssl: False``

- Using a virtualenv is recommended (python 3.5+ is required)

::

    virtualenv -p python3 venv
    source venv/bin/activate
    pip install python-grid5000

4.1 Get node information
~~~~~~~~~~~~~~~~~~~~~~~~

.. code:: python

    import os

    from grid5000 import Grid5000

    conf_file = os.path.join(os.environ.get("HOME"), ".python-grid5000.yaml")
    gk = Grid5000.from_yaml(conf_file)

    node_info = gk.sites["nancy"].clusters["grisou"].nodes["grisou-1"]
    print("grisou-1 has {threads} threads and has {ram} bytes of RAM".format(
        threads=node_info.architecture["nb_threads"],
        ram=node_info.main_memory["ram_size"]))

4.2 Job filtering
~~~~~~~~~~~~~~~~~

.. code:: python

    import os


    from grid5000 import Grid5000

    conf_file = os.path.join(os.environ.get("HOME"), ".python-grid5000.yaml")
    gk = Grid5000.from_yaml(conf_file)

    # state=running will be placed in the query params
    running_jobs = gk.sites["rennes"].jobs.list(state="running")
    print(running_jobs)

    # get a specific job by its uid
    job = gk.sites["rennes"].jobs.get("424242")
    print(job)

4.3 Submit a job
~~~~~~~~~~~~~~~~

.. code:: python

    import os
    import time

    from grid5000 import Grid5000

    conf_file = os.path.join(os.environ.get("HOME"), ".python-grid5000.yaml")
    gk = Grid5000.from_yaml(conf_file)

    # This is equivalent to gk.sites.get("rennes")
    site = gk.sites["rennes"]

    job = site.jobs.create({
        "name":"pyg5k",
        "command": "sleep 3600",
    })

    while job.state != "running":
        job.refresh()
        print("Waiting for the job [%s] to be running" % job.uid)
        time.sleep(10)

    print(job)
    print("Assigned nodes : %s" % job.assigned_nodes)

4.4 Deploy an environment
~~~~~~~~~~~~~~~~~~~~~~~~~

.. code:: python

    import os
    import time

    from grid5000 import Grid5000

    conf_file = os.path.join(os.environ.get("HOME"), ".python-grid5000.yaml")
    gk = Grid5000.from_yaml(conf_file)

    # This is equivalent to gk.sites.get("rennes")
    site = gk.sites["rennes"]

    job = site.jobs.create({
        "name":"pyg5k",
        "command": "sleep 3600",
        "types": ["deploy"]
    })

    while job.state != "running":
        job.refresh()
        print("Waiting the job [%s] to be running" % job.uid)
        time.sleep(10)

    print("Assigned nodes : %s" % job.assigned_nodes)

    deployment = site.deployments.create({"nodes": job.assigned_nodes,
                                          "environment": "debian9-x64-min"})

    while deployment.status != "terminated":
        deployment.refresh()
        print("Waiting for the deployment [%s] to be finished" % deployment.uid)
        time.sleep(10)

    print(deployment.result)

4.5 Get Storage accesses
~~~~~~~~~~~~~~~~~~~~~~~~

.. code:: python

    import os

    from grid5000 import Grid5000


    conf_file = os.path.join(os.environ.get("HOME"), ".python-grid5000.yaml")
    gk = Grid5000.from_yaml(conf_file)

    print(gk.sites["rennes"].storage["msimonin"].access.list())

4.6 Set storage accesses (e.g for vms)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code:: python

    from netaddr import IPNetwork
    import os
    import time

    from grid5000 import Grid5000


    conf_file = os.path.join(os.environ.get("HOME"), ".python-grid5000.yaml")
    gk = Grid5000.from_yaml(conf_file)
    site = gk.sites["rennes"]

    job = site.jobs.create({
        "name":"pyg5k",
        "command": "sleep 3600",
        "resources": "slash_22=1+nodes=1"
    })

    while job.state != "running":
        job.refresh()
        print("Waiting the job [%s] to be running" % job.uid)
        time.sleep(5)

    subnet = job.resources_by_type['subnets'][0]
    ip_network = [str(ip) for ip in IPNetwork(subnet)]

    # create acces for all ips in the subnet
    access = site.storage["msimonin"].access.create({'ipv4': ip_network,
                                                      'termination': {"job": job.uid, 
                                                                      "site": site.uid}})

4.7 Get vlan(s)
~~~~~~~~~~~~~~~

.. code:: python

    import os

    from grid5000 import Grid5000


    conf_file = os.path.join(os.environ.get("HOME"), ".python-grid5000.yaml")
    gk = Grid5000.from_yaml(conf_file)

    site = gk.sites["rennes"]

    # Get all vlans
    vlans = site.vlans.list()
    print(vlans)

    # Get on specific
    vlan = site.vlans.get("4")
    print(vlan)

    vlan = site.vlans["4"]
    print(vlan)

    # Get vlan of some nodes
    print(site.vlansnodes.submit({"nodes": ["paravance-1.rennes.grid5000.fr", "paravance-2.rennes.grid5000.fr"]}))


    # Get nodes in vlan
    print(site.vlans["4"].nodes.list())

4.8 TODO Set nodes in vlan
~~~~~~~~~~~~~~~~~~~~~~~~~~

- Putting primary interface in a vlan

  .. code:: python

      from netaddr import IPNetwork
      import os
      import time

      from grid5000 import Grid5000


      conf_file = os.path.join(os.environ.get("HOME"), ".python-grid5000.yaml")
      gk = Grid5000.from_yaml(conf_file)
      site = gk.sites["rennes"]

      job = site.jobs.create({
          "name":"pyg5k",
          "command": "sleep 3600",
          "resources": "{type='kavlan'}/vlan=1+nodes=1",
          "types": ["deploy"]
      })

      while job.state != "running":
          job.refresh()
          print("Waiting the job [%s] to be runnning" % job.uid)
          time.sleep(5)

      deployment = site.deployments.create({"nodes": job.assigned_nodes,
                                            "environment": "debian9-x64-min", 
                                            "vlan": job.resources_by_type["vlans"][0]})

      while deployment.status != "terminated":
          deployment.refresh()
          print("Waiting for the deployment [%s] to be finished" % deployment.uid)
          time.sleep(10)

      print(deployment.result)

- Putting the secondary interface in a vlan

  .. code:: python

      from netaddr import IPNetwork
      import os
      import time

      from grid5000 import Grid5000

      def _to_network_address(host, interface):
            """Translate a host to a network address
            e.g:
            paranoia-20.rennes.grid5000.fr -> paranoia-20-eth2.rennes.grid5000.fr
            """
            splitted = host.split('.')
            splitted[0] = splitted[0] + "-" + interface

            return ".".join(splitted)

      conf_file = os.path.join(os.environ.get("HOME"), ".python-grid5000.yaml")
      gk = Grid5000.from_yaml(conf_file)
      site = gk.sites["rennes"]

      job = site.jobs.create({
          "name":"pyg5k",
          "command": "sleep 3600",
          "resources": "{type='kavlan'}/vlan=1+{cluster='paranoia'}nodes=1",
          "types": ["deploy"]
      })

      while job.state != "running":
          job.refresh()
          print("Waiting the job [%s] to be runnning" % job.uid)
          time.sleep(5)

      vlanid = job.resources_by_type["vlans"][0]

      # we hard code the interface but this can be discovered in the node info
      # TODO: write the code here to discover
      nodes = [_to_network_address(n, "eth2") for n in job.assigned_nodes]
      print(nodes)

      # set in vlan
      site.vlans[vlanid].submit({"nodes": nodes})


