:orphan:



.. _sphx_glr_auto_integrations_flytekit_plugins_modin_examples:

Modin
======

Modin is a pandas-accelerator that helps handle large datasets.
Pandas works gracefully with small datasets since it is inherently single-threaded, and designed to work on a single CPU core.
With large datasets, the performance of pandas drops (becomes slow or runs out of memory) due to single core usage.
This is where Modin can be helpful.

Instead of optimizing pandas workflows for a specific setup, we can speed up pandas workflows by utilizing all the resources (cores) available in the system using the concept of ``parallelism``, which is possible through modin. `Here <https://modin.readthedocs.io/en/stable/getting_started/why_modin/pandas.html#scalablity-of-implementation>`__ is a visual representation of how the cores are utilized in case of Pandas and Modin.


Installation
------------

.. code:: bash

   pip install flytekitplugins-modin


How is Modin different?
-----------------------

Modin **scales** the Pandas workflows by changing only a **single line of code**.

The plugin supports the usage of Modin DataFrame as an input to and output of a task/workflow, similar to how a pandas DataFrame can be used.



.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="In this example, let&#x27;s understand how effortlessly the Modin DataFrames can be used with tasks ...">

.. only:: html

 .. figure:: /auto/integrations/flytekit_plugins/modin_examples/images/thumb/sphx_glr_knn_classifier_thumb.png
     :alt: KNN Classifier

     :ref:`sphx_glr_auto_integrations_flytekit_plugins_modin_examples_knn_classifier.py`

.. raw:: html

    </div>


.. toctree::
   :hidden:

   /auto/integrations/flytekit_plugins/modin_examples/knn_classifier
.. raw:: html

    <div class="sphx-glr-clear"></div>



.. only :: html

 .. container:: sphx-glr-footer
    :class: sphx-glr-footer-gallery


  .. container:: sphx-glr-download sphx-glr-download-python

    :download:`Download all examples in Python source code: modin_examples_python.zip </auto/integrations/flytekit_plugins/modin_examples/modin_examples_python.zip>`



  .. container:: sphx-glr-download sphx-glr-download-jupyter

    :download:`Download all examples in Jupyter notebooks: modin_examples_jupyter.zip </auto/integrations/flytekit_plugins/modin_examples/modin_examples_jupyter.zip>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_