Neptune plugin#
Neptune is an experiment tracker for large-scale model training. It allows AI researchers to monitor their model training in real time, visualize and compare experiments, and collaborate on them with a team. This plugin enables seamless use of Neptune within Flyte by configuring links between the two platforms. You can find more information about how to use Neptune in their documentation.
Installation#
To install the Flyte Neptune plugin, run the following command:
pip install flytekitplugins-neptune
Example usage#
For a usage example, see the Neptune example.
Local testing#
To run Neptune example locally:
Create an account on Neptune.
Create a project on Neptune.
In the example, set
NEPTUNE_PROJECT
to your project name.Add a secret using Flyte’s Secrets manager with
key="neptune-api-token"
andgroup="neptune-api-group"
If you want to see the dynamic log links in the UI, then add the configuration in the next section.
Flyte deployment configuration#
To enable dynamic log links, add the plugin to Flyte’s configuration file:
plugins:
logs:
dynamic-log-links:
- neptune-run-id:
displayName: Neptune
templateUris: "{{ .taskConfig.host }}/{{ .taskConfig.project }}?query=(%60flyte%2Fexecution_id%60%3Astring%20%3D%20%22{{ .executionName }}-{{ .nodeId }}-{{ .taskRetryAttempt }}%22)&lbViewUnpacked=true"