# Athina

## Docs

- [Athina AI](https://docs.athina.ai/overview.md): Athina is a collaborative AI development platform that lets teams build, test and monitor production-grade AI applications.
- [Getting Started with Athina](https://docs.athina.ai/getting-started.md)
- [Introduction](https://docs.athina.ai/datasets/overview.md)
- [Create a Dataset](https://docs.athina.ai/datasets/create-dataset/create-dataset.md): You can currently create datasets in Athina in the following ways:
- [From Logs](https://docs.athina.ai/datasets/create-dataset/create-dataset-from-logs.md): You can import your inference logs to create a dataset in Athina.
- [Upload File](https://docs.athina.ai/datasets/create-dataset/create-dataset-from-file.md): You can upload a JSONL, CSV, or JSON file to create a dataset in Athina.
- [Via Python SDK](https://docs.athina.ai/datasets/create-dataset/create-dataset-via-python-sdk.md): You can use our Python SDK to create a dataset in Athina.
- [Via API](https://docs.athina.ai/datasets/create-dataset/create-dataset-via-api.md): You can use simple POST API requests to add rows to a dataset.
- [Import a HuggingFace Dataset](https://docs.athina.ai/datasets/create-dataset/import_huggingface_dataset.md)
- [Generate a Synthetic Dataset](https://docs.athina.ai/datasets/create-dataset/generate-synthetic-dataset.md): You can generate synthetic datasets in Athina.
- [Overview](https://docs.athina.ai/datasets/dynamic-columns/dynamic-columns.md): Dynamic columns let you run prompts, code execution, retrievals, and more on your datasets
- [Run Prompt](https://docs.athina.ai/datasets/dynamic-columns/dynamic-columns-run-prompt.md): Send a prompt to a language model to generate an AI response
- [API Call](https://docs.athina.ai/datasets/dynamic-columns/dynamic-columns-api-call.md): Fetch some data from an API endpoint
- [Classification](https://docs.athina.ai/datasets/dynamic-columns/dynamic-columns-classification.md): Classify text into pre-defined labels using an LLM
- [Extract Entities](https://docs.athina.ai/datasets/dynamic-columns/dynamic-columns-extract-entities.md): Extract entities from a previous column using an LLM
- [Code Execution](https://docs.athina.ai/datasets/dynamic-columns/dynamic-columns-code-execution.md): Execute a python function on every row
- [Run evaluations (UI)](https://docs.athina.ai/datasets/run-eval.md): Run evaluations on Athina IDE in a few clicks
- [View Metrics](https://docs.athina.ai/datasets/metrics.md): You can view and compare metrics across datasets in Athina.
- [Run Experiments](https://docs.athina.ai/datasets/run-experiment.md): Re-generate a dataset with a new prompt or a new model and compare the results side-by-side
- [Compare datasets](https://docs.athina.ai/datasets/compare-datasets.md): You can compare multiple datasets side-by-side in Athina.
- [Join Datasets](https://docs.athina.ai/datasets/join-datasets.md): Import columns from another dataset.
- [Export / Download Datasets](https://docs.athina.ai/datasets/export-datasets.md): Export datasets to a file.
- [SQL Notebook](https://docs.athina.ai/datasets/sql.md): Run SQL queries on your datasets
- [Dataset Automations](https://docs.athina.ai/datasets/automations.md): Automate workflows with project automations.
- [Manage Datasets](https://docs.athina.ai/datasets/management.md): Organize datasets with Projects and Tags
- [Delete a Dataset](https://docs.athina.ai/datasets/delete-dataset.md): You can delete a dataset in Athina by following these steps.
- [Athina Evals](https://docs.athina.ai/evals/overview.md)
- [Quick Start](https://docs.athina.ai/evals/quickstart.md)
- [Online Evals](https://docs.athina.ai/evals/online-evals.md)
- [Offline Evals](https://docs.athina.ai/evals/offline-evals.md)
- [Preset Evaluators](https://docs.athina.ai/evals/preset-evals.md)
- [Custom Evals](https://docs.athina.ai/evals/custom-evals.md)
- [Running Evals in UI](https://docs.athina.ai/evals/running-evals-in-ui.md)
- [Running Evals via SDK](https://docs.athina.ai/evals/running-evals-via-sdk.md)
- [Running Evals in CI/CD](https://docs.athina.ai/evals/running-evals-ci-cd.md)
- [Open Source Evaluations](https://docs.athina.ai/evals/why-athina/open-source-evals.md): The philosophy behind `athina-evals`, our open-source evaluation library.
- [Why Athina Evals](https://docs.athina.ai/evals/why-athina/why-athina-evals.md)
- [Eval Cookbooks](https://docs.athina.ai/evals/cookbooks.md)
- [Flows](https://docs.athina.ai/flows/overview.md): Build complex pipelines with Flows.
- [Concepts](https://docs.athina.ai/flows/concepts.md): Learn the basics of flows and blocks.
- [Variables in Flows](https://docs.athina.ai/flows/variables.md): Understand how variables work in flows.
- [Sharing Flows](https://docs.athina.ai/flows/share-flows.md): Learn how to share flows publicly or with selected users.
- [Flow Templates](https://docs.athina.ai/flows/templates.md): Flow Templates are public flows that you can copy and use in your own flows.
- [Blocks](https://docs.athina.ai/flows/blocks/overview.md): Blocks and flows are composable and can be used to create powerful workflows.
- [Search](https://docs.athina.ai/flows/blocks/search.md): Search the web for information.
- [Code Execution](https://docs.athina.ai/flows/blocks/code_execution.md): Execute Python code in a sandbox.
- [Knowledge Retrieval](https://docs.athina.ai/flows/blocks/knowledge.md): Retrieve documents from a knowledge base.
- [Annotation](https://docs.athina.ai/annotations/overview.md): Allow your team to annotate and label your datasets.
- [Annotation Metrics](https://docs.athina.ai/annotations/metrics.md): Understand annotation progress and agreement rates across your datasets.
- [Configure an Annotation Project](https://docs.athina.ai/annotations/configure-project.md): Learn how to set up and configure an annotation project in Athina AI.
- [View Annotated Data](https://docs.athina.ai/annotations/view-annotated-data.md): Learn how to inspect completed annotations and monitor dataset coverage.
- [Review Annotation Entries](https://docs.athina.ai/annotations/review-entries.md): Learn how to review and validate completed annotations directly within the dataset.
- [Export Annotated Data](https://docs.athina.ai/annotations/export-data.md): Export labeled datasets from Athina in your preferred format.
- [Annotation Permissions](https://docs.athina.ai/annotations/permissions.md): Manage who can create, access, and contribute to annotation projects.
- [Overview](https://docs.athina.ai/prompts/overview.md): Athina offers a powerful prompt management system that allows you to create, edit, manage, test and version prompts.
- [Concepts](https://docs.athina.ai/prompts/concepts.md): Understand how prompts are structured in Athina
- [Prompt Syntax](https://docs.athina.ai/prompts/syntax.md): Learn how to write prompts in Athina's Playground.
- [Create Prompt Template](https://docs.athina.ai/prompts/create-prompt.md): You can create prompts in Athina's Prompt Playground, or via API
- [Prompt Versioning](https://docs.athina.ai/prompts/prompt-versioning.md): You can version your prompts in Athina's Prompt Playground, or via API.
- [Delete Prompt Slug](https://docs.athina.ai/prompts/delete-prompt.md): You can delete a prompt slug in Athina's Prompt Playground, or via API
- [List Prompt Slugs](https://docs.athina.ai/prompts/list-prompts.md): You can list all prompt slugs stored on Athina via API or Python SDK
- [Duplicate Prompt Slug](https://docs.athina.ai/prompts/duplicate-prompt.md): You can duplicate a prompt slug in Athina's Prompt Playground, or via API
- [Run Prompt](https://docs.athina.ai/prompts/run-prompt.md): You can run prompts in Athina's Prompt Playground, or via API
- [Multi-Prompt Playground](https://docs.athina.ai/prompts/run-multiple-prompts.md): You can run multiple prompts in Athina's Prompt Playground, or via API
- [Run Evaluations on a Prompt Response](https://docs.athina.ai/prompts/prompt-evals.md): You can evaluate prompt responses in Athina's Playground.
- [Organize Prompts](https://docs.athina.ai/prompts/organization.md): You can organize your prompts in Athina's Prompt Management System
- [Athina Monitoring](https://docs.athina.ai/monitoring/overview.md): Advanced Monitoring & Analytics. Your production environment will thank you.
- [Inference Trace](https://docs.athina.ai/monitoring/inference-trace.md)
- [Analytics and Insights](https://docs.athina.ai/monitoring/analytics.md)
- [Query Topic Classification](https://docs.athina.ai/monitoring/topic-classification.md)
- [How can I export my logged inferences?](https://docs.athina.ai/monitoring/export-data.md)
- [Continuous Evaluation](https://docs.athina.ai/monitoring/continuous-eval.md): Athina can run continuous evaluations on your logs to monitor model performance in production
- [Model Performance Metrics](https://docs.athina.ai/monitoring/performance-metrics.md)
- [Custom Models](https://docs.athina.ai/settings/custom-models.md): You can configure custom models on Athina in the Settings page.
- [Sampling Evals](https://docs.athina.ai/settings/sampling-evals.md): You can set up sampling rules for continuous evals in the Settings.
- [Credits](https://docs.athina.ai/settings/credits.md): Understand how execution credits work in Athina.
- [Integrations](https://docs.athina.ai/integrations.md): Athina has integrations with the following projects/packages:
- [Self-Hosting](https://docs.athina.ai/self-hosting/on-AWS.md)
- [Self-Hosting On Azure](https://docs.athina.ai/self-hosting/on-Azure.md)
- [Logging](https://docs.athina.ai/api-reference/logging/overview.md): To get started with Athina's Monitoring, the first step is to start logging your inferences.
- [Logging Attributes](https://docs.athina.ai/api-reference/logging/logging-attributes.md)
- [Log via API Request](https://docs.athina.ai/api-reference/logging/log-via-api-request.md)
- [Log via Python SDK](https://docs.athina.ai/api-reference/logging/log-via-python-sdk.md)
- [Log via TypeScript SDK](https://docs.athina.ai/api-reference/logging/log-via-typescript-sdk.md)
- [Langchain](https://docs.athina.ai/api-reference/logging/langchain.md)
- [🚅 LiteLLM](https://docs.athina.ai/api-reference/logging/lite-llm.md)
- [OpenAI Chat Completion](https://docs.athina.ai/api-reference/logging/openai-chat-1x.md): If you're using OpenAI chat completions in Python, you can get set up in just 2 minutes
- [OpenAI Chat Completion](https://docs.athina.ai/api-reference/logging/openai-chat-0x.md): If you're using OpenAI chat completions in Python, you can get set up in just 2 minutes
- [OpenAI Assistant](https://docs.athina.ai/api-reference/logging/openai-assistant.md)
- [Tracing](https://docs.athina.ai/api-reference/logging/tracing.md): Athina LLM Application Tracing captures the full context of an execution including retrieval, generation, api calls, and more
- [Tracing via API](https://docs.athina.ai/api-reference/logging/tracing-via-api.md): Tracing to Athina can be done via simple API requests.
- [Tracing using Python decorators](https://docs.athina.ai/api-reference/logging/tracing-using-python-decorators.md): Tracing to Athina can be done using Python decorators.
- [Tracing for Langchain (Python)](https://docs.athina.ai/api-reference/logging/tracing-for-langchain.md): Athina Tracing integrates with Langchain using Langchain Callbacks (Python). Thereby, our SDK automatically creates a nested trace for every run of your Langchain application.
- [Supported Models](https://docs.athina.ai/api-reference/logging/supported-models.md)
- [Update Logs By ID](https://docs.athina.ai/api-reference/logging/updates/update-logs-by-id.md)
- [Update Logs By External Reference ID](https://docs.athina.ai/api-reference/logging/updates/update-logs-by-external-reference-id.md)
- [List All Datasets](https://docs.athina.ai/api-reference/datasets/list.md): You can list all datasets using the Python SDK.
- [Create a Dataset](https://docs.athina.ai/api-reference/datasets/create-dataset.md): You can create a dataset programmatically using the Python SDK.
- [Add Rows to Dataset](https://docs.athina.ai/api-reference/datasets/add-rows-to-dataset.md): You can add rows to an existing dataset using the Python SDK.
- [Get Dataset](https://docs.athina.ai/api-reference/datasets/get-dataset.md): You can get a specific dataset by ID or name.
- [Delete Dataset](https://docs.athina.ai/api-reference/datasets/delete-dataset.md): You can delete a dataset by ID via the Python SDK.
- [Update Cells in a Dataset](https://docs.athina.ai/api-reference/datasets/update-cell-via-api.md): You can update cells in a dataset via the Python SDK or directly via the API.
- [Running Evals](https://docs.athina.ai/api-reference/evals/running-evals/run-single-eval.md)
- [Running a suite of evals](https://docs.athina.ai/api-reference/evals/running-evals/run-eval-suite.md): Here is a sample of all the code you need to run a suite of evals.
- [Loading data for Evals](https://docs.athina.ai/api-reference/evals/loading-data/loading-data-for-eval.md)
- [Loading data via Llama-Index](https://docs.athina.ai/api-reference/evals/loading-data/loading-data-via-llama-index.md)
- [Preset Evals](https://docs.athina.ai/api-reference/evals/preset-evals/overview.md)
- [Context Sufficiency](https://docs.athina.ai/api-reference/evals/preset-evals/rag/context-sufficiency.md)
- [Response Faithfulness](https://docs.athina.ai/api-reference/evals/preset-evals/rag/response-faithfulness.md)
- [Answer Completeness](https://docs.athina.ai/api-reference/evals/preset-evals/rag/answer-completeness.md)
- [Groundedness](https://docs.athina.ai/api-reference/evals/preset-evals/rag/groundedness.md)
- [RAGAS](https://docs.athina.ai/api-reference/evals/preset-evals/rag/ragas.md)
- [PII Detection](https://docs.athina.ai/api-reference/evals/preset-evals/safety/pii-detection.md)
- [Prompt Injection](https://docs.athina.ai/api-reference/evals/preset-evals/safety/prompt-injection.md)
- [OpenAI Content Moderation](https://docs.athina.ai/api-reference/evals/preset-evals/safety/open-ai-content-moderation.md)
- [Guardrails](https://docs.athina.ai/api-reference/evals/preset-evals/guardrails.md)
- [JSON Evals](https://docs.athina.ai/api-reference/evals/preset-evals/json-evals.md)
- [Summarization QA](https://docs.athina.ai/api-reference/evals/preset-evals/summarization-qa.md)
- [Function Based Evaluators](https://docs.athina.ai/api-reference/evals/preset-evals/function-evals.md)
- [Grounded Evals](https://docs.athina.ai/api-reference/evals/preset-evals/grounded-evals.md)
- [Conversation Evals](https://docs.athina.ai/api-reference/evals/preset-evals/conversation-evals.md)
- [API Call](https://docs.athina.ai/api-reference/evals/custom-evals/api-call.md)
- [LLM-as-a-Judge (Custom Prompt Eval)](https://docs.athina.ai/api-reference/evals/custom-evals/custom-prompt.md)
- [Pairwise Evaluation](https://docs.athina.ai/api-reference/evals/custom-evals/pairwise-evaluation.md)
- [Evaluation with Custom Python Code](https://docs.athina.ai/api-reference/evals/custom-evals/custom-code-eval.md)
- [Custom Grading Criteria](https://docs.athina.ai/api-reference/evals/custom-evals/grading-criteria.md)
- [Write your own LLM Eval class](https://docs.athina.ai/api-reference/evals/custom-evals/create-your-own-eval.md)
- [Overview](https://docs.athina.ai/api-reference/graphql-api/overview.md)
- [Getting Started](https://docs.athina.ai/api-reference/graphql-api/getting-started.md)
- [Sample GraphQL Queries](https://docs.athina.ai/api-reference/graphql-api/sample-queries.md)
- [Query the Graph API using cURL or Python](https://docs.athina.ai/api-reference/graphql-api/curl-python-examples.md): We have provided examples of how to query the Athina AI GraphQL API using cURL and Python. You can use these examples to fetch data from the API and integrate it into your applications.
- [OpenAI Completions](https://docs.athina.ai/api-reference/logging/misc/openai-completion-0x.md): _If you're using OpenAI completions in Python, you can get set up in just **2 minutes**_
- [OpenAI Completions](https://docs.athina.ai/api-reference/logging/misc/openai-completion-1x.md): _If you're using OpenAI completions in Python, you can get set up in just **2 minutes**_
- [Getting Started Guides](https://docs.athina.ai/guides/overview.md)
- [Prompt Comparison](https://docs.athina.ai/guides/prompts/prompt-comparison.md): Learn how to compare multiple prompts using Athina AI.
- [Prompt Versioning](https://docs.athina.ai/guides/prompts/prompt-versioning.md): A guide on managing and tracking prompts with prompt versioning in Athina AI.
- [Integrate AWS Bedrock Models](https://docs.athina.ai/guides/datasets/aws-bedrock-model.md): Add and use custom Large Language Models in Athina.
- [Preparing Data for Fine-Tuning](https://docs.athina.ai/guides/datasets/preparing-data-for-fine-tuning.md): Step-by-Step Guide to Optimizing Your Dataset for Fine-Tuning Models in Athina.
- [Get Data from S3 Bucket](https://docs.athina.ai/guides/datasets/get-data-from-s3-bucket.md): Step-by-step guide on retrieving S3 data in to Athina.
- [Run an Experiment to compare and evaluate responses from different models](https://docs.athina.ai/guides/datasets/comparing-models.md)
- [Comparing datasets using Athina IDE](https://docs.athina.ai/guides/datasets/comparing-datasets.md)
- [Prototype and Evaluate a Prompt Chain](https://docs.athina.ai/guides/datasets/prototype-and-evaluate-prompt-chain.md): You can prototype and evaluate a prompt chain in Athina IDE
- [Run Prompts and Evaluate](https://docs.athina.ai/guides/datasets/run-prompts-and-evaluate.md)
- [Pairwise Evaluation](https://docs.athina.ai/guides/evals/pairwise-evals.md): Step by step pairwise evaluation guide to compare model outputs using Athina AI.
- [Evaluations in CI/CD Pipeline](https://docs.athina.ai/guides/evals/evals-in-cicd-pipeline.md): Automating Evaluations using Athina AI in CI/CD Pipelines.
- [Which evaluations to use for RAG applications?](https://docs.athina.ai/guides/evals/rag-eval-guide.md)
- [Prompt Injection: Attacks and Defenses](https://docs.athina.ai/guides/evals/prompt-injection.md)
- [Running evals as real-time guardrails](https://docs.athina.ai/guides/evals/running-evals-guardrails.md)
- [Different stages of evaluation](https://docs.athina.ai/guides/evals/llm-eval-workflows.md)
- [How can I improve the performance / reliability of my evals?](https://docs.athina.ai/guides/evals/improving-eval-performance.md)
- [How to Measure Retrieval Accuracy in RAG Applications Using Athina IDE](https://docs.athina.ai/guides/evals/measuring-retrieval-accuracy-in-rag.md)
- [Pairwise Evaluation](https://docs.athina.ai/guides/evals/pairwise-evaluation.md)
- [Evaluate Conversations](https://docs.athina.ai/guides/evals/evaluate-conversations.md): Step-by-step evaluation of a multi-turn conversation using Athina.
- [RAG Evaluators](https://docs.athina.ai/guides/evals/rag-evals.md)
- [Compare Multiple Models](https://docs.athina.ai/guides/experiments/compare-multiple-models.md): Step by step guide on how to compare multiple models using Athina AI.
- [Create and Share Flow](https://docs.athina.ai/guides/flows/create-and-share-flow.md): Learn how to create and share AI workflows using the Flow Builder in Athina AI.
- [Where is data stored?](https://docs.athina.ai/faqs/data-policy.md)
- [Q. Can Athina's observability platform be deployed on-prem?](https://docs.athina.ai/faqs/on-prem.md)
- [Q. Will the Athina logging SDK increase my latency?](https://docs.athina.ai/faqs/logging/logging-latency.md)
- [Is this SDK going to make a proxy request to OpenAI through Athina?](https://docs.athina.ai/faqs/logging/proxy.md)
- [Can I log inferences from any model?](https://docs.athina.ai/faqs/logging/can-i-log-using-any-model.md)
- [Grouping Inferences](https://docs.athina.ai/faqs/logging/how-can-i-log-conversations.md)
- [Why use LLM-as-a-judge for evaluations?](https://docs.athina.ai/faqs/evals/why-llm-judge.md)
- [Can I choose which model to use for running evaluations?](https://docs.athina.ai/faqs/evals/choosing-models.md)
- [How do you manage costs for LLM evaluation?](https://docs.athina.ai/faqs/evals/managing-costs.md)
- [Why Not Use Traditional Evaluation Metrics?](https://docs.athina.ai/faqs/evals/why-not-traditional-metrics.md)

## OpenAPI Specs

- [openapi](https://docs.athina.ai/api-reference/openapi.json)

## Optional

- [Open-Source Evals](https://github.com/athina-ai/athina-evals)
- [Blog](https://blog.athina.ai/)
- [Email us](mailto:hello@athina.ai)
- [Book a call](https://cal.com/shiv-athina/30min)