Question 1

What is TryEval?

Accepted Answer

TryEval is a platform for evaluating AI models and endpoints. You bring your datasets and define what "good" looks like — we run evaluations using statistical metrics, LLM-as-a-Judge scoring, and optional human review, then give you scorecards with pass/fail results so you can ship with confidence.

Question 2

Can I try TryEval without signing up?

Accepted Answer

Yes. Try Free lets you run a full evaluation in your browser with no account required. Your dataset stays in your browser's memory, API calls go directly to your LLM provider, and your API key is never sent to TryEval. Nothing is stored after you close the tab.

Question 3

What's the difference between Try Free and a paid account?

Accepted Answer

Try Free is a one-off browser-based evaluation with preset rubrics only — results are not saved. When you sign up, you get a free account with 20 evaluation credits, the ability to save results, and access to features like custom rubrics, team collaboration, and data retention that scale with your plan. See Pricing for full details.

Question 4

How do I get started?

Accepted Answer

You can Try Free right now — no sign-up needed. Upload a CSV, pick a metric pack, connect your LLM API key, and run your evaluation. To save results and unlock more features, sign up for a free account.

Question 5

What are evaluation credits?

Accepted Answer

Credits are how we meter evaluations. Each evaluation row costs 1 credit, with the cost increasing based on token size for LLM-as-a-Judge runs. Every plan includes a credit allotment — from 20 one-time credits on the free plan to unlimited on Enterprise. See Pricing for plan-specific details.

Question 6

What's the difference between preset rubrics and custom rubrics?

Accepted Answer

Preset rubrics are built-in metric packs covering common evaluation needs — General Quality, RAG/Retrieval, Safety & Compliance, Conversational, and Content Generation. They're available on all plans including Try Free. Custom rubrics let you define your own criteria, scoring scales, and pass thresholds tailored to your use case. Custom rubrics are available on the Starter plan and above.

Question 7

What is LLM-as-a-Judge?

Accepted Answer

LLM-as-a-Judge is an evaluation method where a large language model acts as a judge for your AI's outputs. You define rubrics with scoring criteria, and the judge LLM evaluates each row of your dataset against those criteria — returning scores, pass/fail results, reasoning, and evidence for every metric. It's available on all plans.

Question 8

What are statistical metrics?

Accepted Answer

Statistical metrics are automated, reference-based measures that compare your AI's output against expected output. TryEval supports ROUGE, METEOR, BERTScore, BLEU, F1 Score, Exact Match, and Toxicity detection. They're available on all plans and don't require an LLM API key.

Question 9

What is Human-in-the-Loop (HITL)?

Accepted Answer

Human-in-the-Loop means evaluations where human annotators or reviewers score and verify AI outputs alongside automated metrics. It adds a layer of human judgement for nuanced assessments that automated methods might miss. HITL is available on the Growth plan and above.

Question 10

What are judge personas?

Accepted Answer

Judge personas are saved judge configurations — including model presets and evaluation parameters — for running consistent evaluations across different runs and team members. Judge personas are coming soon and will be available on the Growth plan and above. We'll announce when they're ready.

Question 11

What's included in the free plan?

Accepted Answer

The free plan includes 20 evaluation credits (one-time), statistical evaluations, LLM-as-a-Judge with preset rubrics, dataset-based evaluations, 7-day data retention, a 1 GB dataset size limit, and a single seat. See the full comparison on Pricing.

Question 12

How does billing work for paid plans?

Accepted Answer

Paid plans (Starter and Growth) are billed monthly with a set number of included credits. If you exceed your credit allotment, additional charges may apply depending on your plan. Enterprise plans have custom pricing. Full plan details and credit allocations are on Pricing.

Question 13

Can I upgrade or downgrade my plan?

Accepted Answer

Yes. You can change your plan from your account settings. Upgrades take effect immediately, and downgrades take effect at the start of your next billing cycle.

Question 14

What's your refund policy?

Accepted Answer

We handle refund requests on a case-by-case basis. For refunds, email support@tryeval.com. You can also review our Terms of Service for more details.

Question 15

Where is my data stored?

Accepted Answer

Your evaluation data is stored securely in the cloud. Data retention depends on your plan — 7 days on the free plan, 1 month on Starter, 6 months on Growth, and custom policies for Enterprise. On the Enterprise plan, you can control how data is managed and run deployments within your own infrastructure. See our Privacy Policy for details.

Question 16

Do you train on my data?

Accepted Answer

No. We do not use your content or evaluation data to train our or any third-party AI models. Your data is used solely to provide the evaluation service you've requested. This commitment is detailed in our Privacy Policy.

Question 17

Is my data safe during Try Free?

Accepted Answer

Yes. During Try Free, your dataset stays entirely in your browser's memory. API calls go directly from your browser to your LLM provider — they never pass through TryEval servers. Your API key is never stored or transmitted to us. All results are computed locally and nothing persists after you close the tab.

Question 18

How do I get help?

Accepted Answer

Email us at contact@tryeval.com — we're happy to help with setup, troubleshooting, or any questions about the platform. You can also explore our Features page for a detailed overview of what TryEval offers, or check Pricing for plan comparisons.

Frequently Asked Questions

Getting Started

Evaluation & Features

Plans & Billing

Security & Data

Support

Still have questions?