Getting Started

Running the HumanEval generation example

Quick Start

Prerequisites

• Python 3.10, 3.11 or 3.12

• pip

Installation

pip install celi-framework

Run a Simple Demo (no LLM-setup required):

  python -m celi_framework.main \
  --job-description=celi_framework.examples.human_eval.job_description.job_description \
  --tool-config='{"single_example":"HumanEval/3"}' \
  --simulate-live

This example simulates using CELI to solve problem #3 of the HumanEval benchmark programming problem set. It uses cached versions of the LLM outputs so it doesn’t require an API key or make any paid LLM calls on your behalf. The result will be put in the target/drafts directory.

Running this demo should take a few minutes. You will be able to see how CELI tackles the problem and the LLM calls it makes, along with the responses.

Explore the Results

Check the output in the target/drafts directory.

Run the Example Live

The code above uses a cached version of the LLM results. To meaningfully run CELI on anything new, you will need to make new LLM calls, which will require an OpenAI API key (or your own local LLM; see LLM Support).

  python -m celi_framework.main \
  --job-description=celi_framework.examples.human_eval.job_description.job_description \
  --openai-api-key=<Insert your OpenAI API key here> \

Again, you can check your output in the target/drafts directory.