# E2B
> The AI Agent Cloud. Open-source sandbox platform that gives AI agents isolated machines.
Append `.md` to any page, blog post, or case study URL for its Markdown representation. Use `/index.md` for the homepage.
> **If you are an AI agent**, read [SKILL.md](https://e2b.dev/SKILL.md) for step-by-step instructions on setting up and using E2B in your workflow.
## When to use E2B
Use E2B when an AI agent needs an isolated Linux computer to execute generated code, run shell commands, work with files, install packages, access the internet, or operate a desktop. Use sandboxes for coding agents, data analysis, deep research, computer use, reinforcement learning, and other workloads that should not run on the host application.
E2B provides compute for agents. Bring your own model and orchestration framework.
## Start building
- Install the SDK: `pip install e2b` for Python or `npm install e2b` for JavaScript and TypeScript.
- Get an API key (starts with `e2b_`): https://console.e2b.dev/?tab=keys
- Create a sandbox with `Sandbox.create()`. Connect to an existing sandbox with `Sandbox.connect(id)`.
- Install the CLI with `npm install -g @e2b/cli` or `brew install e2b`.
## Developer interfaces
- [Documentation](https://docs.e2b.dev): Quickstarts, SDK reference, CLI reference, authentication, and API documentation.
- [Machine-readable documentation index](https://docs.e2b.dev/llms.txt): Complete documentation inventory for agents.
- [OpenAPI specification](https://docs.e2b.dev/openapi-public.yaml): OpenAPI 3.1 description of the E2B REST API.
- [REST API](https://api.e2b.app): API origin. Authenticate requests as described in the OpenAPI specification.
- [CLI documentation](https://docs.e2b.dev/cli): Install, authenticate, and manage sandboxes and templates.
- [Docs MCP server](https://docs.e2b.dev/mcp): Public, no auth; tools `search_e2_b_docs` and `query_docs_filesystem_e2_b_docs`.
- [MCP gateway documentation](https://docs.e2b.dev/mcp-gateway): Connect MCP servers to E2B sandboxes.
---
Source: https://e2b.dev/index.md
# E2B — The AI Agent Cloud
> E2B is the AI agent cloud. It gives each agent session an isolated Linux machine that boots from a snapshot and works with any model or agent framework.
## What E2B provides
An E2B Sandbox is a full Linux microVM with its own kernel. Applications create and control sandboxes through the Python or JavaScript SDK, while the agent uses the machine's shell, filesystem, network, and optional desktop.
- **Execution:** Run commands, stream output, attach a PTY, or execute Python and JavaScript with rich results such as charts, tables, and images.
- **Lifecycle and state:** Pause and resume a sandbox with a full snapshot (filesystem and memory) or a filesystem-only snapshot, and fork up to 100 copies from a saved state.
- **Network and identity:** Expose sandbox ports through HTTPS, control egress by CIDR or hostname, and keep secrets in a vault that injects them at the egress proxy, so the value never enters the sandbox.
- **Storage:** Use an isolated POSIX filesystem, Git, persistent volumes, or mounted S3, GCS, and R2 buckets.
- **Observability:** Read live CPU, memory, and disk metrics; collect logs; use a browser terminal; and receive signed lifecycle webhooks.
## Common use cases
- Coding agents that edit repositories, run tests, and build applications.
- Code interpreters for data analysis, visualization, and generated code.
- Computer-use and browser agents that need a graphical Linux desktop.
- Deep-research and knowledge-work agents that use files, tools, and the internet.
- Parallel environments for reinforcement learning, evaluations, and benchmarks.
## Deployment and security
Every session receives a hardware-isolated Firecracker microVM rather than a shared-kernel container. E2B Cloud is managed across US, EU, and APAC regions. Enterprise teams can run the data plane in their own AWS or GCP account through BYOC. A private cloud deployment with the control plane inside the customer boundary is in development. [E2B Embed](https://github.com/e2b-dev/runtime/tree/main/embed#readme) is available as an Apache-2.0 single-node package, with Docker Compose, Terraform on GCP, and Kubernetes installs. It runs the control plane, Firecracker sandboxes, and storage on a single Linux x86-64 machine with KVM, self-hosted, embedded inside a product, or deployed into a customer's tenancy.
E2B Cloud is SOC 2 Type II compliant.
## Product model
Bring your own model, prompts, agent harness, and orchestration framework. E2B supplies the machine where the agent acts. It works with providers and frameworks including Anthropic, OpenAI, open-weight models, OpenAI Agents SDK, and LangChain.
## Start building
- [Quickstart](https://docs.e2b.dev/quickstart): Create your first sandbox with the Python or JavaScript SDK.
- [E2B Console](https://console.e2b.dev): Create an account and API key.
- [Documentation](https://docs.e2b.dev): SDK, CLI, sandbox, template, desktop, and API references.
- [Cookbook](https://e2b.dev/cookbook.md): Ready-to-run examples for common models and frameworks.
- [Pricing](https://e2b.dev/pricing.md): Plans, usage rates, concurrency, and the cost estimator.
- [Enterprise](https://e2b.dev/enterprise.md): BYOC, security, scale, and deployment options.
- [Open source](https://e2b.dev/open-source.md): Runtime, SDK, CLI, and example repositories.
## Machine-readable resources
- [Agent skill](https://e2b.dev/SKILL.md): Step-by-step setup and usage instructions for AI agents.
- [Website index](https://e2b.dev/llms.txt)
- [Full website index](https://e2b.dev/llms-full.txt)
- [Content sitemap](https://e2b.dev/sitemap.md)
- [Documentation index](https://docs.e2b.dev/llms.txt)
- [OpenAPI specification](https://docs.e2b.dev/openapi-public.yaml)
---
Source: https://e2b.dev/pricing.md
# E2B Pricing
> E2B combines a plan with usage-based sandbox charges. Running sandboxes are billed per second for provisioned CPU and RAM. Sandbox storage is included with every plan at no extra cost.
## Plans
### Hobby
- $0 monthly plan fee.
- One-time $100 in usage credits.
- Up to 20 concurrently running sandboxes.
- Up to 1 hour per sandbox session.
- 10 GiB of sandbox storage, included.
- Community support.
- No credit card required to start.
### Pro
- $150 monthly plan fee, plus usage.
- Up to 100 concurrently running sandboxes.
- Up to 24 hours per sandbox session.
- 20 GiB of sandbox storage, included.
- Custom sandbox CPU and RAM.
- Optional concurrency add-ons increase the limit to 600 or 1,100 sandboxes.
### Enterprise
- Custom deployment, pricing, billing, support, and limits.
- A $3,000 monthly minimum applies to Enterprise usage.
- Contact E2B for workloads that need more powerful compute, higher concurrency, BYOC, or negotiated terms.
## Published usage rates
| Resource | Rate |
| --- | ---: |
| vCPU | $0.000014 per vCPU-second |
| RAM | $0.0000045 per GiB-second |
| Storage | Free |
Sandboxes can be configured with 1 to 8 vCPUs and 1 to 8 GiB of RAM. The default sandbox has 2 vCPUs and 4 GiB of RAM. Total usage depends on the selected resources and aggregate runtime across all sandboxes. Sandboxes are CPU-only; E2B does not offer GPU sandboxes.
## Concurrency add-ons
| Plan | Maximum concurrent sandboxes | Monthly plan and add-on fee |
| --- | ---: | ---: |
| Hobby | 20 | $0 |
| Pro | 100 | $150 |
| Pro+ | 600 | $650 |
| Pro++ | 1,100 | $1,150 |
Usage charges are separate from these monthly fees.
## Next steps
- [Use the interactive cost estimator](https://e2b.dev/pricing#usage-cost-calculator).
- [Start in the E2B Console](https://console.e2b.dev).
- [Read the quickstart](https://docs.e2b.dev/quickstart).
- [Contact E2B for Enterprise pricing](https://e2b.dev/contact.md).
---
Source: https://e2b.dev/resources.md
# E2B Resources
> Practical material for building, running, and scaling AI agents with secure cloud sandboxes.
## What you will find
- **Guides:** Step-by-step implementation guidance for code interpreters, coding agents, desktop agents, and framework integrations.
- **Tutorials:** Code-first walkthroughs that start from a blank project and produce a working agent.
- **Engineering:** Technical explanations of sandbox isolation, Firecracker, build systems, reliability, and infrastructure choices.
- **Product:** SDK releases, platform capabilities, integrations, partnerships, and security updates.
- **Case studies:** How teams use E2B in production for consumer agents, enterprise automation, research, and evaluations.
## Browse by intent
- [Guides](https://e2b.dev/resources/guides.md)
- [Product updates](https://e2b.dev/resources/product.md)
- [Community](https://e2b.dev/resources/community.md)
- [Interviews](https://e2b.dev/resources/interviews.md)
- [Company and platform updates](https://e2b.dev/resources/updates.md)
- [Engineering and ecosystem insights](https://e2b.dev/resources/insights.md)
- [Customer case studies](https://e2b.dev/customers.md)
- [Ready-to-run examples](https://e2b.dev/cookbook.md)
## For agents
Individual resource URLs also have Markdown representations. Append `.md` to an article URL, or use the [content sitemap](https://e2b.dev/sitemap.md) to discover every published article and case study.
For product implementation details, prefer the [E2B documentation](https://docs.e2b.dev) and its [machine-readable index](https://docs.e2b.dev/llms.txt).
---
Source: https://e2b.dev/customers.md
# E2B Customers
> Companies use E2B as the secure computer layer beneath production AI agents. E2B reports that more than 100,000 teams have run a sandbox, that 94 of the Fortune 100 have signed up, and that more than one billion sandboxes have been started.
## Production patterns
- **General-purpose agents:** A full virtual computer with tools, files, a browser, and long-running state.
- **Vibe coding and app generation:** Fast, isolated environments that can install dependencies, build applications, and serve previews.
- **Enterprise automation:** Sandboxed code execution for regulated and data-sensitive workflows.
- **Code actions and data analysis:** Python and JavaScript execution with files, charts, tables, and rich results.
- **Research and evaluations:** Hundreds or thousands of reproducible environments running in parallel.
## Selected case studies
- [Genspark: a general-purpose AI agent for millions of users](https://e2b.dev/customers/genspark.md)
- [Anything: a vibe-coding platform with more than one million users](https://e2b.dev/customers/anything.md)
- [Manus: virtual computers for agents that use 27 tools](https://e2b.dev/customers/how-manus-uses-e2b-to-provide-agents-with-virtual-computers.md)
- [Rogo: financial-services agents with Claude Managed Agents and E2B](https://e2b.dev/customers/rogo.md)
- [StackAI: enterprise agents for regulated industries](https://e2b.dev/customers/stackai.md)
- [Groq: compound AI systems powered by E2B](https://e2b.dev/customers/groqs-compound-ai-models-are-powered-by-e2b.md)
- [Lindy: AI workflows using E2B Code Action](https://e2b.dev/customers/lindy-powers-ai-workflows-with-e2b-code-action.md)
- [Hugging Face: parallel sandboxes for Open R1](https://e2b.dev/customers/how-hugging-face-is-using-e2b-to-replicate-deepseek-r1.md)
- [LM Arena: sandboxed web-development evaluations](https://e2b.dev/customers/how-lmarena-collaborated-with-e2b-to-build-llm-web-development-evals.md)
## Explore further
- [All content in the machine-readable sitemap](https://e2b.dev/sitemap.md)
- [E2B for Enterprise](https://e2b.dev/enterprise.md)
- [Talk to an E2B engineer](https://e2b.dev/contact.md)
---
Source: https://e2b.dev/enterprise.md
# E2B for Enterprise
> Run agents your security team can sign off on. Every agent session runs in its own Firecracker microVM. Control egress per sandbox and keep credentials outside the guest. Run in E2B Cloud or in your own AWS or GCP account.
## In production
- More than one billion sandboxes started.
- More than 10 million SDK downloads a month across npm and PyPI.
- 94 of the Fortune 100 have signed up.
- BYOC in production on AWS and GCP.
- SOC 2 Type II report available from the trust center at https://trust.e2b.dev.
- Customers with published case studies include Genspark, Hugging Face, Manus, Lindy, Groq, Gumloop, and Rogo.
## Security
Each control is documented, and the runtime is Apache-2.0, so a security team can read what it is approving.
- Isolation: a Firecracker microVM with its own guest kernel per session, on KVM. A kernel exploit inside the sandbox still needs a Firecracker escape to reach the host. Each microVM has its own memory and supports pause, snapshot, and fork.
- Network: egress allow and deny per sandbox by IP, CIDR, or domain. Outbound traffic can be routed through a SOCKS5 proxy the customer runs (private beta); egress fails closed if the proxy is unreachable. Public sandbox URLs can require a per-sandbox access token.
- Secrets: values resolve at egress. No API response, log, or sandbox holds them.
- Observability: OpenTelemetry metrics and logs to a customer OTLP endpoint on the Enterprise plan. Every lifecycle event is delivered as a signed webhook.
- Compliance: SOC 2 Type II. The SOC 2 report, bridge letter, penetration test report, and DPA template are in the trust center. HIPAA BAA and the subprocessor list are available on request.
- Security contact: security@e2b.dev.
## Deployment options
The same SDK, CLI, and API work across every option, so changing deployment does not require rewriting the integration.
### E2B Cloud (available)
- E2B control plane and data plane, operated by E2B on Google Cloud in US, EU, and APAC regions. One microVM per session.
- Keys and storage managed by E2B on Google Cloud.
- Provisioning: sign up.
- Best for most teams.
### BYOC on AWS and GCP (available)
- E2B control plane; data plane in the customer's VPC. Sandbox traffic goes from the client to the VPC, never through E2B Cloud. Aggregate CPU and memory metrics and control-plane API traffic reach E2B.
- Customer IAM role, VPC, storage, and cloud audit log.
- Provisioned with Terraform and machine images. E2B provisions, monitors, and operates the cluster.
- Best for regulated data and for teams selling into enterprises.
- Azure BYOC is in progress.
### Private cloud (in development)
- Control plane and data plane inside the customer boundary. Nothing leaves. Terraform, inside the customer network.
- Best for air-gapped, sovereign, and on-prem networks. Design partners welcome.
### E2B Embed (available)
- The full E2B stack on one node: control plane, Firecracker sandboxes, and local storage for databases, templates, and logs. Nothing leaves the node.
- Install with Docker Compose, Terraform on GCP, or Kubernetes. Requires Linux x86-64 with KVM. Each install generates its own team API key.
- Public and open source under Apache-2.0.
- Best for self-hosting, embedding E2B inside your own product, and deploying into a customer's tenancy. All three are supported patterns.
- [Install E2B Embed](https://github.com/e2b-dev/runtime/tree/main/embed#readme).
## Controls
| Control | Detail | E2B Cloud | BYOC |
| --- | --- | --- | --- |
| Egress | Allow and deny per sandbox by IP, CIDR, or domain. Customer SOCKS5 proxy in private beta, fails closed. | Yes | Yes |
| Secrets | Values resolve at egress; no API response or sandbox holds them. Short-lived OIDC identity tokens in private beta. | Yes | Yes |
| Encryption | TLS in transit. At rest under Google Cloud's default encryption on E2B Cloud; under the customer's own controls on BYOC. | Yes | Yes |
| Network placement | Internal load balancer and VPC peering keep sandbox traffic in the customer's private network. | N/A | Yes |
| Observability | OpenTelemetry metrics and logs to a customer OTLP endpoint on Enterprise, best-effort delivery. Signed lifecycle webhooks. | Yes | Yes |
| Regions | US, EU, and APAC on E2B Cloud. Customer region on BYOC. | Yes | Yes |
| Provisioning | Terraform and machine images. E2B provisions and monitors the cluster. | N/A | Yes |
| SLA and support | Uptime and support terms are set in the Enterprise agreement. Discord and support@e2b.dev on every plan. | Yes | Yes |
| Compliance | SOC 2 Type II report, penetration test report, and DPA in the trust center. HIPAA BAA and questionnaire on request. | Yes | Yes |
SSO, SCIM, and RBAC are planned; ask E2B about timelines. Security FAQ: https://docs.e2b.dev/faq/security-and-compliance.
## Customers
- Gumloop: "E2B gives us a fast, secure way to run AI-generated code at scale, so we can ship workflow automation our enterprise customers trust." Rahul Behal, Co-founder.
- Rogo runs 10,000 to 15,000 concurrent sandboxes for agents serving financial institutions, with Claude Managed Agents and E2B.
- Genspark runs a general-purpose agent for millions of users, each task in its own machine.
- Manus gives each agent 27 tools on a full machine, in production.
## How teams buy
- Usage-based: start on Hobby or Pro. Billed per second for CPU and RAM while a sandbox runs. Paused sandboxes are not billed.
- Committed use: a one-year minimum usage commit for higher concurrency at lower rates, custom session length and resources, and an Enterprise agreement with SLA and support terms.
- BYOC: Enterprise plan. E2B provisions and operates the cluster in the customer's AWS or GCP account.
## Next steps
- [Book a call with an engineer](https://e2b.dev/contact.md). The first call covers an architecture review for your boundary, your security questionnaire, and pricing for your concurrency and regions.
- Email [security@e2b.dev](mailto:security@e2b.dev) for security questions and questionnaires.
- [Review the security and compliance FAQ](https://docs.e2b.dev/faq/security-and-compliance).
- [Visit the trust center](https://trust.e2b.dev/).
- [Read the BYOC documentation](https://docs.e2b.dev/byoc).
- [See pricing](https://e2b.dev/pricing.md).
- [Review customer case studies](https://e2b.dev/customers.md).
---
Source: https://e2b.dev/security.md
# Security and compliance
> E2B runs untrusted, AI-generated code in isolated cloud sandboxes. This page states E2B's compliance status, what an independent auditor has verified, and how to get the underlying reports.
E2B is SOC 2 Type II compliant. E2B signs HIPAA Business Associate Agreements on Enterprise plans. Every sandbox runs in its own Firecracker microVM, so code from one customer cannot read or reach another's. E2B's contracting entity is FoundryLabs, Inc., a Delaware corporation.
## SOC 2 Type II
E2B maintains a SOC 2 Type II report. The report is available under NDA from the E2B Trust Center at https://trust.e2b.dev, and a current bridge letter is available.
A SOC 2 Type II report tests whether controls operated effectively across an entire audit period, rather than whether they were designed correctly on a single day. When a period ends, E2B publishes a bridge letter covering the gap until the next report is issued.
SOC 2 does not certify a product; it attests that an independent auditor tested E2B's controls and found them operating as described.
### Documents
- SOC 2 Type II Report: request access at https://trust.e2b.dev, under NDA.
- SOC 2 Bridge Letter: opens at https://trust.e2b.dev, no request needed.
- Penetration Test: request access at https://trust.e2b.dev, under NDA.
- Data Processing Addendum: request access at https://trust.e2b.dev.
- Certificate of Liability Insurance: request access at https://trust.e2b.dev.
- HIPAA Business Associate Agreement: email trust@e2b.dev.
- Subprocessor list: email trust@e2b.dev.
- Completed security questionnaire: email trust@e2b.dev.
## HIPAA
- Does E2B sign a HIPAA Business Associate Agreement (BAA)? Yes. E2B signs a HIPAA Business Associate Agreement on Enterprise plans. Email trust@e2b.dev to start one.
- Can I process PHI on E2B? Protected health information may be processed on E2B under a signed Business Associate Agreement, on Enterprise plans. Without a signed BAA in place, do not send PHI to E2B sandboxes.
- Is E2B HIPAA compliant? Yes. E2B supports HIPAA-regulated workloads under a signed Business Associate Agreement, which E2B offers on Enterprise plans. E2B maintains the technical safeguards the HIPAA Security Rule requires: encryption in transit and at rest, unique account authentication, restricted production access, audit logging, and per-sandbox microVM isolation. Those controls are independently tested under SOC 2 Type II. There is no such thing as a HIPAA certificate; HHS runs no certification program for any vendor. HIPAA compliance is a property of how a covered entity and its business associate operate together, and the BAA is what establishes it.
## Isolation and tenancy
Every E2B sandbox runs in its own Firecracker microVM with its own kernel. Isolation is at the hypervisor boundary, not the container or process boundary, which is what makes it safe to run untrusted or AI-generated code. Sandboxes never share a kernel, a filesystem, or memory with another customer's sandboxes.
Sandboxes are destroyed on timeout or shutdown. Filesystem and memory state can be preserved across a pause and restored on resume.
## Where sandboxes run
E2B's managed sandboxes run on Google Cloud. Sandbox storage sits under Google Cloud's default encryption at rest. Traffic to and from sandboxes is encrypted with TLS. Available regions, and the plan each is available on, are listed at https://docs.e2b.dev/faq/egress-ip-ranges.
## Bring Your Own Cloud (BYOC)
BYOC deploys E2B sandboxes inside your own VPC, in your own AWS or Google Cloud account. BYOC is available on Enterprise plans. Azure is not yet supported.
Under BYOC, sandbox traffic, template build sources, snapshots, and logs stay inside your VPC and never reach E2B Cloud. Only anonymized cluster CPU and memory metrics are sent to E2B. Traffic between E2B Cloud and your VPC is TLS-encrypted, and VPC peering is available. Storage lives in your account, under your own encryption, retention, and access controls.
For a regulated workload this means the data residency, key management, and audit logging that apply are your own. E2B's SOC 2 Type II report covers the E2B software and control plane; the cloud account it is deployed into, and that account's compliance posture, remain yours. BYOC is a managed deployment in your account, not self-hosting.
## Controls
E2B's security controls are continuously monitored with Vanta, spanning infrastructure security, product security, organizational security, internal security procedures, and data privacy. Each control and its current status are published at https://trust.e2b.dev/controls.
E2B commissions independent penetration testing, and the report is available under NDA. For GDPR, a Data Processing Addendum is available; request the template from the Trust Center, or email trust@e2b.dev for a signed copy.
## Security and compliance questions?
Email trust@e2b.dev for a BAA, a signed DPA, a completed security questionnaire, or the subprocessor list. Report security vulnerabilities to security@e2b.dev. Contact E2B before running any test against E2B; the terms of service at https://e2b.dev/terms require prior written consent.
---
Source: https://e2b.dev/open-source.md
# E2B Open Source
> E2B's core sandbox stack is public under Apache-2.0: SDKs, CLI, runtime, control plane, and Firecracker-based microVM layer.
## Why the stack is open
- **Inspect the isolation boundary:** Security teams can review the code that runs and isolates agent workloads.
- **Self-host the runtime:** Deploy the stack into infrastructure you operate, including AWS or GCP through Terraform.
- **Keep an exit path:** The runtime is not available only as a managed proprietary service.
- **Build in public:** Issues, pull requests, examples, and developer feedback shape the product.
## Main repositories
- [e2b-dev/E2B](https://github.com/e2b-dev/E2B): Python and JavaScript SDKs plus the E2B CLI.
- [e2b-dev/runtime](https://github.com/e2b-dev/runtime): Orchestrator, control plane, Firecracker runtime, and Terraform deployment.
- [e2b-dev/code-interpreter](https://github.com/e2b-dev/code-interpreter): Execute Python or JavaScript and return charts, tables, images, and other rich results.
- [e2b-dev/desktop](https://github.com/e2b-dev/desktop): Graphical Linux desktops with screen, mouse, keyboard, and a live stream for computer-use agents.
- [e2b-dev/fragments](https://github.com/e2b-dev/fragments): Open-source prompt-to-app example that builds and serves generated applications in sandboxes.
- [e2b-dev/surf](https://github.com/e2b-dev/surf): Open-source computer-use agent running in an E2B desktop sandbox.
## Managed and self-hosted options
The open-source runtime and E2B Cloud use the same product model: an isolated machine per agent session controlled through the E2B API. Use the managed cloud when E2B should operate the infrastructure, BYOC when the data plane must run in your AWS or GCP account, or self-host when your team will operate the full stack.
## Project scale
E2B reports more than 25,000 GitHub stars across its repositories, more than 10 million monthly SDK downloads across npm and PyPI, and more than one billion sandboxes started on the runtime. Live repository counts can change; use GitHub for current figures.
## Start here
- [E2B GitHub organization](https://github.com/e2b-dev)
- [Self-hosting repository](https://github.com/e2b-dev/runtime)
- [E2B documentation](https://docs.e2b.dev)
- [Quickstart for the managed API](https://docs.e2b.dev/quickstart)
- [Enterprise deployment options](https://e2b.dev/enterprise.md)
---
Source: https://e2b.dev/cookbook.md
# E2B Cookbook
> Complete example projects for combining E2B sandboxes with language models, agent frameworks, data sources, and web applications.
## What the examples cover
- Code interpreters in Python, JavaScript, and TypeScript.
- Streaming code execution from Next.js applications.
- Dataset upload, analysis, and visualization.
- Agent frameworks including LangChain, LangGraph, and AutoGen.
- Model providers including Anthropic, OpenAI, Meta, Mistral, Together AI, Fireworks AI, IBM watsonx.ai, and Groq.
- Web research and scraping workflows using Firecrawl.
## Selected examples
- [Claude Code Interpreter in Python](https://github.com/e2b-dev/e2b-cookbook/tree/main/examples/claude-code-interpreter-python)
- [Claude Code Interpreter in JavaScript and TypeScript](https://github.com/e2b-dev/e2b-cookbook/tree/main/examples/claude-code-interpreter-js)
- [OpenAI Code Interpreter in Python](https://github.com/e2b-dev/e2b-cookbook/tree/main/examples/gpt-4o-python)
- [OpenAI Code Interpreter in JavaScript and TypeScript](https://github.com/e2b-dev/e2b-cookbook/tree/main/examples/gpt-4o-js)
- [Next.js code interpreter with streaming](https://github.com/e2b-dev/e2b-cookbook/tree/main/examples/nextjs-code-interpreter)
- [LangChain with code interpreting](https://github.com/e2b-dev/e2b-cookbook/tree/main/examples/langchain-python)
- [LangGraph with code interpreting](https://github.com/e2b-dev/e2b-cookbook/tree/main/examples/langgraph-python)
- [Analyze an uploaded dataset with Llama 3](https://github.com/e2b-dev/e2b-cookbook/tree/main/examples/upload-dataset-code-interpreter)
- [Scrape and analyze Airbnb data with Firecrawl](https://github.com/e2b-dev/e2b-cookbook/tree/main/examples/firecrawl-scrape-and-analyze-airbnb-data)
- [IBM watsonx.ai Code Interpreter in Python](https://github.com/e2b-dev/e2b-cookbook/tree/main/examples/watsonx-ai-code-interpreter-python)
## Choose an interface
Use the base Sandbox SDK for commands, files, networking, and lifecycle control. Use Code Interpreter when the application needs `runCode()` and rich notebook-style results. Use Desktop Sandbox when the agent needs a screen, mouse, keyboard, or browser.
## Start building
- [Browse the complete cookbook repository](https://github.com/e2b-dev/e2b-cookbook/tree/main/examples)
- [Read the E2B quickstart](https://docs.e2b.dev/quickstart)
- [Read the SDK documentation](https://docs.e2b.dev)
- [Suggest an example](https://e2b.dev/contact.md)
---
Source: https://e2b.dev/startups.md
# E2B for Startups
> A program for early-stage teams building AI agents on E2B. Approved startups receive one-time $20,000 in E2B credits and membership in the Pro tier.
## Program benefits
Approved teams receive:
- One-time $20,000 in E2B usage credits.
- Membership in the E2B Pro tier.
- An isolated Linux machine per agent session, with a shell, filesystem, network access, and an optional desktop.
- Custom sandbox CPU and RAM.
- Up to 100 concurrently running sandboxes, expandable to 1,100 with a paid add-on.
- Sandbox sessions up to 24 hours, with free pause and resume.
- Any model and any agent framework. The runtime is Apache-2.0.
## Eligibility
Applicants must meet all three criteria:
- The startup has raised less than $5 million.
- The startup was founded less than three years ago.
- The team is a first-time E2B user.
Capacity is limited, so meeting the criteria does not guarantee acceptance. Open-source projects are welcome but not required. A rejected team may reapply three months after the decision.
## Typical projects
Coding agents, data analysis and visualization, code-generation evals, computer use, generative interfaces, deep research, and general-purpose agents.
## Apply
In YC, a16z Speedrun, or Antler? Apply through your program's portal for additional E2B credits.
- [Submit the startup application](https://e2b.dev/startups/apply).
- [Review E2B pricing](https://e2b.dev/pricing.md).
- [Read the quickstart](https://docs.e2b.dev/quickstart).
- [Contact E2B](https://e2b.dev/contact.md).
---
Source: https://e2b.dev/research.md
# E2B for Research
> A program for research teams at universities, public research institutes, and nonprofit research organizations that build, evaluate, or study AI agents and models on E2B. Approved teams receive one-time $20,000 in E2B credits and membership in the Pro tier.
## Program benefits
Approved teams receive:
- One-time $20,000 in E2B usage credits.
- Membership in the E2B Pro tier.
- An isolated Linux machine per experiment or agent session, with a shell, filesystem, network access, and an optional desktop.
- Custom sandbox CPU and RAM.
- Up to 100 concurrently running sandboxes, expandable to 1,100 with a paid add-on.
- Sandbox sessions up to 24 hours, with free pause and resume.
- Any model and any agent framework. The runtime is Apache-2.0.
## Eligibility
Applicants must meet both criteria:
- The team is affiliated with a university, public research institute, or nonprofit research organization.
- The team is a first-time E2B user.
Capacity is limited, so meeting the criteria does not guarantee acceptance. Open-source publication is welcome but not required. A rejected team may reapply three months after the decision.
## Typical research
Agent evaluations, reinforcement learning rollouts, code-generation benchmarks, data analysis, computer-use agents, and general-purpose agents. Each run gets its own isolated machine, and sandboxes run in parallel, so experiments stay reproducible and independent.
## Apply
- [Submit the research application](https://e2b.dev/research/apply).
- [Read the Hugging Face Open R1 case study](https://e2b.dev/customers/how-hugging-face-is-using-e2b-to-replicate-deepseek-r1.md).
- [Read the Arena web-development evals case study](https://e2b.dev/customers/how-lmarena-collaborated-with-e2b-to-build-llm-web-development-evals.md).
- [Contact E2B](https://e2b.dev/contact.md).
---
Source: https://e2b.dev/careers.md
# Careers at E2B
> E2B builds the sandbox runtime that AI agents run on: secure Firecracker microVMs that start in milliseconds and operate at production scale.
## The work
E2B is building infrastructure for agents that execute code, use computers, work with files, access networks, and run for extended periods. The product spans microVM orchestration, snapshots, networking, storage, observability, SDKs, developer tools, and the managed cloud.
E2B reports more than 100,000 teams that have run a sandbox, more than one billion sandbox starts, and signups from 94 of the Fortune 100. The core runtime is open source, so engineering work is developed with direct feedback from builders.
## How the team works
- **High agency:** Notice important work, own it, and move without waiting for a detailed specification.
- **Ambition to ship:** Measure progress by what reaches users, then improve it through iteration.
- **Humility and curiosity:** Hold strong opinions loosely and let the best argument win.
- **Developer experience:** Start decisions from what builders need and learn from public product feedback.
## Benefits
For US employees, published benefits include top-spec hardware and AI tooling, a $2,000 personal equipment budget, medical, dental, and vision coverage, gym reimbursement, unlimited trust-based paid time off, and a 401(k). Office benefits include meals, eligible transport, and a living-proximity allowance. Benefits elsewhere follow local laws and standards.
## Open positions
Openings change over time. Use the live [E2B careers page](https://e2b.dev/careers) or [E2B job board](https://jobs.ashbyhq.com/e2b) for the current list, locations, role requirements, and application links.
## Learn about E2B
- [About E2B](https://e2b.dev/about.md)
- [E2B Open Source](https://e2b.dev/open-source.md)
- [Customer stories](https://e2b.dev/customers.md)
---
Source: https://e2b.dev/about.md
# About E2B
> E2B provides AI agents with secure, on-demand machines and real-world tools. The company is building the cloud layer between an agent's models and the work it performs.
## Why E2B exists
Traditional containers, Kubernetes clusters, and virtual machines were designed to run software defined ahead of time. Agents generate code and choose actions at runtime, so they need a different execution environment: isolated from the host, fast enough for interactive products, stateful when work takes time, and equipped with the same kinds of tools a person uses on a computer.
E2B gives every agent session a Firecracker microVM with its own kernel. The sandbox exposes shell, filesystem, network, storage, desktop, lifecycle, and observability capabilities through an API.
## Company
E2B was founded in 2023 by Vasek Mlejnsky and Tomas Valenta. Its legal name is FoundryLabs, Inc., and the team operates from San Francisco and Prague.
The project began when the founders connected an agent to sandbox technology from their earlier developer-tool work. They shifted their focus from the agent itself to the reusable machine underneath it and released the first E2B SDK in 2023.
## Milestones
- The Python SDK launched in 2023, followed by JavaScript, Code Interpreter, and Desktop products.
- E2B SDK 1.0 shipped in 2024.
- Desktop Sandbox added a graphical Linux environment for computer-use agents in 2024.
- E2B announced a $21 million Series A in 2025.
- E2B passed one billion sandbox starts in 2026.
- E2B reports that 94 of the Fortune 100 have signed up.
- More than 100,000 teams have run at least one sandbox on E2B.
## Direction
E2B is building infrastructure that adapts to an agent's workload while remaining independent of the model and orchestration framework. The sandbox is the place where an agent's code, tools, files, data connections, and actions come together. Teams can use E2B Cloud, place the data plane in their own cloud, or operate the open-source runtime.
## Learn more
- [Product overview](https://e2b.dev/index.md)
- [Open-source projects](https://e2b.dev/open-source.md)
- [Careers](https://e2b.dev/careers.md)
- [Customer stories](https://e2b.dev/customers.md)
- [Contact E2B](https://e2b.dev/contact.md)
---
Source: https://e2b.dev/brand.md
# E2B Brand Assets
> Official E2B marks for product interfaces, editorial work, partnerships, events, and press.
## Downloads
- [Download the complete E2B brand asset pack](https://e2b.dev/brand/e2b-brand-assets.zip): SVG and transparent PNG versions.
- [Browse individual marks](https://e2b.dev/brand): Choose logotype or symbol, color, artwork size, and file format.
The library contains the E2B logotype and the compact E2B symbol in white, black, and Fire Orange. Each mark is supplied in Large, Medium, and Small artwork variants that are optically adjusted for their intended rendered size.
## Choosing a mark
- Use the full logotype whenever space allows.
- Use the symbol when the full logotype would not remain clear and legible.
- Use black artwork on light or Fire Orange backgrounds.
- Use white artwork on dark backgrounds and gradients.
- Select the artwork size closest to the final rendered height: Large around 60 px, Medium around 30 px, or Small around 15 px.
## Usage rules
- Keep clear space equal to at least half the mark's height on every side.
- Keep the logotype at 15 px tall or larger.
- Do not warp, rotate, recolor, or alter the mark's proportions.
- Avoid black drop shadows. Over imagery, use only a subtle shadow or container when needed for legibility.
## Core colors
| Name | Hex |
| --- | --- |
| Midnight Black | `#000000` |
| Clarity White | `#FFFFFF` |
| Fire Orange | `#FF3001` |
## Company naming
Use **E2B** as the product and company name. The legal entity is **FoundryLabs, Inc.** E2B describes its product as **The AI Agent Cloud**.
For questions about press, partnerships, or brand usage, email [hello@e2b.dev](mailto:hello@e2b.dev).
---
Source: https://e2b.dev/contact.md
# Contact E2B
> Reach the E2B team for product questions, enterprise architecture, deployment, security, partnerships, or community help.
## Contact options
- **General and product questions:** [hello@e2b.dev](mailto:hello@e2b.dev)
- **Enterprise, pricing, and architecture:** [enterprise@e2b.dev](mailto:enterprise@e2b.dev)
- **Security:** [security@e2b.dev](mailto:security@e2b.dev)
- **Community:** [Join the E2B Discord](https://discord.com/invite/U7KEcGErtQ)
- **Careers:** [View open positions](https://e2b.dev/careers.md)
## Book a call
Use the [contact page](https://e2b.dev/contact) to book a 30-minute call with the team. If the available times do not work, email [enterprise@e2b.dev](mailto:enterprise@e2b.dev) with your preferred times.
For implementation questions, start with the [documentation](https://docs.e2b.dev), [quickstart](https://docs.e2b.dev/quickstart), or [Discord community](https://discord.com/invite/U7KEcGErtQ).
---
Source: https://e2b.dev/resources/build-an-agent-workbench-on-openais-agents-api.md
Published: 2026-09-10
# Build an Agent Workbench on OpenAI's Agents API
OpenAI's [Agents API](https://developers.openai.com/api/docs/guides/agents-api/overview) is a hosted Codex runtime that connects to sandbox providers like E2B. OpenAI owns the model, the harness, and session lifecycle, and E2B owns where the agent's code executes. The following example is a workbench built on top of it.
- [Full cookbook example](https://github.com/e2b-dev/e2b-cookbook/tree/main/examples/openai-agents-api-python-sdk)
- [Docs](https://docs.e2b.dev/agents/openai-agents-api)
OpenAI supports two ways to provision E2B: [application-managed](https://developers.openai.com/api/docs/guides/agents-api/environments/providers/e2b#application-managed) and [webhook-managed](https://developers.openai.com/api/docs/guides/agents-api/environments/providers/e2b#webhook-managed). This example is application-managed: your application calls the E2B SDK directly to start, connect, and stop the sandbox, so the sandbox's lifecycle lives in your own code alongside the Agents API session.
## Booting warm from a template
E2B sandboxes are built on Firecracker microVMs, so each session gets an isolated machine that cold-starts in under 60ms, optimizing chats for user experience.
The workbench builds its sandbox from a template with the Codex exec-server already baked in instead of installing dependencies at request time.
Create the sandbox and connect to its terminal with the E2B CLI:
```bash
e2b sbx create openai-agents-api-python-sdk
```
## Every session in its own machine
The workbench consists of a Flask backend that pairs an Agents API session with an E2B sandbox for every chat, and a React frontend built on TanStack Router and Query.
Each session gets an E2B sandbox running the Codex executor; OpenAI runs the agent and maintains session state. When a user sends the first prompt, the backend creates an Agents API session and an E2B sandbox together, then streams the turn back to the frontend as it runs.
Scoping the sandbox to the chat instead of sharing one sandbox across the whole backend process means each conversation's agent can't see or touch another's filesystem. Teardown is per-chat too, and closing one conversation doesn't affect anyone else's session.
## What's wired up
- **Tools**: OpenAI docs MCP, web search scoped to openai.com, and a function tool with the full result round trip
- **Sessions**: Streaming events, follow-ups, cancel, delete, and persistence across a backend restart
- **Pause and fork**: Branching a chat from its snapshot into a new sandbox and session
- **Visibility**: A workspace file viewer and live executor logs
---
Source: https://e2b.dev/resources/devin-outposts.md
Published: 2026-07-21
# Use E2B as the Execution Layer For Devin Outposts
Devin was the moment the industry realized agents could do real engineering work. As agents become part of everyday software development, they need the same infrastructure as the developers they work alongside: the same dependencies, tooling, and access to internal systems. For many organizations, that also means keeping execution inside their own cloud.
Devin Outposts lets you run Devin sessions inside infrastructure you control. By pairing Outposts with our Bring Your Own Cloud (BYOC) offering, each Devin session runs on an E2B microVM that you can configure with the CPU, RAM, and disk your workload needs. Devin can securely access private repositories, internal APIs, package registries, and other VPC-only services while execution remains entirely within your cloud boundary.
E2B also adds snapshot functionality to every Outposts session. If a Devin session needs to be paused, it can be resumed quickly without losing its filesystem, running processes, or other state. Because E2B's microVMs are built on Firecracker, snapshotting and restoring are built in, and Devin can pick up exactly where it left off almost instantly.
**Get started:** [E2B + Devin Guide](https://docs.e2b.dev/agents/devin-outposts)
We're excited to see what developers build with Devin and E2B.
---
Source: https://e2b.dev/resources/e2b-sandboxes-are-now-available-through-stripe-projects.md
Published: 2026-06-10
# E2B sandboxes are now available through Stripe Projects
We're excited to join the Stripe Projects developer preview as a partner.
If you've built anything with AI coding agents, you've run into the same wall: the agent can write the code, but getting it somewhere to run requires a human to step in. You provision infrastructure, generate an API key, name it something like agent-testing-foo-1234, and hope you remember which agent it belongs to when something breaks.
It's a structural gap in how agents interact with the services they depend on. There's no standard protocol for an agent to say "I need a sandbox" and receive authenticated, scoped credentials back without a developer acting as an intermediary every time.
When Stripe brought us in as a partner on the Agentic Provisioning Protocol, the goal was to fix exactly that. Now, the agent can use Stripe Projects to discover, provision, and authenticate a secure E2B sandbox, getting credentials in its environment so it can immediately start running code, without a human touching a dashboard.
**Stripe as the trust layer**
Stripe Projects acts as the trust authority between developers, their agents, and service providers like E2B. Because Stripe already knows who you are and has your payment method, it can vouch for you to any provider in the network, so provisioning an E2B sandbox becomes a single CLI command:
```bash
stripe projects add e2b
```
There's no signup flow and no need to manually retrieve your API keys. The agent runs one command and gets back working credentials.
## **What this means for credentials in practice**
Credentials are delivered as environment variables and synced directly to your environment. When someone joins the team, switches machines, or a new agent session starts, simply run:
```bash
stripe projects env --sync
```
Rotation works the same way. One command issues a new key, revokes the old one, and updates the environment:
```bash
stripe projects rotate e2b
stripe projects env --sync
```
**Why we co-designed the protocol**
Secure, reliable code execution is core to what E2B does, so the credentials and ownership model mattered a lot to us. Resources need to live in the user's own account. Credentials need to be scoped, auditable, and rotatable. Provisioning needs to be deterministic enough that an agent can rely on it.
When Stripe reached out, we saw an opportunity to help get this right. Stripe handles payment via a Shared Payment Token. Your card details never leave Stripe, and E2B never sees them. The whole flow is auditable and repeatable from the CLI.
**What agents can do once provisioned**
Once an E2B sandbox is provisioned, the credentials are available in the environment and an agent can take it from there with the full E2B SDK: creating and managing sandboxes, executing code, handling long-running tasks, taking snapshots, and managing lifecycle, without any human in the loop. And with E2B's prebuilt templates for [OpenCode](https://docs.e2b.dev/agents/opencode), [Codex](https://docs.e2b.dev/agents/codex), [Claude Code](https://docs.e2b.dev/agents/claude-code), and [Amp](https://docs.e2b.dev/agents/amp), they're ready to run right away.
E2B sandboxes run on Firecracker microVMs for kernel-level isolation, support any language or framework, and can run for up to 24 hours. The Stripe Projects provisioning step is the bridge from "I need compute" to "I have a fully authenticated E2B environment."
**Getting started**
Install the Stripe CLI, run `stripe projects init my-app`, and select E2B to provision a sandbox.
```bash
brew install stripe/stripe-cli/stripe && stripe plugin install projects
stripe login
stripe projects init my-app
stripe projects add e2b
stripe projects env --sync
```
We'd love to hear about what you build with Stripe Projects and E2B. Reach out at hello\[at]e2b.dev or tag us on [LinkedIn](https://www.linkedin.com/company/96484695/admin/dashboard/) or [X](https://x.com/e2b).
We're excited to continue evolving the standard and to work with Stripe on sharing a more official specification soon.
If you want your platform to integrate with E2B, reach out at hello\[at]e2b.dev.
---
Source: https://e2b.dev/resources/not-affected-by-copy-fail-heres-why.md
Published: 2026-04-30
# E2B Sandboxes Aren't Affected by Copy Fail (CVE-2026-31431). Here's why.
Theori disclosed CVE-2026-31431 yesterday. A 732-byte PoC Python script. Deterministic root on essentially every Linux system shipped since 2017. If you run untrusted code, this is the kind of bug worth stopping for.
**E2B sandboxes are not affected. By design**.
## Background
`algif_aead` is a kernel module that exposes the crypto API to userspace via `AF_ALG` sockets. A 2017 in-place optimization let page cache pages land in a writable scatterlist during AEAD operations. Combined with `splice()`, an unprivileged process gets a deterministic 4-byte write into the kernel's page cache, the in-memory copy of any file the system reads.
Aim those 4 bytes at `/usr/bin/su`. Run `su`. You're root.
Three properties make this lethal in shared-kernel environments:
- **Deterministic**.
- **Cross-tenant**. The page cache is shared host-wide. A write from one container reaches every other tenant on that host.
- **Invisible**. Corruption only exists in memory. Disk forensics show the original file.
Theori named Kubernetes clusters, CI runners, and "cloud SaaS running user code" as priority targets for patches. That last category covers **most** sandbox-as-a-service providers.
## Why E2B is not affected.
**Per-sandbox kernels**. Every E2B sandbox runs in its own [Firecracker microVM](/resources/firecracker-vs-qemu). Own kernel, own memory, own page cache. Bugcrowd's analysis of Copy Fail puts it cleanly: the boundaries that hold are the ones that don't share a kernel. A kernel exploit inside the sandbox cannot reach the host: the attacker would need a Firecracker escape on top of the kernel bug.
**The module isn't compiled in**. We build custom minimal kernels for agent sandboxes. These run code interpreters, drive browsers and CLIs, power RL rollouts, and host long-running agents. `CONFIG_CRYPTO_USER` and `CONFIG_CRYPTO_USER_API_AEAD` have never been enabled, cause they're not needed by AI agents.
```text
# CONFIG_CRYPTO_USER is not set
# CONFIG_CRYPTO_USER_API_AEAD is not set
```
``Fewer features mean fewer places for vulnerabilities.
## If your sandbox provider runs on containers
Ask them three questions today:
1. Have you patched every host kernel?
2. What was your exposure window?
3. Could one tenant have written into another tenant's page cache before the patch?
## What you should do
Patch to mainline `a664bf3d603d`. Ubuntu 26.04 (Resolute) and later are unaffected.
If you can't patch immediately:
```bash
echo "install algif_aead /bin/false" > /etc/modprobe.d/disable-algif.conf
rmmod algif_aead 2>/dev/null || true
```
Container platforms running untrusted workloads: block `AF_ALG` socket creation with seccomp even after patching. Almost nothing legitimate uses it.
Security and isolation won't affect 99% of your day-to-day. The 1% when it does affect you outweighs the other 99% combined. The reason we chose VM-based architecture is that 1%.
---
Source: https://e2b.dev/resources/e2b-is-now-in-agents-sdk.md
Published: 2026-04-15
# E2B is now supported in the OpenAI Agents SDK
We're happy to announce that E2B is a sandbox provider in the new [OpenAI Agents SDK](https://developers.openai.com/api/docs/guides/agents).
With this release, agents built with the OpenAI Agents SDK will get their own sandboxes featuring a custom environment with resource isolation, security boundaries, and persistence, requiring no infrastructure setup. Your agents will be able to:
- Edit files and run shell commands in isolated environments
- Maintain temporary workspace state across steps
- Generate frontend output with live preview URLs
- Produce artifacts you can review before publishing
- Run multiple sandboxes in parallel for concurrent workloads
## **Integration takes just a few lines of code**
You can set up computer use with just a few lines of code. You just need to define your tools, the agent to use them, and something to get started.
## **See it in action**
In the demo below, we walk through the full workflow of creating a landing page:
1. **Set up a sandbox session** with the E2B SDK and define a builder agent with file-editing capabilities.
2. **Generate a landing page from scratch**: the agent writes HTML/CSS, deploys into the sandbox, and serves it at a preview URL.
3. **Iterate with diffs**: instead of rewriting everything, the agent applies patches for faster turnaround.
4. **Compare variants side by side**: each variant lives at its own URL, making visual comparison straightforward.
5. **Parallelize**: spin up multiple sandboxes simultaneously, each running an independent agent, to generate several variants at once.
The result: multiple landing page variants, each pre-viewable at its own URL, generated in parallel with minimal code.
## **Sample applications and use cases for the OpenAI Agent SDK**
Browse examples in the [E2B Cookbook](https://github.com/e2b-dev/e2b-cookbook/tree/main/examples).
- Computer use
- Webpage building
- Parallelize data science tasks for grid searches and more
- Parallelize code review
## **Get started**
You'll need an [E2B](https://e2b.dev/sign-up) account and an [OpenAI API](https://openai.com/api/) account. Check out the repos:
- [E2B Runtime](https://github.com/e2b-dev/runtime)
- [E2B SDK](https://github.com/e2b-dev/E2B)
- [OpenAI Agents Python SDK](https://github.com/openai/openai-agents-python)
- [OpenAI release post](https://openai.com/index/the-next-evolution-of-the-agents-sdk/)
Note: Sandboxing is currently supported in the Python Agents SDK. Stay tuned for TypeScript support.
---
Source: https://e2b.dev/resources/yc-companies-ai-agents.md
Published: 2026-03-02
# Where AI Agents Are Heading: What We Learned from Recent YC Startups
AI agents are having a moment. Coding agents like [Claude Code](https://docs.anthropic.com/en/docs/claude-code), [Codex](https://openai.com/index/codex), and [Cursor](https://www.cursor.com/) are going mainstream, and highly autonomous ones are making a comeback, gaining the trust of enterprises. [Manus](/customers/how-manus-uses-e2b-to-provide-agents-with-virtual-computers) was acquired by Meta after just a few months of existence, and [Genspark](https://x.com/genspark_ai/status/1999358283052974414?s=20) (recently going viral with their Super Bowl ad) is serving Fortune 100 companies with 5M+ users. [OpenClaw](https://openclaw.ai/), the open-source AI agent that went massively viral, created 1.5 million agents in just weeks — showing that demand for autonomous agents is real and accelerating. Meanwhile, plenty of people still say AI is a bubble.
At [E2B](https://e2b.dev), we are collaborating with selected agentic startups to help them grow and scale with our open-source cloud infrastructure. Our [startup program](https://e2b.dev/startups) is designed to support the companies that we believe will define the next generation of agent-powered software. From the latest batches, we talked to teams that are pushing the boundaries of what agents can do.
The founders behind these startups come from diverse backgrounds — some backed by top accelerators like Y Combinator and a16z, others supported by programs like Founders, Inc. and Z Fellows, and several are still pre-funding, building purely on conviction. Looking across these teams, several patterns become clear about where agents are actually headed — and what infrastructure they'll need to get there.
## Every Company Is Becoming an Agent Company
One of our biggest takeaways: agents are not a vertical anymore. They have become a layer in the product, and this layer has become an expected feature by many enterprises that are in turn starting to adopt the agentic product.
The numbers speak for themselves. Nearly [50% of some of the recent YC batches](https://pitchbook.com/news/articles/y-combinator-is-going-all-in-on-ai-agents-making-up-nearly-50-of-latest-batch) are AI agent companies. When we looked at the latest batch (YC W26 by the time of writing this article), we have identified even more. [80% of enterprises](https://www.salesmate.io/blog/future-of-ai-agents/?st_source=ai_mode#:~:text=By%202026%2C%20IDC%20expects%20AI,tools%20to%20proactive%20decision%2Dmakers.) are expected to implement AI agents by end of 2026.
The startups we selected for the E2B program aren't anymore just "AI agent companies" in the way we usually think about them. They're insurance companies, logistics companies, video production companies, DevOps platforms, and developer tools companies. The agent is how they deliver value, not what they sell to end users anymore.
> "We run Claude Code agents to generate Remotion video templates." — Resonate
[Prox](https://www.getprox.com/) (YC F26) runs ticket resolution agents for logistics companies, [Arcline](https://arcline-ai.com) (YC W26) lets agents do legal documents drafting. These teams don't think about "AI agents" as a category, but they rather solve problems like insurance claims, shipping tickets, or video production. They focus on the end product, which is also the reason they are outsourcing the infrastructure building and maintenance to E2B AI cloud.
## Everything is Code
If we still try to point a finger at the most common use case, it is not surprisingly coding agents. That is, agents that write and run code to accomplish their tasks. The most popular ones — [Claude Code](https://docs.anthropic.com/en/docs/claude-code), [Codex](https://openai.com/index/codex), [Cursor](https://www.cursor.com/), and [Gemini CLI](https://github.com/google-gemini/gemini-cli) — are rapidly becoming the default way developers write software. Essentially, almost anything agents need to do comes down to writing code and running it in an isolated sandbox environment.
Claude Code alone went from $0 to [$1B in annualized revenue](https://x.com/selinazwang/status/1995975322953810055) in roughly six months after its public launch — the fastest product to hit that milestone. [Startups are the main early adopters](https://x.com/AnthropicAI/status/1916871421696917613), which tells you where the market will be moving. Agentic startup now primarily build products using other coding agents, or integrating them directly into products or offering them to enterprises in a secure way.
> "[E2B](https://www.linkedin.com/company/e2b-dev/) powers the orchestration layer that lets you go wild with --dangerously-skip-permissions without blowing up your local machines." — Runtime
[Rivet](https://rivet.gg/) (YC W23) is one of these examples, using E2B sandboxes for their [Sandbox Agent SDK](https://github.com/rivet-dev/sandbox-agent) — with many users running coding agents orchestrated through Rivet on E2B. Fix.fast is building a general-purpose productivity agent working in the E2B sandboxes. [Syntropy](http://syntropy.io/) (YC W26) uses E2B for agentic code execution — running unit and integration tests that the AI agents generate.
One of the critical requirements for agent infrastructure is concurrency — the ability to run many agent instances in parallel. Teams are running dozens of Claude Code and Codex instances simultaneously inside E2B sandboxes to accelerate engineering workflows.
> "We are running many Claude code and Codex instances in parallel to accelerate the team technical design process in large teams." — Scott AI (YC W25)
The maturity of the coding agent category tells us something about the broader market: once a category of agents proves out, adoption is fast and the focus immediately shifts to differentiation on top of the execution layer. Browser agents and vertical agents could be next.
## Agents Deserve Real Computers
E2B has been the pioneer in the AI sandbox category, starting from code interpreter use case, but evolving into a versatile long-running environment where agents install files, do research, or use diversity of tools. Since 2023 when we started, we've watched what teams expect from sandbox providers evolve. The expectations on the AI-first infrastructure have been rising, and features like persistence or good way of scaling have become the standard.
Manus, [Genspark](https://www.genspark.ai/), [OpenClaw](https://openclaw.ai/), and other general autonomous agents demonstrate that agents need to use computers the same way humans do. OpenClaw in particular showed the world what happens when you give an AI agent full system access — email, calendars, messaging, browsers, file systems — and let it act autonomously. The result was the [fastest-growing open-source project in GitHub history](https://x.com/aakashgupta/status/2021366401500664207): 9K to 182K stars in 60 days, outpacing Linux, React, and TensorFlow.
But viral adoption also highlighted why [security and sandboxing](https://www.crowdstrike.com/en-us/blog/what-security-teams-need-to-know-about-openclaw-ai-super-agent/) are non-negotiable. [Critical authentication bypasses](https://aisle.com/blog/aisle-tops-openclaw-disclosures) and other issues were quickly discovered in OpenClaw. Problems like agents accessing operating system or critical data can be solved by the right isolation layer in the infrastructure.
> "Our E2B use case is Manus-like sandboxes for browser use, web research, voice calls, and image generation" — Waldium
## Build With Us
We're always looking for the next generation of agent-first companies to support through the [E2B startup program](https://e2b.dev/startups). The program includes E2B credits and all benefits of the E2B Pro tier plan. If you're building something ambitious with AI agents, we'd love to hear from you — apply for the next batch on the website, or get in touch at [startups@e2b.dev](mailto:startups@e2b.dev).
―
E2B is the open-source cloud infrastructure for AI agents. Secure virtual machines that spin up in milliseconds. [Get started →](https://e2b.dev)
---
Source: https://e2b.dev/resources/postmortem-service-disruption-on-jan-13-2026.md
Published: 2026-01-14
# Postmortem: Service disruption on Jan 13, 2026
On January 13, E2B had a brief infrastructure outage following a continuous production rollout to upgrade our systems. A control-plane failure in one of our regions blocked the creation of new sandboxes and began causing malfunctions in already-running sandboxes.
Service is fully restored. We sincerely apologize to all customers affected. We know sandboxes are on the critical path for agentic workflows, and this outage may have impacted important end-user workflows. We have applied immediate safeguards and are rolling out longer-term improvements to reduce the likelihood and impact of a recurrence. Template data and paused sandboxes were not affected by this incident.
## Customer impact
- **Single node failure**: At 4:31 PM PST sandboxes on one node experienced issues with requests to their management API. We stopped scheduling sandboxes on the node and started investigating this as a workload node issue. At this point sandboxes were being scheduled on the other nodes and there was no impact on their functionality.
- **Degraded service**: 5:02 PM to 5:12 PM PST. Elevated failures across several nodes occurred while we were triaging the problem and we began stabilizing the control plane.
- **Outage**: 5:12 PM to 5:35 PM PST. New sandbox creations began to fail, and existing ones started malfunctioning.
- **Recovery**: 5:35 PM to 5:46 PM PST—Sandbox creation and functionality restored, but capacity was temporarily constrained while additional workload nodes were coming online. Most requests were succeeding by ~5:42 PM, with a small number of errors tapering off by 5:46 PM PST.
- **Full recovery**: 5:46 PM PST.
## Detection
We pinpointed the root cause of the incident via high control-plane CPU usage reports, elevated sandbox-creation failures, and degradation signals in production monitoring. The first signs of the possible incident were reported by users experiencing problems with the sandbox management API during the single node failure.
## Root cause
A production deployment of the orchestrator increased the likelihood of triggering a Nomad scheduling failure mode involving reserved host ports and port preemption. Nomad began repeatedly rejecting placement plans (port planning failures and port collisions). During this churn, the control plane experienced increased resource pressure (CPU, disk activity, and memory), and the affected control plane nodes began failing health checks and restarting.
With multiple control plane nodes unstable and without enough time for individual nodes to recover, the cluster could not reliably maintain quorum or conduct leader elections. Without a stable leader and quorum, Nomad couldn't schedule new allocations, and Consul couldn’t provide internal cluster routing, which affected several services and prevented starting or interacting with sandboxes.
## Resolution
We restored service by re-establishing a healthy Nomad server quorum and re-applying job state:
- Brought additional server capacity online to restore a stable control plane quorum.
- Rebooted the affected control plane nodes so they could rejoin cleanly.
- Re-deployed Nomad jobs to the recovered cluster, restoring scheduling and sandbox functionality.
## Corrective actions
- **Monitoring**: Add tighter alerts for memory pressure and early CPU anomalies, as well as plan-failure and port-collision signals; expand Nomad/Consul dashboards.
- **Stability**: Speed up replacement of unhealthy control-plane servers; maintain sufficient headroom to keep quorum during node instability.
- **Safer deploys**: Adjust orchestrator port/deploy strategy to avoid possibly risky scheduling patterns; expand pre-deploy checks. In the long term, migrate to a different deploy strategy.
- **Resilience**: Improve persistence and recovery so the control plane can recover cleanly after reboots. Focus on a multi-region failover so a regional control-plane failure does not affect platform availability.
---
Source: https://e2b.dev/resources/docker-e2b-partner-to-introduce-mcp-support-in-e2b-sandbox.md
Published: 2025-10-23
# Docker & E2B partner to introduce MCP support in E2B Sandbox
Today, we’re launching native MCP support in the E2B Sandbox. We’ve partnered with the Docker team to bring over 200+ MCP tools from the official [Docker’s MCP catalog](https://hub.docker.com/mcp) to every sandbox.
Watch the announcement video with Mark Cavage (President & COO at Docker), Tushar Jain (Head of Product & Engineering at Docker), and Vasek Mlejnsky (CEO at E2B).
[Model Context Protocol](https://modelcontextprotocol.io/) (MCP) is an open-source standard that makes it easy to connect AI systems like AI agents to external applications and use those external applications as LLM tools. In short, MCP tools make AI agents more useful and give them easy access to real world applications. However, the MCP tools can suffer from [various](https://www.docker.com/blog/mcp-horror-stories-the-supply-chain-attack/) [critical](https://www.docker.com/blog/mcp-horror-stories-the-supply-chain-attack/) [security](https://www.docker.com/blog/mcp-security-issues-threatening-ai-infrastructure/) issues that can put your infrastructure at risk. Additionally, having your agents use MCP tools in sandboxes has been until now painful and required a significant amount of sandbox customization making developers spend their time on manually setting up various MCP tools and frameworks inside the sandbox instead of focusing on building the agent.
We’re bringing the MCP support to E2B sandboxes and partnering with Docker to give developers quick and easy access to hundreds of type-safe MCPs in a secure sandbox. The MCPs have been verified and are safe to consume through a single unified interface. What previously took tens of minutes to hours of setup now takes a few seconds.
E2B has already been powering secure sandboxes for agent code paths, and Docker has already produced hundreds of trusted MCP servers. Together, we built a completely safe and trustworthy foundation for connecting AI agents with real-world tools.
This partnership with Docker spans both the content collaboration and the open-source MCP gateway. Together, we’re building on the momentum of a growing ecosystem of open, secure, and interoperable tools—while later introducing commercial extensions on top of it.
The MCP tools in your E2B sandbox can be reached from both inside (accessing the MCP gateway running on localhost) and outside (via the sandbox URL), allowing your local code and sandboxed agents to share the same tools securely.
**JavaScript**
```typescript
import Sandbox from 'e2b'
const sandbox = await Sandbox.create({
mcp: {
browserbase: {
apiKey: process.env.BROWSERBASE_API_KEY!,
geminiApiKey: process.env.GEMINI_API_KEY!,
projectId: process.env.BROWSERBASE_PROJECT_ID!,
},
exa: {
apiKey: process.env.EXA_API_KEY!,
},
notion: {
internalIntegrationToken: process.env.NOTION_API_KEY!,
},
},
});
```
**Python**
```python
import os
from e2b import Sandbox
sandbox = await Sandbox.create(
mcp={
"browserbase": {
"apiKey": os.environ["BROWSERBASE_API_KEY"],
"geminiApiKey": os.environ["GEMINI_API_KEY"],
"projectId": os.environ["BROWSERBASE_PROJECT_ID"],
},
"exa": {
"apiKey": os.environ["EXA_API_KEY"],
},
"notion": {
"internalIntegrationToken": os.environ["NOTION_API_KEY"],
},
}
)
```
MCP has become a widely adopted standard for connecting tools to LLMs. By bringing native MCP support to the E2B sandboxes, we’re making it easy for developers and AI agents to securely access all the tools from Docker's MCP catalog such as Browserbase, Stripe, GitHub, Context7, Grafana, or Notion and make their agents more useful in just a few lines of code.
Each MCP tool runs as a Docker container inside the E2B sandbox. The E2B SDK gives you autocomplete, type validation, and one unified interface to control and configure each MCP tool.
### **How to start**
Get started by installing the latest version of the E2B SDK and following quickstart examples in our [documentation](https://docs.e2b.dev/mcp).
**JavaScript**
`npm install e2b@latest`
**Python**
`pip install --upgrade e2b`
### **Towards secure and vetted MCP tools**
Our partnership with Docker allows us to bring many official and verified MCP servers in a secure, isolated environment.
The combination of Docker’s powerful MCP catalog, wide developer adoption, and our secured sandboxes make running MCP servers not only safer but also very easily accessible for all the AI developers building at the forefront of what’s possible with the state-of-the-art models.
We’ve given early access to the sandbox MCP support to selected of E2B users and partners. We want to thank Genspark, Groq, Exa, Browserbase, and others for providing valuable feedback and supporting this launcj.
> "E2B MCP with code unlocks an exciting paradigm of MCP in code while keeping the dev experience simple and intuitive."
\- Benjamin Klieger, Compound AI Lead
> “Genspark uses E2B Sandbox with MCP integration to let users run 200+ MCPs from Docker’s official catalog with minimal setup, giving access to tools like Notion, Stripe, Browserbase, or Exa. E2B is an essential layer in our infrastructure, enabling us to securely run any MCP directly from GitHub inside isolated sandboxes.”
\- Kay Zhu, cofounder and CTO of Genspark
---
Source: https://e2b.dev/resources/introducing-build-system-2-0.md
Published: 2025-10-16
# Introducing Build System 2.0
We’ve released **E2B Build System 2.0** — a faster and simpler way to create custom sandboxes in E2B.
With the new system you don’t need Dockerfiles, extra config files, or manual CLI build commands. You just write code. The build step still exists, but it runs automatically when you execute your template.
## **Why we rebuilt it**
The old build system was functional, but:
- Required juggling config files, Dockerfiles, and CLI commands.
- Builds were slow, mostly local, and debugging was painful.
- Hard to expose or extend build logic for your own users.
Build System 2.0 addresses these problems head-on.
## **What’s new**
### **1. Better Developer Experience**
- **Before**: e2b template build + config + Dockerfile
- **Now**: Just run your code → the template builds automatically/
What previously took a config file, a Docker file, and running a terminal command is now just running a script with a few lines of code. This enables type hints, linters, dynamic builds inside apps, and exposing build logic directly to users.
**Javascript**:
```javascript
Template()
.fromImage("node:24")
.copy("src/", ".")
.runCmd("npm install")
```
**Python**:
```python
Template()
.from_image("node:24")
.copy("src/", ".")
.run_cmd("npm install")
```
### **2. Agentic Experience**
Templates are now expressed as code, so AI IDEs and agents like Cursor or Claude Codecan parse, suggest, and generate them without extra config. This makes it straightforward to plug the build system into an LLM as a tool. This makes it straightforward to plug the build system into an LLM as a tool.
### **3. Speed**
- Intelligent caching
- Parallel uploads
- Server-side optimizations
Builds are **up to 14× faster** when cached, and ~2× faster without cache. They now run on E2B infra — making the feedback loops shorter, performance more stable, and have clearer error messages. They now run on E2B infra — making the feedback loops shorter, performance more stable, and have clearer error messages.
### **4. Painless Migration**
We kept backwards compatibility and added multiple migration paths:
1. `e2b template migrate` (CLI)``
2. `Template().fromDockerfile()/Template(path).from_dockerfile(path)` → parses your Dockerfile``
3. `fromImage(tag)/from_image(tag)` → continue building from an existing base image
## **Helper Methods**
The API now includes helpers for common tasks such as:
**JavaScript**
```javascript
Template()
.fromUbuntu("24.04")
.pipInstall(["numpy", "pandas"])
.aptInstall(["curl", "wget"])
.gitClone("https://github.com/user/repo.git", { branch: "main" })
.setEnvs({ DEBUG: "true" })
.runCmd("npm start")
```
**Python**
```python
Template()
.from_ubuntu("24.04")
.pip_install(["numpy", "pandas"])
.apt_install(["curl", "wget"])
.git_clone("https://github.com/user/repo.git", branch="main")
.set_envs({ DEBUG: "true" })
.run_cmd("npm start")
```
## **Get Started with Build System 2.0**
If you’re on the latest E2B SDK, you already have Build System 2.0. Here’s how to try it out:
### **1. Install or update the SDK**
Make sure you’re on the newest version:
**JavaScript**
`npm install e2b@latest`
**Python**
`pip install --upgrade e2b`
### **2. Create a simple template**
Define your build directly in code:
**JavaScript**
```javascript
import { Template, waitForPort } from "e2b"
const template = Template()
.fromImage("node:24")
.copy("src/", ".")
.runCmd("npm install")
.setStartCmd("npm start", waitForPort(3000))
```
**Python**
```python
from e2b import Template, wait_for_port
template = (
Template()
.from_image("python:3.11")
.pip_install(["requests", "numpy"])
.copy("app.py", ".")
.run_cmd("python app.py", wait_for_port(3000))
)
```
### **3. Build the template**
The build step is automatic — just run code:
**JavaScript**
```javascript
await Template.build(template, {
alias: "my-template",
onBuildLogs: defaultBuildLogger(),
})
```
**Python**
```python
Template.build(template, alias="my-template", on_build_logs=default_build_logger())
```
### **4. Run your sandbox**
Once the template is built, launch a sandbox from it:
**JavaScript**
```javascript
import { Sandbox } from "e2b"
const sandbox = await Sandbox.create("my-template")
const result = await sandbox.commands.run("echo Hello E2B")
console.log(result.output)
```
**Python**
```python
from e2b import Sandbox
sandbox = Sandbox.create("my-template")
result = sandbox.commands.run("echo Hello E2B")
print(result.output)
```
### **5. Migrating existing templates**
If you already have templates built with the old system, you can migrate:
`e2b template migrate`
Or update your code with:
**JavaScript**
`Template().fromDockerfile("./e2b.Dockerfile")`
**Python**
`Template().from_dockerfile("./e2b.Dockerfile").`
### **Resources**
- [E2B documentation](https://docs.e2b.dev)
- [Template migration guide docs](https://docs.e2b.dev/migration/template-v2)
---
Source: https://e2b.dev/resources/javascript-guide-run-openai-codex-in-an-e2b-sandbox.md
Published: 2025-08-25
# JavaScript Guide: Run OpenAI Codex in an E2B Sandbox
OpenAI Codex turns natural-language instructions into code edits and files. Run it inside an **E2B Sandbox** and you get an isolated, long-running environment where the agent can safely create files, install tools, and execute without interactive prompts.
## **What we’ll build**
- Spin up an E2B sandbox from the **prebuilt openai-codex template**.
- Inject your [**OpenAI API key**](https://platform.openai.com/api-keys) into the sandbox.
- Run a Codex prompt that creates a Hello World file.
- Print the output and shut the sandbox down.
#### See [full example](https://github.com/e2b-dev/e2b-cookbook/tree/main/examples/openai-codex-in-sandbox-js) in the E2B Cookbook.
## **Components we’ll use**
- **E2B JS SDK** – creates and controls the sandbox.
- **Prebuilt template**: openai-codex – Codex CLI is preinstalled.
- **`dotenv`** – loads [`E2B_API_KEY`](https://console.e2b.dev/?tab=keys) from `.env`.
## **Prerequisites**
- **Node.js 18**+
- [E2B account](https://console.e2b.dev) + [`E2B_API_KEY`](https://console.e2b.dev/?tab=keys)
- [**OpenAI API key**](https://platform.openai.com/api-keys)
## **Outline**
1. Create the project
2. Add the files (package.json, src/index.ts)
3. Set `E2B_API_KEY` in `.env`
4. Install and run
## **1) Create the project**
```bash
mkdir openai-codex-in-sandbox-js
cd openai-codex-in-sandbox-js
mkdir -p src
```
Create and commit .**gitignore before .env** so secrets never enter git history.
Target structure:
```text
openai-codex-in-sandbox-js/
├─ src/
│ └─ index.ts
├─ package.json
├─ .gitignore
└─ .env
```
## **2) Files to add**
### **File**: **`package.json`**
```json
{
"name": "openai-codex-in-sandbox",
"version": "1.0.0",
"main": "index.js",
"type": "module",
"scripts": {
"start": "tsx src/index.ts"
},
"keywords": [],
"author": "",
"license": "ISC",
"description": "",
"dependencies": {
"dotenv": "^16.5.0",
"e2b": "^1.4.0",
"tsx": "^4.19.4"
}
}
```
### **File**: .**`gitignore`**
```text
.env
node_modules/
```
### **File**: .**`env`**
```bash
E2B_API_KEY="YOUR_E2B_API_KEY"
```
### **File**: **`src/index.ts`**
**This script loads .env (so the SDK sees E2B\_API\_KEY), creates the sandbox from the openai-codex template while injecting your OpenAI key, runs a Codex exec prompt that generates index.html, prints the result, and then shuts the sandbox down**.
Load configuration and SDK:
```javascript
import { Sandbox } from 'e2b'
import dotenv from 'dotenv'
dotenv.config()
```
Create the sandbox (OpenAI key passed inline) and log its ID:
```javascript
const templateName = 'openai-codex'
const sbx = await Sandbox.create(templateName, {
envs: {
OPENAI_API_KEY: '',
},
})
console.log('Sandbox created', sbx.sandboxId)
```
Run a Codex prompt that creates a file (no per-command timeout):
```javascript
// Print help for Codex
// const result = await sbx.commands.run('codex --help')
// console.log(result.stdout)
// Run a prompt with Codex
const result = await sbx.commands.run(
`codex exec --skip-git-repo-check --dangerously-bypass-approvals-and-sandbox "Create a hello world index.html"`,
{ timeoutMs: 0 }
)
console.log(result.stdout)
```
Clean up:
```javascript
sbx.kill()
```
## **3) Install & run**
```bash
npm install
npm run start
```
**Expected**: console output from Codex. The generated index.html lives inside the sandbox filesystem.
#### See [full example](https://github.com/e2b-dev/e2b-cookbook/tree/main/examples/openai-codex-in-sandbox-js) in the E2B Cookbook.
---
Source: https://e2b.dev/resources/python-guide-run-openai-codex-in-an-e2b-sandbox.md
Published: 2025-08-25
# Python Guide: Run OpenAI Codex in an E2B Sandbox
OpenAI Codex turns natural-language instructions into code edits and files. Run it inside an **E2B Sandbox** and you get an isolated, long-running environment where the agent can safely create files, install tools, and execute without interactive prompts.
## **What we’ll build**
- Spin up an E2B sandbox from the **prebuilt openai-codex template**.
- Inject your **OpenAI API key** into the sandbox.
- Run a Codex prompt that creates a Hello World file.
- Print the output and shut the sandbox down.
#### See [full example](https://github.com/e2b-dev/e2b-cookbook/tree/main/examples/openai-codex-in-sandbox-python) in the E2B Cookbook.
## **Components we’ll use**
- **E2B Python SDK** – creates and controls the sandbox.
- **Prebuilt template**: openai-codex – Codex CLI is preinstalled.
- **OpenAI API** – used by Codex via OPENAI\_API\_KEY.
## **Prerequisites**
- **Python 3.11**+
- [E2B account](https://console.e2b.dev) + [`E2B_API_KEY`](https://console.e2b.dev/?tab=keys)
## **Outline**
1. Create the project
2. Add the files (pyproject.toml, main.py)
3. Set E2B\_API\_KEY in .env
4. Install with pip and run
## **1) Create the project**
```bash
mkdir openai-codex-in-sandbox-python
cd openai-codex-in-sandbox-python
mkdir -p src/openai_codex_in_sandbox_python
```
Create and commit `.gitignore` before `.env` so secrets never enter git history.
Target structure:
```text
openai-codex-in-sandbox-python/
├─ src/
│ └─ openai_codex_in_sandbox_python/
│ └─ main.py
├─ pyproject.toml
├─ .gitignore
└─ .env
```
## **2) Files to add**
#### **File**: **`pyproject.toml`**
```toml
[project]
name = "openai-codex-in-sandbox-python"
version = "0.1.0"
description = "Add your description here"
readme = "README.md"
requires-python = ">=3.11"
dependencies = [
"e2b",
"python-dotenv",
]
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
```
#### **File**: **`src/openai_codex_in_sandbox_python/main.py`**
This script loads `.env` (so the SDK sees `E2B_API_KEY`), creates the sandbox from the `openai-codex` template while injecting your OpenAI key, runs a Codex exec prompt that generates index.html, prints the result, and then shuts the sandbox down.
Load configuration and SDK:
```python
from dotenv import load_dotenv
from e2b import Sandbox
load_dotenv()
```
Create the sandbox (OpenAI key passed inline) and log its ID:
```python
template_name = 'openai-codex'
sbx = Sandbox(
template_name,
envs={
"OPENAI_API_KEY": "",
},
timeout=60 * 5, # 5 minutes; match README (remove this line to match the file exactly)
)
print("Sandbox created", sbx.sandbox_id)
```
Run a Codex prompt that creates a file (no per-command timeout):
```python
# Print help for Codex
# result = sbx.commands.run('codex --help', request_timeout=0, timeout=0)
# print(result.stdout)
# Run a prompt with Codex
result = sbx.commands.run(
"codex exec --skip-git-repo-check --dangerously-bypass-approvals-and-sandbox 'Create a hello world index.html'",
timeout=0,
)
print(result.stdout)
```
Clean up:
```python
sbx.kill()
```
#### **File**: .**`gitignore`**
```text
.venv/
.env
__pycache__/
*.pyc
```
#### **File**: .**`env`**
```bash
E2B_API_KEY="YOUR_E2B_API_KEY"
```
## **3) Set up and install**
```bash
python -m venv .venv
# macOS / Linux
source .venv/bin/activate
# Windows (PowerShell)
# .venv\Scripts\Activate.ps1
pip install -e .
```
## **4) Run the example**
```bash
python src/openai_codex_in_sandbox_python/main.py
```
**Expected**: console output from Codex. The generated index.html lives inside the sandbox filesystem.
#### See [full example](https://github.com/e2b-dev/e2b-cookbook/tree/main/examples/openai-codex-in-sandbox-python) in the E2B Cookbook.
---
Source: https://e2b.dev/resources/series-a.md
Published: 2025-07-28
# We Raised $21M to Give Fortune 100 Cloud for AI Agents
Today, we’re excited to announce that E2B has raised a $21M Series A funding round led by global software investor Insight Partners, with participation from Decibel, Sunflower Capital, and Kaya, along with prominent angels such as Docker's former CEO Scott Johnston. This investment comes as **88% of the Fortune 100** are already signed up on our platform, validating our vision of giving every enterprise AI agent its own **cloud computer** to work with. We’ve grown from a handful of early adopters to hundreds of millions of cloud sandboxes initiated at more than half of Fortune 500. Agents are already dramatically boosting human productivity, with compelling stats:
- Code copilots generate at least a [quarter of the world’s code](https://archive.is/X43PU).
- [JPMC saved 360,000 hours](https://medium.com/@arahmedraza/how-jpmorgan-uses-ai-to-save-360-000-legal-hours-a-year-6e94d58a557b) of manual work per year with document processing agents.
Yet, despite these gains, agent productivity remains capped due to inadequate infrastructure and tooling. Today's agents lack secure, scalable environments necessary to fully unleash their capabilities. And while enterprises are expecting to automate away between [15% to 50% of manual tasks with agents](https://www.mckinsey.com/capabilities/mckinsey-digital/our-insights/superagency-in-the-workplace-empowering-people-to-unlock-ais-full-potential-at-work), we simply are not giving the agents the right tools and compute infrastructure to complete them. E2B solves this. And we’ve raised our $21M Series A to deliver a secure, scalable suite of infrastructure tools needed to deliver production-grade agentic workflows in enterprise.
We do this by giving AI agents instant-on, secure cloud machines equipped with the specialized tools they need to perform complex, multi-step tasks seamlessly and safely.
## **Our Enterprise Traction and Customer Success to Date**
It’s not often a young company can say its product is used by the vast majority of Fortune 100 firms. We’re excited that enterprises across industries have embraced E2B as the backbone for their AI initiatives. From AI research labs to high-security enterprise environments, E2B’s **sandboxed cloud runtime** is becoming the de facto, open-source standard for safe, scalable code execution by AI - from research labs to high-security enterprise environments:
- Hugging Face and LMArena use E2B to [securely scale AI research](/customers/how-hugging-face-is-using-e2b-to-replicate-deepseek-r1) with tens of thousands of concurrent machines with almost zero setup time.
- Perplexity and Lindy [integrate and scale agentic features](/customers) in days instead of weeks.
- Groq relies on E2B for [high‑speed, secure code execution](/customers/groqs-compound-ai-models-are-powered-by-e2b).
- Manus runs [fully functional virtual computers](/customers/how-manus-uses-e2b-to-provide-agents-with-virtual-computers) to power autonomous multi‑agent systems.
We’re proud to be supporting dozens of use cases we never imagined when we started, and learning from each of them to further improve our product.
## **From LLMs to Autonomous Agents: Why Production-Grade Agentic Workflows Need Better Infrastructure**
E2B is becoming the open-source standard for sandboxes for agentic workflows. In the near future, this will extend into a standardized interface allowing AI agents to seamlessly control and interact with diverse sandbox environments—from Linux interpreters and Windows VMs to Chrome browsers and operate on your infrastructure of choice - be it Kubernetes, Google Cloud, Azure, AWS, Cloudflare, or your in-house VPC. This approach transforms how agents execute complex tasks, ensuring they can securely chain operations, maintain state, and interact with external systems without ad-hoc infrastructure. By providing a shared, modular architecture, E2B encourages community-driven extensibility and innovation, aiming to establish a universal standard similar to web protocols.
Expanding the open-source standard, E2B will provide pluggable modules such as **Secrets Vault** for secure credential management, **Sandbox Observability** for real-time monitoring and compliance, and **Shared Context** for persistent, collaborative state management. These add-ons will enable enterprises to tailor their agent environments precisely to security and operational requirements. Ultimately, E2B's vision is a robust, open ecosystem where autonomous agents run safely, transparently, and flexibly—accelerating adoption from pilots to full-scale deployment.
**Scaling the Future: Enterprise-Ready Infrastructure for AI Agents**
It’s obvious that AI agents are here to stay. We've built the infrastructure to make it happen.
At the same time, we remain focused on practical needs—delivering a platform that's robust, reliable, and scalable enough for today's most demanding users. Our technology is already proving its worth at Fortune 100 scale.
Since October last year, we've supported **hundreds of millions of sandbox sessions**, with customers like Perplexity successfully testing our platform at high scale from the start. Now, with our latest funding, we'll expand our engineering, product, and go-to-market teams in San Francisco, accelerate development of our sandbox platform, and meet growing demand worldwide.
To our customers, thank you for trusting us with such critical parts of your stack – your stories of E2B in action inspire us daily. To the Fortune 100 companies on board, thank you for pushing us to meet the highest bars for security, compliance, and reliability. And to the AI builders and dreamers: we see you, and we share your excitement for what’s possible. Agents that debug software, agents that analyze and chart data, generating expert reports, agents that help train new models with reinforcement learning – all of these are being built on E2B right now, and many more are yet to come.
In 10 years, we see AI agents becoming as commonplace as apps on your iPhone, seamlessly integrated into daily tasks. Just as an iPhone app needs iOS, every agent will rely on its own secure and dedicated computing environment. With E2B, we are laying the groundwork for that future now, one sandbox at a time. We’re grateful to have incredible investors like **Insight Partners** on this journey with us, and we’re more motivated than ever to deliver on our promise: giving AI agents the cloud machines they need to think and to act.
*Thank you for being part of our story. Here’s to the next chapter of E2B* – [*come build with us*](https://e2b.dev/careers)!
---
Source: https://e2b.dev/resources/firecracker-vs-qemu.md
Published: 2025-03-03
# Firecracker vs QEMU
If you want to run untrusted code like code generated by LLMs you will quickly realize that you need a way to sandbox the code. You could of course run the code in a container, but that is generally not considered secure enough. You could also run the code in a traditional virtual machine, but that is probably overkill and too resource intensive for short-lived tasks. This is where Firecracker and QEMU come in.
Both Firecracker and QEMU are virtualization solutions that can be used to run untrusted code. They are both KVM-based and offer robust isolation, but they serve distinctly different needs and come with their own sets of tradeoffs.
Firecracker, developed by AWS, has gained significant attention for being lightweight, fast, and security-focused. It powers AWS Lambda's serverless infrastructure, handling tens of trillion function invocations monthly \[16]. Its minimal footprint and rapid boot times make it particularly attractive for ephemeral workloads.
On the other hand, QEMU has established itself as the Swiss Army knife of virtualization. With its rich feature set and extensive hardware support, it powers everything from development environments to production servers. While it may not match Firecracker's lightning-fast boot times, QEMU offers unparalleled flexibility, supporting full system emulation, hardware acceleration, and a wide range of (sometimes legacy) virtual devices that modern development workflows depend on.
This comparison will go into both virtualization solutions, exploring their architectures, strengths, and limitations. So let's go and sandbox some untrusted code!
## Virtualization Basics
Virtualization lets you run multiple isolated operating systems on a single physical machine. Modern CPUs include special instructions that make virtualization faster and more secure. If you're a bit rusty on the basics, here are the key components:
**Virtual Machine (VM**) is an isolated environment that acts like a complete computer, with its own CPU, memory, storage, and network resources. VMs run their own operating systems and can't directly interfere with each other.
**Virtual Machine Monitor (VMM**) or **Hypervisor** is the software layer that creates and manages VMs. It controls how VMs access the physical hardware and ensures they remain isolated from each other. VMMs come in two types:
- Type 1 hypervisors run directly on hardware (like VMware ESXi)
- Type 2 hypervisors run on top of an operating system (like VirtualBox)
**KVM (Kernel-based Virtual Machine**) is Linux's built-in hypervisor. When enabled, it turns the Linux kernel into a Type 1 hypervisor, letting you run VMs with near-native performance. Both QEMU and Firecracker use KVM for CPU virtualization.
## QEMU
QEMU stands for Quick EMUlator. It's widely used in production environments to run virtual machines for web servers, databases, and development environments. It can run as a pure emulator to mimic different CPU architectures, or as a virtualizer using KVM for near-native performance \[[0](#sources)].
QEMU handles common VM workloads like running Linux servers, Windows instances, and containerized applications. Developers use it to test software across different operating systems, run CI/CD pipelines, and deploy production services. It's particularly useful when you need to run multiple isolated environments on a single machine. When running with KVM, QEMU delegates CPU and memory management to the Linux kernel while handling I/O and device emulation itself. This split design lets QEMU support a wide range of devices while maintaining good performance for basic compute tasks.
### What makes QEMU special?
- **Extensive Platform Support**: You can run almost anything on QEMU. From ancient s390x \[[17](#sources)] systems to modern full-desktop environments, if you need it, QEMU probably supports it \[[5](#sources)]
- **Full Emulation and KVM Acceleration**: Support for both emulation and hardware-accelerated KVM virtualization. \[[0](#sources)]
- **Ecosystem and Integration**: QEMU's long history means there's a vast array of tools and integrations available.
- **MicroVM Mode**: QEMU MicroVMs \[[13](#sources)], more similar to Firecracker. Less devices supported, faster boot times (4x as fast as normal QEMU, still 3x slower than Firecracker)
### What are the tradeoffs?
With age and versatility comes a different set of tradeoffs:
- **Complexity and Security**: With nearly 2 million lines of C code \[[6](#sources)], QEMU's attack surface is significantly larger, with a history of CVEs. \[[7](#sources)]
- **Performance**: QEMU's full-featured nature means slower boot times and higher memory usage, which can be a bottleneck for ephemeral or lightweight workloads. \[[8](#sources)]\[[14](#sources)]
- **Developer Experience**: The vast options (the manpage is over 47 pages long \[[11](#sources)]) can be overwhelming, leading to a steeper learning curve.
## Firecracker
Firecracker is a virtual machine monitor (VMM) built by AWS specifically for running serverless workloads. It's a stripped-down virtualization tool that focuses on speed and security rather than flexibility. Using KVM under the hood, Firecracker creates lightweight virtual machines called microVMs \[[1](#sources)].
Unlike traditional VMs that might take several seconds to start, Firecracker VMs can boot up in as little as 125ms. AWS built it to power Lambda and Fargate \[[2](#sources)], where they need to quickly spin up isolated environments for running customer code. Companies like [E2B](https://e2b.dev/) use Firecracker to run AI generated code securily in the cloud, while Fly.io uses it to run lightweight container-like VMs at the edge \[[4](#sources), [5](#sources)].
Firecracker runs as a single process on your host machine and exposes a simple API for creating and managing VMs. Each microVM is minimal - it only includes the devices and functionality needed to run a basic Linux kernel. This minimalist approach helps keep the codebase small and reduces potential security vulnerabilities.
### What makes Firecracker special?
- **Small and Secure**: Written in Rust with only 50k lines of code (as of 2020), significantly reducing the attack surface and overall complexity \[[12](#sources)]
- **Fast Boots**: Starts VMs 3x faster than QEMU MicroVMs and 10x faster than normal QEMU emulation \[[8](#sources)]
- **Minimal Memory Usage**: Each microVM has less than 5MB RAM overhead \[[9](#sources)]
- **Simple Architecture**: Single process with a straightforward API, making it easy to manage and integrate \[[12](#sources)]
- **Resource Efficient**: Designed for overcommitting hardware resources, perfect for serverless workloads
### What are the tradeoffs?
- **Limited Device Support**: Minimal device emulation - no support for legacy hardware like floppy disks
- **No GPU Support**: Currently lacks PCIe support, making GPU passthrough impossible (work has been stopped for 2025) \[[10](#sources)]
- **Smaller Ecosystem**: While growing with adopters like E2B and Fly.io, the ecosystem is still smaller than QEMU's \[[3](#sources), [4](#sources)]
## So what?
Of course, the decision is very nuanced and specific to your use case. In general:
- Choose Firecracker if you're looking for speed, security, and efficiency for serverless computing or edge computing scenarios with modern hardware support.
- Choose QEMU if you need versatility, support for a broader range of hardware, full system emulation, or if you're dealing with legacy systems or development environments requiring extensive device emulation.
## Sources
1. [QEMU](https://www.qemu.org/)
2. [Firecracker](https://firecracker-microvm.github.io/)
3. [Firecracker Paper](https://www.usenix.org/system/files/nsdi20-paper-agache.pdf)
4. [E2B](https://github.com/e2b-dev/runtime)
5. [Fly.io](https://fly.io/docs/reference/architecture/)
6. [QEMU Build Platforms](https://www.qemu.org/docs/master/about/build-platforms.html)
7. [QEMU OpenHub](https://openhub.net/p/qemu)
8. [QEMU CVEs](https://www.cvedetails.com/vulnerability-list/vendor_id-7506/Qemu.html)
9. [Firecracker Boot Time](https://doi.org/10.1145/3447786.3456248)
10. [Firecracker Memory Footprint](https://firecracker-microvm.github.io/#:~:text=This%2C%20along%20with%20a%20streamlined,a%20%3C%205%20MiB%20memory%20footprint.)
11. [Firecracker PCIe](https://github.com/firecracker-microvm/firecracker/issues/4845)
12. [QEMU manpage](https://manpages.debian.org/jessie/qemu-system-x86/qemu-system-x86_64.1.en.html)
13. [Firecracker Github](https://github.com/firecracker-microvm/firecracker)
14. [QEMU MicroVMs](https://www.qemu.org/docs/master/system/i386/microvm.html)
15. [Performance analysis of KVM-based microVMs orchestrated by Firecracker and QEMU](https://dreadl0ck.net/papers/Firebench.pdf)
16. [Firecracker Credits](https://github.com/firecracker-microvm/firecracker/blob/main/CREDITS.md)
17. [AWS Lambda](https://aws.amazon.com/blogs/aws/aws-lambda-turns-ten-the-first-decade-of-serverless-innovation/)
18. [](https://aws.amazon.com/blogs/aws/aws-lambda-turns-ten-the-first-decade-of-serverless-innovation/)[S390x](https://www.qemu.org/docs/master/system/target-s390x.html)
## Additional Reading
- [Memory Ballooning in Unikernels: QEMU, Firecracker, and oh my!](https://nanovms.com/dev/tutorials/memory-ballooning-unikernels-qemu-firecracker-oh-my)[](https://hocus.dev/blog/qemu-vs-firecracker)
- [Why we migrated from Firecracker to QEMU](https://hocus.dev/blog/qemu-vs-firecracker) and the accompanying [HN discussion](https://news.ycombinator.com/item?id=36666782)[](https://lwn.net/Articles/775736/)
- [The Firecracker virtual machine monitor](https://lwn.net/Articles/775736/)[](https://github.com/firecracker-microvm/firecracker-demo)
- [4000 Firecracker MicroVMs on one server](https://github.com/firecracker-microvm/firecracker-demo)
---
Source: https://e2b.dev/resources/replicating-cursors-agent-mode-with-e2b-and-agentkit.md
Published: 2025-02-24
# Replicating Cursor’s Agent Mode with E2B and AgentKit
Cursor is one of the fastest-growing AI products, completely changing how developers code. Initially released in November 2024, Cursor Agent mode can now read terminal outputs and lint errors and interact with MCP servers, unleashing a new wave of autonomous coding capabilities.
Coding Agents is now the hottest topic in town. In this article, we will see how to replicate Cursor’s Agent mode capabilities by using [E2B Sandboxes](https://docs.e2b.dev) and [AgentKit](https://github.com/inngest/agentkit) and following best practices to build useful Agent tools.
## Our Coding Agent capabilities
Before jumping into the code, let’s list the capabilities from Cursor’s Agent mode that we will replicate. Our Coding Agent, built with AgentKit and E2B, will focus on three main capabilities:
- **Generating a complete working project** from a given prompt
- **Fixing bugs or performing refactoring** on an existing codebase
- **Run terminal commands** to assess the program’s health. For example, run tests
Building a Coding Agent with such capabilities requires some core concepts:
- **E2B Sandboxes** will replace Cursor’s Editor environment, providing a quick, safe environment to interact with
- **The AgentKit library** will allow our Agent to interact with the Sandbox and develop a certain level of autonomy, enabling it to react and learn from its actions.
Let’s now dig into our Coding Agent implementation, layer by layer.
## Building our Coding Agent
Creating an Agent differs from building a RAG workflow or other AI workflows with pre-defined steps.
First, the prompt is no longer used to instruct some “generative” (ex: classify) or transformative (ex: translate) actions but rather to guide the LLM with open-ended instructions to follow.
Also, an Agent requires “context pulling” (the LLM asks for context) versus “context pushing,” which we usually find in RAG workflows. “Context pulling” is achieved by providing the LLM with a set of available tools to interact with its environment to perform actions or retrieve context.
Finally, an Agent relies on an “autonomous loop” in which the LLM decides when the workflow is complete.
Let’s see these patterns in action:
### Shaping the prompt
An Agent prompt follows a structure documented as [ReAct](https://arxiv.org/abs/2210.03629) (Reason and Act). It is usually composed of three parts: Goal, Return Format, Warnings, and Context.
Our Coding Agent, like Cursor, relies on Claude Sonnet 3.5 which excels at coding tasks. Anthropic recommends a slight variation of the prompt format called [CoT](https://docs.anthropic.com/en/docs/build-with-claude/prompt-engineering/chain-of-thought#example-writing-donor-emails-basic-cot) (Chain of Thoughts), resulting in the below prompt:
Our prompt follows the [structured guided Chain of Thoughts format](https://docs.anthropic.com/en/docs/build-with-claude/prompt-engineering/chain-of-thought#example-writing-donor-emails-structured-guided-cot).
```text
You are a coding agent help the user to achieve the described task.
When running commands, keep in mind that the terminal is non-interactive, remind to use the '-y' flag when running commands.
Once the task completed, you should return the following information:
Think step-by-step before you start the task.
```
Our Coding Agent is now implemented as follows with AgentKit’s `createAgent()`:
```typescript
import { createAgent, anthropic } from "@inngest/agent-kit"
const agent = createAgent({
name: "Coding Agent",
description: "An expert coding agent",
system: `You are a coding agent help the user to achieve the described task.
When running commands, keep in mind that the terminal is non-interactive, remind to use the '-y' flag when running commands.
Once the task completed, you should return the following information:
Think step-by-step before you start the task.
`,
model: anthropic({
model: "claude-3-5-sonnet-latest",
max_tokens: 4096,
}),
})
```
Let’s now focus on the most critical part of building Coding Agents: Designing and building tools.
### Building the tools
Providing tools to an Agent to enable it to interact with its environment relies on Function Calling, also called [Tool Calling](https://docs.anthropic.com/en/docs/build-with-claude/tool-use).
Tool Calling is achieved by providing a description of tools (tool name, description, and parameters as JSON Schema) that the LLM can request to use to achieve its goal.
Let’s build tools enabling our Coding Agent to interact with a dedicated E2B Sandbox to [CRUD](https://en.wikipedia.org/wiki/Create,_read,_update_and_delete) on files and run terminal commands, just like Cursor does.
**A naive approach to tools building**
An intuitive approach to building tools can be to directly map our Agent to the API we want to use, for example, with E2B:
- a `runCode` tool, described as “run code in the sandbox” and taking a `code: string` parameter
- a `terminal` tool, described as “run terminal commands” and taking a `command: string` parameter
This approach, while logical, makes it harder for the Agent to achieve its goals as the following behavior rapidly starts to rise:
- The LLM will begin to use the `runCode` tool to run code that will create code files
- Sometimes, it will try to achieve the same goal by going through the `terminal` tool to perform some `echo "..." > /path/to/file.tsx` commands
The above behaviors often lead to some string escape errors, which create endless error loops.
Instead of mapping our Agent tools to the E2B APIs, we will follow the tool design best practices [recently introduced by Anthropic](https://www.youtube.com/watch?v=LP5OCa20Zpg).
**Best practice of tools building**
Designing helpful Agent tools requires us to have some “LLM empathy” by asking ourselves which tools we need to achieve similar capabilities.
By asking ourselves, “What would I need in this situation?” the list of tools will expand with the following tools:
- `createOrUpdateFiles`, described as "Create or update files in the sandbox”
- `readFiles`, described as "Read files from the sandbox”
These two new tools map to the [E2B Sandbox filesystem API](https://docs.e2b.dev/filesystem/read-write).
Here is the `createOrUpdateFiles` created using `createTool()` from AgentKit:
```typescript
import { createAgent, anthropic, createTool } from "@inngest/agent-kit"
const agent = createAgent({
name: "Coding Agent",
description: "An expert coding agent",
system: `You are a coding agent help the user to achieve the described task.
When running commands, keep in mind that the terminal is non-interactive, remind to use the '-y' flag when running commands.
Once the task completed, you should return the following information:
Think step-by-step before you start the task.
`,
model: anthropic({
model: "claude-3-5-sonnet-latest",
max_tokens: 4096,
}),
tools: [
// create or update file
createTool({
name: "createOrUpdateFiles",
description: "Create or update files in the sandbox",
parameters: z.object({
files: z.array(
z.object({
path: z.string(),
content: z.string(),
})
),
}),
handler: async ({ files }, { network }) => {
const sandbox = getSandbox(network);
try {
await Promise.all(
files.map((file) => sandbox?.files.write(file.path, file.content)
);
return `Files created or updated: ${files.map((f) => f.path).join(", ")}`;
} catch (e) {
console.error("error", e);
return "Error: " + e;
}
},
}),
]
})
```
Let’s take a closer look at the createOrUpdateFiles tool:
- `createOrUpdateFiles` is created using [`createTool()`](https://agentkit.inngest.com/reference/create-tool) and assigned to our Agent. Here, AgentKit will handle the tool calling mechanism and forward the tool result back to the LLM.
- **You will notice that errors are also returned to the Agent**. This enables our Agent to react to such errors and call this tool again (with different parameters) or decide to take another action.
- The implementation of `getSandbox()` is in the repository. This helps ensure that a dedicated E2B sandbox runs while [increasing its timeout if necessary](https://docs.e2b.dev/sandbox).
Interestingly, the tools provided to our Coding Agent are similar to the ones available to Cursor’s Agent: terminal commands, Editor actions (read, create files), and code evaluation.
Let’s now make our Coding Agent autonomous by enabling it to react to its thinking process and actions (tools).
### Making it autonomous
AgentKit enables us to add autonomy to our [`Agent`](https://agentkit.inngest.com/concepts/agents) via the concept of a [`Network`](https://agentkit.inngest.com/concepts/networks). Adding our Agent to a Network will allow us to run it in a loop until it completes its goal:
```typescript
import { createAgent, anthropic, createTool, createNetwork } from "@inngest/agent-kit"
const agent = createAgent(/* our Agent and its Tools */);
const network = createNetwork({
name: "coding-agent-network",
agents: [agent],
maxIter: 10,
defaultRouter: ({ network }) => {
if (network?.state.kv.has("task_summary")) {
return;
}
return agent;
},
});
async function main() {
const result = await network.run(process.argv.slice(2).join(" "));
console.log(result.state.kv.get("task_summary"));
}
main();
```
Our Network will run our Agent into a loop, enabling it to react to the output of its tools (e.g., terminal commands) and iterate on the plan created to reach its goals (e.g., “Create a Next.js TodoList project).”
The `defaultRouter` function is called after each iteration of our Agent, enabling us to define when the Agent has completed its goal. If no `task_summary` has been inserted in the state, the Agent will perform a maximum of 10 iterations.
But where does this `task_summary` come from?
It is added to the state when the Agent responds with a `` answer (check the prompt). It is achieved by leveraging the AgentKit Agent lifecycle hooks to update the network state dynamically:
```typescript
const agent = createAgent({
name: "Coding Agent",
description: "An expert coding agent",
// ... prompt and tools
lifecycle: {
onResponse: async ({ result, network }) => {
const lastAssistantMessageText =
lastAssistantTextMessageContent(result);
if (lastAssistantMessageText) {
if (lastAssistantMessageText.includes("")) {
network?.state.kv.set("task_summary", lastAssistantMessageText);
}
}
return result;
},
},
```
Our Coding Agent is now complete!
Its code is [available on GitHub](https://github.com/e2b-dev/e2b-cookbook/tree/main/examples/agentkit-coding-agent/) and can run locally with a valid Anthropic Claude API Key.
## Putting it all together: demos time!
Let’s now look at how our Coding Agent behaves with typical Cursor Agent prompts.
### Scaffold a complete Next.js TodoList with unit tests
After cloning the repository and installing its dependencies with `pnpm i`, let’s run the following prompt:
```bash
pnpm run start "Create a Next.js TodoList demo and its associated unit tests. Finally run the tests with coverage"
```
Our Coding Agent will create a plan and start executing it:
```text
--- Iteration #0 ---
Agent response > undefined
terminal < npx create-next-app@latest todo-list-demo --typescript --tailwind --eslint --app --src-dir --use-npm --yes
terminal result > Creating a new Next.js app in /home/user/todo-list-demo.
[...]
Success! Created todo-list-demo at /home/user/todo-list-demo
--- Iteration #1 ---
Agent response > undefined
terminal < cd todo-list-demo && npm install --save-dev @testing-library/react @testing-library/jest-dom @testing-library/user-event jest jest-environment-jsdom
terminal result >
added 329 packages, and audited 709 packages in 14s
[...]
--- Iteration #2 ---
Agent response > undefined
createOrUpdateFiles < [ 'todo-list-demo/src/components/TodoList.tsx' ]
--- Iteration #3 ---
Agent response > undefined
createOrUpdateFiles < [ 'todo-list-demo/src/components/__tests__/TodoList.test.tsx' ]
--- Iteration #4 ---
Agent response > undefined
createOrUpdateFiles < [ 'todo-list-demo/jest.config.js' ]
--- Iteration #5 ---
Agent response > undefined
createOrUpdateFiles < [ 'todo-list-demo/jest.setup.js' ]
--- Iteration #6 ---
Agent response > undefined
terminal < cd todo-list-demo && npm pkg set scripts.test="jest" scripts.test:coverage="jest --coverage"
terminal result >
--- Iteration #7 ---
Agent response > undefined
createOrUpdateFiles < [ 'todo-list-demo/src/app/page.tsx' ]
--- Iteration #8 ---
Agent response > undefined
terminal < cd todo-list-demo && npm run test:coverage
terminal result >
> todo-list-demo@0.1.0 test:coverage
> jest --coverage
--------------|---------|----------|---------|---------|-------------------
File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
--------------|---------|----------|---------|---------|-------------------
All files | 100 | 85.71 | 100 | 100 |
TodoList.tsx | 100 | 85.71 | 100 | 100 | 28
--------------|---------|----------|---------|---------|-------------------
--- Iteration #9 ---
Agent response > Coding Agent
Now that we have completed all the steps, let me provide a task summary:
[...]
--- Iteration #10 ---
Coding Agent
Now that we have completed all the steps, let me provide a task summary:
1. Created a new Next.js project with TypeScript and Tailwind CSS
2. Installed testing dependencies (Jest, Testing Library)
3. Created the TodoList component with the following features:
- Add new todos
- Toggle todo completion
- Delete todos
- Support for Enter key to add todos
4. Created comprehensive unit tests covering:
- Component rendering
- Adding todos
- Toggling todo completion
- Deleting todos
- Enter key functionality
5. Configured Jest with coverage reporting
6. Executed tests with coverage results:
- Statements: 100%
- Branches: 85.71%
- Functions: 100%
- Lines: 100%
The TodoList demo is now complete and fully tested. The coverage report shows excellent test coverage with only one uncovered branch case.
```
I highly encourage you to [clone the repository](https://github.com/e2b-dev/e2b-cookbook) (examples/agentkit-coding-agent/) and try some different prompts. You can even update the program to load an existing project in the sandbox and ask our Coding Agent to fix some bugs.
## Conclusion
As we saw together, combining E2B and [AgentKit](https://github.com/inngest/agentkit/) only requires a few hundred lines of code to build a competent Coding Agent. The most essential part of creating a Coding Agent (and any kind of Agent) resides in the design of its tools, which consists of putting yourself in the shoes of the LLM.
You might say, “Right, but Cursor also performs searches on the web and can navigate documentations”. The good news is that AgentKit also supports Anthropic’s MCP servers ([Model Context Protocol](https://github.com/modelcontextprotocol)), which could enable our Coding Agent to interact with external tools such as Neon to create databases on the fly or [Stagehand](https://github.com/browserbase/mcp-server-browserbase) to browse the web autonomously:
```typescript
import { createAgent, anthropic } from "@inngest/agent-kit"
const agent = createAgent({
name: "Coding Agent",
description: "An expert coding agent",
// prompt, model configuration
mcpServers: [
// enables our Coding Agent to browse the web
{
name: "stagehand",
transport: {
type: "sse",
url: "http://localhost:3005",
requestInit: {
headers: {
"BROWSERBASE_API_KEY": "",
"BROWSERBASE_PROJECT_ID": "browserbase-project-id",
"OPENAI_API_KEY": "openai-api-key",
},
},
},
},
],
})
```
By configuring MCP servers to this Coding Agent, you’ll get an experience similar to the current version of Cursor Agent mode.
Finally, AgentKit will soon support parallel tools, providing an experience similar to Cursor Agent, which allows you to edit multiple files simultaneously.
If you enjoy this Coding Agent experiment, check [the E2B Fragment project](https://github.com/e2b-dev/fragments), an open-source alternative to v0, Cursor, and Stackblitz.
---
Source: https://e2b.dev/resources/scaling-firecracker-using-overlayfs-to-save-disk-space.md
Published: 2025-02-12
# Scaling Firecracker: Using OverlayFS to Save Disk Space
If you want to run a lot of [Firecracker](https://firecracker-microvm.github.io/) instances like [E2B](https://docs.e2b.dev) does, you will quickly realize that copying the root filesystem for each instance is not the best idea. Even with small root filesystems like [Alpine Linux](https://www.alpinelinux.org/), you will probably have a few hundred megabytes of data to copy for each instance. Now multiply that with a few thousand instances and you will run into some serious space issues. Of course there is a Linux solution for this problem: OverlayFS. This blog post will explain OverlayFS and the copy-on-write (COW) technique and how to use it with Firecracker!
## OverlayFS and Copy-on-Write (COW)
[OverlayFS](https://docs.kernel.org/filesystems/overlayfs.html) is a Linux filesystem that lets you layer one filesystem on top of another, creating a merged view of both.
Imagine you have two directories: one that you can read from and write to (the upper directory), and another that you can only read from (the lower directory). The upper directory starts completely empty and will only contain files that are modified or added after mounting. When you combine these using OverlayFS, it creates a merged directory that looks like all the files from both directories are in one place. This is super handy because you don't need to copy all the files from the lower directory; you just reference them. This is what you see in the diagram, where files 'a.txt', 'b.txt', and 'c.txt' from both directories appear together in the merged directory, even though they might only exist in one of the directories.
And that's what we want to use with Firecracker! The lower directory is the read-only root filesystem (for example, the Alpine Linux root filesystem), and the upper directory is the writable layer that we can modify (for example, the user's code).
When you want to write or modify a file from the lower directory, OverlayFS uses a technique called [copy-on-write](https://en.wikipedia.org/wiki/Copy-on-write). Instead of changing the original file, it makes a copy of that file in the upper directory and applies the changes to this copy. So, if you modify 'b.txt' which comes from the lower directory, a copy of 'b.txt' is made in the upper directory where the changes are made, leaving the original in the lower directory untouched. This ensures the integrity of the base system and reduces the amount of data that needs to be copied.
Now that we understand the basics, let's see how we can use OverlayFS in Firecracker!
## How to Use OverlayFS in Firecracker
At this point I assume you already have a basic Firecracker setup running. If not, I'd recommend following the [Firecracker Init Lab](https://github.com/alexellis/firecracker-init-lab) or the [official getting started guide](https://github.com/firecracker-microvm/firecracker/blob/main/docs/getting-started.md) to get started.
### Step 1: Create read-only Root Filesystem
We'll start by creating a read-only root filesystem, which I assume you already have at `rootfs.ext4`.
First, create a new directory to modify the root filesystem and then mount the root filesystem to it:
```bash
mksquashfs /tmp/rootfs rootfs.ext4 -noappend
```
After mounting it we need to add a few new directories and most importantly, the `overlay-init`
```bash
umount /tmp/rootfs
rm -rf /tmp/rootfs
```
The overlay-init script is a simple script that sets up the overlay filesystem and then calls the actual init process. You can either build that yourself or use a [slightly modified version](https://github.com/njapke/overlayfs-in-firecracker/blob/main/overlay-init) from containerd:
```bash
#!/bin/sh
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"). You may
# not use this file except in compliance with the License. A copy of the
# License is located at
#
# http://aws.amazon.com/apache2.0/
#
# or in the "license" file accompanying this file. This file is distributed
# on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
# express or implied. See the License for the specific language governing
# permissions and limitations under the License.
# Parameters:
# 1. rw_root -- path where the read/write root is mounted
# 2. work_dir -- path to the overlay workdir (must be on same filesystem as rw_root)
# Overlay will be set up on /mnt, original root on /mnt/rom
pivot() {
local rw_root work_dir
rw_root="$1"
work_dir="$2"
/bin/mount \
-o noatime,lowerdir=/,upperdir=${rw_root},workdir=${work_dir} \
-t overlay "overlayfs:${rw_root}" /mnt
pivot_root /mnt /mnt/rom
}
# Overlay is configured under /overlay
# Global variable $overlay_root is expected to be set to either:
# "ram", which configures a tmpfs as the rw overlay layer (this is
# the default, if the variable is unset)
# - or -
# A block device name, relative to /dev, in which case it is assumed
# to contain an ext4 filesystem suitable for use as a rw overlay
# layer. e.g. "vdb"
do_overlay() {
local overlay_dir="/overlay"
if [ "$overlay_root" = ram ] ||
[ -z "$overlay_root" ]; then
/bin/mount -t tmpfs -o noatime,mode=0755 tmpfs /overlay
else
/bin/mount -t ext4 "/dev/$overlay_root" /overlay
fi
mkdir -p /overlay/root /overlay/work
pivot /overlay/root /overlay/work
}
# If we're given an overlay, ensure that it really exists. Panic if not.
if [ -n "$overlay_root" ] &&
[ "$overlay_root" != ram ] &&
[ ! -b "/dev/$overlay_root" ]; then
echo -n "FATAL: "
echo "Overlay root given as $overlay_root but /dev/$overlay_root does not exist"
exit 1
fi
do_overlay
# invoke the actual system init program and proceed with the boot
# process.
exec /sbin/init $@
```
Theres not a lot of magic going on here, the overlay-init script just sets up the overlay filesystem and then calls the actual init process.
Once thats done, we are going to create a [Squashfs](https://docs.kernel.org/filesystems/squashfs.html) image of the rootfs directory. Squashfs is a compressed read-only filesystem that is very fast to load and can be mounted as a read-only filesystem.
```bash
mksquashfs /tmp/rootfs rootfs.ext4 -noappend
```
Finally, you can unmount the rootfs and remove the temporary directory:
```bash
umount /tmp/rootfsrm -rf /tmp/rootfs
```
### Step 2: Create Writable Overlay
Now if you want to persist the overlay filesystem you will have to create a new ext4 image. If you want a 5GB large filesystem, you can create it with the following command:
```bash
dd if=/dev/zero of=overlay.ext4 bs=1M count=5120
```
The cool thing here is that this is a sparse file. If you run `ls -lh overlay.ext4` you will see that it says `5.0G`. Only if you run `du -h overlay.ext4` you will see that its actually 0 bytes big. Only when you add files to it, it will start taking up space!
Now we need to format the filesystem with ext4:
```bash
mkfs.ext4 overlay.ext4
```
That's it! You will have to repeat the step for your overlay filesystem for each instance, but not for the root filesystem. Next we can actually configure Firecracker to use the overlay filesystem.
### Step 3: Run Firecracker with OverlayFS
Configuring your Firecracker instance to use OverlayFS is pretty straightforward. Instead of only specifying the root drive, you're going to add the overlay drive as well.
The config for your first drive will look like this:
```json
{
"drive_id": "rootfs",
"path_on_host": "rootfs.img",
"is_root_device": true,
"partuuid": null,
"is_read_only": true,
"cache_type": "Unsafe",
"rate_limiter": null
}
```
And your second drive will look like this:
```json
{
"drive_id": "overlayfs",
"path_on_host": "overlay.ext4",
"is_root_device": false,
"partuuid": null,
"is_read_only": false,
"cache_type": "Unsafe",
"rate_limiter": null
}
```
Lastly, you need to tell your kernel to use the overlay-init and set the overlay\_root. For that, simply add``
```bash
init=/sbin/overlay-init overlay_root=/vdb
```
to the kernel args. `vdb` is the identifier for your second drive (vda would be the first read-only root drive. If you have more than two drives it might be `vdc`, `vdd`, etc.). If you don't want to have persistent storage, you can use a ramdisk instead of a disk image with `overlay_root=ram`.
## Conclusion
Now, instead of copying a root filesystem of a few hundred megabytes for each instance, you can use OverlayFS to share the same root filesystem across multiple instances. This saves a lot of space and time, making it easier to manage a lot of instances, and especially useful for running a lot of small instances like **E2B** does.
## Sources
- [OverlayFS](https://en.wikipedia.org/wiki/OverlayFS)
- [Squashfs](https://docs.kernel.org/filesystems/squashfs.html)
- [Copy-on-write](https://en.wikipedia.org/wiki/Copy-on-write)
- [Firecracker with OverlayFS](https://github.com/firecracker-microvm/firecracker/discussions/3061)
- [OverlayFS in Firecracker](https://github.com/njapke/overlayfs-in-firecracker)
- [Firecracker Demo](https://github.com/firecracker-microvm/firecracker-demo)
---
Source: https://e2b.dev/resources/how-i-taught-an-ai-to-use-a-computer.md
Published: 2025-01-20
# How I taught an AI to use a computer
#### Table of contents
- [An open source computer use agent](#an-open-source-computer-use-agent)
- [Technical Challenges](#technical-challenges)
- [Challenge 1: Security](#challenge-1-security)
- [Challenge 2: Clicking on things](#challenge-2-clicking-on-things)
- [Challenge 3: Reasoning](#challenge-3-reasoning)
- [Digression: Agent frameworks are mostly useless](#digression-agent-frameworks-are-mostly-useless)
- [Challenge 4: Deploying Niche LLMs](#challenge-4-deploying-niche-llms)
- [Challenge 5: Streaming the display](#challenge-5-streaming-the-display)
- [Thoughts on the future](#thoughts-on-the-future)
- [APIs and Accessibility APIs](#apis-and-accessibility-apis)
- [Authentication and Sensitive Information](#authentication-and-sensitive-information)
- [Conclusion](#conclusion)
## An open source computer use agent
I made this! It’s an LLM-powered tool that can use all the functionalities of a personal computer.
It takes a command like “Search the internet for cute cat pictures” and uses LLM-based reasoning to operate the mouse and keyboard of the computer on autopilot.
How is this different than other tools that exist already? It’s [**fully open source**](https://github.com/e2b-dev/secure-computer-use/) and uses **only open weight models**. That means that anyone can run and modify my project in any way.
The computer use agent is a work in progress and has limited accuracy, but is showing noticeable improvement every few days. In this article, I’ll give you a tour of how it works. The short explanation is as follows:
The agent **takes many screenshots and asks Meta’s Llama 3.3 LLM what to do next** (click, type, etc.) until the response is that that the task is finished.
Technically, there are a few more components in the system. Here’s an in-depth flow chart of the program and **all of the critical components**:
This schematic, of course, is just a snapshot of what I have right now, which took me about a month to develop. The LLMs and tools in the diagram will rapidly change as I experiment.
## Technical Challenges
To solve this, I had some pretty daunting challenges.
1. **Security**: Isolating the operating system in a safe, controlled environment
2. **Clicking on things**: Enabling the AI to click precisely to manipulate UI elements
3. **Reasoning**: Enabling the AI to decide what to do next (or when to stop) based on what it sees
4. **Deploying niche LLMs**: Hosting open source models, specifically OS-Atlas, in a cost-effective way
5. **Streaming the display**: Finding a low latency way to show and record video of the sandbox
### Challenge 1: Security
The ideal environment to run an AI agent should be easy to use, performant, and secure. Giving an AI agent direct access to your personal computer and file system is dangerous! It could delete files, or perform other irreversible actions.
Rather than give the agent access to my computer, I used [**E2B**](https://e2b.dev/). E2B is a cloud platform that provides secure sandboxes meant to augment AI agents. It’s most common use-case is to run Python code (to generate Perplexity’s charts, for example) but it now supports running a full-fledged Ubuntu system with GUI applications. Thus, it’s perfect for this project.
### Challenge 2: Clicking on things
Now, we’re getting to the fun part. LLM-based “computer use” is fairly straightforward when the interface is text-based, and you can get far with just text-based commands.
However, there are some applications you will basically never be able to use without a mouse. Thus, for a comprehensive computer use agent, we need this feature.
I was also not satisfied with solutions that used traditional computer vision models as a “bridge” between the screen and LLM. They did great for recognizing text and some icons, but they had no idea what was a text field vs. a button or some other element.
Then, I came upon some promising research out of China on building “grounded VLMs.” This is a vision LLM with the ability to output precise coordinates referencing the input image. Both Gemini and Claude have this ability, but are neither are open source nor published. The OS-Atlas team, on the other hand, has published their weights on Hugging Face and outlined [**the fascinating training process in this paper**](https://arxiv.org/pdf/2410.23218).
### Challenge 3: Reasoning
The power of LLM-based agents is that they can decide between multiple actions, and make educated decisions using the most recent information.
Over the past year, we’ve seen a gradual increase in LLMs’ abilities to make these decisions. The first approach was to simply prompt the LLM to output actions in a given text format, and to add the result of the action to the chat history before calling the LLM again. All following approaches have been roughly the same, with fine-tuning used to compliment the system prompts. This general ability was called **function calling**, while the term **tool-use** is now more popular.
The combination of vision to inform tool-use in a single LLM call is a fairly new thing that hasn’t seen much mileage yet. I tried a few different open source models to get this, and I’ll summarize the following part briefly since it will probably be outdated in a couple of weeks anyway. In my agent, I used:
- **Llama-3.2-90B-Vision-Instruct** to view the sandbox display, and decide on next steps to take
- **Llama 3.3-70B-Instruct** to take the decision from Llama 3.2 and rephrase it in tool-use format
- **OS-Atlas-Base-7B** as a tool that can be called by the agent to perform a click action given a prompt of what to click
### Digression: Agent frameworks are mostly useless
If you’ve looked into building AI agents, you’ve probably asked the question: Why are there so many frameworks out there?
In my personal experience, the utility of these frameworks is to abstract 1) LLM input formatting and output parsing, 2) the agent prompts and 3) the agent run loop. Since I want to keep my run loop very simple, the main use in a framework would be to handle the interface with the LLM provider, especially for tool-use and images. However, most providers are now standardizing towards the OpenAI tool-use format **anyways**, and when there are exceptions it’s often not clear from the documentation if the framework handles them. And as for the system prompts, I really **don’t** want this to be abstracted, since this is one part of the code I need to adjust all the time.
If you’ve had a different experience than the above—That’s cool, I’d love to hear your thoughts!
Also, one big lesson that I have learned about tool use is that it’s not really a single feature. It’s a whole hodgepodge of LLM fine-tuning, various prompts, and string formatting and parsing on either the API side or on the client side. It is just so hard to make a framework (and keep it updated) to fit together all these parts without the developer needing to look inside.
### Challenge 4: Deploying Niche LLMs
Since I want my agent to run fast, I wanted to run the LLM inference in the cloud. I also wanted it to work out-of-the-box for curious people like yourself.
Unfortunately, this was much easier said than done. There are numerous inference hosting providers, and they all have there different points of friction. Fortunately, Llama 3.2 and 3.3 are fairly common, and I found OpenRouter, Fireworks AI, and the official Llama API to be pretty good options. They all provide “serverless” hosting, which essentially means that you only pay marginal costs and no fixed costs.
But, there were no such options for OS-Atlas. I reached out to a number of inference providers, and what I eventually learned is that economies of scale make it prohibitive for hosts to put out serverless versions of infrequently used models. With few users, it’s hard for them to distribute the costs of the hosting and the engineering time amongst these users.
I ended up using a free Hugging Face Space to call OS-Atlas. This is relatively slow (takes a few seconds for each call) and is rate-limited (a few dozen calls per hour) but it gets the job done for now.
### Challenge 5: Streaming the display
In order to see what the AI is doing, we want to get live updates from the Sandbox’s screen. I wondered if I do this using ffmpeg. After bashing random shell commands for a while, I found the right magic incantations:
Server: `ffmpeg -f x11grab -s 1024x768 -framerate 30 -i $DISPLAY -vcodec libx264 -preset ultrafast -tune zerolatency -f mpegts -listen 1 http://localhost:8080`
Client: `ffmpeg -reconnect 1 -i http://servername:8080 -c:v libx264 -preset fast -crf 23 -c:a aac -b:a 128k -f mpegts -loglevel quiet - | tee output.ts | ffplay -autoexit -i -loglevel quiet -`
The first command basically creates a video streaming server over HTTP which can stream to one client at a time. The second command captures the stream, and simultaneously writes it to a .ts file, and displays it in a GUI.
This works fine over the internet. The server is some kind of built in feature of FFmpeg, but has the limitation that it can only stream to one client at a time. Therefore, the client must use the tee command to split the stream so it can be both saved and displayed. (Please don’t ask me anything about codecs or any of the other flags up there!) In the future, the plan is to either reduce the latency of the stream or replace it entirely with a VNC connection.
## Thoughts on the future
This has mainly been an article of what I built with the tools available to me. A major goal of the project was that it is operating system agnostic, application agnostic, and to the extent possible, LLM agnostic. This worked, although
### APIs and Accessibility APIs
One recurring theme that came up in my work was the question of whether computer use agents in general should lean more heavily on APIs (coded pathways) or GUI only (pure vision). The answer is clearly: Agents **should** make use of APIs as much as possible, but most software is just not made to be controlled this way.
That’s why in my testing, I wanted to make sure the agent could open a web browser, click on the URL bar, type some text, etc., even though there is an equivalent shell command that can do the same thing. When designing a computer use agent, we should also consider the non-visual interfaces that are available to us, and here are a few:
1. **Standard APIs**: These include APIs such as the file system API, the Microsoft Office API, or the Gmail REST API, which provide structured access to useful functionalities.
2. **Code Execution**: This involves running scripts or commands, such as executing Bash or Python code to launch an application or parse the contents of a file.
3. **Accessibility APIs**: The OS or desktop environment often provides accessibility APIs that allow direct interaction with the GUI hierarchy. Unfortunately, support on Linux tends to be worse than macOS or Windows.
4. **Document Object Model (DOM**): The DOM enables interaction with web pages in a semi-structured, text-based manner.
5. **Model Context Protocol (MCP**): The Model Context Protocol is a is a newly introduced API specifically designed to both provide context and actions in an agent-friendly manner.
Given the number of options, it’s somewhat of a tragedy that we have to rely on vision which is a far more burdensome task for an AI. This is especially true for #3, since better accessibility APIs would be beneficial for many (vision impaired) humans as well. It would be amazing if everything could work like Zapier, where everything is connected with the right adapters. We can only hope!
### Authentication and Sensitive Information
Another huge open question is how to securely handle authentication. The **insecure approach** would be to provide the agent with the same level of access as the user. A **secure approach would be to scope permissions**, as is commonly used by OAuth apps, iOS apps, etc. such as the example below:
In our agent we’ve avoided this problem entirely by creating a fresh, isolated sandbox with no user data or credentials. But this also doesn’t solve the problem.. If a secure approach isn’t available to users, they tend to create an insecure one. Therefore, it’s important to already start thinking about the following:
- Ways to provide computer use agents with **scoped access to APIs**: For example, a computer use agent uses a traditional API to view the user’s email inbox without the ability to delete or send emails
- Ways to **redact sensitive information** passed to the LLM, and restore it in the LLM output: For example, a user can set secrets, such as CREDIT\_CARD\_NUMBER, which can be passed to tools but not seen by the LLM
## Conclusion
The AI computer use agent I made is a prototype that can use the computer about as well as I could when I was five or six. It still has a lot of trouble planning next steps and often doesn’t know where to focus its attention on the screen. For example, it may not notice if a text field is selected or not, or it may lose sight of the original goal when presented with a full screen of text. This is not at all surprising for an LLM.
That said, reasoning with vision is an area where we expect to see a lot of improvement in open source models on a monthly basis, and even while I’ve been writing this article, new models have been released that I’m excited to try out. Meanwhile, I’m also excited to augment the agent’s abilities by adding additional APIs to the agent’s toolbox.
If this is a problem that’s interesting to you, check out [**the source code**](https://github.com/e2b-dev/secure-computer-use/) and [**reach out to me**](https://www.linkedin.com/in/jamesmurdza/).
---
Source: https://e2b.dev/resources/typescript-code-interpreter-with-o1-and-gpt-4o.md
Published: 2024-09-19
# Code interpreter with o1 and GPT-4o
This AI data scientist can take any dataset - including a messy, unsanitized one - and design and train a supervised machine learning model from it. In this example, we'll take Kaggle's [Titanic: Machine Learning from Disaster challenge](https://www.kaggle.com/competitions/titanic), and ask the AI data scientist to train a decision tree which predicts based on various characteristics of the passengers which passengers will survive the sinking of the Titanic.
We'll do this by using o1, OpenAI's powerful new reasoning model in conjunction with GPT-4o and E2B's Code Interpreter SDK. We'll use the following components:
1. [o1-mini](https://openai.com/index/introducing-openai-o1-preview/) to generate a detailed plan with multiple code blocks.
2. [GPT-4o-mini](https://openai.com/index/gpt-4o-mini-advancing-cost-efficient-intelligence/) to extract and consolidate one final code block from the plan.
3. [E2B Code Interpreter SDK](https://github.com/e2b-dev/code-interpreter): To run the code in a secure cloud sandbox.
In summary, we use o1 to generate the code, and E2B's SDK to run the code and return the results. The code processes the data in the uploaded CSV files, cleans the data, and architects and trains a machine learning model, displaying a chart of the model's performance.
**Full code for this guide** Find the full code in [our cookbook on GitHub](https://github.com/e2b-dev/e2b-cookbook/tree/main/examples/openai-js).
### Key links
- [E2B Docs](https://docs.e2b.dev)
- [E2B Code Interpreter SDK](https://github.com/e2b-dev/code-interpreter)
- [E2B Cookbook](https://github.com/e2b-dev/e2b-cookbook/tree/main)
- [Follow E2B](https://x.com/e2b_dev)
- [Get started with OpenAI API](https://platform.openai.com/docs/overview)
- [Introducing o1](https://openai.com/index/introducing-openai-o1-preview/)
### Outline
1. Prerequisites
2. Install the SDKs
3. Set up API keys and model instructions
4. Add code interpreting capabilities and initialize the models
5. Upload the dataset
6. Put everything together
7. Run the program and see the results
8. Resources
## Setup
### 1. Prerequisites
Ensure you have **Node.js** installed on your system. Obtain your API keys from OpenAI and E2B: you can get your OpenAI API key [here](https://platform.openai.com/account/api-keys) and your E2B API key [here](https://docs.e2b.dev).
Download the Titanic dataset (both `train.csv` and `test.csv`) from [Kaggle](https://www.kaggle.com/competitions/titanic/data) and place the files in the same directory as your TypeScript script. Create a `.env` file in your project directory and add your API keys as follows:
```bash
OPENAI_API_KEY=your_openai_api_key
E2B_API_KEY=your_e2b_api_key
```
### 2. Install the SDKs
Install the required packages using `npm`:
```bash
npm install
```
### 3. Set up the API keys and model instructions
Import necessary libraries and load your API keys from the `.env` file:
```typescript
import fs from "node:fs";
import { OpenAI } from "openai";
import { CodeInterpreter, Result } from "@e2b/code-interpreter";
import { ProcessMessage } from "@e2b/code-interpreter";
import * as dotenv from "dotenv";
dotenv.config();
```
Two prompts are defined, one for o1 and one for GPT-4o. The o1 prompt assigns a data scientist role and explains the schema of the uploaded CSV. If you choose different data, you need to update the prompt accordingly.
```typescript
const O1_PROMPT = `
You're a data scientist analyzing survival data from the Titanic Disaster. You are given tasks to complete and you run Python code to solve them.
Information about the Titanic dataset:
- It's in the \`/home/user/train.csv\` and \`/home/user/test.csv\` files
- The CSV files are using \`,\` as the delimiter
- They have following columns:
- PassengerId: Unique passenger ID
- Pclass: 1st, 2nd, 3rd (Ticket class)
- Name: Passenger name
- Sex: Gender
- Age: Age in years
- SibSp: Number of siblings/spouses aboard
- Parch: Number of parents/children aboard
- Ticket: Ticket number
- Fare: Passenger fare
- Cabin: Cabin number
- Embarked: Port of Embarkation (C = Cherbourg, Q = Queenstown, S = Southampton)
Generally, you follow these rules:
- ALWAYS FORMAT YOUR RESPONSE IN MARKDOWN
- ALWAYS RESPOND ONLY WITH CODE IN CODE BLOCK LIKE THIS:
\`\`\`python
{code}
\`\`\`
- the python code runs in jupyter notebook.
- every time you generate python, the code is executed in a separate cell. it's okay to multiple calls to \`execute_python\`.
- display visualizations using matplotlib or any other visualization library directly in the notebook. don't worry about saving the visualizations to a file.
- you have access to the internet and can make API requests.
- you also have access to the filesystem and can read/write files.
- install all packages before using by running \`!pip install {package}\`.
- you can run any python code you want, everything is running in a secure sandbox environment.
`;
```
The GPT-4o prompt is assigned the role of a software engineer, generating a complete, single block of code from the plan.
```typescript
const GPT_4O_PROMPT = `
You are an expert software engineer. Based on the execution plan you receive, you will create a single Python script that does everything in the plan. It will be executed in a single Python notebook cell.
`;
```
### 4. Add code interpreting capabilities and initialize the models
We first define helper functions to extract the code outputted from GPT-4o, the final model used. This function extracts all code between \`\`\`typescript markers, which are called code fences and used by GPT to delimit the code.
```typescript
// Function to extract code blocks from a response
function matchCodeBlocks(llmResponse: string): string {
const regex = /```python\n([\s\S]*?)```/g;
let matches: string[] = [];
let match: RegExpExecArray | null;
while ((match = regex.exec(llmResponse)) !== null) {
if (match[1]) {
matches.push(match[1]);
}
}
if (matches.length > 0) {
const code = matches.join("\n");
console.log("> LLM-generated code:");
console.log(code);
return code;
}
return "";
}
```
Now, we set up the chain which uses both of the above prompts to first generate the code, then consolidate it. We also initialize the OpenAI client. The chat function takes care of the interaction with the LLM. It calls the E2B code interpreter anytime there is code to be run.
```typescript
// Function to interact with both models: o1 and gpt-4o
async function chat(
codeInterpreter: CodeInterpreter,
userMessage: string,
): Promise {
console.log(
`\n${"=".repeat(50)}\nUser Message: ${userMessage}\n${"=".repeat(50)}`,
);
// First, get the plan from o1-mini
try {
const responseO1 = await openai.chat.completions.create({
model: "o1-mini", // Assuming this model is accessible
messages: [
{ role: "user", content: O1_PROMPT },
{ role: "user", content: userMessage },
],
});
const contentO1 = responseO1.choices[0].message.content;
if (contentO1 === null) {
throw Error(`Chat content is null.`);
}
// Then, use gpt-4o to extract code
const response4o = await openai.chat.completions.create({
model: "gpt-4o",
messages: [
{ role: "system", content: GPT_4O_PROMPT },
{ role: "user", content: `${GPT_4O_PROMPT}\n${contentO1}` },
],
});
const content4o = response4o.choices[0].message.content;
if (content4o === null) {
throw Error(`Chat content is null.`);
}
console.log("Code from gpt-4o:", content4o);
// Extract Python code from the gpt-4o response
const pythonCode = matchCodeBlocks(content4o);
if (pythonCode == "") {
throw Error(`Failed to match any Python code in model's response:\n${content4o}`);
}
// Run the Python code using the code interpreter
const codeInterpreterResults = await codeInterpret(
codeInterpreter,
pythonCode,
);
return codeInterpreterResults;
} catch (error) {
console.error("Error when running code interpreter:", error);
return [];
}
}
```
Now we define the function that will use the code interpreter by E2B. Everytime the LLM assistant decides that it needs to execute code, this function will be used. Read more about the Code Interpreter SDK [here](https://docs.e2b.dev/quickstart).
```typescript
// Function to run the Python code using the code interpreter
async function codeInterpret(
codeInterpreter: CodeInterpreter,
code: string,
): Promise {
console.log("Running code interpreter...");
const exec = await codeInterpreter.notebook.execCell(code, {
onStderr: (msg: ProcessMessage) =>
console.log("[Code Interpreter stderr]", msg),
onStdout: (stdout: ProcessMessage) =>
console.log("[Code Interpreter stdout]", stdout),
});
if (exec.error) {
console.log("[Code Interpreter ERROR]", exec.error);
throw new Error(exec.error.value);
}
return exec.results;
}
```
### 5. Upload the dataset
The CSV data files are uploaded to the code interpreter before running any code. The file paths must match the CSV files in the directory. In this case, because we are tasking the AI with a supervised learning task, we have both a testing and training dataset. Read more about the file upload [here](https://docs.e2b.dev/filesystem/upload).
```typescript
// Function to upload Kaggle dataset files
async function uploadDataset(codeInterpreter: CodeInterpreter) {
console.log(
"Uploading testing and training datasets to Code Interpreter sandbox...",
);
const testCsv = fs.readFileSync("./test.csv");
const testCsvPath = await codeInterpreter.uploadFile(testCsv, "test.csv");
console.log("Uploaded test.csv at", testCsvPath);
const trainCsv = fs.readFileSync("./train.csv");
const trainCsvPath = await codeInterpreter.uploadFile(trainCsv, "train.csv");
console.log("Uploaded train.csv at", trainCsvPath);
}
```
### 6. Put everything together
Finally, we put everything together and let the AI data scientist upload the data, run an analysis, and generate a PNG file with a chart. You can update the task for the assistant in this step. For example, you can ask it to visualize the decision tree or a confusion matrix instead of the learning curve. If you decide to change the dataset you are using, don't forget to update the prompt too.
```typescript
async function run() {
const codeInterpreter = await CodeInterpreter.create();
try {
// Upload the Titanic dataset to the sandbox
await uploadDataset(codeInterpreter);
// Let the model analyze the dataset
const codeInterpreterResults = await chat(
codeInterpreter,
"Clean the data, train a decision tree to predict the survival of passengers, and visualize the learning curve. Then run the model on the test dataset and print the results.",
);
console.log("codeInterpreterResults:", codeInterpreterResults);
if (codeInterpreterResults.length > 0) {
const result = codeInterpreterResults[0];
console.log("Result object:", result);
// Handle the result, e.g., save any generated images
if (result && result.png) {
fs.writeFileSync("result.png", Buffer.from(result.png, "base64"));
} else {
console.log("No image data available.");
}
} else {
console.log("No results returned.");
}
} catch (error) {
console.error("An error occurred:", error);
} finally {
await codeInterpreter.close();
}
}
```
### 7. Run the program and see the results
When you run the script, the following steps will occur:
1. **Dataset Upload**: The Titanic datasets (`train.csv` and `test.csv`) are uploaded to the sandbox environment.
2. **Code Generation**: The `o1-mini` model generates a detailed plan with code blocks to perform data cleaning, model training, and visualization.
3. **Code Extraction**: The `gpt-4o-mini` model extracts the final executable TypeScript code from the plan.
4. **Code Execution**: The extracted code is executed in the E2B Code Interpreter sandbox.
5. **Results**: The model trains a decision tree classifier, visualizes the learning curve, and prints predictions on the test dataset.
Example Output:
```text
Uploading testing and training datasets to Code Interpreter sandbox...
Uploaded at /home/user/train.csv
==================================================
User message: Clean the data, train a decision tree to predict the survival of passengers, and visualize the learning curve.
==================================================
> LLM-generated code:
# Install necessary packages
!pip install pandas numpy matplotlib seaborn scikit-learn
# Import libraries
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import train_test_split, learning_curve
from sklearn.preprocessing import LabelEncoder
from sklearn.metrics import accuracy_score
# Load the data (Make sure to replace these paths with the correct ones)
train = pd.read_csv('/home/user/train.csv')
test = pd.read_csv('/home/user/test.csv')
# Display first few rows
print(train.head())
# Data Cleaning
# Combine train and test for consistent preprocessing
combined = pd.concat([train, test], sort=False)
# ... [code continues] ...
```
[](https://github.com/e2b-dev/e2b-cookbook/tree/main/examples/openai-js)[**Full code for this guide**](https://github.com/e2b-dev/e2b-cookbook/tree/main/examples/openai-js)
Find the full code in [our cookbook on GitHub](https://github.com/e2b-dev/e2b-cookbook).
#### Resources
- [E2B Docs](https://docs.e2b.dev)
- [E2B Code Interpreter SDK](https://github.com/e2b-dev/code-interpreter)
- [E2B Cookbook](https://github.com/e2b-dev/e2b-cookbook/tree/main)
- [Follow E2B](https://x.com/e2b_dev)
- [Get started with OpenAI API](https://platform.openai.com/docs/overview)
- [Introducing o1](https://openai.com/index/introducing-openai-o1-preview/)
---
Source: https://e2b.dev/resources/python-code-interpreter-with-o1-and-gpt-4o.md
Published: 2024-09-18
# Python code interpreter with o1 and GPT-4o
This AI data scientist can take any dataset - including a messy, unsanitized one - and design and train a supervised machine learning model from it. In this example, we'll take Kaggle's [Titanic: Machine Learning from Disaster challenge](https://www.kaggle.com/competitions/titanic), and ask the AI data scientist to train a decision tree which predicts based on various characteristics of the passengers which passengers will survive the sinking of the Titanic.
We'll do this by using o1, OpenAI's powerful new reasoning model in conjunction with GPT-4o and E2B's Code Interpreter SDK. We'll use the following components:
1. [o1-mini](https://openai.com/index/introducing-openai-o1-preview/) to generate a detailed plan with multiple code blocks.
2. [GPT-4o-mini](https://openai.com/index/gpt-4o-mini-advancing-cost-efficient-intelligence/) to extract and consolidate one final code block from the plan.
3. [E2B Code Interpreter SDK](https://github.com/e2b-dev/code-interpreter): To run the code in a secure cloud sandbox.
In summary, we use o1 to generate the code, and E2B's SDK to run the code and return the results. The code processes the data in the uploaded CSV files, cleans the data, and architects and trains a machine learning model, displaying a chart of the model's performance.
**Find the full code in our cookbook on GitHub**.
#### Key links
- [E2B Docs](https://docs.e2b.dev)
- [E2B Code Interpreter SDK](https://github.com/e2b-dev/code-interpreter)
- [E2B Cookbook](https://github.com/e2b-dev/e2b-cookbook/tree/main)
- [Follow E2B](https://x.com/e2b_dev)
- [Get started with OpenAI API](https://platform.openai.com/docs/overview)
- [Introducing o1](https://openai.com/index/introducing-openai-o1-preview/)
#### Outline
1. Prerequisites
2. Install the SDKs
3. Set up API keys and model instructions
4. Add code interpreting capabilities and initialize the models
5. Upload the dataset
6. Put everything together
7. Run the notebook and see the results
8. Resources
## Setup
### 1. Prerequisites
Ensure you have Python 3.8+ installed on your system. You will also need Jupyter Notebook or JupyterLab to run the notebook. Obtain your API keys from OpenAI and E2B: You can get your OpenAI API key [here](https://platform.openai.com/account/api-keys) and your E2B API key [here](https://docs.e2b.dev).
Download the Titanic dataset (both `train.csv` and `test.csv` ) from [Kaggle](https://www.kaggle.com/competitions/titanic/data) and place the files in the same directory as your notebook. Create a `.env` file in your project directory and add your API keys as follows:
```bash
OPENAI_API_KEY=your_openai_api_key
E2B_API_KEY=your_e2b_api_key
```
### 2. Install the SDKs
Install the required packages using pip:
```python
%pip install openai=1.45.1 e2b_code_interpreter=0.0.10 python-dotenv==1.0.1
```
### 3. Set up the API keys and model instructions
Import necessary libraries and load your API keys from the `.env` file:
```python
import os
from dotenv import load_dotenv
load_dotenv()
# Get your API keys
OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")
E2B_API_KEY = os.getenv("E2B_API_KEY")
```
Two prompts are defined, one for o1 and one for GPT-4o. The o1 prompt assigns a data scientist role and explains the schema of the uploaded CSV. If you choose different data you need to update the prompt accordingly.
````python
O1_PROMPT = """You're a data scientist analyzing survival data from the Titanic Disaster. You are given tasks to complete and you run Python code to solve them.
Information about the Titanic dataset:
- It's in the `/home/user/train.csv` and `/home/user/test.csv` files
- The CSV files use `,` as the delimiter
- They have the following columns:
- PassengerId: Unique passenger ID
- Pclass: 1st, 2nd, 3rd (Ticket class)
- Name: Passenger name
- Sex: Gender
- Age: Age in years
- SibSp: Number of siblings/spouses aboard
- Parch: Number of parents/children aboard
- Ticket: Ticket number
- Fare: Passenger fare
- Cabin: Cabin number
- Embarked: Port of Embarkation (C = Cherbourg, Q = Queenstown, S = Southampton)
Generally, you follow these rules:
- **Always format your response in Markdown**
- **Always respond only with code in code blocks like this:**
```python
{code}
```
- The Python code runs in a Jupyter notebook.
- Every time you generate Python code, it is executed in a separate cell.
- Display visualizations using `matplotlib` or any other visualization library directly in the notebook.
- You have access to the internet and can make API requests.
- You also have access to the filesystem and can read/write files.
- Install all packages before using them by running `!pip install {package}`.
- You can run any Python code you want; everything is running in a secure sandbox environment.
"""
````
The GPT-4o prompt is assigned the role of a software engineer, generating a complete, single block of code from the output of o1.
```python
GPT_4O_PROMPT = """You are an expert software engineer that receives an execution plan, and then creates a single Python script that does everything in the plan. It will be executed in a single Python notebook cell."""
```
### 4. Add code interpreting capabilities and initialize the models
We first define a helper function to extract the code outputted from GPT-4o, the final model used. This function extracts all code between \`\`\` markers, which are called code fences and used by GPT to deliminate the code.
```python
import re
def match_code_blocks(llm_response):
pattern = re.compile(r'```python\n(.*?)\n```', re.DOTALL) # Match everything between ```python and ```
matches = pattern.findall(llm_response)
if matches:
code = "\n".join(matches)
print("> LLM-generated code:")
print(code)
return code
return ""
```
Now, we set up the chain which uses both of the above prompts to first generate the code, then consolidate it. We also initialize the OpenAI client. The chat function takes care of the interaction with the LLM. It calls the E2B code interpreter anytime there is a code to be run.
```python
from openai import OpenAI
def chat(e2b_code_interpreter, user_message):
print(f"\n{'='*50}\nUser message: {user_message}\n{'='*50}")
client = OpenAI(api_key=OPENAI_API_KEY)
# Get the plan from o1-mini
response_o1 = client.chat.completions.create(
model="o1-mini",
messages=[
{"role": "user", "content": O1_PROMPT},
{"role": "user", "content": user_message}
]
)
content_o1 = response_o1.choices[0].message.content
# Use gpt-4o-mini to extract the final code
response_4o = client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{"role": "system", "content": GPT_4O_PROMPT},
{"role": "user", "content": f"This is the plan I received, please write fully functional code that I can run in one notebook cell, and list all its dependencies: {content_o1}"}
]
)
content_4o = response_4o.choices[0].message.content
python_code = match_code_blocks(content_4o)
if python_code != "":
code_interpreter_results = code_interpret(e2b_code_interpreter, python_code)
return code_interpreter_results
else:
print(f"Failed to match any Python code in model's response: {content_4o}")
return []
```
Now we define the function that will use the code interpreter by E2B. Everytime the LLM assistant decides that it needs to execute code, this function will be used. Read more about the Code Interpreter SDK [here](https://docs.e2b.dev/quickstart).
```python
def code_interpret(e2b_code_interpreter, code):
print("Running code interpreter...")
exec = e2b_code_interpreter.notebook.exec_cell(
code,
on_stderr=lambda stderr: print("[Code Interpreter]", stderr),
on_stdout=lambda stdout: print("[Code Interpreter]", stdout),
)
if exec.error:
print("[Code Interpreter ERROR]", exec.error)
else:
return exec.results
```
### 5. Upload the dataset
The CSV data files are uploaded to the code interpreter before running any code. The file paths must match the CSV files in the directory. In this case, because we are tasking the AI with a supervised learning task, we have both a testing and training data set. Read more about the file upload [here](https://docs.e2b.dev/filesystem/upload).
```python
def upload_dataset(code_interpreter):
print("Uploading testing and training datasets to Code Interpreter sandbox...")
with open("./test.csv", "rb") as f:
remote_path = code_interpreter.upload_file(f)
print("Uploaded test.csv at", remote_path)
with open("./train.csv", "rb") as f:
remote_path = code_interpreter.upload_file(f)
print("Uploaded train.csv at", remote_path)
```
### 6. Put everything together
Finally we put everything together and let the AI data scientist upload the data, run an analysis, and generate a PNG file with a chart. You can update the task for the assistant in this step. For example, you can ask it to visualize the decision tree or a confusion matrix instead of the learning curve. If you decide to change the dataset you are using, don't forget to update the prompt too.
```python
from e2b_code_interpreter import CodeInterpreter
with CodeInterpreter(api_key=E2B_API_KEY) as code_interpreter:
# Upload the dataset to the code interpreter sandbox
upload_dataset(code_interpreter)
code_results = chat(
code_interpreter,
"Clean the data, train a decision tree to predict the survival of passengers, and visualize the learning curve. Then run the model on the test dataset and print the results."
)
if code_results:
first_result = code_results[0]
else:
raise Exception("No code interpreter results")
```
### 9. Run the program and see the results
When you run the notebook, the following steps will occur:
1. **Dataset upload**: The Titanic datasets ( train.csv and test.csv ) are uploaded to the sandbox environment.
2. **Code generation**: The o1-mini model generates a detailed plan with code blocks to perform data cleaning, model training, and visualization.
3. **Code extraction**: The gpt-4o-mini model extracts the final executable Python code from the plan.
4. **Code execution**: The extracted code is executed in the E2B Code Interpreter sandbox.
5. **Results**: The model trains a decision tree classifier, visualizes the learning curve, and prints predictions on the test dataset.
Example Output:
```text
Uploading testing and training datasets to Code Interpreter sandbox...
Uploaded at /home/user/train.csv
==================================================
User message: Clean the data, train a decision tree to predict the survival of passengers, and visualize the learning curve.
==================================================
> LLM-generated code:
# Install necessary packages
!pip install pandas numpy matplotlib seaborn scikit-learn
# Import libraries
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import train_test_split, learning_curve
from sklearn.preprocessing import LabelEncoder
from sklearn.metrics import accuracy_score
# Load the data (Make sure to replace these paths with the correct ones)
train = pd.read_csv('/home/user/train.csv')
test = pd.read_csv('/home/user/test.csv')
# Display first few rows
print(train.head())
# Data Cleaning
# Combine train and test for consistent preprocessing
combined = pd.concat([train, test], sort=False)
# ... [code continues] ...
```
**Full code for this guide** Find the full code in [our cookbook on GitHub](https://github.com/e2b-dev/e2b-cookbook/tree/main/examples/openai-python).
#### Resources
- [E2B Docs](https://docs.e2b.dev)
- [E2B Code Interpreter SDK](https://github.com/e2b-dev/code-interpreter)
- [E2B Cookbook](https://github.com/e2b-dev/e2b-cookbook/tree/main)
- [Follow E2B](https://x.com/e2b_dev)
- [Get started with OpenAI API](https://platform.openai.com/docs/overview)
- [Introducing o1](https://openai.com/index/introducing-openai-o1-preview/)
---
Source: https://e2b.dev/resources/automating-spreadsheet-workflows-with-ai.md
Published: 2024-09-11
# Automating spreadsheet workflows with AI (Manaflow)
[Manaflow](https://manaflow.ai/) is a YC startup that uses AI to automate repetitive office work often done in spreadsheets. Using natural language, you can program AI agents via Notion-like UI, create & call custom tools, populate Manasheet columns, and execute your repetitive workflows on a cron schedule.
I asked [Lawrence Chen](https://www.linkedin.com/in/lawrencecchen/), the CTO of Manaflow, to share more about Manaflow.
#### What was your journey to building Manaflow? What problem does it solve?
The idea for Manaflow emerged from our conversations with over 200 business managers, directors, and operators across various industries. We noticed that small-to-mid-sized businesses (SMBs) are constantly overwhelmed with numerous manual workflows and heavily underutilized technology.
Throughout America, there are millions of white-collar workers with spreadsheets open, where each column represents a step of a task and each row represents a case of the task. At SMBs, these processes are traditionally manual, time-consuming, and tedious, hindering scalability.
We knew that we wanted to build something that could even the automation playing field for these underdog businesses against the larger, tech-savvy corporations with teams of engineers. At first, it started as one of the many side projects that we hacked together, but it eventually grew into a startup as we kept talking to businesses about our idea, who were impressed by the technology and wanted to use our product.
Manaflow solves the pain by automating manual, repetitive office work that businesses do on a daily basis, especially those involving data analysis, calling APIs, and business actions. With automation, Manaflow enables more efficient scalability and frees up human resources for more strategic work while reducing errors.
#### How does the product work under the hood? What is the role of AI agents and "tools"?
Manaflow’s primary interface is a spreadsheet called a Manasheet where each column represents a step in the workflow and each row corresponds to an instance of a task. The workflow powering each spreadsheet is programmed using natural language, allowing non-technical users to describe tasks and steps in plain English, eliminating the need for coding skills.
We have a list of custom tools that our users can call or create themselves that AI agents can call to execute different steps of the task. Each spreadsheet has an internal dependency graph to determine the execution order for each column. This also enables checkpoints, which allow for human intervention and audits.
In addition, Manaflow has integrations with various external services, authenticated platforms, and APIs, enabling seamless automation of diverse business operations, from retrieving data from Google searches to processing Stripe invoices. Real-time monitoring and updates on the spreadsheets that serve as admin dashboards provide transparency and allow human operators to effectively manage workflow progress.
Lastly, operation managers can program AI agents to populate data into the cells and execute Manasheets for you. These AI agents are essentially managers that handle data entry and execution of the task for you if need be – they can be programmed to run on a cron schedule or via a button.
#### What are your techstack choices and why?
Manaflow is built primarily with TypeScript, Next.js, Postgres, and PartyKit. Each Manasheet is backed by a Postgres table. Manasheet executions spawn Durable Objects that execute code on [E2B sandboxes](https://docs.e2b.dev/sandbox). We made these technical decisions in order to unify user-facing code interpreter sessions with every Manasheet cell execution. This made it easy to observe individual Manasheet cells’ code interpreter runs within a chat/notebook interface.
#### How and why are you using E2B? What other alternatives have you considered?
We are using [E2B](https://docs.e2b.dev) sandboxes to provide a [code interpreter](https://github.com/e2b-dev/code-interpreter) environment. When we started building, we relied on Pyodide to evaluate Python and StackBlitz to evaluate JS/TS in the user’s browser. However, some people wanted to install dependencies, execute compute-intensive code, run tasks on a schedule, and make requests to services that had CORS. At this point, we knew we had to move code execution out of the user’s browser.
> "We found that E2B’s startup times were significantly faster than everything else we tried."
We considered using Fly.io Machines, Modal, and Google Cloud Run, and implementing our own Jupyter clusters on basic cloud compute primitives. However, when evaluating these options, we found that E2B’s startup times were significantly faster than everything else we tried. And with the ease of use of the @e2b/code-interpreter package, choosing E2B was a no-brainer.
#### What is your vision for the future? What is next on your roadmap?
Automation will transform office work. But, rather than relying on third parties to build automations, we believe knowledge workers themselves will be empowered to automate their own tasks without needing to code. Today, operations managers are essentially expert workflow programmers at heart, orchestrating manual human tasks to achieve broader objectives. In the future, these managers will shift to directing AI agents instead.
Internal tools and software are widespread across companies of all sizes today. We believe that in the future, AI agents will take over the operation of these internal tools, which is a manual process traditionally done by humans. AI agents will automatically update new states on admin dashboards, with humans overseeing the process. The concept of internal tools will change forever with AI agents – instead of building physical internal tools, AI agents will automate processes end-to-end on one consolidated platform.
We hope to be the infrastructure layer above the foundation models so that businesses can seamlessly integrate state-of-the-art AI models into their current workflows.
#### See Manaflow in action
#### Learn more
- [Manaflow - website](https://manaflow.ai/)
- [Manaflow - LinkedIn page](https://www.linkedin.com/company/manaflow-ai/)
- [Manaflow - X page](https://x.com/manaflowai)
- [Lawrence Chen - website](https://lawrencecchen.com/)
- [Lawrence Chen - LinkedIn profile](https://www.linkedin.com/in/lawrencecchen/)
- [Lawrence Chen - X profile](https://x.com/lawrencecchen)
---
Source: https://e2b.dev/resources/guide-code-interpreting-with-groq-and-e2b.md
Published: 2024-09-09
# Code Interpreting with Groq and E2B in Python
This AI data analyst can plot a linear regression chart based on CSV data. It uses [LLMs powered by Groq](https://console.groq.com/docs/models), and the [Code Interpreter SDK](https://github.com/e2b-dev/code-interpreter) by E2B for the code interpreting capabilities. The SDK quickly creates a secure cloud sandbox powered by [Firecracker](https://github.com/firecracker-microvm/firecracker). Inside this sandbox is a running Jupyter server that the LLM can use.
Read more about models powered by Groq [here](https://console.groq.com/docs/models).
The AI agent performs a data analysis task on an uploaded CSV file, executes the AI-generated code in the sandboxed environment by E2B, and returns a chart, saving it as a PNG file. The code is processing the data in the CSV file, cleaning the data, and performing the assigned analysis, which includes plotting a chart.
🔗 [**Full code on Github**](https://github.com/groq/groq-api-cookbook/tree/main/tutorials/08-integrations/e2b-code-interpreting/code-interpreter-python)
#### Key links
- [Code Interpreter SDK](https://github.com/e2b-dev/code-interpreter)
- [E2B docs](https://docs.e2b.dev)
- [E2B Cookbook](https://github.com/e2b-dev/e2b-cookbook/tree/main)
- [Groq Cookbook](https://github.com/groq/groq-api-cookbook)
#### Outline
1. Prerequisites
2. Install the SDKs
3. Set up the API keys and model instructions
4. Add code interpreting capabilities and initialize the model
5. Upload the dataset
## Setup
### 1. Prerequisites
Create the `main.ipynb` file.
Get the E2B API key [here](https://docs.e2b.dev/api-key) and the Groq API key [here](https://console.groq.com/keys).
If you want to use environment variables, copy the `env.template` file, and save it to a `.gitignore` file.
Download the CSV file from [here](https://www.kaggle.com/datasets/nishanthsalian/socioeconomic-country-profiles/code) and upload it to the same directory as your program. Name it `data.csv`.
### 2. Install the SDKs
```bash
pip install groq==0.6.0 e2b-code-interpreter==0.0.10 dotenv==1.0.0
```
### 3. Set up the API keys and model instructions
In this step you upload your E2B and Groq API keys to the program. In the JS & TS case, the API keys are stored in the `.env` file, in the Python case, they are added directly to the notebook. You pick the model of your choice by uncommenting it. There are some recommended models that are great at code generation, but you can add a different one from [here](https://console.groq.com/docs/models).
The model is assigned a data scientist role and explained the uploaded CSV. You can choose different data but need to update the instructions accordingly.
```python
import os
from dotenv import load_dotenv
import os
import json
import re
from groq import Groq
from e2b_code_interpreter import CodeInterpreter
load_dotenv()
# TODO: Get your Groq API key from https://console.groq.com/keys
GROQ_API_KEY = os.getenv("GROQ_API_KEY")
# TODO: Get your E2B API key from https://docs.e2b.dev
E2B_API_KEY = os.getenv("E2B_API_KEY")
# Choose from the codegen models:
MODEL_NAME = 'llama-3.1-70b-versatile'
# MODEL_NAME = 'llama-3.1-8b-instant'
# MODEL_NAME = 'llama3-groq-70b-8192-tool-use-preview'
# MODEL_NAME = 'llama3-groq-8b-8192-tool-use-preview'
# MODEL_NAME = 'llama3-70b-8192'
# MODEL_NAME = 'gemma2-9b-it'
# See the complete list of Groq models here: https://console.groq.com/docs/models.
SYSTEM_PROMPT = """You're a Python data scientist. You are given tasks to complete and you run Python code to solve them.
Information about the csv dataset:
- It's in the `/home/user/data.csv` file
- The CSV file is using , as the delimiter
- It has the following columns (examples included):
- country: "Argentina", "Australia"
- Region: "SouthAmerica", "Oceania"
- Surface area (km2): for example, 2780400
- Population in thousands (2017): for example, 44271
- Population density (per km2, 2017): for example, 16.2
- Sex ratio (m per 100 f, 2017): for example, 95.9
- GDP: Gross domestic product (million current US$): for example, 632343
- GDP growth rate (annual %, const. 2005 prices): for example, 2.4
- GDP per capita (current US$): for example, 14564.5
- Economy: Agriculture (% of GVA): for example, 10.0
- Economy: Industry (% of GVA): for example, 28.1
- Economy: Services and other activity (% of GVA): for example, 61.9
- Employment: Agriculture (% of employed): for example, 4.8
- Employment: Industry (% of employed): for example, 20.6
- Employment: Services (% of employed): for example, 74.7
- Unemployment (% of labour force): for example, 8.5
- Employment: Female (% of employed): for example, 43.7
- Employment: Male (% of employed): for example, 56.3
- Labour force participation (female %): for example, 48.5
- Labour force participation (male %): for example, 71.1
- International trade: Imports (million US$): for example, 59253
- International trade: Exports (million US$): for example, 57802
- International trade: Balance (million US$): for example, -1451
- Education: Government expenditure (% of GDP): for example, 5.3
- Health: Total expenditure (% of GDP): for example, 8.1
- Health: Government expenditure (% of total health expenditure): for example, 69.2
- Health: Private expenditure (% of total health expenditure): for example, 30.8
- Health: Out-of-pocket expenditure (% of total health expenditure): for example, 20.2
- Health: External health expenditure (% of total health expenditure): for example, 0.2
- Education: Primary gross enrollment ratio (f/m per 100 pop): for example, 111.5/107.6
- Education: Secondary gross enrollment ratio (f/m per 100 pop): for example, 104.7/98.9
- Education: Tertiary gross enrollment ratio (f/m per 100 pop): for example, 90.5/72.3
- Education: Mean years of schooling (female): for example, 10.4
- Education: Mean years of schooling (male): for example, 9.7
- Urban population (% of total population): for example, 91.7
- Population growth rate (annual %): for example, 0.9
- Fertility rate (births per woman): for example, 2.3
- Infant mortality rate (per 1,000 live births): for example, 8.9
- Life expectancy at birth, female (years): for example, 79.7
- Life expectancy at birth, male (years): for example, 72.9
- Life expectancy at birth, total (years): for example, 76.4
- Military expenditure (% of GDP): for example, 0.9
- Population, female: for example, 22572521
- Population, male: for example, 21472290
- Tax revenue (% of GDP): for example, 11.0
- Taxes on income, profits and capital gains (% of revenue): for example, 12.9
- Urban population (% of total population): for example, 91.7
Generally, you follow these rules:
- ALWAYS FORMAT YOUR RESPONSE IN MARKDOWN
- ALWAYS RESPOND ONLY WITH CODE IN CODE BLOCK LIKE THIS:
``python
{code}
``
- the Python code runs in jupyter notebook.
- every time you generate Python, the code is executed in a separate cell. it's okay to make multiple calls to `execute_python`.
- display visualizations using matplotlib or any other visualization library directly in the notebook. don't worry about saving the visualizations to a file.
- you have access to the internet and can make api requests.
- you also have access to the filesystem and can read/write files.
- you can install any pip package (if it exists) if you need to be running `!pip install {package}`. The usual packages for data analysis are already preinstalled though.
- you can run any Python code you want, everything is running in a secure sandbox environment
"""
```
### 4. Add code interpreting capabilities and initialize the model
Now we define the function that will use the code interpreter by E2B. Everytime the LLM assistant decides that it needs to execute code, this function will be used. Read more about the Code Interpreter SDK [here](https://docs.e2b.dev/quickstart).
We also initialize the Groq client. The function for matching code blocks is important because we need to pick the right part of the output that contains the code produced by the LLM. The chat function takes care of the interaction with the LLM. It calls the E2B code interpreter anytime there is a code to be run.
```python
def code_interpret(e2b_code_interpreter, code):
print("Running code interpreter...")
exec = e2b_code_interpreter.notebook.exec_cell(
code,
on_stderr=lambda stderr: print("[Code Interpreter]", stderr),
on_stdout=lambda stdout: print("[Code Interpreter]", stdout),
# You can also stream code execution results
# on_result=...
)
if exec.error:
print("[Code Interpreter ERROR]", exec.error)
else:
return exec.results
client = Groq(api_key=GROQ_API_KEY)
pattern = re.compile(
r"```python\n(.*?)\n```", re.DOTALL
) # Match everything in between ```python and ```
def match_code_blocks(llm_response):
match = pattern.search(llm_response)
if match:
code = match.group(1)
print(code)
return code
return ""
def chat_with_llm(e2b_code_interpreter, user_message):
print(f"\n{'='*50}\nUser message: {user_message}\n{'='*50}")
messages = [
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": user_message},
]
response = client.chat.completions.create(
model=MODEL_NAME,
messages=messages,
)
response_message = response.choices[0].message
python_code = match_code_blocks(response_message.content)
if python_code != "":
code_interpreter_results = code_interpret(e2b_code_interpreter, python_code)
return code_interpreter_results
else:
print(f"Failed to match any Python code in model's response {response_message}")
return []
```
### 5. Upload the dataset
The CSV data is uploaded programmatically, not via AI-generated code. The code interpreter by E2B runs inside the E2B sandbox. Read more about the file upload [here](https://docs.e2b.dev/filesystem/upload).
```python
def upload_dataset(code_interpreter):
print("Uploading dataset to Code Interpreter sandbox...")
dataset_path = "./data.csv"
if not os.path.exists(dataset_path):
raise FileNotFoundError("Dataset file not found")
try:
with open(dataset_path, "rb") as f:
remote_path = code_interpreter.upload_file(f)
if not remote_path:
raise ValueError("Failed to upload dataset")
print("Uploaded at", remote_path)
return remote_path
except Exception as error:
print("Error during file upload:", error)
raise error
```
### 6. Put everything together
Finally we put everything together and let the AI assistant upload the data, run an analysis, and generate a PNG file with a chart. You can update the task for the assistant in this step. If you decide to change the CSV file you are using, don't forget to update the prompt too.
```python
with CodeInterpreter(api_key=E2B_API_KEY) as code_interpreter:
# Upload the dataset to the code interpreter sandbox
upload_dataset(code_interpreter)
code_results = chat_with_llm(
code_interpreter,
"Make a chart showing linear regression of the relationship between GDP per capita and life expectancy from the data. Filter out any missing values or values in wrong format.",
)
if code_results:
first_result = code_results[0]
else:
raise Exception("No code interpreter results")
# This will render the image
# You can also access the data directly
# first_result.png
# first_result.jpg
# first_result.pdf
# ...
first_result
```
### 7. Run the program and see the results
The file is generated within the notebook. The plot shows the linear regression of the relationship between GDP per capita and life expectancy from the CSV data.
```text
Uploading dataset to Code Interpreter sandbox...
Uploaded at /home/user/data.csv
==================================================
User message: Make a chart showing linear regression of the relationship between GDP per capita and life expectancy from the data. Filter out any missing values or values in wrong format.
==================================================
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.linear_model import LinearRegression
import numpy as np
# Load the data
data = pd.read_csv('/home/user/data.csv')
# Filter out missing values and values in wrong format
data = data[pd.notnull(data['GDP per capita (current US$)'])]
data = data[pd.notnull(data['Life expectancy at birth, total (years)'])]
data['GDP per capita (current US$)'] = pd.to_numeric(data['GDP per capita (current US$)'], errors='coerce')
data['Life expectancy at birth, total (years)'] = pd.to_numeric(data['Life expectancy at birth, total (years)'], errors='coerce')
data = data.dropna(subset=['GDP per capita (current US$)', 'Life expectancy at birth, total (years)'])
# Create a linear regression model
X = data[['GDP per capita (current US$)']]
y = data['Life expectancy at birth, total (years)']
...
plt.title('Linear Regression of GDP per capita vs Life expectancy')
plt.legend()
plt.show()
Running code interpreter...
```
🔗 [**Full code on Github**](https://github.com/groq/groq-api-cookbook/tree/main/tutorials/08-integrations/e2b-code-interpreting/code-interpreter-python)
### Resources
- [Code Interpreter SDK](https://github.com/e2b-dev/code-interpreter)
- [E2B docs](https://docs.e2b.dev)
- [E2B Cookbook](https://github.com/e2b-dev/e2b-cookbook/tree/main)
- [Groq Cookbook](https://github.com/groq/groq-api-cookbook)
---
Source: https://e2b.dev/resources/guide-groq-js.md
Published: 2024-09-09
# Code Interpreting with Groq and E2B in JavaScript
This AI data analyst can plot a linear regression chart based on CSV data. It uses [LLMs powered by Groq](https://console.groq.com/docs/models), and the [Code Interpreter SDK](https://github.com/e2b-dev/code-interpreter) by E2B for the code interpreting capabilities. The SDK quickly creates a secure cloud sandbox powered by [Firecracker](https://github.com/firecracker-microvm/firecracker). Inside this sandbox is a running Jupyter server that the LLM can use.
Read more about models powered by Groq [here](https://console.groq.com/docs/models).
The AI agent performs a data analysis task on an uploaded CSV file, executes the AI-generated code in the sandboxed environment by E2B, and returns a chart, saving it as a PNG file. The code is processing the data in the CSV file, cleaning the data, and performing the assigned analysis, which includes plotting a chart.
🔗 [Full code on GitHub](https://github.com/e2b-dev/e2b-cookbook/tree/main/examples/groq-code-interpreter-js)
#### Key links
- [Code Interpreter SDK](https://github.com/e2b-dev/code-interpreter)
- [E2B docs](https://docs.e2b.dev)
- [E2B Cookbook](https://github.com/e2b-dev/e2b-cookbook/tree/main)
- [Groq Cookbook](https://github.com/groq/groq-api-cookbook)
#### Outline
1. Prerequisites
2. Install the SDKs
3. Set up the API keys and model instructions
4. Add code interpreting capabilities and initialize the model
5. Upload the dataset
## Setup
### 1. Prerequisites
Create the `main.ipynb` file.
Get the E2B API key [here](https://docs.e2b.dev/api-key) and the Groq API key [here](https://console.groq.com/keys).
If you want to use environment variables, copy the `env.template` file, and save it to a `.gitignore` file.
Download the CSV file from [here](https://www.kaggle.com/datasets/nishanthsalian/socioeconomic-country-profiles/code) and upload it to the same directory as your program. Name it `data.csv`.
### 2. Install the SDKs
```bash
pip install groq==0.6.0 e2b-code-interpreter==0.0.10 dotenv==1.0.0
```
### 3. Set up the API keys and model instructions
In this step you upload your E2B and Groq API keys to the program. In the JS & TS case, the API keys are stored in the `.env` file, in the Python case, they are added directly to the notebook. You pick the model of your choice by uncommenting it. There are some recommended models that are great at code generation, but you can add a different one from [here](https://console.groq.com/docs/models).
The model is assigned a data scientist role and explained the uploaded CSV. You can choose different data but need to update the instructions accordingly.
```python
import os
from dotenv import load_dotenv
import os
import json
import re
from groq import Groq
from e2b_code_interpreter import CodeInterpreter
load_dotenv()
# TODO: Get your Groq API key from https://console.groq.com/keys
GROQ_API_KEY = os.getenv("GROQ_API_KEY")
# TODO: Get your E2B API key from https://docs.e2b.dev
E2B_API_KEY = os.getenv("E2B_API_KEY")
# Choose from the codegen models:
MODEL_NAME = 'llama-3.1-70b-versatile'
# MODEL_NAME = 'llama-3.1-8b-instant'
# MODEL_NAME = 'llama3-groq-70b-8192-tool-use-preview'
# MODEL_NAME = 'llama3-groq-8b-8192-tool-use-preview'
# MODEL_NAME = 'llama3-70b-8192'
# MODEL_NAME = 'gemma2-9b-it'
# See the complete list of Groq models here: https://console.groq.com/docs/models.
SYSTEM_PROMPT = """You're a Python data scientist. You are given tasks to complete and you run Python code to solve them.
Information about the csv dataset:
- It's in the `/home/user/data.csv` file
- The CSV file is using , as the delimiter
- It has the following columns (examples included):
- country: "Argentina", "Australia"
- Region: "SouthAmerica", "Oceania"
- Surface area (km2): for example, 2780400
- Population in thousands (2017): for example, 44271
- Population density (per km2, 2017): for example, 16.2
- Sex ratio (m per 100 f, 2017): for example, 95.9
- GDP: Gross domestic product (million current US$): for example, 632343
- GDP growth rate (annual %, const. 2005 prices): for example, 2.4
- GDP per capita (current US$): for example, 14564.5
- Economy: Agriculture (% of GVA): for example, 10.0
- Economy: Industry (% of GVA): for example, 28.1
- Economy: Services and other activity (% of GVA): for example, 61.9
- Employment: Agriculture (% of employed): for example, 4.8
- Employment: Industry (% of employed): for example, 20.6
- Employment: Services (% of employed): for example, 74.7
- Unemployment (% of labour force): for example, 8.5
- Employment: Female (% of employed): for example, 43.7
- Employment: Male (% of employed): for example, 56.3
- Labour force participation (female %): for example, 48.5
- Labour force participation (male %): for example, 71.1
- International trade: Imports (million US$): for example, 59253
- International trade: Exports (million US$): for example, 57802
- International trade: Balance (million US$): for example, -1451
- Education: Government expenditure (% of GDP): for example, 5.3
- Health: Total expenditure (% of GDP): for example, 8.1
- Health: Government expenditure (% of total health expenditure): for example, 69.2
- Health: Private expenditure (% of total health expenditure): for example, 30.8
- Health: Out-of-pocket expenditure (% of total health expenditure): for example, 20.2
- Health: External health expenditure (% of total health expenditure): for example, 0.2
- Education: Primary gross enrollment ratio (f/m per 100 pop): for example, 111.5/107.6
- Education: Secondary gross enrollment ratio (f/m per 100 pop): for example, 104.7/98.9
- Education: Tertiary gross enrollment ratio (f/m per 100 pop): for example, 90.5/72.3
- Education: Mean years of schooling (female): for example, 10.4
- Education: Mean years of schooling (male): for example, 9.7
- Urban population (% of total population): for example, 91.7
- Population growth rate (annual %): for example, 0.9
- Fertility rate (births per woman): for example, 2.3
- Infant mortality rate (per 1,000 live births): for example, 8.9
- Life expectancy at birth, female (years): for example, 79.7
- Life expectancy at birth, male (years): for example, 72.9
- Life expectancy at birth, total (years): for example, 76.4
- Military expenditure (% of GDP): for example, 0.9
- Population, female: for example, 22572521
- Population, male: for example, 21472290
- Tax revenue (% of GDP): for example, 11.0
- Taxes on income, profits and capital gains (% of revenue): for example, 12.9
- Urban population (% of total population): for example, 91.7
Generally, you follow these rules:
- ALWAYS FORMAT YOUR RESPONSE IN MARKDOWN
- ALWAYS RESPOND ONLY WITH CODE IN CODE BLOCK LIKE THIS:
``python
{code}
``
- the Python code runs in jupyter notebook.
- every time you generate Python, the code is executed in a separate cell. it's okay to make multiple calls to `execute_python`.
- display visualizations using matplotlib or any other visualization library directly in the notebook. don't worry about saving the visualizations to a file.
- you have access to the internet and can make api requests.
- you also have access to the filesystem and can read/write files.
- you can install any pip package (if it exists) if you need to be running `!pip install {package}`. The usual packages for data analysis are already preinstalled though.
- you can run any Python code you want, everything is running in a secure sandbox environment
"""
```
### 4. Add code interpreting capabilities and initialize the model
Now we define the function that will use the code interpreter by E2B. Everytime the LLM assistant decides that it needs to execute code, this function will be used. Read more about the Code Interpreter SDK [here](https://docs.e2b.dev/quickstart).
We also initialize the Groq client. The function for matching code blocks is important because we need to pick the right part of the output that contains the code produced by the LLM. The chat function takes care of the interaction with the LLM. It calls the E2B code interpreter anytime there is a code to be run.
```python
def code_interpret(e2b_code_interpreter, code):
print("Running code interpreter...")
exec = e2b_code_interpreter.notebook.exec_cell(
code,
on_stderr=lambda stderr: print("[Code Interpreter]", stderr),
on_stdout=lambda stdout: print("[Code Interpreter]", stdout),
# You can also stream code execution results
# on_result=...
)
if exec.error:
print("[Code Interpreter ERROR]", exec.error)
else:
return exec.results
client = Groq(api_key=GROQ_API_KEY)
pattern = re.compile(
r"```python\n(.*?)\n```", re.DOTALL
) # Match everything in between ```python and ```
def match_code_blocks(llm_response):
match = pattern.search(llm_response)
if match:
code = match.group(1)
print(code)
return code
return ""
def chat_with_llm(e2b_code_interpreter, user_message):
print(f"\n{'='*50}\nUser message: {user_message}\n{'='*50}")
messages = [
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": user_message},
]
response = client.chat.completions.create(
model=MODEL_NAME,
messages=messages,
)
response_message = response.choices[0].message
python_code = match_code_blocks(response_message.content)
if python_code != "":
code_interpreter_results = code_interpret(e2b_code_interpreter, python_code)
return code_interpreter_results
else:
print(f"Failed to match any Python code in model's response {response_message}")
return []
```
### 5. Upload the dataset
The CSV data is uploaded programmatically, not via AI-generated code. The code interpreter by E2B runs inside the E2B sandbox. Read more about the file upload [here](https://docs.e2b.dev/filesystem/upload).
```python
def upload_dataset(code_interpreter):
print("Uploading dataset to Code Interpreter sandbox...")
dataset_path = "./data.csv"
if not os.path.exists(dataset_path):
raise FileNotFoundError("Dataset file not found")
try:
with open(dataset_path, "rb") as f:
remote_path = code_interpreter.upload_file(f)
if not remote_path:
raise ValueError("Failed to upload dataset")
print("Uploaded at", remote_path)
return remote_path
except Exception as error:
print("Error during file upload:", error)
raise error
```
### 6. Put everything together
Finally we put everything together and let the AI assistant upload the data, run an analysis, and generate a PNG file with a chart. You can update the task for the assistant in this step. If you decide to change the CSV file you are using, don't forget to update the prompt too.
```python
with CodeInterpreter(api_key=E2B_API_KEY) as code_interpreter:
# Upload the dataset to the code interpreter sandbox
upload_dataset(code_interpreter)
code_results = chat_with_llm(
code_interpreter,
"Make a chart showing linear regression of the relationship between GDP per capita and life expectancy from the data. Filter out any missing values or values in wrong format.",
)
if code_results:
first_result = code_results[0]
else:
raise Exception("No code interpreter results")
# This will render the image
# You can also access the data directly
# first_result.png
# first_result.jpg
# first_result.pdf
# ...
first_result
```
### 7. Run the program and see the results
The file is generated within the notebook. The plot shows the linear regression of the relationship between GDP per capita and life expectancy from the CSV data.
```text
Uploading dataset to Code Interpreter sandbox...
Uploaded at /home/user/data.csv
==================================================
User message: Make a chart showing linear regression of the relationship between GDP per capita and life expectancy from the data. Filter out any missing values or values in wrong format.
==================================================
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.linear_model import LinearRegression
import numpy as np
# Load the data
data = pd.read_csv('/home/user/data.csv')
# Filter out missing values and values in wrong format
data = data[pd.notnull(data['GDP per capita (current US$)'])]
data = data[pd.notnull(data['Life expectancy at birth, total (years)'])]
data['GDP per capita (current US$)'] = pd.to_numeric(data['GDP per capita (current US$)'], errors='coerce')
data['Life expectancy at birth, total (years)'] = pd.to_numeric(data['Life expectancy at birth, total (years)'], errors='coerce')
data = data.dropna(subset=['GDP per capita (current US$)', 'Life expectancy at birth, total (years)'])
# Create a linear regression model
X = data[['GDP per capita (current US$)']]
y = data['Life expectancy at birth, total (years)']
...
plt.title('Linear Regression of GDP per capita vs Life expectancy')
plt.legend()
plt.show()
Running code interpreter...
```
🔗 [Full code on GitHub](https://github.com/e2b-dev/e2b-cookbook/tree/main/examples/groq-code-interpreter-js)
### Resources
- [Code Interpreter SDK](https://github.com/e2b-dev/code-interpreter)
- [E2B docs](https://docs.e2b.dev)
- [E2B Cookbook](https://github.com/e2b-dev/e2b-cookbook/tree/main)
- [Groq Cookbook](https://github.com/groq/groq-api-cookbook)
---
Source: https://e2b.dev/resources/llama-3-1-with-code-interpreter-in-javascript.md
Published: 2024-08-14
# Llama 3.1 with code interpreter in JavaScript
Llama 3.1, the latest open-source model by Meta, features multi-step reasoning, integrated tool search, and a code interpreter. The LLM comes in three sizes: 8B, 70B, and 405B. Especially in the 405B version, Llama 3.1 challenges top LLMs in areas like general knowledge, math, tool use, and multilingual translation.
In this post, we will look closer at the code generation and code execution capabilities of Llama 3.1. We will give a step-by-step tutorial for securely running the LLM-generated code with E2B, in a Python or JavaScript/TypeScript version.
## Approaches to run code with Llama 3.1
Similar to other LLMs (e.g., GPT or Claude), there are two different options to generate code with Llama 3.1. You can utilize the built-in function calling support (if your LLM provider supports that), or you can add your own way to handle code generation.
### 1. Built-in function calling support
This approach is native to Llama 3.1 and easy to use. The function calling means that you are allowed to add “tools” that the LLM can decide to call.
The tools are usually defined in JSON, detailing their name, description, and the necessary input schema required to execute Python code in a Jupyter notebook environment.
It is recommended to pick 70B or 405B if you want to have a full conversation with function calling. Function calling doesn't necessarily imply code execution capabilities. Llama 3.1 models are trained to identify prompts that can be answered with their built-in code interpreter tool and provide the ppropriate Python function calls to achieve the result. You still have to make an implementation that produces valid results for the given query.
In general, the tools can serve various use cases, from searching the internet to running the generated code. They often include API calls to third-party apps.
Llama 3.1 also offers [built-in support for tools](https://llama.meta.com/docs/model-cards-and-prompt-formats/llama3_1/#built-in-tooling):
- Brave Search used to perform web searches.
- Wolfram Alpha used to perform complex mathematical calculations.
- Code Interpreter used to run the Python code generated by the LLM
This means that the LLM has been fine-tuned to more accurately make use of these functions, but you are still in charge of implementing them.
Even though Llama 3.1 supports the tool calling, I have struggled with it while generating bigger pieces of code. The model often stopped the generation before completing the code. This led me to use the second option - adding my own way of instructing Llama 3.1 to generate complete and good-quality code.
### 2. Manual approach with Markdown
When a LLM lacks built-in support for function calling, we can still achieve the same result. In this approach, we instruct Llama 3.1 to return just plain markdown code blocks that we manually parse and pass to the code interpreter.
We instruct the model on how to generate code in a suitable format, and we programmatically add a way to parse the code and have it prepared for running.
In the system prompt, we prompt the model to return a code response that matches the desired format. That means an output formatted correctly in Markdown with Python code blocks. We need to also specify the parsing of the code for the code interpreter we will use to run the code.
This approach might seem more difficult, but it is more universal and applicable beyond Llama 3.1, regardless of whether the particular LLM supports function calling.
Both approaches can use the E2B Code Interpreter SDK for securely executing the code generated by Llama 3.1. Executing the code produced by an LLM is a problem separate from the code generation, and there are different approaches to that. We chose the Code Interpreter SDK which runs the AI-generated code inside an open-source secure cloud sandbox, which is specifically made for running untrusted AI-generated code.
In this guide, we will demonstrate an example of the second approach.
In case you are interested in the first approach, there is a great [tutorial by Together AI](https://docs.together.ai/docs/function-calling) for the native function calling with Llama 3.1.
## Guide: Code interpreting with Llama 3.1 and E2B Code Interpreter SDK
🔗 [Full code on GitHub](https://github.com/e2b-dev/e2b-cookbook/tree/main/examples/together-ai-code-interpreter-python)
We will show how to build an AI assistant that analyzes a CSV file with socioeconomic data, runs code to analyze them, and generates a chart as a result.
The assistant will be powered by Llama 3.1 on Together AI and using the open-source Code Interpreter SDK by E2B. The E2B Code Interpreter SDK quickly creates a secure cloud sandbox powered by Firecracker. Inside this sandbox is a running Jupyter server that the LLM can use.
The Code Interpreter SDK works for both approaches we mentioned (built-in function calling or manually parsing the code). It is used to execute the AI-generated code, regardless of what approach and what LLM was used to provide the code.
### Key links
- [Get started with E2B](https://docs.e2b.dev)
- [Get started with Llama](https://llama.meta.com/)
- [Get started with Together AI](https://www.together.ai/)
- [Follow E2B](https://x.com/e2b)
### Outline
1. Prerequisites
2. Install the SDKs
3. Set up the API keys and model instructions
4. Add code interpreting capabilities and initialize the model
5. Upload the dataset
6. Put everything together
7. Run the program and see the results
## Setup
### 1. Prerequisites
Create an `index.ts` file for the main program, and \`.env\` file that looks like this:
```python
# TODO: Get your Together AI API key from https://api.together.xyz/settings/api-keys
TOGETHER_API_KEY = ""
# TODO: Get your E2B API key from https://docs.e2b.dev
E2B_API_KEY = ""
```
Get the E2B API key [here](https://docs.e2b.dev/api-key) and the Together AI API key [here](https://api.together.xyz/settings/api-keys).
Download the CSV file from [here](https://www.kaggle.com/datasets/nishanthsalian/socioeconomic-country-profiles/code) and upload it to the same directory as your program. Rename it to `data.csv`.
### 2. Install the SDKs
```bash
npm install @e2b/code-interpreter@0.0.5 together-ai@0.6.0-alpha.4 dotenv@16.4.5
```
### 3. Set up the API keys and model instructions
In this step we set up the E2B and Together API keys. They are stored in the `.env` file.
You pick the model of your choice by uncommenting it. There are some recommended models that are great at code generation, but you can add a different one from [here](https://api.together.ai/models).
The model is assigned a data scientist role and explained the uploaded CSV. If you use your custom CSV file, don’t forget to update the system prompt accordingly.
```typescript
import fs from 'node:fs'
import { CodeInterpreter, Result, ProcessMessage } from '@e2b/code-interpreter'
import * as dotenv from 'dotenv'
import Together from 'together-ai/index.mjs'
dotenv.config()
const TOGETHER_API_KEY = process.env.TOGETHER_API_KEY || ''
const E2B_API_KEY = process.env.E2B_API_KEY || ''
if (!TOGETHER_API_KEY) {
console.error('Error: TOGETHER_API_KEY is not provided. Please set the TOGETHER_API_KEY in your environment variables.')
process.exit(1)
}
if (!E2B_API_KEY) {
console.error('Error: E2B_API_KEY is not provided. Please set the E2B_API_KEY in your environment variables.')
process.exit(1)
}
// Choose from the codegen models:
const MODEL_NAME = 'meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo'
// const MODEL_NAME = 'meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo'
// const MODEL_NAME = 'meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo'
// const MODEL_NAME = 'Qwen/Qwen2-72B-Instruct'
// const MODEL_NAME = "codellama/CodeLlama-70b-Instruct-hf"
// const MODEL_NAME = "deepseek-ai/deepseek-coder-33b-instruct"
// See the complete list of Together AI models here: https://api.together.ai/models.
const SYSTEM_PROMPT = `
You're a python data scientist. You are given tasks to complete and you run Python code to solve them.
Information about the csv dataset:
- It's in the \`/home/user/data.csv\` file
- The CSV file is using , as the delimiter
- It has the following columns (examples included):
- country: "Argentina", "Australia"
- Region: "SouthAmerica", "Oceania"
- Surface area (km2): for example, 2780400
- Population in thousands (2017): for example, 44271
- Population density (per km2, 2017): for example, 16.2
- Sex ratio (m per 100 f, 2017): for example, 95.9
- GDP: Gross domestic product (million current US$): for example, 632343
- GDP growth rate (annual %, const. 2005 prices): for example, 2.4
- GDP per capita (current US$): for example, 14564.5
- Economy: Agriculture (% of GVA): for example, 10.0
- Economy: Industry (% of GVA): for example, 28.1
- Economy: Services and other activity (% of GVA): for example, 61.9
- Employment: Agriculture (% of employed): for example, 4.8
- Employment: Industry (% of employed): for example, 20.6
- Employment: Services (% of employed): for example, 74.7
- Unemployment (% of labour force): for example, 8.5
- Employment: Female (% of employed): for example, 43.7
- Employment: Male (% of employed): for example, 56.3
- Labour force participation (female %): for example, 48.5
- Labour force participation (male %): for example, 71.1
- International trade: Imports (million US$): for example, 59253
- International trade: Exports (million US$): for example, 57802
- International trade: Balance (million US$): for example, -1451
- Education: Government expenditure (% of GDP): for example, 5.3
- Health: Total expenditure (% of GDP): for example, 8.1
- Health: Government expenditure (% of total health expenditure): for example, 69.2
- Health: Private expenditure (% of total health expenditure): for example, 30.8
- Health: Out-of-pocket expenditure (% of total health expenditure): for example, 20.2
- Health: External health expenditure (% of total health expenditure): for example, 0.2
- Education: Primary gross enrollment ratio (f/m per 100 pop): for example, 111.5/107.6
- Education: Secondary gross enrollment ratio (f/m per 100 pop): for example, 104.7/98.9
- Education: Tertiary gross enrollment ratio (f/m per 100 pop): for example, 90.5/72.3
- Education: Mean years of schooling (female): for example, 10.4
- Education: Mean years of schooling (male): for example, 9.7
- Urban population (% of total population): for example, 91.7
- Population growth rate (annual %): for example, 0.9
- Fertility rate (births per woman): for example, 2.3
- Infant mortality rate (per 1,000 live births): for example, 8.9
- Life expectancy at birth, female (years): for example, 79.7
- Life expectancy at birth, male (years): for example, 72.9
- Life expectancy at birth, total (years): for example, 76.4
- Military expenditure (% of GDP): for example, 0.9
- Population, female: for example, 22572521
- Population, male: for example, 21472290
- Tax revenue (% of GDP): for example, 11.0
- Taxes on income, profits and capital gains (% of revenue): for example, 12.9
- Urban population (% of total population): for example, 91.7
Generally, you follow these rules:
- ALWAYS FORMAT YOUR RESPONSE IN MARKDOWN
- ALWAYS RESPOND ONLY WITH CODE IN CODE BLOCK LIKE THIS:
\`\`\`python
{code}
\`\`\`
- the Python code runs in jupyter notebook.
- every time you generate Python, the code is executed in a separate cell. it's okay to make multiple calls to \`execute_python\`.
- display visualizations using matplotlib or any other visualization library directly in the notebook. don't worry about saving the visualizations to a file.
- you have access to the internet and can make api requests.
- you also have access to the filesystem and can read/write files.
- you can install any pip package (if it exists) if you need to be running \`!pip install {package}\`. The usual packages for data analysis are already preinstalled though.
- you can run any Python code you want, everything is running in a secure sandbox environment
`
```
### 4. Add code interpreting capabilities and initialize the model
Now we define the function that will use the code interpreter by E2B. Every time the LLM assistant decides that it needs to execute code, this function will be used. Read more about the Code Interpreter SDK [here](https://docs.e2b.dev/quickstart).
We also initialize the Together AI client. The function for matching code blocks is important because we need to pick the right part of the output that contains the code produced by the LLM. The chat function takes care of the interaction with the LLM. It calls the E2B code interpreter anytime there is a code to be run.
```typescript
const together = new Together()
async function codeInterpret(codeInterpreter: CodeInterpreter, code: string): Promise {
console.log('Running code interpreter...')
const exec = await codeInterpreter.notebook.execCell(code, {
onStderr: (msg: ProcessMessage) => console.log('[Code Interpreter stderr]', msg),
onStdout: (stdout: ProcessMessage) => console.log('[Code Interpreter stdout]', stdout)
})
if (exec.error) {
console.error('[Code Interpreter ERROR]', exec.error)
throw new Error(exec.error.value)
}
return exec.results
}
async function chat(codeInterpreter: CodeInterpreter, userMessage: string): Promise {
console.log(`\n${'='.repeat(50)}\nUser Message: ${userMessage}\n${'='.repeat(50)}`)
const messages = [
{ role: 'system', content: SYSTEM_PROMPT },
{ role: 'user', content: userMessage }
]
try {
const response = await together.chat.completions.create({
model: MODEL_NAME,
messages: messages
})
const responseMessage = response.choices[0].message.content
const codeBlockMatch = responseMessage.match(/```python\n([\s\S]*?)\n```/)
if (codeBlockMatch && codeBlockMatch[1]) {
const pythonCode = codeBlockMatch[1]
console.log('CODE TO RUN')
console.log(pythonCode)
const codeInterpreterResults = await codeInterpret(codeInterpreter, pythonCode)
return codeInterpreterResults
} else {
console.error('Failed to match any Python code in model\'s response')
return []
}
} catch (error) {
console.error('Error during API call:', error)
throw error
}
}
```
### 5. Upload the dataset
The CSV data is uploaded programmatically, not via AI-generated code. The code interpreter by E2B runs inside the E2B sandbox. Read more about the file upload [here](https://docs.e2b.dev/filesystem/upload).
```typescript
async function uploadDataset(codeInterpreter: CodeInterpreter): Promise {
console.log('Uploading dataset to Code Interpreter sandbox...')
const datasetPath = './data.csv'
if (!fs.existsSync(datasetPath)) {
throw new Error('Dataset file not found')
}
const fileBuffer = fs.readFileSync(datasetPath)
try {
const remotePath = await codeInterpreter.uploadFile(fileBuffer, 'data.csv')
if (!remotePath) {
throw new Error('Failed to upload dataset')
}
console.log('Uploaded at', remotePath)
return remotePath
} catch (error) {
console.error('Error during file upload:', error)
throw error
}
}
```
### 6. Put everything together
Finally we put everything together and let the AI assistant upload the data, run an analysis, and generate a PNG file with a chart.
You can update the task for the assistant in this step. If you decide to change the CSV file you are using, don't forget to update the prompt too.
```typescript
async function run() {
const codeInterpreter = await CodeInterpreter.create()
try {
const remotePath = await uploadDataset(codeInterpreter)
console.log('Remote path of the uploaded dataset:', remotePath)
const codeInterpreterResults = await chat(
codeInterpreter,
// Task for the model
'Make a chart showing linear regression of the relationship between GDP per capita and life expectancy from the data. Filter out any missing values or values in wrong format.'
)
console.log('codeInterpreterResults:', codeInterpreterResults)
const result = codeInterpreterResults[0]
console.log('Result object:', result)
if (result && result.png) {
fs.writeFileSync('image_1.png', Buffer.from(result.png, 'base64'))
console.log('Success: Image generated and saved as image_1.png')
} else {
console.error('Error: No PNG data available.')
}
} catch (error) {
console.error('An error occurred:', error)
} finally {
await codeInterpreter.close()
}
}
run()
```
### 7. Run the program and see the results
The resulting chart is saved to the same directory as a PNG file. The plot shows the linear regression of the relationship between GDP per capita and life expectancy from the CSV data.
```text
> together-code-interpreter@1.0.0 start
> tsx index.ts
(node:21539) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
Uploading dataset to Code Interpreter sandbox...
Uploaded at /home/user/data.csv
Remote path of the uploaded dataset: /home/user/data.csv
==================================================
User Message: Make a chart showing linear regression of the relationship between GDP per capita and life expectancy from the data. Filter out any missing values or values in wrong format.
==================================================
CODE TO RUN
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.linear_model import LinearRegression
# Load the data
data = pd.read_csv('/home/user/data.csv', delimiter=',')
# Filter out missing values or values in wrong format
data = data.dropna(subset=['GDP per capita (current US$)', 'Life expectancy at birth, total (years)'])
# Convert columns to numeric
data['GDP per capita (current US$)'] = pd.to_numeric(data['GDP per capita (current US$)'], errors='coerce')
data['Life expectancy at birth, total (years)'] = pd.to_numeric(data['Life expectancy at birth, total (years)'], errors='coerce')
# Filter out any remaining non-numeric values
data = data.dropna(subset=['GDP per capita (current US$)', 'Life expectancy at birth, total (years)'])
# Fit linear regression model
X = data['GDP per capita (current US$)'].values.reshape(-1, 1)
y = data['Life expectancy at birth, total (years)'].values.reshape(-1, 1)
model = LinearRegression().fit(X, y)
# Plot the data and the regression line
plt.scatter(X, y, color='blue')
plt.plot(X, model.predict(X), color='red')
plt.xlabel('GDP per capita (current US$)')
plt.ylabel('Life expectancy at birth, total (years)')
plt.show()
Running code interpreter...
codeInterpreterResults: [
...
...
...
Success: Image generated and saved as image_1.png
```
🔗 [Full code on GitHub](https://github.com/e2b-dev/e2b-cookbook/tree/main/examples/together-ai-code-interpreter-python)
### Resources
- [Code Interpreter SDK](https://github.com/e2b-dev/code-interpreter)
- [E2B docs](https://docs.e2b.dev)
- [E2B Cookbook](https://github.com/e2b-dev/e2b-cookbook/tree/main)
---
Source: https://e2b.dev/resources/llama-3-1-with-code-interpreter-in-python.md
Published: 2024-08-14
# Llama 3.1 with code interpreter in Python
Llama 3.1, the latest open-source model by Meta, features multi-step reasoning, integrated tool search, and a code interpreter. The LLM comes in three sizes: 8B, 70B, and 405B. Especially in the 405B version, Llama 3.1 challenges top LLMs in areas like general knowledge, math, tool use, and multilingual translation.
In this post, we will look closer at the code generation and code execution capabilities of Llama 3.1. We will give a step-by-step tutorial for securely running the LLM-generated code with E2B, in a Python or JavaScript/TypeScript version.
## Approaches to run code with Llama 3.1
Similar to other LLMs (e.g., GPT or Claude), there are two different options to generate code with Llama 3.1. You can utilize the built-in function calling support (if your LLM provider supports that), or you can add your own way to handle code generation.
### 1. Built-in function calling support
This approach is native to Llama 3.1 and easy to use. The function calling means that you are allowed to add “tools” that the LLM can decide to call.
The tools are usually defined in JSON, detailing their name, description, and the necessary input schema required to execute Python code in a Jupyter notebook environment.
It is recommended to pick 70B or 405B if you want to have a full conversation with function calling. Function calling doesn't necessarily imply code execution capabilities. Llama 3.1 models are trained to identify prompts that can be answered with their built-in code interpreter tool and provide the ppropriate Python function calls to achieve the result. You still have to make an implementation that produces valid results for the given query.
In general, the tools can serve various use cases, from searching the internet to running the generated code. They often include API calls to third-party apps.
Llama 3.1 also offers [built-in support for tools](https://llama.meta.com/docs/model-cards-and-prompt-formats/llama3_1/#built-in-tooling):
- Brave Search used to perform web searches.
- Wolfram Alpha used to perform complex mathematical calculations.
- Code Interpreter used to run the Python code generated by the LLM
This means that the LLM has been fine-tuned to more accurately make use of these functions, but you are still in charge of implementing them.
Even though Llama 3.1 supports the tool calling, I have struggled with it while generating bigger pieces of code. The model often stopped the generation before completing the code. This led me to use the second option - adding my own way of instructing Llama 3.1 to generate complete and good-quality code.
### 2. Manual approach with Markdown
When a LLM lacks built-in support for function calling, we can still achieve the same result. In this approach, we instruct Llama 3.1 to return just plain markdown code blocks that we manually parse and pass to the code interpreter.
We instruct the model on how to generate code in a suitable format, and we programmatically add a way to parse the code and have it prepared for running.
In the system prompt, we prompt the model to return a code response that matches the desired format. That means an output formatted correctly in Markdown with Python code blocks. We need to also specify the parsing of the code for the code interpreter we will use to run the code.
This approach might seem more difficult, but it is more universal and applicable beyond Llama 3.1, regardless of whether the particular LLM supports function calling.
Both approaches can use the E2B Code Interpreter SDK for securely executing the code generated by Llama 3.1. Executing the code produced by an LLM is a problem separate from the code generation, and there are different approaches to that. We chose the Code Interpreter SDK which runs the AI-generated code inside an open-source secure cloud sandbox, which is specifically made for running untrusted AI-generated code.
In this guide, we will demonstrate an example of the second approach.
In case you are interested in the first approach, there is a great [tutorial by Together AI](https://docs.together.ai/docs/function-calling) for the native function calling with Llama 3.1.
## Guide: Code interpreting with Llama 3.1 and E2B Code Interpreter SDK
🔗 [**Full code on GitHub**](https://github.com/e2b-dev/e2b-cookbook/tree/main/examples/together-ai-code-interpreter-python)
We will show how to build an AI assistant that analyzes a CSV file with socioeconomic data, runs code to analyze them, and generates a chart as a result.
The assistant will be powered by Llama 3.1 on Together AI and using the open-source Code Interpreter SDK by E2B. The E2B Code Interpreter SDK quickly creates a secure cloud sandbox powered by Firecracker. Inside this sandbox is a running Jupyter server that the LLM can use.
The Code Interpreter SDK works for both approaches we mentioned (built-in function calling or manually parsing the code). It is used to execute the AI-generated code, regardless of what approach and what LLM was used to provide the code.
### Key links
- [Get started with E2B](https://docs.e2b.dev)
- [Get started with Llama](https://llama.meta.com/)
- [Get started with Together AI](https://www.together.ai/)
- [Follow E2B](https://x.com/e2b_dev)
### Outline
1. Prerequisites
2. Install the SDKs
3. Set up the API keys and model instructions
4. Add code interpreting capabilities and initialize the model
5. Upload the dataset
6. Put everything together
7. Run the program and see the results
## Setup
### 1. Prerequisites
Create a `main.ipynb` file.
Get the E2B API key [here](https://docs.e2b.dev/api-key) and the Together AI API key [here](https://api.together.xyz/settings/api-keys).
Download the CSV file from [here](https://www.kaggle.com/datasets/nishanthsalian/socioeconomic-country-profiles/code) and upload it to the same directory as your program. Rename it to `data.csv`.
### 2. Install the SDKs
```bash
pip install together==0.6.0 e2b-code-interpreter==0.0.10 dotenv==1.0.0
```
### 3. Set up the API keys and model instructions
In this step we set up the E2B and Together API keys. They are added directly to the notebook.
You pick the model of your choice by uncommenting it. There are some recommended models that are great at code generation, but you can add a different one from [here](https://api.together.ai/models).
The model is assigned a data scientist role and explained the uploaded CSV. If you use your custom CSV file, don’t forget to update the system prompt accordingly.
````python
import os
from dotenv import load_dotenv
import os
import json
import re
from together import Together
from e2b_code_interpreter import CodeInterpreter
load_dotenv()
# TODO: Get your Together AI API key from https://api.together.xyz/settings/api-keys
TOGETHER_API_KEY = os.getenv("TOGETHER_API_KEY")
# TODO: Get your E2B API key from https://docs.e2b.dev
E2B_API_KEY = os.getenv("E2B_API_KEY")
# Choose from the codegen models:
MODEL_NAME = "meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo"
# MODEL_NAME = "meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo"
# MODEL_NAME = "meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo"
# MODEL_NAME = "codellama/CodeLlama-70b-Instruct-hf"
# MODEL_NAME = "deepseek-ai/deepseek-coder-33b-instruct"
# MODEL_NAME = "Qwen/Qwen2-72B-Instruct"
# See the complete list of Together AI models here: https://api.together.ai/models.
SYSTEM_PROMPT = """You're a Python data scientist. You are given tasks to complete and you run Python code to solve them.
Information about the csv dataset:
- It's in the `/home/user/data.csv` file
- The CSV file is using , as the delimiter
- It has the following columns (examples included):
- country: "Argentina", "Australia"
- Region: "SouthAmerica", "Oceania"
- Surface area (km2): for example, 2780400
- Population in thousands (2017): for example, 44271
- Population density (per km2, 2017): for example, 16.2
- Sex ratio (m per 100 f, 2017): for example, 95.9
- GDP: Gross domestic product (million current US$): for example, 632343
- GDP growth rate (annual %, const. 2005 prices): for example, 2.4
- GDP per capita (current US$): for example, 14564.5
- Economy: Agriculture (% of GVA): for example, 10.0
- Economy: Industry (% of GVA): for example, 28.1
- Economy: Services and other activity (% of GVA): for example, 61.9
- Employment: Agriculture (% of employed): for example, 4.8
- Employment: Industry (% of employed): for example, 20.6
- Employment: Services (% of employed): for example, 74.7
- Unemployment (% of labour force): for example, 8.5
- Employment: Female (% of employed): for example, 43.7
- Employment: Male (% of employed): for example, 56.3
- Labour force participation (female %): for example, 48.5
- Labour force participation (male %): for example, 71.1
- International trade: Imports (million US$): for example, 59253
- International trade: Exports (million US$): for example, 57802
- International trade: Balance (million US$): for example, -1451
- Education: Government expenditure (% of GDP): for example, 5.3
- Health: Total expenditure (% of GDP): for example, 8.1
- Health: Government expenditure (% of total health expenditure): for example, 69.2
- Health: Private expenditure (% of total health expenditure): for example, 30.8
- Health: Out-of-pocket expenditure (% of total health expenditure): for example, 20.2
- Health: External health expenditure (% of total health expenditure): for example, 0.2
- Education: Primary gross enrollment ratio (f/m per 100 pop): for example, 111.5/107.6
- Education: Secondary gross enrollment ratio (f/m per 100 pop): for example, 104.7/98.9
- Education: Tertiary gross enrollment ratio (f/m per 100 pop): for example, 90.5/72.3
- Education: Mean years of schooling (female): for example, 10.4
- Education: Mean years of schooling (male): for example, 9.7
- Urban population (% of total population): for example, 91.7
- Population growth rate (annual %): for example, 0.9
- Fertility rate (births per woman): for example, 2.3
- Infant mortality rate (per 1,000 live births): for example, 8.9
- Life expectancy at birth, female (years): for example, 79.7
- Life expectancy at birth, male (years): for example, 72.9
- Life expectancy at birth, total (years): for example, 76.4
- Military expenditure (% of GDP): for example, 0.9
- Population, female: for example, 22572521
- Population, male: for example, 21472290
- Tax revenue (% of GDP): for example, 11.0
- Taxes on income, profits and capital gains (% of revenue): for example, 12.9
- Urban population (% of total population): for example, 91.7
Generally, you follow these rules:
- ALWAYS FORMAT YOUR RESPONSE IN MARKDOWN
- ALWAYS RESPOND ONLY WITH CODE IN CODE BLOCK LIKE THIS:
```python
{code}
```
- the Python code runs in jupyter notebook.
- every time you generate Python, the code is executed in a separate cell. it's okay to make multiple calls to `execute_python`.
- display visualizations using matplotlib or any other visualization library directly in the notebook. don't worry about saving the visualizations to a file.
- you have access to the internet and can make api requests.
- you also have access to the filesystem and can read/write files.
- you can install any pip package (if it exists) if you need to be running `!pip install {package}`. The usual packages for data analysis are already preinstalled though.
- you can run any Python code you want, everything is running in a secure sandbox environment
"""
````
### 4. Add code interpreting capabilities and initialize the model
Now we define the function that will use the code interpreter by E2B. Every time the LLM assistant decides that it needs to execute code, this function will be used. Read more about the Code Interpreter SDK [here](https://docs.e2b.dev/quickstart).
We also initialize the Together AI client. The function for matching code blocks is important because we need to pick the right part of the output that contains the code produced by the LLM. The chat function takes care of the interaction with the LLM. It calls the E2B code interpreter anytime there is a code to be run.
```python
def code_interpret(e2b_code_interpreter, code):
print("Running code interpreter...")
exec = e2b_code_interpreter.notebook.exec_cell(
code,
on_stderr=lambda stderr: print("[Code Interpreter]", stderr),
on_stdout=lambda stdout: print("[Code Interpreter]", stdout),
# You can also stream code execution results
# on_result=...
)
if exec.error:
print("[Code Interpreter ERROR]", exec.error)
else:
return exec.results
client = Together(api_key=TOGETHER_API_KEY)
pattern = re.compile(
r"```python\n(.*?)\n```", re.DOTALL
) # Match everything in between ```python and ```
def match_code_blocks(llm_response):
match = pattern.search(llm_response)
if match:
code = match.group(1)
print(code)
return code
return ""
def chat_with_llm(e2b_code_interpreter, user_message):
print(f"\n{'='*50}\nUser message: {user_message}\n{'='*50}")
messages = [
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": user_message},
]
response = client.chat.completions.create(
model=MODEL_NAME,
messages=messages,
)
response_message = response.choices[0].message
python_code = match_code_blocks(response_message.content)
if python_code != "":
code_interpreter_results = code_interpret(e2b_code_interpreter, python_code)
return code_interpreter_results
else:
print(f"Failed to match any Python code in model's response {response_message}")
return []
```
### 5. Upload the dataset
The CSV data is uploaded programmatically, not via AI-generated code. The code interpreter by E2B runs inside the E2B sandbox. Read more about the file upload [here](https://docs.e2b.dev/filesystem/upload).
```python
def upload_dataset(code_interpreter):
print("Uploading dataset to Code Interpreter sandbox...")
dataset_path = "./data.csv"
if not os.path.exists(dataset_path):
raise FileNotFoundError("Dataset file not found")
try:
with open(dataset_path, "rb") as f:
remote_path = code_interpreter.upload_file(f)
if not remote_path:
raise ValueError("Failed to upload dataset")
print("Uploaded at", remote_path)
return remote_path
except Exception as error:
print("Error during file upload:", error)
raise error
```
### 6. Put everything together
Finally we put everything together and let the AI assistant upload the data, run an analysis, and generate a PNG file with a chart.
You can update the task for the assistant in this step. If you decide to change the CSV file you are using, don't forget to update the prompt too.
```python
with CodeInterpreter(api_key=E2B_API_KEY) as code_interpreter:
# Upload the dataset to the code interpreter sandbox
upload_dataset(code_interpreter)
code_results = chat_with_llm(
code_interpreter,
"Make a chart showing linear regression of the relationship between GDP per capita and life expectancy from the data. Filter out any missing values or values in wrong format.",
)
if code_results:
first_result = code_results[0]
else:
raise Exception("No code interpreter results")
# This will render the image
# You can also access the data directly
# first_result.png
# first_result.jpg
# first_result.pdf
# ...
first_result
```
### 7. Run the program and see the results
The resulting chart is generated within the notebook environment. The plot shows the linear regression of the relationship between GDP per capita and life expectancy from the CSV data.
```text
Uploading dataset to Code Interpreter sandbox...
Uploaded at /home/user/data.csv
==================================================
User message: Make a chart showing linear regression of the relationship between GDP per capita and life expectancy from the data. Filter out any missing values or values in wrong format.
==================================================
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.linear_model import LinearRegression
# Load the data
data = pd.read_csv('/home/user/data.csv', delimiter=',')
# Clean the data
data = data.dropna(subset=['GDP per capita (current US$)', 'Life expectancy at birth, total (years)'])
data['GDP per capita (current US$)'] = pd.to_numeric(data['GDP per capita (current US$)'], errors='coerce')
data['Life expectancy at birth, total (years)'] = pd.to_numeric(data['Life expectancy at birth, total (years)'], errors='coerce')
# Fit the linear regression model
X = data['GDP per capita (current US$)'].values.reshape(-1, 1)
y = data['Life expectancy at birth, total (years)'].values.reshape(-1, 1)
model = LinearRegression().fit(X, y)
# Plot the data and the regression line
plt.scatter(X, y, color='blue')
...
plt.xlabel('GDP per capita (current US$)')
plt.ylabel('Life expectancy at birth, total (years)')
plt.show()
Running code interpreter...
```
🔗 [**Full code on GitHub**](https://github.com/e2b-dev/e2b-cookbook/tree/main/examples/together-ai-code-interpreter-python)
### Resources
- [Code Interpreter SDK](https://github.com/e2b-dev/code-interpreter)
- [E2B docs](https://docs.e2b.dev)
- [E2B Cookbook](https://github.com/e2b-dev/e2b-cookbook/tree/main)
---
Source: https://e2b.dev/resources/earning-enterprises-trust-with-your-ai-product.md
Published: 2024-07-09
# Athena Intelligence: Earning enterprises' trust with your AI product
When people talk about AI agents, one particular thing they ask very often is the option to analyze their own data, like PDF, Web, or Excel. That’s exactly what [Athena Intelligence](https://athenaintel.com/) is great at. Athena is the first artificial data analyst, which automates time-consuming tasks so that analysts can focus on strategic work.
Recently, they announced the code execution feature. This is a major next step for code generation - you can now let the AI assistant write code to enable complex analytics without leaving the chat interface.
I talked to [Ben Reilly](https://www.linkedin.com/in/patrickbenreilly/), the founding platform engineer about building a trustworthy AI product, what is going on “behind the scenes” of Athena, and what enterprises care about the most.
##### **You are building Athena intelligence. How did you start, and what were the beginnings like**?
The story starts with the founder [Brendan Geils](https://www.linkedin.com/in/brendongeils/). The idea behind Athena is a one-stop shop for enterprise data analytics. We aim to be the place where enterprise data teams, whether that means analysts or engineers, go to help understand their data.
As LLMs have gotten better and better, we've seen a very obvious use case in our platform. Athena harnesses LLMs to provide enterprise analytics teams with everything they need out of the box to move fast and get productive answers to the business as fast as possible.
##### **For readers who haven't tried the product, can you describe the basic building blocks and features of Athena**?
Athena offers many interfaces. The original is chat, but we now offer AI-powered reports, spreadsheets, and other interfaces.
Currently, users can leverage a variety of tools within the chat, including accessing information from PDFs and spreadsheets in your workspace. Athena can use this information to answer questions about your business in real-time. Athena also enables an auto-pilot mode to do this, where workflows you previously had Athena help you with become fully automated.
I find the roadmap really exciting as it aligns with our future goals and takes advantage of advancements in the field. I believe our platform will only become more useful as the models improve.
> "The vision is to build a platform that brings value to the most technical person in the room but is also immediately useful for the least technical one."
##### **How is the experience with Athena different for technical versus non-technical people and teams**?
A data analyst can work without having to deal with any complexity, while a technical person can get more specific and have greater control by writing custom components.
The vision is to build a platform that brings value to the most technical person in the room but is also immediately useful for the least technical one. That is, for someone who doesn’t know Python or SQL, but still has some basic understanding of data analytics and has a question to answer with data.
These workflows are enabled by [E2B](https://e2b.dev/), which enables trust in our platform for clients building a microVM and handling the code processes there.
##### **Regardless of technical skills of the user, what are your favorite examples of workflows done with Athena**?
Some examples include preparing complex reports, for example of a competitive landscape, synthesizing information across hundreds of documents, or researching important metrics.
Teams can even include Athena in their email and let it handle their tasks without needing to log in and initiate a conversation. Plus, any attachments or rich media in the email are automatically uploaded to your Olympus Drive.
##### **There are a lot of security questions when building with AI. How do you make users trust the product**?
Imagine you hire a brilliant team of PhDs who have already worked on groundbreaking research. You bring them onto your team and give them all your Slack messages and all the context that they would theoretically need. You ask them to solve a tough problem, and they come back and they say 42. Would you believe them just like that?
The idea of observability and customizability is an integral part of how we will work with LLMs in the future. Even if GPT-7 was a genius, we need the AI products to be observable and tweakable.
If the Athena agent performs a complex task, the user can later closely examine the decision the agent made and check whether it was a good decision given the information provided. As the platform evolves and we keep adding new features and making improvements, this aspect remains crucial.
Importantly, we never train models on any customer data. We take extreme care with data privacy - It is essential for enterprise applications. We can also deploy Athena to customer environments, including more sensitive networks like AWS Govcloud.
##### **What are other things you are proud of, that give you a competitive advantage**?
We need to ship fast too. When Anthropic dropped a new LLM ([Claude-3.5 Sonnet](https://www.anthropic.com/news/claude-3-5-sonnet)), 30 minutes later it was already available on Athena, and we opened up free account signups for the next 24 hours to allow teams to check out that new model.
##### **I am excited about the new code execution feature you launched recently. Can you share more**?
We use [E2B](https://e2b.dev/) for code execution under the hood. The whole process starts with a code written by an LLM for you. Previously, the process consisted of manually copying the code to a Jupyter Notebook, running it, checking if it works, dealing with any errors, and repeating this process.
With the code execution feature, we integrated this entire workflow into Athena, making it a first-class citizen across the platform. It's quite good now and will only improve in the future.
So now, every time an LLM writes Python code, Athena identifies it and executes it using the isolated E2B sandbox. If there are errors, the LLM automatically makes fixes and produces a new output for the user.
We see this unlocking lots of workflows where the agent can do more complex reasoning than without a code execution environment.
> "*With code execution, the LLM has instant feedback on whether or not the code it wrote actually works*."
##### **How else can the code execution feature improve the quality of results for users? Apart from not having to manually copy-paste and run the code**.
The most tangible thing is that with code execution, the LLM has instant feedback on whether or not the code it wrote actually works. It can immediately test its own output and compute verifiably correct complex operations. For example, multiplying several numbers together or doing complex math that an LLM would not verifiably give the right answer for.
That's a pretty useful primitive in the platform, and a big step in the LLM-powered software.
I am observing that AI programmers and human programmers have a lot of parallels. All LLMs can produce good code output, but some are obviously better than others. If you think about it, even a human can stare at a Python script for twenty-four hours and it still doesn’t have to be perfect or all things checked unless it has a compiler.
We think LLMs should have the same tools as humans when trying to answer the questions in the right way.
##### **A lot of people are used to AI chatbots now. What do you think is the “it” AI product of the future**?
There are all sorts of technical questions which are worth answering but for us, we just try to maintain a razor focus on how the enterprise customers want to interact with this new technology.
You could go off and explore all sorts of cool research questions about LLMs. How do you make the context window bigger? How do you control attention? How do you have LLMs use tools in the most effective way? How do you string them together? All these questions are worth asking, a lot of that is gonna get solved, and our job at Athena is to write the future of how people interact with LLMs in a useful way.
I think chat is not the final version of the AI software. Chat is what we're comfortable with now, partially because ChatGPT was the first thing that launched, but the Fortune 500s that are interested in using this technology, probably aren't going to be using chat forever.
Eventually, it's going to have to turn into a workflow, which is more intuitive and probably resembles the places where we do knowledge work now, like spreadsheets and reports.
I would say the most pressing issue that we're working with is not a technical challenge, but sort of a human question of how we actually want to use this technology. That is, how does the UI look like such that it allows us to use this technology in the most useful way possible.
##### **Do the large customers really care about LLMs and that Athena is “AI-powered**”?
That's a good question. At the end of the day and especially at the enterprise level, things have to be useful. People are interested in Athena, but they're interested because the AI makes the product very useful. Not because it has AI for the sake of AI. [Sklearn](https://scikit-learn.org/stable/) solves Linear Regression better than ChatGPT would produce, and we think it will stay that way for the foreseeable future. So we try to build with that in mind.
##### **My last question is: What are your plans and vision for the future**?
As a high-level solution, Athena is the enterprise data platform that connects your entire team and empowers everyone with LLMs. Many of the features on our platform are already great and extremely useful, and we are committed to further development.
Our ultimate goal is to be the most immediately and tangibly useful platform possible for the enterprise, serving the largest companies in the world. We want to unlock the tedious portions of knowledge work, and allow Analysts to use their time to think strategically about what will help their business.
### Learn more
- [Athena Intelligence](https://athenaintel.com/)
- [Athena Intelligence - X](https://x.com/AthenaIntell)
- [Athena Intelligence - LinkedIn](https://www.linkedin.com/company/athena-intelligence-ai/)
- [Athena Intelligence - YouTube](https://www.youtube.com/@athenaintelligence)
- [Ben Reilly (the founding platform engineer) - LinkedIn](https://www.linkedin.com/in/patrickbenreilly/)
---
Source: https://e2b.dev/resources/langgraph-with-code-interpreter-guide-with-code.md
Published: 2024-06-17
# Give LangGraph code execution capabilities
This example shows how to add code interpreting to an LLM using the [Code Interpreter SDK](https://github.com/e2b-dev/code-interpreter) and [LangGraph](https://langchain-ai.github.io/langgraph/).
🔗 [**Full code on GitHub**](https://github.com/e2b-dev/e2b-cookbook/tree/main/examples/langgraph-python)
## Why Code Interpreter SDK
The [E2B Code Interpreter SDK](https://github.com/e2b-dev/code-interpreter) quickly creates a secure cloud sandbox powered by [Firecracker](https://github.com/firecracker-microvm/firecracker). Inside this sandbox is a running Jupyter server that the LLM can use.
In general, the Code Interpreter SDK allows you to build custom code interpreters. For example, you can [install custom packages](https://docs.e2b.dev/template/quickstart), have access to the internet, [use the filesystem](https://docs.e2b.dev/filesystem), or [connect your cloud storage](https://docs.e2b.dev/storage/cloud-buckets).
The Code Interpreter SDK works with [any LLM](https://github.com/e2b-dev/e2b-cookbook/tree/main), in this example, we are using OpenAI's [GPT-3.5 Turbo](https://platform.openai.com/docs/models/gpt-3.5-turbo) to plot a sine wave.
### Key links
- [Get started with E2B](https://docs.e2b.dev)
- [Get started with LangGraph](https://langchain-ai.github.io/langgraph/)
- [Follow E2B](https://twitter.com/e2b_dev?lang=en)
### Overview
1. Install dependencies
2. Get API keys, prompt, and tools
3. Implement the method for code interpreting
4. Implement the methods for calling tool, create workflow and invoke the LangGraph app
5. Run the program
## Setup
### 1. Install dependencies
We start by installing the [E2B code interpreter SDK](https://github.com/e2b-dev/code-interpreter) and [LangChain Python SDK](https://console.groq.com/).
```python
pip install langgraph e2b-code-interpreter langchain langchainhub langchain-openai
```
### 2. Define API keys, prompt, and tools
Now we store your the [E2B API KEY](https://docs.e2b.dev/api-key) and [OPENAI API KEY](https://platform.openai.com/settings).
```python
import os
# TODO: Get your OpenAI API key from https://platform.openai.com/api-keys
os.environ["OPENAI_API_KEY"] = ""
# TODO: Get your E2B API key from https://docs.e2b.dev
os.environ["E2B_API_KEY"] = ""
```
### 3. Implement the method for code interpreting
This part includes the tool definition that uses the [E2B Code Interpreter SDK](https://github.com/e2b-dev/code-interpreter). We'll be using this to get the E2B code interpreter tool and to format the output of the tool.
First, we import all necessary libraries.
```python
import os
import json
from typing import Any
from langchain_core.tools import Tool
from pydantic.v1 import BaseModel, Field
from e2b_code_interpreter import CodeInterpreter
from langchain_core.messages import ToolMessage
```
Then we define classes to allow the LLM use a code interpreter as a tool. The class `LangchainCodeInterpreterToolInput` defines the input schema for the tool using [Pydantic](https://docs.pydantic.dev/latest/), specifying that the input will be a string of Python code.
The class `CodeInterpreterFunctionTool` calls arbitrary code against a Python Jupyter notebook. It requires an `E2B_API_KEY` to create a sandbox.
We define the `format_to_tool_messages` function to identify for each agent's action whether it corresponds to a specific tool.
```python
class RichToolMessage(ToolMessage):
raw_output: dict
class LangchainCodeInterpreterToolInput(BaseModel):
code: str = Field(description="Python code to execute.")
class CodeInterpreterFunctionTool:
tool_name: str = "code_interpreter"
def __init__(self):
# Instantiate the E2B sandbox - this is a long lived object
# that's pinging E2B cloud to keep the sandbox alive.
if "E2B_API_KEY" not in os.environ:
raise Exception(
"Code Interpreter tool called while E2B_API_KEY environment variable is not set. Please get your E2B api key here https://docs.e2b.dev and set the E2B_API_KEY environment variable."
)
self.code_interpreter = CodeInterpreter()
def close(self):
self.code_interpreter.close()
def call(self, parameters: dict, **kwargs: Any):
code = parameters.get("code", "")
print(f"***Code Interpreting...\n{code}\n====")
execution = self.code_interpreter.notebook.exec_cell(code)
return {
"results": execution.results,
"stdout": execution.logs.stdout,
"stderr": execution.logs.stderr,
"error": execution.error,
}
# langchain does not return a dict as a parameter, only a code string
def langchain_call(self, code: str):
return self.call({"code": code})
def to_langchain_tool(self) -> Tool:
tool = Tool(
name=self.tool_name,
description="Execute python code in a Jupyter notebook cell and returns any rich data (eg charts), stdout, stderr, and error.",
func=self.langchain_call,
)
tool.args_schema = LangchainCodeInterpreterToolInput
return tool
@staticmethod
def format_to_tool_message(
tool_call_id: str,
output: dict,
) -> RichToolMessage:
"""
Format the output of the CodeInterpreter tool to be returned as a RichToolMessage.
"""
# TODO: Add info about the results for the LLM
content = json.dumps(
{k: v for k, v in output.items() if k not in ("results")}, indent=2
)
return RichToolMessage(
content=content,
raw_output=output,
tool_call_id=tool_call_id,
)
```
### 4. Implement the methods for calling tool, create workflow and invoke the LangGraph app
Now we define the `format_to_tool_messages` function to identify for each agent's action whether it corresponds to a specific tool. If it does, the function formats the action and observation into messages and appends them to the `messages` list, ensuring no duplicates.
We create a prompt template that will be used by the agent to generate responses. We define and invoke the agent, during which we specify the prompt, which is to plot and show sinus.
```python
from typing import List
from langchain_openai import ChatOpenAI
from langgraph.graph import END, MessageGraph
# Define the function that determines whether to continue or not
def should_continue(messages) -> str:
last_message = messages[-1]
# If there is no function call, then we finish
if not last_message.tool_calls:
return END
else:
return "action"
# Handle tools execution
def execute_tools(messages, tool_map) -> List[RichToolMessage]:
tool_messages = []
for tool_call in messages[-1].tool_calls:
tool = tool_map[tool_call["name"]]
if tool_call["name"] == CodeInterpreterFunctionTool.tool_name:
output = tool.invoke(tool_call["args"])
message = CodeInterpreterFunctionTool.format_to_tool_message(
tool_call["id"],
output,
)
tool_messages.append(message)
else:
content = tool.invoke(tool_call["args"])
tool_messages.append(RichToolMessage(content, tool_call_id=tool_call["id"]))
return tool_messages
# 1. Pick your favorite llm
llm = ChatOpenAI(model="gpt-3.5-turbo-0125", temperature=0)
# 2. Initialize the code interpreter tool
code_interpreter = CodeInterpreterFunctionTool()
code_interpreter_tool = code_interpreter.to_langchain_tool()
tools = [code_interpreter_tool]
tool_map = {tool.name: tool for tool in tools}
# 3. Define the graph
workflow = MessageGraph()
workflow.add_node("agent", llm.bind_tools(tools))
workflow.add_node("action", lambda x: execute_tools(x, tool_map))
# Conditional agent -> action OR agent -> END
workflow.add_conditional_edges(
"agent",
should_continue,
)
# Always transition `action` -> `agent`
workflow.add_edge("action", "agent")
workflow.set_entry_point("agent")
app = workflow.compile()
# 4. Invoke the app
result = app.invoke([("human", "plot and show sinus")])
code_interpreter.close()
def display_results(messages):
for message in result:
if hasattr(message, 'raw_output'):
if message.raw_output["results"]:
rs = message.raw_output["results"]
for r in rs:
display(r)
display_results(result)
```
### 5. Run the program
Finally, we run the program. The task given to the agent was to plot and show a sine function.
```text
***Code Interpreting...
import matplotlib.pyplot as plt
import numpy as np
x = np.linspace(0, 2*np.pi, 100)
y = np.sin(x)
plt.plot(x, y)
plt.title('Sine Wave')
plt.xlabel('x')
plt.ylabel('sin(x)')
plt.grid(True)
plt.show()
====
```
🔗 [**Full code on GitHub**](https://github.com/e2b-dev/e2b-cookbook/tree/main/examples/langgraph-python)
### Key links
- [Get started with E2B](https://docs.e2b.dev)
- [Get started with LangCraph](https://langchain-ai.github.io/langgraph/)
- [Follow E2B](https://twitter.com/e2b_dev?lang=en)
---
Source: https://e2b.dev/resources/build-langchain-agent-with-code-interpreter.md
Published: 2024-06-13
# Build LangChain agent with code interpreter
This example shows how to add code interpreting to an LLM using the [Code Interpreter SDK](https://github.com/e2b-dev/code-interpreter) and [LangChain](https://www.langchain.com/).
🔗 [**Full code on GitHub**](https://github.com/e2b-dev/e2b-cookbook/tree/main/examples/langchain-python)
## Why Code Interpreter SDK
The [E2B Code Interpreter SDK](https://github.com/e2b-dev/code-interpreter) quickly creates a secure cloud sandbox powered by [Firecracker](https://github.com/firecracker-microvm/firecracker). Inside this sandbox is a running Jupyter server that the LLM can use.
In general, the Code Interpreter SDK allows you to build custom code interpreters. For example, you can [install custom packages](https://docs.e2b.dev/template/quickstart), have access to the internet, [use the filesystem](https://docs.e2b.dev/filesystem), or [connect your cloud storage](https://docs.e2b.dev/storage/cloud-buckets).
The Code Interpreter SDK works with [any LLM](https://github.com/e2b-dev/e2b-cookbook/tree/main), in this example, we are using OpenAI's [GPT-3.5 Turbo](https://platform.openai.com/docs/models/gpt-3.5-turbo) to plot a sine wave.
### Key links
- [Get started with E2B](https://docs.e2b.dev)
- [Get started with LangChain](https://python.langchain.com/v0.2/docs/introduction/)
- [Follow E2B](https://twitter.com/e2b_dev?lang=en)
### Overview
1. Install dependencies
2. Get API keys, prompt, and tools
3. Implement the method for code interpreting
4. Implement the methods for formatting messages, create and invoke the LangChain agent
5. Run the program
## Setup
### 1. Install dependencies
We start by install the [E2B code interpreter SDK](https://github.com/e2b-dev/code-interpreter) and [LangChain Python SDK](https://console.groq.com/).
```bash
pip install e2b-code-interpreter langchain langchainhub langchain-openai
```
### 2. Define API keys, prompt, and tools
Now we store your the [E2B API KEY](https://docs.e2b.dev/api-key) and [OPENAI API KEY](https://platform.openai.com/settings).
```python
import os
# TODO: Get your OpenAI API key from https://platform.openai.com/api-keys
os.environ["OPENAI_API_KEY"] = ""
# TODO: Get your E2B API key from https://docs.e2b.dev
os.environ["E2B_API_KEY"] = ""
```
### 3. Implement the method for code interpreting
This part includes the tool definition that uses the [E2B Code Interpreter SDK](https://github.com/e2b-dev/code-interpreter). We'll be using this to get the E2B code interpreter tool and to format the output of the tool.
The class `LangchainCodeInterpreterToolInput` defines the input schema for the tool using [Pydantic](https://docs.pydantic.dev/latest/), specifying that the input will be a string of Python code.
```python
import os
import json
from typing import Any, List
from langchain_core.tools import Tool
from pydantic.v1 import BaseModel, Field
from e2b_code_interpreter import CodeInterpreter
from langchain_core.messages import BaseMessage, ToolMessage
from langchain.agents.output_parsers.tools import (
ToolAgentAction,
)
class LangchainCodeInterpreterToolInput(BaseModel):
code: str = Field(description="Python code to execute.")
```
We then create the `CodeInterpreterFunctionTool` class, which handles the code interpreting. The constructor (`__init__` method) initializes the `CodeInterpreter`, which creates a long-running sandbox instance.
Note that at the end of the class definition, we are filtering out the `"results"` key from the `observation` dictionary. The [`result`](https://docs.e2b.dev/code-interpreting/analyze-data-with-ai) [object](https://docs.e2b.dev/code-interpreting/analyze-data-with-ai) represents the data to be displayed as a result of executing a cell in a Jupyter notebook.
We need to filter it out because, by default, LangChain attaches all LLM-generated output, but the `result` object can contain multiple types of data, such as text, images, plots, etc. represented as a string that would be difficult for the LLM to process and could take a large portion of the context window.
```python
class CodeInterpreterFunctionTool:
"""
This class calls arbitrary code against a Python Jupyter notebook.
It requires an E2B_API_KEY to create a sandbox.
"""
tool_name: str = "code_interpreter"
def __init__(self):
# Instantiate the E2B sandbox - this is a long lived object
# that's pinging E2B cloud to keep the sandbox alive.
if "E2B_API_KEY" not in os.environ:
raise Exception(
"Code Interpreter tool called while E2B_API_KEY environment variable is not set. Please get your E2B api key here https://docs.e2b.dev and set the E2B_API_KEY environment variable."
)
self.code_interpreter = CodeInterpreter()
def call(self, parameters: dict, **kwargs: Any):
code = parameters.get("code", "")
print(f"***Code Interpreting...\n{code}\n====")
execution = self.code_interpreter.notebook.exec_cell(code)
return {
"results": execution.results,
"stdout": execution.logs.stdout,
"stderr": execution.logs.stderr,
"error": execution.error,
}
def close(self):
self.code_interpreter.close()
# langchain does not return a dict as a parameter, only a code string
def langchain_call(self, code: str):
return self.call({"code": code})
def to_langchain_tool(self) -> Tool:
tool = Tool(
name=self.tool_name,
description="Execute python code in a Jupyter notebook cell and returns any rich data (eg charts), stdout, stderr, and error.",
func=self.langchain_call,
)
tool.args_schema = LangchainCodeInterpreterToolInput
return tool
@staticmethod
def format_to_tool_message(
agent_action: ToolAgentAction,
observation: dict,
) -> List[BaseMessage]:
"""
Format the output of the CodeInterpreter tool to be returned as a ToolMessage.
"""
new_messages = list(agent_action.message_log)
# TODO: Add info about the results for the LLM
content = json.dumps(
{k: v for k, v in observation.items() if k not in ("results")}, indent=2
)
new_messages.append(
ToolMessage(content=content, tool_call_id=agent_action.tool_call_id)
)
return new_messages
```
### 4. Implement the methods for formatting messages, create and invoke the LangChain agent
Now we define the `format_to_tool_messages` function to identify for each agent's action whether it corresponds to a specific tool. If it does, the function formats the action and observation into messages and appends them to the `messages` list, ensuring no duplicates.
We create a prompt template that will be used by the agent to generate responses. We define and invoke the agent, during which we specify the prompt, which is to plot and show sinus.
```python
from typing import List, Sequence, Tuple
from langchain_core.prompts import ChatPromptTemplate
from langchain.agents import AgentExecutor
from langchain_openai import ChatOpenAI
from langchain_core.messages import BaseMessage
from langchain_core.runnables import RunnablePassthrough
from langchain.agents.output_parsers.tools import (
ToolAgentAction,
ToolsAgentOutputParser,
)
def format_to_tool_messages(
intermediate_steps: Sequence[Tuple[ToolAgentAction, dict]],
) -> List[BaseMessage]:
messages = []
for agent_action, observation in intermediate_steps:
if agent_action.tool == CodeInterpreterFunctionTool.tool_name:
new_messages = CodeInterpreterFunctionTool.format_to_tool_message(
agent_action,
observation,
)
messages.extend([new for new in new_messages if new not in messages])
else:
# Handle other tools
print("Not handling tool: ", agent_action.tool)
return messages
# 1. Pick your favorite llm
llm = ChatOpenAI(model="gpt-3.5-turbo-0125", temperature=0)
# 2. Initialize the code interpreter tool
code_interpreter = CodeInterpreterFunctionTool()
code_interpreter_tool = code_interpreter.to_langchain_tool()
tools = [code_interpreter_tool]
# 3. Define the prompt
prompt = ChatPromptTemplate.from_messages(
[("human", "{input}"), ("placeholder", "{agent_scratchpad}")]
)
# 4. Define the agent
agent = (
RunnablePassthrough.assign(
agent_scratchpad=lambda x: format_to_tool_messages(x["intermediate_steps"])
)
| prompt
| llm.bind_tools(tools)
| ToolsAgentOutputParser()
)
agent_executor = AgentExecutor(
agent=agent,
tools=tools,
verbose=True,
return_intermediate_steps=True,
)
# 5. Invoke the agent
result = agent_executor.invoke({"input": "plot and show sinus"})
code_interpreter.close()
# Each intermediate step is a Tuple[ToolAgentAction, dict]
result["intermediate_steps"][0][1]["results"][0]
```
### 5. Run the program
Finally, we run the program. The task given to the agent was to plot and show a sine function.
```text
> Entering new AgentExecutor chain...
Invoking: `code_interpreter` with `{'code': "import matplotlib.pyplot as plt\nimport numpy as np\n\nx = np.linspace(0, 2*np.pi, 100)\ny = np.sin(x)\n\nplt.plot(x, y)\nplt.title('Sine Wave')\nplt.xlabel('x')\nplt.ylabel('sin(x)')\nplt.grid(True)\nplt.show()"}`
***Code Interpreting...
import matplotlib.pyplot as plt
import numpy as np
x = np.linspace(0, 2*np.pi, 100)
y = np.sin(x)
plt.plot(x, y)
plt.title('Sine Wave')
plt.xlabel('x')
plt.ylabel('sin(x)')
plt.grid(True)
plt.show()
====
{'results': [], 'stdout': [], 'stderr': [], 'error': None}Here is a plot of the sine wave.
> Finished chain.
```
🔗 [**Full code on GitHub**](https://github.com/e2b-dev/e2b-cookbook/tree/main/examples/langchain-python)
### Key links
- [Get started with E2B](https://docs.e2b.dev)
- [Get started with LangChain](https://python.langchain.com/v0.2/docs/introduction/)
- [Follow E2B](https://twitter.com/e2b_dev?lang=en)
---
Source: https://e2b.dev/resources/how-to-analyze-your-csv-files-with-llama-3.md
Published: 2024-06-05
# How to analyze your CSV files with Llama 3
In this guide, we will show how to upload your own CSV file for an AI assistant to analyze. The assistant is powered by [Meta's Llama 3](https://ai.meta.com/blog/meta-llama-3/) and executes its actions in the secure sandboxed environment via the [E2B Code Interpreter SDK](https://github.com/e2b-dev/code-interpreter).
🔗 [**Full code on GitHub**](https://github.com/e2b-dev/e2b-cookbook/blob/main/examples/upload-dataset-code-interpreter/upload_dataset.ipynb)
## Why Code Interpreter SDK
The [E2B Code Interpreter SDK](https://github.com/e2b-dev/code-interpreter) quickly creates a secure cloud sandbox powered by [Firecracker](https://github.com/firecracker-microvm/firecracker). Inside this sandbox is a running Jupyter server that the LLM can use.
In general, the Code Interpreter SDK allows you to build custom code interpreters. For example, you can [install custom packages](https://docs.e2b.dev/template/quickstart), have access to the internet, [use the filesystem](https://docs.e2b.dev/filesystem), or [connect your cloud storage](https://docs.e2b.dev/storage/cloud-buckets). The Code Interpreter SDK works with [any LLM](https://github.com/e2b-dev/e2b-cookbook/tree/main).
### Key links
- [Get started with E2B](https://docs.e2b.dev)
- [Get started with Llama](https://llama.meta.com/docs/get-started/)
- [Get started with Groq](https://wow.groq.com/docs/)
- [Follow E2B](https://twitter.com/e2b_dev?lang=en)
### Overview
1. Install dependencies
2. Get API keys, prompt, and tools
3. Implement the method for code interpreting
4. Implement the method for calling LLM and parsing tools
5. Implement method for uploading dataset to code interpreter sandbox
6. Put everything together
## Setup
### 1. Install dependencies
First, we install the [E2B code interpreter SDK](https://github.com/e2b-dev/code-interpreter) and [Groq's Python SDK](https://console.groq.com/).
```bash
pip install groq e2b_code_interpreter
```
### 2. Get API keys, prompt, and tools
Then we store the [Groq](https://console.groq.com/) and [E2B](https://docs.e2b.dev) API keys and set the model name for the Llama 3 instance we will use. In the system prompt we define the rules for the interaction with Llama. We define our tools - there will be just one tool for executing Python code.
```python
# TODO: Get your Groq AI API key from https://console.groq.com/
GROQ_API_KEY = ""
# TODO: Get your E2B API key from https://docs.e2b.dev
E2B_API_KEY = ""
# Or use 8b version
# MODEL_NAME = "llama3-8b-8192"
MODEL_NAME = "llama3-70b-8192"
SYSTEM_PROMPT = """You're a Python data scientist that is analyzing daily temperature of major cities. You are given tasks to complete and you run python code to solve them.
Information about the the temperature dataset:
- It's in the `/home/user/city_temperature.csv` file
- It has following columns (examples included):
- `Region`: "North America", "Europe"
- `Country`: "Iceland"
- `State`: for example "Texas" but can also be null
- `City`: "Prague"
- `Month`: "June"
- `Day`: 1-31
- `Year`: 2002
- `AvgTemperature`: temperature in celsiu, for example 24
Generally you follow these rules:
- the python code runs in jupyter notebook.
- every time you call `execute_python` tool, the python code is executed in a separate cell. it's okay to multiple calls to `execute_python`.
- display visualizations using matplotlib or any other visualization library directly in the notebook. don't worry about saving the visualizations to a file.
- you have access to the internet and can make api requests.
- you also have access to the filesystem and can read/write files.
- you can install any pip package (if it exists) if you need to but the usual packages for data analysis are already preinstalled.
- you can run any python code you want, everything is running in a secure sandbox environment
"""
tools = [
{
"type": "function",
"function": {
"name": "execute_python",
"description": "Execute python code in a Jupyter notebook cell and returns any result, stdout, stderr, display_data, and error.",
"parameters": {
"type": "object",
"properties": {
"code": {
"type": "string",
"description": "The python code to execute in a single cell.",
}
},
"required": ["code"],
},
},
}
]
```
### 3. Implement the method for code interpreting
We define the main function that uses the E2B code interpreter to execute code in a Jupyter Notebook that's running inside the E2B sandbox. We'll be calling this function a little bit further when we're parsing the Llama's response with tool calls.
```python
def code_interpret(e2b_code_interpreter, code):
print("Running code interpreter...")
exec = e2b_code_interpreter.notebook.exec_cell(
code,
on_stderr=lambda stderr: print("[Code Interpreter]", stderr),
on_stdout=lambda stdout: print("[Code Interpreter]", stdout),
# You can also stream code execution results
# on_result=...
)
if exec.error:
print("[Code Interpreter ERROR]", exec.error)
else:
return exec.results
```
### 4. Implement the method for calling LLM and parsing tools
Now we're going to define and implement `chat_with_llama` method. In this method, we'll call the LLM with our `tools` dictionary, parse the output, and call our `code_interpret` method we defined above.
See the [Groq documentation](https://wow.groq.com/docs/) to get started.
```python
import os
import json
import re
from groq import Groq
client = Groq(api_key=GROQ_API_KEY)
def chat_with_llama(e2b_code_interpreter, user_message):
print(f"\n{'='*50}\nUser message: {user_message}\n{'='*50}")
messages = [
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": user_message}
]
response = client.chat.completions.create(
model=MODEL_NAME,
messages=messages,
tools=tools,
tool_choice="auto",
max_tokens=4096,
)
response_message = response.choices[0].message
tool_calls = response_message.tool_calls
if tool_calls:
for tool_call in tool_calls:
function_name = tool_call.function.name
function_args = json.loads(tool_call.function.arguments)
if function_name == "execute_python":
code = function_args["code"]
code_interpreter_results = code_interpret(e2b_code_interpreter, code)
return code_interpreter_results
else:
raise Exception(f"Unknown tool {function_name}")
else:
print(f"(No tool call in model's response) {response_message}")
return []
```
### 5. Implement method for uploading dataset to code interpreter sandbox
The file gets uploaded to the E2B sandbox where our code interpreter is running. We get the file's remote path in the `remote_path` variable.
```python
def upload_dataset(code_interpreter):
print("Uploading dataset to Code Interpreter sandbox...")
with open("./city_temperature.csv", "rb") as f:
remote_path = code_interpreter.upload_file(f)
print("Uploaded at", remote_path)
```
### 6. Put everything together
Finally, we put all the pieces together. We instantiate a new code interpreter instance using
`with CodeInterpreter(api_key`=`E2B_API_KEY) as code_interpreter:`
and then call the `chat_with_llama` method with our user message and the `code_interpreter` instance.
```python
from e2b_code_interpreter import CodeInterpreter
with CodeInterpreter(api_key=E2B_API_KEY) as code_interpreter:
# Upload the dataset to the code interpreter sandbox
upload_dataset(code_interpreter)
code_results = chat_with_llama(
code_interpreter,
"Plot average temperature over the years in Algeria"
)
if code_results:
first_result = code_results[0]
else:
raise Exception("No code results")
# This will render the image
# You can also access the data directly
# first_result.png
# first_result.jpg
# first_result.pdf
# ...
first_result
```
```text
Uploading dataset to Code Interpreter sandbox...
Uploaded at /home/user/city_temperature.csv
==================================================
User message: Plot average temperature over the years in Algeria
==================================================
Running code interpreter...
```
🔗 [**Full code on GitHub**](https://github.com/e2b-dev/e2b-cookbook/blob/main/examples/upload-dataset-code-interpreter/upload_dataset.ipynb)
### Key links
- [Get started with E2B](https://docs.e2b.dev)
- [Get started with Llama](https://llama.meta.com/docs/get-started/)
- [Get started with Groq](https://wow.groq.com/docs/)
- [Follow E2B](https://twitter.com/e2b_dev?lang=en)
---
Source: https://e2b.dev/resources/building-ai-workflow-automation-for-enterprises.md
Published: 2024-06-03
# Gumloop: Building AI workflow automation for enterprises
[Max Brodeur-Urbas](https://www.linkedin.com/in/max-brodeur-urbas-1a4b25172/?originalSubdomain=ca) is the founder of [Gumloop](https://www.gumloop.com/) - a platform for automating any workflow without needing to code, using movable building blocks. Gumloop has customers from banks and big corporations to individuals running their whole businesses on the platform. What are the secrets behind their success?
We talked about:
- How they started as an AutoGPT wrapper
- Why Max thinks AI products would sometimes be better off with less AI
- Views on security, generative UI, future of agents, and AI code execution with [E2B](https://e2b.dev/)
- Why they rebranded AgentHub to Gumloop.
## From AutoGPT wrapper to a successful company
##### **Tell me about the beginnings of Gumloop. When and how did you start**?
It all started last year when [AutoGPT](https://github.com/Significant-Gravitas/AutoGPT) came out in March 2023. At that time, I was using ChatGPT daily and already understood its limitations. AutoGPT convinced me, at least for a few days, that agents were going to be something crazy, so we started building on top of the AutoGPT GitHub repository as a side project.
##### **So Gumloop started as a wrapper around AutoGPT… Among all the agentic experiments at that time, how did you get your first users**?
Spending time on the [AutoGPT Discord server](https://discord.com/invite/autogpt), we noticed thousands of people were joining the server each day, but most of these were nontechnical. They didn’t even know what GitHub was, or how to clone a repo or install dependencies, but they wanted to use AutoGPT.
So we just took the most recent version of AutoGPT, hosted it in the cloud, and made a simple GUI for it in a browser. People were writing to the Support channel with problems like “What command do I use to run this repo”, so we just pasted a link to Gumloop (at that time AgentHub) as an answer and people often checked that.****
##### **How did the first version of the product look**?
We shipped the first version in approximately 48 hours. It was my first time using React. This is how the first version looked like on day one:
This all happened within a few weeks of the launch of the AutoGPT GitHub repo.
##### **At that time your company was called AgentHub and you just recently rebranded to Gumloop. Where did the original name come from**?
Under the assumption that agents were useful, which was wrong at least at that time, we wanted to make a hub for agents. Our hypothesis was that if you make something useful, you would want to share it with someone else. And there was no way of easily sharing your prompt for AutoGPT. We had a vision of a marketplace where people could rent agents. Something like some startups are building now - a marketplace for AI employees.
And the domain for AgentHub was cheap.
## Building an app with reliable AI agents
##### **What happened after you launched the first version**?
I was getting dozens of messages on Discord. People were complaining about the agents. Especially non-technical people didn’t understand that I wasn’t the person building AutoGPT itself, but just the UI. And at that time, AutoGPT was failing in many tasks, so people blamed me for that. They wanted me to debug their agents constantly.
##### **Did you consider building on alternatives to AutoGPT**?
I was watching the AI agents community and quickly moved on from the super autonomous agents and didn’t even try other alternatives. I remember [BabyAGI](https://github.com/yoheinakajima/babyagi) was one of the other popular projects at the time but we never experimented with it.
##### **How did you overcome this stage and move towards an enterprise-grade product**?
It started with one important realization. I noticed people demanded simple tasks like “Scrape this website and analyze it.” or “Get information from this website and summarize it.” There was a desperate need from non-technical people for AI to do something useful, even if their tasks often weren’t that sophisticated.
These automations could very feasibly be built with a few scripts and working directly with different APIs but for these users, learning how to code was completely out of the question. They were looking for a simple solution, not a new career path.
Agents felt like an exciting possible solution but they were a bandaid on the problem. Throwing AI at every step of the way will only make things expensive and unreliable. We wanted a simpler way for people to reliably convert their tasks to valuable automation. This is how our automation framework was born.
##### **So your strategy is to take part of the responsibilities from agents if they are better done with deterministic software**?
Yes, exactly. The approach we are taking is using less AI in the workflow and creating something more reliable.
##### **Do you think the future of AI agents lies in focusing on narrow tasks, like with Gumloop, or do generic agents still have a chance**?
I think what we’re building is the shortest path to reliable, affordable AI being leveraged in businesses.
Until true AGI arrives, the only people able to utilize AI to its full potential are engineers building with it. We’re lowering that technical bar so anyone can build complex flows with AI and see immediate business value.
## Is Gumloop a Zapier on steroids?
##### **Now we are getting more into the question of how Gumloop works. Can you talk about the product and how it is built**?
We are using Tailwind, CSS, React, and Typescript for the frontend, and Python for the backend.
The frontend provides a hyper-flexible canvas where the user can define their custom flows, we then run them in the cloud in a scalable way so that even workflows can be looped thousands of times to hyper-scale operations.
##### **My first thought was that Gumloop reminds me of Zapier and I have seen your users call Gumloop “Zapier on steroids” too. What are the differences**?
Gumloop as the framework is very flexible, it’s almost like a visual programming language. There is always the tradeoff between how steep the learning curve is, versus what you can achieve with it.
You can have a ‘low floor’ which makes getting started simple, or a high ceiling which allows power users to build extremely complex automations. Our ceiling is extremely high compared to competitors. So high in fact that users are building entire businesses on Gumloop. We’re now working on lowering that floor next so that absolutely anyone can get started with ease.
##### **How are your customers utilizing the wide range of possibilities? What are some examples of what people are building with Gumloop**?
We have some very large startups and corporations as enterprise customers but the most impressive use cases are people who have built entire businesses on our app. For example, one of our enterprise users is using Gumloop to process academic papers for universities. His whole data pipeline and the service he provides is built on our platform.
He provides the expertise on how to build the automation and then sends it back to his customers at a much higher markup. He was able to build it all without hiring any AI engineer, so the value is equivalent to the salaries of several engineers.
##### **On your website, you have a lot of different templates for use cases from Sales to Hiring and Education. How customizable are the workflows in Gumloop**?
That’s our biggest strength. We are completely horizontal so we can really automate anything. The only missing part might be a particular integration, but we can quickly implement that for our customers.
On the other hand, it can do anything, but it’s hard to market. A lot of time we see the failure in creativity. People don’t really know what to do with Gumploop, so they need to see the examples to get the idea.
##### **We talked about removing responsibilities from AI agents. Are there still any autonomous parts within your platform**?
Almost none of our features are Agentic anymore on Gumloop. The closest to agentic decision-making is when you have an AI step, and set up conditional flows based on the response. E.g. “If the agent responds with this, let’s do this”. This happens for example with categorization and data extraction flows.
If you add more decisions like this in a row, it might seem autonomous, but even this is a deterministic process that depends on the previous output.
We do have some experimental web browsing agents built into the platform which are our take on the web voyager project. These are more proofs of concept however than something people should use in production.
## About security and code execution in the cloud
##### **We talked about removing responsibilities from AI agents. Are there still any autonomous parts within your platform**?
We have a big library of nodes. Nodes are building blocks that you drag onto the canvas to provide functionality, and each of the nodes has its specific purpose.
We have a “run code” node that a lot of users choose when the existing nodes library doesn’t suffice for their use case. That is, once you want something very custom, e.g. your own way to manipulate and format data, we have to run code for that.
That’s why we use [E2B](https://e2b.dev/), which we see as the only way to do safe code execution within our automation. The E2B sandbox environment is the solution we are sure won’t destroy our servers with malicious code. The E2B runtime lets users code in Python or JavaScript, define dynamic inputs and outputs, and run code in the node, so it gives customers the flexibility to go the extra mile.
##### **What’s the most popular use case today**?
There are some patterns, but the most exciting ones are so niche that they seem useless to the general population. The bigger the business, the more boring and niche the work becomes. A lot of companies are just doing the same specific thing thousands of times a month, so giving someone else the same template wouldn’t bring much value.
Among more generic use cases, web scraping is very popular, e.g., using external data from the internet to enrich flows, scrapping subreddits, summarizing reports of what customers are complaining about… A lot of people have businesses that aggregate information, like databases or directories of sources. These are the kinds of users approaching us. Keeping information up to date is important and often requires automation.
Another big use case is document processing. We found out that a lot of companies, especially those interacting with the government, have to deal with a lot of annoying forms and documents. More old-aged businesses like lawyers or people working in shipping and logistics have to fill in tons of annoying paperwork, so they get pretty excited about the ability to process documents in a smart way. Whether it is categorizing data, or processing it for their CRM. This is the boring paperwork to automate with Gumloop.
##### **In the beginning, you talked about the benefit of no-code UI. What do you offer for more technical users, like developers, apart from customizing the “run code” node**?
We have the more “advanced” category of nodes, where you can call your own arbitrary API in the automation. For example, people make get requests to their own servers to receive the product information. You can use the code execution, you can script in Selenium without using code. We have a “web agent scraper” with discrete actions like “scroll”, “click” “hover”, or “screenshot” allowing you to automate your browser in a selenium-like way.
##### **How can people integrate Gumloop into their existing products**?
We can trigger the automations with Webhook. This is how people integrate the Gumloop automations into their own products. Generally, all automations have one or a few input nodes with input data, but with Webhook, you can pass in values as input nodes. For example, you put in the first and last name via the Webhook, trigger it via API, and in turn, it researches the person and pings you on Slack about it.
##### **How agnostic is Gumloop? Are there any techstack limitations or lock-ins**?
Yes, it works with anything. We are completely model-agnostic too, so every time a new LLM comes out, users can just run the automations with a new model easily without changing anything.
Any integration we don’t have now is possible to add as long as they have a public-facing API for us to integrate with. We can ship features like this within days when customers need them.
The only “lock-in” is that the Gumloop automations stay in the Gumloop platform since they’re a totally custom format that is executed in quite a uniquely scalable way on our backend.
##### **You are building an enterprise-grade product and your customers are often companies. Are there any security and compliance challenges you had to overcome**?
We have had several requests for different certifications, e.g. a request for SOC 2. These typically come from bigger companies like banks, that require a lot of certifications or even on-prem hosting. Some companies want to run Gumloop in their own locally hosted LLM such that no data would leave. This is on our roadmap, but at this moment we haven’t had an urgent need to focus on this yet..
##### **Apart from security, what are other important aspects of building a product for enterprises**?
What I haven’t mentioned is that it’s quite easy to collaborate in teams. Most of our power users don’t work on the platform alone but have a project together with coworkers, share the best prompts for individual flows, and just work on the automations together.
## Why AgentHub rebranded to Gumloop
##### **What else is on your roadmap recently**?
We recently released parallel node execution that really sped up some automations. Let’s say you are processing a thousand websites and performing the same tasks on them. That might take a day. However, since we added the parallel batch mode, you can process many websites at once, which makes these processes take not a day, but just an hour.
We are also adding a lot of AI features that allow users to build more easily. We are trying to lower the floor starting with Gumloop. The complexity is what can make new-coming people leave before using the platform in a meaningful way, so we are solving this problem.
##### **What is your approach to generative UI**?
I think if you nail generative UI, you get the ultimate personalization and onboarding experience. We are trying to let users describe what they want to build, the same way they would describe it to a human, and encode our understanding of how to approach it to an LLM that is able to do it at scale and help out.
I think it is done well and allows us to completely change the journey of some users. The ability to demonstrate to people what they can build, instead of hoping that some content on your page will speak to them or some example you published will address their needs. That is, I see generative UI as a problem of execution and user experience problem more than a technical problem. At least with regard to generating automation on our platform, we know that it’s possible, but the question is how to make it enjoyable for users.
##### **What are the biggest challenges for you at this moment**?
I think the biggest thing is not having enough engineering hours in a day. We are a team of two full-time engineers, plus doing a lot of sales, marketing, and hiring. It’s hard to increase velocity when you’re working 14-hour days every day.
##### **My last question is about your recent rebranding. Why did you go from AgentHub to Gumloop**?
We rebranded to Gumloop, because instead of “AgentHub”, people were hearing “AsianHub”, and that’s not something you want to google. Another reason was some points from Paul Graham's essay “[Change Your Name](https://paulgraham.com/name.html?ref=blog.gumloop.com)”. And finally, we are not technically building agents (which non-technical people often don’t know anyway), and the original direction of a hub for agents isn’t relevant anymore.
The name Gumloop suggests connecting things with some sticky substance and looping one instance of
### Discover more
- [Gumloop.com](https://www.gumloop.com/)
- [Gumloop - X (Twitter) profile](https://x.com/gumloop_ai)
- [Max Brodeur-Urbas - X (Twitter) profile](https://x.com/MaxBrodeurUrbas)
- [Gumloop - LinkedIn profile](https://www.linkedin.com/company/gumloop/posts/?feedView=all)
- [Gumloop - video tutorials](https://www.youtube.com/@Gumloop_Ai)
- [Gumloop - Discord server](https://discord.com/invite/xtbrafmzC7)
---
Source: https://e2b.dev/resources/guide-ai-code-execution-with-mistrals-codestral.md
Published: 2024-05-30
# AI Code Execution with Mistral's Codestral
Mistral AI just announced [Codestral](https://mistral.ai/news/codestral/): a open-weight model designed for code generation tasks. It outperforms other models in a long-range eval for code generation and is fluent in 80+ programming languages. We test capabilities of the new Mistral's model on data analysis tasks, using the [Code Interpreter SDK](https://github.com/e2b-dev/code-interpreter) by [E2B](https://docs.e2b.dev).
Codestral doesn't support using tools for code execution yet, so in this Python example, we added the code interpreting capabilities. We are going to build an AI agent that performs data analysis tasks on provided data, in a form of a csv file.
🔗 [**Full code on GitHub**](https://github.com/e2b-dev/e2b-cookbook/tree/main/examples/codestral-code-interpreter-python)
## Why Code Interpreter SDK
E2B's code interpreter SDK quickly creates a secure cloud sandbox powered by [Firecracker](https://github.com/firecracker-microvm/firecracker). Inside this sandbox is a running Jupyter server that the LLM can use.
In general, the Code Interpreter SDK allows you to build custom code interpreters. For example, you can install custom packages, have access to the internet, use the filesystem, or connect your cloud storage. The Code Interpreter SDK works with [any LLM](https://github.com/e2b-dev/e2b-cookbook/tree/main). (in this example, we are using it with the Mistral's newest model).
### Key links
- [Get started with E2B](https://docs.e2b.dev)
- [Get started with Mistral models](https://mistral.ai/)
- [Read more about the new Codestral model](https://mistral.ai/news/codestral/)
- [Follow E2B](https://twitter.com/e2b_dev?lang=en)
### Overview
1. Installing dependencies
2. Defining API keys and prompt
3. Implementation of method for code interpreting
4. Adding tmethod for calling Codestral and parsing its response
5. Adding method for data upload
6. Putting everything together
## Setup
### 1. Installing dependencies
We start by install the [E2B code interpreter SDK](https://github.com/e2b-dev/code-interpreter) and [Mistral's Python SDK](https://console.mistral.ai/).
```python
%pip install mistralai e2b_code_interpreter
```
### 2. Defining API keys and prompt
Let's define our variables with API keys for Mistral and E2B together with the model ID and prompt. We won't be defining any tools because Codestral doesn't support tool usage yet.
````python
# TODO: Get your Mistral API key from https://console.mistral.ai
MISTRAL_API_KEY = ""
# TODO: Get your E2B API key from https://docs.e2b.dev
E2B_API_KEY = ""
MODEL_NAME = "codestral-latest"
SYSTEM_PROMPT = """You're a python data scientist that is analyzing daily temperature of major cities. You are given tasks to complete and you run python code to solve them.
Information about the the temperature dataset:
- It's in the `/home/user/city_temperature.csv` file
- The CSV file is using `,` as the delimiter
- It has following columns (examples included):
- `Region`: "North America", "Europe"
- `Country`: "Iceland"
- `State`: for example "Texas" but can also be null
- `City`: "Prague"
- `Month`: "June"
- `Day`: 1-31
- `Year`: 2002
- `AvgTemperature`: temperature in celsiu, for example 24
Generally, you follow these rules:
- ALWAYS FORMAT YOUR RESPONSE IN MARKDOWN
- ALWAYS RESPOND ONLY WITH CODE IN CODE BLOCK LIKE THIS:
```python
{code}
```
- the python code runs in jupyter notebook.
- every time you generate python, the code is executed in a separate cell. it's okay to multiple calls to `execute_python`.
- display visualizations using matplotlib or any other visualization library directly in the notebook. don't worry about saving the visualizations to a file.
- you have access to the internet and can make api requests.
- you also have access to the filesystem and can read/write files.
- you can install any pip package (if it exists) if you need to by running `!pip install {package}`. The usual packages for data analysis are already preinstalled though.
- you can run any python code you want, everything is running in a secure sandbox environment
"""
````
As an equivalent to missing function calling, we instruct the model to return messages in Markdown and then parse and extract the Python code block on our own.
```python
import re
pattern = re.compile(r'```python\n(.*?)\n```', re.DOTALL) # Match everything in between ```python and ```
def match_code_block(llm_response):
match = pattern.search(llm_response)
if match:
code = match.group(1)
print(code)
return code
return ""
```
### 3. Implementation of method for code interpreting
Here's the main function that use the E2B code interpreter SDK. We'll be calling this function a little bit further when we're parsing the Codestral's response with tool calls.
```python
def code_interpret(e2b_code_interpreter, code):
print("Running code interpreter...")
exec = e2b_code_interpreter.notebook.exec_cell(
code,
on_stderr=lambda stderr: print("[Code Interpreter]", stderr),
on_stdout=lambda stdout: print("[Code Interpreter]", stdout),
# You can also stream code execution results
# on_result=...
)
if exec.error:
print("[Code Interpreter ERROR]", exec.error)
else:
return exec.results
```
### 4. Adding tmethod for calling Codestral and parsing its response
Now we're going to define and implement `chat` method. In this method, we'll call the Codestral LLM, parse the output to extract any Python code block, and call our `code_interpret` method we defined above.
```python
from mistralai.client import MistralClient
client = MistralClient(api_key=MISTRAL_API_KEY)
def chat(e2b_code_interpreter, user_message):
print(f"\n{'='*50}\nUser message: {user_message}\n{'='*50}")
messages = [
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": user_message}
]
# Codestral doesn't support tools/function calling yet
response = client.chat(
model=MODEL_NAME,
messages=messages,
)
response_message = response.choices[0].message
python_code = match_code_block(response_message.content)
if python_code != "":
code_interpreter_results = code_interpret(e2b_code_interpreter, python_code)
return code_interpreter_results
else:
print(f"Failed to match any Python code in model's response {response_message}")
return[]
```
### 5. Adding method for data upload
Now we implement a method to upload our dataset to the code interpreter sandbox. The file gets uploaded to the E2B sandbox where our code interpreter is running. We get the file's remote path in the `remote_path` variable.
```python
def upload_dataset(code_interpreter):
print("Uploading dataset to Code Interpreter sandbox...")
with open("./city_temperature.csv", "rb") as f:
remote_path = code_interpreter.upload_file(f)
print("Uploaded at", remote_path)
```
### 6. Putting everything together
In this last step, we put all the pieces together. We instantiate a new code interpreter instance using `with CodeInterpreter(api_key=E2B_API_KEY) as code_interpreter:` and then call the `chat` method with our user message and the `code_interpreter` instance.
```python
from e2b_code_interpreter import CodeInterpreter
with CodeInterpreter(api_key=E2B_API_KEY) as code_interpreter:
# Upload the dataset to the code interpreter sandbox
upload_dataset(code_interpreter)
code_results = chat(
code_interpreter,
"Plot average temperature over the years in Algeria"
)
if code_results:
first_result = code_results[0]
else:
raise Exception("No code interpreter results")
# This will render the image
# You can also access the data directly
# first_result.png
# first_result.jpg
# first_result.pdf
# ...
first_result
```
This is how the resulting plot generated by the LLM looks like, based on the dataset.
🔗 [**Full code on GitHub**](https://github.com/e2b-dev/e2b-cookbook/tree/main/examples/codestral-code-interpreter-python)
### Key links
- [Get started with E2B](https://docs.e2b.dev)
- [Get started with Mistral models](https://mistral.ai/)
- [Read more about the new Codestral model](https://mistral.ai/news/codestral/)
- [Follow E2B](https://twitter.com/e2b_dev?lang=en)
---
Source: https://e2b.dev/resources/build-ai-data-analyst-with-sandboxed-code-execution-using-typescript-and-gpt-4o.md
Published: 2024-05-29
# Build AI data analyst with sandboxed code execution using TS, and GPT-4o
This is an example that tests the capabilities of the OpenAI's GPT-4o model. We let it generate a chart and reason about it. We use the E2B Code Interpreter SDK for running the LLM-generated code tasks in a secure and isolated cloud environment.
🔗 [**Full code on GitHub**](https://github.com/e2b-dev/e2b-cookbook/tree/main/examples/openai-js)
## Why Code Interpreter SDK
In this TypeScript example, we will show how to give GPT-4o a code interpreter powered by the open-source [Code Interpreter SDK](https://github.com/e2b-dev/code-interpreter) by E2B. This SDK quickly creates a secure cloud sandbox powered by [Firecracker](https://github.com/firecracker-microvm/firecracker). Inside the E2B sandbox is a running Jupyter server that the LLM can use.
In general, the Code Interpreter SDK allows you to build custom code interpreters. For example, you can install custom packages, have access to the internet, use the filesystem, or connect your cloud storage. The Code Interpreter SDK works with [any LLM](https://github.com/e2b-dev/e2b-cookbook/tree/main). (in this example, we are using it with the OpenAI's newest GPT model.)
#### Key links
- [Get started with E2B](https://docs.e2b.dev)
- [Get started with OpenAI](https://openai.com/index/openai-api/)
- [Learn more about GPT-4o](https://openai.com/index/hello-gpt-4o/)
- [Follow E2B](https://twitter.com/e2b_dev?lang=en)
#### Overview
1. Installations and prerequisites
2. Saving API keys
3. Imports and configuration
4. Setup of model and tools
5. Creating code interpreter
6. Defining interaction with model
7. Running the program
## Setup
### 1. Installations and prerequisites
First, initialize a new `npm` project and install all the dependencies we are going to need.
```bash
npm init -y
npm i @e2b/code-interpreter dotenv openai
npm i --save-dev @types/node ts-node tsx typescript
```
Add the script to start the program. Your "main" file should be called `app.ts`.
```json
"start": "tsx app.ts"
```
Your `package.json` file should look like this:
```json
{
"name": "gpt-4o-code-interpreter",
"version": "1.0.0",
"description": "This is an example of running LLM-generated code tasks in a secure and isolated cloud environment using the E2B Code Interpreter SDK.",
"main": "app.ts",
"scripts": {
"start": "tsx app.ts"
},
"keywords": [],
"author": "",
"license": "ISC",
"devDependencies": {
"@types/node": "^20.12.11",
"ts-node": "^10.9.2",
"tsx": "^4.9.3",
"typescript": "^5.4.5"
},
"dependencies": {
"@e2b/code-interpreter": "^0.0.5",
"dotenv": "^16.4.5",
"openai": "^4.46.1"
}
}
```
### 2. Saving API keys
Now we create the .env file where you store your the [E2B API KEY](https://docs.e2b.dev/api-key) and [OPENAI API KEY](https://platform.openai.com/settings). You can copy [this](https://github.com/e2b-dev/e2b-cookbook/blob/main/examples/openai-js/.env.template) [`.env.template`](https://github.com/e2b-dev/e2b-cookbook/blob/main/examples/openai-js/.env.template) to your `.env` file:
```bash
# TODO: Get your E2B API key from https://docs.e2b.dev/api-key
E2B_API_KEY=""
# TODO: Get your OpenAI API key from https://platform.openai.com/settings.
OPENAI_API_KEY=""
```
### 3. Imports and configuration
In the `app.ts`, we import the file system module, OpenAI's API, the E2B code interpreter for code execution, and `dotenv` for managing environment variables.
We load the environment variables from the `.env`.
```typescript
import fs from 'node:fs'
import { OpenAI } from 'openai'
import { CodeInterpreter, Result } from '@e2b/code-interpreter'
import { ProcessMessage } from '@e2b/code-interpreter'
import * as dotenv from 'dotenv'
import { ChatCompletionTool, ChatCompletionMessageParam } from 'openai/resources/index'
dotenv.config()
```
### 4. Setup of model and tools
Then we define constants and a system prompt to guide the behavior of our Python data scientist assistant. The `MODEL_NAME` constant specifies the LLM we are using, which is `gpt-4o`. The `SYSTEM_PROMPT` is a detailed set of instructions for overall behavior of the AI agent that we are building. We are focusing on analyzing and visualizing data, but you can adjust the prompt according to your use case.
```typescript
const MODEL_NAME = 'gpt-4o'
const SYSTEM_PROMPT = `
## your job & context
you are a python data scientist. you are given tasks to complete and you run python code to solve them.
- the python code runs in jupyter notebook.
- every time you call \`execute_python\` tool, the python code is executed in a separate cell. it's okay to multiple calls to \`execute_python\`.
- display visualizations using matplotlib or any other visualization library directly in the notebook. don't worry about saving the visualizations to a file.
- you have access to the internet and can make api requests.
- you also have access to the filesystem and can read/write files.
- you can install any pip package (if it exists) if you need to but the usual packages for data analysis are already preinstalled.
- you can run any python code you want, everything is running in a secure sandbox environment.
`
```
We list the array of tools that the agent will be able to use. In this case, we give it only one tool called `execute_python` that will leverage the E2B code interpreter.
The tools is defined as a "function" which is an abstract concept that contains the 'parameters' schema ensuring that when the agent "decides" to use a tool, it receives all the necessary information in the correct format.
```typescript
// Creating a list of tools available for the agents
const tools: Array = [
{
'type': 'function',
'function': {
'name': 'execute_python',
'description': 'Execute python code in a Jupyter notebook cell and returns any result, stdout, stderr, display_data, and error.',
'parameters': {
'type': 'object',
'properties': {
'code': {
'type': 'string',
'description': 'The python code to execute in a single cell.',
},
'unit': {'type': 'string', 'enum': ['celsius', 'fahrenheit']},
},
'required': ['code'],
},
}
}
]
```
### 5. Creating code interpreter
In this part, we define a `codeInterpret` function to execute Python code using the E2B [Code Interpreter SDK](https://github.com/e2b-dev/code-interpreter). The function takes two parameters: an instance of the `CodeInterpreter` and the Python code to be executed as a string.
```typescript
// Definine the function to execute code, using the E2B Code Interpreter SDK as a tool
async function codeInterpret(codeInterpreter: CodeInterpreter, code: string): Promise {
console.log('Running code interpreter...')
const exec = await codeInterpreter.notebook.execCell(code, {
onStderr: (msg: ProcessMessage) => console.log('[Code Interpreter stderr]', msg),
onStdout: (stdout: ProcessMessage) => console.log('[Code Interpreter stdout]', stdout),
// You can also stream additional results like charts, images, etc.
})
if (exec.error) {
console.log('[Code Interpreter ERROR]', exec.error)
throw new Error(exec.error.value)
}
return exec.results
}
const openai = new OpenAI() // Initialize openai client
```
Then we define a `chat` function that initializes an array of messages with GPT-4o. It accepts an instance of the `CodeInterpreter`, a user message as a string, and an optional base64-encoded image string. The latter is useful to showcase the capabilities of GPT-4o to "see" visual output it generated and react to it.
The chat function also checks for tool calls, particularly looking for the `execute_python` function. If found, it extracts the code to be executed and calls the previously defined `codeInterpret` function to run the code in the [E2B sandbox](https://docs.e2b.dev/sandbox) (a secure cloud environment).
```typescript
// Define function to chat with the model
async function chat(codeInterpreter: CodeInterpreter, userMessage: string, base64_image?: string): Promise {
console.log(`\n${'='.repeat(50)}\nUser Message: ${userMessage}\n${'='.repeat(50)}`)
const messages: Array = [
{
role: 'system',
content: SYSTEM_PROMPT,
},
]
if (base64_image) {
messages.push({
role: 'user',
content: [
{
type: 'text',
text: userMessage,
},
{
type: 'image_url',
image_url: {
url: `data:image/jpeg;base64,${base64_image}`
}
}
]
})
} else {
messages.push({ role: 'user', content: userMessage })
}
try {
const response = await openai.chat.completions.create({
model: MODEL_NAME,
messages: messages,
tools: tools,
tool_choice: 'auto'
})
for (const choice of response.choices) {
if (choice.message.tool_calls && choice.message.tool_calls.length > 0) {
for (const toolCall of choice.message.tool_calls) {
if (toolCall.function.name === 'execute_python') {
let code: string
if (typeof toolCall.function.arguments === 'object' && 'code' in toolCall.function.arguments) {
code = (toolCall.function.arguments as { code: string }).code
} else {
code = toolCall.function.arguments as string
}
console.log('CODE TO RUN')
console.log(code)
const codeInterpreterResults = await codeInterpret(codeInterpreter, code)
return codeInterpreterResults
}
}
} else {
console.log('Answer:', choice.message.content)
}
}
} catch (error) {
console.error('Error during API call:', error)
}
return []
}
```
### 6. Defining interaction with model
In this step, we put together what we have defined and use it in the `run` function. The function creates an instance of the `CodeInterpreter`. It then makes a call to the `chat` function, asking the LLM to do the task specified in the prompt, in this case generating a random differential equation and a chart with its solution.
You can easily modify the prompt for other data analysis and coding examples.
After the first output, the function makes a second call to the `chat` function. Here it is asking the GPT-4o model to interpret the plot it just created and generate a new plot with different initial conditions.
```typescript
async function run() {
const codeInterpreter = await CodeInterpreter.create()
// Let the model do the task
try {
const codeInterpreterResults = await chat(
codeInterpreter,
'Write a second order differential equation with initial conditions and plot its solutions on a chart, add the conditions to description.'
)
console.log('codeInterpreterResults:', codeInterpreterResults)
const result = codeInterpreterResults[0]
console.log('Result object:', result)
if (result && result.png) {
fs.writeFileSync('image_1.png', Buffer.from(result.png, 'base64'))
} else {
console.log('No PNG data available.')
return
}
const codeInterpreterResults2 = await chat(
codeInterpreter,
'Based on what you see, can you tell me what is the differential equation whose solutions are plotted in the chart? Then make a new plot with the same equation, but different initial conditions.',
result.png
)
console.log('codeInterpreterResults:', codeInterpreterResults2)
const result2 = codeInterpreterResults2[0]
console.log('Result object:', result2)
if (result2 && result2.png) {
fs.writeFileSync('image_2.png', Buffer.from(result2.png, 'base64'))
} else {
console.log('No PNG data available.')
}
} catch (error) {
console.error('An error occurred:', error)
} finally {
await codeInterpreter.close()
}
}
run()
```
### 7. Running the program
Finally, we can start the program. If the results contain PNG data, it is saved to a new file in your directory. (If no PNG data is available, it logs a message.)
```bash
npm run start
```
🔗 [**Full code on GitHub**](https://github.com/e2b-dev/e2b-cookbook/tree/main/examples/openai-js)
### Key links
- [Get started with E2B](https://docs.e2b.dev)
- [Get started with OpenAI](https://openai.com/index/openai-api/)
- [Learn more about GPT-4o](https://openai.com/index/hello-gpt-4o/)
- [Follow E2B](https://twitter.com/e2b_dev?lang=en)
---
Source: https://e2b.dev/resources/launching-the-code-interpreter-sdk.md
Published: 2024-05-06
# Launching the Code Interpreter SDK
Building a good product with underlying AI agents means overcoming the challenges of hallucinations, and unreliability, and navigating the agent to use the right tools. One way to solve these problems is to equip agents with code execution capabilities. We see more and more agents powered by a code interpreter.
Examples include [**Flint**](https://www.flintk12.com/) (AI tutoring assistant), [**Athena Intelligence**](https://athenaintel.com/) (enterprise data analysis) or [**Maisa**](https://maisa.ai/) (knowledge processing unit).
The power of code interpreters is also shown with open-source AI software developers like [**OpenDevin**](https://github.com/OpenDevin/OpenDevin).
## Code Interpreter SDK
At E2B, we are giving AI agents their own computers. We are building the code interpreting layer for AI apps and agents, allowing them to run the LLM-generated actions in a secure and isolated cloud environment.
We just released the [Code Interpreter SDK](https://github.com/e2b-dev/code-interpreter) - open-source building block for AI developers. The SDK makes it easy to add code interpreting to AI apps. The Code Interpreter SDK is built on top of our open-source [runtime for AI agents](https://github.com/e2b-dev/e2b). Start with [our docs](https://docs.e2b.dev).
## Features
The Code Interpreter SDK, and also the core E2B SDK works with any LLM and any popular AI framework like LangChain, AutoGen, or CrewAI.
The SDK has Python or JS version, and supports streaming content like charts and stdout, stderr. It runs on serverless and edge functions and executes the AI-generated code in secure sandboxed environments. It is 100% open source (including [runtime](https://github.com/e2b-dev/runtime)).
For inspiration, see our [Cookbook](https://github.com/e2b-dev/e2b-cookbook) with examples of using the Code Interpreter SDK with [LangChain](https://github.com/e2b-dev/e2b-cookbook/tree/main/examples/langchain-python), [Claude](https://github.com/e2b-dev/e2b-cookbook/tree/main/examples/claude-code-interpreter-python), [Llama 3](https://github.com/e2b-dev/e2b-cookbook/blob/main/examples/groq-code-interpreter-python/groq_code_interpreter.ipynb), [Next.js](https://github.com/e2b-dev/e2b-cookbook/tree/main/examples/nextjs-code-interpreter), and more.
## Built with the Code Interpreter SDK
We built the special SDK for code interpreting, becaue this is the main use-case we are observing among our customers. Examples of companies using the E2B code interpreting layer for their product are:
- Cognosys - AI agent automating everyday tasks like summarizing emails or creating market reports
- [PGA](https://www.pga.com/) - One of the world's largest sports organizations
- [Menza](https://menza.ai/) - Company transforming unstructured data into insights
- [Flint](https://www.flintk12.com/) - AI tutoring for personalized learning
- [Athena Intelligence](https://athenaintel.com/) - Data analyst for enterprise-level companies
- [Maisa](https://maisa.ai/) - AI system that improves reasoning of LLMs.
## Cookbook examples
##### **The SDK works with any LLM**
- [Anthropic Claude 3 Opus with code interpreter](https://github.com/e2b-dev/e2b-cookbook/tree/main/examples/claude-code-interpreter-python)
- 🦙 [Llama 3 with code interpreter](https://github.com/e2b-dev/e2b-cookbook/blob/main/examples/groq-code-interpreter-python/groq_code_interpreter.ipynb)
- [Codestral with code interpreter](https://github.com/e2b-dev/e2b-cookbook/tree/main/examples/codestral-code-interpreter-python)
##### **And you can try it with popular AI frameworks**
- 🦜⛓️ [LangChain with code interpreter](https://github.com/e2b-dev/e2b-cookbook/tree/main/examples/langchain-python)
- 🦜🕸️ [LangGraph with code interpreter](https://github.com/e2b-dev/e2b-cookbook/tree/main/examples/langgraph-python)
- [Autogen with secure sandboxed code interpreter](https://github.com/e2b-dev/e2b-cookbook).
#### Contact us
Need help setting up E2B? We are happy to connect! Give us any feedback on your experience with E2B.
## We are hiring!
Check out the [open positions](https://e2bdev.notion.site/Careers-at-E2B-2163f176991f43f69b0984bf2a142920) at E2B. We’re a pre-seed startup with a [small team](https://www.notion.so/Careers-at-E2B-2163f176991f43f69b0984bf2a142920?pvs=21) focused on shipping. We work in-person from our office in San Francisco.
We’re backed by founders like [Guillermo Rauch](https://twitter.com/rauchg) (CEO of [Vercel](https://vercel.com/)), [Paul Copplestone](https://twitter.com/kiwicopple) (CEO of [Supabase](https://supabase.com/)), [Juraj Masar](https://www.linkedin.com/in/jurajmasar/) (CEO of [Better Stack](https://betterstack.com/)), [Jakub Jurových](https://www.linkedin.com/in/jakubjurovych/) (CEO of [Deepnote](https://deepnote.com/join-us)) or [Flo Crivello](https://twitter.com/Altimor) (CEO of [Lindy](https://lindy.ai/)), together with people from companies like Stripe, Retool, Figma, OpenAI, and Google.
---
Source: https://e2b.dev/resources/how-to-add-code-interpreter-to-llama-3.md
Published: 2024-04-22
# How to add code interpreter to Llama 3
At this moment, Llama 3 is one of the most capable open-source models. In this guide, we give Llama 3 code interpreter capabilities and test it on data analysis and data visualization task.
🔗 [**Full code on GitHub**](https://github.com/e2b-dev/e2b-cookbook/blob/main/examples/groq-code-interpreter-python/groq_code_interpreter.ipynb)
## Code Interpreter SDK
We will show how to build a code interpreter with Llama 3 on [Groq](https://groq.com/), and powered by **open-source** [**Code Interpreter SDK**](https://github.com/e2b-dev/code-interpreter) by E2B. The E2B Code Interpreter SDK quickly creates a secure cloud sandbox powered by [Firecracker](https://github.com/firecracker-microvm/firecracker). Inside this sandbox is a running Jupyter server that the LLM can use.
#### Key links
- [Get started with E2B](https://docs.e2b.dev)
- [Get started with Llama](https://llama.meta.com/docs/get-started/)
- [Get started with Groq](https://wow.groq.com/docs/)
- [Follow E2B](https://twitter.com/e2b_dev?lang=en)
#### Overview
1. Setup
2. Configuration and API keys
3. Creating code interpreter
4. Calling Llama 3
5. Connecting Llama 3 and code interpreter
### 1. Setup
We will be working in Jupyter notebook. First, we install the E2B code interpreter SDK and [Groq's Python SDK](https://console.groq.com/).
```python
%pip install groq e2b_code_interpreter
```
```text
Collecting groq
Downloading groq-0.5.0-py3-none-any.whl (75 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 75.0/75.0 kB 937.3 kB/s eta 0:00:00
Collecting e2b_code_interpreter
Downloading e2b_code_interpreter-0.0.3-py3-none-any.whl (10.0 kB)
Requirement already satisfied: anyio<5,>=3.5.0 in /usr/local/lib/python3.10/dist-packages (from groq) (3.7.1)
Requirement already satisfied: distro<2,>=1.7.0 in /usr/lib/python3/dist-packages (from groq) (1.7.0)
Collecting httpx<1,>=0.23.0 (from groq)
Downloading httpx-0.27.0-py3-none-any.whl (75 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 75.6/75.6 kB 4.2 MB/s eta 0:00:00
Requirement already satisfied: pydantic<3,>=1.9.0 in /usr/local/lib/python3.10/dist-packages (from groq) (2.7.0)
Requirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from groq) (1.3.1)
Requirement already satisfied: typing-extensions<5,>=4.7 in /usr/local/lib/python3.10/dist-packages (from groq) (4.11.0)
Collecting e2b>=0.14.11 (from e2b_code_interpreter)
Downloading e2b-0.14.14-py3-none-any.whl (100 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100.3/100.3 kB 4.8 MB/s eta 0:00:00
Requirement already satisfied: websocket-client<2.0.0,>=1.7.0 in /usr/local/lib/python3.10/dist-packages (from e2b_code_interpreter) (1.7.0)
Requirement already satisfied: idna>=2.8 in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3.5.0->groq) (3.7)
Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3.5.0->groq) (1.2.1)
Collecting aenum>=3.1.11 (from e2b>=0.14.11->e2b_code_interpreter)
Downloading aenum-3.1.15-py3-none-any.whl (137 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 137.6/137.6 kB 2.5 MB/s eta 0:00:00
Requirement already satisfied: aiohttp>=3.8.4 in /usr/local/lib/python3.10/dist-packages (from e2b>=0.14.11->e2b_code_interpreter) (3.9.5)
Collecting jsonrpcclient>=4.0.3 (from e2b>=0.14.11->e2b_code_interpreter)
Downloading jsonrpcclient-4.0.3-py3-none-any.whl (7.0 kB)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.10/dist-packages (from e2b>=0.14.11->e2b_code_interpreter) (2.8.2)
Requirement already satisfied: requests>=2.31.0 in /usr/local/lib/python3.10/dist-packages (from e2b>=0.14.11->e2b_code_interpreter) (2.31.0)
Requirement already satisfied: urllib3>=1.25.3 in /usr/local/lib/python3.10/dist-packages (from e2b>=0.14.11->e2b_code_interpreter) (2.0.7)
Collecting websockets>=11.0.3 (from e2b>=0.14.11->e2b_code_interpreter)
Downloading websockets-12.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (130 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 130.2/130.2 kB 4.2 MB/s eta 0:00:00
Requirement already satisfied: certifi in /usr/local/lib/python3.10/dist-packages (from httpx<1,>=0.23.0->groq) (2024.2.2)
Collecting httpcore==1.* (from httpx<1,>=0.23.0->groq)
Downloading httpcore-1.0.5-py3-none-any.whl (77 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 77.9/77.9 kB 4.7 MB/s eta 0:00:00
Collecting h11<0.15,>=0.13 (from httpcore==1.*->httpx<1,>=0.23.0->groq)
Downloading h11-0.14.0-py3-none-any.whl (58 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.3/58.3 kB 3.9 MB/s eta 0:00:00
Requirement already satisfied: annotated-types>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=1.9.0->groq) (0.6.0)
Requirement already satisfied: pydantic-core==2.18.1 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=1.9.0->groq) (2.18.1)
Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp>=3.8.4->e2b>=0.14.11->e2b_code_interpreter) (1.3.1)
Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp>=3.8.4->e2b>=0.14.11->e2b_code_interpreter) (23.2.0)
Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp>=3.8.4->e2b>=0.14.11->e2b_code_interpreter) (1.4.1)
Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp>=3.8.4->e2b>=0.14.11->e2b_code_interpreter) (6.0.5)
Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp>=3.8.4->e2b>=0.14.11->e2b_code_interpreter) (1.9.4)
Requirement already satisfied: async-timeout<5.0,>=4.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp>=3.8.4->e2b>=0.14.11->e2b_code_interpreter) (4.0.3)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.8.2->e2b>=0.14.11->e2b_code_interpreter) (1.16.0)
Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests>=2.31.0->e2b>=0.14.11->e2b_code_interpreter) (3.3.2)
Installing collected packages: aenum, websockets, jsonrpcclient, h11, httpcore, httpx, e2b, groq, e2b_code_interpreter
Successfully installed aenum-3.1.15 e2b-0.14.14 e2b_code_interpreter-0.0.3 groq-0.5.0 h11-0.14.0 httpcore-1.0.5 httpx-0.27.0 jsonrpcclient-4.0.3 websockets-12.0
```
### 2. Configuration and API keys
Then we store the Groq and E2B API keys and set the model name for the Llama 3 instance you will use. In the system prompt we define sets the rules for the interaction with Llama.
```python
# TODO: Get your Groq AI API key from https://console.groq.com/
GROQ_API_KEY = ""
# TODO: Get your E2B API key from https://docs.e2b.dev
E2B_API_KEY = ""
# Or use 8b version
# MODEL_NAME = "llama3-8b-8192"
MODEL_NAME = "llama3-70b-8192"
SYSTEM_PROMPT = """you are a python data scientist. you are given tasks to complete and you run python code to solve them.
- the python code runs in jupyter notebook.
- every time you call `execute_python` tool, the python code is executed in a separate cell. it's okay to multiple calls to `execute_python`.
- display visualizations using matplotlib or any other visualization library directly in the notebook. don't worry about saving the visualizations to a file.
- you have access to the internet and can make api requests.
- you also have access to the filesystem and can read/write files.
- you can install any pip package (if it exists) if you need to but the usual packages for data analysis are already preinstalled.
- you can run any python code you want, everything is running in a secure sandbox environment"""
tools = [
{
"type": "function",
"function": {
"name": "execute_python",
"description": "Execute python code in a Jupyter notebook cell and returns any result, stdout, stderr, display_data, and error.",
"parameters": {
"type": "object",
"properties": {
"code": {
"type": "string",
"description": "The python code to execute in a single cell.",
}
},
"required": ["code"],
},
},
}
]
```
### 3. Creating code interpreter
We define the main function that uses the E2B code interpreter to execute code in a Jupyter Notebook cell. We'll be calling this function a little bit further when we're parsing the Llama's response with tool calls.
```python
def code_interpret(e2b_code_interpreter, code):
print("Running code interpreter...")
exec = e2b_code_interpreter.notebook.exec_cell(
code,
on_stderr=lambda stderr: print("[Code Interpreter]", stderr),
on_stdout=lambda stdout: print("[Code Interpreter]", stdout),
# You can also stream code execution results
# on_result=...
)
if exec.error:
print("[Code Interpreter ERROR]", exec.error)
else:
return exec.results
```
### 4. Calling Llama 3
Now we're going to define and implement `chat_with_llama` method. In this method, we'll call the LLM with our `tools` dictionary, parse the output, and call our `code_interpret` method we defined above.
See the [Groq documentation](https://wow.groq.com/docs/) to get started.
```python
import os
import json
import re
from groq import Groq
client = Groq(api_key=GROQ_API_KEY)
def chat_with_llama(e2b_code_interpreter, user_message):
print(f"\n{'='*50}\nUser message: {user_message}\n{'='*50}")
messages = [
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": user_message}
]
response = client.chat.completions.create(
model=MODEL_NAME,
messages=messages,
tools=tools,
tool_choice="auto",
max_tokens=4096,
)
response_message = response.choices[0].message
tool_calls = response_message.tool_calls
if tool_calls:
for tool_call in tool_calls:
function_name = tool_call.function.name
function_args = json.loads(tool_call.function.arguments)
if function_name == "execute_python":
code = function_args["code"]
code_interpreter_results = code_interpret(e2b_code_interpreter, code)
return code_interpreter_results
else:
raise Exception(f"Unknown tool {function_name}")
else:
print(f"(No tool call in model's response) {response_message}")
return []
```
### 5. Connecting Llama 3 and code interpreter
Finally, we can instantiate the code interpreter and pass the E2B API key. Then we call the `chat_with_llama` method with our user message and the `code_interpreter` instance.
```python
from e2b_code_interpreter import CodeInterpreter
with CodeInterpreter(api_key=E2B_API_KEY) as code_interpreter:
code_results = chat_with_llama(
code_interpreter,
"Visualize a distribution of height of men based on the latest data you know"
)
if code_results:
first_result = code_results[0]
else:
print("No code results")
exit(0)
# This will render the image
# You can also access the data directly
# first_result.png
# first_result.jpg
# first_result.pdf
# ...
first_result
```
```text
==================================================
User message: Visualize a distribution of height of men based on the latest data you know
==================================================
Running code interpreter...
```
🔗 [**Full code on GitHub**](https://github.com/e2b-dev/e2b-cookbook/blob/main/examples/groq-code-interpreter-python/groq_code_interpreter.ipynb)
---
Source: https://e2b.dev/resources/open-source-alternatives-to-devin.md
Published: 2024-04-18
# Open-Source Alternatives to Devin
A startup called Cognition AI recently caused controversy by releasing an AI assistant called Devin. In a demo video, this AI software developer was seen working on tasks typically done by high-paid software engineers. One of the creators' goals is to enable Devin to contribute code successfully to large, complex codebases, making a lot of technical people fear about their jobs.
There are quite many coding AI agents and assistants already, but Devin caught everyone’s attention because of these reasons things:
- [Amazing demo](https://www.youtube.com/watch?v=fjHtjT7GO1c&ab_channel=Cognition) showing the assistant completing a wide range of more advanced tasks
- Nice UI, which many agents still don't have
- The ability to not only generate, but execute code
- Scoring relatively high in a SWE-bench.
## What is the SWE-bench?
In October 2023, a [paper](https://arxiv.org/abs/2310.06770) called “SWE-bench: Can Language Models Resolve Real-World GitHub Issues?” introduced the SWE benchmark.
The previous coding benchmarks for LLMs all have a similar format. They present a coding exercise to be solved and evaluate models’ responses by running the generated function on some held-out unit tests. Such tests do not represent the real world tasks accurately.
SWE-bench is a set of 2,294 tasks based on pairs of issues and pull requests from popular Python repositories on GitHub. They form an evaluation set requiring models to understand problems in the context of a real codebase, find and isolate bugs among thousand-line files, and generate solutions that could interact with multiple parts of code.
The best-performing model, Claude 2, is able to solve a mere 1.96% of the issues. Advances on SWE-bench represent steps toward LLMs that are more practical, intelligent, and autonomous.]
### Code execution
Chatbots like ChatGPT, Perplexity, and Phind can generate code, but Devin went further. It can also run, test, and implement the code, using a sandbox as runtime. (Read more about how [sandboxes](https://docs.e2b.dev) might work for running LLM-generated code).
Devin is the latest example of the emerging AI agents powered by a [code interpreter](/resources/llm-powered-code-interpreters) - which makes it extra powerful and able to take action instead of just giving a piece of passive advice. It's not the first agent with code execution (this can be also achieved by [Open Interpretrer](https://docs.openinterpreter.com/integrations/e2b), [AutoGen](https://github.com/microsoft/autogen), or [ChatGPT Data Analyst](https://chat.openai.com/)), but such capability is still not that common.
## Open-source alternatives
The way Devin works has still been seen by most people only in the demo as only a few selected individuals have received access. Luckily, people were quick to build open-source alternatives to try. (Some of them, like [Devika](https://x.com/mufeedvh/status/1770696907943219338) started as a fun hackathon-style projects. Creator of Devika made the first version in about 20 hours.)
Open-source is a preferred choice by many developers, and even the original Devin even started working on one of the other “Devins".
Here are some of the popular open-source alternatives to Devin and their capabilities and benchmarks.
### 1. OpenDevin
One of the main reasons Devin got so popular and even reached general public people is the UI where you can see the agents work, without having to use terminal.OpenDevin offers a nice UI as well, and it has received over 20.7k stars. The techstack currenyly used in the project include FastAPI, uvicorn, LiteLLM, Docker, Ruff, MyPy, LlamaIndex, and React.OpenDevin executes code via Docker, though there is ongoing work to add an option for executing code in the [E2B sandboxed environment](https://github.com/OpenDevin/OpenDevin/pull/727). The [sandbox](https://docs.e2b.dev) is a secure micro VM made for running AI agents and AI-generated code in the cloud. With that, it would be very easy for OpenDevin to download things from internet, use filesystem, a browser, or ability to run different languages besides Python.
### 2. AutoCodeRover
Even though AutoCodeRover doesn’t reach the popularity of OpenDevin yet, it was able to resolve approximately 16% of issues on the SWE-bench (totaling 2,294 GitHub issues) and about 22% of issues on the SWE-bench lite (totaling 300 GitHub issues).
How did AutoCodeRover achieve such results? It operates in two stages. First, in the context retrieval stage, the LLM is equipped with code search APIs to navigate the codebase and gather relevant context. The code search APIs are Structure-aware. Instead of searching over files using plain string matching, the agent searches for relevant code context (such as methods or classes) in the abstract syntax tree.
Then comes the patch generation where the LLM attempts to write a patch based on the retrieved context. When a test suite is available, [AutoCodeRover](https://github.com/nus-apr/auto-code-rover) can leverage test cases to achieve an even higher repair rate by performing statistical fault localization.
### 3. Devika
Devika is another Agentic AI software engineer with over 15k GitHub stars that aims to compete with Devin by Cognition AI. Devika supports Claude 3, GPT-4, GPT-3.5, and Local LLMs via Ollama. Its architecture includes a Code Writing Module that generates code based on the plan, a Browser Interaction Module that enables Devika to extract information from websites.
In their [demo](https://www.youtube.com/watch?v=GBvNxHiquKM&ab_channel=WorldofAI), Devika is creating a Game of Life using pygame. This is the same task Devin team showed in their famous demo.
To start, Devika requires you to have API key of OpenAI or Claude, Bing, and Netlify, but it is still great to be able to try an agent that accomplishes tasks similar to Devin.
### 4. Anterion
[Anterion](https://www.youtube.com/watch?v=J-KZNFVcAxU&ab_channel=Anterion) AI Agent aims to extend the capabilities from GitHub issues to open-ended general engineering tasks, and their frontend is inspired by OpenDevin.
[According to the creators](https://www.youtube.com/watch?v=J-KZNFVcAxU&ab_channel=Anterion), they have big plans, for example, adding Vercel, and more features in the future. In their demo, they used the agent to get the number of stars from popular AI software engineer repositories.
### 5. MetaGPT
MetaGPT is a multi-agent framework that has existed longer than other Devin alternatives. However, following the release of Devin, the MetaGPT team [aims to replace most of the Devin capabilities](https://x.com/MetaGPT_/status/1767892377883095237) and the creators shared on X (Twitter) that they exceeded the Devin’s score (with details to be released later).I particularly like their [Data Interpreter](https://docs.deepwisdom.ai/main/en/DataInterpreter/) agent that you can try for data visualization, solving math problems, web crawling, and more.
### 6. AutoDev
AutoDev has one big advantage and that is a multilingual support. It supports languages like Python, and JavaScript/TypeScript, but also Rust, Java, Kotlin, Golang, or C/C++/OC.
AutoDev is still not that easy to access for a normal user and requires coding knowledge to set up. That makes it slightly disadvantageous among other alternatives, but it's still in its early stage.
In its demo, AutoDev shows capabilities like debugging and customizable prompts.
### 7. Devon
Devon is not as well-known as an open-source alternative to Devin by [Entropy Research](https://github.com/entropy-research), with ~200 GitHub stars. It’s an open-source SWE agent built with Python and designed to assist software engineers with the development and maintenance of software. While it does not offer an extensive description yet, it highlights features such as reliable multi-file editing and the use of tools like git.The creators shared with me that they are working on putting Devon through the SWE-bench, so we can stay tuned to see the score.
### 8. SWE-agent
This Devin alternative was made by the SWE-bench authors, and it scores 12.3% on the full SWE-bench.There are changes and innovations in [SWE-agent](https://swe-agent.com/) compared to Devin. It executes code locally via Docker. It uses a constrained "Agent-Computer Interface" (ACI), making the agent more user-friendly for LLMs. Only a few commands are allowed: run code, look for code, edit code, and submit changes to GitHub.
The agent's code goes through a syntax check (linter) before submission, and if the syntax is incorrect, the agent receives feedback and is compelled to rewrite the code. The agent can only read 100 lines of code at a time, simplifying the language model's understanding of the code.
The creators [highlight](https://x.com/jyangballin/status/1775114448513958134) that LLMs require carefully designed agent-computer interfaces, akin to how humans appreciate good user interface (UI) design. For instance, when the LM makes a mistake with indentation, the SWE-agent editor prevents it and provides feedback.
## So… are Devins replacing software engineers?
I’ve seen a lot of people asking whether engineers and developers are really being replaced by teams of Devins. But is it that big of a threat for developers? The Devin demo was followed by other videos [debunking its capabilities](https://www.youtube.com/watch?v=tNmgmwEtoWE&ab_channel=InternetofBugs) and even claiming that Cognition is overselling their agent.
Looking closely at the SWE-bench, it is a great, but still not a 100% perfect measure either. The bench only includes Python tasks at this moment. If you’re an organization building mobile applications or work with Java, Go, Swift or Typescript then progress against SWE-bench by AI software engineers or models has limited use to you.
In conclusion, “Devins” are not there yet to replace engineers, but you can still follow their progress closely, for example on this [GitHub map](https://github.com/e2b-dev/awesome-devins?tab=readme-ov-file). Explore the full list of open-source Devins and make a pull request if you want to include a new Devin-like project.
---
Source: https://e2b.dev/resources/reacteval-building-llm-benchmark-for-frontend.md
Published: 2024-03-18
# ReactEval: Building LLM Benchmark for Frontend
James Murdza created [GitWit](https://gitwit.dev/), an AI-powered online editor for creating React applications. To be able to evaluate the LLM agents within GitWit, James is building [ReactEval](https://github.com/gitwitorg/react-eval), one of the first LLM benchmarks for frontend. We talked about how he automates executing hundreds of runs for each test, how ReactEval helps in building better products, and his view on the AI space.
## Journey to creating GitWit and ReactEval
##### Can you share a bit about how you started building GitWit and ReactEval and introduce these to readers?
I started working on GitWit about one year ago. Back then, code generation with LLMs was fairly new. The initial idea was to make programming new applications as easy as possible, particularly for less experienced developers. I focused on things that were really difficult for developers - like managing dependencies or finding working boilerplate code - and built GitWit to automate such tasks using AI.
Since then, GitWit has evolved to a visual editor that allows you to make frontend React applications using AI.
ReactEval solves a problem that we came across while building GitWit. Figuring out how well an LLM agent is doing at solving a particular task is an ongoing problem for a lot of people, that there haven't been a lot of great solutions for. Very early on OpenAI created something called [HumanEval](https://github.com/openai/human-eval) which tests GPT on solving Python problems. But that was three years ago and in the meantime, the use cases for code generation have become much more advanced.
I realized that the problem I was solving with GitWit generating React JavaScript code wasn't so unique so I wanted to create a common open-source framework where people could measure what they're doing both internally and then externally to compare different models and agents. And we use [E2B](https://docs.e2b.dev) for building ReactEval.
##### Before we make a deep dive into ReactEval, what was the journey to GitWit? From what I understand, today, GitWit is an online tool to build web apps while augmenting your own coding skills with AI. How did the product evolve towards this in the last few months?
We started building an AI agent that you would feed with a single prompt like “Make me a NextJS app with Stripe integration that sells shirts” or something like that, and then the agent would try and come up with a code repository. It would try and create all the files needed for the whole application.
That was the first product that got a lot of usage, but it's just so difficult to solve the general software development use case, and I also realized that the longer the agent is running autonomously, the less the human can actually do to contribute with their own skills.
That’s why we pivoted to building an editor that gives you a live preview of the application you're building. In this case, the goal is to make the code generation run as fast as possible, allowing a human user to give their input as soon as the generation is finished. We're trying to create this natural development process that doesn't feel like you're ever waiting for the AI to finish anything.
##### So you moved toward more frequent iteration with the human instead of waiting for the AI to make everything from scratch…
Yes, in the previous version, you would be waiting for a few minutes for the AI agent to generate the whole app. Now, you're generally waiting for the agent to produce around a hundred lines of code, which takes around 10 seconds making GitWit faster than for example [Vercel’s V0](https://v0.dev/). And in the upcoming version, it will be even faster.
##### How do I, as an end user, start using GitWit?
You just visit [GitWit.dev](https://gitwit.dev/) and you can immediately create a new app without even signing in. Then you have a working app, which you can just edit there with the code inputs. Then you can use AI prompts that make twists in the app and can rewrite it - for the prompting feature you have to sign up.
In conclusion, right now people can make a basic frontend MVP with GitWit, and then export that. You can also create a button to share the app directly with other people and get a unique URL for others to use your app.
##### That is pretty exciting! What is on your roadmap currently?
One thing that we're working on right now, and we're about to release is a version where the AI can handle multiple files initially to simplify things for the editor.
Now we just use the AI to edit one file at a time which works really well, but we are going to allow it now to refactor into multiple files. That means let's say you want. to build an app where you make a header, make a photo, make a sidebar at a map at a list view, … Then the AI will automatically split those components up into different files, which can then be reused. The idea is that you would just directly connect it to make your apps, but right now you can just click a button and then it gives you the whole source code as a zip file.
## Running browser tests in sandboxes
##### Benchmarking and evaluation of LLMs and AI agents have been a huge topic almost since the boom of AI agents. Can you share your thinking behind creating ReactEval?
When you want to do LLM benchmarking, where things get difficult is when you try to evaluate an LLM, which is by nature non-deterministic, so it's not just enough to run a couple of tests. I need to run the same test at least ten times in order to see if a certain if the LLM is even working well on a certain task.
For example, if I hypothetically want to make sure that GitWit is really good at adding a button (which it *is* good at by the way), I can perform ten tests, and get the same result nine times, but one time it doesn’t work. If I had just done five tests, maybe I wouldn't have gotten this important piece of information.
That just shows we need to perform hundreds of tests to achieve a good level of confidence. Every time we make a change, we need to test the app a hundred times to confirm that it is still performing the right way.
To summarize, I now need to run hundreds of runs in each test, and during the runs, I actually have to build a React application, run it in a web browser, and then observe if there are any errors. To do this manually would take me hours.
##### How do you then automate the tests in ReactEval to be able to run that many tests and not spend a whole day by that?
To automate all these processes we use [E2B sandboxes](https://docs.e2b.dev) for the code execution in a web browser. It's very lightweight. In particular, we use a [custom sandbox](https://docs.e2b.dev/template/quickstart) - its job is to build a React JavaScript application, open it in a browser, and then check for any errors. Each run within each test has its own separate sandbox instance.
When performing these tests, I just run the sandbox a hundred times with each code that I want to test, and then record the results so we can put together a report of errors and analyze what caused them.
##### What types of errors are you usually getting when running the tests?
There are two kinds of error messages.
Either you receive an exit code in case you get a standard error from the testing process itself, for example, if the app fails to build because of a TypeScript error.
Another type of error is the one in the JavaScript console. Once the app builds and runs in the browser using Puppeteer, then you may still get errors while the app is running.
During the tests, we record both kinds of errors, but we check mainly the errors that come out of the browser because the most frustrating part with GitWit is when it generates some code that causes an error in the browser. It happens a significant amount of the time using GPT because it has limited knowledge. GPT performs very well at coding but it knows very little about what things are compatible with each other so it might install two different packages that are just completely not compatible with each other, so the code can look completely fine, but the program still doesn’t run in the end.
##### What were other difficulties you encountered when finding the optimal solution for running the tests?
The most challenging part of the whole process was setting up computing resources for the evaluation process. Before trying E2B I experimented with [AWS](https://aws.amazon.com/), but I'm not an expert in AWS so this took me a lot of time.
E2B sandbox turned out to be a lot easier than using AWS, specifically for quickly creating a container and accessing the filesystem.
##### Can you describe in more detail what is happening with the sandbox and how its features help with running the tests?
The benefit of using the sandbox for performing a test is that we can simply copy a base project and don't have to reinstall everything for React from scratch.
Our big goal is to make it faster to install everything in the sandbox from the beginning. The sandbox works just like a Docker container that has Puppeteer and Chrome in it. When we want to run a test, we open the sandbox and rewrite the template code inside.
The sandbox we spawn already has our template for a blank React app inside, and this template gets modified before we start running the test. The sandbox allows us to perform operations and change files inside it, so we change the application code, build the application, and then run it in a browser.
## Comparison of LLMs
##### Everyone is anticipating the new GPT and the models get better and better at reasoning. How do you think the new LLMS will impact what you’re building?
OpenAI team has said that they will never let GPT get as outdated again as it is now. However, there is a trade-off between being up-to-date and speed. If you look at GPT-4, it is almost three times smaller than GPT-3.
We use ReactEval to demonstrate this trade-off for LLMs and solve questions like how much one LLM is slower, given it has 20% better performance than another LLM. For example, if one wants to switch to Mistral, we can evaluate the pros and cons and quantify the performance when we run it through ReactEval.
We are planning to soon publish results of how well the different LMS compare for producing React code.
##### So, the users can switch between LLMs, based on the tests performed by ReactEval. What are other actionable things to do, based on your findings from the tests?
Yes, as a very fundamental thing, you can switch models, but you can also do anything else imaginable to improve your app.
For example, you can implement retrieval augmented generation (RAG) and add context to your prompts with documentation from different libraries. We can measure whether that's improving your project, or having no effect, how big the effect is, or how the performance changes compared to let’s say two weeks ago.
The main improvement that we've made so far is with dependencies, which is a really annoying problem for many developers. No model is trained on a hundred percent up-to-date data, they all have a cut-off date, but we allow the models to use any newer packages than the date that they were trained on. That means we need to figure out what versions the models are referencing and install the right versions, which the model itself doesn’t know (that's a scary thing).
## Open-sourcing the product
##### You are using ReactEval to improve your product - GitWit. Are you offering ReactEval to other companies to use?
Yes, we're working with one other company in the same space right now to enable them to do the same thing we're doing. It's very easy, just a few lines of code that need to be changed to get ReactEval to work for their products. We want to offer what we built with ReactEval to other companies in the space as well. I think collaboration would just be a win for everyone.
##### You built quite an impressive product with ReactEval. How do you communicate the advantages of evaluating LLMs and LLM-powered apps to your users?
The downside of working with text prompts is that users don't know the full scope of what they can do with the product. We have to record a lot of demos to show people what's even possible with ReactEval.
I made some videos where we explained what was going on in the code - this is helping a lot of people to understand all of the hard work that needs to go on before you have something that “magically” works. The demos brought more companies that we hadn’t heard about before to collaborate with us.
##### Sounds good! What’s your next step with ReactEval now?
I think showing the product in demo videos is still not enough, so now we are open-sourcing our code generation algorithms because it's a long journey and we're going to need a lot of people working together.
##### How frontend-specific is your solution? In general, what are other approaches to the evaluation of LLMs and LLM apps?
One option is to analyze users’ historical data. We don't have thousands of users using it every day, so this approach wouldn’t work for us. Another approach is creating an evaluation platform where you can manage the data of the evaluation.
For us, the hard part of the process isn't the data management, but the problem that we solved with the [E2B sandbox](https://docs.e2b.dev). Which is, how can we build and test hundreds of React applications in 10 minutes because, for example, my own computer wouldn't have enough RAM to do that.
We just need to run hundreds of apps in the browser and the rest of the process is mostly just storing the results in JSON files or database, which is a piece of cake compared to that.
Note that what we're doing is very specific to this frontend generation, and it won't apply to other areas very much.
### Discover more
- [GitWit - webpage](https://gitwit.dev/)
- [ReactEval - GitHub](https://github.com/gitwitorg/react-eval)
- [James Murdza - X (Twitter) profile](https://twitter.com/jamesmurdza)
- [GitWit - X (Twitter) profile](https://twitter.com/GitWitDev)
---
Source: https://e2b.dev/resources/building-open-source-codebase-copilot-with-code-execution-layer.md
Published: 2024-03-12
# Building Open-Source Codebase Copilot with Code Execution Layer
**Ted Spare** is one of three co-founders of [Rubric Labs](https://rubriclabs.com/) - an AI-focused software studio. The Rubric Labs team works with startups and financial firms, builds AI-enabled products and features with them, and invests in open-source. They also build their own AI projects like assistants for various coding tasks or [calendar management](https://cal.com/ai) (Cal).
We asked Ted what it's like to build an AI-powered GitHub copilot, how he makes techstack choices, and how he leverages the open-source community.
## Introducing a GitHub codebase copilot
##### One of your popular projects within Rubric Labs is [Maige](https://maige.app/). Can you please introduce it?
Maige is an AI companion for codebase managers or open-source maintainers. It can do anything that a person could do with the GitHub API, from labeling issues to reviewing pull requests. It can even start to write basic code and run it in [E2B sandboxes](https://docs.e2b.dev?ref=e2b-blog).
##### I’ve seen that Maige is in the alpha stage as of today. Who are your users and how do you work with them?
Maige has been installed on more than 2,000 GitHub repos. For example, the issues-labeling use case is being used on [Cal.com](http://cal.com/), a repository with over 27,000 stars and hundreds of collaborators or contributors. We spend a lot of time with users, watching them go through the onboarding flow. That helps us to spot papercuts and see how they interact with the product.
## Techstack considerations
##### What does the AI agent’s workflow look like and what is the underlying architecture?
Open-source maintainers integrate Maige with their repository (this takes about three clicks) and they get started writing custom rules to specify what the Maige agent should do based on certain events. For example, the agent can react every time an issue related to a certain topic is created.
Maige has a very lean event-driven architecture. When a repo is added, and then later on when an issue is opened or a comment is added, there are a whole set of actions that have to take place. The set of actions can be customized through natural language.
##### Can you describe the tech stack of your product, as well as your thought process when choosing individual components?
Maige runs in a serverless function and then it spins up the Node.js environment in [E2B](https://e2b.dev/). As a team of TypeScript developers, our frontend choices are Next.js, then Tailwind, and Schadcn.
We use OpenAI embedding endpoints and the latest GPT-4 Turbo because it's currently the best at what we want to do, but we have our eyes on the [Code Llama](https://huggingface.co/codellama) family of models and [Groq](https://groq.com/) hardware for inference. It's very fast.
The vector database we chose is [Weaviate](https://weaviate.io/) hosted on [Railway](https://railway.app/) - which has been working greatly for us.
We have [Planetscale MySQL](https://planetscale.com/) as the database chosen for the other parts. We focused on portability and the ability to self-host which is available to all users, since [Maige is open-source](https://github.com/RubricLab/maige).
Self-hosting shows users that we're not trying to lock them in and they can continue using the infrastructure even if the team behind the product switches gears and stops maintaining the product. I’ve noticed on tech Twitter that not locking people in one type of tech stack is being put an even higher emphasis on recently.
## Execution layer for AI-generated code
##### You mentioned running code in E2B sandboxes. Why do you use it and what exactly is happening in the sandbox?
E2B is officially communicated as a cloud sandbox for AI agents and that's exactly how we use it.
Every repository that integrates with Maige gets its own sandbox for the AI agent to run in. The repository is cloned in the sandbox, the dependencies can be installed there, and the agent can run shell commands the way a junior developer would. It can run tests or try to reproduce an issue or an error or even write code, commit it, and open a pull request. That is, we use E2B as the kind of developer workspace for Maige, and every repo gets its own isolated sandbox instance.
##### What alternatives were you considering before choosing E2B for this purpose?
We probably would have used just [AWS](https://aws.amazon.com/free/?gclid=Cj0KCQjw-r-vBhC-ARIsAGgUO2C3x1L33z14opqMc_OjxMObfdgNfi8PO0ZfqXry6Tzk_sXRxbAU77kaAmB_EALw_wcB&trk=f17b4b4e-aa1b-4189-b0c4-81a19b53f625&sc_channel=ps&ef_id=Cj0KCQjw-r-vBhC-ARIsAGgUO2C3x1L33z14opqMc_OjxMObfdgNfi8PO0ZfqXry6Tzk_sXRxbAU77kaAmB_EALw_wcB:G:s&s_kwcid=AL!4422!3!645186168166!e!!g!!aws!19579892551!148838343321&all-free-tier.sort-by=item.additionalFields.SortRank&all-free-tier.sort-order=asc&awsf.Free%20Tier%20Types=*all&awsf.Free%20Tier%20Categories=*all). We are not infrastructure engineers so that would mean a great challenge to make it scalable and cost-effective in-house. That's also why we chose E2B.
##### How do you ensure the sandboxes are tailored exactly to the needs of the Maige copilot?
We set up the [custom sandbox](https://docs.e2b.dev/template/quickstart) for writing and committing code. We just clone the user's repository, sign in to GitHub as a bot user, open a pull request, and have it attributed to the right bot.
## Leveraging the open-source community
##### You mentioned Maige is open-source. Is this a strategic decision for you? What benefits of open-source do you consider important?
Of course, it’s a personal preference, but I am naturally attracted to open source because I see how impactful such projects are and that they don’t have to rely on one person. That's really important. Any tool that makes scaling and maintaining a project easier is useful.
Another consideration is that I see costs to develop software dropping with AI. Thanks to AI it's harder and harder to build a moat around proprietary software. Going the open-source way, I believe Maige is aligned with the trend of software becoming commoditized.
Finally, we like to work with open-source customers because naturally their use cases are shown in public, which helps form the network around the product. In that way, we can provide a lot of open-source project discounts and free credits.
##### What does the space of GitHub AI agents look like currently?
There are more players in the space, like [Sweep.dev](https://sweep.dev/), [Dosu.dev](https://dosu.dev/), and [Codegen](https://www.codegen.com/) which is heavily funded and focused on large repositories. I see room for all of them if we grow the pie and allow small teams to scale a community around a commercial open-source company, which is a trend we see for more and more products.
## Challenges of building AI agents
##### What are some of the biggest struggles that you have to overcome when building Maige?
There is always the technical challenge of AI agents being non-deterministic and generative AI not being very good at writing code and it's even harder to get it to insert code or modify code. AI agents still can get stuck in loops or give up the task, so the question is how to make them more effective.
##### The security topic is a big part of AI discussions. How important is it for your users and how do you approach it?
To ensure security, we are conservative with major outputs and are confident in features before putting them into production.
Within Maige, we ask for some generic permissions, for example, to open a pull request. Even if that raises concerns for some users, we address those by reminding users of branch protection rules and that the AI agent isn’t allowed to commit code to their production environment without review.
## Future outlook
##### What’s your plan for the upcoming months?
We want to get deeper into front-end development with a vision-based AI and be able to identify accessibility issues or just UI bugs. There is a lot of room for growth there.
We also want to improve the code search feature. Now we embed the full code base that integrates Maige with a vector database and this kind of search still has room for improvement.
##### With the advancements of LLMs and AI techstack overall, how do you think the future AI space development will impact Maige?
According to betting markets, we can expect GPT-5 around October. We are seeing [open-source models getting better](https://rubriclabs.com/blog/designing-for-abundant-intelligence) and I think that even having GPT 3.5 level models that are 10 or 100x cheaper or faster could yield some really interesting results. Maige uses an AI agent so it runs in an open-ended loop and if we could have that loop 100x faster and cheaper, it could be a lot more exploratory and a lot less efficient, but also do more interesting things.
That’s what I'm really excited about.
## Discover more
- [Rubric Labs web](https://rubriclabs.com/)
- [Maige web](https://maige.app/)
- [Ted Spare - X (Twitter) profile](https://twitter.com/TedSpare)
- [Rubric Labs - X (Twitter) profile](https://twitter.com/rubriclabs)
---
Source: https://e2b.dev/resources/customize-sandbox-compute.md
Published: 2024-02-28
# You Can Now Customize CPU and RAM for Your Sandbox
We allowed Pro users to customize the number of vCPUs and RAM size for the sandboxes. The sandbox compute now also has a [dedicated page](https://docs.e2b.dev/sandbox) in our documentation. Customers can now choose any combination of the CPU and RAM based on the following tables. Learn more on the [Pricing](https://e2b.dev/pricing) docs page.
### How to customize your sandbox compute
To customize your sandbox compute, you need to build your custom sandbox template using the [E2B CLI](https://docs.e2b.dev/cli). Run [`e2b template build`](https://docs.e2b.dev/template/build) and specify the `--cpu-count` and `--memory-mb` options.
If you want to try the E2B sandboxes for running your AI app or agent, start [here](https://docs.e2b.dev). Let us know at [hello@e2b.dev](mailto:hello@e2b.dev) if you need any support.
---
Source: https://e2b.dev/resources/up-to-5x-faster-sandboxes.md
Published: 2024-02-26
# Up to 5x Faster Sandboxes
We added support for [Huge Pages](https://www.perplexity.ai/search/what-is-huge-QjvkVL5vQs26enuyxEZj8Q) to our sandboxes.
This new feature makes the start of memory-intensive tasks in sandboxes **up to 5x faster**. For example, reading 0,5 GB of data for the first time after the sandbox starts used to take about **4,9s**. It now takes only **0,76s** thanks to the memory improvements.
Huge pages let you use **2 MiB chunks** of memory instead of **4 KiB chunks**, which are the default. Because there is an overhead each time sandboxes need to ask for more memory, being able to ask for bigger chunks of memory means you can ask fewer times with less total overhead.
Huge pages are enabled by default in all **newly** created custom sandboxes. If you have an existing custom sandbox, you need to rebuild it by calling `e2b build` in the directory with the `e2b.toml` file.
### Getting started
If you want to try the sandboxes, start [here](https://docs.e2b.dev) and let us know at [hello@e2b.dev](mailto:hello@e2b.dev) if you need any support.
---
Source: https://e2b.dev/resources/community-contribution-autogen-code-interpreter-with-e2b.md
Published: 2024-02-19
# AutoGen Code Interpreter with E2B
Last week, our community contributor built an open-source [cookbook example](https://github.com/e2b-dev/e2b-cookbook) of a code interpreter using AutoGen agents.
The LLM-generated code in this example project is executed in the cloud, using [E2B sandbox](https://docs.e2b.dev). By default, AutoGen agents do the code execution locally via Docker which can be [limiting](/resources/limitations-of-running-ai-agents-locally) for some use cases or possess some [risks](/resources/microsoft-s-autogen).
### The E2B Sandbox
The [E2B Sandbox](https://docs.e2b.dev) is a secure way to run your AI app. It is a long-running cloud environment where you can let any LLM (GPTs, Claude, local LLMs, etc) use tools exactly like you would do locally. E2B is fully [open sourced](https://github.com/e2b-dev/e2b) including the infrastructure layer. To try E2B sandboxes for free, start with [documentation](https://docs.e2b.dev).
### Credits
The author of this example is [Keegan McCallum](https://github.com/keeganmccallum), the founder of Xler.ai. Xler is a general-purpose multi-agent platform built on top of AutoGen. It also provides production-grade services like evaluation, deployment, monitoring, and fine-tuning.
### Want to show your example?
Write us at [hello@e2b.dev](mailto:hello@e2b.dev) or make a pull request to the [E2B cookbook](https://github.com/e2b-dev/e2b-cookbook). We have a strong preference for accepting projects such as LLM-powered code interpreters or coding AI agents. The condition is that the app uses E2B sandboxes.
Check out other open-source community examples in the [E2B cookbook.](https://github.com/e2b-dev/e2b-cookbook)
---
Source: https://e2b.dev/resources/crewai-vs-autogen-for-code-execution-ai-agents.md
Published: 2024-02-16
# CrewAI vs AutoGen for Code Execution AI Agents
A new paper [More Agents Is All You Need](https://arxiv.org/abs/2402.05120) finds that, simply via a sampling-and-voting method, the performance of LLMs scales with the number of agents instantiated. This can imply that the popularity of multi-agent frameworks is justified.
CrewAI, also [called AutoGen 2.0](https://www.toolify.ai/ai-news/developing-autonomous-ai-agents-with-crewai-autogen-20-installation-guide-541783), is a recently popular multi-agent framework. I tested CrewAI and compared it to AutoGen, mainly regarding the LLM-generated code execution capabilities.
CrewAI is built on top of LangChain and allows one to orchestrate multiple agents working on a user-defined task. Same as [AutoGen](https://microsoft.github.io/autogen/docs/Use-Cases/agent_chat/), CrewAI is open-source and uses the concept of agents with different roles, but on top of that, CrewAI allows agents to delegate work to each other.
## Why the hype?
There are several explanations for CrewAI's popularity. It is quick to set up, and works well for a variety of interesting use cases with clear guides and demos, e.g.:
- [Stock Analysis](https://github.com/crewAIInc/crewAI-examples/tree/main/crews/stock_analysis)
- [Creating Instagram posts](https://www.youtube.com/watch?v=NY97B2jDCo8&ab_channel=MervinPraison)
- [Trip Planner](https://github.com/crewAIInc/crewAI-examples/tree/main/crews/trip_planner)
- [Landing Page Generator.](https://github.com/crewAIInc/crewAI-examples/tree/main/crews/landing_page_generator)
## Code execution comparison
### AutoGen
What I like about AutoGen is that it is [execution-capable](https://microsoft.github.io/autogen/docs/FAQ/#code-execution) of the code output it produces. That is, when I wanted to analyze and visualize a dataset, AutoGen agents generated a code for it, executed the code via Docker, and saved the resulting chart as a PDF file on my computer.
By default, AutoGen [currently uses](https://microsoft.github.io/autogen/blog/2024/01/23/Code-execution-in-docker/) Docker containers to execute Python code. They even [added a Code Interpreter example](https://microsoft.github.io/autogen/blog/2023/11/13/OAI-assistants/) made with a new (experimental) agent called the GPTAssistantAgent that lets you add the new OpenAI assistants into AutoGen-based workflows.
Executing LLM-generated code locally via Docker may be limiting o for some use cases and possesses some risks, but there exists a cloud alternative. In this [open-source code interpreter example](https://github.com/e2b-dev/e2b-cookbook), the code produced by AutoGen agents is running in an isolated cloud environment.
### CrewAI
When asked for similar data analysis tasks, CrewAI by default generates a text report. It works well with search tools like [LangChain DuckDuckGo Search](https://python.langchain.com/docs/integrations/tools/ddg), but to perform more complex data analysis tasks, it would need tools that allow code execution of the LLM-generated code.
I haven’t found a quick way to add such tools, but it still should be possible to integrate them. In some examples, like [generating a landing page](https://github.com/crewAIInc/crewAI-examples/tree/main/crews/landing_page_generator), CrewAI uses other (custom) tools, like writing a new file with content.
### LangChain tools for code execution
Lang Chain offers several [Tools](https://python.langchain.com/docs/modules/agents/tools/) where LLM-generated code gets automatically executed.
One example is the [Pandas Dataframe](https://python.langchain.com/docs/integrations/toolkits/pandas) where a Python agent is used to execute the LLM-generated Python code.
Another example is [Python REPL](https://python.langchain.com/docs/integrations/tools/python) which can execute Python commands.
There is even one Langchain tool for remote code execution. [Bearly Code Interpreter](https://python.langchain.com/docs/integrations/tools/bearly) allows safe LLM code execution by evaluating Python code in a sandbox environment. This environment resets on every execution.
Apart from these, users can even build their [custom Langchain tools](https://python.langchain.com/docs/modules/agents/tools/custom_tools) for code execution and add them to CrewAI.
In conclusion, LangChain tools are able to execute code snippets for example via the Python runtime environment.
### Limitations
Running LLM-generated code can pose a security risk in general. Either because a user asks the LLM to generate malicious code or the LLM generates malicious code accidentally.
Even the official LangChain tool Pandas Dataframe explicitly mentions “This can be bad if the LLM generated Python code is harmful. Use cautiously.”
LangChain recently received feedback to add a more secure way of running the LLM-generated code, e.g., the same way AutoGen does.
## Conclusion
I can understand the popularity of both AutoGen and CrewAI as they have proven the ability to deliver some interesting and useful examples quickly. While CrewAI is younger than AutoGen, it would be cool to see benchmarks and evals from both frameworks to make it easier for developers to make the right decision when deciding.
I heard from some developers that they chose CrewAI because they were already familiar with LangChain, and others argued that AutoGen is more customizable. However, when discussing with developers, most said that they don’t see a big difference between CrewAI and AutoGen as they accomplish similar tasks.
---
Source: https://e2b.dev/resources/llm-powered-code-interpreters.md
Published: 2024-02-14
# LLM-powered Code Interpreters
When I was talking to developers and community contributors about their projects with E2B, a few of them asked what is meant by "**code interpreter" in the context of AI agents**, and especially, how they differ from other AI coding apps. I summarized my understanding of this term and examples of popular LLM-powered code interpreters.
## Traditional definition
In the context of "traditional" software, a [code interpreter]() is a type of program that reads and executes instructions written in a programming language. The interpreter translates ("interprets") high-level programming language into low-level machine language which the computer can understand and execute.
In the past, these code instructions were usually written by a human programmer. Just recently, we have added AI to the equation. Since the [boom of AI agents in 2023](/resources/the-state-of-ai-agents-reliability-sdks-benchmarking-and-market-trends), developers have been building code interpreters that include an intermediary in the form of an AI agent or assistant translating human prompts written in natural language into the code instructions for computers.
## LLM-powered code interpreter
How do AI code interpreters fit into the world of AI agents? One of the key features of LLM-powered code interpreters is the ability to execute code, as opposed to only generating pieces of code that you as a user subsequently have to copy-paste and execute within your own program. The code execution capabilities are not a necessary part of every AI coding agent or assistant.
The code execution capability implies for example an ability to create a new file on a given path, download files, or provide the result of an operation written as a code (e.g. executing a code that results in generating a chart based on data provided).
## Examples
These are popular examples of LLM-powered code interpreters, as described above:
- [Open Interpreter](https://openinterpreter.com/) - controls your computer via a terminal
- [Autogen](https://microsoft.github.io/autogen/) agents - Autogen is Microsoft's framework for AI agents. Autogen is Microsoft’s framework for AI agents. Autogen’s agents have the ability to execute code, for example, visualize your data in the form of a chart and save it as a file.
- [ChatGPT's Code Interpreter feature](https://openai.com/blog/chatgpt-plugins#code-interpreter) - one of the plugins for ChatGPT has the ability to execute code. (Without the plugin, ChatGPT just generates code and other content.)
In comparison, examples of products that (to this day) do not have code interpreter capabilities:
- LLM-powered chatbots like [ChatGPT](https://chat.openai.com/) (without code interpreter plugin), [Phind](https://www.phind.com/), [Perplexity](https://www.perplexity.ai/)
- [GitHub Copilot](https://github.com/features/copilot) —This coding assistant is able to automate users’ work and produce code in a text format, but not for example changing the content of your directory.
#### Read more
See a [blog post](/resources/limitations-of-running-ai-agents-locally) about the limitations of executing LLM-generated code locally.
---
Source: https://e2b.dev/resources/limitations-of-running-ai-agents-locally.md
Published: 2024-02-09
# Limitations of Running AI Agents Locally
Like many developers, I have been recently building my own coding AI agent.
I was inspired by a lot of popular code interpreters and AI coding frameworks I have seen lately, like [Open Interpreter](https://openinterpreter.com/), [Autogen](https://github.com/microsoft/autogen), or [ChatGPT Code Interpreter plugin](https://openai.com/blog/chatgpt-plugins#code-interpreter).
The coding agents' value for daily usage has increased after they have **evolved from just chatting, to doing the actual work**. Developers are equipping their agents with the ability to **execute LLM-generated code output**.
Imagine you are using an AI agent that can analyze and visualize given data, but instead of just providing you a code that you can copy-paste to do the task, the agent conducts the next step and also runs the code to produce a chart and saves it to your local filesystem.
For example, when I used AutoGen to analyze stock data. The AI agents generated the required code, executed it, and saved the resulting data analysis chart in a PDF format on my computer:
When a human user chooses to execute the code, the output opens in a new window like this:
The code execution is often done locally via containers, e.g., Docker, like in the case of Autogen. Many developers that I talked to started building their own in-house solutions for running AI code.
Running the AI output on the user's computer is an achievable, but risky way. Here are a few obvious and less obvious issues that I had, and noticed other AI developers mentioning:
## 1. Security + isolation
Allowing AI tools to autonomously run untrusted LLM-generated operations on users' computers can be problematic.
Take for example Docker containers, that (according to [their docs](https://docs.docker.com/engine/security/)) possess a risk of providing incomplete isolation, either independently, or when used in combination with kernel vulnerabilities.
When using containers, ensuring isolation is very difficult even when adding barriers.
## 2. User-centric approach
The approach to building AI agents has begun as developer-centric, which is understandable for any emerging field or technology.
This can result in end users facing difficulties with all the AI products that were built using Docker or another type of local solution. The majority of AI assistants with nice user experience I came across were browser apps - people often ask how to filter AI tools based on whether they provide nice UI. It's worth noting that apps don't necessarily have to be used solely on desktop and some provide better functionalities on a mobile version.
Non-technical users may struggle with installing an app locally, let alone controlling it via a terminal. In such a setting, it's also challenging to allow end users to collaborate, and share their files with their app in a simple way or share outputs within a team. Within a browser UI, users can share resources and templates so they don't have to build the same thing from scratch.
## 3. Scalability
Another challenge that is particularly important for agent-building frameworks is deploying AI agent instances so they can scale. An AI platform or service that accommodates thousands of users, each developing its own AI applications, will require thousands of containers, as each agent instance needs to run in its own isolated environment.
## 4. Session length
A lot of developers want their end users to be able to return to their work after some time or even after closing and re-opening the app. Developers need to ensure long-running sessions for their AI products.
In conclusion, end users apply their UX and security standards from using "traditional" software to AI apps.
Issues like security and data privacy have been discussed even more in relation to AI.
Within the new AI tech stack that is being formed, developers need to think of a solution that would be tailored for building AI products.
#### E2B - Remote execution of AI-generated output
[E2B](https://github.com/e2b-dev/E2B) (5.7K+ ⭐️) provides the cloud runtime for AI agents and apps.
The E2B Sandbox is a secure way to run your AI app. It is a long-running cloud environment where you can let any LLM (GPTs, Claude, local LLMs, etc) use tools as you would do locally.
Check out the quick start in the docs to start with E2B for free: [https://docs.e2b.dev](https://docs.e2b.dev).
---
Source: https://e2b.dev/resources/ai-agents-in-2024.md
Published: 2024-01-02
# AI Agents in 2024
We are E2B. Try our [cloud runtime for AI agents](https://docs.e2b.dev?ref=framer-ai-agents-in-2024). Check out our [GitHub](https://github.com/e2b-dev/e2b?ref=framer-ai-agents-in-2024), and support us with a star if you like what we’re building. ✴️
What will the new year bring to the AI field? We asked developers and AI startup founders what they think will happen with AI products in 2024.
Big thanks for the insights go to:
- [Ismail Pelaseyed](https://twitter.com/pelaseyed?lang=cs), founder of [Superagent](https://www.superagent.sh/)
- [Ted Spare](https://tedspare.com/), co-founder of [Rubric Labs](https://rubriclabs.com/)
- [Jonathan Harel](https://twitter.com/HaJongler), co-founder of Fine
- [Kaveen Kumarasinghe](https://kaveenk.com/), student at University of Waterloo, contractor at [OpenAI](https://openai.com/)
- [John Xie](https://twitter.com/johnxie?lang=cs), co-founder of [Taskade](https://www.taskade.com/)
- [Rahul Behal](https://www.rahulbehal.com/), co-founder of [AgentHub](https://www.agenthub.dev/)
- [Max Musing](https://twitter.com/MaxMusing), founder of [Basedash](https://www.basedash.com/)
- [Brendon Geils](https://twitter.com/BrendonGeils), founder of [Athena Intelligence](https://athenaintel.com/)
## 1. What do you predict for AI agents/AI products in 2024?
It's positive to see that developers perceive 2024 as the year of AI agents. See what are their predictions for the new year.
**Ismail, Superagent**:
We will see a lot of LLM extensions/adapters. A concept similar to database extensions. These extensions would augment the abilities of the LLM without using prompt techniques. Examples of this would be extensions for structured outputs (function calling) similar to [outlines-dev](https://github.com/outlines-dev/outlines) or [JSONformer](https://python.langchain.com/docs/integrations/llms/jsonformer_experimental) packages on GitHub. Future examples would be adapters for memorization of new content (an alternative to RAG).
**Ted, Rubric Labs**: AI agents will show results in 2024. Barriers to entry will continue to drop through low-code tools and education.
**Jonathan, Fine**: Product will become the main moat for AI startups.
**Kaveen, Waterloo & OpenAI**: There will be a significant focus on personalization and context-awareness, allowing AI to provide more tailored and relevant interactions within environments that we use daily. I think we’ll also see more drivable and coherent collaborative AI, where agents work in tandem with humans to accomplish tasks.
**John, Taskade**: AI agents will become even more integrated into our daily workflows, especially in project management and collaborative spaces. They’ll not only automate tasks but also provide deeper insights and predictions, helping us make smarter decisions faster.
**Rahul, AgentHub**: Browsing agents that interface with web browsers at a high level will be able to robustly and reliably autonomously perform at the same level as humans or better for 90% of the things we spend our time with browsers doing. Tasks like shopping on Amazon, browsing/posting to social media, reading/sending emails, etc.
**Max, Basedash**: Agents will be functional with minimal prompt engineering, and remarkable with sophisticated prompt engineering.
**Brendon, Athena**: My prediction for AI agents (or AI products in general) is that the way we guide Agents will not come primarily from chat inputs.
## 2. What was your favorite (or most used) AI product in 2023?
The last year was a boom for AI-powered tools and a lot of new startups building AI agents. See some developers' top picks of 2023.
**Ismail, Superagent**: [**Perplexity AI**](https://www.perplexity.ai/).
**Ted, Rubric Labs**: [Perplexity](https://www.perplexity.ai/). Real-time search results make LLM chat much more useful since much of what I’m looking for is \<2-year-old docs.
**Jonathan, Fine**:[Ollie.ai](https://ollie.ai/) was pretty neat, found out a few cool things I can gift friends and myself!
**Kaveen, Waterloo & OpenAI**: My most used AI product in 2023 was the [OpenAI API](https://openai.com/blog/openai-api), specifically GPT-4 and, more recently, the [GPT-4 vision](https://platform.openai.com/docs/guides/vision) preview!
**John, Taskade**: It’s hard to pick just one given 100% biased towards [Taskade AI](https://www.taskade.com/), but I’m particularly impressed with the advancements in AI Workflow Generators and Custom AI Agents by Taskade. These tools have significantly streamlined task management and creative processes for me.
**Rahul, AgentHub**:[Cursor](https://cursor.sh/). It just completely changed and enhanced my development workflow and experience. I had several issues with both Co-Pilot and only using ChatGPT, Cursor solves most of them. It’s not perfect, but the product is getting better every day.
**Max, Basedash**:[Raycast AI](https://www.raycast.com/) proved to me that speed is often more important than capabilities.
**Brendon, Athena**: I’ll give this to our favorite framework, [Langchain](https://www.langchain.com/). Their team’s ability to ship days if not hours after AI news hit made it our first stop for AI integration.
To get inspired, check out the [landscape](https://github.com/e2b-dev/awesome-ai-agents) of AI products and agents, sorted by their use cases.
## 3. What is your AI-related resolution for 2024?
Developers shared a broad spectrum of goals for the upcoming year, from building a product that complements human work to replacing one real job with AI.
**Ismail, Superagent**: Keep shipping.
**Ted, Rubric Labs**: To spend more time listening to worries around AI from family and friends. Disruption can be scary. If we're about to see AI generate the greatest wealth in history, I think it can only be sustained if there's widespread buy-in from regular people.
**Jonathan, Fine**: I hope to reach the end of 2024 with an AI that helps me manage my own life.
**Kaveen, Waterloo & OpenAI**: To stay ahead of the latest developments in generative AI. This also entails continuing to build and maintain a strong network of professionals to draw information and inspiration from.
**John, Taskade**: We want to ensure that we keep building a tool that enhances human creativity and efficiency, rather than replacing it.
**Rahul, AgentHub**: Strive to be more platform-agnostic, experimenting with different types of models and services including OSS. And building in a modular way so models can be swapped in and out easily.
**Max, Basedash**: Render one job title obsolete with agents.
**Brendon, Athena**: Talk more politely to any AI…. just in case.
What is your view on the future of AI agents? Join the discussion on our [Discord server](https://discord.com/invite/U7KEcGErtQ).
---
Source: https://e2b.dev/resources/will-openai-s-gpts-kill-ai-agents.md
Published: 2023-11-15
# Will OpenAI’s GPTs Kill AI Agents?
We are E2B. We provide sandboxed cloud environments made for AI agents and AI apps. Try our Custom Sandboxes. Check out our GitHub, and support us with a star if you like it.
OpenAI introduced GPTs - separate instances of ChatGPT that everyone can configure for their use cases with different instructions, knowledge, and actions.
Does it mean that all the agent-ish products and open-source projects built by developers for months are now dead? Here are some differences between building GPTs and "independent" AI agents.
### Experimental nature
People began to build GPTs just a few days ago. There are similarities to what we have seen in the first hype around AI agents. Everyone is now experimenting with GPTs and they certainly have a long way to take. We can foresee that GPTs are yet to find their most viable use cases and potentially turn into something more serious.
OpenAI often emphasizes the community around GPTs, they are great for sharing on social media.
### Techstack limitations
GPTs are tied to OpenAI's ecosystem, implying some limitations in a techstack overall.
- GPTs are not open-source, hence you cannot build on top of existing GPT
- GPTs are limited in the choice of LLM - using just GPT.
- The current version of GPTs has limitations, e.g. being able to upload a maximum of 10 files.
Using that action button you saw above, GPTs can be easily integrated into with other systems, such as your email, a travel site, or corporate payment software. You can start to see the birth of true agents as a result.
### Builders' skills
Primary people building "independent" AI agents are developers, while GPTs attract no-code people and once again make AI accessible to a broader public. A lot of GPTs were built with just writing a few prompts in plain English. We have seen this with agent-building frameworks like Superagent where non-technical people can define their agent within existing UI.
The developer-centric counterpart of GPTs is the OpenAI Assistant API.
### Tasks and capabilities
"Independent" agents are turning into various types of personal assistants. We have **MultiOn** that can order a meal or buy a plane ticket for you. We have **Fine.dev** that provides virtual SW developer teammates, **Cursor**, or **Aide** assisting you with coding or **Open Interpreter** that operates via user's terminal.
Even though they have diverse purposes, from a personal trainer to a startup co-founder, most GPTs are still **chatbots**. I see parallels with Character AI where everyone has their custom character for fun. Another strong use-case is generating other things than text, e.g. DALL-E for images.
Of course, there are exceptions, e.g. the Zapier GPTs that can be granted to access your calendar or Slack.
### Technical and security challenges
Agents are often criticized for being unreliable and not ready for enterprise-level adoption. The same problems are faced by GPTs. Hallucinations, providing different results on the same prompt, and not really understanding the underlying processes are challenges typical for any product of a stochastic nature. These can be partly overcome by more advanced models or by building a product around an agent that would compensate for the lack of reliability.
"When builders customize their own GPT with actions or knowledge, the builder can choose if user chats with that GPT can be used to improve and train our models."
### Revenue paths
So far, GPTs lack the characteristics of concrete products. Even though there have been discussions about GPTs killing agent-building startups, we now lack a way to make a business out of GPTs. The GPTs are exclusive to users with ChatGPT Premium, and there are no options for pricing strategy or differentiation of product tiers.
### Different use-cases
While "independent" agents see main use cases in coding, debugging, and related tasks, GPTs offer use cases across fields. Many of them are useful for marketing or lifestyle purposes.
You can see this on the visual landscapes of popular OpenAI GPTs vs popular agents (both open and closed source) below.
### Are GPTs even agents?
Here we go again with the ambiguously defined "AI agents". I have seen GPTs being called "Almost Agents". In his recent post, Bill Gates made a few points about what differentiates agents from just bots (like Clippy):
- Proactive in suggesting solutions to user's needs
- Able to accomplish tasks across applications
- Improving over time.
OpenAI made the term "agents" great again, even though most GPTs still lack the degree of autonomy we expect from agents, e.g. using "tools" outside of their chat interface.
### Will GPTs replace AI Agents?
In my opinion, OpenAI responded to a trend and quickly brought GPTs to the market. While they might be useful for introducing agents to the general public, they are still in an early stage, even though the idea is promising and will probably be further developed in the upcoming months.
GPTs are really beneficial for the daily users of ChatGPT - allowing them to utilize pre-made GPTs instead of iterating on a perfect prompt.
OpenAI hinted that the current state of GPTs is just the beginning. They are soon launching a GPT app store, offering creators the benefit of promotion. However, the creators will face very high competition due to the low entry barriers of GPT building.
However, in the future, many GPTs will still require developing more features and functionalities outside of OpenAI’s UI, the same way developers have been building more complex products around AI agents.
---
Source: https://e2b.dev/resources/e2b-sandbox.md
Published: 2023-11-07
# Code Interpreter Sandbox
We are [E2B.](https://e2b.dev/?ref=ai-agents-vs-developers) We provide sandboxed cloud environments for AI-powered apps and agentic workflows.
Try our [Sandbox Runtime for LLMs](https://docs.e2b.dev?ref=october-newsletter)[.](https://github.com/e2b-dev)
[We are open-source](https://github.com/e2b-dev), so please check out our [GitHub](https://github.com/e2b-dev/e2b), and support us with a star. ✴️
### E2B Sandbox
LLM Sandboxes are **cloud environments, general-purpose** machines powered by Ubuntu. Sandboxes are an ideal fit for AI assistants like coding copilots, code interpreters, AI data analysts, AI browser assistants, and other AI-powered apps. Read more about it [here](https://docs.e2b.dev/sandbox).
### E2B Custom Sandboxes
While OpenAI announced major updates, E2B silently launched Custom Sandboxes today.
Follow [our guide](https://docs.e2b.dev/template/quickstart) on how to create your own Custom Sandbox.
You can create your own Custom Sandboxes for different purposes, from data analysis through AI internet browsing to very popular code execution. You can use Template Files for building the Custom Sandboxes.
**The Code Interpreter Sandbox is just one of the Custom Sandboxes**, and it may seem similar to what was released by OpenAI at the DevDay conference.
#### How is the OpenAI Code Interpreter different from the E2B Code Interpreter Sandbox?
This is the question we have been asked a lot. The differences include:
1. The [E2B Code Interpreter Sandbox](https://docs.e2b.dev/sandbox) is just a sandbox - without any LLM "connected" to it. Our Sandbox can be **controlled with SDK** (`run_code`, `install_pkg`, `create_file`, etc) and gives you the freedom to **connect it to (any) LLM**.
On the other hand, you **control the OpenAI Code Interpreter by talking to an AI assistant**.
1. While the OpenAI CI API is a good fit if you want something working right out of the box, E2B Sandbox comes in handy if you want **more granular control** over what's and when is happening. We give you [**complete environment customization**](https://docs.e2b.dev/template/quickstart).
2. E2B is (partly) [open-source](https://github.com/e2b-dev) and we are going full open-source soon.
---
Source: https://e2b.dev/resources/openai-devday.md
Published: 2023-11-07
# OpenAI DevDay
This is a view by the E2B team, so the thoughts and comments are based on our experience with the AI space. [E2B](https://e2b.dev/) provides sandboxed cloud environments for AI-powered apps and agentic workflows.
Check out our [sandbox runtime for LLMs](https://docs.e2b.dev?ref=october-newsletter)[.](https://github.com/e2b-dev)[](https://github.com/e2b-dev)
[We are open-source](https://github.com/e2b-dev), so please check out our [GitHub](https://github.com/e2b-dev/e2b), and support us with a star. ✴️
The highly anticipated OpenAI DevDay is likely to be remembered as the biggest AI event in 2023.
For weeks, rumors have been circulating, predicting that the way we use ChatGPT is about to change completely, and the announcements will kill many AI startups. After watching the [DevDay Opening Keynote](https://www.youtube.com/watch?v=U9mJuUkhUzk) together with our community, we are discussing the [major announcements](https://openai.com/blog/new-models-and-developer-products-announced-at-devday).
## The announcements
Sam Altman shared on the stage three major updates:
1. **GPT-4 Turbo launch** - with longer context, more control, better knowledge, new modalities, customization, and higher rate limits
2. **GPTs** - Customized versions of ChatGPT
3. **Assistants API** - a playground for building AI assistants (the GPTs)
These are the three big news in detail:
### 1. GPT-4 Turbo
When we talked to medium/enterprise companies this year, many of them have AI projects just waiting for a lower price and better latency to launch. The new GPT-4 Turbo model is addressing this, among other things.
Starting now, it is much cheaper than GPT-4, in particular:
- 3 times less for input tokens.
- 2 times less for output tokens.
This is indeed great news for developers of AI agents and apps.
The six major updates in GPT 4 Turbo are:
#### 01. Context length
While GPT-4 supports up to 8k (in some cases up to 32k) context length, GPT-4 Turbo offers 128k context length. That is approximately 300 pages of a standard book, so it could remember what happened to Tolkien’s [hobbits](https://www.amazon.com/Hobbit-J-R-Tolkien/dp/054792822X) throughout the book.
Sam mentioned that Turbo also is **more accurate**.
#### 02. More control
Reliability has been a huge problem for AI developers, given the unpredictable outputs of LLM models.
Addressing feedback from developers, there will be more control over the model's responses and outputs. This could partly compensate for the LLMs' stochasticity.
First, OpenAI announces a “JSON load” feature which ensures that the model will respond with valid JSON. That will make the API calls much easier.
Second, the GPT-4 Turbo is significantly better at function calling. You can now call many functions at once.
The model will do better at following instructions in general.
Finally, Altman announced a beta of [reproducible outputs](https://platform.openai.com/docs/guides/text-generation) as a new feature. That allows to pass the seed parameter to the model, which will make it return consistent outputs. That provides the user a higher degree of control over model behavior.
#### 03. Updated knowledge
The knowledge cutoff has been extended to April 2023 and will continue to see improvements.
Additionally, OpenAI is surfing on the current hype, adding built-in RAG. RAG [sparked a lot of interest](https://www.ai.engineer/summit/) among developers and became a major topic at conferences. OpenAI is now introducing a retrieval feature in its platform. This allows users to incorporate information from external documents or databases into their projects.
#### 04. New modalities
Surprising no one, DALL·E 3, GPT-4 Turbo with Vision, and the new text-to-speech (TTS) model are all going to into the OpenAI API now.
Developers can [integrate DALL·E 3](https://platform.openai.com/docs/guides/images) to ChatGPT Plus and Enterprise users, directly into their apps and products through our Images API by specifying dall-e-3 as the model.
#### 05. Customization
Fine-tuning has proven to be highly effective for GPT-3.5 since its launch a few months ago. Starting immediately, OpenAI is extending this approach to the 16k version of the model.
OpenAI is inviting active fine-tuning users to [apply](https://openai.com/form/custom-models) for an experimental GPT-4 fine-tuning [Custom Models program](https://openai.com/form/custom-models).
It should allow close collaboration between the researchers and companies to create highly customized models for specific use cases. This includes modifying all aspects of the model training process, including domain-specific pre-training and post-training tailored to a particular domain.
#### 06. Higher rate limits
OpenAI is doubling tokens per minute for GPT-4 customers and allowing rate limit changes in API settings.
They're introducing Copyright Shield to cover legal costs for copyright claims in ChatGPT Enterprise and the API, emphasizing they don't train models using API or ChatGPT Enterprise data.
“And let me be clear,” adds Sam Altman. “This is a good time to remind people, that we do not train on data from the API or ChatGPT Enterprise ever.”
### 2. GPTs - Customized versions of ChatGPT
The OpenAI DevDay was the (first ever) conference for developers. However, the launch with the biggest hype, GPTs, is consumer-facing.
You may recall that OpenAI indicated in February that it intended to allow users to [define their own customizable AI agents](https://synthedia.substack.com/p/openai-to-offer-chatgpt-customization). The rumors prior to the DevDay were true - here come the OpenAI customizable GPTs.
You can now create custom versions of ChatGPT **without knowing how to code**. GPTs combine instructions, extra knowledge, and any combination of skills.
Example GPTs available for ChatGPT Plus and Enterprise users, with more users to follow.
OpenAI decided to include the AI community that is shaping the future, proven for example by the fact that “[ChatGPT Is More Famous, but Character.AI Wins on Engagement](https://www.similarweb.com/blog/insights/ai-news/character-ai-engagement/)”. (Users of Character.AI - a chatbot interface where you can customize your chat avatars - allegedly spend an average of two hours per day on the site.)
GPT Store will be launched soon, featuring creations by verified builders.
#### Why “GPTs”?
OpenAI avoided the term “AI agent” and used "[GPTs](https://openai.com/blog/introducing-gpts)", even though they follow the characteristics of agents.
It may be due to better connecting “GPT” with already publicly accepted “ChatGPT”. “GPTs” may be more relatable to the broader public. The emphasis is put on using natural language to program.
Recall the definition of [LLM Powered Autonomous Agents](https://lilianweng.github.io/posts/2023-06-23-agent/) by Lilian Weng from OpenAI where agents were specified by
- Long-term memory
- Planning
- Tool use.
I see some parallels with GPTs, which have:
- Expanded Knowledge
- Custom Instructions
- Actions.
### 3. Assistants API
The [assistants API](https://platform.openai.com/docs/assistants/overview) is basically a **developer-facing part of the GPTs**. It works as self-coding API-level agents.
It simplifies the process for developers to create their own AI-powered assistants with well-defined objectives and the ability to call models and tools. You **no longer need to include all previous messages** for context when sending a new one to the API.
OpenAI also shipped a new [**playground**](https://platform.openai.com/playground?mode=assistant) to build the assistants. What is interesting to me is how OpenAI just targets both no-code "developers" and traditional developers with the Assistants API. I was expecting something more d
The Assistants API includes:
- Better function calling
- Built-in conversation management
- Python sandbox
- Memory
It offers using two tools so far:
- Retrieval
- Code interpreter (Which was called "Advanced Data Analyst" until recently).
We can probably expect more tools to be added soon, and there is already an option to add your custom tool. **The tools are essentially just OpenAI Functions**.
[**How is the OpenAI Code Interpreter different from E2B Sandboxes**](/resources/e2b-sandbox)?
### What are the implications?
OpenAI is shifting from focusing solely on AGI to prioritizing the commoditization of software development and building a platform. This will have a huge impact on coding and prototyping.
Companies react quickly to the updates. Langchain, the most popular framework for building AI agents, quickly announced OpenGPTs as an alternative to OpenAI’s GPTs.
The announcements certainly put many companies in danger, e.g. vector database startups jeopardized by the Assistants API's retrieval.
Hopefully, most startups and companies won't go out of business. Instead, we'll see even more AI companies building and adding completely new AI features to their products.
For the AI developer community, 2024 will certainly be the production year, and we are excited to see what’s coming.
### About OpenAI DevDay
OpenAI’s [first developer conference](https://openai.com/blog/announcing-openai-devday) took place in San Francisco, CA, on November 6, 202.
The goal was to bring hundreds of developers from around the world together with the team at OpenAI to preview new tools and exchange ideas
---
Source: https://e2b.dev/resources/e2b-october-update.md
Published: 2023-10-31
# October Update
Enjoy our monthly recap of new AI products, guides, articles, podcasts, and E2B updates!
We are [E2B.](https://e2b.dev/?ref=ai-agents-vs-developers) We provide sandboxed cloud environments for AI-powered apps and agentic workflows. Check out our [sandbox runtime for LLMs](https://docs.e2b.dev?ref=october-newsletter)[.](https://github.com/e2b-dev)
[We are open-source](https://github.com/e2b-dev), so please check out our [GitHub](https://github.com/e2b-dev/e2b), and support us with a star. ✴️
## 🛠️ Product
### LangChain integration
This month, we announced the official [E2B integration into LangChain](https://python.langchain.com/docs/integrations/tools/e2b_data_analysis). The E2B [LLM Sandbox](https://docs.e2b.dev?ref=changelog) is ideal for building your own **AI Data Analysts** and Code Interpreters.
A big advantage of the E2B Sandbox is a safe code execution - allowing for example installing packages dynamically, generating charts, or file upload & download.
See the final code in the [official LangChain documentation](https://python.langchain.com/docs/integrations/tools/e2b_data_analysis?ref=build-ai-data-analyst-with-langchain-and-e2b).
### Guides and examples
We shared a list of tutorials and E2B use cases on our [Discord](https://discord.com/invite/U7KEcGErtQ). Check it out in the #guides channel or submit your own use-case if you want to be featured.
Read our [newest guide](/resources/build-ai-data-analyst-with-langchain-and-e2b) where we create an example of a LangChain agent that uses E2B cloud sandbox and GPT-4 to analyze your uploaded data.
### Superagent integration
E2B is now integrated into [Superagent](https://www.superagent.sh/), meaning there is added support for a Code Execution feature. You can try it here.
This integration allows the AI agents to actually [run code in a sandboxed environment](https://docs.e2b.dev), which helps for example with creating charts, doing requests, or analyzing files. To try Superagent with the E2B feature, create a Code interpreter API and then select it for your agent to use.
Join the [Superagent Discord](https://discord.com/invite/mhmJUTjW4b) to stay updated and discuss your creations.
## 📚 Knowledge
### New on E2B blog
- 📜 [AI data analyst in cloud sandbox](/resources/build-ai-data-analyst-with-langchain-and-e2b) - A guide to building with LangChain and E2B
- 📜 [Microsoft's AutoGen](/resources/microsoft-s-autogen) - A guide to code-executing agents
- 🤖 [AI Agents vs Developers](/resources/ai-agents-vs-developers) - About the challenges of memory, security, testing, and data privacy of LLM products
If you like our content, please support us on [Medium](https://medium.com/e-two-b).
### Team's picks
- 🎧 [Bundling/Unbundling AI](https://another-podcast.simplecast.com/episodes/bundling-unbundling-ai) (podcast)
- 💬 [Announcing the general availability of the world’s first commercial cloud computer](https://oxide.computer/blog/the-cloud-computer) (article)
- 🎧 [NVIDIA CEO Jensen Huang](https://podcasts.apple.com/gb/podcast/nvidia-ceo-jensen-huang/id1050462261?i=1000631419166) (podcast)
- 🎧 [Why AI Agents Don't Work (yet) - with Kanjun Qiu of Imbue](https://www.latent.space/p/imbue?utm_campaign=email-post&r=2qp5vy&utm_source=substack&utm_medium=email#details) (podcast)
- 💬 [OpenAI plans major updates to lure developers with lower costs, sources say](https://www.reuters.com/technology/openai-plans-major-updates-lure-developers-with-lower-costs-sources-2023-10-11/) (article)
### New LLM products, [agents](https://github.com/e2b-dev/awesome-ai-agents), and [AI tools](https://github.com/e2b-dev/awesome-sdks-for-ai-agents) to check out
- 🤖 Adepts' [Fuyu-8B:](https://www.adept.ai/blog/fuyu-8b) A Multimodal Architecture for AI Agents
- 🤖 AgentLabs - A platform for interoperability & AI agent management
- 🤖 [BondAI](https://github.com/krohling/bondai) - An AI Agent with an easy-to-use CLI, RESTful/WebSocket API, Pre-built Docker image, and a lot of integrated tools
- 🤖 [MemGPT](https://memgpt.ai/) - Teaching LLMs memory management for unbounded context
- 🤖 Wispy - A web-browsing AI assistant that can summarize content, compose content, explain things, or create quizzes for topics you are learning
Are you building AI agents, frameworks, or tooling for AI apps that are not featured in our database of [agents](https://github.com/e2b-dev/awesome-ai-agents) or [SDKs](https://github.com/e2b-dev/awesome-sdks-for-ai-agents)? Tell us about your product [here](https://docs.google.com/forms/d/e/1FAIpQLScndOs3bQ8aBtqqqhvlC_qgJmg16jmwOfEoLter6I921vfmvQ/viewform).
We are happy and proud to share with you that the E2B SDK has been downloaded [over 3,700 times on NPM](https://npmcharts.com/compare/@e2b/sdk?interval=30&log=false) and [over 22,000 times on PyPI](https://www.pepy.tech/projects/e2b?versions=0.*) since it's been out! Thank you for your support!
If you have tried E2B or had trouble running it, we will be grateful if you [share your feedback with us](https://docs.google.com/forms/d/e/1FAIpQLSdBfhmXqDmB7ye6qB7SMUBHgD61jPcFkhb1-RWtNdzctrkpXQ/viewform). It helps us to prioritize the right features.
- 💬 If you have any **questions**, just reply to this email or tag @**e2b-team** on our [Discord](https://discord.com/invite/U7KEcGErtQ)
- ⭐️ If you like our work, consider giving us a star on [GitHub](https://github.com/e2b-dev)
- 👉 Follow us on [X (Twitter)](https://twitter.com/e2b_dev) and [LinkedIn](https://www.linkedin.com/company/e2b-dev/)
---
Source: https://e2b.dev/resources/build-ai-data-analyst-with-langchain-and-e2b.md
Published: 2023-10-25
# AI Data Analyst in Cloud Sandbox with LangChain & E2B
We are E2B. We provide sandboxed cloud environments for AI-powered apps and agentic workflows. Check out our sandbox runtime for LLMs.
In this guide, we will create an example of a LangChain agent that uses E2B cloud sandbox and GPT-4 to analyze your uploaded data.
**See the final guide and code in the official LangChain documentation here**.
## Why build your own agent with E2B?
E2B's cloud environments are runtime sandboxes for LLMs. They are an ideal fit for building AI assistants like code interpreters or advanced data-analyzing tools. We can use E2B's Data Analysis Sandbox for our use case.
Compared to assistants running their code locally, e.g. via Docker, the Data Analysis Sandbox allows for safe code execution in a remote environment.
That is a secure way to run the unpredictable LLM-generated code on your computer without the potential harm that such code can cause to your machine, e.g., unauthorized access to vulnerable data.
We will create an assistant that will use OpenAI’s GPT-4 and E2B's Data Analysis sandbox to perform analysis on uploaded files using Python.
Let's get to hacking!
### 1. Get API keys, import packages
First, we have to ensure that we have the latest version of E2B.
```bash
pip install -U e2b langchain
```
We import `E2BDataAnalysisTool` and other necessary modules from LangChain. We get our OpenAI API key [here](https://platform.openai.com/account/api-keys), and our E2B API key [here](https://docs.e2b.dev/api-key) and set them as environment variables.
🔎 Find the full OpenAI API documentation [here](https://openai.com/blog/openai-api).
```python
import os
from langchain.chat_models import ChatOpenAI
from langchain.tools import E2BDataAnalysisTool
from langchain.agents import initialize_agent, AgentType
os.environ["E2B_API_KEY"] = ""
os.environ["OPENAI_API_KEY"] = ""
```
### 2. Initialize the E2B tool for the LangChain agent
💡When creating an instance of the `E2BDataAnalysisTool`, you can pass callbacks to listen to the output of the sandbox. This is useful, for example, when creating a more responsive UI. Especially with the combination of streaming output from LLMs. We define a Python function save\_artifact, which is used to handle and save charts created by Matplotlib. When a chart is generated with `plt.show()`, this function is called to print a message about the newly generated Matplotlib chart, downloads it as bytes, and then saves it to a directory named "charts."
```python
# Artifacts are charts created by matplotlib when `plt.show()` is called
def save_artifact(artifact):
print("New matplotlib chart generated:", artifact.name)
# Download the artifact as `bytes` and leave it up to the user to display them (on frontend, for example)
file = artifact.download()
basename = os.path.basename(artifact.name)
# Save the chart to the `charts` directory
# Make sure the "charts" directory exists
with open(f"./charts/{basename}", "wb") as f:
f.write(file)
e2b_data_analysis_tool = E2BDataAnalysisTool(
on_stdout=lambda stdout: print("stdout:", stdout),
on_stderr=lambda stderr: print("stderr:", stderr),
on_artifact=save_artifact,
)
```
### 3. Upload your data file
You can choose your own CSV data file to upload to the E2B sandbox. In our example, we chose a file about Netflix TV shows. You can download the file [here](https://storage.googleapis.com/e2b-examples/netflix.csv).
The following code reads a file named "netflix.csv" from the local file system. It then uses the `e2b_data_analysis_tool.upload_file` method to upload the contents of this file to the sandbox and print the path where the file is saved in the sandbox.
```python
with open("./netflix.csv") as f:
remote_path = e2b_data_analysis_tool.upload_file(
file=f,
description="Data about Netflix tv shows including their title, category, director, release date, casting, age rating, etc.",
)
print(remote_path)
```
##### ▶**️ Code output**
```text
name='netflix.csv'
remote_path='/home/user/netflix.csv'
description='Data about Netflix tv shows including their title, category, director, release date, casting, age rating, etc.'
```
### 4. Create tools and initialize the agent
Now we get to set up the LangChain agent. It will be using GPT-4 and the `e2b_data_analysis_tool` we created earlier.
```python
tools = [e2b_data_analysis_tool.as_tool()]
llm = ChatOpenAI(model="gpt-4", temperature=0)
agent = initialize_agent(
tools, llm, agent=AgentType.OPENAI_FUNCTIONS, verbose=True, handle_parsing_errors=True
)
```
### 5. Execute a query
We initiate the execution of the agent with a specific query or task.
```python
agent.run(
"What are the 5 longest movies on netflix released between 2000 and 2010? Create a chart with their lengths."
)
```
##### ▶**️ Code output**
### 6. Sandbox advanced features
E2B also allows you to install both Python and system (via `apt`) packages dynamically during runtime like this:
```python
# Install Python package
e2b_data_analysis_tool.install_python_packages('pandas')
```
##### ▶**️ Code output**
Additionally, you can download any file from the sandbox like this:
```python
# The path is a remote path in the sandbox
files_in_bytes = e2b_data_analysis_tool.download_file('/home/user/netflix.csv')
```
Lastly, you can run any shell command inside the sandbox via `run_command`.
```python
# Install SQLite
e2b_data_analysis_tool.run_command("sudo apt update")
e2b_data_analysis_tool.install_system_packages("sqlite3")
# Check the SQLite version
output = e2b_data_analysis_tool.run_command("sqlite3 --version")
print("version: ", output["stdout"])
print("error: ", output["stderr"])
print("exit code: ", output["exit_code"])
```
##### ▶**️ Code output**
### 7. Close the sandbox
When your agent is finished, don't forget to close the sandbox.
```python
e2b_data_analysis_tool.close()
```
## Output
If you try this example with our [Netflix file](https://storage.googleapis.com/e2b-examples/netflix.csv), the output should be automatically saved into your local directory like this:
**See the final guide and code in the official LangChain documentation** [**here**](https://python.langchain.com/docs/integrations/tools/e2b_data_analysis?ref=build-ai-data-analyst-with-langchain-and-e2b). **See** [**E2B docs**](https://docs.e2b.dev) **here**.
- Need help or want to share feedback? Join our [Discord](https://discord.com/invite/U7KEcGErtQ).
- If you like the guide, please support us with a star on [GitHub](https://github.com/e2b-dev/e2b).
- Follow us on [X (Twitter)](https://twitter.com/e2b_dev).
- You can also reach us at [hello@e2b.dev](mailto:hello@e2b.dev).
---
Source: https://e2b.dev/resources/ai-agents-vs-developers.md
Published: 2023-10-20
# AI Agents vs. Developers
We are E2B. We provide sandboxed cloud environments for AI-powered apps and agentic workflows. Check out our sandbox runtime for LLMs.
The traditional human-computer interaction now often involves an intermediary in the form of an AI assistant, which is specified by uncertainty. That is the root cause of many problems, but also brings many new possibilities and products.
Following the latest post on the State of AI Agents, I summarized the challenges of memory, security, testing, and data privacy of LLM products. I looked at how AI Engineers overcome these problems, e.g. with agent-specific products, and multi-agent frameworks.
I also asked the following LLM developers for their views and current approach to building:
- **Vasilje Markovic** (founder of PromethAI)
- **José María Lago** (co-founder of GenWorlds and Yeager AI)
- **Adam Silverman** (co-founder of AgentOps)
- **Kevin Rohling** (founder of BondAI)
Enjoy the reading!
### **Outline**
1. There is a new ecosystem being formed
2. Security and data privacy must be addressed before enterprise adoption
3. RAG holds a central place in the discussing agents' memory
4. Testing AI products needs a special approach
5. Building may become more affordable
6. Inter-agent communication is becoming a major topic
7. Conclusion… so do agents work yet?
## 1. There is a new ecosystem being formed
The boom of new agents in the Summer of 2023 was followed by the development of **frameworks, libraries, and tools** that support their functioning.
Agents give AI apps new capabilities through a **combination of reasoning and planning**, **self-reflection**, **tool usage**, and **memory**.
In the emerging AI-centered ecosystem, **LLMs are compared to kernel processes**.
Hundreds to thousands of people started to identify as **AI Engineers**, but **millions more are needed to fulfill the growing demand**, according to reports \[1], \[2]. A few days ago, San Francisco even hosted **the first AI Engineer summit**.
Relatively new LLM and agent-specific frontend and backend solutions, platforms for memory management, building frameworks, and SDKs are on the rise and are starting to work together.
A great example of new products working together is
- **AgentLabs** as an AI frontend with built-in Chat UI, attachments handler, and authentication
- **E2B** as a safe sandbox runtime for executing code output
- **OpenAI’s GPT-4** as a model
Similarly, you can build your code interpreter with Pyodide and Langchain, build agents with AutoGen and Chainlit UI, and many more. Even **Langchain**, the most popular agent framework, offers **over 500 integrations**.
## 2. Security and data privacy must be addressed before enterprise adoption
What are the outcomes of letting a LLM produce **untrusted code** run the user’s operating system?
A great example is the widely popular **Open Interpreter** which uses a **terminal as an interface**. The code doesn’t have to necessarily cause any irreversible damage. However, occasional uncomfortable outcomes still have been reported, for example, a GPT-generated command on `pip install new package` breaking the Open Interpreter dependency.
**The big problem is** making agents work in an arbitrary environment and runtime (custom environments). We need to be able to either **execute the code securely** or **test whether the code is correct** so LLM can fix itself.
José María Lago from GenWorlds adds to this:
“**The challenge is having the right mix between deterministic and non-deterministic (agents’) routines**. More mission-critical tasks require more deterministism. At GenWorlds, we **allow the developer to choose where to draw that line**.”
The challenges of untrusted code can be solved by running the code in secure cloud environments. If LLMs are a new kernel, then a cloud for running agents and AI apps is a new runtime.
Kevin Rohling, the founder of BondAI, thinks that in general, securing an agent isn't all that different than securing traditional software.
“For the products I've been developing, the security/code execution challenges are generally solved by running inside a heavily sandboxed Kubernetes node,” says Kevin.
“I highly restrict which outbound calls are allowed (e.g., obviously white list OpenAI), while I don't allow any inbound calls. I also heavily restrict what data the agent has access to.”
#### Data privacy and retention
Individual developers build chatbots using LLMs on private data. However, when it comes to the enterprise level, integrating AI solutions often requires an official legal data retention policy. It's not surprising that AI apps (with the most renowned ChatGPT at the forefront) are restricted or completely banned by many companies.
## 3. RAG holds a central place in discussing agents' memory
Recently, there has been hype \[3], \[4] around Retrieval Augmented Generation (RAG) as an efficient way to maximize the potential of LLMs without having to train your own. RAG combines retrieval and generation methods and its benefit for LLM apps is enabling developers to build on their private data.
If we compare “traditional” vector DBs, RAG, and large context windows, each has its own limitations.
#### Limitations of RAG
One unsolved problem for RAG is permissions, that is, figuring out which users are supposed to have access to what information in a chatbot or agent. Other potential challenges are a limited context window, having trouble utilizing the right data sources, or performing well in summary across the entire corpus.
#### Problems with vector databases
I discussed this topic with Vasilje Markovic, the founder of PromethAI, who is researching this topic.
“Vector databases have a problem with long upload times and can't handle loads. Retrieval doesn’t work as well, production needs dictate switching to something older that works,” says Vasilje. “A potential solution may be elastic search or relational databases.”
He adds that a pure semantic search doesn't add much value. “Since we are unable to understand dimensions such as time with only embeddings, we need to store more contextual information than just pure factual info.”
A big question analyzed in a new paper is whether vector databases are actually dangerous - it turns out that text embeddings can be inverted \[5].
#### RAG vs finetuning vs longer context window
1. Unlike finetuning, RAG allows security and data privacy, because you can decide at any moment, what documents should you give access to.
2. Expanding LLM context windows is seen as a solution to hallucination \[6]. However, recent studies \[7] reveal LLMs' struggles to extract information from large contexts, especially when the information is buried inside the middle portion of the context. Moreover, costs increase linearly with larger contexts.
#### How to solve the problems
“Adding a series of memory abstractions powered by a "central LLM connector" fine-tuned to manage memory components coupled with a traditional DB or a Graph DB would be one approach to the issue,” says Vasilje from PromethAI. Kevin from BondAI shares his own approach towards memory management:
“I created specialized agents that manage their own memory and I allow one "admin" agent to delegate tasks to those other agents,” comments Kevin. “This prevents a single agent from having to keep the whole problem context in memory at once. I also limit the maximum token size that any tool can respond with and architect my tools to prevent large files/content from polluting the memory space. A semantic search layer in the form of large documents helps a lot.”
## 4. Testing AI products needs a special approach
In the AI ecosystem, testing is crucial for ensuring quality code, just as in traditional software.
The co-founder and CPO of Codium, Dedy Kredo, gave a speech about how the industry's reliance on metrics such as code coverage is insufficient and can even be misleading, and the AI agents community needs to revolutionize our approach to testing.
Dedy believes that GAN-like architecture is the solution for testing AI-generated code. “GAN worked as two neural networks, one generating and one as a critic of the other. With transformers coming in 2017, the focus has been solely on the generative part.” \[8]
Adam Silverman from AgentOps shared that the team at AgentOps is developing a test suite for agents that uses custom benchmarks in addition to WebArena, AG benchmark, and BOLAA.
We have written a lot about testing and talked to founders of agent-centered products, e.g. the founder of Lindy AI, founders of Sweep AI, or the founder of Superagent.
## 5. Building may become more affordable
Leaked updates by OpenAI say it is going to make AI-based software application development cheaper and faster. The changes are supposed to happen next month and should include adding memory storage to developer tools, potentially reducing application costs by up to 20 times. \[9]
## 6. Inter-agent communication is becoming a major topic
In the past few months, end users started to take agents as something that does all the work for them.
That is far from true. Instead of trying to create powerful multi-purpose agents, they have been becoming invisible” part of a more complex product.
Using multi-agent conversations with multiple agents, each focused on a specific topic working together on the same project can create opportunities for complex workflows. Examples of popular multi-agent frameworks are AutoGen by Microsoft, ChatDev, and GenWorlds.
****Overall, building agents’ frameworks is still a hard battle to win. The re-built solutions may be limiting in terms of tech stack. Even one of the most popular frameworks for agents' creation, Langchain, has struggled with criticism from the community, e.g. for the difficulty of setting up, too much abstraction, or no clear benefits.
In the end, for many developers, a simple solution like Python + OpenAI may be better. The agent frameworks may still be great for well-established workflows.
### Does self-healing code work?
With the agents’ iteration and multi-agent communication, it is natural to ask about self-healing loops between agents.
However, a new paper from the Google DeepMind team shows that LLMs cannot “repair themselves” and achieve improvement through self-reflection, even with multiple agents.
"We investigate the potential of multi-agent debate as a means to improve reasoning. In this method, multiple instances of an LLM critique each other’s responses. However, our results reveal that its efficacy is no better than self-consistency when considering an equivalent number of responses, …"
However, AutoGen by Microsoft creates agents such that for example one writes a code, and another agent immediately debugs it, and it seems to work in examples and use cases. Also, GPT-4 becomes 30% more accurate when asked to critique itself. \[10]
## 7. Conclusion… so do agents work yet?
Agents have the potential to become a central piece of the LLM app architecture, but they still struggle with being production-ready. \[11]
“I think the only successful agents are the ones that do really small things. So very specific, small things like fix the color of this button on the website or like change the color of this button,” says Kanjun Qiu, the CEO and co-founder of Imbue, in the Latent Space episode.
A use case that is already working well is **agents for coding**, including data visualization (e.g. SuperAgent, Cursor, Aide). As we discussed in a previous post, agents are directed toward becoming a small part of a more complex product.
Hopefully, with LLMs consistently making progress toward better reliability, there is a bright future for agents. LLMs improve every few months, so consistency rises over time and becomes less of a worry.
Are you building AI agents, tools, SDKs, or frameworks and do you want to talk about your experience? Shoot us a message.
This post was written by E2B - sandbox runtime for LLMs. Any feedback and discussion is appreciated.
### References
- \[1] The Verge (article)
- \[2] The New Stack (article)
- \[3] Latent Space (podcast)
- \[4] AI Engineer Summit (event page)
- \[5] Text Embeddings Reveal (Almost) As Much As Text (paper)
- \[6] Pinecone Blog (article)
- \[7] Lost in the Middle: How Language Models Use Long Contexts (paper)
- \[8] AI Engineer Summit (video, 25:00)
- \[9] Reuters (article)
- \[10] Reflexion: Language Agents with Verbal Reinforcement Learning (paper)
- \[11] A16Z (article)
---
Source: https://e2b.dev/resources/microsoft-s-autogen.md
Published: 2023-10-08
# Microsoft's AutoGen - A guide to code-executing agents
After the initial hype around AI agents, there has been a cooling-off period as people realize that **AI agents are not** ***that*** **autonomous**. An agent won’t create the whole complex program dreamed up by a no-code user. Usually, until reaching a desired quality, agents' output needs multiple iterations.
These iterations may not be just human-agent, but rather among a higher number of agents specialized in narrow areas. For example, one agent writes a code specified by the end user, another agent then takes over and debugs the code, then hands it to another agent who can visualize the data, and so on.
[Recently launched](https://www.microsoft.com/en-us/research/blog/autogen-enabling-next-generation-large-language-model-applications/) AutoGen by Microsoft has gained especially big popularity among multi-agent [frameworks](https://github.com/e2b-dev/awesome-sdks-for-ai-agents).
## Simple Guide to AutoGen
What is special about AutoGen is that it is execution-capable of the code output it produces.
We will hence focus on that feature and create a simple data visualization Python script, where we explore different types of AutoGen pre-defined agent classes, and demonstrate how AutoGen generates and runs code. I hope it helps understand the principles of AutoGen.
### 1. Install AutoGen
Start with installing AutoGen from their [documentation](https://microsoft.github.io/autogen/docs/Getting-Started) or from [GitHub](https://github.com/microsoft/autogen).
```bash
pip install pyautogen
pip install "pyautogen[blendsearch]" for optional dependencies
```
AutoGen offers good support on their [Discord](https://discord.gg/E5YAjB5gRY), even though I don’t know whether it’s official. It also provides a page with concisely written [Examples](https://microsoft.github.io/autogen/docs/Examples) from which I choose the data analyst one to try.
### 2. Import packages
AutoGen has a default abstract class called [Agent](https://microsoft.github.io/autogen/docs/reference/agentchat/agent) that can communicate with other agents and perform actions. Agents can differ in what actions they perform in the receive method. We import `AssistantAgent` and `UserProxyAgent` classes, which are both subclasses of a more generic class - [ConversableAgent.](https://microsoft.github.io/autogen/docs/reference/agentchat/conversable_agent) (We will get to this later.)
```python
from autogen import AssistantAgent, UserProxyAgent
```
### 3. Get API Keys
Now, we get our API keys. I store mine as the `.env` variable.
```python
import os
import openai
from dotenv import load_dotenv
load_dotenv()
```
### 4. Create the agents
In this step, we can define a set of agents with specialized capabilities and roles.
We create an instance of the [`AssistantAgent class`](https://microsoft.github.io/autogen/docs/reference/agentchat/assistant_agent) representing the chatbot that will respond to the user input and an instance of the [`UserProxyAgent class`](https://microsoft.github.io/autogen/docs/reference/agentchat/user_proxy_agent/) representing the user that will initiate the conversation.
The LLM inference configuration in AssistantAgent can be configured via `llm_cofig`.
```python
assistant = AssistantAgent(name="assistant")
user_proxy = UserProxyAgent(name="user_proxy")
```
### 5. Define the interaction
After creating the agents, the script initiates a chat between the user and the chatbot by calling the `initiate_chat` method on the `user_proxy` instance. The `initiate_chat` method takes two arguments: the assistant instance, which represents the chatbot, and a message string that contains the task description.The script then creates a text completion request using the `openai.Completion.create` method.The `config_list` parameter is set to a list that contains a dictionary with the model name, API base URL, API type, and API key.The prompt parameter is set to a string that contains the text to be completed. The `Completion.create` method sends a request to the OpenAI API and returns a response that contains the completed text.
```python
user_proxy.initiate_chat(
assistant,
message="""Hello, today you are my data analyst assistant and you should help me visualize data, make predictions, and explain your thinking.""",
)
response = oai.Completion.create(
config_list=[
{
"model": "chatglm2-6b",
"api_base": "http://localhost:8000/v1",
"api_type": "open_ai",
"api_key": "NULL", # just a placeholder
}
],
prompt="Hi",
)
print(response)
```
### 6. Create a chat completion request
Finally, we create a chat completion request using the `openai.ChatCompletion.create` method. The `config_list` parameter is set to the same list as before, and the messages parameter is set to a list that contains a dictionary with the role and content of the user's message.The `ChatCompletion.create` method sends a request to the OpenAI API and returns a response that contains the chatbot's response to the user's message.
```python
response = oai.ChatCompletion.create(
config_list=[
{
"model": "chatglm2-6b",
"api_base": "http://localhost:8000/v1",
"api_type": "open_ai",
"api_key": "NULL",
}
],
messages=[{"role": "user", "content": "Hi"}]
)
print(response)
```
## Output
As I mentioned earlier, AssistantAgent and UserProxyAgent classes are both subclasses of a more generic class - [ConversableAgent.](https://microsoft.github.io/autogen/docs/reference/agentchat/conversable_agent)
The [AssistantAgent](https://microsoft.github.io/autogen/docs/reference/agentchat/assistant_agent) (a subclass of ConversableAgent) is designed to solve a task with LLM. This agent doesn't execute code by default and expects the user to execute the code. After the AssistantAgent produces code output, the user can execute the code by pressing Enter.
I chose my agent to visualize data, which is a task requiring multiple steps like planning the process, writing the code, and executing it in visual form. That should best show its capabilities.
First, the program answers to my default intro message:
```python
message="""Hello, today you are my data analyst assistant and you should help me visualize data, make predictions, and explain your thinking."""
```
I now instruct the agent to plot a chart of NVDA and TESLA stock price change YTD. It then prints user input and devises an action plan - which may include even installing new libraries.
The agent returned a code that contains an error that is indicated under user\_proxy. Here, the user\_proxy is used as another agent that provides feedback to the assistant, as opposed to a human instructing the agent with a prompt to fix the code.
The assistant makes another iteration that seems to be functioning code. This was a nice example of [self-healing code](https://stackoverflow.blog/2023/06/07/self-healing-code-is-the-future-of-software-development/).
The following diagram summarizes the workflow of iterating between multiple agents.
## Code execution
The output explains what happens if the user (you) decides to run the code. You can always execute the proposed code by pressing “enter”.
When a human user chooses to execute the code, the output opens in a new window like this:
It is possible to configure various arguments of the AutoGen agents. It is [open-source](https://github.com/microsoft/autogen), and I like how their [docs](https://microsoft.github.io/autogen/docs/reference/agentchat/conversable_agent/) are structured.
When modifying the ConversableAgent class, you can change the code\_execution\_config argument in the [\_\_init\_\_ method](https://microsoft.github.io/autogen/docs/reference/agentchat/conversable_agent#__init__) to even disable the execution of the code.
You can also modify the way to execute code blocks, single code blocks, or function calls, by overriding `execute_code_blocks`, `run_code`, `and execute_function` methods respectively.
The code from AutoGen agents is executed locally via `use_docker` - Bool value of whether to use docker to execute the code, or str value of the docker image name to use or None when code execution is disabled.
## Potential limitations
Why would you want to keep a close eye on the execution of the code run locally via Docker?
As the [Docker security article](https://docs.docker.com/engine/security/) mentions,
*One primary risk with running Docker containers is that the default set of capabilities and mounts given to a container may provide incomplete isolation, either independently, or when used in combination with kernel vulnerabilities*.
Granting autonomous AI tools access to executing code locally may be a challenge, especially for enterprise users.
#### Alternative solutions may be:
1. A better isolation of containers achieved by adding some barriers between them. However, containers like [Docker](https://www.docker.com/) would still use shared resources as the kernel.
2. Another option is using [sandboxed cloud environments](https://docs.e2b.dev). This provides security for running any code, starting processes, using the filesystem, and so on.
Another challenge with agent frameworks is scalability when the product acquires hundreds or thousands of users each developing their own AI applications, which would require thousands of containers.
This problem is solved for example by using cloud with [E2B SDK](https://docs.e2b.dev?ref=framer-microsoft-s-autogen).
## AutoGen Use Cases
I found a few examples of how people try AutoGen. It seems like it is still experimenting with the framework mostly for fun purposes, but maybe a time shows whether AutoGen becomes regularly used for work purposes too.
1. [Snake Game Development with AutoGen](https://github.com/Poly186-AI-DAO/AutoGen-Snake-Game) - A project structured around a group chat setup where different agents collaborate to bring the snake game to life. [YouTube video](https://www.youtube.com/watch?v=gnn1H4H81IY)
2. [Enhanced Agents](https://github.com/Andyinater/AutoGen_EnhancedAgents) - Debuting with a MemoryEnabledAgent with improvements in context/token control, portability, and PnP functionality
3. [Scene Writer](https://github.com/abhilashi/ai-explorations/blob/main/ai_scene_writer.py) - A simulation of a fictional scene with AI screenwriters, a couple of assistant agents, and a critique
4. [AgentXP](https://twitter.com/oscarmoxon/status/1708603929011863871) - A self-improving agent that is eventually able to write itself
5. [Meme creator](https://colab.research.google.com/github/githubpradeep/notebooks/blob/main/autogen_meme_creator.ipynb)
6. [Agentcy](https://github.com/amadad/agentcy) - An example with agents’ roles such as Account Manager, Strategist, Marketer, Researcher, or Designer
## Resources
- [Paper: AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation](https://arxiv.org/abs/2308.08155)
- [AutoGen Documentation](https://microsoft.github.io/autogen/docs/reference/agentchat/conversable_agent)
- [GitHub - AutoGen](https://github.com/microsoft/autogen)
- [Docker Documentation - Docker Security](https://docs.docker.com/engine/security/)
- [My tweet about AutoGen](https://x.com/tereza_tizkova/status/1707771482779127923?s=20)
- [AutoGen: Enabling next-generation large language model applications](https://www.microsoft.com/en-us/research/blog/autogen-enabling-next-generation-large-language-model-applications/)
---
Source: https://e2b.dev/resources/github-copilot-vs-cursor-so-vs-aide-battle-of-ai-coding-tools.md
Published: 2023-09-22
# Battle of AI coding tools
There has been a boom of AI-powered coding tools, like [GitHub Copilot](https://github.com/features/copilot), [Sweep](https://sweep.dev/), [GPT Engineer](https://github.com/AntonOsika/gpt-engineer), [codium](https://www.codium.ai/), or [Open Interpreter](https://openinterpreter.com/) recently trending on global GitHub. They have been a big topic, as people are trying to test as many of them as possible.
I compared the more established GitHub Copilot with two newly launched AI coding copilots, [Cursor](https://www.cursor.so/) and [Aide](https://codestory.ai/), both built as a modification of [VSCode](https://code.visualstudio.com/).
They all intend to make the developer experience easier, are compatible with Windows, MacOS, and Linux, and offer writing, suggesting, debugging, and explaining of code. I compared their features and tested each of them in multiple “categories” on a simple programming task.
### GitHub Copilot
[Copilot](https://github.com/features/copilot) is a two-year-old tool created by Microsoft-backed OpenAI. It has been trained on a selection of English language and source code from publicly available sources, including code in public repositories on GitHub.
### Cursor
[Cursor](https://www.cursor.so/) by Anyspehere is an AI-powered code editor that recently gained [big popularity](https://github.com/getcursor/cursor). The IDE is a fork of VSCode, it can generate code from scratch, ask questions about your codebase, edit code with prompts, debug the code, or explain it. Cursor is currently being developed by a small team and is described by them as an “[attempt at a new way to write code](https://anysphere.co/).
### Aide
[Codestory](https://codestory.ai/) is a YC23 startup, that is a few months old, and the team consists of just two people. Their product, Aide, is an AI-first IDE, currently in the alpha stage, and only supporting JS/TS projects, however already used regularly.
Aide founders realize that SW development is not just about writing code, but also debugging, refactoring, testing, reviewing code, and planning for new features. For example, “Aide agent can invoke to do multi-file edits,” says Sandeep Pani, the CEO of Codestory.
## Comparison
**Links**
- Companies: [GitHub](https://github.com/about), [Anysphere](https://anysphere.co/), [Codestory](https://www.ycombinator.com/launches/JCn-codestory-is-an-ai-powered-mod-of-vscode)
- Founders' Linkedin profiles: [Sualeh A.](https://www.linkedin.com/company/anysphere/people/), [Michael T.](https://www.linkedin.com/in/michael-t-5b1bbb122/), [Arvid Lunnemark](https://www.linkedin.com/in/arvid-lunnemark/), [Aman Sanger](https://www.linkedin.com/in/aman-sanger-482243171/), [Sandeep Pani](https://www.linkedin.com/in/sandeep-kumar-pani/), [Naresh Ramesh](https://www.linkedin.com/in/naresh-ramesh/)
I tested each of the tools with GPT-4, and compared them across the following categories:
1. How easy they are to set up
2. How well they generate code from scratch
3. How they suggest code
4. Ability to edit
5. Debugging
6. Code explanation
7. Support and communication
Since I am not a full-time developer, I chose a beginner-level program to test the tools on. I created a simple version of a [Blackjack](https://pi.math.cornell.edu/~mec/2006-2007/Probability/Blackjack.htm) card game, where a player decides whether to “hit” (take more cards) or “stand” (skip taking more cards) and then competes with a dealer in getting the sum of their card values closer to 21, without exceeding it.
You can check out the files with code written by myself, Copilot, Cursor, and Aide, in my public Blackjack repository.
## 1. Setting up
First, I looked at how easy it is to install these tools and get them running. All three tools were quick to set up and I started using them almost immediately.
### GitHub Copilot
You can [set up Copilot](https://docs.github.com/en/copilot/getting-started-with-github-copilot) via JetBrains IDEs, Vim/Neovim, VS, or VSCode, which was my choice. Before you can start using GitHub Copilot, you will need to set up a free trial or subscription for your personal account. You can then enable Copilot on your GitHub or install it as a VSCode extension. I struggled a bit with installation, for example, it took me some time to allow the Chat feature, which was in the pre-released version.
### Cursor
Cursor is downloadable from the [landing page](https://www.cursor.so/). Their UI is almost identical to VSCode. The time between installing and starting to use Cursor was really short for me, compared to the other two.
### Aide
You can download Aide directly from their [landing page](https://codestory.ai/) or ensure the latest version from the [releases page](https://github.com/codestoryai/binaries/releases).
[Installation](https://codestory.ai/) is easy.
1. Open a JS/TS project folder.
2. Press Cmd + Shift + P to open the command palette and search for Import settings and keybindings from VSCode. Hit enter to run the command. For this command to work, I first had to sign up to GitHub via Aide and clone my GitHub repo for this command to work.
Aide supports importing all your settings and extensions from VSCode.
### 🥇Winner: Aide
**Reasoning**: I found setting it up the smoothest and liked the structure of their documentation that includes a comprehensive step-by-step guide, and also limitations, given the very early product. I struggled to find Cursor’s docs and to set up all Copilot’s features quickly.
## 2. Code generation
Next, I compared, what the tools provide on my prompt. I used variations of “Please write an interactive blackjack card game program.”
### GitHub Copilot
To access the code-writing feature of Copilot, you need to enable the pre-released version of the product in the VSCode. Once enabled, you can use the chat feature to input prompts and receive code. This feature is not yet available in the released version.
The code generated was satisfactory and fully functional. I was able to run it and play a simple version of the game.
### Cursor
In Cursor, you have two basic options to communicate with the Agent.
- Command K lets you edit the highlighted code or ask anything about it.
- Command L opens a chat interface.
I used the latter to generate the entire codebase based on my prompt. However, when I tried generating a Blackjack game, it failed to complete the task to my satisfaction - the provided code doesn't provide an interactive card game when run.
### Aide
The Aide agent is incapable of generating code from scratch. Instead, it relies on the code uploaded by the user, searching through the existing codebase for the desired results, as shown in the image.
I asked Sandeep Pani, the CEO of Codestory, about Aide’s ability to generate code. “Aide does not know how to generate code from scratch today,” says Sandeep. “We do think there exists a workflow where code generation from scratch becomes important (e.g., if you are working with a new library or API) and plan to add support for it in the upcoming weeks.”
### 🥇 Winner: Copilot
**Reasoning**: Cursor provided non-functional code, and Aide doesn’t have yet the feature of code generation.
## 3. Code completion
GitHub CopilotRegarding suggesting code, GitHub Copilot managed to create my desired program basically from scratch by suggesting smaller code snippets correctly. It can suggest for example entire functions or methods. I was pleasantly surprised that it copied my style of coding, putting an explanation comment behind every piece of code.For each suggestion, you can pick from (usually) 1-2 options, by switching the suggested snippets with arrows. If you use chat to make the Copilot agent finalize the whole program at once, it will do it.
The only drawback is that the Copilot can get into an infinite loop of suggestions.
### Cursor
Cursor managed to complete the blackjack game code. The code it produced as a completion was not an interactive game, but just a mechanism of blackjack that runs automatically.
However, the Cursor agent realized this drawback when asked (it explained that “In the current implementation, the player automatically draws cards until their score is 21 or higher, which isn't how a typical game of Blackjack is played.)” and suggested a modification of a play method in the code to make it interactive.
I appreciate how the agent communicates one step beyond just providing an answer and suggests improvements proactively.
### Aide
I asked Aide to complete a JS blackjack code. It first searched through my codebase and provided a few queries and selected files to use (image 1).
It took a longer time to complete the code, compared to Copilot and Cursor. It seemed like Aide tried to improve the existing codebase solution it found by suggesting new code snippets to add (images 2, 3).
I often came across the error “Code modification generation failure.”
### 🥇 Winner: Copilot
**Reasoning**: The Copilot agent suggests code in smaller parts and offers multiple versions, which helps me, as a beginner programmer, to get insight into the process. It provided the correct solution on the first try.
## 4. Code editing
### Copilot
The Copilot agent easily edits the chosen code. This feature is working even without the pre-released version.
However, it didn't finish my task to put a comment to each line of code, which it did only partially.
### Cursor
Cursor edited my code correctly.
### Aide
Aide isn’t able to edit code right now.
### 🥇 Winner: Cursor
**Reasoning**: The Cursor agent was the only one that completed the given task of editing code.
## 5. Code debugging
CopilotCopilot managed to identify and fix all the bugs that I created in the Python code, and I had no objection.CursorCursor debugging made me confused.I made some errors for Cursor, plus I expected it to fix the bugs the agent itself made when generating the code.It didn’t manage to do any of that, and the code wasn’t functioning even after debugging.On the other hand, Cursor sometimes debugged even the correct parts by adding redundant code, e.g., adding a card to a player’s score by re-writing the variable, even though the card was already added there before by .append.
### Aide
For Aide, I appreciated that it not only suggested a proper fix, but also explained my bug in the chat interface.
I struggled to find an option to highlight and debug particular code snippets. Sometimes it doesn't react correctly on debugging (image 2).
However, I still value the error explanation, instead of just suggesting improvements.
### 🥇 Winner: Copilot
**Reasoning**: Copilot provided the most reliable debugging, even though it was simple and didn’t explain the bugs.
## 6. Code explanation
I fed Copilot and Cursor agents with my version of the blackjack game python code, so I could benchmark the explanation on the identical program. For Aide, I use the JS version.
### Copilot
### Cursor
### Aide
I failed to ask Aide to explain my code. When I asked to explain the highlighted code, it wrote what mistakes it contained, instead of explaining the structure.
### 🥇 Winner: Cursor
**Reasoning**: Cursor provides an explanation with higher granularity, and refers to particular parts of code while it also provides more complex comments on the code, including a high-level description.
## 7. Support & Communication
This category is equally important to me as the technical features.CopilotGitHub Copilot already gained thousands of users, so it is no wonder that they don’t offer any quick support. I think the only way to get support is via the official [GitHub support](https://support.github.com/) page.CursorI contacted Cursor on their contact e-mail, asking a few questions. I have been waiting for their response for few days now.AideAide answered my e-mail until the second day. They provided lengthy answers to my questions and were nice. Also, I appreciate that Aide agents showed me the links for Discord and e-mail support.
### 🥇 Winner: Aide
**Reasoning**: They were responsive and willing to help.
## Conclusion
It is difficult to state one winner. From my view as a beginner programmer, I would use each of the three tools occasionally for help in the specific areas they work for.
Given the early stage of Cursor and Aide, GitHub Copilot indeed feels more like a mature product. However, each has its pros and cons.
#### Overall, these were my biggest struggles with each tool:
- **Copilot**: Setting up the chat, price (no free option)
- **Cursor**: Doing too much work at one time, but with not a great quality
- **Aide**: Knowing how to control it (e.g. how to refer to a particular code when chatting about it), and frequent error messages that I didn’t understand
#### Overall, these were the biggest strengths I see in each tool:
- **Copilot**: The suggestions for code snippets
- **Cursor**: Intuitive UI, generating entire codebase from scratch
- **Aide**: Explanations during debugging code, support, price
**Disclaimer**: This review is highly subjective, and your experience may differ by the type of your work, seniority level, or purpose of using the tools. If you have any comments on this comparison, please contact me at tereza@e2b.dev
#### More reading and listening on the coding tools
- [More open or closed-source agent-powered coding tools](https://github.com/e2b-dev/awesome-ai-agents)
- [Latent space podcast about Cursor with a founder of Anysphere](https://www.latent.space/p/cursor#details)
- [Introducing GitHub Copilot X - a leveled-up Copilot](https://github.com/features/copilot)
---
Source: https://e2b.dev/resources/about-building-tools-for-llm-agent-with-flo-crivello-ceo-at-lindy-ai.md
Published: 2023-09-14
# Building tools for LLM agents with Flo Crivello - CEO at Lindy AI
[Lindy](https://www.lindy.ai/) is an AI assistant whose primary objective is to save users' time. Lindy is described as “supremely reliable and attentive to detail.” We had an interview with its creator, [Flo Crivello](https://twitter.com/Altimor), about his methodology in developing Lindy AI and internal tooling for the agent.
## Overall approach towards building agents
Flo comments on how they are building Lindy.
“We think of our approach as two halves. An AI agent needs:
1. The correct collection of tools
2. To know how to utilize them."
They are building a framework focused on the second point in particular - integrations aren't the big risk here.
Right now, the Lindy AI team is especially focused on techniques here that will get the agent to self-improve / learn from its own experience how to use its tools better and better.“
## Main use cases and ideal users
Lindy AI is a personal assistant for making a user's life more efficient. It can assist with all daily tasks, from managing user's schedule and composing emails to sending contracts, and more.
“Our ideal user is a senior manager in the technology space,” says Flo.
## Reliability
Lindy has made great progress in reliability. When unsure about a task or when about to perform a high-stakes action, the agent asks the end user for confirmation.
"These confirmations become increasingly unnecessary as time goes by, both because we make Lindy smarter, and because she learns the user's preferences," explains Flo.
## Building agents tools
We asked Flo how they currently approach agent's debugging, monitoring, and tracing, what are the main struggles in this area, and how they are planning to solve them.
"We build a wide range of internal tools for example for tracing, and monitoring." shares Flo. "The agents' debugging in principle is similar to regular debugging: you examine the logs you have, try to reproduce them, then try different solutions until the bug is fixed. "
The Lindy AI team has developed all the agent tooling in-house.
"Our most important tool is probably the tracer, which shows step by step what the agent did exactly to fulfill a user query," says Flo. "It is similar to [Langsmith](https://smith.langchain.com/), but way better in our opinion."
**Lindy AI team also has tools for**:
- Seeing the "lessons" that the agent is learning over time (the Memory in the screenshot above) and editing them
- Reviewing the tools available, and editing the instructions about when and how to use each tool for the agent
- Editing global rules or action-level rules
- Monitoring benchmarks
## Other challenges
During the discussion, Flo mentioned additional problems that they are presently aiming to resolve, specifically fine-tuning and cognitive architecture.
"Right now, coming up with the right cognitive architecture is insanely challenging for us," says Flo, "but we wouldn’t outsource it, since we believe that’s our purpose. "
"Fine-tuning our model is also very painful, especially because we require a big model (40B+ parameters) and a big context window (8k+)," adds Flo. "I foresee that once we have that model, deploying it for inference at scale will be another significant challenge."
---
Source: https://e2b.dev/resources/the-state-of-ai-agents-reliability-sdks-benchmarking-and-market-trends.md
Published: 2023-09-07
# The State of AI Agents
Over the last few months, we have looked into [around 100 agents](https://github.com/e2b-dev/awesome-ai-agents) with various use cases, studied [SDKs and frameworks for agents](https://github.com/e2b-dev/awesome-sdks-for-ai-agents), and discussed challenges faced by agents with founders of [Cognosys](/resources/about-deployment-evaluation-and-testing-of-agents-with-sully-omar-the-ceo-of-cognosys-ai), [Aomni](/resources/david-zhang-from-aomni-gives-his-view-agents-reliability-debugging-and-orchestration), [Superagent](/resources/discussing-agents-challenges-with-ismail-pelaseyed-the-founder-of-superagent), [Sweep](/resources/sweep-founders-share-learnings-from-building-an-ai-coding-assistant), and more.
Here is what we learned about products built on top of agents, their challenges, standardization, and the future.
## 1. The space lacks consensus on the definition of an AI agent
There is still some ambiguity in the terms like "agents", "AI agents", "autonomous agents", or "LLM agents".
We define an agent (using interchangeably with the other variations) similarly to Shawn Wang, aka “Swyx” (founder of [smol ai](https://github.com/smol-ai)), [Matt Schlicht](https://www.mattprd.com/p/the-complete-beginners-guide-to-autonomous-agents) (CEO of Octane AI), and mainly [Lilian Weng](https://lilianweng.github.io/posts/2023-06-23-agent/) from OpenAI.
AI agents possess three main capabilities.
1. They combine reasoning and acting. The agent uses LLMs like GPT-3.5 and GPT-4 to understand, execute, and reflect on tasks.
2. They have both short and long-term memory.
3. Agents can use "tools" by calling external APIs - for example, it can browse the web, use apps, read and write files, make payments, and even control a user's laptop.
These qualities distinct agents from semi or non-autonomous LLM-powered apps. When compared with “mainstream” automation - where you set up a range of triggers based on data or system states and configure what happens next - AI agents can work in unpredictable environments where there's a lot of new information.
## 2. Agents switch from a standalone product to an “invisible” feature
Defining agents correctly may not be needed soon, as the trend is moving from popular standalone agents, often trying to solve a broad variety of problems at the expense of quality, to agents being just an unmentioned part of a bigger product.
Companies work on agent-powered assistants as an additional feature in existing products. Examples include Hyperwrite AI's [Otherside](https://github.com/e2b-dev/awesome-ai-agents#othersides-ai-assistant), which serves as a personal assistant for daily tasks, [MultiOn](https://multion.ai/), a personal life assistant, and [Deepnote’s AI Copilot](https://deepnote.com/blog/introducing-deepnote-ai).
We see an increase in the complexity of the agents-centered projects. [Sweep](https://github.com/e2b-dev/awesome-ai-agents#sweep), for instance, is an open-source GitHub assistant with a significant amount of code built around the AI agent. Another example is [Grit.io](https://www.grit.io/) - a tool for automated code migrations and dependency upgrades.
## 3. Agents still have a long way to enterprise-level reliability
The main incentives for enterprises to use agents are saving costs and money. However, they are still hesitant towards agents until they become more reliable.
“For enterprise customers, we are talking at least ~99.9% reliability," thinks David Zhang, the founder of Aomni Agent.
The end users have high standards for fast software, while LLM-powered agents sometimes run slow. Sully Omar, the CEO of Cognosys, [comments](/resources/about-deployment-evaluation-and-testing-of-agents-with-sully-omar-the-ceo-of-cognosys-ai): "In traditional SW engineering, around 200 milliseconds is already considered slow. For agents and LLM apps, latency is a big issue, with LLM calls taking more than 30 seconds."
In general, developers of agents currently struggle with testing, evaluating, debugging, latency, and monitoring. One particular example of a common problem is identifying at what step their agent broke and why.
Another big question that runs through the entire AI industry is that of privacy, security, and data retention policy.
## 4. Agents are in need of specific SDKs and frameworks
Agent developers differ in the paradigms they choose for solving the said challenges.
They either build on top of existing tools, create their own internal solutions, or adopt some of the products built specifically for agents, many still in an early stage or in alpha/beta version.
##### Existing “traditional software” solutions
David Zhang, the founder of Aomni, points out how [a lot of agent developers try to reinvent the wheel](/resources/david-zhang-from-aomni-gives-his-view-agents-reliability-debugging-and-orchestration) with new frameworks and SDKs, instead of building on top of existing technology.
Developers chose solutions for equivalents of agents’ problems in traditional software, e.g.
- [Inngest](https://www.inngest.com/) for orchestration and debugging of agents
- [Sentry](https://sentry.io/welcome/) for observability
- [LlamaIndex](https://www.llamaindex.ai/) for data integration.
##### Agent-specific solutions
The traditional software solutions still fail for very agent-specific challenges given by the nature of LLMs. One example is debugging agents, which is essentially playing around with prompts, and the [lack of an agent equivalent of real-time debugging](/resources/david-zhang-from-aomni-gives-his-view-agents-reliability-debugging-and-orchestration).
We have met with developers of agents like [Grit](https://www.grit.io/) or [Sweep](https://sweep.dev/), who are either building their completely custom infrastructure or trying to use existing technologies to at least somehow fit their agent use-case. As [mentioned by Swyx](https://www.latent.space/p/aug-2023), the infrastructure complement to multi-agent systems is agent clouds. E2B has built [AI playgrounds](https://docs.e2b.dev?ref=framer-the-state-of-ai-agents-reliability-sdks-benchmarking-and-market-trends), sandboxed cloud environments for agents or AI apps, that are especially useful for the coding use-case of agents.
There are more projects tailored for AI agents or LLM apps, most often frameworks for building, monitoring, and analytics.
## 5. The community is looking for standards for autonomous agents
As we're moving closer and closer to more advanced agents, the [community is having discussions](/resources/agent-protocol-developers-community-setting-a-new-standard) about establishing a common “framework” to help the agent ecosystem grow faster and simplify the work.
Particular questions include how to design realistic benchmarks for better evaluation of agents' performance, and also to incorporate safety considerations.
### Benchmarking
The [benchmarking effort](https://github.com/Significant-Gravitas/Auto-GPT-Benchmarks) (a benchmarking tool for [Agent Evals](https://github.com/agbenchmark/agent-evals/tree/main)) by AutoGPT originates from a need to truly understand the agent’s ongoing processes and to determine whether the modifications made to an agent genuinely enhance its performance.
The biggest challenges with designing the agents’ benchmarks are cost, time, and choosing the most optimal design of tests. There is a tradeoff between the diversity and uniqueness of the testing environment versus realism and naturality.
“If an agent fails a simple test, it won’t pass the more difficult ones. Part of the challenge is hence structuring tests in the correct order” said Silen Naihin, an R&D lead at AutoGPT, in the [X space about agents benchmarking](https://twitter.com/TechySwift/status/1689669584683503618?s=20).
##### Other benchmarking efforts:
- [WebArena](https://webarena.dev/) - A realistic web environment for building agents
- [MACHIAVELLI benchmark](https://aypan17.github.io/machiavelli/) - An environment is based on human-written, text-based Choose-Your-Own-Adventure games containing over half a million scenes with millions of annotations.
### The Agent Protocol
The Agent Protocol, adopted in the AutoGPT [benchmarks](https://github.com/Significant-Gravitas/Auto-GPT-Benchmarks), is a tech stack agnostic way to standardize and hence benchmark and compare AI agents.
It is an OpenAPI specification v3-based protocol - a list of endpoints, which the agent should expose with predefined response models, and defines an interface for interacting with your agent. Developers of LLM apps, such as [AutoGPT, LemonAI, or BabyAGI](https://github.com/AI-Engineers-Foundation/agent-protocol#open-source-agents-and-projects-that-have-adopted-agent-protocol) are currently adopting the protocol.
The protocol serves as a single communication interface with agents, making it also easier to develop developer tools that work with agents out of the box.
## 6. Agents are moving in the vertical direction
The hype where people experimented with the first open-source agent projects like [AutoGPT](https://github.com/e2b-dev/awesome-ai-agents#autogpt) or [BabyAGI](https://github.com/e2b-dev/awesome-ai-agents#babyagi) is starting to gradually calm down. End users are now looking to solve specific problems.
Agent use cases are being narrowed down to achieve perfection in one specific role. Today’s most common use cases are [coding, personal daily tasks, or research](https://github.com/e2b-dev/awesome-ai-agents).
The future of software will likely include apps powered by dozens of “small” AI agents serving specific purposes and interacting with each other. Agents will need their own secure cloud space to seamlessly communicate and conduct their tasks with autonomy.
We may expect a further shift towards a vertical market, for example, one app with different underlying agents designed for code writing, code debugging, code migration, e-mail communication, calendar planning, and task management.
#### Communication with end users
To increase the ratio of returning users, developers focus on showcasing real tangible results and use cases instead of over-explaining how the agent works and why people should use it.
Sully Omar, the founder of Cognosys AI, enhances, how users care about tangible results, rather than underlying technology. [“For example, offering users different models is redundant if they do not understand which is the most suitable for their needs,”](/resources/about-deployment-evaluation-and-testing-of-agents-with-sully-omar-the-ceo-of-cognosys-ai)
A famous example of avoiding description of the technology itself is Apple, [not mentioning “AI” at all](https://www.businessinsider.com/why-ai-artificial-intelligence-wasnt-mentioned-apple-wwdc-tim-cook-2023-6) during an important presentation, or not mentioning “metaverse” because [“the average person doesn't know what it means](https://www.businessinsider.com/tim-cook-apple-avoids-term-metaverse-facebook-2022-10)”.
## Conclusion
Agents still have a long way to enterprise-level reliability. There are still challenges to overcome with [agent-specific SDKs, frameworks, and tools](https://github.com/e2b-dev/awesome-sdks-for-ai-agents). The biggest ones are debugging, monitoring, deployment, and benchmarking of agents. The Agent Protocol is one of the efforts to standardize agents and improve their communication and benchmarking.
The space switches from agents as a standalone code to “agent as a feature”, being part of a more complex product. Agent developers are focusing on more narrow use cases and learning to communicate better with end users.
The most common use cases of agent technology are coding, personal assistance with daily tasks, and search. We see that the future of software includes autonomous LLM agents.
For trying out autonomous agents, check out the [overview of popular AI agents](https://github.com/e2b-dev/awesome-ai-agents).
---
Source: https://e2b.dev/resources/sweep-founders-share-learnings-from-building-an-ai-coding-assistant.md
Published: 2023-09-01
# Sweep Founders Share Learnings from Building an AI Coding Assistant
**Coding and code debugging** have been the most common use case of AI agents, just some of the many examples including [Smol Developer](https://github.com/smol-ai/developer), [GPT Engineer](https://github.com/AntonOsika/gpt-engineer), [AutoPR](https://github.com/irgolic/AutoPR), ReactAgent, and [Bloop](https://bloop.ai/). [**Sweep**](https://github.com/sweepai) **represents this category of agents in** [**YC S23**](https://www.ycombinator.com/companies/sweep).
[Sweep](https://sweep.dev/) works by **generating code on GitHub issue submissions**, e.g. your bugs and feature requests. It generates code in the form of **pull requests**, which the user can **comment on and iterate** until reaching their desired result. The following diagram shows the pipeline of turning an issue into a pull request.
We asked the [Sweep](https://sweep.dev/) founders - [William Zeng](https://twitter.com/wwzeng1) and [Kevin Lu](https://twitter.com/kevinlu1248) - for their view on the current agents' space, and the challenges they are working on to solve for the agents.
## Users and Use Cases
[**Sweep**](https://github.com/sweepai) **works with Github issues**. Given an issue, the [Sweep](https://sweep.dev/) agent plans how to solve it, writes code, and turns the issues directly into a pull request (without an IDE). It uses **embedding-based code search**.
The **initial startup time usually takes 3-5 minutes**, depending on the codebase. Users can iterate on results by adding comments to pull requests.
As many agents for developers, it is **open-source**, and getting increasingly popular, with over [5000 stars on GitHub](https://github.com/orgs/sweepai/repositories).
“We target really strong developers and try to help be as efficient as possible” say the founders.
The current total usage is **around 100 pull requests a day**. What the end users build with [Sweep](https://sweep.dev/) differs, since the developers are utilizing it for their existing projects.
## Current Challenges
Kevin and William share their experience with agent instances failing. “When an agent fails, it is in **approximately 20% cases caused by prompts, and 80% all kinds of other issues**.” The bug has often a combination of root causes.
Sometimes the agent straight away gets the wrong idea or uses the wrong format for the result of the task. If there is a bug, they can go and twist a prompt a little bit, and fix it.
“Given that the **agent operates in a linear manner**, it is usually easy to determine what caused the issue and decompose the process into steps,” said Kevin and William.
The agent’s workflow is pretty constrained and so, there is no need to stop it in the middle. Instead, when a bug happens, **restarting the whole process is the most pragmatic way**.
## Debugging tools
**Debugging has been a struggle for most agents’ developers**, and there is no strong consensus about agent-specific [tools or frameworks](https://github.com/e2b-dev/awesome-sdks-for-ai-agents) used for debugging agents' prompts and errors in general.
“With Sweep, the process of debugging usually involves **looking through the history of the conversation with the agent**,” summarize the Sweep founders, who **created their own internal tool to visualize the chats**.
“We built this **chat visualizer** in 2 hours,” say Kevin and William. “If a user contacts us, **with their permission to view their logs for debugging purposes, we can visualize what happened and fix what needs to be fixed, then click redeliver**."
End users cannot access the visualization, so they usually contact the Sweep team. “This happens about once a day”, William and Kevin say.
The following image shows [Sweep](https://sweep.dev/)’s internal visualization of agents’ bugs.
## Communication with users
The **user experience is very server-centralized**, as Kevin and William comment: “If the servers have an issue, everyone has an issue.” The [Sweep](https://sweep.dev/) team often **learns about issues directly from the end users** and consequently assists in fixing them.
The Sweep team aspires for users to engage their agents in meaningful work-related tasks, not just playing with [test repositories](https://github.com/sweepai/create-react-app) or experimenting. “Since we found at the start that developers are still using AI agents for the sake of exploring new technologies and having fun, we focused our strategy on **limiting the usage of Sweep**. This way, the **users are mindful and use Sweep for the actual work**.”
They hence limit to **5 uses a month of** [**GPT-4**](https://openai.com/gpt-4), **the rest of GPT-3.5**.
## Conclusion
Users can work with [Sweep](https://sweep.dev/) independently, without assistance. William and Kevin never stop developing their agent and **making even bigger progress toward agents becoming regular work tools**. “Currently, we are reading a lot into the old style of indexing code. The goal is to become proficient in both code understanding and agent understanding.”
If you want to try [Sweep](https://github.com/sweepai/sweep), you can install it via [https://github.com/sweepai/sweep](https://github.com/sweepai/sweep). After installation, you can add the repository you want the agent to work on, make a ticket (e.g. writing tests), add the label "sweep" and watch the AI agent do the work.
If you are interested in open-source agents for developers and coding, check the whole [AI agents database](https://github.com/e2b-dev/awesome-ai-agents).
---
Source: https://e2b.dev/resources/about-deployment-evaluation-and-testing-of-agents-with-sully-omar-the-ceo-of-cognosys-ai.md
Published: 2023-08-28
# About deployment, evaluation, and testing of agents with Sully Omar, the CEO of Cognosys AI
Cognosys provides a closed-source UI for creating AI agents. Their vision is to develop an easy-to-use consumer-facing product to assist non-technical individuals in completing specific daily tasks.
We asked the founder of Cognosys, [Sully Omar](https://twitter.com/SullyOmarr), about his experience with building a product for no-code users in the Agents space.
### Users and Basic Architecture
Cognosys is a web-based version of AutoGPT/babyAGI working in "loops" - a series of tasks. The agent generates output based on provided objectives and iterates until completion. It puts high-level tasks into smaller ones, calls an LLM, and iterates until the task is done. The whole process takes a few seconds and requires zero coding. Sully emphasizes that the crucial moment is the first trial of Cognosys. “The people that have **found value within first experience with the agents**, are the ones who become **returning users**.”
There are more options offered for the agent’s “specialization”.
“Currently, we are focusing on **narrowing down the use cases** to just a few,” explains Sully. “People find the biggest value in letting the agent **dig into the internet**.”
The agent for searching over the internet, similar to [**Perplexity AI**](https://www.perplexity.ai/), is currently the most popular one. The research agent takes an objective, conducts internet research, synthetizes it, and provides links to relevant sources.
### Overcoming Agents Challenges
Sully comments on the current problems of agent developers.
“Locally, **monitoring of LLM agent’s steps** is easy, but **tracking what is happening at scale on the aggregate level is the most important challenge** to solve, for any company using LLMs in general.” For tracing agent runs, Cognosys uses mostly its own UI.
An important concern is how much information users should receive about the agents. According to Sully, for example, **offering users a choice between GPT-3.5 and GPT-4 is useless** if they do not understand which models are suitable for their needs. He believes that the primary concern of users is whether the agent can perform the expected tasks.
## 1. Deployment
The Cognosys team started by using the [**Vercel edge function**](https://vercel.com/features/edge-functions), which had **a limit of 60 seconds for a timeout**. However, this posed a problem for Cognosys, since occasionally, the agent needs more time to execute.
They have tried [**Cloud Functions**](https://cloud.google.com/functions), which didn’t yield optimal results. Now they use **an instance of** [**Cloud Run**](https://cloud.google.com/run) **that all main systems run on**. “The advantage is that **we get a unified API via an API Gateway** for agents and can easily spin up tens of agents for a single user,”
There are issues that are associated with LLM calls in general. **Serverless functions are meant to take 10-50 milliseconds**. With LLM calls taking much more time, it doesn't make sense for Cognosys to use serverless architecture. They do use serverless for minor things, e.g. updating users’ profiles.
## 2. Observability
The Cognosys team is exploring a variety of tools, using different **infra plugins for observability**, which is a challenge due to multiple factors contributing to the success or failure of agents.
“We have tried [**Sentry**](https://sentry.io/welcome/), [**Google Cloud**](https://cloud.google.com/), [**Google Cloud Platform**](https://cloud.google.com/gcp?utm_source=google&utm_medium=cpc&utm_campaign=emea-emea-all-en-bkws-all-all-trial-e-gcp-1011340&utm_content=text-ad-none-any-DEV_c-CRE_500236788675-ADGP_Hybrid+%7C+BKWS+-+EXA+%7C+Txt+~+GCP+~+General%23v3-KWID_43700060393213451-aud-1641092902540:kwd-87853815-userloc_9048063&utm_term=KW_gcp-NET_g-PLAC_&&gad=1&gclid=Cj0KCQjwoeemBhCfARIsADR2QCunVw3Ik-GCi1d1E_3uLJSLKNP2vdpjV_KCneskLRfd49cfS7n6rVsaApsGEALw_wcB&gclsrc=aw.ds).” names Sully. “Another one we are starting to look at and that is agnostic to agents, but still in beta version, is [**Langsmith**](https://smith.langchain.com/).”
The key aspect of observability is understanding which tools the agent uses throughout the process and whether they are the right choice.
### 3. Testing and Evals
“**Evaluation is currently a big challenge** for autonomous agents in general, due to the LLMs nature,” says Sully. “How do you define good output, especially for the longer and more complex runs requiring many steps, where **we lack the simple input-output relation**?”
We discussed how the **subjectivity of good versus bad results** is one of the root causes of agents' evaluation struggles.
“There are two parts to evaluate. The objective part to evaluate is the binary form, for example, whether the agent did, or didn’t order a meal or booked a flight. The other and more tricky part is to evaluate how well the agent wrote a text or how quality research it did.”
## 4. Debugging
Cognosys has **its own system of the retrial of the agents’ steps** when it fails. It notifies the end user by saying that the instance failed, and they can run the agent again.
**They don’t share error details with end users**. “Giving them too many insights can get the non-technical users confused,” says Sully. However, users are mostly able to solve the problems themselves, by simply **reruning the agent**.
## 5. Latency
In traditional SW engineering, **around 200 milliseconds is considered slow**. For AI agents in general, latency is a big issue, with **LLM calls taking more than 30 seconds**. Cognosys agents usually run anywhere from 60 seconds to even 5 minutes sometimes.
“Currently, the agent uses [GPT-4](https://openai.com/gpt-4), which takes quite a long time to take action,” says Sully. “But people expect results quickly, and waiting even a minute until an agent provides the result makes them unsatisfied.”
## Conclusion
Sully realizes that the whole agents' space is still in the **early phase**.
“There is not that much functionality yet, so a big use-case at the beginning was just that the agent is fun to play with and try what it can do,” says Sully.
“But we want to continue focusing on a **few valuable specializations** for the agent. It’s very easy to want to do everything with the agent, but with the current models, it is impossible to do all these things well. And once the users get frustrated, they leave and never return.”
The Cognosys team is working on a **new version of their platform**. “We are excited about our next iteration that would solve some of the agents' issues, like latency”.
“Our plan for the future is to make the system more robust and easier to use, and have users more aware of capabilities.”
---
Source: https://e2b.dev/resources/david-zhang-from-aomni-gives-his-view-agents-reliability-debugging-and-orchestration.md
Published: 2023-08-23
# David Zhang from Aomni gives his view agents' reliability, debugging and orchestration
[**David Zhang**](https://twitter.com/dzhng) is the founder of [**Aomni**](https://www.aomni.com/) - one of not so many agents in the **business intelligence** category. We asked David about [Aomni](https://www.aomni.com/) users, the challenges he has been working on recently, and his view on the agents’ journey toward reliability.
### Intro
[Aomni](https://www.aomni.com/) is an AI agent that **crawls the web and ingests a vast amount of unstructured data**. It takes user's research goal, creates a research plan, completes it one by one, and sends user the result over email. This process takes 15-20 minutes to complete, and you can use it **3 times a day for free**.
## Users and Use Cases
The primary use of [Aomni](https://www.aomni.com/) is **sales market research and account planning**. It trains a personalized AI agent to handle account planning busywork, **saving sales reps 10+ hours every week** by researching their prospects and doing the legwork needed to help reps with outreach and relationship building, allowing reps to focus on building relationships and closing deals. Most [Aomni](https://www.aomni.com/) users are not in the AI field and are completely unaware of agent technology.
## Enterprise-level reliability
We discussed the readiness of agents for widespread adoption.
“One thing that I noticed recently is that **many builders are still not fully focusing on enterprise-level reliability**,” thinks David. “For enterprise customers, we are talking at least ~99.9% reliability. That’s what people expect and what agent developers need to aim for.”
David points out that the end users of Aomni (and many other agent-powered companies) are not in the AI field. They just see the product as any other software and put their **high-reliability standards** on it.
“There is a **tradeoff between generality versus reliability of the agent technology**. As a proxy, we can say that more general agent projects get less reliable. Take some of the popular [open-source agents](https://github.com/e2b-dev/awesome-ai-agents#open_hands-open-source-projects) as examples. You sure can ask a very multi-purpose agent to plan your wedding or to code for you. But it's going to **cost a lot of tokens** and these agents are not known as the perfectly reliable ones.” Usage - e.g. how expensive the agent gets - is another important variable to include in this tradeoff.
David predicts that new agents coming to the market in the next 1-2 years will be **very specific in certain niches** and that’s why he focuses on **narrowing down the focus of** [**Aomni**](https://www.aomni.com/).
“I **limit the number of tools** that [Aomni](https://www.aomni.com/) uses,” David adds, “and the number of decisions the agent needs to make to reach the final result.”
## Solving agents’ challenges
We asked David about [SDKs and tools](https://github.com/e2b-dev/awesome-sdks-for-ai-agents) he has been using for overcoming challenges, such as debugging, monitoring, and orchestration.
“Many agents are workflow orchestration type of systems,” says David. He then shared the agent’s task queue and explained how he uses [Inngest](https://www.inngest.com/) for orchestration and debugging.
On the left of the image below, we can see all the agent runs, both successful and unsuccessful. On the right, there are all different steps that the agent took in a certain run, including reasons for failures that have happened.
However, David points out the **limitations of** [**Inngest**](https://www.inngest.com/). “It doesn't let you get in the middle of the process and **retry the agent from a certain step**. Such tasks are very agents-specific.”
David’s take on the agents' debugging is that he is noticing a lot of agent developers **trying to reinvent the wheel with new frameworks and SDKs**. Instead, they should build on top of existing technology.
“However, there is still a space for innovation, for example for having one **platform for monitoring token usage and measuring prompt effectiveness**.”
David enhances the necessity of **time-travel debugging**, e.g., resuming the execution of an agent in a certain step.
“It is a pretty common concept in SW engineering, so for me, it is a matter of time before someone adds the agent logic on the top of one of the existing engines,” David predicts.
## Communication with users
David has been very busy communicating with users on the [Aomni discord](https://discord.com/invite/a367ncqEsm/?utm_source=awesome-ai-agents). When an agent fails, the user just sees a message, with an option to try another query and to share the error message.
“**The Aomni agents are relatively reliable** and don’t fail that much, so I didn’t need anything complicated for the error messages,” says David. “I am in the middle of **adding a** [**PostHog**](https://posthog.com/?utm_source=google&utm_medium=cpc&utm_campaign=Brand-Campaign-OtherEU&utm_content=pure-brand&gclid=Cj0KCQjw3JanBhCPARIsAJpXTx4sK2SHsgHRJl0MlLqpnzAY1UQnZUGYy0r7OYam-8B3bFoKgoyBCo8aAuMyEALw_wcB) **integration** for product analytics.”
In David’s opinion, agent developers need to find a good way to **collect human feedback** and sort and manage examples of agents’ use cases.
“Creating something like a repository of agents’ use cases with good examples could be very useful.”
Try the Aomni agent [here](https://www.aomni.com/?auth=signin), and join their [Discord community](https://discord.com/invite/a367ncqEsm/?utm_source=awesome-ai-agents) to stay updated.
---
Source: https://e2b.dev/resources/discussing-agents-challenges-with-ismail-pelaseyed-the-founder-of-superagent.md
Published: 2023-08-22
# Discussing agents' tracing, observability, and debugging with Ismail Pelaseyed, the founder of Superagent
We talked with [Ismail Pelaseyed](https://twitter.com/pelaseyed) about AI agents, their current challenges, and potential solutions.
Ismail is the founder of [Superagent](https://www.superagent.sh/) - **a framework for web or no-code developers**. Being active for around two months, Superagent already gained **around 2000 users and over 1500 agents running in the production every da**y. It is a completely [open-source](https://github.com/homanp/superagent) platform, with a cloud version free to use.
## Users and Use Cases
Superagent tries to approach **no-code developers** to **integrate agents into their applications**.
They have two main types of agents built with Superagent:
1\. **Chatbots** - **Retrieval types of agents**, used to Q&A over structured or unstructured documents.
2\. **Action-based assistant** - Run either through a **chat interface** or just through an API.
## Architecture and Techstack
[Superagent](https://www.superagent.sh/) has a **database layer on top of the agents' infrastructure**. When a user creates an agent, they create an entry into that database with a configuration of how an agent should run. They have an **API service hosted on** [**render.com**](http://render.com/) which picks the configuration depending on the API call, creates the agent on the fly, and runs it.
They are currently using [Langchain](https://github.com/e2b-dev/awesome-sdks-for-ai-agents/tree/main#langchain), which has a very **prompt-based agent infrastructure**, to call various LLMs and [LlamaIndex](https://github.com/e2b-dev/awesome-sdks-for-ai-agents/tree/main#llamaindex) to ingest various data.
They are deep into replacing currently used frameworks, moving away from the core Langchain Agent-LLM loop (e.g. [ReAct](https://arxiv.org/abs/2210.03629), [Chain of Thought](https://www.promptingguide.ai/techniques/cot)).
The [Superagent](https://www.superagent.sh/) team has a **wrapper around some individual types of agents** which they have tested and are **ready to be run in production**. These agents can then be easily configured without the need to consider various components that are involved in running agents, mainly:
- **Memory**
- **Vectorizations**
- **The agent's acting and whole logic**.
## Current Challenges
The **agent architecture** **is still in its early stages**, making configuration for specific use cases challenging.
“*Many people attempt to create agents but often give up due to its inability to meet their desired functionality*, " says Ismail. “*Our goal is to develop agents that are not only demo-worthy but also practical and useful for business purposes*.”
Ismail has been noticing the **many and many specific ways** in which people want to use their agents **for the same use case**. “*People really do have specific ways of working on the same thing, which therefore requires deep developers’ understanding*.”
Creating efficient agents that work effectively in production for **specific use cases** demands significant expertise at this stage. Especially with high pressure being placed on ensuring user-friendliness for no-code users.
With that come the challenges of **debugging, tracing, monitoring, and observability** of the agents.
## Debugging the Agents
The current process of solving a bug is very simple. If an agent instance fails, end users go to the [Superagent Discord](https://discord.com/invite/mhmJUTjW4b). There they describe the bug, for example, that a retrieval agent did not use a document to make a query, but hallucinated instead.
“*It is not the actual software that is failing, it is* ***mostly the setup that the user has***. *It can be prompt, it can be the document being ingested, or anything else. That is what we are trying to debug*.”
Ismail emphasizes looking for a powerful debugging solution that would be **user-centric**. It is not the developer team, but the end users who are not currently able to debug the agent themselves.
## Tracing and Observability
**The tracing currently lacks WHY and HOW**. That is, the user sees that an agent instance failed, but doesn't understand WHY exactly it happened and HOW to fix the problem. This problem can be solved on an individual level but will grow big when Superagent gains even more regular users and bigger customers.
## Agents's tools and SDKs
To overcome the challenges of tracing and debugging, the Superagent team can envision themselves using an **equivalent of** [**Sentry**](https://sentry.io/welcome/) (a software error tracking and monitoring platform), which could provide **insights into the origin of a bug**. They have tried [**Langsmith**](https://github.com/e2b-dev/awesome-sdks-for-ai-agents/tree/main#langsmith) (a web-based GUI to test and monitor calls from LLM apps), which, however, won't give the root cause of the bug and possible solutions.
“*We need a tool that would* ***also tell a user what they should do with the bug***,” says Ismail.
The [Superagent](https://www.superagent.sh/) team has a UI where end users can see agents’ runs. It is very similar to Langsmith or any other type of logging framework.
When a user faces a problem, the [Superagent](https://www.superagent.sh/) team can log into the system, see what the agent tries to do in each step, and then **visualize the log**. This is in **principle similar to regular server logs visualizations**.
## Conclusion
Right now the [Superagent](https://www.superagent.sh/) team focuses on **re-doing their whole infrastructure**, aiming to **run the agents in a separate queue**, and creating a queue system, instead of running them on the fly.
“*We would go for big* ***changes in our infrastructure*** *if it solves the discussed problems*,” says Ismail.
Without a doubt, in the very young agents space ([AutoGPT](https://news.agpt.co/), the first well-known agent, launched only in March 2023), [Superagent](https://www.superagent.sh/) and other agents are steadily gaining traction among end users. Despite the current drawbacks, this shows the **real demand for agents**.
If you want to try [Superagent](https://www.superagent.sh/) yourself, check the [web](https://www.superagent.sh/), [GitHub](https://github.com/homanp/superagent), or join the [Discord community](https://www.superagent.sh/).
---
Source: https://e2b.dev/resources/agent-protocol-developers-community-setting-a-new-standard.md
Published: 2023-08-07
# Agent Protocol: Developers community setting a new standard
The [agent space](https://github.com/e2b-dev/awesome-ai-agents) is evolving rapidly, with new agents launching every week. They differ in use cases, e.g. coding assistance ([Sweep](https://sweep.dev/), [Clippy,](https://github.com/ennucore/clippy/) [yAgents](https://github.com/yeagerai/yeagerai-agent)), business intelligence ([Aomni](https://www.aomni.com/)), research ([GPT Researche](https://github.com/assafelovic/gpt-researcher)r), and more. But more importantly, they differ in their architecture and approach toward building the agents.
It is clear that a standardized protocol is essential and has recently been a big subject of [community discussions](https://www.blog.e2b.dev/log/discussed-agent-protocol-with-community). By establishing a common framework, we aim to help the agent [ecosystem](https://www.blog.e2b.dev/log/ai-agents-in-the-wild) grow faster and simplify the work. The need for a standard becomes even more crucial within applications.
Our approach towards the Agent Protocol is starting with a minimal core and iteratively building on it. We are using [guidance from agent developers](https://www.blog.e2b.dev/log/discussed-agent-protocol-with-community), trying to reflect their needs and lead the Protocol as an open-source community effort towards the gold standard in the agents field.
The development and invention of the Agent Protocol was significantly influenced and spearheaded by E2B's founding engineer - [Jakub Novak](https://www.linkedin.com/in/novak-kuba/). He has served as the lead inventor and code contributor of the project.
## What is The Agent Protocol
The Agent Protocol itself is the most important part. It is defined as an OpenAPI specification of endpoints.
### Key Components
1. Protocol Specification - Central to the Agent Protocol is the OpenAPI specification - a list of endpoints that the agent should expose. Picture it as something that wraps your agent in a web server that allows for communication with your agent (and in between agents in the future).
2. SDK - The SDK wraps your agent within a web server. In enables communication with your agent, and potential inter-agent interactions in the future. The wrap is created to make setting up the protocol super simple and user-friendly, while also making sure there are no limitations.
3. Client SDK
The Client SDK allows end users to interact with the agent easily. Thanks to the standard the users can try multiple agents without the need for any additional adjustments (or very minimal) in their code.
## Overcoming agents' challenges
The incentive of adopting the Protocol lies in overcoming current agents’ problems, that are being repeatedly described by their developers. The most common struggles have been the development and debugging of the agent (both in the building phase and production), monitoring, evals, tracing, observability, and deployment of agents.
The Protocol brings some immediate benefits for solving current challenges.
### Building dev tools for better agents reliability
The agents have a long way to go before agents' reliability starts approaching at least 90%. They have to improve debugging agent prompts, tracing agents’ fails, monitoring the logs, or rerunning certain steps of an agent instance. Not talking about communicating bugs and root causes of problems to end users.
So far, developers have been working on in-house tools to solve these problems, as well as trying tooling available on the market, but there is no efficient way to make agents enterprise-ready.
On top of the Protocol, developers can build general dev tools for the development, testing, and debugging of agents. They won’t need to write boilerplate API and you can focus on developing their agent.
Moreover, if one person (for example your team member) has started using the protocol, other people should be able to use and integrate other agents more easily, without any additional implementation.
### Benchmarking
Setting a general simple standard that would allow for easy-to-use benchmarking of agents. Another immediate benefit that adopting the Agent Protocol brings is the ease with which you can use the benchmarks.
### Great developer experience
In general, one of the primary goals of the protocol is a great developer experience and simple implementation on the end of agent developers. You just start your agent and that’s all you have to do.
*Listen to a* [*recording from the Twitter Space*](https://www.blog.e2b.dev/log/discussed-agent-protocol-with-community) *where founders and builders gave their view on the future approach towards agents and discussed the Agent Protocol*.
## Your Input Matters!
At E2B, the community is at the core of our mission. You can find all information about installation and usage in [the official Agent Protocol repo](https://github.com/e2b-dev/agent-protocol).
If you're missing a feature or have a great idea for improvement, or just want to discuss the protocol, don't hesitate to reach out.
Open an issue or reach us at [hello@e2b.dev](mailto:hello@e2b.dev). You can also join AutoGPT discord or E2B discord.
---
Source: https://e2b.dev/resources/chatgpt-plugin-by-e2b.md
Published: 2023-08-01
# ChatGPT Plugin by E2B
The E2B plugin for ChatGPT is like [the official code interpreter plugin](https://openai.com/blog/chatgpt-plugins#code-interpreter), but on steroids.
It gives your ChatGPT an access to a full cloud environment that is sandboxed and running on E2B infrastructure. That means:
- Access to Linux OS
- Using filesystem (create, list, and delete files and dirs)
- Run any terminal command
- Access to the internet
- Run any programming language (out of the box we support Nodejs, Go, Bash, Rust, Python3, PHP, Java, Perl, .NET)
We’ve built these cloud environments to be used specifically by AI Agents. With the E2B plugin, your ChatGPT becomes like an AI agent that’s your personal software developer.
The plugin is [open source](https://github.com/e2b-dev/chatgpt-plugin) and we’re looking for beta testers and contributors. We’ll be releasing our plugin to the ChatGPT store soon.
### [Test the plugin here](https://github.com/e2b-dev/chatgpt-plugin)
Try out the E2B ChatGPT Plugin [here](https://github.com/e2b-dev/chatgpt-plugin) and tell us how you liked it via [Discord](https://discord.gg/35NF4Y8WSE) or by tagging [@e2b\_dev](https://twitter.com/e2b_dev) on twitter. Your feedback is very welcome.
---
Source: https://e2b.dev/resources/discussed-agent-protocol-with-community.md
Published: 2023-07-25
# We discussed the Agent Protocol with the community
We organized a Twitter space with hundreds of people attending in total.
Together with 26 speakers, we discussed current challenges faced by agents, e.g. their deployment, debugging in real time or tooling. Founders and builders gave their view on the future approach towards agents and we talked about the [Agent Protocol](https://github.com/e2b-dev/agent-protocol).
Listen [here](https://soundcloud.com/e2b-443059138/twitter-space-the-agent-protocol) and enjoy.
Thanks everyone for joining, for example:
- Div Garg - [@DivGarg9](https://twitter.com/DivGarg9), founder of [Multion AI](https://www.multion.ai/) ([@MultiON\_AI](https://twitter.com/MultiON_AI))
- Ismail Pelaseyed - [@pelaseyed](https://twitter.com/pelaseyed), founder of [Superagent](https://www.superagent.sh/)
- Ed Shrager - [@edshrag](https://twitter.com/edshrag), founder of Summari ([@getsummari](https://twitter.com/getsummari))
- Yohei Nakajima - [@yoheinakajima](https://twitter.com/yoheinakajima), creator of [BabyAGI](https://github.com/yoheinakajima/babyagi)
- Merwane Hamadi - [@merwanehamadi](https://twitter.com/merwanehamadi), [AutoGPT](https://news.agpt.co/) ([@Auto\_GPT](https://twitter.com/Auto_GPT)) contributor
- Sully Omarr - [@SullyOmarr](https://twitter.com/SullyOmarr), building Cognosys ([@Cognosysai](https://twitter.com/Cognosysai))
- Brennan Woodruff - [@BrennanWoodruff](https://twitter.com/BrennanWoodruff), Co-Founder of GoCharlie ([@Gocharlieai](https://twitter.com/Gocharlieai))
- Owen Colegrove - [@ocolegro](https://twitter.com/ocolegro), building [Automata](https://github.com/emrgnt-cmplxty/automata)
- David Zhang - [@dzhng](https://twitter.com/dzhng), founder of [Aomni](https://www.aomni.com/) ([@aomniapp](https://twitter.com/aomniapp))
- Alex Reibman - [@AlexReibman](https://twitter.com/AlexReibman), founder of [AgentOps](https://www.agentops.ai/)
- Kevin Lu - [@KevinLu45010771](https://twitter.com/KevinLu45010771), co-founder of [Sweep AI](https://github.com/sweepai) ([@sweep\_\_ai](https://twitter.com/sweep__ai))
- Lev Chizhov - [@ennucore](https://twitter.com/ennucore), building [Clippy](https://github.com/ennucore/clippy/)
- Justin Torre - [@justinstorre](https://twitter.com/justinstorre), buidling [Helicone](https://www.helicone.ai/)
---
Source: https://e2b.dev/resources/we-gave-ai-agents-a-cloud-playground.md
Published: 2023-06-29
# We gave AI Agents a cloud playground
Today, we launched an AI agent deployed on E2B. Meet "smol developer" - your very own personal junior developer, and at the same time a powerful AI agent. You can start working with smol developer within a few seconds.
#### How does it work?
You can follow the following three steps.
Once you sign-in via your GitHub and assign a (new or existing) repository, it becomes the center of the smol developer’s work. You can tell the smol what to do, collaborate with them, add comments to their pull requests, to which they will react.
The agent is a generalist and you can assign them any of your repositories, or you can create a new one and build anything. You are offered a few templates for making apps, a website, web scraper or chrome extension.
Compared to running locally or via a framework, in E2B, the agent has their own environment in the cloud - they run via their own firecracker VMs.
Share your experience with us.
---
Source: https://e2b.dev/resources/ai-agents-in-the-wild.md
Published: 2023-06-16
# AI agents in the wild
They are all over here and more of them are coming soon.
In the last months, AI agents have gained huge popularity - [AutoGPT](https://github.com/Significant-Gravitas/Auto-GPT) has reached 140,000 stars on GitHub and we’re seeing new companies being started on an almost weekly basis. Ranging from agents reviewing code ([Sweep A](https://sweep.dev/)I) to virtual personal assistants like [Lindy](https://www.lindy.ai/).
We mapped the AI agents landscape and explored over 50 agents, which we put together into a [list](https://github.com/e2b-dev/awesome-ai-agents). Here are our conclusions about the current state of the ecosystem.
### What is an AI autonomous agent?
Developers and founders have been eager to contribute to the agents’ list with various agent-ish projects. We have to ask the prerequisite question though. How do you define an autonomous AI agent? How would one differentiate between an AI agent and LLM-based chatbot, for example?
Agents use both short-term memory (in-context learning) and long-term memory (retrieval of information from an external vector store), have the capability to plan, break down objectives into smaller tasks by “thinking” step-by-step, and reflect on their own performance. Moreover, a lot of agents can use tools, such as scraping websites, using Google Calendar, or running commands in the developer’s terminal. Implementing tool usage has been made easier for developers by the recent [OpenAI's explicit support for functions](https://openai.com/blog/function-calling-and-other-api-updates).
We aimed to include projects and companies based on this empirical definition.
### Five key learnings
1. Among 56 projects, we observed an approximately 50:50 ratio of open source vs closed source projects.
2. The closed source projects are generally better at communicating use cases for their agent.
3. The most popular use case is coding assistants, followed by productivity assistants, and general purpose agents.
4. There’s a high number of no-code/low-code platforms for building custom agents. Users of these platforms must be somewhat familiar with technicalities of LLMs to be able to use them efficiently.
5. Developers of agents most often struggle with testing/evaluating, debugging, monitoring, and identifying at what step their agent broke and why.
### What's next
The hype around AI agents is strong, however, as much as AI agents are exciting and promising there’s still a lot of challenges, especially around their reliability. We think AI agents are on a path to become the future of software and become common the same way for example websites are common nowadays. However, to get there, there needs to be first better tooling for developers. Especially tooling that will help developers make their agents more reliable and give developers bigger confidence their agents are working as intended.
---
Source: https://e2b.dev/customers/lark-case-study.md
Published: 2026-09-17
# How Lark Uses E2B to Safely Test Apps with Customer Data
Lark solves a new type of problem in the industry: AI coding agents write code faster than anyone can verify it works. Lark is an AI test engineer. It maps out a customer's product and workflows, writes a full suite of end-to-end tests, and keeps running them as the product changes. Customers run hundreds of these tests at once, and the load is spiky by nature: when a wave of pull requests merge at the same time, test volume spikes with it. Each test runs in its own sandbox against the customer's application, sometimes even with real customer data and credentials in play.
That latter part is what makes the infrastructure hard. Lark is running agents against customer systems, and the isolation between test runs has to hold every single time.
## Testing Customer Environments
Lark's agents generate and execute test cases. This means launching browsers, calling APIs, running scripts, and validating actual user flows, then producing artifacts like screenshots and logs attached to tests. Early versions of the product ran this workflow on local dev machines and traditional CI runners. That worked until Lark's agents needed to run arbitrary workflows autonomously, at which point the team needed a secure environment to execute the code in.
## Isolation Is Essential on Sensitive Data
Lark has to work within any customer's system — in live environments with user data and production credentials in reach. That raises the stakes on isolation. Every customer test needed to run in its own ephemeral environment, spin up clean, execute, and disappear, with zero chance of one run seeing another's data.
## Docker Support Narrowed the Field Fast
Lark's tests sometimes need to spin up a customer's own development environment inside the sandbox to validate against it, which requires them to run Docker containers within the sandbox itself. That requirement ruled out most providers before the evaluation really started: most sandbox providers isolate agents using containers, and running Docker inside a container usually means either privileged mode, which breaks isolation, or no way to run it at all, since the sandbox doesn't expose the kernel features Docker needs to function.
## Custom Templates Ready for an Agent to Test
A developer describes what they want tested in natural language, and Lark turns the request into an execution plan. It then spins up an isolated E2B sandbox from a custom template with the full testing environment already baked in, including browsers, testing frameworks, Python, and other language runtimes. Because dependencies do not need to be installed for every run, tests can start quickly and results return without disrupting the user experience.
To keep work and results flowing between its platform and each sandbox, Lark built an open-source project, runtimeuse. It handles bidirectional communication, pushing tests into the sandbox and streaming results back to Lark in real time so users can watch them execute. Once the test is complete, the sandbox is destroyed.
## Built for Running Tests Across the Whole Codebase
Lark is designed for many test executions running at once, particularly around CI pipelines, pull requests, and large batch validation jobs. That load spikes hard during active development hours and around large releases, which are the moments when a team most needs its tests to run. Having E2B handle sandbox provisioning meant Lark didn't have to build its own capacity planning for the burst traffic, letting them spin up thousands of concurrent sandboxes seamlessly.
## Docker-in-Sandbox, Without Building It Themselves
Running Docker containers inside sandboxes limits the field to a handful of providers, and E2B's MicroVM isolation meant Lark didn't have to build and operate that isolation layer on its own, saving them at least several weeks of building out their own sandbox infrastructure.
## Looking Ahead
As AI coding agents write more code, more of it will need to be continuously tested. Lark expects both its customer base and sandbox usage to grow, including among larger organizations with stringent data and security requirements. As it scales, the team plans to continue relying on E2B as its tried-and-true infrastructure for testing agents.
---
Source: https://e2b.dev/customers/paperinstruments.md
Published: 2026-09-03
# How Paper Instruments uses E2B to cut idle GPU time for RL rollouts
Paper Instruments trains frontier models to do knowledge work: consulting, finance, investment banking, and law. They partnered with Perplexity to develop [DRACO](https://research.perplexity.ai/articles/evaluating-deep-research-performance-in-the-wild-with-the-draco-benchmark), a benchmark for its flagship Advanced Deep Research product. Most recently, they released DiligenceBench with Thoughtful Lab, which is a benchmark that grades agents against equity-research tasks with rubrics built from thousands of weighted criteria.
Training frontier models against these benchmarks requires infrastructure that can scale. In practice, that means running thousands of concurrent rollouts, each one with an agent operating on its own filesystem and external application state over many steps. These rollouts need to be isolated from each other to keep training signals clean. Additionally, the data generated by rollouts (the agent's actions, file changes, intermediate outputs, rewards/grades from the benchmark rubrics) is wired directly into the pipeline in a structured and traceable way so that there is visibility into how data actually influences the models. That visibility starts with the sandbox each rollout runs in.
> "Our core IP centers on plumbing data generation through training so that we have visibility into how each task and its provenance actually influences the models we train. On the infrastructure side, running RL with a policy gradient method like GRPO or CISPO means orchestrating thousands of concurrent rollouts at once."
\- Joey Besgen, Co-Founder, Paper Instruments
## The challenge
### Isolating thousands of RL rollouts
Each RL rollout needs its own file system and persistent state for the length of the trajectory, and it’s important that the solver and the grader are completely isolated from one another to prevent reward hacking. Additionally, parallel rollouts running in parallel must not affect each other. Cross-contamination between rollouts, or exposure to systems outside the sandbox, can corrupt the reward signal the model is training against.
### Latency is a GPU bill
RL training has an inherently spiky workload: hundreds of sandboxes spin up together, run agent actions and tool calls against the file system/external state, then grade the results and tear down. Every second a rollout spends waiting on a slow command or a slow cold start impacts GPU utilization, and long waits for sandbox actions accumulate to drive up training cost directly.
### Build vs. buy on Firecracker
Paper Instruments evaluated all of the top sandbox providers, and E2B’s command execution latency and resilience to production spikes proved to be the best for their workflows.
The other alternative to a managed sandbox layer was building one, but standing up and operating that orchestration layer in-house would have pulled engineering time away from the model and data work that gives Paper Instruments its edge.
## The solution
Paper Instruments runs its RL rollouts inside E2B sandboxes, giving every concurrent trajectory its own isolated file system and its own state for the length of the run.
### Cutting cold-start and command latency
In Paper Instruments' own benchmarking, E2B's tool execution came in up to 3 times faster than other sandbox providers they evaluated. Faster startup and faster command execution both translate directly into less idle GPU time and lower training cost.
### Driving faster startup with snapshots
In RL training, every rollout needs to start from the exact same "world": the file system, running services, and any external state the agent operates on, so that differences in outcomes reflect the policy being trained and not inconsistencies in the starting environment. Paper Instruments starts with an E2B sandbox [template](https://docs.e2b.dev/template/quickstart), initializes each environment and any required services, then [snapshots](https://docs.e2b.dev/sandbox/snapshots#sandbox-snapshots) the running sandbox. Rollouts can then spin up directly from the appropriate snapshot, restoring the captured file system and memory state instead of repeating the setup process. This lets hundreds of concurrent rollouts launch from consistent, pre-initialized environments while minimizing idle GPU time.
> "Another thing E2B is great at is that you can snapshot both the file system and the memory. That drives faster startup, because we can snapshot running servers inside the sandbox. When a job launches hundreds of rollouts against the same starting state, that snapshot is what eliminates most of the cold-start work for every rollout.”
\- Joey Besgen, Co-Founder, Paper Instruments
### Developer experience for humans and agents
Besgen points to onboarding as a differentiator that showed up twice: once for his own team reading the docs, and once for the agents that also read them. That meant they could go from zero to one a lot faster. Spending less time on perfecting sandbox infrastructure allowed the team to focus more efforts on improving training data and the training recipe, which meant they could iterate on training results in a much shorter timeframe.
> "E2B provides a very good developer experience. They make it easy to spin up sandboxes, easy to snapshot the running state so we can resume quickly for later training runs, and gives us good intelligence into what's happening inside those sandboxes so we can debug during training."
\- Joey Besgen, Co-Founder, Paper Instruments
## Looking ahead
Paper Instruments' bet is that synthetic data, sampled from existing policies and pushed slightly out of distribution, can drive high Pareto performance. That bet depends on being able to run large sweeps of RL experiments cheaply and repeatedly, which is what E2B helps solve today. As Paper Instruments scales up training and evaluation for its next models, the volume of concurrent runs is only set to multiply on E2B.
---
Source: https://e2b.dev/customers/replicas.md
Published: 2026-08-27
# How Replicas Cut Customer Latency 60% with E2B
Replicas runs coding agents in the cloud for engineering teams. A developer can ping the agent from Slack, assign it a Linear ticket, or tag it on a GitHub pull request. Replicas then starts an agent that clones the repository, installs dependencies, runs the real stack, and returns with a pull request.
The premise sounds simple: give an agent a sandbox and let it write code. In practice, that agent needs to run builds, boot databases, start development servers, and use the same tools an engineer runs locally. The environment underneath it cannot be only a place to execute code. It has to behave like a full computer.
## The challenge
### Agents need a complete development environment
Customers expect the agent to work like an engineer: check out a branch, run the build, start services, open the application, and verify that a change works before submitting it. That workload needs memory, disk, nested services, and an environment that stays available for the duration of the task.
> “Agents need the same environment engineers have - a full computer. Those are big: a lot of memory, a lot of disk. Spinning up, running, and winding down a lot of large machines reliably matters a lot in use cases like ours.”
\- Connor Loi, Founder, Replicas
### Docker-based sandboxes slowed down the workload
Before E2B, Replicas used a Docker-based sandbox. That became a bottleneck because a common Replicas workflow is to run Docker workloads inside the agent environment. Nesting those services on top of an already containerized sandbox made customer workloads slower and heavier.
### Reliability became urgent during fundraising
The sandbox reliability problem peaked near the end of Replicas' Y Combinator batch, just as the team was preparing to fundraise. Building the infrastructure internally was not realistic. Replicas evaluated raw AWS EC2 instances and five sandbox providers; E2B was the fastest and most reliable option they tested.
> “I originally started with a competitor, but the search for something stable kept dragging on right as the team was heading into the end of its YC batch and preparing to fundraise. The sandbox infrastructure was basically the last thing I wanted to think about.”
\- Connor Loi, Founder, Replicas
## The solution
### Warm environments turn a ticket into a running agent
Replicas captures each customer's configured workspace as an E2B [Snapshot](https://docs.e2b.dev/sandbox/snapshots), including the cloned repository, installed dependencies, and development setup. When somebody pings Replicas in Slack or assigns it a Linear ticket, Replicas starts a new sandbox from that snapshot. The agent inherits the prepared environment and can begin work without rebuilding the customer's setup.
### MicroVM isolation with room to grow
Replicas compared providers on more than feature checklists. It needed infrastructure the team could trust years into the future. E2B's MicroVM isolation, snapshots, open-source architecture, and responsive support gave Replicas a stable foundation for long-running coding agents.
> “We wanted an offering that balanced a focus on open source with stability and fast support, and that was E2B.”
\- Connor Loi, Founder, Replicas
### It worked on the first try
The difference appeared immediately after the switch. E2B was the only provider that worked on the first try without additional flakiness to diagnose. Resolving the sandbox problem during the most important week of the YC batch let the team focus on its product and customers instead of infrastructure.
## The results
- **More than 60% lower customer latency** for heavy Replicas workloads after moving to E2B.
- **Three weeks saved** on sandbox debugging during the final stretch of YC and fundraising.
- **More than 20 YC companies** now run engineering work through Replicas.
> “Reliability is the only thing that matters, and lack of reliability was the main reason for churn at Replicas. You want to pick infrastructure that's never going to let you down, so you have time to work on the thing that matters most: the agent experience.”
\- Connor Loi, Founder, Replicas
## Looking ahead
Replicas is building toward fleets of agents that run engineering workflows end to end: triaging tickets, writing code, responding to CI failures and review comments, and merging changes without anyone in the room. As Replicas serves larger engineering teams and enterprise contracts, it is leaning further into E2B's MicroVM isolation and flexible deployment options so every agent receives an environment engineers can trust.
---
Source: https://e2b.dev/customers/effectiveai.md
Published: 2026-08-05
# How Effective AI Runs Secure, Multi-Agent Insurance Workflows at Scale
Effective AI helps insurance organizations collectively writing more than $180 billion in direct written premiums launch and maintain insurance products. Its platform uses specialized agents for market benchmarking, rate-plan design, actuarial analysis, regulatory filings, and compliance workflows.
As those agents moved beyond document analysis and began producing the actual models, spreadsheets, and files insurance teams rely on, Effective AI needed a secure execution layer that could support complex, customer-specific workflows at enterprise scale. With E2B, the company supports more than 10,000 user sessions each month.
> “E2B gives Effective AI a secure execution layer at large scale, turning complex filing, pricing, and compliance work into faster product launches.”
\- Arijit Banerjee, Co-Founder, Effective AI
## The challenge: moving from insurance intelligence to finished work
### Turning analysis into usable outputs
Early versions of Effective AI helped insurance teams reason over filings, manuals, and regulatory documents. As customers brought more of the insurance product lifecycle onto the platform, they needed agents that could produce the artifacts required to act on those decisions.
That meant building actuarial models, generating spreadsheets, constructing rating engines, transforming files, and running customer-defined workflows. Effective AI needed an environment where agents could work with files and dependencies, connect to approved systems, and maintain state across complex workflows.
### Deploying AI at scale in a highly-regulated industry
Effective serves insurance organizations with strict compliance requirements. Workflows may handle sensitive policyholder information, proprietary pricing models, claims histories, and loss and exposure data. Each agent workload must therefore run in isolation and receive only the files, credentials, tools, and systems it is authorized to access.
### Scaling for variable, multi-agent workloads
Effective AI’s workloads vary dramatically. A table-extraction task may finish in seconds, while a more complicated workflow like building a rating plan, comparing competitor filings, or running a portfolio analysis, may take up to 16 hours. Individual rater-building workflows may even require hundreds of agents running across tens of sandboxes, making low-latency handoffs between agents essential. Other use cases include long-lived sessions that wake up daily or weekly to perform recurring tasks, such as monitoring for new filings that match specific criteria.
> “Startup time matters because our workflows are not one agent doing one task. A single customer workflow can fan out across hundreds of agents, and even small delays compound quickly.”
\- Arijit Banerjee, Co-Founder, Effective AI
### Staying focused on insurance
Building an internal sandbox platform would have required Effective AI to solve isolation, provisioning, lifecycle management, observability, and scaling, pulling engineers away from building the features that differentiate the product.
## The solution: a secure execution layer built on E2B
Effective built its execution layer on E2B from the start. Clear documentation helped the team get up and running quickly, while fast sandbox startup times and the option to self-host gave Effective additional confidence in E2B.
> "We got started and never looked back."
\- Arijit Banerjee, Co-Founder, Effective AI
### From a request to an executable workflow
A user submits a request in natural language on Effective - say, build a rating plan, compare competitor filings, or run a portfolio analysis. Their orchestration layer then decomposes it into specialized tasks handled by parallel agents: filing research, table extraction, rater construction, and actuarial calculations.
Each session gets an E2B sandbox booted from a pre-warmed execution environment. Effective uses E2B [templates](https://docs.e2b.dev/template/quickstart) to define a base execution environment with pre-installed libraries and common insurance agent dependencies. A periodic build job layers each customer’s specific tools, code, and context onto that base template, then saves the configured environment as a [snapshot](https://docs.e2b.dev/sandbox/snapshots). New sessions boot from the customer’s snapshot and retrieve only the code changes made since the snapshot was created.
Agents then execute code inside the sandbox and coordinate through Effective’s custom [multi-agent runtime](https://effectiveailabs.com/blog/multi-agent-runtime). When agents need to work from the same files, dependencies, or in-progress outputs, they share a sandbox; otherwise, the runtime uses a Jupyter PreforkProvisioner to launch isolated agent kernels from a preloaded daemon with copy-on-write memory sharing, reducing per-notebook overhead and allowing hundreds of agentic functions to run efficiently on the same machine.
E2B’s [pause-and-resume](https://docs.e2b.dev/sandbox/persistence) feature lets Effective preserve a workflow’s state without keeping its sandbox running continuously, supporting the longer-running and recurring sessions.
> “A generic sandbox is useful, but our customers need agents that work inside their world. Templates let us start each workflow with the customer’s tools, dependencies, and context already in place.”
\- Arijit Banerjee, Co-Founder, Effective AI
### Extending security controls into agent execution
Each workload runs in its own hardware-isolated E2B MicroVM, creating a strong boundary between tenants and preventing customer code, data, and agent activity from crossing into another customer’s environment. Sandboxes are treated as ephemeral execution environments: only the files and data needed for a workflow are loaded into the sandbox, and the entire environment is destroyed when the workflow is complete, with persistent files stored in Effective’s platform.
### Keeping engineers focused on the product
By building on E2B, Effective avoided creating and operating its own sandbox infrastructure, saving the equivalent of one to two full-time engineering hires.
Instead, the team can focus on embedding insurance expertise into the platform and improving how customers research markets, develop rating plans, validate filings, and operationalize their underwriting and actuarial workflows.
## The results
Today, Effective AI:
- **Scales beyond 1,000 concurrent E2B sandboxes** supporting complex workflows across hundreds of agents.
- **Gracefully handles spiky workloads** from workflows that wake up daily or weekly
- **Saved an estimated 14% of engineering capacity** who would otherwise have been dedicated to building and operating a secure sandboxing platform.
## Looking ahead
Effective AI is expanding into even more specialized workflows across underwriting, forms generation, generalized linear modeling, regulatory filing preparation, and submission support.
As its agents take on more of the technical work behind insurance product development, E2B provides the secure compute foundation needed to turn customer data, code, and institutional knowledge into finished work that insurance teams can review and use that will grow with Effective AI’s platform.
---
Source: https://e2b.dev/customers/anything.md
Published: 2026-07-20
# How Anything Is Building a Vibe-Coding Empire with 1M+ Users
Over one million people use Anything to build apps without writing code. A user describes what they want to build, and the agent creates everything - from the frontend and backend to deployment and App Store submission.
What looks like a simple conversation hides a demanding AI workload in production.
Getting from a prompt to a working app needs to feel immediate and magical for users. Behind this is a complete development environment that has to be provisioned, ready in seconds so the experience feels as immediate as chatting with an LLM.
## The challenge: making every execution environment feel like a real developer workstation.
### Agents need real tools
Generating code is easy. Getting it to execute like it would on a real developer machine is the hard part. Installing dependencies, starting development servers, compiling native mobile apps, launching browsers, and debugging code. The challenge is giving the agent an environment that feels like a developer's own computer, not just somewhere to execute code.
That means providing a complete development environment with a filesystem, native build toolchains, and browsers, so the agent can build, run, and test applications the same way a human developer would.
> If the execution environment can't support one of those workflows, the coding agent gets stuck, the user gets frustrated, and ultimately, they can't get the job done
\- Marcus Lowe, CEO, Anything
### Scaling for viral growth
Additionally, Anything's workload is unpredictable. A surge of users, whether from a product launch or a viral social media moment, can require thousands of sandboxes to spin up almost instantly.
The infrastructure has to scale just as quickly, provisioning thousands of isolated execution environments on demand so users can start building immediately without introducing noticeable latency, which directly affects the user experience.
> “Our business is extremely high-volume... Oftentimes, the workload is quite spiky as well, so we need to be able to spin up thousands of sandboxes on a moment's notice if we happen to be going viral on social media.
\- Marcus Lowe, CEO, Anything
### The build vs buy decision
Anything considered building their own sandbox infrastructure, but decided against it after estimating it would take two to three senior engineers months to reach production, with no guarantee the resulting system would scale.
## The solution: real computers for agents
E2B provides infrastructure that scales with Anything’s spiky workloads while keeping sandbox startup fast enough that users don't notice the infrastructure behind the scenes. E2B gives each user in an Anything session its own MicroVM: a lightweight virtual machine that behaves like a real Linux computer while providing strong isolation between users. That means Anything's agents don't need special-case tooling or restricted runtimes; they can use the same package managers, development servers, browsers, and build tools they would on a developer's laptop.
> "E2B is more or less like a true representation of a Linux box that a coding agent would expect to use whenever they were coding if they were working locally on your machine."
\- Marcus Lowe, CEO, Anything
Anything was able to integrate with E2B's SDK quickly, start sending their production traffic almost immediately, and validate that the execution layer could scale alongside the rest of the product.
Anything uses E2B Templates to give every new project a preconfigured starting environment. As each app evolves, the platform continuously creates updated templates containing the latest version of the user's code, so new sessions start from the current state instead of an outdated baseline.
Anything also makes use of E2B snapshots to resume running development servers instantly, eliminating setup work between sessions. When users return to their project, the environment resumes instead of starting over. For interactive coding, eliminating repeated startup work makes the product feel substantially faster.
> "I still don't understand how you guys do that. That part is really cool."
\- Marcus Lowe, CEO, Anything
## The result: built for consumer scale
**Concurrency**: Thousands of concurrent sandboxes
**Scalability**: Workloads for 1M+ unique users supported
**Headcount savings:** 2-3 senior infrastructure engineers over a few months
## Scaling with Anything
Today, Anything runs thousands of E2B sandboxes in parallel, serving more than one million users. What started as a build-versus-buy decision has become core infrastructure for the product, allowing the team to focus on AI software engineering while relying on E2B to provide fast, isolated Linux environments at consumer scale.
> "As we grow beyond millions of users, we know and trust that E2B will continue to grow with us. As new users sign up, they'll always have a hot and ready sandbox with their code loaded in it ready to go."
\- Marcus Lowe, CEO, Anything
---
Source: https://e2b.dev/customers/rogo.md
Published: 2026-06-12
# Rogo Delivers AI for Financial Institutions with Claude Managed Agents & E2B
## **State-of-the-art orchestration on top, kernel-level isolation, and 10,000 to 15,000 concurrent sandboxes underneath**
Rogo is an AI platform for financial institutions, helping bankers and investors execute complex research, analysis, and deal workflows. Their customers include hundreds of leading firms globally, whose teams send tens of thousands of queries to AI agents every day.
Two constraints shape everything Rogo builds. The work is heterogeneous: a question might need a narrow specialist or a broad generalist, and routing it wrong degrades the answer. And the customers are financial institutions, so any code an agent generates has to run in a sandbox with full auditability, traceability, and per-session isolation. Those two constraints map onto two layers: the agent harness, and the execution layer underneath it.
## **Two layers: Claude Managed Agents and E2B**
Claude Managed Agents allow Rogo to configure specialist and generalist agents. Rogo was drawn to Managed Agents for its orchestration and post-training capabilities. Because Anthropic develops both the harness and the underlying models, the platform can benefit from improvements to Claude's capabilities without requiring teams to maintain every aspect of orchestration themselves.
Rogo built its own agent harness for finance-specific workflows. Evaluating it alongside Claude Managed Agents highlighted how the two approaches could complement one another, with a natural division between finance-specific logic and general-purpose orchestration. E2B is the execution layer. When an agent generates code, it runs inside an isolated E2B sandbox.
The flow is straightforward. A user asks an agent for a workflow to be done. Rogo routes it to a generalist or a specialist. Claude Managed Agents is designed to handle the agent flow. E2B provides an isolated container for any code execution. Artifacts and the final answer stream back to the user.
The result is managed-agent convenience without giving up isolation, control, or speed.
> “With Claude Managed Agents, we easily configured state-of-the-art model orchestration, and with E2B, we got the trust and isolation our clients require.”
\- Alex Wang, Applied AI, Rogo.
## **The harness: specialists and generalists without the plumbing**
Rogo uses both specialist and generalist agents — specialists for highly scoped tasks with a clear deliverable, and generalists for open-ended work. A single agent might carry both general Q&A and the ability to build a full information memorandum, the roughly 90-page document that covers everything about a company. Bundling unrelated capabilities into a single agent, however, deteriorates overall performance.
With Claude Managed Agents, Rogo can easily configure dedicated agents that each know only what their task requires. For example, the CIM (Confidential Information Memorandum) builder — the builder for the roughly 90-page document covering everything about a company for potential buyers — needs only the context to build that document. Scoping data sources, tools, and abilities to a single task is straightforward out of the box, as is setting system instructions per agent — and Rogo plans to maintain both approaches for the foreseeable future.
Scoping agents to a single task also makes parallelization possible, which can be particularly useful for complex workflows. For example, creating a CIM is a high-stakes process with multiple sections, each requiring different types of analysis. Rather than forcing all of that context into a single agent, the work can be split into focused subtasks, such as separate agents for the market narrative and financial analysis. This helps manage complexity and can improve the quality of the final output.
> "Claude Managed Agents allows you to abstract away the harness, so you get all the benefits of a state-of-the-art harness for free."
\- Alex Wang, Applied AI, Rogo.
Offloading context management, error recovery, and graceful failure lets Rogo's team spend its time on the parts specific to finance, not on the orchestration plumbing that every agent product has to rebuild.
## **The execution layer: isolation, scale, and rapid startup**
The harness decides what runs. E2B decides where it runs, and for regulated industries, that is the part that has to be right.
**Per-session isolation, by default**. Any code an agent generates runs in its own E2B sandbox with full auditability and traceability. E2B runs each sandbox as an isolated MicroVM, so isolation is enforced at the kernel level rather than by a shared container. Teams with stricter perimeters can run the same setup inside their own cloud through BYOC; Rogo chose E2B's managed infrastructure to avoid operating it themselves.
> "We work with clients from hundreds of financial firms, so per-session isolation is a hard requirement."
\- Alex Wang, Applied AI, Rogo.
**Scale that holds in production**. With tens of thousands of users firing concurrent queries, Rogo needs to spin up sandboxes on the order of 10,000 to 15,000 at once. Doing that without investing in their own infrastructure was the deciding factor.
> "E2B offers the flexibility to scale in production that is tough to match."
\- Alex Wang, Applied AI, Rogo.
**Networking and streaming that other providers didn't have**. E2B gave Rogo native support for domain-based network allowlists and pre-signed URL streaming. The allowlists keep agent network access scoped to approved domains; pre-signed URL streaming returns artifacts to the user without routing data through extra hops.
**Startup time, because the product is latency-sensitive**. Rogo pre-configures E2B templates with the scripts and Python packages that its agents need, so a sandbox comes up ready to run instead of installing dependencies on every run.
> "On a latency-sensitive product, that startup time makes all the difference. It helps craft a magical experience for our end users, who go from problems to solutions in no time."
\- Alex Wang, Applied AI, Rogo.
## **Security is the through line**
Rogo's customers are in highly regulated industries, and they delegate the cutting edge of AI to Rogo on the condition that it stays secure. That is the bar the pairing is built to clear: Claude Managed Agents simplifies how Rogo configures specialist and generalist agents, and E2B supplies the isolation for secure code execution.
> "At Rogo, top-of-the-line AI infra built with security by design is paramount. E2B gives us just that."
\- Alex Wang, Applied AI, Rogo.
---
Source: https://e2b.dev/customers/stackai.md
Published: 2026-06-08
# How StackAI Runs Enterprise AI Agents for Regulated Industries
StackAI is a no-code enterprise AI platform used by enterprise teams to build and deploy agentic workflows on sensitive data. Founded in 2023 by two MIT PhDs, Tony Rosinol and Bernard Aceituno, it lets users automate document- and process-heavy work by dragging and dropping nodes, plugging in any LLM, and connecting natively to enterprise systems like SharePoint and SAP.
Those workflows run code. Users trigger it, and so do the LLMs. In regulated industries, that code cannot run anywhere it can touch another customer's data or the host. As StackAI's customer workflows grew more sophisticated, the team hit a wall. The team needed secure, isolated execution baked into every workflow, at scale, and off-the-shelf options like REPL did not meet the bar.
## The problem: building sandbox infrastructure is a full-time job
StackAI initially built its own sandbox system to support user-triggered and LLM-triggered code execution. It held up at small scale. As demand grew, the team found itself maintaining provisioning, network isolation, session persistence, and enterprise-grade access control in-house. None of it scaled cleanly, and all of it pulled engineers off the product - especially as adoption rapidly grew.
> "*We built our own code sandbox VM, and it neither scaled well nor had all the features we needed. We couldn’t afford the time to take away from building revenue-generating features for 5-6 of our platform engineers to build and maintain the infrastructure*".
\- Tony Rosinol, Co-Founder at StackAI
The team’s requirements for an ideal solution were specific:
- **Real isolation by default**. StackAI's customers operate in banking, defense, and healthcare. Agent-generated code has to run in an environment hardened against escape, not a shared container. E2B runs every sandbox as an isolated VM.
- **Fast, easy provisioning**. A sandbox spins up per execution. If that step is slow or fiddly, it shows up in every workflow run.
- **Scale**. Hundreds of agent workflows in production per customer, with headroom to grow.
> "*What stood out initially was how fast E2B was to get going from running a few sandboxes for evaluation to hundreds of thousands of sandboxes. It just worked*."
\- Tony Rosinol, Co-Founder at StackAI
## The integration: three capabilities, one runtime
After the initial evaluation, the StackAI team quickly scaled E2B usage across three core use cases.
**Custom Code**. Users write and execute Python or JavaScript directly inside agent workflows through the Code Node. E2B handles the sandboxing, so every execution runs in a secure, isolated environment regardless of workflow complexity.
**Computer Use and RPA**. Users first map out what a successful session looks like by recording and identifying the right steps. StackAI then replays those steps within E2B sandboxes, enabling agents to interact with desktop environments and enterprise software securely and reliably.
**Browser Use**. Agents that navigate the web, scrape data, or drive legacy and browser-based tools run those sessions inside E2B, keeping every session isolated and secure.
The throughline across all three is the same: isolation that customers in regulated industries can trust, provisioning the team does not have to manage, and performance that holds at scale.
> "With E2B, it's really easy to provision sandboxes. They're secure by default, highly capable, and very performant. E2B is one of the only providers that delivers the enterprise security we need at the scale and throughput that helps us create experiences that delight our end users and solve their problems."
\- JJ Miller, Senior Product Manager at StackAI
## What's next: a sandbox for every agent
StackAI's roadmap treats the sandbox as core infrastructure, not a feature. The plan is for every agent to get its own sandbox. Not just for code execution, but as a persistent environment with its own datastore: a place to keep data, run processes, and maintain state across sessions.
That bet depends on the runtime keeping up.
> "Sandboxes are a key primitive for agents going forward. For us to keep growing as a company, we need a partner that can grow with us - we’re happy to have found one in E2B."
\- JJ Miller, Senior Product Manager at StackAI
**Follow StackAI for more**:
[StackAI X](https://x.com/stackai) [StackAI LinkedIn](https://www.linkedin.com/company/stackai/) [StackAI website](https://www.stackai.com/)
---
Source: https://e2b.dev/customers/genspark.md
Published: 2026-05-07
# How Genspark Uses E2B to Run a General-Purpose AI Agent for Millions of Users
Genspark's Super Agent orchestration handles everything from deep research and slide creation to making phone calls and analyzing data. Under the hood, Genspark uses E2B infrastructure to give its agents secure, isolated virtual machines where they can execute code, browse the web, and complete complex tasks autonomously.
## **A General-Purpose Agent Needs a General-Purpose Runtime**
Unlike single-purpose AI tools, Genspark’s Super Agent orchestration is designed to handle any task a user throws at it. A user might ask it to research a competitor, build a financial model, generate a presentation, or analyze a dataset — all in a single session.
This means the execution environment can’t be a simple code sandbox. Genspark’s agents need full virtual machines where they can:
- **Execute arbitrary code** — Python, JavaScript, shell scripts, whatever the task requires
- **Install packages and dependencies** on the fly, adapting to each task
- **Run long-running workflows** — research sessions that span dozens of steps, not just quick code snippets
- **Access the web** — browsing, scraping, and pulling live data as part of multi-step tasks
- **Maintain a persistent state** — keeping context, files, and intermediate results across an entire agent session
E2B’s Firecracker micro VMs provide exactly this: lightweight virtual machines that spin up in milliseconds, run securely in complete isolation, and support the full range of operations a general-purpose agent needs.
## **Enterprise-Ready from Day One**
Genspark isn’t just a consumer product — enterprises rely on it for business-critical workflows. When a Fortune 500 company uses Genspark’s agent to analyze internal data or generate reports, the execution environment needs to meet enterprise standards. When choosing a sandbox provider, but more generally, an AI cloud to run your agents on, you need to be very honest in answering the following questions:
- **Security & data isolation** — Does this sandbox solution make your users more or less secure?
- **Reliability** — How much can our users bet on this tool?
- **Scalability** — Can our users really grow 100x with this sandbox provider?
- **Speed** — Is this tool slowing down your users? If yes, how much?
With E2B, each agent session runs in its own isolated Firecracker micro VM. No data leaks between sessions, no risk to Genspark’s core infrastructure. From a single user researching a topic to enterprise teams running hundreds of parallel agent tasks, E2B also scales with demand.
\- Kay Zhu Co-founder & CTO, Genspark
## **Why E2B Over Building In-House**
For a company growing as fast as Genspark (from zero to $250M ARR in 12 months) building and maintaining custom execution infrastructure would be a massive distraction. This kind of infrastructure typically requires 3-5 dedicated engineers and months of development.
E2B lets Genspark’s engineering team focus entirely on making the agent smarter and more capable, while E2B handles the execution layer: spinning up isolated virtual machines, managing resources, ensuring security, and scaling to millions of sessions.
## **What’s Next**
Genspark is at the forefront of a new category: general-purpose AI agents capable of truly doing anything. As agent capabilities expand — longer sessions, more complex multi-step workflows, deeper enterprise integrations — the demands on the execution environment will only grow. E2B’s infrastructure is built for exactly this trajectory: secure, fast, scalable virtual machines that give AI agents the full computing environment they need to get real work done.
*E2B is the open-source cloud infrastructure for AI agents. Give your agents secure virtual machines that spin up in milliseconds. Building something similar*? [*Book a call*](https://e2b.dev/contact) *with our engineers to learn more*.
---
Source: https://e2b.dev/customers/gumloop-case-study.md
Published: 2026-02-27
# Gumloop Has Run on E2B for Over Two Years — Serving Enterprises Like Shopify and Instacart
## Two Years, One Outage
Gumloop is one of E2B’s longest-standing customers. Code execution is a core part of their infrastructure — powering every custom workflow node, every integration, and every agent tool call. In over two years of running on E2B, co-founder Rahul Behal recalls only a single outage.
> “You are super reliable. There’s only been one outage I can think of in the past two years. You’re doing a really good job at reliability and support, and the product is really good. Just keep doing the exact same thing you’re doing.” — Rahul Behal, Co-founder, Gumloop
When E2B works, Gumloop’s enterprise customers never have to think about what’s running under the hood. For a platform trusted by banks and large corporations, that’s the point.
## Enterprise-Grade from Day One
Gumloop chose E2B early and never left. Rahul points to several reasons:
- Reliability — Effectively zero downtime over two years
- Developer experience — Easy to set up, simple to integrate
- Cost — Affordable at scale for enterprise workloads
- Open source — The #1 reason: the option to self-host for data residency
> “The number one reason is that it’s open source and we have the option to self-host. As we get more and more enterprises, we’re deploying Gumloop on their cloud infrastructure. Having the ability to self-host is important to us from a data residency perspective.” — Rahul Behal, Co-founder, Gumloop
For enterprise customers in regulated industries, data residency isn’t optional. E2B’s open-source infrastructure gives Gumloop the flexibility to run sandboxes wherever their customers need them.
## How Gumloop Uses E2B
Gumloop has evolved from a workflow builder into a full AI automation platform with agents. E2B powers code execution across both:
- Agent code sandbox — When a Gumloop agent needs to run code, it calls an E2B sandbox under the hood, executes, and returns results to the conversation
- Custom workflow nodes — Users describe what they need in plain English, AI writes the code, and E2B executes it every time the node runs
- Integration nodes — When users connect services like Gmail or Salesforce, AI generates the integration code and E2B runs it in a sandbox
Virtually nobody writes code manually — every piece of code running in E2B is AI-generated from user prompts.
> “The vast majority of people use agents nowadays. In terms of tools, you can add a code sandbox tool — and this uses E2B under the hood.” — Rahul Behal, Co-founder, Gumloop
## Growing Together
Gumloop’s trajectory from a side project built in a Vancouver bedroom to serving Shopify and Instacart reflects a broader shift: enterprises are adopting AI automation at scale, and they need infrastructure they can trust.
E2B has been that infrastructure for Gumloop from the start. As their agent capabilities grow and enterprise customers explore self-hosted deployments, E2B’s role only becomes more central — the same reliable foundation, now serving the next generation of enterprise AI workflows.
---
Source: https://e2b.dev/customers/groqs-compound-ai-models-are-powered-by-e2b.md
Published: 2025-05-20
# Groq's Compound AI Systems Are Powered by E2B
In April 2025, Groq [launched Compound Beta](https://console.groq.com/docs/compound), its first compound AI system that combines LLMs with the ability to search the web and execute code. Behind this advancement is E2B's AI infrastructure. It enables Groq to securely run code execution at scale while maintaining their renowned speed advantage.
Compound Beta represents a significant leap forward in AI systems that can take action beyond just generating text. By integrating E2B's secure code execution environment, Groq has built a system that combines the reasoning abilities of large language models with real-world interaction capabilities.
## **Building a Complex AI System**
For AI to solve real-world problems effectively, it needs to do more than predict the next token in a sequence. It needs to access up-to-date information, perform calculations, and interact with external data sources.
Compound Beta uses iterative, server-side tool execution to answer complex queries. The system can autonomously decide when and how to use tools such as web search and code execution, potentially running them multiple times before returning a response. This allows it to handle tasks that require both reasoning and real-time data processing:
- Searching the web for current information
- Running and validating code for computations and data analysis
- Returning grounded answers using live information or logical reasoning.
## **E2B: The Engine Behind Groq's Code Execution**
When Groq set out to build Compound, they needed a solution for code execution that would maintain their performance edge while ensuring security and reliability. E2B's sandbox environment proved to be the perfect fit.
> "We needed a fast, secure, and scalable way to let Compound Beta execute code. E2B was the obvious choice. E2B’s impressive API interface made their sandbox infrastructure require minimal effort to integrate, and their team has been incredibly responsive. They are truly invested in our success."
\- Benjamin Klieger, Compound AI Lead at Groq
Under the hood, E2B uses Firecracker microVMs—lightweight virtual machines that provide complete isolation for running untrusted code. For Groq, this means:
1. **Security**: Each piece of code runs in a fully isolated environment, preventing any potential security risks
2. **Speed**: E2B sandboxes start up quickly maintaining Groq's performance advantage
3. **Scalability**: The AI system scales seamlessly to handle thousands of concurrent requests
4. **Reliability**: Consistent execution environment ensures predictable results
> "E2B was the only solution that could match our requirements for both security and speed. The fast sandbox startup time was a game-changer for us."
\- Benjamin Klieger, Compound AI Lead at Groq
Groq's implementation of E2B for Compound Beta focused on three key aspects:
1. **Seamless API Integration**: The team integrated E2B's code execution capabilities as a new module directly into their existing infrastructure, allowing developers to access LLMs with server-side code execution with just a simple model name change.
2. **Advanced Orchestration**: Groq developed sophisticated orchestration systems that decide when and how to use code execution versus web search, with full control over tool selection.
3. **Performance Optimization**: The team worked to ensure the entire pipeline, from initial request to final response, maintained Groq's industry-leading performance standards.
## **Why Groq Chose E2B Over Alternatives**
The Groq team evaluated several options for code execution before selecting E2B:
- **Docker containers** were too slow to spin up, adding significant latency to responses
- **Custom solutions** would have required months of engineering time and ongoing maintenance
E2B offered the right balance of security, performance, and developer experience. The solution's focus on AI-specific use cases made it particularly well-suited for Compound Beta's requirements.
## **Results and Future Plans**
Since launching in April 2025, Compound Beta has demonstrated impressive performance. On Groq's RealtimeEval benchmark, designed to measure search and computation capabilities on current events and live data, Compound Beta outperformed competing solutions from major providers.
Looking ahead, Groq plans to expand Compound's capabilities with additional tools and improved orchestration. The team is also exploring more advanced use cases that leverage E2B's sandbox technology:
1. **Extended persistence**: Allowing users to maintain state across multiple interactions
2. **Multi-language support**: Expanding beyond Python to additional programming languages
3. **Filesystem interactions**: Supporting more complex workflows involving file manipulations
## **Learn more**
- [Groq Compound Beta launch post](https://console.groq.com/docs/compound)
- [Compound Beta documentation](https://console.groq.com/docs/agentic-tooling/compound-beta)
- [Groq's RealtimeEval benchmark](https://github.com/groq/realtime-eval)
---
Source: https://e2b.dev/customers/lindy-powers-ai-workflows-with-e2b-code-action.md
Published: 2025-05-19
# Lindy Powers AI Workflows With E2B Code Action
Lindy AI, the workflow automation platform, recently launched their highly requested power-user feature: custom Python and JavaScript code execution directly within workflows. Behind this new capability is E2B's secure code execution infrastructure, allowing Lindy users to run custom code safely without leaving their automation environment.
This integration represents a significant advancement in no-code/low-code platforms, bridging the gap between visual workflow builders and the flexibility of custom code. By partnering with E2B, Lindy has empowered its users to tackle more complex automation challenges while maintaining the platform's user-friendly approach.
## **Adding Code Power to No-Code**
While Lindy's visual workflow builder has enabled thousands of users to automate their work, there was a persistent demand for the ability to execute custom code for more advanced use cases.
\- Luiz Scheidegger, Head of Engineering at Lindy
The new Code action feature enables Lindy users to:
- Perform advanced data manipulation and analysis
- Create custom API integrations
- Implement complex conditional logic
- Execute everyday calculations within their workflows
This bridges a critical gap in the platform's capabilities. Now, when users encounter a task that's difficult to achieve with standard actions, they can seamlessly insert custom code without leaving the Lindy environment.
## **The Security Challenge of User-Generated Code**
Implementing code execution in a production SaaS platform presents significant technical and security challenges. Lindy needed a solution that would:
1. Speed of implementation compared to building in-house
2. Securely isolate user code to prevent security vulnerabilities
3. Scale efficiently to handle thousands of workflow runs
4. Start quickly to maintain the responsiveness users expect
5. Support both Python and JavaScript, the most requested languages
6. Integrate seamlessly with their existing infrastructure
\- Luiz Scheidegger, Head of Engineering at Lindy
## **Integrating E2B for Secure Code Execution**
After evaluating building code support in-house, Lindy chose E2B as the foundation for their Code action feature. In particular, the security and isolation posture of E2B code environments were a very attractive proposition for Lindy. E2B's architecture offers several benefits that were critical for Lindy's implementation:
1. **Isolation**: Each code execution runs in its own Firecracker microVM, providing complete isolation from other users and the host system.
2. **Fast startup**: E2B sandboxes initialize in approximately 150ms, ensuring that adding code to workflows doesn't introduce noticeable delays.
3. **Language support**: Native support for both Python and JavaScript, the two languages most requested by Lindy users.
4. **Simple integration**: E2B's API made it straightforward to integrate code execution capabilities into Lindy's existing platform.
Lindy's implementation of the Code action feature focused on maintaining the platform's user-friendly approach while adding powerful new capabilities.
The team designed a simple interface that allows users to:
- Select their preferred programming language (Python or JavaScript)
- Write or paste their code in a syntax-highlighted editor
- Define input variables that connect to other parts of their workflow
- Specify output variables to use the code's results in subsequent steps
\- Luiz Scheidegger, Head of Engineering at Lindy
Behind the scenes, when a workflow with a Code action runs:
1. Lindy prepares the code execution environment using E2B's SDK
2. Input variables from previous workflow steps are passed to the sandbox
3. The user's code executes in an isolated E2B environment
4. Results are captured and passed back to the Lindy workflow engine
5. The workflow continues with subsequent actions, potentially using the code's output
## **Customer Impact and Future Plans**
The Code action feature has been in beta for several weeks and is now rolling out to all Lindy users. Initial feedback has been overwhelmingly positive. @ Lindy team: Any user feedback on the Code Action.
The new capability has also expanded Lindy's market appeal:
- Technical users who previously felt limited by no-code tools now have the flexibility they need
- Teams can create more sophisticated workflows without involving developers
- Companies can standardize on Lindy for a wider range of automation needs
Looking ahead, Lindy plans to expand the Code action feature with additional capabilities:
1. Allowing AI to help end users write code
2. UI optimizations - syntax highlighting and line numbers in the code editor
3. The ability for users to install packages
For Lindy's users, this new capability represents a significant expansion of what's possible within the platform. Tasks that previously required custom integrations or external services can now be handled directly within Lindy workflows.
## **Learn more**
- [Lindy](https://www.lindy.ai/) website
- [E2B integration](https://www.lindy.ai/integrations/e2b)
- [LinkedIn launch post](https://www.linkedin.com/feed/update/urn:li:activity:7323774203596296192/)
---
Source: https://e2b.dev/customers/how-manus-uses-e2b-to-provide-agents-with-virtual-computers.md
Published: 2025-05-06
# How Manus Uses E2B to Provide Agents With Virtual Computers
When Manus launched its general-purpose AI agent, it quickly became one of the most talked-about new platforms in the AI space. Behind the viral launch is a technically sophisticated multi-agent system that actually manages to execute real-world workflows end to end.
Manus is not a single LLM agent—it’s a more complex coordination system. Prompts go first to a planner agent that decomposes them into a sequence of subtasks. Then, executor agents carry them out using a variety of tools, from web browsing and file search to running commands in a terminal.
For this, the Manus agent needs a full cloud computer, and that’s why it relies on E2B - a secure cloud platform designed for AI agents to run untrusted code securely and at scale.
## It’s Not Enough to Just Run Code
For AI agents to actually get things done and act like autonomous humans, they must be able to perform tasks from data analysis to using a terminal.
\- Tao Zhang, Co-founder at Manus
A lot of the tasks go down to code, which is like the core of the agent. But a mere code execution isn’t enough. Manus agent needs its virtual computer, the same as if it were a human researcher working on the task end-to-end, sometimes taking even dozens of minutes.
Under the hood, E2B uses Firecracker microVMs—ephemeral, lightweight virtual machines originally developed by AWS. These VMs serve as whole virtual computers for Manus. Inside the sandbox, agents can run Python, JavaScript, Bash, and more.
E2B sandboxes can run for hours in persistent sessions while the agent decides in each iteration which action to perform in the sandbox. For paid users, they can save information in the E2B sandbox for up to 14 days.
The tools span from using the Chromium browser to visit URLs, save images, and scroll, through executing terminal commands to using the filesystem to create, edit, or delete files.
This means that agents in Manus can act more like real researchers or developers, keeping context between steps, updating plans, and producing complex artifacts—all within the same isolated sandbox session. It’s important to be able to pause and resume the sandbox sessions, for example, when the agent needs to check something with the user, requires credentials to access certain websites, or when passing the “verify you are human” tests.
## Why Manus Chose E2B, and not Alternatives
Manus team started building the agent, they tried to find a solution that would offer a fast and scalable environment for the agent.
During the first testing phase, they tried Docker. The problem was, it was really slow (10 - 20 seconds to spawn), but most importantly, Docker, as a container solution, doesn’t have full functionality of an operating systems. And Manus team needed real operating system so the agent can perform actions like installing apps or Python packages. That’s why they started searching for a solution built specifically for LLM-powered applications - and they discovered E2B.
\- Tao Zhang, Co-founder at Manus
When deciding, Manus also went for E2B because of these factors:
- **Speed**: A new sandbox can spin up in ~150ms, fast enough to keep up with the users’ standards
- **Good DX**: Manus team was able to implement and deploy E2B in half a day
- **Scalability**: Manus is gaining a lot of users quickly, and they need each user to have the agent work in a separate instance from the others
- **Self-hosting option**: Manus is running E2B on their machines. E2B self-hosting is easy to manage
Could Manus have built this infrastructure themselves? Technically, yes—but it would have taken months of work by a dedicated infra team. Re-creating and maintaining an infrastructure stack from scratch would have required 3–5 full-time infra engineers.
For most teams building agent platforms, that’s a distraction from product and research work. With E2B, Manus was able to ship faster and focus entirely on improving their multi-agent orchestration instead of reinventing cloud runtime infrastructure for their agents.
### Future Plans
Developers often ask about Manus selecting E2B over container alternatives. Looking ahead, Manus aims to extend agent capabilities across various operating systems, including Windows and Android. Since not all information and services exist solely on the web, environments like virtual Android significantly expand what their agents can access and accomplish.
\- Tao Zhang, Co-founder at Manus
Currently, Manus is already used for a variety of applications beyond just creating reports and charts. Manus' team is travelling around the world to talk to users and learn about [different use cases](https://manus.im/share/bR4qMz0bx9xXm1CYb2GV8h).
For example, the Chief Editor of Financial Times Chinese showcased a 3D-printed model representing the past decade of US national debt, generated entirely through Manus.
Another example Manus team seen recently in Dubai, where a social media consultant leveraged Manus to develop comprehensive content strategies for clients. The agent, after being provided with the client’s website and social media profiles, generated complete one-year strategies including audience targeting, titles, content, hook sentences, and channel-specific recommendations. The resulting 50+ page document costs approximately $6-7 to produce, yet commands thousands in consulting fees.
Manus has a lot ahead, and this is just the beginning. By pushing boundaries and learning from real-world use cases, the team is committed to creating an AI platform that delivers exceptional value across industries while remaining accessible to all users.
## Learn more
- [Manus landing page](https://manus.im/)
- [Manus LinkedIn](https://www.linkedin.com/company/manus-im/)
- [Manus X](https://x.com/ManusAI_HQ)
---
Source: https://e2b.dev/customers/how-hugging-face-is-using-e2b-to-replicate-deepseek-r1.md
Published: 2025-03-31
# How Hugging Face Is Using E2B to Replicate DeepSeek-R1
Shortly after the release of DeepSeek-R1, Hugging Face launched the [Open R1 project](https://github.com/huggingface/open-r1) to reverse-engineer the missing pieces of DeepSeek’s data and training pipeline.
A key part of that pipeline involves [*reinforcement learning with verifiable rewards*](https://huggingface.co/papers/2411.15124), where a large language model (LLM) is trained to solve problems that can be checked for correctness against ground-truth answers. For example, given a simple math problem like “What is 1+1?”, the LLM receives a binary reward of 1 or 0 based on whether it produces the correct answer or not. As shown by DeepSeek-R1, maximising these rewards with reinforcement learning enables LLMs to obtain robust reasoning capabilities that translate into high accuracy on various public benchmarks:
For domains like mathematics, verifying whether an LLM’s output is correct can be achieved by parsing strings with libraries like [Math-Verify](https://github.com/huggingface/Math-Verify). However, for domains like [competitive programming](https://huggingface.co/papers/2502.06807), the reward is obtained by executing LLM-generated code and comparing the result against the expected outcome from a set of test cases:
The problem here is that executing LLM-generated code locally poses many [risks](https://huggingface.co/docs/smolagents/main/tutorials/secure_code_execution#local-code-execution); the model may accidentally produce programs that corrupt your hard drive, or worse, `rm -rf` your entire home directory!
The solution is to use E2B Sandboxes for secure code execution! Let’s take a look at how the Hugging Face team uses Sandboxes in the Open R1 project.
## Using E2B for code execution
Hugging Face uses E2B Sandboxes as part of the reward function for code execution. During training, the LLM-generated code is executed in the isolated E2B sandboxes. Currently, this reward function targets competitive programming competitions like [CodeForces](https://codeforces.com/), where solutions are executed against a set of test cases and the overall success rate is returned as the final reward.
> “We found E2B is a simple and cost-effective platform for this. It was extremely easy to set up.”
\- Lewis Tunstall, Research Engineer at Hugging Face
```python
def code_reward(completions, **kwargs) -> list[float]:
"""Reward function that evaluates code snippets using the E2B code interpreter.
Assumes the dataset contains a `verification_info` column with test cases.
"""
if not is_e2b_available():
raise ImportError(
"E2B is not available and required for this reward function. Please install E2B with "
"`pip install e2b-code-interpreter` and add an API key to a `.env` file."
)
# TODO: add support for other languages in E2B: https://docs.e2b.dev/code-interpreting/supported-languages
"""Returns a reward function that evaluates code snippets in a sandbox."""
evaluation_script_template = """
import subprocess
import json
def evaluate_code(code, test_cases):
passed = 0
total = len(test_cases)
exec_timeout = 5
for case in test_cases:
process = subprocess.run(
["python3", "-c", code],
input=case["input"],
text=True,
capture_output=True,
timeout=exec_timeout
)
if process.returncode != 0: # Error in execution
continue
output = process.stdout.strip()
# TODO: implement a proper validator to compare against ground truth. For now we just check for exact string match on each line of stdout.
all_correct = True
for line1, line2 in zip(output.split('\\n'), case['output'].split('\\n')):
all_correct = all_correct and line1.strip() == line2.strip()
if all_correct:
passed += 1
success_rate = (passed / total)
return success_rate
code_snippet = {code}
test_cases = json.loads({test_cases})
evaluate_code(code_snippet, test_cases)
"""
code_snippets = [extract_code(completion[-1]["content"]) for completion in completions]
verification_info = kwargs["verification_info"]
scripts = [
evaluation_script_template.format(code=json.dumps(code), test_cases=json.dumps(json.dumps(info["test_cases"])))
for code, info in zip(code_snippets, verification_info)
]
language = verification_info[0]["language"]
if not all(v["language"] == language for v in verification_info):
raise ValueError("All verification_info must have the same language", verification_info)
try:
rewards = run_async_from_sync(scripts, language)
except Exception as e:
print(f"Error from E2B executor: {e}")
rewards = [0.0] * len(completions)
return rewards
```
Hugging Face chose E2B Sandboxes, the isolated cloud environments for executing AI-generated code. The environment is specialized for LLMs, making it easy, for example, to extract errors, or have the LLM reference to previously defined variables, functions, etc. Inside the E2B Sandbox, it’s easy to run code, start programs, start long-running processes, use the filesystem, upload data to the Sandbox, and download any type of file from the Sandbox.
E2B Sandboxes also fulfill important requirements:
- **Security**: E2B's use of Firecrackers by AWS for creating isolated environments is a secure way to run LLM-generated code.
- **Speed**: When a new Sandbox session is started, E2B starts a small VM in the cloud. All this takes about 150-170 ms. This is critical with methods like reinforcement learning, where one tries to minimise idle time on GPUs waiting for the rewards to be computed.****
- **Price**: E2B Sandboxes are cheap to run, as single training run costs up to a few dollars of E2B.
## Integrating E2B Sandboxes in Open R1
Integrating E2B Sandboxes with Open R1’s reinforcement learning pipeline was quite straight forward and involved the following steps:
- Define a template for the LLM-code to be executed, along with the reward to be computed (success rate in the case of competitive programming)
```python
evaluation_script_template = """
import subprocess
import json
def evaluate_code(code, test_cases):
passed = 0
total = len(test_cases)
exec_timeout = 5
for case in test_cases:
process = subprocess.run(
["python3", "-c", code],
input=case["input"],
text=True,
capture_output=True,
timeout=exec_timeout
)
if process.returncode != 0: # Error in execution
continue
output = process.stdout.strip()
all_correct = True
for line1, line2 in zip(output.split('\\n'), case['output'].split('\\n')):
all_correct = all_correct and line1.strip() == line2.strip()
if all_correct:
passed += 1
success_rate = (passed / total)
return success_rate
code_snippet = {code}
test_cases = json.loads({test_cases})
evaluate_code(code_snippet, test_cases)
"""
```
- Use the asynchronous Sandbox to launch hundreds of tasks in parallel
```python
async def run_async(scripts: list[str], language: str) -> list[float]:
sbx = await AsyncSandbox.create(timeout=30, request_timeout=3)
# Create a list of tasks for running scripts concurrently
tasks = [run_script(sbx, script, language) for script in scripts]
# Wait for all tasks to complete and gather their results as they finish
results = await asyncio.gather(*tasks)
rewards = list(results) # collect results
# Kill the sandbox after all the tasks are complete
await sbx.kill()
return rewards
```
> “It took just a few hours to implement E2B for code execution”
\- Lewis Tunstall, Research Engineer at Hugging Face
Important features provided by E2B are:
- **Multi-language support**. The Open-r1 project can currently run code in Python, JavaScript, C++ with more languages (e.g., Rust and Lean4) coming in the future. Support for multiple languages is possible by specifying the language inside the E2B Sandbox.
- **Sandbox persistence**. The persistence across calls was needed. E2B allows this using sandbox.run\_code(). The E2B sandboxes use headless Jupyter server so the code is essentially running in a Jupyter-like notebook.
- **Handling multiple instances**. Hugging Face is currently launching **hundreds of** sandboxes per training step in their Open R1 experiments.
## What's next
What does the Open R1 project mean for open reasoning models? Since Hugging Face collected a lot of learnings when building it, it can mean a significant improvement in the quality of open-source LLMs in the next few months.
> “We plan to scale up our reinforcement learning pipeline to target code execution feedback for models like OlympicCoder”
\- Lewis Tunstall, Research Engineer at Hugging Face
## Learn more
- [Open-r1 GitHub repository](https://github.com/huggingface/open-r1)
- [Open-r1 blog post](https://huggingface.co/blog/open-r1)
- [The R1 paper](https://arxiv.org/pdf/2501.12599v1)
---
Source: https://e2b.dev/customers/how-lmarena-collaborated-with-e2b-to-build-llm-web-development-evals.md
Published: 2025-02-03
# How Arena Collaborated with E2B to Build LLM Web Development Evals
[Arena](https://arena.ai/code), a UC Berkeley research team, created WebDev Arena to evaluate the capabilities of large language models (LLMs) in web development. It’s a free open-source arena where two LLMs compete to build a web app.
You can vote on which LLM performs better and view a leaderboard of the best models.
## The challenges of LLM coding evals
Arena faced key technical challenges in testing multiple LLMs building real-time web applications.
When benchmarking LLMs on web development tasks, there's a fundamental need to run the code output. While getting LLMs to generate code is straightforward, running it securely at scale presents several challenges:
**Speed:** For meaningful comparisons, the evaluator needs to see the two LLMs' outputs nearly simultaneously. When you're comparing output from two or multiple models at once, even small delays compound quickly and distort the results of the voting.
**Code execution:** Running coding evaluations means handling large amounts of code simultaneously. Each LLM can generate extensive code snippets - from simple DOM manipulations to complex React components. Running these side-by-side requires significant computational resources.
**Isolation and security:** Each piece of LLM-generated code needs to run in an isolated environment. When you're comparing models like Claude and GPT-4 simultaneously, their code can't interfere with each other or share any execution context. Running LLM-generated code requires complete isolation to prevent any potential system access or interference between different models' executions.
## LLM code execution
These challenges led Arena to implement E2B as their execution environment. Under the hood, each sandbox is a small VM, allowing WebDev Arena to run code from multiple LLMs simultaneously while maintaining strict isolation and performance standards.
Key factors in this decision included:
- Security first - E2B's isolated environments ensure each LLM's code runs separately and securely
- Quick startup - E2B sandboxes start in ~150ms, essential for real-time model comparison
- Reliability at scale - Running multiple sandboxes simultaneously for different LLM battles
The E2B sandboxes provide isolated cloud environments specifically designed for running AI-generated code. E2B is agnostic towards the choice of techstack, frameworks, and models, which lets WebDev Arena run code from different LLMs (e.g., Claude, GPT-4, Gemini, DeepSeek, and Qwen).
> “Very quickly, we had over 50,000 users who started over 230,000 E2B sandboxes.”
\- Aryan Vichare, Member of Technical Staff at LMArena
## Implementing E2B
Arena implemented E2B for WebDev Arena in November 2024. The integration involved working closely with E2B's team to ensure reliable execution of code from multiple LLMs.
The implementation consisted of handling sandbox creation and code execution. Here's how it works:
1. Sandbox Configuration and Creation
The team implemented a flexible sandbox system with configurable timeouts and templates
2. Dynamic Dependency Management
The system supports dynamic package installation based on LLM requirements:
- Automatic detection of additional dependencies
- Custom installation commands for different environments
- Real-time logging of dependency installation status
3. Code Execution Pipeline
The system handles two types of code execution:
- Code Interpreter Mode: For running and evaluating code with detailed output
- Web Development Mode: For deploying and accessing web applications
> “It took 2 hours to get E2B up and running.”
\- Aryan Vichare, Member of Technical Staff at LMArena
## What's next
Since the launch of WebDev Arena with E2B's code execution layer, the platform has run over 50,000 model comparisons, with Claude 3.5 Sonnet currently being on top of the [leaderboard](https://web.lmarena.ai/leaderboard), followed closely by DeepSeek-R1. The team plans to expand the benchmark with more models and plan to do online general-purpose coding evaluations in the future.
> “We plan to expand to more software engineering tasks."
\- Aryan Vichare, Member of Technical Staff at LMArena