Changelog

The official CircleCI Evals Orb makes it easy to integrate LLM evaluations into a CI pipeline, and to review evaluation results without context switching. The output of evaluations run through the Evals Orb is stored in CircleCI, and is accessible as a job artifact and as a PR comment added automatically by CircleCI.

Currently, the Evals Orb exposes commands to run evaluations through two popular LLMOps tools: LangSmith and Braintrust. If your evals leverage a different tool, let us know at ai-feedback@circleci.com. You can also contribute directly to the official Orb, by opening a PR on the public repository.

More resources on evaluating LLM-enabled applications are available in our documentation.

Collapse

Users can now orchestrate their model deployments across multiple environments through the Amazon SageMaker Orb. They can also track and manage their deployments from within the Releases UI.

See our documentation for more information.

Collapse

What’s New

A bug has been fixed in the circle-android utility in the machine image used in the Android orb, where the wait-for-boot step would hang.

The Android cimg and machine images will both be built using Ubuntu Jammy 22.04 in the near future.

Visit the Discuss post for more information.

Collapse

What’s New

The orb source, orb info, and orb/config validate CLI commands have been updated to work successfully with private orbs.

While the orb source and orb info commands operate the same as they do for public orbs, the orb/config validate command is different.

For more information on how to use the updated orb/config validate tool, please refer to the Private Orbs technical documentation.

Collapse

What’s New

We have made some exciting updates to our free plan, including giving teams more build minutes and access to our most popular features formerly only available on paid plans.

Some features on our new Free plan include:

• Up to 6,000 build minutes per month. Choose the right resource class size (S-L) to optimize for speed and power, or to maximize your build minutes.
• Unlimited users. CI and CD work better with a team; bring everyone on board easily.
• The largest free resource classes on Docker, Linux, and Windows so you can choose the right compute for any job.
• Increased concurrency limits: now up to 30x on Linux, Arm, and Windows.
• Streamlined collaboration with private orbs: shareable CircleCI configuration packages.
• Security. Our top priority is protecting our users’ intellectual property and sensitive secrets such as keys, tokens, and credentials. Rest easy with a suite of security features and compliance certifications, including FedRAMP and SOC 2 Type II.

You can learn about the of the changes here:

• CircleCI Blog - https://circleci.com/blog/most-generous-free-cicd-plan/
• Technical Documentation - https://circleci.com/docs/credits/#free-plan

Collapse

What’s New

You can now interact with private orbs via the CircleCI web app. A list of your organization’s orbs has been added on the Organization Settings page, and new private orb details pages that open in the orb registry have also been added as part of this release. The orbs list shows all public and private orbs authored by your organization / in your organization’s namespace(s), while the private orb details pages open in the Orbs registry and are flagged with a new “private” badge. Please note you will only be able to see these pages if you are logged in and authenticated; if you are not logged in, you will receive a 404 error.

With this update, you may select your preferred way to interact with orbs, which can be either the CircleCI CLI or the CircleCI App, while also being able to easily share private orb information with other users in your organization.

Note: This update will not add private orbs to the searchable Orb Registry, and un-published orbs will not have linked details pages.

Collapse