All modelsClaude 3 Opus
Claude 3 Opus represents the most powerful and advanced model in the Claude 3 family. This state-of-the-art AI delivers unparalleled performance on highly complex tasks, demonstrating fluency and human-like understanding that sets it apart from other AI models.
Intended Use
Task automation: plan and execute complex actions across APIs and databases, interactive coding
R&D: research review, brainstorming and hypothesis generation, drug discovery
Strategy: advanced analysis of charts & graphs, financials and market trends, forecasting
Use cases
Multilingual Capabilities: Claude 3 Opus offers improved fluency in non-English languages such as Spanish and Japanese, enabling use cases like translation services and global content creation.
Vision and Image Processing: This model can process and analyze visual input, extracting insights from documents, processing web UI, generating image catalog metadata, and more.
Steerability and Ease of Use: Claude 3 Opus is designed to be easy to steer and better at following directions, giving you more control over model behavior and more predictable, higher-quality outputs.
Performance
Benchmark | GPT-4 Evaluated few-shot | Few-shot SOTA | SOTA Best external model (includes benchmark-specific training) |
VQAv2 VQA score (test-dev) | 77.2% 0-shot | 67.6% Flamingo 32-shot | 84.3% PaLI-17B |
TextVQA VQA score (val) | 78.0% 0-shot | 37.9% Flamingo 32-shot | 71.8% PaLI-17B |
ChartQA Relaxed accuracy (test) | 78.5% | - | 58.6% Pix2Struct Large |
AI2 Diagram (AI2D) Accuracy (test) | 78.2% 0-shot | - | 42.1% Pix2Struct Large |
DocVQA ANLS score (test) | 88.4% 0-shot | - | 88.4% ERNIE-Layout 2.0 |
Infographic VQA ANLS score (test) | 75.1% 0-shot | - | 61.2% Applica.ai TILT |
TVQA Accuracy (val) | 87.3% 0-shot | - | 86.5% MERLOT Reserve Large |
LSMDC Fill-in-the-blank accuracy (test) | 45.7% 0-shot | 31.0% MERLOT Reserve Large | 52.9% MERLOT |
Limitations
Here are some of the limitations we are aware of:
Medical images: Claude 3 is not suitable for interpreting specialized medical images like CT scans and shouldn't be used for medical advice.
Spatial reasoning: Claude 3 struggles with tasks requiring precise spatial localization, such as identifying chess positions.
Hallucinations: the model can provide factually inaccurate information.
Image shape: Claude 3 struggles with panoramic and fisheye images.
Metadata and resizing: Claude 3 doesn't process original file names or metadata, and images are resized before analysis, affecting their original dimensions.
CAPTCHAS: For safety reasons, Claude 3 has a system to block the submission of CAPTCHAs.