A Generative AI Engineer is tasked with developing an application that is based on an open source large language model (LLM). They need a foundation LLM with a large context window. Which model fits…

The correct answer is B. MPT-30B. MPT-30B (MosaicML Pretrained Transformer 30B) was specifically architected to support very large context windows-up to 65,536 tokens-using ALiBi positional embeddings, which allow it to extrapolate beyond its training length. DistilBERT is a small encoder-only model not…

LLM Model Selection and Capabilities

Question

A Generative AI Engineer is tasked with developing an application that is based on an open source large language model (LLM). They need a foundation LLM with a large context window. Which model fits this need?

Options

ADistilBERT
BMPT-30B
CLlama2-70B
DDBRX

How the community answered

(20 responses)

A
5% (1)
B
90% (18)
C
5% (1)

Explanation

MPT-30B (MosaicML Pretrained Transformer 30B) was specifically architected to support very large context windows-up to 65,536 tokens-using ALiBi positional embeddings, which allow it to extrapolate beyond its training length. DistilBERT is a small encoder-only model not designed for generative tasks. Llama2-70B, despite being larger in parameter count, has a native context window of only 4,096 tokens. DBRX is Databricks' model released later and is not the canonical answer for this requirement. MPT-30B was the standout open-source model specifically known for its large context window at the time of this exam's creation.

Topics

#Large Language Models (LLMs)#Context Window#Open Source LLMs#Foundation Models

Community Discussion

No community discussion yet for this question.

Full GENERATIVE-AI-ENGINEER-ASSOCIATE Practice