Correct common LLM assumptions

A language model assigns probabilities to sequences of tokens. A typical autoregressive text model generates a response by predicting a next token, appending it to the sequence, and repeating. The prompt and previously generated output influence each step. This mechanism can support summarization, translation, classification, extraction, and code generation.

A plausible continuation can still contain an incorrect statement. Fluency and factual accuracy need separate assessment. An application should verify outputs according to the consequences of an error.

A tokenizer maps text into token IDs. A token may represent a word, a word fragment, punctuation, or another text unit. Token boundaries depend on the tokenizer. The same sentence may have different token counts across models. Code, identifiers, non-English text, and unusual strings can have very different token densities.

For example, customerAccountId may be split into several pieces. Counting characters or whitespace-separated words cannot establish its exact token count. Use the tokenizer associated with the selected model, or the provider's supported counting mechanism. Special message delimiters and tool definitions may also contribute to request size.

Tokens matter because they affect request limits, generation length, latency, and often billing. A model with a larger context allowance still has finite limits.

The context window is the amount of tokenized information a model can process for a request under its documented accounting rules. Inputs can include instructions, user messages, prior turns, retrieved text, and tool results. Output also consumes capacity in many model interfaces. Separate output limits may apply.

A simplified teaching model uses input tokens + reserved output tokens <= context capacity. Actual APIs may account for reasoning tokens and other content differently. Always check the model's rules before implementing this calculation.

Conversation history usually reaches the model because the application or service supplies it. A model call does not inherently remember every earlier conversation. Trimming or summarizing history changes what information is available for the answer.

Providing a document in a prompt does not ordinarily update the model's weights. Provider data retention and training policies are separate service-level questions.

Many contemporary LLMs use transformer architectures. Attention lets a representation incorporate information from other tokens according to learned relationships. Positional information helps distinguish order. In causal generation, a token can attend to earlier positions without reading future generated tokens.

At this difficulty, explain the purpose of attention and ordering. Deriving attention equations or implementing a transformer belongs in deeper model study.

Chat systems commonly expose message roles to distinguish application instructions, user requests, and model responses. Exact roles and precedence are interface-specific. A chat template translates structured messages into the model's expected representation.

A request might instruct the assistant to summarize a supplied paragraph in three bullets. The paragraph is task data. Clear separation helps communicate intent, but instructions alone do not provide a reliable security boundary.

A hallucination is generated content that is fabricated, unsupported, or inconsistent with relevant evidence. A model might invent a book title, a citation, or a configuration option. Missing information, ambiguous requests, and misleading context can contribute.

Useful responses include supplying relevant evidence, checking citations against that evidence, validating fields, and allowing the application to report insufficient information. Asking the model whether it is certain cannot independently verify an answer. A confident tone is not a calibrated probability of correctness.

An output limit bounds generation. Sampling settings influence which token is chosen from a distribution. Lower temperature generally concentrates sampling toward more probable choices when sampling is enabled. Higher temperature can increase variation. Neither setting establishes factual truth. Supported parameters and their interactions vary by model.

A short output can result from a limit, a stop condition, or a deliberately concise response. Inspect termination information before deciding which explanation applies.