Anatomy of a Prompt That Actually Steers the Model
Diffusion models do not read prompts the way people read sentences. They respond to tokens and learned associations, which is why a structured prompt outperforms an eloquent one. The structure that consistently works orders information by importance: subject first, then medium, style, lighting, composition, and finally technical details.
Subject is the noun phrase plus its action and setting — "an elderly clockmaker repairing a pocket watch at a cluttered workbench" gives the model relationships, not just objects. Specificity here matters more than anywhere else; "beautiful woman" produces the averaged face of the training set, while "freckled woman in her 40s with windswept red hair" produces a person.
Medium decides the entire rendering pipeline: photograph, oil painting, watercolor, 3D render, charcoal sketch. It should come early because it constrains everything after it — film grain makes no sense in a watercolor.
Style narrows within the medium: film noir, Studio Ghibli, 1970s National Geographic, Art Nouveau. Then lighting and composition (next sections), then parameters like aspect ratio.
Order matters for a mechanical reason: most models weight earlier tokens more heavily, and long prompts get truncated or diluted at the tail. Put anything you cannot compromise on in the first ten words, and treat everything after word forty as a suggestion the model may ignore.
Midjourney, Stable Diffusion, DALL-E: Same Words, Different Machines
A prompt is not portable. The three major ecosystems parse weighting, negatives, and parameters differently, and pasting one engine's syntax into another silently does nothing — or worse, becomes literal text in the scene.
Midjourney uses double-colon weighting (ocean::2 sky::0.5), command-line style parameters (--ar 16:9 for aspect ratio, --chaos for variety, --sref for style reference images), and a --no parameter for exclusions (--no text, watermark). It leans stylized and opinionated by default; plain prompts still come out pretty.
Stable Diffusion ecosystems (AUTOMATIC1111, ComfyUI) use parenthesis weighting — (golden hour:1.3) boosts a concept, values below 1 suppress it — and a genuinely separate negative prompt field, which is the main quality lever: deformed hands, extra fingers, watermark, low quality live there, not in the main prompt. Base SD is far more literal than Midjourney, and results swing enormously with the checkpoint model you load.
DALL-E and similar natural-language systems invert the whole approach: no weights, no negative field. They are trained to follow prose, so you write a clear descriptive paragraph and state exclusions as sentences ("no text anywhere in the image"). Token-soup prompts actually perform worse here.
This tool outputs comma-separated keyword prompts — the dialect Midjourney and SD share. For DALL-E-style systems, ask an LLM to rewrite the keyword list as flowing prose first.
Same intent, three dialects:
Midjourney:
lighthouse on a cliff, storm waves::2, oil painting,
dramatic lighting --ar 16:9 --no boats, text
Stable Diffusion:
oil painting of a lighthouse on a cliff,
(storm waves:1.3), dramatic lighting
Negative prompt: boats, text, watermark, blurry
DALL-E style:
An oil painting of a lighthouse on a cliff battered
by storm waves. Dramatic lighting, wide 16:9
composition. No boats and no text in the image.The Vocabulary That Moves the Image: Light, Framing, Glass
Most quality words in circulating prompts are noise. Masterpiece, best quality, highly detailed, 8k are cargo-cult tokens — harmless on some models, meaningless on others. The vocabulary that reliably changes the image comes from photography and cinematography, because the training captions did.
Lighting is the highest-leverage category. Golden hour produces warm low-angle light and long shadows; blue hour a cool dusk glow. Rembrandt lighting gives the lit cheek triangle in portraits; rim lighting separates the subject from a dark background; soft diffused light from a north-facing window is the classic portrait recipe; harsh midday sun creates documentary flatness. Each phrase carries a whole learned scene.
Composition words set the camera you never held: extreme close-up, low-angle shot (power), bird's-eye view, rule of thirds, negative space (minimalist isolation), dutch angle (unease). Lens and film tokens finish the job: 85mm f/1.4 implies creamy background blur; 24mm wide angle implies environmental context and slight distortion; macro implies extreme detail at tiny scale; shot on Portra 400 pulls a specific warm film palette; long exposure smooths water and streaks lights.
Pick one term per category. Stacking golden hour, neon lighting, and overcast does not blend three moods — it makes the model average them into mud.
Flat prompt:
portrait of an old fisherman, masterpiece, 8k,
highly detailed, best quality, amazing
Photographic prompt:
portrait of an old fisherman mending a net,
weathered hands in focus, Rembrandt lighting,
overcast harbor at dawn, 85mm f/1.4, shallow
depth of field, Kodak Portra 400, rule of thirdsIterate Like a Photographer, Not a Slot Machine
The default failure mode of prompt iteration is the slot-machine loop: generate, dislike, rewrite the whole prompt, repeat. Because every run rerolls the random seed, you learn nothing — you cannot tell whether the image changed because of your edit or because of the dice.
The photographer's loop changes one variable at a time against a fixed seed. In Stable Diffusion, set the seed explicitly; in Midjourney, use --seed. With the seed pinned, regenerate after each single edit: swap golden hour for overcast, add one weight, remove one adjective. Now differences in the output are attributable to your change, and after a session you own a private phrasebook of what each token does in your engine.
A practical session shape: run one loose exploration batch (4–8 images, no seed) to find a composition worth pursuing; pin that seed; then do attribute passes — lighting first, then style, then cleanup via negative prompt or --no. Save the full prompt and seed with every keeper; a great image whose recipe you lost is a one-off, not a capability.
And know when to leave the prompt box: fixing one wrong hand or one object is inpainting work (or a Vary Region job), not a reason to re-roll a composition you already like.
Failure Modes and Honest Limits
Some failures are prompt problems; some are model problems; knowing which is which saves hours.
Contradictory tokens produce mud: minimalist plus intricate details, or two competing art styles, average into neither. Concept bleed is subtler — attributes wander between subjects, so "a red-haired woman and a man in a blue coat" often swaps the hair and the coat. Weighting helps; separating the subjects into different regions (or generating separately and compositing) helps more.
Other limits are structural, and no phrasing fixes them reliably: legible text in images (improving, still unreliable — plan to add typography in an editor), hands and interlocking fingers, correct object counts above three or four, and physical plausibility in mechanisms like bicycles and clocks. Budget retouching for these instead of burning fifty generations.
Two verification notes worth taking seriously. Style tokens borrowed from a builder can carry unexpected baggage — a tag you did not research may pull in an aesthetic you did not intend, so test unfamiliar tokens on a fixed seed before relying on them. And for commercial use, check the license terms of your specific generator; if a prompt names a living artist, weigh both the ethics and the increasingly active legal landscape before publishing.