A JSON prompt replaces a vague sentence with a structured brief that the video model reads like a film set. Google published an official guide upon the launch of Veo 3, recommending explicit decomposition into subject, action, scene, camera, and lighting fields (Google, 2024). This page explains why this structure improves shot consistency, how to write it for Sora, Veo 3, Runway Gen-3, and Kling, and how to industrialize it in an ad or storyboard workflow.
TL;DR: The JSON prompt in 30 seconds
A JSON prompt is a key-value object that isolates each dimension of a video shot (subject, action, camera, light, mood, duration). Recent video models, trained on structured cinematic descriptions, leverage these fields better than a free-form sentence.
Key points
- Veo 3 and Sora explicitly recognize
camera,lens,shot_typefields in their official guides.- JSON reduces the number of iterations needed to stabilize a shot by 2 to 4 times, according to public feedback from Runway users.
- A typical structure covers 6 blocks: subject, action, environment, camera, light, mood.
- The format connects directly to OpenAI and Google AI Studio APIs to automate batches of variations.
What is a JSON prompt and how does it differ from a text prompt?
A JSON prompt is a video description encoded in JavaScript Object Notation, a format standardized by ECMA-404. Where a text prompt mixes everything into one sentence, JSON isolates each cinematic variable. OpenAI documents this logic in its prompt engineering guide (OpenAI, 2024): a structured schema reduces interpretation ambiguity.
Concretely, writing “a cat on a sofa filmed in close-up in the morning” leaves the model to guess five parameters. JSON fixes them.
{
"subject": "ginger tabby cat",
"action": "sleeps curled up",
"location": "green velvet sofa",
"camera": {"shot": "close-up", "angle": "eye-level", "movement": "slow push-in"},
"lighting": "golden hour, garden-side window",
"mood": "calm, intimate"
}Why video models read JSON better
Generative video models like Veo 3 or Sora are trained on (video, caption) pairs. Many of these captions come from cinema databases annotated by fields (shot type, lens, lighting). A structured prompt brings your request closer to the training distribution, which improves rendering fidelity.
Why adopt a JSON prompt for Veo 3, Sora, and Runway?
Google DeepMind describes Veo 3 as capable of understanding detailed cinematic instructions in natural or structured language (DeepMind, 2025). Sora 2, released in late 2025, generates up to 60 seconds of coherent video with synchronized audio according to OpenAI documentation. To leverage these capabilities, JSON provides three clear benefits.
Gain in precision
Each field becomes an isolated signal. You can write "lens": "35mm" without the model confusing it with a distance. Google's official Veo guide explicitly lists recognized camera parameters: aerial shot, dolly shot, tracking shot, low angle.
Gain in iteration
Modifying a single key is enough to test a variant. Changing "lighting": "golden hour" to "lighting": "neon noir" keeps everything else constant. For an ad campaign, you generate 20 lighting variants without rewriting 20 sentences.
Gain in collaboration
An art director, a developer, and a growth manager read the same file. No paraphrasing, no loss. JSON becomes the single source of truth for the brief, versionable in Git like any technical artifact.
What is the typical structure of a video JSON prompt?
Six blocks cover 95% of needs. This taxonomy uses the fields documented by Runway for Gen-3 Alpha and the Google Veo guide. The structure below works for Veo 3, Sora 2, Runway Gen-3, and Kling 2.0.
{
"subject": "",
"action": "",
"scene": {
"location": "",
"time_of_day": "",
"weather": ""
},
"camera": {
"shot_type": "medium shot",
"angle": "eye-level",
"movement": "static",
"lens": "50mm"
},
"lighting": "",
"style": {
"genre": "",
"mood": "",
"color_palette": ""
},
"audio": "",
"duration_seconds": 8
}Critical fields according to the target model
- Veo 3: add
audio(dialogue, ambient sound) which has been natively supported since its public launch in May 2024. - Sora 2: specify
physics(improved physical realism) andcontinuityfor multi-shot plans. - Runway Gen-3 Alpha: emphasize
camera.movement, the model excels at complex movements. - Kling 2.0:
negative_promptfield recommended by Kling AI to eliminate artifacts.
How to write a JSON prompt for Veo 3?
Google DeepMind's Veo 3 generates 8 seconds in 1080p with synchronized audio according to the official announcement. The following JSON produces a directly usable coffee advertisement shot.
{
"subject": "young woman, brown hair, beige wool sweater",
"action": "pours an espresso into a white cup, looks up at the camera and smiles",
"scene": {
"location": "minimalist Scandinavian kitchen",
"time_of_day": "early morning",
"weather": "soft natural light"
},
"camera": {
"shot_type": "medium close-up",
"angle": "slight low angle",
"movement": "slow dolly in",
"lens": "35mm anamorphic"
},
"lighting": "window backlight, light fill right",
"audio": "sound of espresso pouring, silent ambiance",
"style": {
"genre": "lifestyle advertisement",
"mood": "cozy, premium",
"color_palette": "cream tones, light wood, sage green"
},
"duration_seconds": 8
}Tip recognized by Veo users
The audio field changes everything. A prompt without sound description often generates a generic soundtrack. Specifying “espresso sound, ambient silence” forces Veo 3 to compose a sound design mix consistent with the image.
How to adapt the JSON prompt to Sora 2 and Runway Gen-3?
Sora 2 and Runway Gen-3 respond differently to the same JSON. Sora 2 excels at physics and multi-shot continuity, Gen-3 at cinematic camera movements according to Runway Research. Adapting means weighting fields according to the model.
Sora 2 variant
{
"subject": "surfer, black wetsuit",
"action": "catches a wave, drops, bottom turn, exits with spray",
"physics": "realistic water, gravity, board mass",
"continuity": "single 12-second shot, no cuts",
"camera": {"shot_type": "wide tracking", "movement": "follow shot drone"},
"lighting": "Pacific sunset backlight",
"duration_seconds": 12
}Runway Gen-3 Alpha variant
{
"subject": "German Shepherd",
"action": "runs in a wheat field",
"camera": {
"shot_type": "low angle wide",
"movement": "handheld tracking, slight shake",
"lens": "24mm",
"speed": "slow motion 120fps"
},
"lighting": "golden hour, anamorphic lens flare",
"style": {"genre": "cinematic, Terrence Malick", "color_palette": "warm golden"}
}The difference? Sora 2 values physics and continuity. Gen-3 rewards camera details and aesthetics referenced to a filmmaker. Testing both on the same brief remains the best method for choosing.
How to integrate the JSON prompt into a professional workflow?
The transition from manual prompting to an industrialized pipeline multiplies generated volumes without diluting quality. Anthropic recommends in its Claude documentation the use of tagged structures for complex prompts, an identical principle for JSON. Three use cases dominate in agencies and for advertisers.
Case 1: Storyboard generated from a script
An ad script is broken down into 6 to 12 shots. Each shot becomes a JSON object. A Python script iterates through the list and sends each object to the Veo 3 or Runway API. Result: an animated storyboard in 20 minutes instead of 2 days.
Case 2: A/B variants for video ads
A paid social brief requires 8 versions of the same scene (palette, mood, framing). The JSON template remains identical, only three keys vary. On LinkedIn Ads or Meta Ads, this fuels a creative test at a near-zero marginal cost. To structure this type of creative-performance pipeline, see our growth marketing approach.
Case 3: Versioned prompt library
The creative team stores its best JSONs in a Git repo. Each success becomes a template. A new project starts by duplicating an existing JSON and modifying the subject. This capitalization transforms individual know-how into an agency asset.
What pitfalls to avoid with a JSON prompt?
Structure does not guarantee quality. A poorly written JSON produces worse results than a simple text prompt. Public feedback on Runway forums and r/StableDiffusion converges on four recurring errors.
Error 1: Over-specifying
Piling up 30 fields stifles the model. Veo 3 and Sora 2 handle 8 to 12 precise fields better than an exhaustive object. Keeping superfluous information out of the JSON improves rendering.
Error 2: Vague values
Writing "lighting": "beautiful light" is useless. Prefer "lighting": "north window backlight, 5600K, white bounce fill". Technical specificity guides the model.
Error 3: Ignoring duration
Veo 3 caps at 8 seconds per generation in May 2024, Sora 2 at 60 seconds, Runway Gen-3 Alpha at 10 seconds. A JSON requesting "duration_seconds": 30 from Veo 3 will be truncated or fail.
Error 4: Forgetting the negative
For Kling and Runway, adding a negative_prompt field with “distorted faces, inconsistent hands, random text” drastically reduces artifacts. The field is ignored by Veo and Sora but doesn't break anything.
What quality gains are observed between text and JSON prompts?
No public academic benchmark directly compares the two formats to date. However, qualitative feedback converges. For complex shots (multi-subject, defined camera movement, precise ambiance), JSON divides the number of iterations needed to achieve a valid rendering. An arXiv paper on prompt structuring shows that LLMs respond better to formatted instructions than to free text, a principle transferable to video models.
In practice, for an ad production with 12 shots to validate, switching to JSON typically reduces prompt-engineering time from 2-3 days to half a day. The API generation cost remains identical. The gain is therefore 100% human.
FAQ: JSON prompt and AI video generation
Does the JSON prompt also work for Midjourney or Stable Diffusion?
Partially. Midjourney v6 and Stable Diffusion XL accept JSON but flatten it into text before processing. The benefit is less clear than with Veo, Sora, or Runway, which are video models trained on field-structured descriptions.
Should the JSON be validated before sending it to the API?
Yes. A trailing comma or a missing quote will crash the parser. A linter like JSONLint or a simple Python json.loads() validates in two seconds. For a batch pipeline of 100 prompts, it's non-negotiable.
Can ChatGPT or Claude generate the JSON prompt?
Yes, and it's even recommended. Describe the shot in French, ask Claude or GPT-5 to produce a JSON compliant with a provided schema. This method reduces the writing time for a complex prompt from 10 minutes to 1 minute, with superior consistency.
Does the JSON prompt replace the traditional storyboard?
No, it complements it. The storyboard remains the client validation language. JSON becomes the technical execution layer between the validated storyboard and the video model. Both coexist in a mature agency workflow.
Which video model to choose in 2026 based on my use case?
For short ads with audio: Veo 3. For long, coherent shots with realistic physics: Sora 2. For cinematic camera control: Runway Gen-3 Alpha. For value for money on high volumes: Kling 2.0. The JSON prompt works on all four, which facilitates cross-comparisons.
Moving to Industrialization
The JSON prompt is not a fad: it is the contractual interface between a creative intention and a video model. Those who structure their prompts gain time, consistency, and reproducibility. Those who stick to text prompts compensate with dozens of iterations.
If you are building an AI video pipeline for your campaigns or product content, Uclic's AI support covers JSON template design, API integration, and production deployment. To go further on creative-performance management, our growth marketing offering handles upstream (brief, audience) and downstream (testing, scaling). Other AI and automation analyses are available on our blog.
