AIGC 专题报告：从文生图到文生视频技术框架与商业化()()S0350521090003 S()2023 11 2 2%1M 3M 12M-8.60 14.79 50.09-5.72-10.23 1.47 Roblox AIGC Roblox(RBLX.N)*2023-09-24 02121 AI+*IT 2023-09-14-W 00020 2023H1 AI*IT 2023-09-08-S 9999.HK*2023-09-03 002315 2023H1*2023-09-03-20%-10%0%10%20%30%40%50%60%港股互联网指数 恒生指数 31 GAN Diffusion model GAN FID Inception Distance score FID FID GAN GAN GAN GAN GAN A100GPU 120 DALL-E 18 200 Parti 100 2 GPU/API/Midjourney 0.030.04 0.05 30%40%20%Stable Diffusion SD Stable Diffusion 1-10B 70B 1 A100 RTX3040 IBS3060 50001w/10 100 DAU 143 A100+AI/C C B AI AI ARPU 3 OpenAI Meta Midjourney Stability AI AI Midjourney Adobe Stable Diffusion OpenAI DALL-E3 ChatGPT 4 AI C B AI Meta Adobe 5 4.5 Stable Diffusion GAN Diffusion AI GAN GAN Transformer GAN CLIP Diffusion.22 Midjourney&.32 5 Midjourney Adobe Open AI DALL-E3 imagen PartiMeta CM3Leon Midjourney Stability AI Stable Diffusion Stability AI SDXL1.0 Clipdrop Stability AI AI Adobe Firefly Adobe AIGC Pixpic B AI AI PicSo.49 CogVi de o:NUWA Phenaki Imagen Video Meta Make-A-Video Magic Video NVIDIA Zeroscope Runway Gen-1 Runway Gen-1&Gen-2 Synthesia AI Lumen5 6 7 St a bl e D i f f u s i o n Disco Diffusion Google partiOpenAI GLIDE Midjourney Meta Make-A-Scene Google Imagen ERNIE-VILG2.02021 5 2022 3 2022 6 2022 4 StableDiffusion OpenAI DALL E2 2023 3 2022 5 CogView22023 6 AIAct AI 2023 7 2021 11 CogView/Text-to-Image AI 3 Stable Diffusion Midjourney GAN GAN 2021 2022 AI 2022 11 AI MiracleVision2022 8 Disco Diffusion 2021 10 ERNIE-VILGStability AI StableDiffusion 1 36 AI 2019 3 Nvidia GauGAN2021 1 OpenAI DALL E Google XMC-GANNvidia GauGAN2 8 Text-to-Video AI 3 Runway Meta Model Scope ZeroScope GAN/VAE/Flow-Based 2020 2016 2019 20162021.04 GODIVA GAN/VAE/Flow-Based IT 36 2 2016.10 VGAN2016.01 2017.01 2018.01 2019.01 2020.01 2021.01 2022.012023.012021.11 NUWA 2023.02 RunwayGen-12023.03 RunwayGen-22022.05 CogVideo2022.07 NUWA-Infinity2022.09 Meta Make-A-Video2022.10 GooglePhenaki2022.10 Google Imagen Video2023.03 NUWA-XL2023.04 Nvidia Video LDM2023.11 MagicVideo2023.05 Nvidia PYoCo2017.08 TGAN2017.12 MoCoGAN2022.02 DIGAN2020 2019.09 DVD-GAN 2017.04 VideoGPT 9 GAN Diffusion GAN)Generator GAN Discriminator Transformer Encoder Decoder 50%token GAN Relativistic GAN GAN Relativistic GAN GAN 10 GAN D2GAN-GAN GAN-StackGAN+DF-GAN DALL-E CogView CogView2 Parti CM3leon Stable Diffusion GLIDE DALL-E 2VGAN TGAN VideoGPT MoCoGAN DVD-GAN DIGANCogVideo GODIVA NUWA PhenakiVideo Diffusion Model Make-A-Video Imagen Video Tune-A-Video Dreamix NUWA-XL Text2Video-Zero VideoLDM PYoCo Midjourney Stable Diffusion Runway 10 GAN Stable Diffusion GLIDE DALL-E 2DALL-E CogView CogView2 Parti CM3leonStackGAN+DF-GAN 11 AI 2022 10 2023 1 NIST RMF 2023 6 AI Act 2023 7 AI 12G AN StackGAN+:Realistic Image Synthesis with Stacked Generative Adversarial Networks Han Zhang DF-GAN:A Simple and Effective Baseline for Text-to-Image Synthesis Ming Tao GANs GAN,Generative Adversarial Networks DF-GAN UPBlock stackGAN+GoGAN GAN GAN G1 GAN GAN mode collapse GAN GAN GAN GAN GAN RNN Bi-LSTM stackGAN+13G AN 3 GAN*UCF-101 IS GAN IS*VGAN 2016.10 GAN 3D 2D 8.31.09VideoGPT 2017.04 VAE GAN 24.69.30TGAN 2017.08 3D 11.85.07MoCoGAN 2017.12MoCoGAN 12.42.07DVD-GAN 2019.09 GAN DVD-GAN 48 256*256 DVD-GAN Kinetics-600 MoCoGAN 32.97 1.7DIGAN 2022.02 a INR b 128 128 128 48 80 29.71.53 14 Auto-regressive Model Transformer Transformer Encoder Decoder tokens Transformer VAE GAN Transf orm er Hierarchical Text-Conditional Image Generation with CLIP Latents Aditya Ramesh CogView:Mastering Text-to-Image Generation via Transformers Ming Ding Google Research 1 2 Meta CM3leon VQ-VAE CogView ERNIE-ViLG VQ-VAE+Transformer token AE(Auto-Encoder)token token token GPT ViT-VQGAN Parti ViT-VQGAN Transformer GPT-2 15 GANs G AN NUWA:Visual Synthesis Pre-training for Neural visUal World creAtion Chenfei Wu CogVideo:Large-scale Pretraining for Text-to-Video Generation via Transformers Wenyi Hong PixelCNN PixelRNN Image Transformer iGPT Video Transformer VQ-VAE GODIVA VideoGPT token NUWA CogVideo Phenaki 16 Hierarchical Text-Conditional Image Generation with CLIP Latents Aditya Ramesh Denoising Diffusion Probabilistic Models Jonathan Ho High-Resolution Image Synthesis with Latent Diffusion Models Robin Rombach Diffusion Model CLIP DALLE-2 Diffusion Model CLIP CLIP Latent Space:Stable Diffusion Latent Space Latent Diffusion Models GPU 10 1 GAN 2 CLIP 3 17C L I P Nvidia Developer Improving Language Understanding by Generative Pre-Training Learning Transferable Visual Models From Natural Language Supervision CLIP Contrastive Language-image Pre-training-Text Encoder Image Encoder-CLIP 1 CLIP 2 CLIP-CLIP OpenAI 4-CLIP 18 4 Video Diffusion Model Video Diffusion Model Google 2022.04 Video Diffusion Model Variance of minibatch Make-A-Video Meta 2022.09Make-A-Video T2I T2V-Imagen Video Google 2022.10Imagen Video Imagen Tune-A-Video 2022.12Tune-A-Video T2I T2V T2V Gen-1 Runway 2023.02Gen-1 Dreamix Google 2023.02Dreamix NUWA-XL 2023.03NUWA-XL Diffusion over Diffusion)NUWA-XL Text2Video-ZeroPicsart AI Resarch(PAIR)UT Austin U of Oregon UIUC2023.03Text2Video-Zero VideoLDM 2023.04VideoLDM LDM DM PYoCo 2023.05PYoCo eDiff-I 19 GAN+Transformer GAN token GAN FID Inception Distance score FID FID GAN GAN GAN GAN GAN A100GPU 120 DALL-E 18 200 Parti 100 GLIDE 35 DALLE-2 35 Imagen 34 Re-Imagen 36 DALLE 120 Cogview 40 Cogview2 60 Parti 200 DFGAN 0.19 20 D i f fu s i o n 36kr OpenAI Consistency Models Meta CM3Leon token Transformer 4.88 FID GAN Adobe GigaGAN GAN GAN GAN GAN 21 24 Runway-Prompt-prompt prompt A I GC Prompt a girl is chasing a cat on the grass,full shot,classicPrompt a man is walking the dog in the school,wide angle,cinematic 22 23+GPU CAD LOGO CT API 24 Mi dj ourney Midjourney 1 A100 1.8-1.9 3 6000 OPENAI Midjourney 80-90/Midjourney 11 1000/1000 2000 500 A100 250 400 80%10%5%5%Midjourney 75008000/0.030.04/1 0.05/30%-40%20%25 GITHUB the gamer station GAN GAN 1/41/3 Meta CM3Leon Parti 200 DALLE 120 1.5 GPU OpenAI Midjourney 80-90%10-20%24 24 24 24 24 2 4 24 26 Midjourney C B AI AI 27 51TCO NLP GPTBERT(Large)GPT-2 T5 GPT-3 PanGu LaMDA ERNIE 3.0ERNIE 3.0 ZeusGLM-130B GPT-4 110M 340M 1.5B 11B 175B 200B 137B 10B 130B 5GB 40GB 750G 45TB 1.1TB 1.56TB 4TB 4TB DFGAN DALL-E Cogview Cogview2 Imagen GLIDE DALL-E2StableDiffusionParti CM3Leon SDXL1.0 19M 12B 4B 6B 3.4B 3.5B 3.5B 800M 20B 7B 3.3B 400M 5B 340M Stable Diffusion SD Stable Diffusion Stable Diffusion SDXL1.0 AI SD AI 1-10B 70B 1 A100 RTX3040 IBS3060 50001w/1 0.1 C B AI 300 C 20%B 60%16%24%AI Stable Diffusion Disco Diffusion 60%16%14%7%2%1%AI AI 10 10-100 100-500 500-3000 3000 28&AI/69/139/339/DALLE1&2 AI 15 115 AI 10/30/100 5/12/20 Stable Diffusion WHEEAI Midjourney AI Basic/Standard/Pro 10/30/60/AI/99/199/1299/4699/Designs.ai Basic/Pro 19/69/AI 9.9 10 2/2/Lumen5 Basic/Starter/Professional 19/59/149/PicSo AI 9.99/49.99/Runway credits Gen-1 Gen-2$12/-625$28/-2250 6Open artAI 20/200/800/5000 5/30/100/500 Synthesia 29/25%264/29 Discord 1 2 3 4 5 Midjourney 10 20 30 40 50 30MJ 9-40 SD 10 GB 3.7 7.5 11.2 14.9 18.6 11.2 A100 GB 40 40 40 40 40 40 1 0.14 0.29 0.43 0.57 0.71 0.43A100 7 GPU 1/7 A100 DAU 100 300 500 800 1000 700MJ 2023 5 1500 DAU/50%1000 3000 5000 8000 10000 7000Google 10 10 5000 10 1 1 1 1 1 1 2 1000 3000 5000 8000 10000 7000/=1 x 2 143 857 2143 4571 7143 3000 10 3.7GB FP32 FP32 4 10*4*108/1024/1024/1024=3.7GB A100 40GB/80GB 40GB A100 GPU 7 40/7=5.73.7 A100 1/7 Google 10 30 1 2 3 4 5 100 150 200 250 300Runway GEN2 100 GB 37.3 55.9 74.5 93.1 111.8 A100 GB 40 40 40 40 40 1 1 2 2 3 3DAU 10 50 100 300 500 DAU/50%200 1000 2000 6000 10000 2 1 1 1 1 1 2 200 1000 2000 6000 10000/=1 x 2 200 2000 4000 18000 30000 100 37GB FP32 FP32 4 100*4*108/1024/1024/1024=37GB A100 40GB/80GB 40GB Google 10 2 5 31 AI AI MidjourneyStable DiffusionRunwayAdobe Firefly/C AI C B ARPU 32 33 DALL-EDisco DiffusionCogView GLIDEERNIE-VILG Make-A-SceneMidjourney DALL-E-2 Stable Diffusion Imagen CogView2 PartiERNIE-VILG 2.0CM3leon DALL-E-3 2021.02 2021.05 2021.11 2021.11 2021.12 2022.03 2022.03 2022.04 2022.04 2022.05 2022.04 2022.06 2023.03 2023.07 2023.10OpenAI OpenAI OpenAI Meta Midjourney OpenAI RunwayGoogle Google Meta OpenAI Diffusion Model 12B 4B 3.5B 10B 4B 3.5B 1B 1.5B 6B 20B 24B350M/760M/7B 250M-30M-145M-35M-900M LAION-5B460M-400MLaion-30M-LAION-400M FIT400M JFT-4B dataset170M-Shutterstock datasets Zero-shot FID-30k 27.5 27.1 12.24 14.7 11.84 10.39 12.6 7.27 24.00 7.23 6.7514.20/6.61/4.88 Speed s-15-25-3.7 9.1-5 34 prompt DALL-EDreamStudio Stable Diffusion v2.1 Midjourney V5 Adobe Firefly Tiamat Midjourney V5 Adobe Firefly Tiamat DALL-E DreamStudio Tiamat Adobe MJ Tiamat 3D 3D Midjourney Tiamat Adobe Firefly Dreamstudio DALL-E M i d jo ur n e y A d o be 35O pen AI DALL-E3 OpenAI 36kr DALL-E2021 2 DALL-E22022 4 GLIDE2021 11 DALL-E 120-Transformer GLIDE 35 Guided Diffusion Autoencoder 3.5 billion DALLE-FID GLIDE DALL-E GLIDE DALL-E 2 35 CLIP prior img decoder CLIP text encoder img encoder DALL-E 2 Prior CLIP Decoder CLIP DALL-E2 Diffusion CLIP DALL-E 2 CLIP OpenAI GLIDE DALL-E32023 9 DALL-E 3 DALLE-2 DALLE-3 1 DALL-E 3 2 DALL-E 3 prompt DALL-E3 ChatGPT ChatGPT 3 10 ChatGPT Plus Enterprise DALLE-2 DALLE-3 36 i m agen Part i Google Research AssemblyAI MarsBit ImagenPartiParti 2022 6 Parti Transformer ViT-VQGAN Parti ViT-VQGAN Parti 350M 750M 3B 20B 1 Parti OpenAI DALL-E 2 2 3B 20B/63.2%75.9%3 20B Zero-Shot FID 30k Parti-3B 8.10 Parti-20B 7.23 A portrait photo of a kangaroo wearing an orange hoodie and blue sunglasses standing on the grass in front of the Sydney Opera House holding a sign on the chest that says Welcome Friends!Imagen 2022 5 15 Imagen Transformer T5 Imagen Imagen 64x64 Imagen 1024x1024 Zero-Shot FID 30k 7.27 Open AI Diffusion DALL-E2 10.39 Imagen DALL-E 2 37M e t a C M 3 L e o n CM3leon CM3leon AI CM3leon Meta CM3leon Transformer CM3leon 4.88 FID Parti CM3leon 7 16 Meta CM3leon 30 token SFT CM3Leon Meta Shutterstock 70 OpenAI DALL-E2 CM3Leon Transformer CM3Leon Retrieval-Augmented Multimodal Language Modeling Meta CM3Leon CM3Leon SFT Meta CM3leon QA MetaAI Scaling Autoregressive Multi-Modal Models:Pretraining and Instruction Tuning AI IT CM3leon Transformer 38M i dj ourney aituts Discord Linkedin Midjourney V1 V5.2 Discord Midjourney Discord Midjourney Midjourney Midjourney Midjourney AI David Holz 2021 Midjourney Discord 1000 1 Midjourney CLIP Diffusion 2022 3 2022 4 2022 7 25 2022 11 5 2023 3 16 0.2 0.3 0.4 0.51.02.03.04.56.010.015.014.013.514.015.015.70.02.04.06.08.010.012.014.016.018.0Jul-22Aug-22Sep-22Oct-22Nov-22Dec-22Jan-23Feb-23Mar-23Apr-23May-23Jun-23Jul-23Aug-23Sep-23Oct-23Midjourney Discord 39St abi l i t yAI St abl e Di f f usi on StabilityAI 36 IT 51CTO SD fine-tune SD Midjourney Midjourney SD Github 2022 8 Stability AI Stable Diffusion 1.0 11 Stable Diffusion 2.0 2023 2 ControlNet Stable Diffusion 2023 4 Stable Diffusion SDXL 6 SDXL 0.9 Stable Diffusion StableDiffusion 2023 7 Stability AI SDXL1.0 33 66 Stability AI SDXL1.0 100 1024 1024 Stable Diffusion Stability AI AI 2022 8 2022 10 Stability AI Lightspeed Venture Partners Coatue Management 1.01 10 AI Stable Diffusion Latent Space diffusion Perceptual Compression VAE GPU 10 40S t a b i l i t y A I SD X L 1.0 Reddit Stable Diffusion 1.5 Midjourney SDX1.0 MJ Stable Diffusion 1.5 SDXL1.0A robot holding a sign in 1930s Walt Disney styleMargot Robbie and Keanu Reeves,film grain,action movie,RAW 4K UHD,masterpieceUkiyo-e Sakura Blossom:A blank greeting card in ukiyo-e style depicting a serene landscape with cherry blossom trees in full bloom,colorful flowers carpeting the ground,a meandering river,and birds chirping joyfully in the backgroundanime guy,short silver hair,black white silver gold clothes,comet,asteroids in background,comet in background,silver eyes,serious/neutral expression,serious/neutral face,forehead armor headband silver and gold,flying,cool,creative,powerful look,4ksoccer player in white jersey celebrating his goal,in the style of zeiss batis 18mm f/2.8 41 Clipdrop Init ML AI 10 AI Init ML 2020 2023 3 AI Stable Diffusion Stability.AI 2022 6 Stability.AI SDXL 0.9 Stable Diffusion-Clipdrop SDXL 0.9 Clipdrop 2023 7 26 Stability.AI SDXL 1.0 Clipdrop 2023 7 1500 Stable Diffusion XL Stable Diffusion Clipdrop 3D Reimage XL Uncrop Clipdrop AIGC AI Clipdrop Free Pro 9/7/Pro Stable Diffusion XL API API credit credit Clipdrop Stability AI AI Clipdrop Clipdrop AIGC Clipdrop AIGC AI Clipdrop Crunchbase eu startups stability ai Lab 212 Similarweb 0510152025303502004006008001000120014001600Clipdrop&42A d o b e F i re f l y A d o b e Adobe 36 prompt Firefly AI Firefly Photoshop Adobe Photoshop AI Photoshop AI Adobe Stable Diffusion Midjourney FireFly Adobe 3 Adobe AI Firefly Adobe Stock 6 Adobe Firefly Photoshop beta 6 Adobe AI Firefly Firefly Firefly Adobe Express Firefly Firefly Adobe AI 100 3D Firefly AI Adobe Stock 2023 9 Adobe AIGC Firefly AI AI 25 Adobe Firefly 5 Firefly AI Midjourney AI Firefly 3D 43 AI 2022 8 AI+ERNIE-ViL 2.0 ERNIE-ViLG Prompt AI AI 2/0.180.25/AI 2/2/2/80/9.9 200/23.9 800/79.9 1/899 AI 3 AI AI 900 9 3 5 AI AI 2300 8.5 3 10 AI AI 6000 8 3 44 AIGC Pixpic App TechWeb AIGC 2022 11 AI AI AI AI AIGC Pixpic AI AI Pixpic AI 5 Pixpic Google Play 500 Tag&5 5/10 0.5/12/30 0.4/20/100 0.2/45 B AI leiphone WHEE 2022 AIGC 2022 AIGC 2023 6 19 AI MiracleVision 6 WHEE WinkStudio 2.0 DreamAvatar AI RoboNeo AI B ToB AIGC+C+B WHEE AI AI AI AI AI MiracleVision Diffusion MiracleVision 2023 7 10 MiracleVision MiracleVision MiracleVision MiracleVision MiracleVision MiracleVision API SDK WHEE 4 46 App App AI 820 AI 38/AI AI 3.9/3.9/3.9/9.9/Y2K 3.9/3.9/0.9/3.9/6.9/9.9/0.9/AI 50 5 10/6.6 100 10 10/9.9 100 AI 4 AI 6 AI 2 VIP 120/AI 1 VIP VIP 12 98 88 128 18 WHEE 47 AI 36 TechVerse AI C 9.9 20 2020 20 AI 36 28 8 Lensa Stable Diffusion+LAION-5B LoRA Stable Diffusion Diffusion Lensa 1020 20 AI Lensa AI 35.99/Lensa 50/3.99 100/5.99 200/7.99 9.9 10 2/2/5/1 60/6 Stable Diffusion 7 20+48 PicSo PicSo AppStore PicSo 2022 iOS IP AI Girl tags AI Art AI Girl PicSo 1 4 9 100 9.99/49.99/paypal 2022.10.20/1.3.1 App Store 2022.12.20/1.8.0 2023.2.14/1.10.1 2023.2.21/1.11.0 AI Girl 2023.6.21/1.15.0 49 50 6 Token AIGC/AI GODIVA N WA Cogvideo Make-A-Video Imagen Video Phenaki MagicVideo Tune-A-Video Text2Video-Zero Runway Gen-1 NUWA-XL VideoLDM PYoCo 2021.04 2021.11 2022.05 2022.09 2022.10 2022.10 2022.11 2022.12 2023.03 2023.02 2023.03 2023.04 2023.05 Meta Google Google Picsart AI Resarch(PAIR)UT Austin U of Oregon UIUCRunway Nvidia Nvidia AR()AR()Transformer Diffusion Model Transformer Diffusion Model Diffusion Model Diffusion Model Diffusion Diffusion Model Diffusion Model Diffusion ModelTransformer Diffusion Model Diffusion Model 0.9B 9.4B 11.6B 1.8B 3.1B Howto100M 1.36-2.9M-727K 241K-5.4M-2.3B-20M-14M-60M-LAION-400M-15M-50M-LAION-400M-Laion 5B 10M 50M Webvid10M 7M-UCF-101 MSR-VTTDAVIS 42 140 Prompt240M 6.4M clips166 38000 1440 1080 10.7M-52K 1.2B-22.5M-128*128 480*480 1280*768 1280*768 256*256 512*512 256*256 1280*2048 4 5 5 2-2.5 11 Zero-Shot Zero-Shot 32/24/24/30/24-32 UCF-101 IS 23.55/25