ultimate-ai-content-pipeline

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Ultimate AI Content Pipeline

终极AI内容流水线

Skill by ara.so — Marketing Skills collection.

由ara.so提供的技能 —— 营销技能合集。

Overview

概述

Ultimate AI Content Pipeline is a comprehensive TypeScript-based content automation system that transforms a single keyword into fully-researched articles and rendered videos. It combines:

Auto-research: Web scraping from TechCrunch, a16z, Twitter/X, LinkedIn
AI content generation: Multi-format content creation with Claude 3 and OpenAI
Video rendering: Automatic infographic and video generation using Remotion
Multi-language support: Simultaneous English and Vietnamese content generation

The system creates a complete content production pipeline from research to publication-ready assets.

终极AI内容流水线是一个基于TypeScript的综合性内容自动化系统，可将单个关键词转化为经过充分调研的文章和渲染完成的视频。它整合了以下功能：

自动调研：从TechCrunch、a16z、Twitter/X、LinkedIn进行网页抓取
AI内容生成：借助Claude 3和OpenAI创建多格式内容
视频渲染：使用Remotion自动生成信息图和视频
多语言支持：同时生成英文和越南语内容

该系统构建了从调研到可发布资产的完整内容生产流程。

Installation

安装

bash

undefined

bash

undefined

Clone the repository

git clone https://github.com/pennydinh/marketing-pineline-share.git cd marketing-pineline-share

Install dependencies

npm install

or

yarn install

or

pnpm install

undefined

pnpm install

undefined

Configuration

配置

Create a

.env.local

file in the project root:

bash

undefined

在项目根目录创建

.env.local

文件：

bash

undefined

AI Provider Keys

ANTHROPIC_API_KEY=your_anthropic_key OPENAI_API_KEY=your_openai_key

Research API Keys

RAPIDAPI_KEY=your_rapidapi_key

Optional: Custom endpoints

NEXT_PUBLIC_API_URL=http://localhost:3000

Remotion Configuration

REMOTION_TIMEOUT=120000

undefined

REMOTION_TIMEOUT=120000

undefined

Project Structure

项目结构

marketing-pineline-share/
├── src/
│   ├── app/              # Next.js app router
│   ├── components/       # React components
│   ├── lib/             # Core utilities
│   │   ├── ai/          # AI provider integrations
│   │   ├── research/    # Web scraping modules
│   │   └── video/       # Remotion video generation
│   ├── types/           # TypeScript definitions
│   └── utils/           # Helper functions
├── remotion/            # Remotion video templates
└── public/              # Static assets

marketing-pineline-share/
├── src/
│   ├── app/              # Next.js app router
│   ├── components/       # React components
│   ├── lib/             # Core utilities
│   │   ├── ai/          # AI provider integrations
│   │   ├── research/    # Web scraping modules
│   │   └── video/       # Remotion video generation
│   ├── types/           # TypeScript definitions
│   └── utils/           # Helper functions
├── remotion/            # Remotion video templates
└── public/              # Static assets

Core Components

核心组件

1. Research Module

1. 调研模块

The research module crawls and aggregates content from multiple sources:

typescript

import { researchTopic } from '@/lib/research/aggregator';

interface ResearchResult {
  articles: Article[];
  insights: string[];
  trends: string[];
  sources: string[];
}

// Research a topic automatically
const result = await researchTopic({
  keyword: 'AI marketing automation',
  sources: ['techcrunch', 'a16z', 'twitter', 'linkedin'],
  timeframe: '24h',
  language: 'en'
});

console.log(result.insights);
// ["AI adoption in marketing increased 45% YoY", ...]

调研模块可从多个来源抓取并聚合内容：

typescript

import { researchTopic } from '@/lib/research/aggregator';

interface ResearchResult {
  articles: Article[];
  insights: string[];
  trends: string[];
  sources: string[];
}

// 自动调研主题
const result = await researchTopic({
  keyword: 'AI marketing automation',
  sources: ['techcrunch', 'a16z', 'twitter', 'linkedin'],
  timeframe: '24h',
  language: 'en'
});

console.log(result.insights);
// ["AI adoption in marketing increased 45% YoY", ...]

2. AI Content Generation

2. AI内容生成

Generate content in multiple formats using Claude or OpenAI:

typescript

import { generateContent } from '@/lib/ai/content-generator';

interface ContentOptions {
  format: 'toplist' | 'pov' | 'case-study' | 'how-to';
  tone: 'expert' | 'friendly' | 'humorous';
  language: 'en' | 'vi';
  provider: 'claude' | 'openai';
}

// Generate a toplist article
const content = await generateContent({
  keyword: 'AI marketing tools',
  research: result, // from research module
  format: 'toplist',
  tone: 'expert',
  language: 'en',
  provider: 'claude'
});

console.log(content.title);
console.log(content.body);
console.log(content.metadata);

使用Claude或OpenAI生成多格式内容：

typescript

import { generateContent } from '@/lib/ai/content-generator';

interface ContentOptions {
  format: 'toplist' | 'pov' | 'case-study' | 'how-to';
  tone: 'expert' | 'friendly' | 'humorous';
  language: 'en' | 'vi';
  provider: 'claude' | 'openai';
}

// 生成榜单类文章
const content = await generateContent({
  keyword: 'AI marketing tools',
  research: result, // 来自调研模块
  format: 'toplist',
  tone: 'expert',
  language: 'en',
  provider: 'claude'
});

console.log(content.title);
console.log(content.body);
console.log(content.metadata);

3. Multi-Language Generation

3. 多语言生成

Create content in both English and Vietnamese simultaneously:

typescript

import { generateBilingual } from '@/lib/ai/bilingual';

const bilingualContent = await generateBilingual({
  keyword: 'content automation',
  research: result,
  format: 'how-to',
  tone: 'friendly'
});

// Access English version
console.log(bilingualContent.en.title);
console.log(bilingualContent.en.body);

// Access Vietnamese version
console.log(bilingualContent.vi.title);
console.log(bilingualContent.vi.body);

同时创建英文和越南语内容：

typescript

import { generateBilingual } from '@/lib/ai/bilingual';

const bilingualContent = await generateBilingual({
  keyword: 'content automation',
  research: result,
  format: 'how-to',
  tone: 'friendly'
});

// 访问英文版本
console.log(bilingualContent.en.title);
console.log(bilingualContent.en.body);

// 访问越南语版本
console.log(bilingualContent.vi.title);
console.log(bilingualContent.vi.body);

4. Video Generation with Remotion

4. 基于Remotion的视频生成

Transform content into video automatically:

typescript

import { renderVideo } from '@/lib/video/renderer';
import { bundle } from '@remotion/bundler';
import { renderMedia } from '@remotion/renderer';

interface VideoConfig {
  template: 'infographic' | 'slideshow' | 'reels';
  aspectRatio: '16:9' | '9:16' | '1:1';
  duration: number;
}

// Generate video from content
const video = await renderVideo({
  content: content,
  config: {
    template: 'infographic',
    aspectRatio: '9:16', // For Reels/TikTok
    duration: 30
  }
});

console.log(video.outputPath);
// => /public/videos/output_12345.mp4

自动将内容转化为视频：

typescript

import { renderVideo } from '@/lib/video/renderer';
import { bundle } from '@remotion/bundler';
import { renderMedia } from '@remotion/renderer';

interface VideoConfig {
  template: 'infographic' | 'slideshow' | 'reels';
  aspectRatio: '16:9' | '9:16' | '1:1';
  duration: number;
}

// 从内容生成视频
const video = await renderVideo({
  content: content,
  config: {
    template: 'infographic',
    aspectRatio: '9:16', // 适配Reels/TikTok
    duration: 30
  }
});

console.log(video.outputPath);
// => /public/videos/output_12345.mp4

API Routes

API路由

POST /api/research

Research a topic and gather insights:

typescript

// Request
const response = await fetch('/api/research', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    keyword: 'AI content creation',
    sources: ['techcrunch', 'twitter'],
    timeframe: '24h'
  })
});

const data = await response.json();
// { articles: [...], insights: [...], trends: [...] }

调研主题并收集洞察信息：

typescript

// 请求示例
const response = await fetch('/api/research', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    keyword: 'AI content creation',
    sources: ['techcrunch', 'twitter'],
    timeframe: '24h'
  })
});

const data = await response.json();
// { articles: [...], insights: [...], trends: [...] }

POST /api/generate

Generate content from research:

typescript

// Request
const response = await fetch('/api/generate', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    keyword: 'marketing automation',
    research: researchData,
    format: 'toplist',
    tone: 'expert',
    language: 'en',
    provider: 'claude'
  })
});

const content = await response.json();
// { title: "...", body: "...", metadata: {...} }

基于调研结果生成内容：

typescript

// 请求示例
const response = await fetch('/api/generate', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    keyword: 'marketing automation',
    research: researchData,
    format: 'toplist',
    tone: 'expert',
    language: 'en',
    provider: 'claude'
  })
});

const content = await response.json();
// { title: "...", body: "...", metadata: {...} }

POST /api/render-video

Render video from content:

typescript

// Request
const response = await fetch('/api/render-video', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    contentId: content.id,
    template: 'reels',
    aspectRatio: '9:16'
  })
});

const video = await response.json();
// { videoUrl: "...", duration: 30, size: "..." }

基于内容渲染视频：

typescript

// 请求示例
const response = await fetch('/api/render-video', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    contentId: content.id,
    template: 'reels',
    aspectRatio: '9:16'
  })
});

const video = await response.json();
// { videoUrl: "...", duration: 30, size: "..." }

Complete Pipeline Example

完整流水线示例

End-to-end content generation workflow:

typescript

import { ContentPipeline } from '@/lib/pipeline';

async function createCompleteContent(keyword: string) {
  const pipeline = new ContentPipeline({
    anthropicKey: process.env.ANTHROPIC_API_KEY!,
    openaiKey: process.env.OPENAI_API_KEY!,
    rapidApiKey: process.env.RAPIDAPI_KEY!
  });

  // Step 1: Research
  console.log('📡 Researching topic...');
  const research = await pipeline.research({
    keyword,
    sources: ['techcrunch', 'a16z', 'twitter'],
    timeframe: '24h'
  });

  // Step 2: Generate bilingual content
  console.log('🧠 Generating content...');
  const content = await pipeline.generateBilingual({
    keyword,
    research,
    format: 'toplist',
    tone: 'expert'
  });

  // Step 3: Render videos
  console.log('🎬 Rendering videos...');
  const videos = await pipeline.renderVideos({
    content: content.en,
    templates: [
      { template: 'reels', aspectRatio: '9:16' },
      { template: 'infographic', aspectRatio: '16:9' }
    ]
  });

  return {
    research,
    content,
    videos
  };
}

// Execute pipeline
const result = await createCompleteContent('AI marketing automation');
console.log('✅ Pipeline complete!');
console.log(`Generated ${result.videos.length} videos`);

端到端内容生成工作流：

typescript

import { ContentPipeline } from '@/lib/pipeline';

async function createCompleteContent(keyword: string) {
  const pipeline = new ContentPipeline({
    anthropicKey: process.env.ANTHROPIC_API_KEY!,
    openaiKey: process.env.OPENAI_API_KEY!,
    rapidApiKey: process.env.RAPIDAPI_KEY!
  });

  // 步骤1：调研
  console.log('📡 正在调研主题...');
  const research = await pipeline.research({
    keyword,
    sources: ['techcrunch', 'a16z', 'twitter'],
    timeframe: '24h'
  });

  // 步骤2：生成双语内容
  console.log('🧠 正在生成内容...');
  const content = await pipeline.generateBilingual({
    keyword,
    research,
    format: 'toplist',
    tone: 'expert'
  });

  // 步骤3：渲染视频
  console.log('🎬 正在渲染视频...');
  const videos = await pipeline.renderVideos({
    content: content.en,
    templates: [
      { template: 'reels', aspectRatio: '9:16' },
      { template: 'infographic', aspectRatio: '16:9' }
    ]
  });

  return {
    research,
    content,
    videos
  };
}

// 执行流水线
const result = await createCompleteContent('AI marketing automation');
console.log('✅ 流水线执行完成！');
console.log(`已生成 ${result.videos.length} 个视频`);

Custom Remotion Components

自定义Remotion组件

Create custom video templates:

typescript

// remotion/compositions/CustomTemplate.tsx
import { AbsoluteFill, Sequence, useCurrentFrame } from 'remotion';

export const CustomTemplate: React.FC<{
  title: string;
  points: string[];
  duration: number;
}> = ({ title, points, duration }) => {
  const frame = useCurrentFrame();
  
  return (
    <AbsoluteFill style={{ backgroundColor: '#1a1a1a' }}>
      <Sequence from={0} durationInFrames={30}>
        <div style={{ 
          fontSize: 48, 
          color: 'white',
          textAlign: 'center',
          marginTop: 100
        }}>
          {title}
        </div>
      </Sequence>
      
      {points.map((point, index) => (
        <Sequence 
          key={index}
          from={30 + (index * 90)}
          durationInFrames={90}
        >
          <div style={{ 
            fontSize: 32, 
            color: 'white',
            padding: 40
          }}>
            {point}
          </div>
        </Sequence>
      ))}
    </AbsoluteFill>
  );
};

typescript

// remotion/index.ts
import { registerRoot } from 'remotion';
import { CustomTemplate } from './compositions/CustomTemplate';

registerRoot(() => {
  return (
    <>
      <Composition
        id="CustomTemplate"
        component={CustomTemplate}
        durationInFrames={300}
        fps={30}
        width={1080}
        height={1920}
      />
    </>
  );
});

创建自定义视频模板：

typescript

// remotion/compositions/CustomTemplate.tsx
import { AbsoluteFill, Sequence, useCurrentFrame } from 'remotion';

export const CustomTemplate: React.FC<{
  title: string;
  points: string[];
  duration: number;
}> = ({ title, points, duration }) => {
  const frame = useCurrentFrame();
  
  return (
    <AbsoluteFill style={{ backgroundColor: '#1a1a1a' }}>
      <Sequence from={0} durationInFrames={30}>
        <div style={{ 
          fontSize: 48, 
          color: 'white',
          textAlign: 'center',
          marginTop: 100
        }}>
          {title}
        </div>
      </Sequence>
      
      {points.map((point, index) => (
        <Sequence 
          key={index}
          from={30 + (index * 90)}
          durationInFrames={90}
        >
          <div style={{ 
            fontSize: 32, 
            color: 'white',
            padding: 40
          }}>
            {point}
          </div>
        </Sequence>
      ))}
    </AbsoluteFill>
  );
};

注册模板：

typescript

// remotion/index.ts
import { registerRoot } from 'remotion';
import { CustomTemplate } from './compositions/CustomTemplate';

registerRoot(() => {
  return (
    <>
      <Composition
        id="CustomTemplate"
        component={CustomTemplate}
        durationInFrames={300}
        fps={30}
        width={1080}
        height={1920}
      />
    </>
  );
});

Content Formats

内容格式

Toplist Format

榜单格式

typescript

const toplist = await generateContent({
  format: 'toplist',
  // Generates: "Top 10 AI Marketing Tools in 2024"
  // Structure: Numbered list with descriptions, pros/cons
});

typescript

const toplist = await generateContent({
  format: 'toplist',
  // 生成示例："2024年十大AI营销工具"
  // 结构：带描述、优缺点的编号列表
});

POV (Point of View) Format

POV（观点）格式

typescript

const pov = await generateContent({
  format: 'pov',
  // Generates: Opinion piece with strong viewpoint
  // Structure: Thesis → Arguments → Conclusion
});

typescript

const pov = await generateContent({
  format: 'pov',
  // 生成示例：带有明确观点的评论文章
  // 结构：论点 → 论据 → 结论
});

Case Study Format

案例研究格式

typescript

const caseStudy = await generateContent({
  format: 'case-study',
  // Generates: Real-world example analysis
  // Structure: Problem → Solution → Results
});

typescript

const caseStudy = await generateContent({
  format: 'case-study',
  // 生成示例：真实案例分析
  // 结构：问题 → 解决方案 → 结果
});

How-To Format

教程格式

typescript

const howTo = await generateContent({
  format: 'how-to',
  // Generates: Step-by-step tutorial
  // Structure: Intro → Steps → Tips → Conclusion
});

typescript

const howTo = await generateContent({
  format: 'how-to',
  // 生成示例：分步教程
  // 结构：引言 → 步骤 → 技巧 → 结论
});

Troubleshooting

故障排除

API Rate Limits

API速率限制

typescript

import { retry } from '@/lib/utils/retry';

// Automatic retry with exponential backoff
const content = await retry(
  () => generateContent(options),
  { maxRetries: 3, delay: 1000 }
);

typescript

import { retry } from '@/lib/utils/retry';

// 带指数退避的自动重试
const content = await retry(
  () => generateContent(options),
  { maxRetries: 3, delay: 1000 }
);

Video Rendering Timeout

视频渲染超时

bash

undefined

bash

undefined

Increase timeout in .env.local

在.env.local中增加超时时间

REMOTION_TIMEOUT=300000 # 5 minutes

undefined

REMOTION_TIMEOUT=300000 # 5分钟

undefined

Memory Issues with Large Research

大型调研结果的内存问题

typescript

// Paginate research results
const research = await researchTopic({
  keyword: 'AI tools',
  maxArticles: 20,  // Limit results
  summarize: true   // Pre-summarize content
});

typescript

// 分页获取调研结果
const research = await researchTopic({
  keyword: 'AI tools',
  maxArticles: 20,  // 限制结果数量
  summarize: true   // 预先汇总内容
});

Claude API Errors

Claude API错误

typescript

// Fallback to OpenAI
try {
  const content = await generateContent({
    provider: 'claude',
    ...options
  });
} catch (error) {
  console.log('Claude failed, trying OpenAI...');
  const content = await generateContent({
    provider: 'openai',
    ...options
  });
}

typescript

// 降级到OpenAI
try {
  const content = await generateContent({
    provider: 'claude',
    ...options
  });
} catch (error) {
  console.log('Claude调用失败，尝试OpenAI...');
  const content = await generateContent({
    provider: 'openai',
    ...options
  });
}

Development Commands

开发命令

bash

undefined

bash

undefined

Start development server

启动开发服务器

npm run dev

Build for production

构建生产版本

npm run build

Start production server

启动生产服务器

npm start

Preview Remotion compositions

预览Remotion模板

npm run remotion:preview

Render specific Remotion video

渲染指定Remotion视频

npm run remotion:render -- CustomTemplate output.mp4

undefined

npm run remotion:render -- CustomTemplate output.mp4

undefined

Best Practices

最佳实践

Cache research results to avoid redundant API calls
Use bilingual generation when targeting multiple markets
Pre-process content before video rendering for better results
Monitor API usage to stay within rate limits
Test video templates with Remotion preview before batch rendering
Store generated content in a database for reuse and scheduling

This skill enables AI coding agents to effectively use the Ultimate AI Content Pipeline for automated, AI-powered content creation workflows.

缓存调研结果，避免重复API调用
使用双语生成，覆盖多语言市场
视频渲染前预处理内容，提升输出效果
监控API使用情况，避免超出速率限制
批量渲染前用Remotion预览测试模板
将生成的内容存储到数据库，便于复用和调度

该技能可让AI编码代理有效利用终极AI内容流水线，实现自动化的AI驱动内容创作工作流。