Competitor Research Automation

Apify → Claude AI → Google Sheets → GitHub Pages

āœ“ 100% Free Tools

šŸ”„ Automation Pipeline

šŸ›’
Data Sources
Amazon/Google/YT/IG
→
šŸ¤–
Apify Scrapers
Free scraping actors
→
🧠
Claude AI
Demand Signal Analysis
→
šŸ“Š
Google Sheets
Apps Script DB
→
🌐
GitHub Pages
Free hosting
šŸ¤– Scraper
🧠 AI Analysis
šŸ“Š Signal Matrix
āš™ļø Setup Guide
šŸ“œ Scripts

šŸ“¦ Select Data Sources

Choose which platforms to scrape competitor data from (Apify free actors)

šŸ›’
Amazon Reviews
⭐
Google Reviews
ā–¶ļø
YouTube Comments
šŸ“ø
Instagram Comments

šŸ“‹ Scraper Output

Live data from Apify actors

// Output will appear here after scraping starts... // Free Apify Actors used: // - amazon-reviews-scraper // - google-maps-reviews-scraper // - youtube-comments-scraper // - instagram-comments-scraper

🧠 Claude AI Analyzer

Paste reviews/comments → Claude extracts Demand Signal Matrix

šŸ“Š Analysis Results

Demand Signal Matrix extracted by Claude

// Claude will analyze your data and extract: // // ā‘  Decision Triggers → Ad hooks, headlines // ā‘” Objection Architecture → FAQs, retargeting // ā‘¢ Conversion Vocabulary → Keywords for ads // ā‘£ Unmet Needs Signal → Offer design, USP // // Paste reviews and click Analyze!

šŸ“Š Demand Signal Matrix — Saved Results

Competitor insights organized by the 4-quadrant framework from the video

Quadrant Signal Source Use Case Frequency Action

šŸ› ļø Step-by-Step Setup (All Free)

Complete guide to set up the automation system

1
Create Free Apify Account

Go to apify.com → Sign up free → Get API token from Settings. Free tier: $5 free credits/month (~500 reviews).

2
Get Claude API Key

Go to console.anthropic.com → Create account → API Keys → New Key. Free trial credits available.

3
Setup Google Sheets Database

Create new Google Sheet → Go to Extensions → Apps Script → Paste the Apps Script code from "Scripts" tab → Deploy as Web App → Copy Web App URL.

4
Fork & Deploy to GitHub Pages

Fork this repo on github.com → Go to Settings → Pages → Source: main branch → Save. Your free URL: yourusername.github.io/repo-name

5
Connect Custom Domain (Optional)

In GitHub Pages settings → Custom domain → Add your domain. In your domain DNS, add CNAME record pointing to yourusername.github.io. Free SSL included.

6
Set Up Automation (Make.com)

Go to make.com (free tier) → Create scenario: Schedule trigger → Apify HTTP call → Claude AI → Google Sheets update. Runs automatically!

šŸ“¦ Free Tools Used

Everything is free — no paid subscriptions needed

āœ… Total Monthly Cost: $0
šŸ¤– Apify.com FREE $5/mo
🧠 Claude API FREE trial
šŸ“Š Google Sheets FREE forever
🌐 GitHub Pages FREE forever
⚔ Make.com FREE 1000 ops
šŸ”’ SSL Certificate FREE via GitHub
šŸ’” Tip: GitHub Pages = free hosting + free SSL + free CDN. No server needed!
āš ļø Store API keys in GitHub Secrets or Google Apps Script Properties — never in code!

šŸ“œ Google Apps Script (Database)

Paste this in Google Sheets → Extensions → Apps Script

// ===== Google Apps Script - Competitor Research DB ===== // Deploy as Web App (Execute as: Me, Access: Anyone) const SHEET_NAME = 'CompetitorData'; function doPost(e) { const data = JSON.parse(e.postData.contents); const sheet = SpreadsheetApp .getActiveSpreadsheet() .getSheetByName(SHEET_NAME); // Auto-create headers if sheet is empty if (sheet.getLastRow() === 0) { sheet.appendRow([ 'Timestamp', 'Source', 'Quadrant', 'Signal', 'UseCase', 'Frequency', 'RawText' ]); } // Save each signal row data.signals.forEach(signal => { sheet.appendRow([ new Date().toISOString(), data.source, signal.quadrant, signal.signal, signal.useCase, signal.frequency, signal.rawText ]); }); return ContentService .createTextOutput(JSON.stringify({status: 'OK', saved: data.signals.length})) .setMimeType(ContentService.MimeType.JSON); } function doGet(e) { const sheet = SpreadsheetApp .getActiveSpreadsheet() .getSheetByName(SHEET_NAME); const rows = sheet.getDataRange().getValues(); const headers = rows[0]; const data = rows.slice(1).map(row => { const obj = {}; headers.forEach((h, i) => obj[h] = row[i]); return obj; }); return ContentService .createTextOutput(JSON.stringify(data)) .setMimeType(ContentService.MimeType.JSON); }

šŸ”„ Make.com Automation JSON

Import this into Make.com to automate the full pipeline

// Make.com Scenario - Paste in Scenarios → Import { "name": "Competitor Research Auto", "flow": [ { "module": "schedule", "cron": "0 9 * * 1", // Every Monday 9am }, { "module": "http.request", "url": "https://api.apify.com/v2/acts/junglee~amazon-crawler/runs", "method": "POST", "body": { "startUrls": ["{{COMPETITOR_URL}}"], "maxItems": 100 } }, { "module": "http.request", "url": "https://api.anthropic.com/v1/messages", "method": "POST", "headers": { "x-api-key": "{{CLAUDE_API_KEY}}" }, "body": { "model": "claude-sonnet-4-20250514", "messages": [{ "role": "user", "content": "Analyze these reviews: {{REVIEWS_DATA}}" }] } }, { "module": "http.request", "url": "{{GOOGLE_APPS_SCRIPT_URL}}", "method": "POST", "body": "{{CLAUDE_RESPONSE}}" } ] }

šŸ“ GitHub Actions Auto-Deploy

Save as .github/workflows/deploy.yml in your repo for automatic deployment

# .github/workflows/deploy.yml # Auto-deploys to GitHub Pages on every push to main name: Deploy to GitHub Pages on: push: branches: [main] schedule: - cron: '0 6 * * 1' # Auto-refresh every Monday jobs: deploy: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - name: Fetch Latest Data from Google Sheets run: | curl "${{ secrets.SHEETS_WEBAPP_URL }}" \ -o data/latest.json - name: Deploy to GitHub Pages uses: peaceiris/actions-gh-pages@v3 with: github_token: ${{ secrets.GITHUB_TOKEN }} publish_dir: ./ # Add these secrets in repo Settings → Secrets: # SHEETS_WEBAPP_URL = your Apps Script web app URL # CLAUDE_API_KEY = your Anthropic API key # APIFY_API_KEY = your Apify API token