Competitor pricing analysis and dynamic pricing recommendation engine

Introduction

Pricing is one of the most dynamic levers in e-commerce and SaaS, yet most businesses update it manually, slowly, or not at all. You probably check competitor prices once a week, maybe less. By the time you've adjusted your own rates, the market has moved again. Meanwhile, you're leaving money on the table or pricing yourself out of deals without realising it.

The real problem isn't gathering competitor data. It's turning that data into action fast enough to matter. A competitor drops their price on Monday; you notice it Wednesday; you update your system Friday; their customers have already switched by then. This gap between insight and action is where profitable opportunities disappear.

What if your pricing could respond to market changes in minutes, not days? What if you could analyse competitor pricing across dozens of products, calculate optimal margins, and push new prices live without touching a single spreadsheet? That's what this workflow does. We'll wire together data collection, analysis, and recommendation generation so that price changes propagate automatically whenever market conditions shift.

The Automated Workflow

This workflow runs on a scheduled trigger (daily or every few hours, your choice) and follows four stages: data collection, enrichment, analysis, and recommendation delivery. We'll use Deepnote for Python-based data manipulation, Finster AI to monitor competitor prices at scale, Terrakotta AI to generate pricing recommendations from that data, and an orchestration platform to tie it all together.

For this advanced setup, I recommend n8n over Zapier or Make because it handles complex conditional logic and long-running Python processes better. If you prefer a visual, low-code interface, Make works fine too. I'll show both approaches below.

Architecture Overview

The workflow moves through these stages:

1. Trigger: Daily schedule or webhook from your e-commerce platform (Shopify, WooCommerce, custom API).
2. Collect: Finster AI fetches competitor prices from configured sources.
3. Enrich: Data is passed to Deepnote for cleaning, deduplication, and time-series analysis.
4. Analyse: Terrakotta AI generates pricing recommendations based on margin targets and competitor data.
5. Push: Recommendations are sent back to your pricing system via API or webhook.

Step 1:

Set Up Finster AI for Competitor Price Collection

Finster AI is a price monitoring service that watches competitor websites and APIs. You configure source URLs or product identifiers, and it returns structured pricing data on demand.

First, authenticate with Finster's API:

Copy
POST https://api.finster-ai.com/v1/auth/token
Content-Type: application/json

{
  "api_key": "your_finster_api_key",
  "api_secret": "your_finster_api_secret"
}

Response:

Copy{
  "access_token": "eyJhbGc...",
  "token_type": "Bearer",
  "expires_in": 3600
}

Next, create a monitor for your key competitors. Let's say you sell SaaS pricing modules and want to track three competitors:

Copy
POST https://api.finster-ai.com/v1/monitors
Authorization: Bearer eyJhbGc...
Content-Type: application/json

{
  "name": "SaaS Competitor Pricing",
  "monitor_type": "price_tracking",
  "sources": [
    {
      "competitor_id": "competitor_a",
      "source_type": "website",
      "url": "https://competitor-a.com/pricing",
      "products": ["starter_plan", "pro_plan", "enterprise_plan"],
      "check_interval_minutes": 360
    },
    {
      "competitor_id": "competitor_b",
      "source_type": "api",
      "endpoint": "https://api.competitor-b.com/v1/pricing",
      "auth_type": "bearer_token",
      "products": ["basic", "standard", "premium"],
      "check_interval_minutes": 360
    },
    {
      "competitor_id": "competitor_c",
      "source_type": "website",
      "url": "https://competitor-c.com/products",
      "products": ["plan_1", "plan_2"],
      "check_interval_minutes": 360
    }
  ],
  "notification_webhook": "https://your-n8n-instance.com/webhook/finster-price-update"
}

Save the returned monitor_id. You'll use this to retrieve pricing data in the workflow. For scheduled workflows, you'll call the data endpoint directly:

Copy
GET https://api.finster-ai.com/v1/monitors/{monitor_id}/latest
Authorization: Bearer eyJhbGc...

This returns:

Copy{
  "monitor_id": "mon_xyz123",
  "last_updated": "2024-01-15T09:30:00Z",
  "data": [
    {
      "competitor_id": "competitor_a",
      "product_id": "starter_plan",
      "price": 29.00,
      "currency": "GBP",
      "timestamp": "2024-01-15T09:25:00Z",
      "discount_applied": false,
      "list_price": 29.00
    },
    {
      "competitor_id": "competitor_a",
      "product_id": "pro_plan",
      "price": 79.00,
      "currency": "GBP",
      "timestamp": "2024-01-15T09:25:00Z",
      "discount_applied": false,
      "list_price": 79.00
    },
    {
      "competitor_id": "competitor_b",
      "product_id": "basic",
      "price": 25.00,
      "currency": "GBP",
      "timestamp": "2024-01-15T09:28:00Z",
      "discount_applied": false,
      "list_price": 25.00
    }
  ]
}

Step 2:

Build the n8n Workflow

Log into your n8n instance and create a new workflow. Start with a Schedule node set to run daily at 6 AM:

Copy
Node: Schedule
Trigger Type: Every day
Hour: 6
Minute: 0
Timezone: Europe/London

Next, add an HTTP Request node to fetch the latest competitor data from Finster:

Copy
Node: HTTP Request
Method: GET
URL: https://api.finster-ai.com/v1/monitors/{{env.FINSTER_MONITOR_ID}}/latest
Authentication: Bearer Token
Bearer Token: {{env.FINSTER_API_TOKEN}}

The {{env.FINSTER_MONITOR_ID}} and {{env.FINSTER_API_TOKEN}} reference environment variables you've set in n8n. This keeps secrets out of your workflow definition.

Now add a Code node to structure the Finster response for Deepnote:

Copy
Node: Code
Language: JavaScript

const data = $input.all();
const finsterData = data[0].json.data;

// Pivot competitor data into a product-centric view
const productPrices = {};

finsterData.forEach(record => {
  const key = record.product_id;
  if (!productPrices[key]) {
    productPrices[key] = {
      product_id: key,
      competitors: [],
      collected_at: new Date().toISOString()
    };
  }
  productPrices[key].competitors.push({
    competitor: record.competitor_id,
    price: record.price,
    currency: record.currency,
    timestamp: record.timestamp
  });
});

return Object.values(productPrices);

This restructures Finster's flat data into a nested format that Deepnote can process more easily.

Step 3:

Send Data to Deepnote for Analysis

Deepnote is a collaborative notebook environment that handles data processing, analysis, and visualisation. You'll use it here to enrich competitor data with your own product catalogue, historical pricing trends, and margin calculations.

Add an HTTP Request node in n8n that sends data to a Deepnote API endpoint (which you've published as a callable endpoint):

Copy
Node: HTTP Request
Method: POST
URL: https://api.deepnote.com/v1/projects/{{env.DEEPNOTE_PROJECT_ID}}/notebooks/{{env.DEEPNOTE_NOTEBOOK_ID}}/execute
Authentication: Bearer Token
Bearer Token: {{env.DEEPNOTE_API_KEY}}
Headers:
  Content-Type: application/json
Body:
{
  "cell_id": "competitor_pricing_analysis",
  "inputs": {
    "competitor_data": {{JSON.stringify($node["Code"].json)}},
    "execution_date": "{{now.toISOString()}}"
  }
}

Inside Deepnote, create a Python notebook with cells that handle the analysis. Here's a sample cell:

Copyimport pandas as pd
import numpy as np
from datetime import datetime, timedelta

competitor_df = pd.DataFrame(competitor_data)

# Load your own product catalogue and cost data
# In production, this would come from your database
your_products = pd.read_csv('gs://your-bucket/product_catalogue.csv')

# Merge competitor prices with your products
merged = your_products.merge(
    competitor_df,
    left_on='sku',
    right_on='product_id',
    how='left'
)

# Calculate average competitor price per product
merged['avg_competitor_price'] = merged.groupby('product_id')['competitors'].transform(
    lambda x: np.mean([comp['price'] for comp in x if isinstance(x, list) and comp])
)

# Calculate price elasticity (simplified: if we're 10% higher, assume 5% volume loss)
merged['price_index'] = merged['current_price'] / merged['avg_competitor_price']
merged['estimated_elasticity'] = np.where(
    merged['price_index'] > 1.0,
    1.0 - (merged['price_index'] - 1.0) * 0.5,
    1.0
)

# Store for next stage
analysis_results = merged.to_json(orient='records')

When the Deepnote cell completes, return the analysis results. You can configure Deepnote to emit a webhook when execution finishes:

Copyimport requests

webhook_url = "https://your-n8n-instance.com/webhook/deepnote-analysis-complete"
requests.post(webhook_url, json={
    "project_id": "{{env.DEEPNOTE_PROJECT_ID}}",
    "analysis_results": analysis_results,
    "timestamp": datetime.now().isoformat()
})

Step 4:

Generate Recommendations with Terrakotta AI

Terrakotta AI is a machine learning service that generates pricing recommendations based on historical data, competitor behaviour, demand signals, and margin targets. It outputs a JSON structure with recommended prices and confidence scores.

Add another HTTP Request node to call Terrakotta:

Copy
Node: HTTP Request
Method: POST
URL: https://api.terrakotta-ai.com/v1/pricing-recommendations
Authentication: Bearer Token
Bearer Token: {{env.TERRAKOTTA_API_TOKEN}}
Body:
{
  "analysis_id": "{{$node["Deepnote Webhook"].json.analysis_id}}",
  "products": {{$node["Deepnote Webhook"].json.analysis_results}},
  "constraints": {
    "minimum_margin_percent": 35,
    "maximum_margin_percent": 55,
    "price_change_limit_percent": 15,
    "currency": "GBP"
  },
  "objectives": {
    "primary": "revenue_maximization",
    "secondary": "market_share_maintenance"
  }
}

Terrakotta's response looks like:

Copy{
  "recommendation_id": "rec_abc789",
  "generated_at": "2024-01-15T09:45:00Z",
  "recommendations": [
    {
      "product_id": "starter_plan",
      "current_price": 29.00,
      "recommended_price": 31.00,
      "price_change_percent": 6.9,
      "confidence_score": 0.87,
      "reasoning": "Competitor average is £32.50; your current position is 11% below market. Raising to £31 captures share without crossing elasticity threshold.",
      "projected_revenue_impact": "+4.2%",
      "projected_margin": "42%"
    },
    {
      "product_id": "pro_plan",
      "current_price": 79.00,
      "recommended_price": 85.00,
      "price_change_percent": 7.6,
      "confidence_score": 0.79,
      "reasoning": "Market leader charging £89; you're competitively positioned at £79. Move to £85 maintains gap while improving margin.",
      "projected_revenue_impact": "+2.8%",
      "projected_margin": "48%"
    }
  ]
}

Step 5:

Apply Recommendations and Notify

Add a Conditional node to filter recommendations by confidence score. Only apply changes above, say, 0.75 confidence:

Copy
Node: Condition
Expression: $node["Terrakotta"].json.recommendations[0].confidence_score > 0.75
Evaluation: AND for all items

For approved recommendations, send them to your e-commerce platform's pricing API. Here's an example for a generic pricing endpoint:

Copy
Node: HTTP Request
Method: POST
URL: https://your-api.example.com/v1/products/bulk-price-update
Authentication: Bearer Token
Bearer Token: {{env.YOUR_API_KEY}}
Body:
{
  "updates": {{$node["Condition"].json.recommendations}},
  "reason": "automated_competitor_analysis",
  "scheduled_effective_date": "{{now.addHours(1).toISOString()}}",
  "require_approval": false
}

Finally, send a summary email or Slack notification with what changed:

Copy
Node: Send Email / Slack
To / Channel: #pricing-team
Subject: Daily Pricing Recommendations Applied
Body:

Automated pricing update completed at {{now.toISOString()}}.

Applied recommendations:
{{$node["Condition"].json.recommendations.map(r => 
  `${r.product_id}: £${r.current_price} → £${r.recommended_price} (${r.price_change_percent}% change, ${(r.confidence_score * 100).toFixed(0)}% confidence)`
).join('\n')}}

Projected revenue impact: +{{$node["Terrakotta"].json.projected_revenue_impact || 'pending'}}

Review changes: https://your-dashboard.example.com/pricing

Make (Integromat) Alternative

If you prefer Make's visual builder, the logic is nearly identical but you'll use Make's built-in modules:

1. Schedule trigger (daily, 6 AM).
2. HTTP module to fetch Finster data.
3. Text aggregator or custom Deepnote connector to format data.
4. Another HTTP module to post to Deepnote.
5. Wait for webhook or use polling to retrieve results.
6. HTTP module to call Terrakotta.
7. Router (conditional) to filter by confidence.
8. HTTP module to update your pricing system.
9. Slack or Email modules for notifications.

Make's visual interface is friendlier for non-technical users, but n8n is more flexible if you need custom logic or error handling.

The Manual Alternative

If you want more control over which recommendations apply, you can pause the workflow before the final price update step and insert a manual approval gate. In n8n, add an Approval node:

Copy
Node: Approval
Assignee: pricing-team-slack-channel
Title: Review Pricing Recommendations
Timeout: 2 hours
Message: {{$node["Terrakotta"].json.recommendations}}

This sends recommendations to a Slack channel where your team can click "Approve" or "Reject" before prices change. It's slower than fully automated, but safer if you want human eyes on larger price moves.

Alternatively, set the workflow to write recommendations to a database table (PostgreSQL, BigQuery, etc.) and create a Streamlit or Looker dashboard where your team reviews and applies changes manually. This gives you audit trails and time to consider qualitative factors that algorithms miss, like brand positioning or upcoming promotions.

Pro Tips

Rate Limiting and Cost Control

Finster and Terrakotta both have rate limits on their free and paid tiers. If you're monitoring 20+ competitors, you'll hit limits quickly. Solution: schedule your workflow to run during off-peak hours (2–4 AM) when your infrastructure is quieter, and batch requests. For Finster, check the X-RateLimit-Remaining header and pause if you're close to your limit. In n8n, use a Wait node:

Copy
Node: Code
const remaining = $node["HTTP Finster"].headers["x-ratelimit-remaining"];
if (remaining < 10) {
  throw new Error("Rate limit approaching, halting execution");
}

Handling Missing or Stale Data

If a competitor's website is down or Finster can't scrape pricing, your analysis breaks. Guard against this:

Copy# In Deepnote
merged['avg_competitor_price'] = merged.groupby('product_id')['competitors'].transform(
    lambda x: np.mean([comp['price'] for comp in x 
                      if isinstance(x, list) and comp and comp.get('timestamp') 
                      and (datetime.now() - datetime.fromisoformat(comp['timestamp'])).days < 7])
)

This ignores prices older than seven days, so stale data doesn't skew recommendations.

Testing Recommendations Before Going Live

Before pushing recommendations to production, run them through a "shadow mode" first. Send recommendations to a logging database for a week without applying them; compare projected impact against actual market movements. If accuracy is above 70%, enable live updates. Use n8n's conditional logic:

Copy
Node: Condition
Expression: $env.PRICING_MODE === "shadow" ? true : false
True: Log to database only
False: Apply to production

Margin Floor Protection

Always enforce minimum margins, even if Terrakotta recommends lower prices to match competitors. Add a validation step in n8n:

Copy
Node: Code
const validated = $node["Terrakotta"].json.recommendations.map(r => {
  const minPrice = r.cost_per_unit * 1.35; // 35% floor
  return {
    ...r,
    recommended_price: Math.max(r.recommended_price, minPrice),
    margin_constrained: r.recommended_price < minPrice
  };
});
return validated;

Timing Coordination Across Time Zones

If you sell globally and competitors span multiple time zones, their prices update at different times. Schedule Finster checks in waves: Europe 6 AM GMT, APAC 12 PM GMT, Americas 4 PM GMT. In n8n, create separate schedules for each region and write to a centralised result database, then kick off Terrakotta once all regions have reported. This ensures you're not recommending prices based on incomplete data.

Cost Breakdown

If you're running this workflow more than once per day (e.g., every 4 hours), expect costs to increase 2–3x. Most businesses find daily or twice-daily updates sufficient for pricing markets that don't move faster than hourly.

This workflow eliminates the manual price-checking grind and puts market-responsive pricing within reach for any business with API access to their e-commerce platform. The hardest part isn't the tools; it's agreeing internally on margin floors and confidence thresholds. Once those are set, the automation is solid and repeatable.