The Meta Ad Library has the data, publicly. Getting it out is the hard part:
This tutorial uses the Meta Ad Library Scraper actor on Apify as the API and keeps the Python to one short file that writes a CSV you can open in Excel or Google Sheets.
export APIFY_TOKEN=your_token_here
pip install "apify-client>=1.7,<2"
No Facebook account, cookies or Meta developer app are involved.
The scraper accepts a numeric page ID, an @username or a facebook.com page link. To get the ID, open the brand in the Ad Library: it is the view_all_page_id= number in the URL. Nike’s is 15087023444. If you pass a plain brand name instead, it is searched as a keyword across all advertisers.
This is python/example.py from the example repo assa83/facebook-ad-library-api. I ran it today, unchanged.
"""Export the Facebook & Instagram ads of a page (or a keyword) from the Meta Ad Library to CSV.
Usage: python example.py 15087023444 --country US --active
python example.py "running shoes" --keyword --country US
"""
import argparse
import csv
import os
import sys
from apify_client import ApifyClient
ACTOR = "lagenuina_agency/meta-ad-library-scraper"
TOKEN = os.environ.get("APIFY_TOKEN")
if not TOKEN:
sys.exit("Set APIFY_TOKEN first: https://console.apify.com/settings/integrations")
FIELDS = ["ad_archive_id", "page_name", "is_active", "start_date_iso", "end_date_iso", "publisher_platform",
"display_format", "ad_text", "title", "cta_text", "link_url", "image_urls", "video_urls", "spend", "ad_library_url"]
def main():
p = argparse.ArgumentParser()
p.add_argument("target", help="page id, @username, facebook.com link, or a keyword with --keyword")
p.add_argument("--keyword", action="store_true", help="search the target as a keyword across all advertisers")
p.add_argument("--country", default="ALL", help="two-letter code or ALL")
p.add_argument("--active", action="store_true", help="only ads running now")
p.add_argument("--limit", type=int, default=200)
a = p.parse_args()
run_input = {"country": a.country, "activeStatus": "active" if a.active else "all", "maxAds": a.limit}
run_input["keywords" if a.keyword else "pageIDs"] = [a.target]
client = ApifyClient(TOKEN)
run = client.actor(ACTOR).call(run_input=run_input)
safe = "".join(c if c.isalnum() else "_" for c in a.target)[:40]
out = f"meta_ads_{safe}.csv"
n = 0
with open(out, "w", newline="", encoding="utf-8") as f:
w = csv.DictWriter(f, fieldnames=FIELDS, extrasaction="ignore")
w.writeheader()
for ad in client.dataset(run["defaultDatasetId"]).iterate_items():
row = {k: (" | ".join(v) if isinstance(v, list) else v) for k, v in ad.items() if k in FIELDS}
w.writerow(row)
n += 1
print(f"{n} ads -> {out}")
if __name__ == "__main__":
main()
How it works:
pageIDs for one advertiser, or keywords to search every advertiser like the Ad Library search box. activeStatus: "active" matches what the website shows by default; "all" is the full archive, often ten times larger.maxAds caps the run, and with it the cost. Leave it high for a full export.a | b strings so each ad is one CSV row.Run on 17 October 2026, Nike’s active US ads, capped at 10:
python example.py 15087023444 --country US --active --limit 10
INFO Scraping 1 target(s), country=US, status=active, type=all, media=all, max 10 ads
INFO Done: 10 ad(s). Ad Library pages=1, GraphQL pages=0, proxy rotations=0.
Status: SUCCEEDED
10 ads -> meta_ads_15087023444.csv
The run took about 6 seconds. A few columns of meta_ads_15087023444.csv (image URLs shortened):
ad_archive_id,page_name,is_active,start_date_iso,publisher_platform,display_format,ad_text,cta_text,link_url
1067080952624010,Nike,True,2026-09-01T07:00:00.000Z,FACEBOOK | INSTAGRAM,DPA,Ball’s in your court. Get the gear that never misses.,Shop now,https://www.nike.com/
968300552951603,Nike,True,2026-09-01T07:00:00.000Z,FACEBOOK | INSTAGRAM,DPA,Get the gear not afraid to put in the work.,Shop now,https://www.nike.com/
2816322322094811,Nike,True,2026-10-15T07:00:00.000Z,FACEBOOK | INSTAGRAM,DPA,Run through summer with gear that can take the heat.,Shop now,https://www.nike.com/
Every row also has image_urls (full-size creative on Facebook’s CDN), video_urls, title, end_date_iso and an ad_library_url you can open to check the ad by hand.
A CSV is only useful if it answers a question. This snippet, run on the file above, shows when the ads started, which formats the brand uses and which messages it repeats:
import csv
from collections import Counter
ads = list(csv.DictReader(open("meta_ads_15087023444.csv", encoding="utf-8")))
print(len(ads), "ads")
print("by start month:", Counter(a["start_date_iso"][:7] for a in ads).most_common())
print("by format:", Counter(a["display_format"] for a in ads).most_common())
print("top messages:")
for text, n in Counter(a["ad_text"] for a in ads).most_common(3):
print(f" {n}x {text}")
10 ads
by start month: [('2026-09', 6), ('2026-10', 3), ('2025-06', 1)]
by format: [('DPA', 10)]
top messages:
3x Get the gear that goes hard on and off the field.
3x Ball’s in your court. Get the gear that never misses.
3x Get the gear not afraid to put in the work.
Even on ten ads the pattern is visible: Nike’s US catalog ads rotate three short lines of copy across many creatives, and a new batch went live in mid-October. On a full export (leave out --active to get the whole archive, and raise --limit) the same counts tell you how often a competitor refreshes creative and which angles survive for months.
Search a keyword instead of a page. python example.py "protein bars" --keyword --country US --active --limit 5 returns ads from every advertiser that mention the phrase. In my run the first five were all Nature Valley catalog ads, which is itself a finding.
n8n, every Monday. The repo includes an n8n workflow that runs the actor on a competitor’s page each Monday at 08:00, flattens the fields and appends them to Google Sheets. Import the JSON, add a Header Auth credential (Authorization: Bearer <token>), set your page IDs and sheet.
AI agents (MCP). Give the actor to Claude, Cursor or any MCP client:
{
"mcpServers": {
"meta-ad-library-scraper": {
"url": "https://mcp.apify.com/?actors=lagenuina_agency/meta-ad-library-scraper"
}
}
}
Then ask “what are Nike’s active Instagram ads in the US promoting this week?” and the agent runs the export for you.
$0.20 per 1,000 ads on every Apify plan, residential proxy included, no start fee. The 15 ads in my two test runs cost a fraction of a cent. Set maxAds or a maximum charge per run to cap spending. Check the actor page for current prices.
DPA, DCO) store copy as placeholders such as ``; the Ad Library does not render the products. You still get the images, CTA and landing page.If the official Ad Library API does not cover the ads you need, or the identity verification is a blocker, a hosted scraper plus forty lines of Python gets a competitor’s Facebook and Instagram ads into a spreadsheet in seconds. Clone the example repo, set APIFY_TOKEN and point it at the page ID you care about.
Disclosure: I built this actor. It is published on Apify by Quelli di Carne Genuina, and I earn from its usage.
]]>If what you want is “check LHR → JFK every morning, keep a history and ping me when it is cheap”, a hosted scraper and a short Python script are enough. This tutorial uses the Google Flights Scraper actor on Apify and the official apify-client package.
By the end you will have fare_tracker.py, which on each run:
fares_history.csv,ALERT line when the fare is at or below your target.export APIFY_TOKEN=your_token_here
pip install "apify-client>=1.7,<2"
I wrote this for the post and ran it today against the live actor. The example repo assa83/google-flights-api has a sibling script that prints every fare instead of logging the cheapest.
"""Track Google Flights fares for one route and alert when the price drops.
Usage: python fare_tracker.py LHR JFK --days-ahead 30 --max-price 450
Run it once a day (cron, GitHub Actions, n8n): every run appends to fares_history.csv.
"""
import argparse
import csv
import datetime
import os
import sys
from apify_client import ApifyClient
ACTOR = "lagenuina_agency/google-flights-scraper"
TOKEN = os.environ.get("APIFY_TOKEN")
if not TOKEN:
sys.exit("Set APIFY_TOKEN first: https://console.apify.com/settings/integrations")
HISTORY = "fares_history.csv"
def main():
p = argparse.ArgumentParser()
p.add_argument("origin", help="IATA airport code, e.g. LHR")
p.add_argument("destination", help="IATA airport code, e.g. JFK")
p.add_argument("--days-ahead", type=int, default=30)
p.add_argument("--max-price", type=float, help="alert when the cheapest fare is at or below this")
p.add_argument("--calendar-days", type=int, default=7, help="also find the cheapest day in the next N days")
p.add_argument("--currency", default="USD")
a = p.parse_args()
client = ApifyClient(TOKEN)
run = client.actor(ACTOR).call(run_input={
"origin": a.origin, "destination": a.destination,
"departureDate": f"{a.days_ahead} days", # relative: a daily run always looks N days ahead
"currency": a.currency, "sortBy": "price", "maxResults": 5,
"calendarDays": a.calendar_days,
})
items = list(client.dataset(run["defaultDatasetId"]).iterate_items())
for e in (i for i in items if i["type"] == "error"):
sys.exit(f"actor error: {e['error']}")
flights = [i for i in items if i["type"] == "flight" and i.get("price") is not None]
if not flights:
sys.exit("no fares found")
best = min(flights, key=lambda f: f["price"])
level = (best.get("priceInsights") or {}).get("priceLevel", "")
days = [i for i in items if i["type"] == "calendarDay" and i.get("lowestPrice") is not None]
cheapest_day = min(days, key=lambda d: d["lowestPrice"]) if days else None
new_file = not os.path.exists(HISTORY)
with open(HISTORY, "a", newline="", encoding="utf-8") as f:
w = csv.writer(f)
if new_file:
w.writerow(["checked_at", "route", "departure_date", "price", "currency", "airlines", "stops", "price_level"])
w.writerow([datetime.datetime.now().isoformat(timespec="minutes"), f"{a.origin}-{a.destination}",
best["departureDate"], best["price"], best["currency"], " / ".join(best["airlines"]),
best["stops"], level])
print(f"Cheapest {a.origin}->{a.destination} on {best['departureDate']}: {best['price']} {best['currency']} "
f"({', '.join(best['airlines'])}, {best['stops']} stop(s)), Google says '{level}'")
if cheapest_day:
print(f"Cheapest day in the next {a.calendar_days}: {cheapest_day['departureDate']} "
f"at {cheapest_day['lowestPrice']} {cheapest_day['currency']}")
if a.max_price is not None and best["price"] <= a.max_price:
print(f"ALERT: {best['price']} {best['currency']} is at or below your {a.max_price:g} target")
if __name__ == "__main__":
main()
The input fields used:
| Field | Meaning |
|---|---|
origin, destination |
IATA airport codes |
departureDate |
YYYY-MM-DD or a relative value like "30 days", so a scheduled run keeps looking the same distance ahead |
sortBy: "price", maxResults |
Cheapest itineraries first, keep 5 |
calendarDays |
Also return the lowest fare for each of the next N departure days |
currency |
Any ISO code; add market (e.g. GB) to see fares as offered in another country |
The output mixes row types: flight rows (itineraries), calendarDay rows (one per date) and error rows. The script splits them by type.
Run on 17 October 2026, London Heathrow to New York JFK, 30 days ahead, with a $550 target:
python fare_tracker.py LHR JFK --days-ahead 30 --max-price 550 --calendar-days 7
INFO LHR→JFK 2026-11-16: 5 itineraries
INFO Done: 12 row(s), 0 route error(s).
Status: SUCCEEDED
Cheapest LHR->JFK on 2026-11-16: 505 USD (JetBlue, 0 stop(s)), Google says 'typical'
Cheapest day in the next 7: 2026-11-17 at 404 USD
ALERT: 505 USD is at or below your 550 target
About ten seconds per run. After two runs, fares_history.csv looks like this:
checked_at,route,departure_date,price,currency,airlines,stops,price_level
2026-10-17T15:35,LHR-JFK,2026-11-16,505,USD,JetBlue,0,typical
2026-10-17T15:39,LHR-JFK,2026-11-16,505,USD,JetBlue,0,typical
Two things in that output are worth more than the price itself:
flight row has a priceInsights block. Shortened:{
"price": 505,
"airlines": ["JetBlue"],
"flightNumbers": ["B6 2220"],
"departureTime": "2026-11-16T08:05",
"durationMinutes": 489,
"stops": 0,
"co2EmissionsKg": 549,
"priceInsights": {
"lowestPrice": 505,
"typicalPriceLow": 400,
"typicalPriceHigh": 570,
"priceLevel": "typical",
"priceHistory": [{ "date": "2026-08-16", "price": 365 }, "..."]
},
"bookingUrl": "https://www.google.com/travel/flights/booking?tfs=..."
}
priceHistory holds about 60 days of Google’s tracked prices for the search. Here it shows the same flight at $365 in mid-August, so “typical” today is still well above where it started. A sensible alert rule uses this: fire when the price is low, or below typicalPriceLow, instead of a number you guessed.
The script appends, so scheduling it is all you need to build a history:
0 7 * * * cd /path/to/tracker && APIFY_TOKEN=... python fare_tracker.py LHR JFK --max-price 450schedule: workflow that runs the script and commits fares_history.csv back to the repo. Store the token as a repository secret.grep ALERT and into your email or Slack webhook of choice.n8n, no code. The example repo has an n8n workflow: every day at 07:00 it searches the route 30 days ahead, a Code node keeps only fares at or below MAX_PRICE, and you attach a Slack or Gmail node. Import the JSON, add a Header Auth credential (Authorization: Bearer <token>) and set your route.
Many routes at once. The actor takes a routes list ([{"origin": "LHR", "destination": "JFK", "departureDate": "2026-12-10"}, ...]), so one run can watch every route you care about.
AI agents (MCP). Give the actor to Claude, Cursor or any MCP client:
{
"mcpServers": {
"google-flights-scraper": {
"url": "https://mcp.apify.com/?actors=lagenuina_agency/google-flights-scraper"
}
}
}
Then ask “what is the cheapest day to fly Milan to New York in the first week of December?” and the agent runs the calendar search.
The actor charges per row: one itinerary or one calendar day. The run above returned 12 rows (5 flights and 7 calendar days) and cost a fraction of a cent. Until 26 October 2026 the price is $0.16 per 1,000 rows on the Apify Free plan ($0.08 on Gold); from 27 October it is $0.80 on Free and $0.56 on Gold. At the new price a daily check like this one is about 4,400 rows a year, roughly $3.50 on the Free plan. Check the actor page for current prices.
resolveReturnFlights to get the matching return flights.market and currency. The same flight can cost differently when searched from the US or the UK. Keep both fixed in a tracker, or the history is not comparable.calendarOnly if you do not need itineraries.Without a public Google Flights API, a small daily script on top of a hosted scraper gets you the part that matters: a fare history you own and an alert that fires on your rules. Copy fare_tracker.py, set APIFY_TOKEN, pick a route and schedule it.
Disclosure: I built this actor. It is published on Apify by Quelli di Carne Genuina, and I earn from its usage.
]]>Two things changed that:
429 Too Many Requests is the error everyone meets. Trends throttles automated traffic hard. A few requests from one IP and you start getting 429s, and then you are writing retry loops, sleeping between calls and rotating proxies instead of analysing data.If you just need the numbers (interest over time, rising queries, interest by region), a simpler option is to let a hosted scraper deal with sessions and rate limits and call it from Python. This tutorial does that with the Google Trends Scraper actor on Apify and the official apify-client package.
By the end you will have a script that writes two CSV files you can open in Excel or import into Google Sheets:
trends_YYYY-MM-DD.csv: one row per keyword and date (searchTerm, geo, date, value, isPartial)rising_YYYY-MM-DD.csv: the rising related queries for each keyword (searchTerm, query, growth)The full code is in the repo assa83/google-trends-api-python, with JavaScript and n8n versions too.
Keep the token out of your code. Put it in an environment variable:
export APIFY_TOKEN=your_token_here
pip install "apify-client>=1.7,<2"
The example targets the 1.x client, where .call() returns a plain dict. If you use apify-client 2.x, check the client changelog for the return type.
Save this as example.py. It is the same file as python/example.py in the repo, and I ran it today against the live actor.
"""Weekly Google Trends export to CSV (Google Sheets ready).
Usage: python example.py "chatgpt" "gemini" --geo US
"""
import argparse
import csv
import datetime
import os
import sys
from apify_client import ApifyClient
ACTOR = "lagenuina_agency/google-trends-scraper"
TOKEN = os.environ.get("APIFY_TOKEN")
if not TOKEN:
sys.exit("Set APIFY_TOKEN first: https://console.apify.com/settings/integrations")
def main():
p = argparse.ArgumentParser()
p.add_argument("terms", nargs="+", help="keywords to track")
p.add_argument("--geo", default="", help="country/region code, empty = worldwide")
p.add_argument("--time-range", default="today 3-m")
a = p.parse_args()
client = ApifyClient(TOKEN)
run = client.actor(ACTOR).call(run_input={
"searchTerms": a.terms,
"geo": a.geo,
"timeRange": a.time_range,
"includeInterestByRegion": False,
"includeRelated": True,
})
items = client.dataset(run["defaultDatasetId"]).list_items().items
today = datetime.date.today().isoformat()
with open(f"trends_{today}.csv", "w", newline="", encoding="utf-8") as f:
w = csv.writer(f)
w.writerow(["searchTerm", "geo", "date", "value", "isPartial"])
for it in items:
for point in it.get("timeline", []):
w.writerow([it["searchTerm"], it.get("geo", ""), point["date"][:10], point.get("value"), point.get("isPartial", False)])
with open(f"rising_{today}.csv", "w", newline="", encoding="utf-8") as f:
w = csv.writer(f)
w.writerow(["searchTerm", "query", "growth"])
for it in items:
for q in it.get("relatedQueries_rising", []):
w.writerow([it["searchTerm"], q["query"], q.get("formattedValue", q.get("value"))])
print(f"{len(items)} keywords -> trends_{today}.csv, rising_{today}.csv")
if __name__ == "__main__":
main()
What it does, step by step:
APIFY_TOKEN and stops with a clear message if it is missing.client.actor(...).call(run_input=...). call() blocks until the run finishes, so there is no polling code to write.timeline, related queries and metadata.The input fields used here:
| Field | Meaning |
|---|---|
searchTerms |
List of keywords, one result per keyword |
geo |
Country or region code such as US, IT, GB; empty string means worldwide |
timeRange |
Same syntax as the Trends website, e.g. today 3-m, today 12-m |
includeInterestByRegion |
Turned off here to keep the output small |
includeRelated |
Adds top and rising related queries |
The actor page (Input tab) documents the rest, including a Trending Now mode.
python example.py "chatgpt" "claude"
The run takes a few seconds per keyword. The console log ends like this:
INFO Scraping 2 keyword job(s)
INFO Done: chatgpt
INFO Done: claude
Status: SUCCEEDED, Message: 2 result(s)
2 keywords -> trends_2026-10-17.csv, rising_2026-10-17.csv
This is an excerpt from a run on 17 October 2026, worldwide, over a one-month window. Over a short window Trends returns daily points. trends_2026-10-17.csv:
searchTerm,geo,date,value,isPartial
chatgpt,Worldwide,2026-09-17,74,False
chatgpt,Worldwide,2026-09-18,64,False
chatgpt,Worldwide,2026-09-19,55,False
...
chatgpt,Worldwide,2026-10-15,99,False
chatgpt,Worldwide,2026-10-16,84,False
chatgpt,Worldwide,2026-10-17,74,True
claude,Worldwide,2026-09-17,66,False
claude,Worldwide,2026-09-18,58,False
...
claude,Worldwide,2026-10-16,90,False
claude,Worldwide,2026-10-17,73,True
And rising_2026-10-17.csv:
searchTerm,query,growth
chatgpt,chatgpt dots,"+1,200%"
claude,opus 5.5,Breakout
claude,claude opus 5.5,Breakout
claude,claude startup program,"+2,650%"
claude,claude for startups,+750%
claude,claude mods,+650%
Some notes on reading this data:
isPartial: True marks the latest point, which is still being collected. Drop it before you compare it with earlier points, or it will look like a sudden fall.If you want the raw JSON rather than CSV, each dataset item looks like this (shortened, from a run for chatgpt in Italy over 12 months):
{
"type": "keyword",
"searchTerm": "chatgpt",
"geo": "IT",
"timeRange": "today 12-m",
"data_granularity": "week",
"timeline": [
{ "date": "2026-10-11T00:00:00.000Z", "value": 88, "isPartial": true }
],
"relatedQueries_top": [
{ "query": "chatgpt gratis", "value": 100, "formattedValue": "100" }
],
"relatedQueries_rising": [
{ "query": "chatgpt atlas", "value": 4250, "formattedValue": "+4,250%" }
],
"trendsUrl": "https://trends.google.com/trends/explore?q=chatgpt&date=today+12-m&hl=en-US&geo=IT"
}
The trendsUrl field is useful for spot checks: open it and compare the chart with your CSV.
The CSV is already in long format, so pivoting it for a chart is short:
import pandas as pd
df = pd.read_csv("trends_2026-10-17.csv", parse_dates=["date"])
df = df[~df["isPartial"]] # drop the incomplete last point
wide = df.pivot(index="date", columns="searchTerm", values="value")
print(wide.tail())
The script names its files by date, so running it on a schedule builds a history without overwriting anything:
0 8 * * 1 cd /path/to/script && python example.py "brand a" "brand b" --geo USThe actor uses pay-per-result pricing with no start fee. One keyword with all its data counts as one result. The price per 1,000 results depends on your Apify plan: $2.40 on the Free plan, going down to $0.24 on Gold. Tracking 10 keywords every week is about 520 results a year. Check the actor page for current prices before you budget.
Apify runs an MCP server, so you can give the actor to Claude, Cursor, VS Code or any other MCP client as a tool, without writing Python:
{
"mcpServers": {
"google-trends-scraper": {
"url": "https://mcp.apify.com/?actors=lagenuina_agency/google-trends-scraper"
}
}
}
Sign in with Apify OAuth when the client asks, or send Authorization: Bearer <APIFY_TOKEN>. Then you can ask questions such as “compare interest in X and Y in Germany over the last 12 months and list the rising queries” and the agent calls the actor for you.
chatgpt 74 next to claude 73 does not mean equal search volume. Compare shapes and changes over time, not absolute values across keywords.If pytrends stopped working for you, or you are tired of handling 429s, moving the scraping to a hosted actor and keeping your Python to about 40 lines of CSV code is a practical trade-off. Clone the example repo, set APIFY_TOKEN and run it.
Disclosure: I built this actor. It is published on Apify by Quelli di Carne Genuina, and I earn from its usage.
]]>This post sets up a small n8n workflow that does it for you. Every Monday at 08:00 it:
You import a ready-made workflow, so you don’t write code. The data comes from the Google Trends Scraper actor on Apify, which n8n calls with a plain HTTP Request node.
Be aware before you start: the workflow file is valid n8n JSON and its Apify call uses the same endpoint and input that I tested from Python. I have not run this exact workflow inside an n8n instance. Node versions differ between n8n releases, so treat it as a starting point and do one manual run before you activate it.
n8n/workflow.json from the repo assa83/google-trends-api-pythonFour nodes in a straight line:
Schedule Trigger ──> Run Apify actor ──> Flatten timeline ──> Append to Google Sheets
(Mon 08:00) (HTTP Request) (Code) (Google Sheets)
POST to Apify’s run-sync-get-dataset-items endpoint. This endpoint starts the actor, waits for it to finish and returns the dataset items in the response, so there is no polling. The node timeout is set to 300,000 ms (5 minutes), the most this endpoint waits.timeline array into separate rows.workflow.json from the repo.The HTTP Request node expects a Header Auth credential named Apify API token.
Apify API tokenAuthorizationBearer followed by your token (with the space after Bearer)The token lives in n8n’s encrypted credential store, not in the workflow JSON. If you export or share the workflow later, the token does not go with it.
Still in Run Apify actor, edit the JSON body:
{
"searchTerms": ["chatgpt", "gemini"],
"geo": "US",
"timeRange": "today 3-m",
"includeInterestByRegion": false
}
searchTerms: your keywords. Each one comes back as one result.geo: a country or region code (US, GB, IT, …). Use "" for worldwide.timeRange: the same syntax as the Trends website, such as today 3-m or today 12-m.The other input fields, including related queries and Trending Now, are on the actor page (Input tab).
Keep the keyword list modest: the synchronous endpoint waits up to 300 seconds. If a long list times out, switch to Apify’s asynchronous /runs endpoint with a webhook.
Create a new Google Sheet. In row 1 of the first tab, write these headers exactly:
| A | B | C | D |
|---|---|---|---|
| searchTerm | geo | date | value |
The node uses automatic mapping, which matches incoming field names to column headers. Without the headers it has nothing to map to.
/d/ and /edit.YOUR_GOOGLE_SHEET_ID with your sheet ID.Sheet1, so change it if your tab has another name, for example in a localised Google account.Before scheduling anything, click Execute workflow (or Test workflow, depending on your n8n version) and check each node:
searchTerm, geo and a timeline array.Flatten timeline should output many small items such as:
{ "searchTerm": "chatgpt", "geo": "US", "date": "2026-10-16", "value": 84 }
If the HTTP node returns 401, the header value is wrong. Check that it reads Bearer <token> with one space and no quotes. If Sheets complains about columns, check the header row from Step 4.
Flip the workflow to Active. The trigger fires every Monday at 08:00 in your n8n instance’s timezone. Check that under the workflow or instance settings if your server runs in UTC and you don’t.
To change the day or hour, open Schedule Trigger and edit the weekly rule.
Each Monday the workflow fetches the last three months, not just the last week. Appending blindly means most rows show up again every week. You have three reasonable options:
Keep only recent days. Change the Code node to keep only the last 7 days, so each run adds just the new week. For example:
// One row per keyword and day, only the last 7 days.
const since = new Date(Date.now() - 7 * 864e5).toISOString().slice(0, 10);
const rows = [];
for (const { json } of $input.all()) {
for (const p of json.timeline ?? []) {
const date = p.date.slice(0, 10);
if (date >= since) rows.push({ json: { searchTerm: json.searchTerm, geo: json.geo, date, value: p.value } });
}
}
return rows;
This snippet is a variant of the one in the repo and I have not run it in n8n, so check its output in a manual test first.
UNIQUE() range.Also remember that the latest point is usually partial (still being collected). The original API response marks it with isPartial: true. The Code node in the workflow does not copy that field, so add isPartial: p.isPartial to the row object if you want to filter it out in Sheets.
Google Trends values are relative, from 0 to 100, where 100 is the peak for that query, region and time range. They are not search volumes. In my tests each keyword came back on its own 0–100 scale, so compare how each line moves over time rather than the absolute values between two keywords.
For a quick report, build a pivot table (date as rows, searchTerm as columns) and add a line chart on it.
n8n costs whatever your plan or server costs. On the Apify side the actor uses pay-per-result pricing with no start fee. One keyword with all its data is one result. The price per 1,000 results goes from $2.40 on the Free plan down to $0.24 on Gold. Ten keywords every Monday is 520 results a year. Check the actor page for current prices.
The same repo has a Python script that writes the data to dated CSV files (trends_YYYY-MM-DD.csv and rising_YYYY-MM-DD.csv) and a Node.js example. I covered the Python version in a separate post. If you work with AI agents, you can also expose the actor as an MCP tool at https://mcp.apify.com/?actors=lagenuina_agency/google-trends-scraper.
The scraper reads only publicly available data and is not affiliated with or endorsed by Google. Use it in line with the applicable terms and laws.
Disclosure: I built this actor. It is published on Apify by Quelli di Carne Genuina, and I earn from its usage.
]]>youtube-transcript-api package. It is a good library, and on your laptop it works on the first try.
Then you deploy it. On AWS, Google Cloud, Azure, Render, Railway or a GitHub Actions runner, the same call fails with RequestBlocked or IpBlocked. YouTube blocks most cloud and datacenter IP ranges for caption requests, and the library’s own README says so: on a server you need rotating residential proxies.
So the real problem is not parsing captions. It is getting a request through from a server IP, every day, without running your own proxy pool.
This tutorial moves that part to a hosted scraper, the YouTube Transcript Scraper actor on Apify, and keeps your Python small. The code below runs anywhere pip install works, because the request to YouTube does not leave from your server.
export APIFY_TOKEN=your_token_here
pip install "apify-client>=1.7,<2"
The examples target the 1.x client, where .call() returns a plain dict.
If you only need the text of one video inside your app, this is all it takes:
import os
from apify_client import ApifyClient
client = ApifyClient(os.environ["APIFY_TOKEN"])
def get_transcript(url, languages=("en",)):
run = client.actor("lagenuina_agency/youtube-transcript-scraper").call(
run_input={"youtube_url": url, "languages": list(languages)}
)
item = client.dataset(run["defaultDatasetId"]).list_items().items[0]
if not item.get("success"):
raise RuntimeError(item.get("error_message", "no transcript"))
return item["non_timestamped"]
print(get_transcript("https://youtu.be/jNQXAC9IVRw")[:80])
Output:
All right, so here we are, in front of the elephants the cool thing about these
languages is an ordered preference list: the first language YouTube has wins, and manual captions beat auto-generated ones in the same language.
For batches (a list of URLs, a playlist or a whole channel), this is the script from the example repo assa83/youtube-transcript-scraper-api. I ran it today, unchanged.
"""Download YouTube transcripts (videos, playlists or a channel) to text files + one CSV.
Usage:
python example.py https://www.youtube.com/watch?v=dQw4w9WgXcQ
python example.py https://www.youtube.com/playlist?list=PL... --max 20
python example.py --channel @TED --max 10 --lang en es
"""
import argparse
import csv
import os
import re
import sys
from apify_client import ApifyClient
ACTOR = "lagenuina_agency/youtube-transcript-scraper"
TOKEN = os.environ.get("APIFY_TOKEN")
if not TOKEN:
sys.exit("Set APIFY_TOKEN first: https://console.apify.com/settings/integrations")
def main():
p = argparse.ArgumentParser()
p.add_argument("urls", nargs="*", help="video / Shorts / playlist URLs or video IDs")
p.add_argument("--channel", help="@handle or channel URL: transcribe its newest videos")
p.add_argument("--max", type=int, default=10, help="max videos per channel or playlist")
p.add_argument("--lang", nargs="+", default=["en"], help="preferred languages, in order")
p.add_argument("--out", default="transcripts", help="output folder")
a = p.parse_args()
if not a.urls and not a.channel:
p.error("pass at least one URL or --channel")
run_input = {"youtube_url": a.urls, "languages": a.lang, "max_videos": a.max, "output_formats": ["srt"]}
if a.channel:
run_input.update(channel=a.channel, channel_transcripts=True)
run = ApifyClient(TOKEN).actor(ACTOR).call(run_input=run_input)
items = ApifyClient(TOKEN).dataset(run["defaultDatasetId"]).list_items().items
os.makedirs(a.out, exist_ok=True)
with open(os.path.join(a.out, "transcripts.csv"), "w", newline="", encoding="utf-8") as f:
w = csv.writer(f)
w.writerow(["video_id", "title", "channel", "language", "source_type", "words", "url", "error"])
for it in items:
w.writerow([it.get("video_id"), it.get("title"), it.get("channel_name"), it.get("language_code"),
it.get("source_type"), it.get("word_count"), it.get("url"), it.get("error_message", "")])
if it.get("success"):
name = re.sub(r"[^\w-]+", "_", f"{it['video_id']}_{it.get('title') or ''}")[:80]
with open(os.path.join(a.out, name + ".txt"), "w", encoding="utf-8") as t:
t.write(f"{it.get('title')}\n{it['url']}\n\n{it['non_timestamped']}\n")
with open(os.path.join(a.out, name + ".srt"), "w", encoding="utf-8") as t:
t.write(it["srt"])
ok = sum(1 for it in items if it.get("success"))
print(f"{ok} transcript(s), {len(items) - ok} without captions/failed -> {a.out}/")
if __name__ == "__main__":
main()
The input fields that matter:
| Field | Meaning |
|---|---|
youtube_url |
One URL or a list: videos, Shorts, youtu.be links, playlists or bare video IDs |
channel + channel_transcripts |
A @handle or channel URL; with channel_transcripts: true its newest videos are transcribed |
max_videos |
How many videos per channel or playlist |
languages |
Ordered language preference |
output_formats |
Extra formats: srt, vtt, text |
Run on 17 October 2026:
python example.py https://www.youtube.com/watch?v=jNQXAC9IVRw https://www.youtube.com/watch?v=dQw4w9WgXcQ
The run log shows exactly the problem this post is about, handled for you:
INFO 2 video(s) to process (languages: en)
INFO YouTube is blocking this IP range, switching to residential proxy.
INFO [Status message]: 2 transcript(s)
Status: SUCCEEDED, Message: 2 transcript(s)
2 transcript(s), 0 without captions/failed -> transcripts/
The whole run took about 12 seconds. transcripts/transcripts.csv:
video_id,title,channel,language,source_type,words,url,error
dQw4w9WgXcQ,Rick Astley - Never Gonna Give You Up (Official Video) (4K Remaster),Rick Astley,en,Manual,487,https://www.youtube.com/watch?v=dQw4w9WgXcQ,
jNQXAC9IVRw,Me at the zoo,jawed,en,Manual,39,https://www.youtube.com/watch?v=jNQXAC9IVRw,
And the start of the .srt file for the second video:
1
00:00:01,360 --> 00:00:03,040
[♪♪♪]
2
00:00:18,640 --> 00:00:21,880
♪ We're no strangers to love ♪
Each dataset item also carries timestamped segments (text, start, duration) if you want to cite the exact second a phrase was said, which is handy for RAG answers that link back to the video.
A whole channel. python example.py --channel @TED --max 10 --lang en es transcribes the ten newest videos of a channel. Listing the channel is free; you pay only for transcripts.
n8n, no code. The example repo has an n8n workflow that runs every morning, transcribes the newest 5 videos of a channel and appends title, URL, language, word count and transcript to Google Sheets. Import the JSON, add a Header Auth credential with Authorization: Bearer <token>, set your channel and sheet.
AI agents (MCP). Give the actor to Claude, Cursor or any MCP client as a tool:
{
"mcpServers": {
"youtube-transcript-scraper": {
"url": "https://mcp.apify.com/?actors=lagenuina_agency/youtube-transcript-scraper"
}
}
}
Then ask “summarise the last three videos of @channel” and the agent fetches the transcripts itself.
Pay per transcript, proxies included, no start fee: $2.20 per 1,000 transcripts on the Apify Free plan, $1.60 on Gold. A three-hour podcast costs the same as a Short. Videos without captions and failed videos are free. The two-video run above cost less than a cent. Check the actor page for current prices.
translation_error field.If youtube-transcript-api works locally and breaks in production, the fix is not more retry code: it is sending the request from an IP YouTube accepts. Clone the example repo, set APIFY_TOKEN and run it from the same server that was getting blocked.
Disclosure: I built this actor. It is published on Apify by Quelli di Carne Genuina, and I earn from its usage.
]]>