WebSub: Get Your New Content Indexed in Seconds, Not Hours

When you publish a new blog post, how long does it take for Google to index it? For most small sites, the answer is hours to days. Google's crawler operates on its own schedule, and sites without high crawl priority can wait 24-72 hours before new content appears in search results.

WebSub eliminates this wait. Instead of waiting for Google to come to you, WebSub pushes a notification to Google (and every other subscriber) the moment you publish. New content can be indexed within seconds of publication.

WebSub is a W3C-recommended protocol formerly known as PubSubHubbub. It is free, open, and supported by Google, most RSS readers, and the Fediverse. I implemented WebSub across all 52 sites in our network. Indexing latency dropped from an average of 8-14 hours to under 5 minutes for most new content.

How WebSub Works

WebSub operates on a publish-subscribe model with three components:

1. Publisher — your website, which publishes new content via an RSS or Atom feed
2. Hub — a relay server that receives your publication notifications and pushes them to subscribers
3. Subscriber — any service that wants to know when you publish new content (Google, Bing, feed readers, Mastodon instances)

The flow:

1. You publish a new blog post
2. Your site notifies the hub: "I have new content"
3. The hub fetches your updated feed
4. The hub pushes the new content to all subscribers

The critical advantage over traditional polling is speed. Without WebSub, subscribers check your feed periodically — maybe every 15 minutes, maybe every hour, maybe every 6 hours. With WebSub, they receive the notification within seconds of publication.

Setting Up WebSub

Step 1: Add Hub Declaration to Your Feed

Your RSS or Atom feed needs to declare which WebSub hub it uses. Add a <link> element to your feed:

For Atom feeds:

<link rel="hub" href="https://pubsubhubbub.appspot.com/" />
<link rel="self" href="https://thecondotrap.com/feed.xml" />

For RSS feeds:

<atom:link rel="hub" href="https://pubsubhubbub.appspot.com/" />
<atom:link rel="self" href="https://thecondotrap.com/feed.xml" />

The hub URL https://pubsubhubbub.appspot.com/ is Google's free, public WebSub hub. It is the most widely used hub and handles the relay infrastructure at no cost.

Step 2: Ping the Hub on Publication

When you publish new content, send an HTTP POST to the hub notifying it that your feed has updated:

curl -X POST https://pubsubhubbub.appspot.com/ \
  -d "hub.mode=publish" \
  -d "hub.url=https://thecondotrap.com/feed.xml"

For static sites built with Eleventy or similar generators, add this ping to your deployment script:

# After deploying the site
curl -s -X POST https://pubsubhubbub.appspot.com/ \
  -d "hub.mode=publish" \
  -d "hub.url=https://thecondotrap.com/feed.xml"

This single HTTP request tells the hub to fetch your updated feed and push the new entries to all subscribers.

Step 3: Automate the Ping

Integrate the hub ping into your CI/CD pipeline so it fires automatically on every deployment:

Netlify build command:

eleventy && curl -s -X POST https://pubsubhubbub.appspot.com/ -d "hub.mode=publish" -d "hub.url=https://thecondotrap.com/feed.xml"

GitHub Actions step:

- name: Ping WebSub hub
  run: |
    curl -s -X POST https://pubsubhubbub.appspot.com/ \
      -d "hub.mode=publish" \
      -d "hub.url=https://thecondotrap.com/feed.xml"

Once automated, you never think about it again. Every deployment triggers a hub notification, which triggers instant content distribution.

What Subscribes to Your WebSub Feed

Google

Google is a WebSub subscriber. When your hub pushes a notification, Google receives it and can crawl and index the new page immediately. This is significantly faster than waiting for Googlebot's regular crawl cycle.

Google's official documentation recommends WebSub as a method for notifying Google of new content. Combined with IndexNow (which notifies Bing, Yandex, and other search engines), WebSub and IndexNow together cover the major search engines for instant indexing.

RSS Feed Readers

Services like Feedly, Inoreader, and NewsBlur subscribe to WebSub-enabled feeds. When you publish, your new post appears in subscribers' feed readers within seconds — not on the next polling cycle (which can be 15-60 minutes for free-tier feed reader accounts).

Fediverse Platforms

Mastodon and other ActivityPub-compatible platforms use WebSub for content distribution. If your blog federates via WriteFreely or the WordPress ActivityPub plugin, WebSub ensures that new posts reach fediverse followers in real time.

Aggregators and Newsletter Tools

Many content aggregation services and automated newsletter tools subscribe to WebSub hubs to detect new content. Publishing via WebSub ensures your content is included in relevant aggregations as quickly as possible.

WebSub vs. IndexNow

WebSub and IndexNow serve similar purposes but through different mechanisms:

WebSub is a publish-subscribe protocol. Subscribers register with your hub and receive push notifications. It is primarily used by Google, feed readers, and fediverse platforms.

IndexNow is a direct ping protocol. You send a notification directly to search engines (Bing, Yandex, Seznam, Naver) telling them specific URLs have changed. Google does not officially support IndexNow (they have their own mechanisms, including WebSub).

The optimal strategy is to implement both. WebSub covers Google and the feed reader ecosystem. IndexNow covers Bing and other search engines. Together, they ensure every major search engine and content platform knows about your new content within seconds.

Combining With Other Protocols

Our network uses a three-protocol stack for content distribution:

1. WebSub — pushes to Google and feed readers
2. IndexNow — pushes to Bing, Yandex, and other search engines
3. Sitemap ping — notifies search engines of sitemap updates (legacy but still functional)

All three protocols fire automatically on every deployment. The combined result is that new content is typically indexed by Google within 5 minutes and by Bing within 15 minutes.

Measuring the Impact

Before WebSub

• Average indexing latency: 8-14 hours for new pages
• Some pages took 48+ hours to appear in search results
• Feed reader updates were delayed 15-60 minutes

After WebSub

• Average indexing latency: under 5 minutes for new pages
• Consistent sub-minute feed reader updates
• Fediverse distribution in real time

The indexing speed improvement has a secondary SEO benefit: for time-sensitive content (market data, breaking analysis, seasonal guides), faster indexing means your content can rank for trending queries before competitors who rely on standard crawl cycles.

Implementation Time and Cost

Adding WebSub to an existing site with an RSS feed takes approximately 15 minutes:

• 5 minutes to add the hub declaration to your feed template
• 5 minutes to add the ping command to your deployment script
• 5 minutes to test and verify

The ongoing cost is $0. Google's public hub handles the relay infrastructure for free. The ping is a single HTTP request per deployment.

WebSub is one of those implementations where the effort is so minimal and the benefit so clear that there is no rational reason to skip it. Fifteen minutes of setup for instant content distribution across Google, feed readers, and the fediverse.

For the complete content distribution protocol stack and real estate analysis, see The Condo Trap and The $100 Dollar Network.