How do I handle spam Webmentions?

Question

Accepted Answer

Just like email and blog comments, Webmentions can attract spam. The good news? The verification process and decentralized nature make spam harder than with traditional comments, and there are proven strategies to keep it under control.

Why Webmentions Are Naturally Spam-Resistant

Unlike comment forms, Webmentions require:

A real website: Spammer needs a publicly accessible URL
Actual linking: The source must actually link to your target
Verification: Your site confirms the link exists before accepting
Domain ownership: Harder to fake than email addresses

This creates higher barriers than filling out a comment form with fake data.

Prevention Strategies

1. Rate Limiting

Prevent bulk spam by limiting requests per IP:

const rateLimit = require('express-rate-limit');

const webmentionLimiter = rateLimit({
  windowMs: 15 * 60 * 1000, // 15 minutes
  max: 10, // 10 mentions per window
  message: 'Too many Webmentions from this IP',
  standardHeaders: true,
  legacyHeaders: false,
});

app.post('/webmention', webmentionLimiter, handleWebmention);

Per-domain limiting:

const mentionCounts = new Map();

function checkDomainRate(source) {
  const domain = new URL(source).hostname;
  const count = mentionCounts.get(domain) || 0;

  if (count > 5) { // 5 mentions per hour per domain
    throw new Error('Rate limit exceeded');
  }

  mentionCounts.set(domain, count + 1);

  // Reset after an hour
  setTimeout(() => {
    mentionCounts.set(domain, 0);
  }, 3600000);
}

2. Domain Verification

Reject Webmentions from suspicious domains:

async function isValidDomain(url) {
  const domain = new URL(url).hostname;

  // Block known spam patterns
  const spamPatterns = [
    /\d{5,}/, // Many numbers
    /[a-z]{20,}/, // Very long random strings
    /\.tk$|\.ml$|\.ga$/, // Free domains often used for spam
  ];

  if (spamPatterns.some(pattern => pattern.test(domain))) {
    return false;
  }

  // Check domain age (optional)
  try {
    const whois = await checkDomainAge(domain);
    if (whois.ageInDays < 30) {
      return false; // Domain too new
    }
  } catch (error) {
    // WHOIS lookup failed, allow it
  }

  return true;
}

3. Content Analysis

Analyze the mentioning content for spam indicators:

function analyzeContent(content) {
  const spamIndicators = {
    excessive_links: (content.match(/<a /g) || []).length > 5,
    suspicious_keywords: /viagra|casino|pills|SEO|cheap/i.test(content),
    excessive_caps: (content.match(/[A-Z]/g) || []).length / content.length > 0.3,
    short_content: content.length < 50,
    no_content: !content || content.trim().length === 0,
  };

  const spamScore = Object.values(spamIndicators)
    .filter(Boolean)
    .length;

  return spamScore >= 2; // Flag as spam if 2+ indicators
}

4. Blocklists

Maintain a blocklist of known spam domains:

const blockedDomains = new Set([
  'known-spam-site.com',
  'another-spammer.com',
]);

// Update from community blocklist
async function updateBlocklist() {
  const response = await fetch('https://webmention-blocklist.example.com/domains.json');
  const domains = await response.json();
  domains.forEach(domain => blockedDomains.add(domain));
}

function isBlocked(url) {
  const domain = new URL(url).hostname;
  return blockedDomains.has(domain);
}

5. Allowlists (Trusted Domains)

Auto-approve mentions from trusted sources:

const trustedDomains = new Set([
  'indieweb.org',
  'example.com',
  // Domains you trust
]);

async function processMention(source, target) {
  const domain = new URL(source).hostname;
  const needsModeration = !trustedDomains.has(domain);

  await saveMention({
    source,
    target,
    status: needsModeration ? 'pending' : 'approved',
    verified: true,
  });
}

Detection Strategies

1. Machine Learning

Train a simple classifier on your mention data:

const brain = require('brain.js');

// Train on your historical data
const net = new brain.NeuralNetwork();

net.train([
  { input: { links: 0.1, caps: 0.2, length: 0.8 }, output: { spam: 0 } },
  { input: { links: 0.9, caps: 0.8, length: 0.1 }, output: { spam: 1 } },
  // More training data...
]);

// Classify new mentions
function classifyMention(mention) {
  const features = {
    links: mention.linkCount / 10,
    caps: mention.capsRatio,
    length: Math.min(mention.length / 500, 1),
  };

  const result = net.run(features);
  return result.spam > 0.7; // Threshold for spam
}

2. Link Analysis

Check where links in the mention point:

async function analyzeLinkTargets(html) {
  const cheerio = require('cheerio');
  const $ = cheerio.load(html);
  const links = $('a[href]').map((i, el) => $(el).attr('href')).get();

  for (const link of links) {
    try {
      const domain = new URL(link).hostname;

      // Check against known malware/phishing databases
      if (await isKnownMalicious(domain)) {
        return { spam: true, reason: 'malicious link' };
      }

      // Check if link redirects to spam
      const finalUrl = await followRedirects(link);
      if (isSpamDomain(finalUrl)) {
        return { spam: true, reason: 'redirects to spam' };
      }
    } catch (error) {
      // Invalid URL in link
      return { spam: true, reason: 'invalid links' };
    }
  }

  return { spam: false };
}

3. Pattern Recognition

Detect spam patterns:

function detectSpamPatterns(mention) {
  const patterns = {
    // Same content sent to multiple targets
    duplicate_content: await checkDuplicateContent(mention.content),

    // Mention sent immediately after domain registered
    suspicious_timing: mention.domainAge < 7,

    // No microformats (legitimate mentions usually have them)
    no_microformats: !mention.hasAuthor && !mention.hasHentry,

    // Source and target unrelated
    unrelated_content: !hasCommonKeywords(mention.content, mention.target),

    // Suspicious author name
    fake_author: /^[A-Z][a-z]+\s[A-Z][a-z]+$/.test(mention.author.name),
  };

  return patterns;
}

Moderation Workflows

1. Automatic Approval with Manual Review

Auto-approve low-risk mentions, queue others:

async function moderateMention(mention) {
  const riskScore = calculateRiskScore(mention);

  if (riskScore < 0.3) {
    // Low risk: auto-approve
    mention.status = 'approved';
  } else if (riskScore < 0.7) {
    // Medium risk: queue for review
    mention.status = 'pending';
    await notifyModerator(mention);
  } else {
    // High risk: auto-reject but save
    mention.status = 'spam';
  }

  await saveMention(mention);
}

function calculateRiskScore(mention) {
  let score = 0;

  if (!mention.hasAuthor) score += 0.3;
  if (mention.linkCount > 3) score += 0.2;
  if (mention.domainAge < 30) score += 0.2;
  if (analyzeContent(mention.content)) score += 0.4;

  return score;
}

2. First-Time Approval

Require manual approval for first mention from a domain:

async function requiresApproval(source) {
  const domain = new URL(source).hostname;

  // Check if we've seen this domain before
  const previousMentions = await db.mentions.count({
    source: { $regex: domain },
    status: 'approved',
  });

  return previousMentions === 0;
}

3. Moderation Dashboard

Build a simple admin interface:

// API endpoint for pending mentions
app.get('/admin/mentions/pending', requireAuth, async (req, res) => {
  const pending = await db.mentions.find({ status: 'pending' })
    .sort({ receivedAt: -1 })
    .limit(50);

  res.json(pending);
});

// Approve/reject actions
app.post('/admin/mentions/:id/approve', requireAuth, async (req, res) => {
  await db.mentions.updateOne(
    { _id: req.params.id },
    { status: 'approved' }
  );
  res.json({ success: true });
});

app.post('/admin/mentions/:id/spam', requireAuth, async (req, res) => {
  const mention = await db.mentions.findOne({ _id: req.params.id });

  // Mark as spam
  await db.mentions.updateOne(
    { _id: req.params.id },
    { status: 'spam' }
  );

  // Add domain to blocklist
  const domain = new URL(mention.source).hostname;
  await db.blocklist.insert({ domain, reason: 'manual spam' });

  res.json({ success: true });
});

Vouch System

Implement the Vouch extension for trust networks:

// Accept vouches from trusted contacts
const trustedVouchers = new Set([
  'https://alice.example.com',
  'https://bob.example.com',
]);

app.post('/webmention', async (req, res) => {
  const { source, target, vouch } = req.body;

  // If vouch provided and from trusted source
  if (vouch && trustedVouchers.has(vouch)) {
    // Verify vouch page links to source
    const vouchValid = await verifyVouch(vouch, source);

    if (vouchValid) {
      // Auto-approve with valid vouch
      return processMention(source, target, { trusted: true });
    }
  }

  // Normal processing without vouch
  processMention(source, target);
  res.status(202).send('Accepted');
});

async function verifyVouch(vouchUrl, sourceUrl) {
  const response = await fetch(vouchUrl);
  const html = await response.text();
  return html.includes(sourceUrl);
}

Cleanup Strategies

1. Regular Audits

Periodically review approved mentions:

async function auditMentions() {
  const mentions = await db.mentions.find({
    status: 'approved',
    createdAt: { $lt: new Date(Date.now() - 30 * 24 * 60 * 60 * 1000) }
  });

  for (const mention of mentions) {
    // Re-verify mention still exists
    const stillValid = await verifyMention(mention.source, mention.target);

    if (!stillValid) {
      // Mark as deleted
      await db.mentions.updateOne(
        { _id: mention._id },
        { status: 'deleted', deletedAt: new Date() }
      );
    }
  }
}

// Run weekly
setInterval(auditMentions, 7 * 24 * 60 * 60 * 1000);

2. Bulk Spam Removal

Remove spam from a specific domain:

async function removeSpamFromDomain(domain) {
  // Find all mentions from domain
  const mentions = await db.mentions.find({
    source: { $regex: domain }
  });

  // Mark as spam
  await db.mentions.updateMany(
    { source: { $regex: domain } },
    { status: 'spam', markedAt: new Date() }
  );

  // Add to blocklist
  await db.blocklist.insert({ domain, reason: 'bulk spam removal' });

  return mentions.length;
}

Best Practices Summary

Prevention:

✅ Implement rate limiting
✅ Verify domain legitimacy
✅ Analyze content for spam indicators
✅ Maintain blocklists and allowlists

Detection:

✅ Check for duplicate content
✅ Analyze link targets
✅ Look for pattern anomalies
✅ Use community spam databases

Moderation:

✅ Auto-approve low-risk mentions
✅ Queue medium-risk for review
✅ Auto-reject high-risk
✅ Require approval for new domains

Cleanup:

✅ Regular audits
✅ Remove dead links
✅ Bulk spam removal tools
✅ Update blocklists

Community Resources

Webmention Blocklist: https://github.com/indieweb/webmention-blocklist
IndieWeb Anti-Spam: https://indieweb.org/anti-spam
Vouch Spec: https://indieweb.org/Vouch

The Balance

The goal isn't zero spam (impossible), but keeping spam at manageable levels while not blocking legitimate mentions. Start conservative, monitor results, and adjust thresholds based on your experience.

Most sites find that:

90%+ of mentions are legitimate
Auto-approval with smart filters works well
Manual review of ~5-10% is reasonable
Spam is much less than traditional comments

With proper strategies, Webmentions can be cleaner and easier to moderate than conventional commenting systems.