Jon Brodkin
– Sep 18, 2020 5:53 pm UTC

The Internet Archive and Cloudflare have teamed up to archive the content of websites that use Cloudflare’s Always Online service, increasing the odds that users will be able to view a recent version of a website during outages. The partnership will increase the number of webpages scanned by the Internet Archive, making the organization’s Wayback Machine more useful to Internet users in general.

“Websites that enable Cloudflare’s Always Online service will now have their content automatically archived, and if by chance the original host is not available to Cloudflare, then the Internet Archive will step in to make sure the pages get through to users,” said an announcement by Mark Graham, director of the Internet Archive’s Wayback Machine.

Cloudflare says its Always Online feature saves “a limited copy of your cached website to keep it online for your visitors” when the origin server is unavailable, ensuring that a website’s “most popular pages are represented.” Using the Wayback Machine will improve the Always Online service, Cloudflare CEO Matthew Prince said.

“The Internet Archive’s Wayback Machine has an impressive infrastructure that can archive the Web at scale,” Prince said.

The partnership will in turn improve the Wayback Machine’s ability to archive the Web. The nonprofit Internet Archive’s system doesn’t crawl the entire Web but has made more than 468 billion archived webpages available and is adding over 1 billion new archived URLs a day, Graham wrote. It does this “via a variety of different methods, such as ‘crawling’ from lists of millions of sites, as submitted by users via the Wayback Machine’s ‘Save Page Now’ feature, [websites] added to Wikipedia articles, referenced in Tweets, and based on a number of other ‘signals’ and sources, such [as] multiple feeds of ‘news’ stories,” Graham explained.

Cloudflare’s Always Online service is now one additional avenue for the Wayback Machine to find and archive websites. “As new URLs are added to sites that use that service they are submitted for archiving to the Wayback Machine,” Graham wrote. “In some cases this will be the first time a URL will be seen by our system and result in a ‘First Archive’ event.” In all cases, these newly archived URLs “will be available to anyone who uses the Wayback Machine.”

Graham predicts that the partnership will let the Internet Archive do a “better job of backing up more of the public Web, and in so doing help make the Web more useful and reliable.”

Users who reach an archived version of a website when a server is offline will see only static pages. “Visitors who interact with dynamic parts of a website, such as a shopping cart or comment box, will see an error page caused by the offline origin web server,” Cloudflare said in a new support page that describes how the integration works. When a website is unreachable, Cloudflare says it will first check “Cloudflare’s cache for a stale or expired version of your website. When none exists, Cloudflare will go to the Internet Archive to fetch and serve static portions of your website.”

The Internet Archive integration is available to Cloudflare’s free users but will only back up the website every 30 days. Cloudflare’s paying customers will get more frequent backups, every 15 days for Pro users and every five days for Business and Enterprise users.

CNMN Collection
WIRED Media Group
© 2020 Condé Nast. All rights reserved. Use of and/or registration on any portion of this site constitutes acceptance of our User Agreement (updated 1/1/20) and Privacy Policy and Cookie Statement (updated 1/1/20) and Ars Technica Addendum (effective 8/21/2018). Ars may earn compensation on sales from links on this site. Read our affiliate link policy.
Your California Privacy Rights | Do Not Sell My Personal Information
The material on this site may not be reproduced, distributed, transmitted, cached or otherwise used, except with the prior written permission of Condé Nast.
Ad Choices


Cloudflare, Wayback Machine, Internet Archive, World Wide Web

World news – CA – Wayback Machine and Cloudflare team up to archive more of the Web

En s’appuyant sur ses expertises dans les domaines du digital, des technologies et des process , CSS Engineering vous accompagne dans vos chantiers de transformation les plus ambitieux et vous aide à faire émerger de nouvelles idées, de nouvelles offres, de nouveaux modes de collaboration, de nouvelles manières de produire et de vendre.

CSS Engineering s’implique dans les projets de chaque client comme si c’était les siens. Nous croyons qu’une société de conseil devrait être plus que d’un conseiller. Nous nous mettons à la place de nos clients, pour aligner nos incitations à leurs objectifs, et collaborer pour débloquer le plein potentiel de leur entreprise. Cela établit des relations profondes et agréables.

Nos services:

  1. Création des sites web professionnels
  2. Hébergement web haute performance et illimité
  3. Vente et installation des caméras de vidéo surveillance
  4. Vente et installation des système de sécurité et d’alarme
  5. E-Marketing

Toutes nos réalisations ici


Please enter your comment!
Please enter your name here