Jon Brodkin
– Sep 18, 2020 5:53 pm UTC

The Internet Archive and Cloudflare have teamed up to archive the content of websites that use Cloudflare’s Always Online service, increasing the odds that users will be able to view a recent version of a website during outages. The partnership will increase the number of webpages scanned by the Internet Archive, making the organization’s Wayback Machine more useful to Internet users in general.

Websites that enable Cloudflare’s Always Online service will now have their content automatically archived, and if by chance the original host is not available to Cloudflare, then the Internet Archive will step in to make sure the pages get through to users,” said an announcement by Mark Graham, director of the Internet Archive’s Wayback Machine.

Cloudflare says its Always Online feature savesa limited copy of your cached website to keep it online for your visitorswhen the origin server is unavailable, ensuring that a website’smost popular pages are represented.Using the Wayback Machine will improve the Always Online service, Cloudflare CEO Matthew Prince said.

The Internet Archive’s Wayback Machine has an impressive infrastructure that can archive the Web at scale,” Prince said.

The partnership will in turn improve the Wayback Machine’s ability to archive the Web. The nonprofit Internet Archive’s system doesn’t crawl the entire Web but has made more than 468 billion archived webpages available and is adding over 1 billion new archived URLs a day, Graham wrote. It does thisvia a variety of different methods, such as ‘crawlingfrom lists of millions of sites, as submitted by users via the Wayback Machine’s ‘Save Page Now’ feature, [websites] added to Wikipedia articles, referenced in Tweets, and based on a number of other ‘signalsand sources, such [as] multiple feeds of ‘newsstories,” Graham explained.

Cloudflare’s Always Online service is now one additional avenue for the Wayback Machine to find and archive websites. “As new URLs are added to sites that use that service they are submitted for archiving to the Wayback Machine,” Graham wrote. “In some cases this will be the first time a URL will be seen by our system and result in a ‘First Archiveevent.In all cases, these newly archived URLswill be available to anyone who uses the Wayback Machine.

Graham predicts that the partnership will let the Internet Archive do abetter job of backing up more of the public Web, and in so doing help make the Web more useful and reliable.

Users who reach an archived version of a website when a server is offline will see only static pages. “Visitors who interact with dynamic parts of a website, such as a shopping cart or comment box, will see an error page caused by the offline origin web server,” Cloudflare said in a new support page that describes how the integration works. When a website is unreachable, Cloudflare says it will first checkCloudflare’s cache for a stale or expired version of your website. When none exists, Cloudflare will go to the Internet Archive to fetch and serve static portions of your website.

The Internet Archive integration is available to Cloudflare’s free users but will only back up the website every 30 days. Cloudflare’s paying customers will get more frequent backups, every 15 days for Pro users and every five days for Business and Enterprise users.

CNMN Collection
WIRED Media Group
© 2020 Condé Nast. All rights reserved. Use of and/or registration on any portion of this site constitutes acceptance of our User Agreement (updated 1/1/20) and Privacy Policy and Cookie Statement (updated 1/1/20) and Ars Technica Addendum (effective 8/21/2018). Ars may earn compensation on sales from links on this site. Read our affiliate link policy.
Your California Privacy Rights | Do Not Sell My Personal Information
The material on this site may not be reproduced, distributed, transmitted, cached or otherwise used, except with the prior written permission of Condé Nast.
Ad Choices


Cloudflare, Wayback Machine, Internet Archive, World Wide Web

World news – THAT – Wayback Machine and Cloudflare team up to archive more of the Web

Building on its expertise in the areas of digital, technologies and processes , CSS Engineering you in your most ambitious transformation projects and helps you bring out new ideas, new offers, new modes of collaboration, new ways of producing and selling.

CSS Engineering is involved in projects each customer as if it were his own. We believe a consulting company should be more than an advisor. We put ourselves in the place of our customers, to align we incentives to their goals, and collaborate to unlock the full potential their business. This establishes deep relationships and enjoyable.

Our services:

  1. Create professional websites
  2. Hosting high performance and unlimited
  3. Sale and video surveillance cameras installation
  4. Sale and Installation of security system and alarm
  5. E-Marketing

All our achievements here


Please enter your comment!
Please enter your name here