Metalink

Metalink is an extensible that describes one or more available for . It specifies files appropriate for the user’s language and ; facilitates file verification and recovery from ; and lists alternate download sources ( ).

The metadata is encoded in and/or in an file with <tt>.meta4</tt> or <tt>.metalink</tt>. The duplicate download locations provide reliability in case one method fails. Some clients also achieve faster download speeds by allowing different chunks/segments of each file to be downloaded from multiple resources at the same time ().

Metalink supports listing multiple partial and full file hashes along with signatures. Most clients only support verifying MD5, SHA-1, and SHA-256, however. Besides FTP and HTTP locations and , it also supports listing the P2P methods , , or any other that uses a .

Contents

Development History

Metalink 3.0 was publicly released in 2005. It was designed to aid in downloading Linux and other large files on release day, when servers would be overloaded (each server would have to be tried manually) and to repair large downloads by replacing only the parts with errors instead of fully re-downloading them. It was initially adopted by , and was used by open source projects such as and . A community developed around it, more download programs supported it (including proprietary ones) and it saw commercial adoption. In 2008, the community took their work to the which resulted in Metalink 4.0 in 2010, described in a Standards Track . Metalink 3.0 (with the extension <tt>.metalink</tt>) and Metalink 4.0 (with the extension <tt>.meta4</tt>) are incompatible because they have a slightly different format. In 2011, another Standards Track described Metalink in .

Client programs

  • aria2 is a utility for downloading files. The supported protocols are , , , BitTorrent, and Metalink.
  • is a computer software project providing a library and command-line tool for transferring data using various protocols.
  • (), a Firefox extension and download manager that has basic Metalink support.
  • (), a Firefox extension for integrating with many Download Managers, supports Metalink.
  • (), a Windows Download Manager, supports Metalink (HTTP/FTP) in version 2.5.
  • 6, a Windows Download Manager, supports Metalink (HTTP/FTP/BitTorrent).
  • is an open source download manager written in .
  • (), ‘s native download manager, supports Metalink in .
  • , a Windows Download Manager, supports Metalink (HTTP/FTP).
  • (), a Mac/Unix/Windows client, supports Metalink downloads along with exporting a library to Metalink for sharing between users.
  • , a Windows GUI FTP client, supports Metalink for adding files to an FTP transfer queue and hash verification.
  • ‘s easyMule (version 1 is ) is a version of with Metalink support.
  • , an utility for -compatible Linux , uses Metalink internally, invisible to the user.
  • , the engine that powers and , supports Metalink for downloading. In the beginning it used aria2 internally, meanwhile (since Jul. 2010) its own libcurl based implementation

Client Libraries

  • libmetalink () is a Metalink library written in . It provides the parsing of Metalink XML files to programs written in C and uses or . It does not handle the actual downloading of files. It is used by and other client programs.
  • metalink-checker () is a command line downloader written in Python that supports multi-source downloads and chunk checksums, as well as checking mirrors. It can also be used as a Python library.

In use

has integrated Metalink into package management with and aria2. has integrated Metalink into package management with . has integrated Metalink into package management with and aria2.

, the Windows-based installer, uses Metadl () to download Ubuntu and takes advantage of Metalink’s features of higher availability and increased reliability. If there are errors in the download, they are repaired, instead of restarting the large download.

Appupdater () for Windows “is similar to apt-get or yum on Linux. It automates the process of installing and maintaining up to date versions of programs.”<!–Also, is this even relevant? Does Appupdater support metalink? –>

Currently, uses Metalinks to distribute their free office suite. offers Metalinks of their downloads. Consortium also provides a Metalink for their large protein database. , a Flash uses Metalinks for downloads and so does , the open-source mathematical software.

The software hosting and collaboration platform Origo generates Metalinks for all hosted releases.

The following Linux distributions use Metalink for distribution: , , , , , , , , , , , , , , , , , StartCom Enterprise Linux, , .

The following based distributions use Metalink for distribution: , , and .

File based search engine findthatfile provides search for Metalinks.

Metalink client feature comparison

A guide to which clients support which Metalink features:

Client
Licensing (MIT/X derivate license) () () Proprietary / $29.95 for Standard, $49.95 for Pro () Proprietary / () Proprietary / for personal, $36.95 commercial
Interface CLI GUI GUI GUI GUI GUI GUI GUI
Metalink 4.0
Runs on Linux
Runs on
Runs on
Resume Downloads
Segmented Downloads  ?
Multiple File support  ?
“location”  ?
OS  ?
Language  ?
HTTP with Transparent Negotiation  ?
Metalink/HTTP  ?
HTTP
FTP
BitTorrent
magnet  ?
MD5 Hashes  ?
SHA-1 Hashes
SHA-256 Hashes
Piece Hashes  ?
PGP Signatures  ?

No clients support at this time.

Metalink generation

Metalink is formatted as XML so it can be either hand-crafted or created with an XML library, useful for automation. These tools are specifically for Metalink.

  • Bouncer is “database driven mirror management app that has three main components – a bounce script, a sentry script to check mirrors and files for integrity and a set of web-based administration tools.” Bouncer supports Metalink via a patch that has not yet been integrated upstream.
  • Geo McFly () is an on the fly generator of metalinks based on the downloader’s geographical location, written in Python.
  • Metalink Editor (GPL) is a Cross Platform GUI Metalink editor written in Python which allows you to create and edit Metalinks.
  • Metalink Library (GPL) is a Cross Platform Metalink library for Python and PHP based on Metalink Editor with many automatic features and an easy command-line interface to generate Metalink and Torrent files.
  • Metalink Generator can also be used to create Metalinks by filling out a Web form.
  • Metalink tools (GPL) Unix/Windows command line tools for generating Metalinks with hashes and mirror/p2p lists.
  • Dynamic online metalinks based on metadata. Automatically created metalinks based on metadata published by each mirror.
  • MirrorManager () is used by the for dynamically listing mirrors.
  • MirrorBrain (, ) is a real-time Metalink generator and download redirector. It can either return Metalinks, or automatically redirect non-Metalink clients (per HTTP redirection) to a mirror server near them. Transparent negotiation of Metalinks, embedding PGP signatures, Metalink/HTTP and other advanced features are supported. It is used for instance by and .

Example Metalink 4.0 .meta4 file

Metalink 4.0 files have the extension <tt>.meta4</tt> and are text files. They are served with the application/metalink4+xml .

<source lang=”xml”>

<?xml version="1.0" encoding="UTF-8"?> <metalink xmlns="urn:ietf:params:xml:ns:metalink"> <published>2009-05-15T12:23:23Z</published> <file name="example.ext"> <size>14471447</size> <identity>Example</identity> <version>1.0</version> <language>en</language> <description> A description of the example file for download. </description> <hash type="sha-256">3d6fece8033d146d8611eab4f032df738c8c1283620fd02a1f2bfec6e27d590d</hash> <url location="de" priority="1">ftp://ftp.example.com/example.ext</url> <url location="fr" priority="1">http://example.com/example.ext</url> <metaurl mediatype="torrent" priority="2">http://example.com/example.ext.torrent</metaurl> </file> </metalink>

</source>

Example Metalink/HTTP header fields

Metalink in HTTP header fields makes use of existing standard such as , Link header fields (for mirrors and P2P), and Instance Digests (for hashes).

<pre> Etag: “thvDyvhfIqlvFe+A9MYgxAfm1q5=” Link: <http://www2.example.com/example.ext>; rel=duplicate Link: <ftp://ftp.example.com/example.ext>; rel=duplicate Link: <http://example.com/example.ext.torrent>; rel=describedby; type=”application/x-bittorrent” Link: <http://example.com/example.ext.meta4>; rel=describedby; type=”application/metalink4+xml” Link: <http://example.com/example.ext.asc>; rel=describedby; type=”application/pgp-signature” Digest: SHA-256=MWVkMWQxYTRiMzk5MDQ0MzI3NGU5NDEyZTk5OWY1ZGFmNzgyZTJlODYzYjRjYzFhOTlmNTQwYzI2M2QwM2U2MQ== </pre>

Example Metalink 3.0 .metalink file

Metalink 3.0 files have the extension <tt>.metalink</tt> and are text files.

<source lang=”xml”>

<?xml version="1.0" encoding="UTF-8"?> <metalink version="3.0" xmlns="http://www.metalinker.org/"> <files> <file name="example.ext"> <verification> <hash type="md5">example-md5-hash</hash> <hash type="sha1">example-sha1-hash</hash> <signature type="pgp"/> </verification> <resources> <url type="ftp" location="us" preference="90">ftp://ftp.example.com/example.ext</url> <url type="ftp" location="uk" preference="90">ftp://ftp.example.net/example.ext</url> <url type="http" location="us" preference="90">http://example.com/example.ext</url>  <url type="http" location="de" preference="90">http://example.net/example.ext</url> <url type="bittorrent" preference="100">http://example.org/example.ext.torrent</url> <url type="rsync"/> <url type="magnet"/> <url type="ed2k"/> </resources> </file> </files> </metalink>

</source>

See Also on BitcoinWiki

External links

Source

http://wikipedia.org/