Cyclic redundancy check


Cyclic redundancy check (CRC)

A cyclic redundancy check (CRC) is an Error detection and correction code commonly used in digital Telecommunications networks and storage devices to detect accidental changes to raw data. Blocks of data entering these systems get a short check value attached, based on the remainder of a Polynomial long division|polynomial division of their contents. On retrieval, the calculation is repeated and, in the event the check values do not match, corrective action can be taken against data corruption. CRCs can be used for Error correcting code|error correction, see Mathematics of cyclic redundancy checks# bitfilters.

CRCs are so called because the check (data verification) value is a redundancy (it expands the message without adding Entropy information) and the CRC algorithm is based on Cyclic codecyclic codes. CRCs are popular because they are simple to implement in binary Computer , easy to analyze mathematically, and particularly good at detecting common errors caused by Noise (electronics)|noise in transmission channels. Because the check value has a fixed length, the function that generates it is occasionally used as a hash function.

The CRC was invented by W. Wesley Peterson in 1961; the 32-bit CRC function of Ethernet and many other standards is the work of several researchers and was published in 1975.

Contents

Introduction

CRCs are based on the theory of cyclic error-correcting codes. The use of systematic cyclic codes, which encode messages by adding a fixed-length check value, for the purpose of error detection in communication networks, was first proposed by W. Wesley Peterson in 1961. Cyclic codes are not only simple to implement but have the benefit of being particularly well suited for the detection of burst errors: contiguous sequences of erroneous data symbols in messages. This is important because burst errors are common transmission errors in many communication channels, including magnetic and optical storage devices. Typically an n-bit CRC applied to a data block of arbitrary length will detect any single error burst not longer than n bits and will detect a fraction of all longer error bursts.

Specification of a CRC code requires definition of a so-called generator polynomial. This polynomial becomes the divisor in a polynomial long division, which takes the message as the dividend and in which the quotient is discarded and the remainder becomes the result. The important caveat is that the polynomial coefficients are calculated according to the arithmetic of a finite field, so the addition operation can always be performed bitwise-parallel (there is no carry between digits). The length of the remainder is always less than the length of the generator polynomial, which therefore determines how long the result can be.

In practice, all commonly used CRCs employ the Galois field of two elements, GF(2). The two elements are usually called 0 and 1, comfortably matching computer architecture.

A CRC is called an n-bit CRC when its check value is n bits long. For a given n, multiple CRCs are possible, each with a different polynomial. Such a polynomial has highest degree n, which means it has terms. In other words, the polynomial has a length of ; its encoding requires bits. Note that most polynomial specifications either drop the Most significant MSB or Least significant LSB, since they are always 1. The CRC and associated polynomial typically have a name of the form CRC-n-XXX as in the #table below.

The simplest error-detection system, the parity bit, is in fact a trivial 1-bit CRC: it uses the generator polynomial (two terms), and has the name CRC-1.

Application

A CRC-enabled device calculates a short, fixed-length binary sequence, known as the check value or CRC, for each block of data to be sent or stored and appends it to the data, forming a codeword.

When a codeword is received or read, the device either compares its check value with one freshly calculated from the data block, or equivalently, performs a CRC on the whole codeword and compares the resulting check value with an expected residue constant.

If the CRC values do not match, then the block contains a data error.

The device may take corrective action, such as rereading the block or requesting that it be sent again. Otherwise, the data is assumed to be error-free (though, with some small probability, it may contain undetected errors; this is the fundamental nature of error-checking).

Cyclic redundancy check

Data integrity

CRCs are specifically designed to protect against common types of errors on communication channels, where they can provide quick and reasonable assurance of the data integrity of messages delivered. However, they are not suitable for protecting against intentional alteration of data.

Firstly, as there is no authentication, an attacker can edit a message and recompute the CRC without the substitution being detected. When stored alongside the data, CRCs and cryptographic hash functions by themselves do not protect against intentional modification of data. Any application that requires protection against such attacks must use cryptographic authentication mechanisms, such as message authentication CRC codes or digital signatures (which are commonly based on cryptographic hash functions).

Secondly, unlike cryptographic hash functions, CRC is an easily reversible function, which makes it unsuitable for use in digital signatures.

Thirdly, CRC is a linear function with a property that as a result, even if the CRC is encrypted with a stream cipher that uses Exclusive XOR as its combining operation (or Block cipher mode of block cipher which effectively turns it into a stream cipher, such as OFB or CFB), both the message and the associated CRC can be manipulated without knowledge of the encryption key; this was one of the well-known design flaws of the Wired Equivalent Privacy (WEP) protocol.

CRC-32 algorithm

This is the algorithm for the CRC-32 variant of CRC. The CRCTable is a memoization of a calculation that would have to be repeated for each byte of the message.

span style="color:blue;"Function/spanCRC32 span style="color:blue;"Input:/span Data: Bytes span style="color:green;"Array of bytes/span span style="color:blue;"Output:/span crc32: UInt32 span style="color:green;"32-bit unsigned crc-32 value/span 
span style="color:green;"Initialize crc-32 to starting value/span crc32 ← 0xffffffff 
span style="color:blue;"for each/span byte span style="color:blue;"in/span data span style="color:blue;"do/span nLookupIndex ← (crc32 xor byte) and 0xFF; crc32 ← (crc shr 8) xor CRCTable[nLookupIndex] span style="color:green;"//CRCTable is an array of 256 32-bit constants/span 
span style="color:green;"Finalize the CRC-32 value by inverting all the bits/span crc32 ← crc32 xor 0xFFFFFFFF 
span style="color:blue;"return/span crc32



Mathematics

Mathematical analysis of this division-like process reveals how to select a divisor that guarantees good error-detection properties. In this analysis, the digits of the bit strings are taken as the coefficients of a polynomial in some variable x—coefficients that are elements of the finite field GF(2), instead of more familiar numbers. The set of binary polynomials is a mathematical ring.

Designing polynomials

The selection of the generator polynomial is the most important part of implementing the CRC algorithm. The polynomial must be chosen to maximize the error-detecting capabilities while minimizing overall collision probabilities.

The most important attribute of the polynomial is its length (largest degree(exponent) +1 of any one term in the polynomial), because of its direct influence on the length of the computed check value.

The most commonly used polynomial lengths are:

  • 9 bits (CRC-8)
  • 17 bits (CRC-16)
  • 33 bits (CRC-32)
  • 65 bits (CRC-64)

A CRC is called an n-bit CRC when its check value is n-bits. For a given n, multiple CRCs are possible, each with a different polynomial. Such a polynomial has highest degree n, and hence terms (the polynomial has a length of ). The remainder has length n. The CRC has a name of the form CRC-n-XXX.

The design of the CRC polynomial depends on the maximum total length of the block to be protected (data + CRC bits), the desired error protection features, and the type of resources for implementing the CRC, as well as the desired performance. A common misconception is that the “best” CRC polynomials are derived from either irreducible polynomials or irreducible polynomials times the factor , which adds to the CRC code the ability to detect all errors affecting an odd number of bits. We can improve this situation. If we use the generator polynomial g(x) = p(x)(1 + x), where p(x) is a primitive polynomial of degree r – 1, then the maximal total block length is 2^{r – 1} – 1, and the code is able to detect single, double, triple and any odd number of errors.

A polynomial g(x) that admits other factorizations may be chosen then so as to balance the maximal total blocklength with a desired error detection power. The BCH codes are a powerful class of such polynomials. They subsume the two examples above. Regardless of the reducibility properties of a generator polynomial of degree r, if it includes the “+1” term, the code will be able to detect error patterns that are confined to a window of r contiguous bits. These patterns are called “error bursts”.

Specification

The concept of the CRC as an error-detecting code gets complicated when an implementer or standards committee uses it to design a practical system. Here are some of the complications:

  • Sometimes an implementation prefixes a fixed bit pattern to the bitstream to be checked. This is useful when clocking errors might insert 0-bits in front of a message, an alteration that would otherwise leave the check value unchanged.
  • Usually, but not always, an implementation appends n 0-bits (n being the size of the CRC) to the bitstream to be checked before the polynomial division occurs. Such appending is explicitly demonstrated in the Computation of cyclic redundancy checks of CRC article. This has the convenience that the remainder of the original bitstream with the check value appended is exactly zero, so the CRC can be checked simply by performing the polynomial division on the received bitstream and comparing the remainder with zero. Due to the associative and commutative properties of the exclusive-or operation, practical table driven implementations can obtain a result numerically equivalent to zero-appending without explicitly appending any zeroes, by using an equivalent,

Standards and common use

Numerous varieties of cyclic redundancy checks have been incorporated into technical standards. By no means does one CRC algorithm, or one of each degree, suit every purpose; Koopman and Chakravarty recommend selecting a polynomial according to the application requirements and the expected distribution of message lengths. There are three polynomials reported for CRC-12,

The polynomials commonly applied are not the most efficient ones possible. Since 1993, Koopman, Castagnoli and others have surveyed the space of polynomials between 3 and 64 bits in size, finding examples that have much better performance (in terms of Hamming distance for a given message size) than the polynomials of earlier protocols, and publishing the best of these with the aim of improving the error detection capacity of future standards. and Hammond, Brown and Liu’s report for the Rome Laboratory, published in May. Both reports contained contributions from the other team. During December 1975, Brayer and Hammond presented their work in a paper at the IEEE National Telecommunications Conference: the IEEE CRC-32 polynomial is the generating polynomial of a Hamming code and was selected for its error detection performance. Even so, the Castagnoli CRC-32C polynomial used in iSCSI or SCTP matches its performance on messages from 58 bits to 131 kbits, and outperforms it in several size ranges including the two most common sizes of Internet packet.

External links

Implementations

CRC Catalogues

See Also on BitcoinWiki

Source

CRC checksum algorithm