Boost Your Site: Parramatta Web Design Fixes

Study of the properties of codes and their fitness

Coding theory is the study of the properties of codes and their respective fitness for specific applications. Codes are used for data compression, cryptography, error detection and correction, data transmission and data storage. Codes are studied by various scientific disciplines—such as information theory, electrical engineering, mathematics, linguistics, and computer science—for the purpose of designing efficient and reliable data transmission methods. This typically involves the removal of redundancy and the correction or detection of errors in the transmitted data.

There are four types of coding:^[1]

1. Data compression (or source coding)
2. Error control (or channel coding)
3. Cryptographic coding
4. Line coding

Data compression attempts to remove unwanted redundancy from the data from a source in order to transmit it more efficiently. For example, DEFLATE data compression makes files smaller, for purposes such as to reduce Internet traffic. Data compression and error correction may be studied in combination.

Error correction adds useful redundancy to the data from a source to make the transmission more robust to disturbances present on the transmission channel. The ordinary user may not be aware of many applications using error correction. A typical music compact disc (CD) uses the Reed–Solomon code to correct for scratches and dust. In this application the transmission channel is the CD itself. Cell phones also use coding techniques to correct for the fading and noise of high frequency radio transmission. Data modems, telephone transmissions, and the NASA Deep Space Network all employ channel coding techniques to get the bits through, for example the turbo code and LDPC codes.

History of coding theory

[edit]

This section is an excerpt from History of information theory.[edit]

The decisive event which established the discipline of information theory, and brought it to immediate worldwide attention, was the publication of Claude E. Shannon's classic paper "A Mathematical Theory of Communication" in the Bell System Technical Journal in July and October 1948.

In this revolutionary and groundbreaking paper, the work for which Shannon had substantially completed at Bell Labs by the end of 1944, Shannon for the first time introduced the qualitative and quantitative model of communication as a statistical process underlying information theory, opening with the assertion that

"The fundamental problem of communication is that of reproducing at one point, either exactly or approximately, a message selected at another point."

With it came the ideas of

• the information entropy and redundancy of a source, and its relevance through the source coding theorem;
• the mutual information, and the channel capacity of a noisy channel, including the promise of perfect loss-free communication given by the noisy-channel coding theorem;
• the practical result of the Shannon–Hartley law for the channel capacity of a Gaussian channel; and of course
• the bit - a new way of seeing the most fundamental unit of information.

Shannon’s paper focuses on the problem of how to best encode the information a sender wants to transmit. In this fundamental work he used tools in probability theory, developed by Norbert Wiener, which were in their nascent stages of being applied to communication theory at that time. Shannon developed information entropy as a measure for the uncertainty in a message while essentially inventing the field of information theory.

The binary Golay code was developed in 1949. It is an error-correcting code capable of correcting up to three errors in each 24-bit word, and detecting a fourth.

Richard Hamming won the Turing Award in 1968 for his work at Bell Labs in numerical methods, automatic coding systems, and error-detecting and error-correcting codes. He invented the concepts known as Hamming codes, Hamming windows, Hamming numbers, and Hamming distance.

In 1972, Nasir Ahmed proposed the discrete cosine transform (DCT), which he developed with T. Natarajan and K. R. Rao in 1973.^[2] The DCT is the most widely used lossy compression algorithm, the basis for multimedia formats such as JPEG, MPEG and MP3.

Source coding

[edit]

The aim of source coding is to take the source data and make it smaller.

Data can be seen as a random variable $\displaystyle X:\Omega \to \mathcal X$, where $\displaystyle x\in \mathcal X$ appears with probability $\displaystyle \mathbb P [X=x]$.

Data are encoded by strings (words) over an alphabet $\displaystyle \Sigma$.

A code is a function

$\displaystyle C:\mathcal X\to \Sigma ^*$ (or $\displaystyle \Sigma ^+$ if the empty string is not part of the alphabet).

$\displaystyle C(x)$ is the code word associated with $\displaystyle x$.

Length of the code word is written as

$\displaystyle l(C(x)).$

Expected length of a code is

$\displaystyle l(C)=\sum _x\in \mathcal Xl(C(x))\mathbb P [X=x].$

The concatenation of code words $\displaystyle C(x_1,\ldots ,x_k)=C(x_1)C(x_2)\cdots C(x_k)$.

The code word of the empty string is the empty string itself:

$\displaystyle C(\epsilon )=\epsilon$

1. $\displaystyle C:\mathcal X\to \Sigma ^*$ is non-singular if injective.
2. $\displaystyle C:\mathcal X^*\to \Sigma ^*$ is uniquely decodable if injective.
3. $\displaystyle C:\mathcal X\to \Sigma ^*$ is instantaneous if $\displaystyle C(x_1)$ is not a proper prefix of $\displaystyle C(x_2)$ (and vice versa).

Entropy of a source is the measure of information. Basically, source codes try to reduce the redundancy present in the source, and represent the source with fewer bits that carry more information.

Data compression which explicitly tries to minimize the average length of messages according to a particular assumed probability model is called entropy encoding.

Various techniques used by source coding schemes try to achieve the limit of entropy of the source. C(x) ≥ H(x), where H(x) is entropy of source (bitrate), and C(x) is the bitrate after compression. In particular, no source coding scheme can be better than the entropy of the source.

Facsimile transmission uses a simple run length code. Source coding removes all data superfluous to the need of the transmitter, decreasing the bandwidth required for transmission.

The purpose of channel coding theory is to find codes which transmit quickly, contain many valid code words and can correct or at least detect many errors. While not mutually exclusive, performance in these areas is a trade-off. So, different codes are optimal for different applications. The needed properties of this code mainly depend on the probability of errors happening during transmission. In a typical CD, the impairment is mainly dust or scratches.

CDs use cross-interleaved Reed–Solomon coding to spread the data out over the disk.^[3]

Although not a very good code, a simple repeat code can serve as an understandable example. Suppose we take a block of data bits (representing sound) and send it three times. At the receiver we will examine the three repetitions bit by bit and take a majority vote. The twist on this is that we do not merely send the bits in order. We interleave them. The block of data bits is first divided into 4 smaller blocks. Then we cycle through the block and send one bit from the first, then the second, etc. This is done three times to spread the data out over the surface of the disk. In the context of the simple repeat code, this may not appear effective. However, there are more powerful codes known which are very effective at correcting the "burst" error of a scratch or a dust spot when this interleaving technique is used.

Other codes are more appropriate for different applications. Deep space communications are limited by the thermal noise of the receiver which is more of a continuous nature than a bursty nature. Likewise, narrowband modems are limited by the noise, present in the telephone network and also modeled better as a continuous disturbance.^{[citation needed]} Cell phones are subject to rapid fading. The high frequencies used can cause rapid fading of the signal even if the receiver is moved a few inches. Again there are a class of channel codes that are designed to combat fading.^{[citation needed]}

The term algebraic coding theory denotes the sub-field of coding theory where the properties of codes are expressed in algebraic terms and then further researched.^{[citation needed]}

Algebraic coding theory is basically divided into two major types of codes:^{[citation needed]}

• Linear block codes
• Convolutional codes

It analyzes the following three properties of a code – mainly:^{[citation needed]}

• Code word length
• Total number of valid code words
• The minimum distance between two valid code words, using mainly the Hamming distance, sometimes also other distances like the Lee distance

Linear block codes have the property of linearity, i.e. the sum of any two codewords is also a code word, and they are applied to the source bits in blocks, hence the name linear block codes. There are block codes that are not linear, but it is difficult to prove that a code is a good one without this property.^[4]

Linear block codes are summarized by their symbol alphabets (e.g., binary or ternary) and parameters (n,m,d_min)^[5] where

1. n is the length of the codeword, in symbols,
2. m is the number of source symbols that will be used for encoding at once,
3. d_min is the minimum hamming distance for the code.

There are many types of linear block codes, such as

1. Cyclic codes (e.g., Hamming codes)
2. Repetition codes
3. Parity codes
4. Polynomial codes (e.g., BCH codes)
5. Reed–Solomon codes
6. Algebraic geometric codes
7. Reed–Muller codes
8. Perfect codes
9. Locally recoverable code

Block codes are tied to the sphere packing problem, which has received some attention over the years. In two dimensions, it is easy to visualize. Take a bunch of pennies flat on the table and push them together. The result is a hexagon pattern like a bee's nest. But block codes rely on more dimensions which cannot easily be visualized. The powerful (24,12) Golay code used in deep space communications uses 24 dimensions. If used as a binary code (which it usually is) the dimensions refer to the length of the codeword as defined above.

The theory of coding uses the N-dimensional sphere model. For example, how many pennies can be packed into a circle on a tabletop, or in 3 dimensions, how many marbles can be packed into a globe. Other considerations enter the choice of a code. For example, hexagon packing into the constraint of a rectangular box will leave empty space at the corners. As the dimensions get larger, the percentage of empty space grows smaller. But at certain dimensions, the packing uses all the space and these codes are the so-called "perfect" codes. The only nontrivial and useful perfect codes are the distance-3 Hamming codes with parameters satisfying (2^r – 1, 2^r – 1 – r, 3), and the [23,12,7] binary and [11,6,5] ternary Golay codes.^[4][5]

Another code property is the number of neighbors that a single codeword may have.^[6] Again, consider pennies as an example. First we pack the pennies in a rectangular grid. Each penny will have 4 near neighbors (and 4 at the corners which are farther away). In a hexagon, each penny will have 6 near neighbors. When we increase the dimensions, the number of near neighbors increases very rapidly. The result is the number of ways for noise to make the receiver choose a neighbor (hence an error) grows as well. This is a fundamental limitation of block codes, and indeed all codes. It may be harder to cause an error to a single neighbor, but the number of neighbors can be large enough so the total error probability actually suffers.^[6]

Properties of linear block codes are used in many applications. For example, the syndrome-coset uniqueness property of linear block codes is used in trellis shaping,^[7] one of the best-known shaping codes.

The idea behind a convolutional code is to make every codeword symbol be the weighted sum of the various input message symbols. This is like convolution used in LTI systems to find the output of a system, when you know the input and impulse response.

So we generally find the output of the system convolutional encoder, which is the convolution of the input bit, against the states of the convolution encoder, registers.

Fundamentally, convolutional codes do not offer more protection against noise than an equivalent block code. In many cases, they generally offer greater simplicity of implementation over a block code of equal power. The encoder is usually a simple circuit which has state memory and some feedback logic, normally XOR gates. The decoder can be implemented in software or firmware.

The Viterbi algorithm is the optimum algorithm used to decode convolutional codes. There are simplifications to reduce the computational load. They rely on searching only the most likely paths. Although not optimum, they have generally been found to give good results in low noise environments.

Convolutional codes are used in voiceband modems (V.32, V.17, V.34) and in GSM mobile phones, as well as satellite and military communication devices.

Cryptography or cryptographic coding is the practice and study of techniques for secure communication in the presence of third parties (called adversaries).^[8] More generally, it is about constructing and analyzing protocols that block adversaries;^[9] various aspects in information security such as data confidentiality, data integrity, authentication, and non-repudiation^[10] are central to modern cryptography. Modern cryptography exists at the intersection of the disciplines of mathematics, computer science, and electrical engineering. Applications of cryptography include ATM cards, computer passwords, and electronic commerce.

Cryptography prior to the modern age was effectively synonymous with encryption, the conversion of information from a readable state to apparent nonsense. The originator of an encrypted message shared the decoding technique needed to recover the original information only with intended recipients, thereby precluding unwanted persons from doing the same. Since World War I and the advent of the computer, the methods used to carry out cryptology have become increasingly complex and its application more widespread.

Modern cryptography is heavily based on mathematical theory and computer science practice; cryptographic algorithms are designed around computational hardness assumptions, making such algorithms hard to break in practice by any adversary. It is theoretically possible to break such a system, but it is infeasible to do so by any known practical means. These schemes are therefore termed computationally secure; theoretical advances, e.g., improvements in integer factorization algorithms, and faster computing technology require these solutions to be continually adapted. There exist information-theoretically secure schemes that provably cannot be broken even with unlimited computing power—an example is the one-time pad—but these schemes are more difficult to implement than the best theoretically breakable but computationally secure mechanisms.

A line code (also called digital baseband modulation or digital baseband transmission method) is a code chosen for use within a communications system for baseband transmission purposes.

Line coding is often used for digital data transport. It consists of representing the digital signal to be transported by an amplitude- and time-discrete signal that is optimally tuned for the specific properties of the physical channel (and of the receiving equipment). The waveform pattern of voltage or current used to represent the 1s and 0s of a digital data on a transmission link is called line encoding. The common types of line encoding are unipolar, polar, bipolar, and Manchester encoding.

This article contains content that may be misleading to readers. Please help improve it by clarifying such content. Relevant discussion may be found on the talk page. (August 2012)

Another concern of coding theory is designing codes that help synchronization. A code may be designed so that a phase shift can be easily detected and corrected and that multiple signals can be sent on the same channel.^{[citation needed]}

Another application of codes, used in some mobile phone systems, is code-division multiple access (CDMA). Each phone is assigned a code sequence that is approximately uncorrelated with the codes of other phones.^{[citation needed]} When transmitting, the code word is used to modulate the data bits representing the voice message. At the receiver, a demodulation process is performed to recover the data. The properties of this class of codes allow many users (with different codes) to use the same radio channel at the same time. To the receiver, the signals of other users will appear to the demodulator only as a low-level noise.^{[citation needed]}

Another general class of codes are the automatic repeat-request (ARQ) codes. In these codes the sender adds redundancy to each message for error checking, usually by adding check bits. If the check bits are not consistent with the rest of the message when it arrives, the receiver will ask the sender to retransmit the message. All but the simplest wide area network protocols use ARQ. Common protocols include SDLC (IBM), TCP (Internet), X.25 (International) and many others. There is an extensive field of research on this topic because of the problem of matching a rejected packet against a new packet. Is it a new one or is it a retransmission? Typically numbering schemes are used, as in TCP. "RFC793". RFCS. Internet Engineering Task Force (IETF). September 1981.

Group testing uses codes in a different way. Consider a large group of items in which a very few are different in a particular way (e.g., defective products or infected test subjects). The idea of group testing is to determine which items are "different" by using as few tests as possible. The origin of the problem has its roots in the Second World War when the United States Army Air Forces needed to test its soldiers for syphilis.^[11]

Information is encoded analogously in the neural networks of brains, in analog signal processing, and analog electronics. Aspects of analog coding include analog error correction,^[12] analog data compression^[13] and analog encryption.^[14]

Neural coding is a neuroscience-related field concerned with how sensory and other information is represented in the brain by networks of neurons. The main goal of studying neural coding is to characterize the relationship between the stimulus and the individual or ensemble neuronal responses and the relationship among electrical activity of the neurons in the ensemble.^[15] It is thought that neurons can encode both digital and analog information,^[16] and that neurons follow the principles of information theory and compress information,^[17] and detect and correct^[18] errors in the signals that are sent throughout the brain and wider nervous system.

• Coding gain
• Covering code
• Error correction code
• Folded Reed–Solomon code
• Group testing
• Hamming distance, Hamming weight
• Lee distance
• List of algebraic coding theory topics
• Spatial coding and MIMO in multiple antenna research
  • Spatial diversity coding is spatial coding that transmits replicas of the information signal along different spatial paths, so as to increase the reliability of the data transmission.
  • Spatial interference cancellation coding
  • Spatial multiplex coding
• Timeline of information theory, data compression, and error correcting codes

• Elwyn R. Berlekamp (2014), Algebraic Coding Theory, World Scientific Publishing (revised edition), ISBN 978-9-81463-589-9.
• MacKay, David J. C. Information Theory, Inference, and Learning Algorithms Cambridge: Cambridge University Press, 2003. ISBN 0-521-64298-1
• Vera Pless (1982), Introduction to the Theory of Error-Correcting Codes, John Wiley & Sons, Inc., ISBN 0-471-08684-3.
• Randy Yates, A Coding Theory Tutorial.

World Wide Web Consortium / W3C

Logo since 1997
Abbreviation	W3C
Formation	1 October 1994; 30 years ago (1994-10-01)
Founder	Tim Berners-Lee
Type	Standards organization
Purpose	Developing protocols and guidelines that ensure long-term growth for the Web
Headquarters	Cambridge, Massachusetts, United States
Location	4 offices Main Office: MIT/CSAIL, US ERCIM, France Keio University/SFC, Japan Beihang University, China[1]
Coordinates	42°21′43″N 71°05′26″W / 42.36194°N 71.09056°W / 42.36194; -71.09056
Region served	Worldwide
Membership	460 member organizations[2]
CEO	Seth Dobbs
Staff	53[3]
Website	w3.org

The World Wide Web Consortium (W3C) is the main international standards organization for the World Wide Web. Founded in 1994 by Tim Berners-Lee, the consortium is made up of member organizations that maintain full-time staff working together in the development of standards for the World Wide Web. As of May 2025,^[update] W3C has 350 members.^[4] The organization has been led by CEO Seth Dobbs since October 2023.^[5] W3C also engages in education and outreach, develops software and serves as an open forum for discussion about the Web.

The World Wide Web Consortium (W3C) was founded in 1994 by Tim Berners-Lee after he left the European Organization for Nuclear Research (CERN) in October 1994.^[6] It was founded at the Massachusetts Institute of Technology (MIT) Laboratory for Computer Science with support from the European Commission, and the Defense Advanced Research Projects Agency, which had pioneered the ARPANET, the most direct predecessor to the modern Internet.^[7] It was located in Technology Square until 2004, when it moved, with the MIT Computer Science and Artificial Intelligence Laboratory, to the Stata Center.^[8]

The organization tries to foster compatibility and agreement among industry members in the adoption of new standards defined by the W3C. Incompatible versions of HTML are offered by different vendors, causing inconsistency in how web pages are displayed. The consortium tries to get all those vendors to implement a set of core principles and components that are chosen by the consortium.

It was originally intended that CERN host the European branch of W3C; however, CERN wished to focus on particle physics, not information technology. In April 1995, the French Institute for Research in Computer Science and Automation became the European host of W3C, with Keio University Research Institute at SFC becoming the Asian host in September 1996.^[9] Starting in 1997, W3C created regional offices around the world. As of September 2009, it had eighteen World Offices covering Australia, the Benelux countries (Belgium, Netherlands and Luxembourg), Brazil, China, Finland, Germany, Austria, Greece, Hong Kong, Hungary, India, Israel, Italy, South Korea, Morocco, South Africa, Spain, Sweden, and, as of 2016, the United Kingdom and Ireland.^[10]

In October 2012, W3C convened a community of major web players and publishers to establish a MediaWiki wiki that seeks to document open web standards called the WebPlatform and WebPlatform Docs.

In January 2013, Beihang University became the Chinese host.^[11]

In 2022 the W3C WebFonts Working Group won an Emmy Award from the National Academy of Television Arts and Sciences for standardizing font technology for custom downloadable fonts and typography for web and TV devices.^[12]

On 1 January 2023, it reformed as a public-interest 501(c)(3) non-profit organization.^[13][14] In October 2023, Seth Dobbs was named as the organization's chief executive officer.^[5]

W3C develops technical specifications for HTML5, CSS, SVG, WOFF, the Semantic Web stack, XML, and other technologies.^[15] Sometimes, when a specification becomes too large, it is split into independent modules that can mature at their own pace. Subsequent editions of a module or specification are known as levels and are denoted by the first integer in the title (e.g. CSS3 = Level 3). Subsequent revisions on each level are denoted by an integer following a decimal point (for example, CSS2.1 = Revision 1).

The W3C standard formation process is defined within the W3C process document, outlining four maturity levels through which each new standard or recommendation must progress.^[16]

After enough content has been gathered from 'editor drafts' and discussion, it may be published as a working draft (WD) for review by the community. A WD document is the first form of a standard that is publicly available. Commentary by virtually anyone is accepted, though no promises are made with regard to action on any particular element commented upon.^[16]

At this stage, the standard document may have significant differences from its final form. As such, anyone who implements WD standards should be ready to significantly modify their implementations as the standard matures.^[16]

A candidate recommendation is a version of a more mature standard than the WD. At this point, the group responsible for the standard is satisfied that the standard meets its goal. The purpose of the CR is to elicit aid from the development community on how implementable the standard is.^[16]

The standard document may change further, but significant features are mostly decided at this point. The design of those features can still change due to feedback from implementors.^[16]

A proposed recommendation is the version of a standard that has passed the prior two levels. The users of the standard provide input. At this stage, the document is submitted to the W3C Advisory Council for final approval.^[16]

While this step is important, it rarely causes any significant changes to a standard as it passes to the next phase.^[16]

This is the most mature stage of development. At this point, the standard has undergone extensive review and testing, under both theoretical and practical conditions. The standard is now endorsed by the W3C, indicating its readiness for deployment to the public, and encouraging more widespread support among implementors and authors.^[16]

Recommendations can sometimes be implemented incorrectly, partially, or not at all, but many standards define two or more levels of conformance that developers must follow if they wish to label their product as W3C-compliant.^[16]

A recommendation may be updated or extended by separately-published, non-technical errata or editor drafts until sufficient substantial edits accumulate for producing a new edition or level of the recommendation. Additionally, the W3C publishes various kinds of informative notes which are to be used as references.^[16]

Unlike the Internet Society and other international standards bodies, the W3C does not have a certification program. The W3C has decided, for now, that it is not suitable to start such a program, owing to the risk of creating more drawbacks for the community than benefits.^[16]

In January 2023, after 28 years of being jointly administered by the MIT Computer Science and Artificial Intelligence Laboratory (located in Stata Center) in the United States, the European Research Consortium for Informatics and Mathematics (in Sophia Antipolis, France),^[17] Keio University (in Japan) and Beihang University (in China), the W3C incorporated as a legal entity, becoming a public-interest not-for-profit organization.^[18]

The W3C has a staff team of 70–80 worldwide as of 2015^[update].^[19] W3C is run by a management team which allocates resources and designs strategy, led by CEO Jeffrey Jaffe^[20] (as of March 2010), former CTO of Novell. It also includes an advisory board that supports strategy and legal matters and helps resolve conflicts.^[21][22] The majority of standardization work is done by external experts in the W3C's various working groups.^[23]

The Consortium is governed by its membership. The list of members is available to the public.^[2] Members include businesses, nonprofit organizations, universities, governmental entities, and individuals.^[24]

Membership requirements are transparent except for one requirement: An application for membership must be reviewed and approved by the W3C. Many guidelines and requirements are stated in detail, but there is no final guideline about the process or standards by which membership might be finally approved or denied.^[25]

The cost of membership is given on a sliding scale, depending on the character of the organization applying and the country in which it is located.^[26] Countries are categorized by the World Bank's most recent grouping by gross national income per capita.^[27]

In 2012 and 2013, the W3C started considering adding DRM-specific Encrypted Media Extensions (EME) to HTML5, which was criticised as being against the openness, interoperability, and vendor neutrality that distinguished websites built using only W3C standards from those requiring proprietary plug-ins like Flash.^{[28][29][30][31][32]} On 18 September 2017, the W3C published the EME specification as a recommendation, leading to the Electronic Frontier Foundation's resignation from W3C.^[33][34] As feared by the opponents of EME, as of 2020^[update], none of the widely used Content Decryption Modules used with EME are available for licensing without a per-browser licensing fee.^[35][36]

W3C/Internet Engineering Task Force standards (over Internet protocol suite):

• About the World Wide Web Consortium
• W3C Technical Reports and Publications
• W3C Process Document
• W3C History
• How to read W3C specs