Just Solve the File Format Problem - User contributions [en]

Microsoft Compound File

2016-03-21T20:33:54Z

Gmcgath:

{{FormatInfo
|formattype=electronic
|subcat=Metaformats
|thiscat=Microsoft Compound File
|locfdd={{LoCFDD|fdd000380}}, {{LoCFDD|fdd000392}}
|pronom={{PRONOM|fmt/111}}
}}
'''Microsoft Compound File''' is a complex container format used by some versions of Microsoft Office, and other Microsoft applications. It has features similar to those of a [[filesystem]] format.

It is also known as '''Compound File Binary File Format''' ('''CFBF''' or '''CFB'''), '''Microsoft Compound Document File Format''', '''OLE Compound Document Format''', '''OLE2 Compound Document Format''', '''Composite Document File''', etc.

The format was not publicly documented by Microsoft until 2008.

It is (or was?) unofficially known as ''LAOLA File Format''.

== Identification ==
Files begin with signature bytes <code>D0 CF 11 E0 A1 B1 1A E1</code>.

Identifying the specific document format is difficult. This is one of the few formats for which the [[file command]] resorts to a hard-coded identification algorithm (see [https://github.com/file/file/blob/master/src/readcdf.c readcdf.c]).

== Related formats ==
See [[:Category:Microsoft Compound File]].

== Specifications ==
* [http://msdn.microsoft.com/en-us/library/dd942138.aspx MSDN: Compound File Binary File Format] → [MS-CFB] PDF
* [http://www.openoffice.org/sc/compdocfileformat.pdf OpenOffice.org's documentation]

== Programs, libraries, and utilities ==
* [http://www.mitec.cz/ssv.html Structured Storage Viewer]
* [https://github.com/libyal/libolecf/wiki libolecf]
* [http://www.decalage.info/python/olefileio OleFileIO_PL - a Python module to read MS OLE2 files]
* [http://videos.didierstevens.com/2014/08/26/oledump-py-beta/ oledump.py beta]
* [https://github.com/unixfreak0037/officeparser officeparser]
* [http://decalage.info/python/oletools python-oletools - python tools to analyze OLE files]

== Links ==
* [[Wikipedia:Compound File Binary Format|Wikipedia article]]
* [http://www.forensicswiki.org/wiki/OLE_Compound_File Forensics Wiki article]
* [http://www.joelonsoftware.com/items/2008/02/19.html Joel on Software: Why are the Microsoft Office file formats so complicated?]
* Some (older) reverse engineered information [http://stuff.mit.edu/afs/athena/astaff/project/mimeutils/share/laola/guide.html here] and a Perl module [http://stuff.mit.edu/afs/athena/astaff/project/mimeutils/share/laola/laola.pl here]
* [http://blog.avira.com/malicious-office-macros-dead/ Malicious Office macros are not dead]
* [http://decalage.info/file_formats_security/office MS Office 97-2003 legacy/binary formats security] - article with lots of resources on MS Office formats, including analysis techniques, tools and parsing libraries

== Editors' notes ==
TODO: Explain the relationship between Compound File format and the format/technology called '''COM Structured Storage''' (or '''OLE Structured Storage''').

[[Category:Document]]
[[Category:Microsoft]]

Electronic Publishing formats

2016-02-18T20:55:51Z

Gmcgath: Stub link for KFX

{{FormatInfo
|formattype=electronic
|thiscat=Electronic Publishing formats
|image=Book.jpg
}}

Formats specific to the storage and transmission of publications in electronic form.

== E-book formats ==

* [[AZW]]
* [[Comic Book Archive]] (.cbr, .cbz, .cbt, .cba, .cb7)
* [[DocBook]]
* [[EPUB]]
* [[IMP (Electronic publishing)|IMP]]
* [[KFX]] (Kindle Format X)
* [[LRF]]
* [[MOBI]]
* [[PRC]]

See also [[Document]], [[Markup]], and [[Web]], since e-publications might also be stored in formats (such as [[PDF]] and [[HTML]]) in those categories, and e-publishing formats may have roots in those formats.

== Other formats ==

* [[News Industry Text Format]]
* [[Portable Web Publications]]
* [[ScienceServer XML format]]

== Identifiers ==

Naming and numbering systems for identifying published material.

* [[ARK]]
* [[DOI]]
* [[ISBN]]
* [[ISSN]]

== Links ==
* [https://scholasticahq.com/innovations-in-scholarly-publishing/announcement/one-of-the-biggest-bottlenecks-in-open-access-publishing-is-typesetting-it-shouldn-t-be Innovations in Scholarly Publishing - One of the biggest bottlenecks in Open Access publishing is typesetting. It shouldn't be.]
* [http://apprenticealf.wordpress.com/2012/09/10/calibre-plugins-the-simplest-option-for-removing-most-ebook-drm/ DeDRM plugin for calibre: the simplest option for removing DRM from most ebooks]
* [http://www.nbcnews.com/tech/gadgets/you-dont-own-your-kindle-books-amazon-reminds-customer-f1C6626211 You don't own your Kindle books, Amazon reminds customer]
* [http://boingboing.net/2013/10/22/even-amazon-cant-keep-up-the.html Even Amazon can't keep up the "you only license ebooks" shuck]
* [http://allthingsd.com/20130906/how-a-man-in-austria-used-legos-to-hack-amazons-kindle-e-book-security/ How a Man in Austria Used Legos to Hack Amazon’s Kindle E-Book Security]
* [http://www.mobileread.com/forums/showpost.php?p=2754561&postcount=319 Adobe is apparently introducing new "hardened" e-book DRM that will break compatibility with older readers]
* [http://www.the-digital-reader.com/2014/02/03/adobe-require-new-epub-drm-july-expects-abandon-existing-users/ Another article on Adobe DRM move]
* [http://sourceforge.net/projects/libebook/ libebook: library to read various e-book formats]
* [http://www.theverge.com/2014/5/9/5688146/the-fight-to-save-endangered-ebooks The fight to save endangered ebooks]
* [http://www.loc.gov/preservation/resources/rfs/textmus.html Library of Congress Recommended Format Specifications: Textual Works]
* [http://www.theguardian.com/books/2014/aug/19/readers-absorb-less-kindles-paper-study-plot-ereader-digitisation Readers absorb less on Kindles than on paper, study finds]
* [http://publishersweekly.com/pw/by-topic/industry-news/libraries/article/64271-check-it-out-with-michael-kelley-how-libraries-preserve-e-books.html Michael Kelley: How Libraries Preserve E-books]
* [http://the-digital-reader.com/2014/10/06/adobe-spying-users-collecting-data-ebook-libraries/ Adobe is Spying on Users, Collecting Data on Their eBook Libraries]
* [http://go-to-hellman.blogspot.com/2014/10/correcting-misinformation-on-adobe.html Correcting Misinformation on the Adobe Privacy Gusher]
* [http://www.theguardian.com/books/booksblog/2012/jun/07/war-and-peace-ebook-nook War and Peace ebook readers find a surprise in its Nooks]
* [http://vgel.me/posts/cracking-online-textbook/ Getting a full PDF from a DRM-encumbered online textbook]

Piano Rolls

2014-07-08T10:21:07Z

Gmcgath: velocity-sensitive piano rolls

{{FormatInfo
|formattype=physical
|released=1896
}}

'''Piano Rolls''' are a form of punched media aimed at the specific application of reproducing music through specifying a sequence of piano keys to be pressed. A player piano reads the punched holes in the roll as it moves through the reader portion of the device, and this triggers the pressing of the appropriate keys under automated control. In the late 19th and early 20th centuries, player pianos competed with another newly-created medium, the [[Gramophone record|phonograph]], as the medium for the public to purchase and play "canned" music instead of doing live musical performances by hand. Famous musician John Philip Sousa famously denounced all of them as "infernal machines" that were killing real music.

Some piano roll formats include velocity information, and in the 1920s famous pianists such as Rachmaninoff and Gershwin made piano roll recordings. They lack many of the subtleties of a live performance, but they're free of the noise and distortion that characterized acoustic recordings of that period, and some of these recordings have been re-released on CD. Most piano rolls, though, include only attack and release information, producing a mechanical sound.

== Characteristics ==

Paper rolls, generally 11 1/4 inch wide. Well preserved ends should angle the paper toward the center where it meets in a tab that will have a hole to connect to the take up roller.

65 note scale introduced in 1896 with 6 holes to the inch on center. Rolls had pins on end.

88 note scale introduced in 1900 with 9 holes to the inch on center.

72 note scale introduced in 1902.

In 1909 United States standardized on the 88 note scale.

== References ==

* [http://en.wikipedia.org/wiki/Piano_roll Wikipedia entry for Piano roll]
* [http://www.player-care.com/rollsizes.html roll sizes]

ISO 8859-1

2014-06-26T15:19:47Z

Gmcgath: Not just for English.

{{FormatInfo
|formattype=electronic
|subcat=Character Encodings
}}
'''ISO 8859-1''', aka Latin-1, is the most widely used encoding of the [[ISO 8859]] family for text in English and other western European languages. [[Windows 1252]] encoding includes all the printing characters of ISO 8859-1 and includes additional printing characters by replacing some [[C1 controls|control characters of the C1 (128-159) range]] (and often web developers, e-mail software, and other people and programs that generate or transmit text, mistakenly identify Windows-1252-encoded documents as ISO-8859-1 in headers and parameters).

ISO 8859-1 was updated to [[ISO 8859-15]], called Latin-9, replacing some of the less used characters and adding the Euro (€) sign. However, those who have decided to update from Latin-1 have generally gone straight to [[Unicode]], so Latin-9 has not become a widespread replacement. [[ISO 8859-9]] or Latin-5, designed for use with Turkish scripts, also differs from Latin-1 in only a few code points.

The Latin-1, Latin-9, and Windows 1252 encodings all incorporate the [[ASCII]] characters in the first 128 code points (0-127), including the [[C0 controls]].

[http://www.kreativekorp.com/charset/encoding.php?file=iso-8859-1.kte Code table]

ISO 8859-7

2014-06-26T14:57:45Z

Gmcgath: Created page with "{{FormatInfo |formattype=electronic |subcat=Character Encodings |released=1987 }} '''ISO 8859-7''', aka Latin/Greek, is an encoding of the ISO 8859 family for Modern Greek..."

{{FormatInfo
|formattype=electronic
|subcat=Character Encodings
|released=1987
}}
'''ISO 8859-7''', aka Latin/Greek, is an encoding of the [[ISO 8859]] family for Modern Greek. A 2003 revision added the drachma, euro, and ypogegrammeni characters without changing any existing codes. The revision is technically called ISO/IEC 8859-7:2003.

The 8859-7 encoding incorporates the [[ASCII]] characters in the first 128 code points (0-127), including the [[C0 controls]].

[http://www.kreativekorp.com/charset/encoding.php?file=iso-8859-7.kte Code table]

ISO 8859-6

2014-06-26T14:24:06Z

Gmcgath: Created page with "{{FormatInfo |formattype=electronic |subcat=Character Encodings |released=1987 }} '''ISO 8859-6''' is an encoding of the ISO 8859 family for the Arabic script. Languages t..."

{{FormatInfo
|formattype=electronic
|subcat=Character Encodings
|released=1987
}}
'''ISO 8859-6''' is an encoding of the [[ISO 8859]] family for the Arabic script. Languages that extend the Arabic script may not be fully supported. The ordering of characters in 8859-6 is logical, not visual; the Arabic language is predominantly right-to-left, but there are important exceptions, such as numbers. The different connecting forms of Arabic letters don't have distinct encodings.

The ISO 8859-6 encoding incorporates the [[ASCII]] characters in the first 128 code points (0-127), including the [[C0 controls]].

[http://www.kreativekorp.com/charset/encoding.php?file=iso-8859-6.kte Code table]

ISO 8859-3

2014-06-26T14:01:35Z

Gmcgath: Created page with "{{FormatInfo |formattype=electronic |subcat=Character Encodings }} ISO 8859-3, aka Latin-3, an encoding of the ISO 8859 family for the Esperanto, Galician, Maltese, and Tu..."

{{FormatInfo
|formattype=electronic
|subcat=Character Encodings
}}
ISO 8859-3, aka Latin-3, an encoding of the [[ISO 8859]] family for the Esperanto, Galician, Maltese, and Turkish languages. [[ISO 8859-9]] (Latin-5) has superseded it for Turkish.

The Latin-3 encoding incorporates the [[ASCII]] characters in the first 128 code points (0-127), including the [[C0 controls]].

[http://www.kreativekorp.com/charset/encoding.php?file=iso-8859-3.kte Code table]

ISO 8859-2

2014-06-26T13:39:57Z

Gmcgath: Add some standard info

{{FormatInfo
|formattype=electronic
|subcat=Character Encodings
}}
ISO 8859-1, aka Latin-2, an encoding of the [[ISO 8859]] family for eastern European languages that use the Latin alphabet. Supported languages are Bosnian, Croatian, Czech, German, Hungarian, Polish, Serbian Latin, Slovak, Slovene, Upper Sorbian, and Lower Sorbian. German letters with umlauts and eszett (ß) are identically encoded in 8859-1 and 8859-2.

The Latin-2 encoding incorporates the [[ASCII]] characters in the first 128 code points (0-127), including the [[C0 controls]].

[http://www.kreativekorp.com/charset/encoding.php?file=iso-8859-2.kte Code table]

ISO 8859-2

2014-06-26T13:31:22Z

Gmcgath: Created page with "{{FormatInfo |formattype=electronic |subcat=Character Encodings }} ISO 8859-1, aka Latin-2, an encoding of the ISO 8859 family for eastern European languages that use the ..."

Markdown

2014-06-26T13:01:53Z

Gmcgath: Changed "official specification" to "official 2004 specification" for clarity

{{FormatInfo
|formattype=electronic
|subcat=Markup
|extensions={{ext|md}}, {{ext|markdown}}, {{ext|mdown}}, {{ext|markdn}}
}}

'''[[Markdown]]''' is a lightweight and human readable [[markup]] format for text formatting created by John Gruber and Aaron Swartz. It is similar to various forms of [[wiki markup]].

There is no formal specification for Markdown, and it has ambiguities that are handled inconsistently by different implementations. There is an initiative<ref>[http://www.w3.org/community/markdown/ Markdown Community Group]</ref> to write such a specification.

== References ==
<references/>

== Resources ==

* [http://daringfireball.net/projects/markdown/ Official 2004 specification and markdown to HTML conversion tool]
* [http://bywordapp.com/markdown/guide.html Byword MultiMarkdown Guide]
* [http://michelf.ca/projects/php-markdown/extra/ Markdown Extra] - includes support for tables, definition lists, footnotes, etc.

== Software ==
* [http://markdownpad.com/ Markdownpad: edit Markdown] (Windows; commercial, with free-of-charge version)
* [http://www.appsonthemove.com/trunk.htm Trunk Notes: note-taking app that uses Markdown] (iOS; commercial, paid)

TeX

2014-06-26T11:54:53Z

Gmcgath: New page

{{FormatInfo
|formattype=electronic
|subcat=Markup
|extensions={{ext|tex}}
|mimetypes={{mimetype|application/x-tex}}
|released=1978
}}

'''TeX''' is a markup language for typography, designed by Donald Knuth. It allows detailed control over layout and is particularly strong in its capabilities for mathematical formulas. It includes a full programming language. The "X" in the name is properly pronounced like the Greek letter chi (or the German ch), though it's often pronounced as "tek."

TeX has a powerful macro facility, and several macro extensions of TeK, such as [[LaTeX]], are widely used.

Library

2013-07-02T13:34:37Z

Gmcgath: Adding my own name to FTL to maintain consistency

Here's a collection of articles related to the File Format problem, this project, or education about the type of endeavor this sort of project entails.

* [http://unsustainableideas.wordpress.com/2012/10/22/open-letter-ms-obsolete-formats/ Open letter to Microsoft on specs for obsolete file formats] by Chris Rusbridge.
* [http://unsustainableideas.wordpress.com/2012/10/15/ppt-4-adventure-learning/ The PowerPoint 4.0 adventure: what did I learn?] by Chris Rusbridge.
* [http://blog.dshr.org/2012/10/cleaning-up-formats-through-time-mess.html Cleaning up the "Formats through Time" mess] by David Rosenthal.
* [http://infotrope.net/2012/11/30/importing-data-is-hard/ Importing data is hard] by Skud
* [http://journal.code4lib.org/articles/8029 The Format Registry Problem] by Gary McGath
* [http://filesthatlast.com/ Files that Last: Digital Preservation for Everygeek] by Gary McGath

JPEG

2012-12-19T02:30:33Z

Gmcgath: Clarify relationship of JPEG, JFIF, and SPIFF

{{FormatInfo
|formattype=electronic
|subcat=Graphics
|extensions={{ext|jpg}} {{ext|jpeg}} {{ext|jpe}} {{ext|jif}} {{ext|jfif}} {{ext|jfi}}
|mimetypes={{mimetype|image/jpeg}}
}}
'''JPEG''', named after the Joint Photographic Experts Group, which created the format, is a lossy compressed format well-suited to photographic images. Line drawings do better with non-lossy compressed bitmaps such as [[GIF]] and [[PNG]].

Properly speaking, JPEG refers to an encoding rather than a file format. [[JFIF]] (JPEG File Interchange Format) further specifies the particular subset of the JPEG standard intended to be used for standardized image files, skipping some of the advanced features of JPEG intended for specific application use. What are commonly called JPEG files are almost always JFIF files. The [[SPIFF]] file format was intended as a replacement for JFIF but never caught on widely.

== References ==
* [http://en.wikipedia.org/wiki/JPEG JPEG (Wikipedia)]
* [http://www.w3.org/Graphics/JPEG/itu-t81.pdf JPEG standard]
* [http://www.w3.org/Graphics/JPEG/jfif3.pdf JFIF standard]

MPEG-4

2012-11-30T16:43:01Z

Gmcgath: Patents and licensing

{{FormatInfo
|formattype=electronic
|subcat=Video
|extensions={{ext|mp4}}
|mimetypes={{mimetype|audio/mp4,video/mp4}}
}}

'''MP4''' is the popular name for the MPEG-4 standard, defined by the Moving Picture Experts Group, codified as ISO/IEC 14496. Saying just what the "MP4 file format" is, is a somewhat complicated question. Parts 12, 14, and 15 specifically refer to "file formats.".

Part 12 is known as ISO Base Media File Format. It is derived largely from Apple's QuickTime format and is described as a "general format for timed sequences of media data."

Part 14 is an extension of Part 12. The recommended extension for it is ".mp4", though ".m4a" is often used for audio-only files.

Part 15 is "MPEG-4 File Format for Advanced Video Coding." This specifies the use of Part 10 encoding with the Part 12 file format. the extension ".avc" is often used with it.

"MP4 audio players" don't always support the MPEG-4 file formats, sometimes supporting only the AAC audio encoding from MPEG-4, Part 3 (aka MPEG-4 Audio). These would more properly be called AAC files, with the extension ".aac".

MPEG-4 is encumbered by numerous patents, and (at least) commercial implementers have to secure licenses. Many of the licensing aspects are handled by a group called MPEG LA.

MPEG-4

2012-11-30T14:57:06Z

Gmcgath: Changed category to video

MPEG-4

2012-11-30T14:55:07Z

Gmcgath: A little more stuff

{{FormatInfo
|formattype=electronic
|subcat=Audio and Music
|extensions={{ext|mp4}}
|mimetypes={{mimetype|audio/mp4,video/mp4}}
}}

'''MP4''' is the popular name for the MPEG-4 standard, defined by the Moving Picture Experts Group, codified as ISO/IEC 14496. Saying just what the "MP4 file format" is, is a somewhat complicated question. Parts 12, 14, and 15 specifically refer to "file formats.".

Part 12 is known as ISO Base Media File Format. It is derived largely from Apple's QuickTime format and is described as a "general format for timed sequences of media data."

Part 14 is an extension of Part 12. The recommended extension for it is ".mp4", though ".m4a" is often used for audio-only files.

Part 15 is "MPEG-4 File Format for Advanced Video Coding." This specifies the use of Part 10 encoding with the Part 12 file format. the extension ".avc" is often used with it.

"MP4 audio players" don't always support the MPEG-4 file formats, sometimes supporting only the AAC audio encoding from MPEG-4, Part 3 (aka MPEG-4 Audio).

MPEG-4

2012-11-30T14:46:13Z

Gmcgath: New page

{{FormatInfo
|formattype=electronic
|subcat=Audio and Music
|extensions={{ext|mp4}}
|mimetypes={{mimetype|audio/mp4,video/mp4}}
}}

'''MP4''' is the popular name for the MPEG-4 standard, defined by the Moving Picture Experts Group, codified as ISO/IEC 14496. Saying just what the "MP4 file format" is, is a somewhat complicated question. Parts 12, 14, and 15 specifically refer to "file formats," but the term "MP4 format" is often taken to refer specifically to Part 10, which is a compression format and not a file format.

Part 12 is known as ISO Base Media File Format. It is derived largely from Apple's QuickTime format and is described as a "general format for timed sequences of media data."

Part 14 is an extension of Part 12. The recommended extension for it is ".mp4", though ".m4a" is often used for audio-only files.

Multics

2012-11-30T14:03:30Z

Gmcgath: New article

{|
|[[Software]]
| >
|[[Operating Systems]]
| >
|[[Multics]]
|}

'''Multics''' was a multitasking, multiuser operating system developed at MIT; its initial installation was made available primary to MIT users, including students. For many students it was their first interactive computer experience, allowing logging on through terminals instead of submitting a batch of cards. There were some other Multics installations, though it was never widely distributed. The Multics project began in 1964. It ran on GE (later Honeywell) hardware.

One of its most distinctive features was the definition of multiple levels of permitted access, called "protection rings," allowing fine-grained security control. Multics was an extremely ambitious design for its time and gave rise to ideas used in subsequent operating systems.

CP/M

2012-11-30T13:51:01Z

Gmcgath: New article

{|
|[[Software]]
| >
|[[Operating Systems]]
| >
|[[CP/M]]
|}

'''CP/M''' (Control Program for Microprocessors) was a popular operating system on early personal computers, sold by Digital Research, Inc. It ran on Intel 8080, Zilog Z-80, and compatible 8-bit microprocessors. Its user interface showed the influence of Digital Equipment Corporation's operating systems.

C1 controls

2012-11-30T13:44:37Z

Gmcgath: Maybe all the descriptions (I can figure out)

{{FormatInfo
|formattype=electronic
|subcat=Character Encodings
}}

The '''C1 controls''' are the control characters (code positions 128-159 decimal) which are defined by ISO/IEC 6429:1992 and are part of the [[ISO-8859]] and other encodings. They are not often used, and in otherwise equivalent Microsoft character sets (e.g., Windows 1252) they are replaced by printing characters. Because of their rarity of use, there is little agreement on their exact meaning and implementation.

{| class="wikitable"
! title="Hexadecimal code point" | Hex
! title="Decimal code point" | Dec
! title="Standard abbreviation" | Abbreviation
! title="Character name" | Name
! title="Description and uses" | Description and uses
|-
|80||128||PAD||Padding Character||Not part of ISO/IEC 6429.
|-
|81||129||HOP||High Octet Preset||Not part of ISO/IEC 6429.
|-
|82||130||BPH||Break Permitted Here||Follows a graphic character where a line break is permitted.
|-
|83||131||NBH||No Break Here||Follows a graphic character where a line break is not permitted.
|-
|84||132||IND||Index||Moves the active position one line down.
|-
|85||133||NEL||Next Line||Yet another line ending.
|-
|86||134||SSA||Start of Selected Area||
|-
|87||135||ESA||End of Selected Area||
|-
|88||136||HTS||Horizontal Tabulation Set||Sets a horizontal tab stop.
|-
|89||137||HTJ||Horizontal Tabulation with Justification||Sets a horizontal tab stop and indicates text should be justified out to the stop.
|-
|8A||138||VTS||Vertical Tabulation Set||Sets a vertical tab stop.
|-
|8B||139||PLD||Partial Line Down||Moves the active position down to a position suitable for subscripts, or undoes PLU.
|-
|8C||140||PLU||Partial Line Up||Moves the active position up to a position suitable for superscripts, or undoes PLD.
|-
|8D||141||RI||Reverse Index||Moves the active position one line up.
|-
|8E||142||SS2||Single-Shift 2||Indicates that the next code only should be interpreted in the G2 character set.
|-
|8F||143||SS3||Single-Shift 3||Indicates that the next code only should be interpreted in the G3 character set.
|-
|90||144||DCS||Device Control String||Introduces a device control sequence, which is terminated by ST (0X96).
|-
|91||145||PU1||Private Use 1||
|-
|92||146||PU2||Private Use 2||
|-
|93||147||STS||Set Transmit State||
|-
|94||148||PCH||Cancel Character||Backspace and cancel the previous character.
|-
|95||149||MW||Message Waiting||
|-
|96||150||SPA||Start of Protected Area||
|-
|97||151||EPA||End of Protected Area||
|-
|98||152||SOS||Start of String||Introduces a control string, which is terminated by ST (0X96).
|-
|99||153||SGCI||Single Graphic Character Introducer||Not part of ISO/IEC 6429.
|-
|9A||154||SCI||Single Character Introducer||Followed by a single printing character or format effector. Meaning uncertain.
|-
|9B||155||CSI||Control Sequence Introducer||
|-
|9C||156||ST||String Terminator||Marks the end of control sequences introduced by several C1 codes.
|-
|9D||157||OSC|Operating System Command||Introduces an operating system command, which is terminated by ST (0X96).
|-
|9E||158||PM||Privacy Message||Introduces a privacy message, which is terminated by ST (0X96).
|-
|9F||159||APC||Application Program Command||Introduces an application program command, which is terminated by ST (0X96).
|}

[[Category:File format details]]

C1 controls

2012-11-30T13:36:34Z

Gmcgath: More descriptions...

{{FormatInfo
|formattype=electronic
|subcat=Character Encodings
}}

The '''C1 controls''' are the control characters (code positions 128-159 decimal) which are defined by ISO/IEC 6429:1992 and are part of the [[ISO-8859]] encoding. They are also part of a number of other character sets derived from ASCII. They are not often used, and in otherwise equivalent Microsoft character sets (e.g., Windows 1252) they are replaced by printing characters.

NOTE: I'm cloning this from the C0 article and saving it occasionally. There will be gross errors till I'm done. Bear with me or dive in. --[[User:Gmcgath|Gmcgath]] ([[User talk:Gmcgath|talk]]) 12:34, 30 November 2012 (UTC)

{| class="wikitable"
! title="Hexadecimal code point" | Hex
! title="Decimal code point" | Dec
! title="Standard abbreviation" | Abbreviation
! title="Character name" | Name
! title="Description and uses" | Description and uses
|-
|80||128||PAD||Padding Character||Not part of ISO/IEC 6429.
|-
|81||129||HOP||High Octet Preset||Not part of ISO/IEC 6429.
|-
|82||130||BPH||Break Permitted Here||Follows a graphic character where a line break is permitted.
|-
|83||131||NBH||No Break Here||Follows a graphic character where a line break is not permitted.
|-
|84||132||IND||Index||Moves the active position one line down.
|-
|85||133||NEL||Next Line||Yet another line ending.
|-
|86||134||SSA||Start of Selected Area||
|-
|87||135||ESA||End of Selected Area||
|-
|88||136||HTS||Horizontal Tabulation Set||Sets a horizontal tab stop.
|-
|89||137||HTJ||Horizontal Tabulation with Justification||Sets a horizontal tab stop and indicates text should be justified out to the stop.
|-
|8A||138||VTS||Vertical Tabulation Set||Sets a vertical tab stop.
|-
|8B||139||PLD||Partial Line Down||Moves the active position down to a position suitable for subscripts, or undoes PLU.
|-
|8C||140||PLU||Partial Line Up||Moves the active position up to a position suitable for superscripts, or undoes PLD.
|-
|8D||141||RI||Reverse Index||Moves the active position one line up.
|-
|8E||142||SS2||Single-Shift 2||Indicates that the next code only should be interpreted in the G2 character set.
|-
|8F||143||SS3||Single-Shift 3||Indicates that the next code only should be interpreted in the G3 character set.
|-
|90||144||DCS||Device Control String||Introduces a device control sequence, which is terminated by ST (0X96).
|-
|91||145||PU1||Private Use 1||
|-
|92||146||PU2||Private Use 2||
|-
|93||147||STS||Set Transmit State||The third of the device-control codes; this one (CTRL-S, also known as XOFF) is often used to pause processes, devices, or output streams, with CTRL-Q (XON) resuming them (though in some cases, any keypress causes output to resume).
|-
|94||148||PCH||Cancel Character||The fourth device-control code; not used as much as DC1 or DC3. In DEC TOPS-20 mainframes, usually resulted in output of system status to terminal.
|-
|95||149||MW||Message Waiting||In transmission protocols, indicates a failure requiring a re-send, or a negative response to a query of whether the process is ready to proceed.
|-
|96||150||SPA||Start of Protected Area||Signals that a correction may now be received in synchronous transmission protocols.
|-
|97||151||EPA||End of Protected Area||Marks the end of a block of data divided into blocks for transmission.
|-
|98||152||SOS||Start of String||Cancels an operation and signals that previously-sent data can be disregarded.
|-
|99||153||SGCI||Single Graphic Character Introducer||Marks the end of a physical medium such as a data-storage tape.
|-
|9A||154||SCI||Single Character Introducer||Used to mark the spot where garbled, missing, or incomplete characters were received due to transmission errors, or various other uses involving place-holder characters. This character (Ctrl-Z) is also used by MS/PC-DOS to mark the end of a file or input stream, calling it EOF (although CTRL-D, EOT, would have been more standards-compliant and is used by Unix-style OSs for this purpose; however, some DEC operating systems used the CTRL-Z convention and this is what was followed by PC-DOS).
|-
|9B||155||CSI||Control Sequence Introducer||Mapped onto the ESC key on keyboards, this usually signals a user attempting to exit a menu or mode. It is also commonly used in printer and terminal control protocols to signal the beginning of a special "escape sequence" where immediately-following characters are interpreted as commands.
|-
|9C||156||ST||Operating System Command||Introduces an operating system command, which is terminated by ST (0X96).
|-
|9D||157||OSC||Group Separator||The second of four separator characters, subordinate to FS, but higher-level than RS and US.
|-
|9E||158||PM||Privacy Message||Introduces a privacy message, which is terminated by ST (0X96).
|-
|9F||159||APC||Application Program Command||Introduces an application program command, which is terminated by ST (0X96).
|}

[[Category:File format details]]

C1 controls

2012-11-30T13:32:06Z

Gmcgath: Working on descriptions

{{FormatInfo
|formattype=electronic
|subcat=Character Encodings
}}

The '''C1 controls''' are the control characters (code positions 128-159 decimal) which are defined by ISO/IEC 6429:1992 and are part of the [[ISO-8859]] encoding. They are also part of a number of other character sets derived from ASCII. They are not often used, and in otherwise equivalent Microsoft character sets (e.g., Windows 1252) they are replaced by printing characters.

NOTE: I'm cloning this from the C0 article and saving it occasionally. There will be gross errors till I'm done. Bear with me or dive in. --[[User:Gmcgath|Gmcgath]] ([[User talk:Gmcgath|talk]]) 12:34, 30 November 2012 (UTC)

{| class="wikitable"
! title="Hexadecimal code point" | Hex
! title="Decimal code point" | Dec
! title="Standard abbreviation" | Abbreviation
! title="Character name" | Name
! title="Description and uses" | Description and uses
|-
|80||128||PAD||Padding Character||Not part of ISO/IEC 6429.
|-
|81||129||HOP||High Octet Preset||Not part of ISO/IEC 6429.
|-
|82||130||BPH||Break Permitted Here||Follows a graphic character where a line break is permitted.
|-
|83||131||NBH||No Break Here||Follows a graphic character where a line break is not permitted.
|-
|84||132||IND||Index||Moves the active position one line down.
|-
|85||133||NEL||Next Line||Used in transmission protocols to request acknowledgement from the other end to make sure connection is still active. In DEC TOPS-20 mainframes, usually resulted in currently-active application outputing status information to terminal.
|-
|86||134||SSA||Start of Selected Area||Sent as response to ENQ message, or used to positively acknowledge receipt of data or messages (as opposed to NAK).
|-
|87||135||ESA||End of Selected Area||On some systems, this causes a bell, buzzer, or beep to sound, or flashes inverse video to alert a system operator. The Apple II had "BELL" on the front side of the "G" key to remind users that Ctrl-G caused this sound effect.
|-
|88||136||HTS||Horizontal Tabulation Set||Moves back one space. Usually deletes last character (e.g., from input string), but on some old terminals it just moved backward without deleting and allowed "overstrike" effects overlaying multiple characters.
|-
|89||137||HTJ||Horizontal Tabulation with Justification||The typewriter "tab key", usually moving to the next tab stop as defined in the particular software being used.
|-
|8A||138||VTS||Vertical Tabulation Set||Move down one line. In Unix-style operating systems, it also moves to the beginning of the next line so that it can be used as a line break (newline) character, while in some other systems and terminals it just moves down without moving to the left, requiring the "CR LF" sequence to break a line.
|-
|8B||139||PLD||Partial Line Down||Moves to vertical tab stops; not used nearly as often as the more-common horizontal tab.
|-
|8C||140||PLU||Partial Line Up||Causes page to eject in printers, and may clear the screen in some terminal emulators. Sometimes used as a logical division of sections of a document.
|-
|8D||141||RI||Reverse Index||Moves the active position one line up.
|-
|8E||142||SS2||Single-Shift 2||Switch to alternate character set (reversed by SI). Used in various systems and terminals to set different characters (e.g., APL or Cyrillic), or change the color or font.
|-
|8F||143||SS3||Single-Shift 3||Return to normal character set (reverses operation of SO).
|-
|90||144||DCS||Device Control String||Signals the start of a sequence of raw data as opposed to normal printable or control characters.
|-
|91||145||PU1||Private Use 1||One of four device-control codes intended to be system-specific. This one (CTRL-Q, also known as XON) is often used to resume operations of a process, device, or output stream that has been paused with CTRL-S (XOFF).
|-
|92||146||PU2||Private Use 2||Another device-control code; not used as much as DC1 and DC3.
|-
|93||147||STS||Set Transmit State||The third of the device-control codes; this one (CTRL-S, also known as XOFF) is often used to pause processes, devices, or output streams, with CTRL-Q (XON) resuming them (though in some cases, any keypress causes output to resume).
|-
|94||148||PCH||Cancel Character||The fourth device-control code; not used as much as DC1 or DC3. In DEC TOPS-20 mainframes, usually resulted in output of system status to terminal.
|-
|95||149||MW||Message Waiting||In transmission protocols, indicates a failure requiring a re-send, or a negative response to a query of whether the process is ready to proceed.
|-
|96||150||SPA||Start of Protected Area||Signals that a correction may now be received in synchronous transmission protocols.
|-
|97||151||EPA||End of Protected Area||Marks the end of a block of data divided into blocks for transmission.
|-
|98||152||SOS||Start of String||Cancels an operation and signals that previously-sent data can be disregarded.
|-
|99||153||SGCI||Single Graphic Character Introducer||Marks the end of a physical medium such as a data-storage tape.
|-
|9A||154||SCI||Single Character Introducer||Used to mark the spot where garbled, missing, or incomplete characters were received due to transmission errors, or various other uses involving place-holder characters. This character (Ctrl-Z) is also used by MS/PC-DOS to mark the end of a file or input stream, calling it EOF (although CTRL-D, EOT, would have been more standards-compliant and is used by Unix-style OSs for this purpose; however, some DEC operating systems used the CTRL-Z convention and this is what was followed by PC-DOS).
|-
|9B||155||CSI||Control Sequence Introducer||Mapped onto the ESC key on keyboards, this usually signals a user attempting to exit a menu or mode. It is also commonly used in printer and terminal control protocols to signal the beginning of a special "escape sequence" where immediately-following characters are interpreted as commands.
|-
|9C||156||ST||Operating System Command||Introduces an operating system command, which is terminated by ST (0X96).
|-
|9D||157||OSC||Group Separator||The second of four separator characters, subordinate to FS, but higher-level than RS and US.
|-
|9E||158||PM||Privacy Message||Introduces a privacy message, which is terminated by ST (0X96).
|-
|9F||159||APC||Application Program Command||Introduces an application program command, which is terminated by ST (0X96).
|}

[[Category:File format details]]

C1 controls

2012-11-30T13:00:40Z

Gmcgath: Real names this time. Still need to fix descriptions.

{{FormatInfo
|formattype=electronic
|subcat=Character Encodings
}}

The '''C1 controls''' are the control characters (code positions 128-159 decimal) which are part of the [[ISO-8859]] standard. They are also part of a number of other character sets derived from ASCII. They are not often used, and in otherwise equivalent Microsoft character sets (e.g., Windows 1252) they are replaced by printing characters.

NOTE: I'm cloning this from the C0 article and saving it occasionally. There will be gross errors till I'm done. Bear with me or dive in. --[[User:Gmcgath|Gmcgath]] ([[User talk:Gmcgath|talk]]) 12:34, 30 November 2012 (UTC)

{| class="wikitable"
! title="Hexadecimal code point" | Hex
! title="Decimal code point" | Dec
! title="Standard abbreviation" | Abbreviation
! title="Character name" | Name
! title="Description and uses" | Description and uses
|-
|80||128||PAD||Padding Character||Marks unused space or padding (e.g., to intentionally slow down terminals or to leave space for added data in memory or storage media). Used in C-based programming languages to mark end of string.
|-
|81||129||HOP||High Octet Preset||Marks the beginning of a header in a message or data structure.
|-
|82||130||BPH||Break Permitted Here||Marks the beginning of the body text of a message, and/or the end of the header.
|-
|83||131||NBH||No Break Here||Marks the end of the body text. Also used as "break character" (Control-C) to terminate a program or process.
|-
|84||132||IND||Index||In Unix-style operating systems, signals end-of-file and is used to log out of a terminal. On Apple II, this character signalled that what followed was a DOS command when it was "printed" to standard output.
|-
|85||133||NEL||Next Line||Used in transmission protocols to request acknowledgement from the other end to make sure connection is still active. In DEC TOPS-20 mainframes, usually resulted in currently-active application outputing status information to terminal.
|-
|86||134||SSA||Start of Selected Area||Sent as response to ENQ message, or used to positively acknowledge receipt of data or messages (as opposed to NAK).
|-
|87||135||ESA||End of Selected Area||On some systems, this causes a bell, buzzer, or beep to sound, or flashes inverse video to alert a system operator. The Apple II had "BELL" on the front side of the "G" key to remind users that Ctrl-G caused this sound effect.
|-
|88||136||HTS||Horizontal Tabulation Set||Moves back one space. Usually deletes last character (e.g., from input string), but on some old terminals it just moved backward without deleting and allowed "overstrike" effects overlaying multiple characters.
|-
|89||137||HTJ||Horizontal Tabulation with Justification||The typewriter "tab key", usually moving to the next tab stop as defined in the particular software being used.
|-
|8A||138||VTS||Vertical Tabulation Set||Move down one line. In Unix-style operating systems, it also moves to the beginning of the next line so that it can be used as a line break (newline) character, while in some other systems and terminals it just moves down without moving to the left, requiring the "CR LF" sequence to break a line.
|-
|8B||139||PLD||Partial Line Down||Moves to vertical tab stops; not used nearly as often as the more-common horizontal tab.
|-
|8C||140||PLU||Partial Line Up||Causes page to eject in printers, and may clear the screen in some terminal emulators. Sometimes used as a logical division of sections of a document.
|-
|8D||141||RI||Reverse Index||Moves to the beginning of the line. In some systems (e.g., Apple II, Commodore 64, and TRS-80, and early Macintosh systems before its OS switched to a Unix-based system), also moves to the next line so that it can be used as a line break character, while in other systems it stays on the same line so that it must be accompanied by a LF character to break a line (but on some printing terminals CR with no LF was used for overstrike effects including underlining by printing underscores). Thus the three different line-break conventions (LF, CR, and CR+LF) arose, which bedevil users of text files to this day. As an input character, CR is generally mapped onto the Enter key, signaling the completion of input.
|-
|8E||142||SS2||Single-Shift 2||Switch to alternate character set (reversed by SI). Used in various systems and terminals to set different characters (e.g., APL or Cyrillic), or change the color or font.
|-
|8F||143||SS3||Single-Shift 3||Return to normal character set (reverses operation of SO).
|-
|90||144||DCS||Device Control String||Signals the start of a sequence of raw data as opposed to normal printable or control characters.
|-
|91||145||PU1||Private Use 1||One of four device-control codes intended to be system-specific. This one (CTRL-Q, also known as XON) is often used to resume operations of a process, device, or output stream that has been paused with CTRL-S (XOFF).
|-
|92||146||PU2||Private Use 2||Another device-control code; not used as much as DC1 and DC3.
|-
|93||147||STS||Set Transmit State||The third of the device-control codes; this one (CTRL-S, also known as XOFF) is often used to pause processes, devices, or output streams, with CTRL-Q (XON) resuming them (though in some cases, any keypress causes output to resume).
|-
|94||148||PCH||Cancel Character||The fourth device-control code; not used as much as DC1 or DC3. In DEC TOPS-20 mainframes, usually resulted in output of system status to terminal.
|-
|95||149||MW||Message Waiting||In transmission protocols, indicates a failure requiring a re-send, or a negative response to a query of whether the process is ready to proceed.
|-
|96||150||SPA||Start of Protected Area||Signals that a correction may now be received in synchronous transmission protocols.
|-
|97||151||EPA||End of Protected Area||Marks the end of a block of data divided into blocks for transmission.
|-
|98||152||SOS||Start of String||Cancels an operation and signals that previously-sent data can be disregarded.
|-
|99||153||SGCI||Single Graphic Character Introducer||Marks the end of a physical medium such as a data-storage tape.
|-
|9A||154||SCI||Single Character Introducer||Used to mark the spot where garbled, missing, or incomplete characters were received due to transmission errors, or various other uses involving place-holder characters. This character (Ctrl-Z) is also used by MS/PC-DOS to mark the end of a file or input stream, calling it EOF (although CTRL-D, EOT, would have been more standards-compliant and is used by Unix-style OSs for this purpose; however, some DEC operating systems used the CTRL-Z convention and this is what was followed by PC-DOS).
|-
|9B||155||CSI||Control Sequence Introducer||Mapped onto the ESC key on keyboards, this usually signals a user attempting to exit a menu or mode. It is also commonly used in printer and terminal control protocols to signal the beginning of a special "escape sequence" where immediately-following characters are interpreted as commands.
|-
|9C||156||ST||Operating System Command||One of four separator characters intended to delimit structured data. FS is the highest-level separator, intended to separate structures which are in turn internally delimited with GS, RS, and US (in descending order). Also used as a "quit and dump core" signal in Unix shells.
|-
|9D||157||OSC||Group Separator||The second of four separator characters, subordinate to FS, but higher-level than RS and US.
|-
|9E||158||PM||Privacy Message||The third of four separator characters, subordinate to FS and GS, but higher-level than US.
|-
|9F||159||APC||Application Program Command||The lowest-level of the separator characters, used to divide strings of ASCII characters which are the base elements of a data structure. A sequence of such US-delimited strings can in turn be used as a higher-level data element separated by other such elements by the RS character, and this structure in turn can be delimited from other such elements by GS, and finally if a fourth level is needed the FS character separates those elements.
|}

[[Category:File format details]]

C1 controls

2012-11-30T12:49:48Z

Gmcgath: Continuing work on basic content

{{FormatInfo
|formattype=electronic
|subcat=Character Encodings
}}

The '''C1 controls''' are the control characters (code positions 128-159 decimal) which are part of the [[ISO-8859]] standard. They are also part of a number of other character sets derived from ASCII. They are not often used, and in otherwise equivalent Microsoft character sets (e.g., Windows 1252) they are replaced by printing characters.

NOTE: I'm cloning this from the C0 article and saving it occasionally. There will be gross errors till I'm done. Bear with me or dive in. --[[User:Gmcgath|Gmcgath]] ([[User talk:Gmcgath|talk]]) 12:34, 30 November 2012 (UTC)

{| class="wikitable"
! title="Hexadecimal code point" | Hex
! title="Decimal code point" | Dec
! title="Standard abbreviation" | Abbreviation
! title="Character name" | Name
! title="Description and uses" | Description and uses
|-
|80||128||PAD||Null character||Marks unused space or padding (e.g., to intentionally slow down terminals or to leave space for added data in memory or storage media). Used in C-based programming languages to mark end of string.
|-
|81||129||HOP||Start of Heading||Marks the beginning of a header in a message or data structure.
|-
|82||130||BPH||Start of Text||Marks the beginning of the body text of a message, and/or the end of the header.
|-
|83||131||NBH||End of Text||Marks the end of the body text. Also used as "break character" (Control-C) to terminate a program or process.
|-
|84||132||IND||End of Transmission||In Unix-style operating systems, signals end-of-file and is used to log out of a terminal. On Apple II, this character signalled that what followed was a DOS command when it was "printed" to standard output.
|-
|85||133||NEL||Enquiry||Used in transmission protocols to request acknowledgement from the other end to make sure connection is still active. In DEC TOPS-20 mainframes, usually resulted in currently-active application outputing status information to terminal.
|-
|86||134||SSA||Acknowledge||Sent as response to ENQ message, or used to positively acknowledge receipt of data or messages (as opposed to NAK).
|-
|87||135||ESA||Bell||On some systems, this causes a bell, buzzer, or beep to sound, or flashes inverse video to alert a system operator. The Apple II had "BELL" on the front side of the "G" key to remind users that Ctrl-G caused this sound effect.
|-
|88||136||HTS||Backspace||Moves back one space. Usually deletes last character (e.g., from input string), but on some old terminals it just moved backward without deleting and allowed "overstrike" effects overlaying multiple characters.
|-
|89||137||HTJ||Horizontal Tab||The typewriter "tab key", usually moving to the next tab stop as defined in the particular software being used.
|-
|8A||138||VTS||Line Feed||Move down one line. In Unix-style operating systems, it also moves to the beginning of the next line so that it can be used as a line break (newline) character, while in some other systems and terminals it just moves down without moving to the left, requiring the "CR LF" sequence to break a line.
|-
|8B||139||PLD||Vertical Tab||Moves to vertical tab stops; not used nearly as often as the more-common horizontal tab.
|-
|8C||140||PLU||Form Feed||Causes page to eject in printers, and may clear the screen in some terminal emulators. Sometimes used as a logical division of sections of a document.
|-
|8D||141||RI||Carriage Return||Moves to the beginning of the line. In some systems (e.g., Apple II, Commodore 64, and TRS-80, and early Macintosh systems before its OS switched to a Unix-based system), also moves to the next line so that it can be used as a line break character, while in other systems it stays on the same line so that it must be accompanied by a LF character to break a line (but on some printing terminals CR with no LF was used for overstrike effects including underlining by printing underscores). Thus the three different line-break conventions (LF, CR, and CR+LF) arose, which bedevil users of text files to this day. As an input character, CR is generally mapped onto the Enter key, signaling the completion of input.
|-
|8E||142||SS2||Shift Out||Switch to alternate character set (reversed by SI). Used in various systems and terminals to set different characters (e.g., APL or Cyrillic), or change the color or font.
|-
|8F||143||SS3||Shift In||Return to normal character set (reverses operation of SO).
|-
|90||144||DCS||Data Link Escape||Signals the start of a sequence of raw data as opposed to normal printable or control characters.
|-
|91||145||PU1||Device Control 1||One of four device-control codes intended to be system-specific. This one (CTRL-Q, also known as XON) is often used to resume operations of a process, device, or output stream that has been paused with CTRL-S (XOFF).
|-
|92||146||PU2||Device Control 2||Another device-control code; not used as much as DC1 and DC3.
|-
|93||147||STS||Device Control 3||The third of the device-control codes; this one (CTRL-S, also known as XOFF) is often used to pause processes, devices, or output streams, with CTRL-Q (XON) resuming them (though in some cases, any keypress causes output to resume).
|-
|94||148||PCH||Device Control 4||The fourth device-control code; not used as much as DC1 or DC3. In DEC TOPS-20 mainframes, usually resulted in output of system status to terminal.
|-
|95||149||MW||Negative Acknowledge||In transmission protocols, indicates a failure requiring a re-send, or a negative response to a query of whether the process is ready to proceed.
|-
|96||150||SPA||Synchronous Idle||Signals that a correction may now be received in synchronous transmission protocols.
|-
|97||151||EPA||End of Transmission Block||Marks the end of a block of data divided into blocks for transmission.
|-
|98||152||SOS||Cancel||Cancels an operation and signals that previously-sent data can be disregarded.
|-
|99||153||SGCI||End of Medium||Marks the end of a physical medium such as a data-storage tape.
|-
|9A||154||SCI||Substitute Character||Used to mark the spot where garbled, missing, or incomplete characters were received due to transmission errors, or various other uses involving place-holder characters. This character (Ctrl-Z) is also used by MS/PC-DOS to mark the end of a file or input stream, calling it EOF (although CTRL-D, EOT, would have been more standards-compliant and is used by Unix-style OSs for this purpose; however, some DEC operating systems used the CTRL-Z convention and this is what was followed by PC-DOS).
|-
|9B||155||CSI||Escape||Mapped onto the ESC key on keyboards, this usually signals a user attempting to exit a menu or mode. It is also commonly used in printer and terminal control protocols to signal the beginning of a special "escape sequence" where immediately-following characters are interpreted as commands.
|-
|9C||156||ST||File Separator||One of four separator characters intended to delimit structured data. FS is the highest-level separator, intended to separate structures which are in turn internally delimited with GS, RS, and US (in descending order). Also used as a "quit and dump core" signal in Unix shells.
|-
|9D||157||OSC||Group Separator||The second of four separator characters, subordinate to FS, but higher-level than RS and US.
|-
|9E||158||PM||Record Separator||The third of four separator characters, subordinate to FS and GS, but higher-level than US.
|-
|9F||159||APC||Unit Separator||The lowest-level of the separator characters, used to divide strings of ASCII characters which are the base elements of a data structure. A sequence of such US-delimited strings can in turn be used as a higher-level data element separated by other such elements by the RS character, and this structure in turn can be delimited from other such elements by GS, and finally if a fourth level is needed the FS character separates those elements.
|}

[[Category:File format details]]

C1 controls

2012-11-30T12:34:32Z

Gmcgath: New version cloned from C0, still working on it.

{{FormatInfo
|formattype=electronic
|subcat=Character Encodings
}}

The '''C1 controls''' are the control characters (code positions 128-159 decimal) which are part of the [[ISO-8859]] standard. They are also part of a number of other character sets derived from ASCII. They are not often used, and in otherwise equivalent Microsoft character sets (e.g., Windows 1252) they are replaced by printing characters.

NOTE: I'm cloning this from the C0 article and saving it occasionally. There will be gross errors till I'm done. Bear with me or dive in. --[[User:Gmcgath|Gmcgath]] ([[User talk:Gmcgath|talk]]) 12:34, 30 November 2012 (UTC)

{| class="wikitable"
! title="Hexadecimal code point" | Hex
! title="Decimal code point" | Dec
! title="Codes used to represent character" | Codes
! title="Standard Acronym" | Acronym
! title="Character name" | Name
! title="Description and uses" | Description and uses
|-
|80||0||^@, \0||NUL||Null character||Marks unused space or padding (e.g., to intentionally slow down terminals or to leave space for added data in memory or storage media). Used in C-based programming languages to mark end of string.
|-
|81||1||^A||SOH||Start of Heading||Marks the beginning of a header in a message or data structure.
|-
|82||2||^B||STX||Start of Text||Marks the beginning of the body text of a message, and/or the end of the header.
|-
|83||3||^C||ETX||End of Text||Marks the end of the body text. Also used as "break character" (Control-C) to terminate a program or process.
|-
|84||4||^D||EOT||End of Transmission||In Unix-style operating systems, signals end-of-file and is used to log out of a terminal. On Apple II, this character signalled that what followed was a DOS command when it was "printed" to standard output.
|-
|85||5||^E||ENQ||Enquiry||Used in transmission protocols to request acknowledgement from the other end to make sure connection is still active. In DEC TOPS-20 mainframes, usually resulted in currently-active application outputing status information to terminal.
|-
|86||6||^F||ACK||Acknowledge||Sent as response to ENQ message, or used to positively acknowledge receipt of data or messages (as opposed to NAK).
|-
|87||7||^G, \a||BEL||Bell||On some systems, this causes a bell, buzzer, or beep to sound, or flashes inverse video to alert a system operator. The Apple II had "BELL" on the front side of the "G" key to remind users that Ctrl-G caused this sound effect.
|-
|88||8||^H, \b||BS||Backspace||Moves back one space. Usually deletes last character (e.g., from input string), but on some old terminals it just moved backward without deleting and allowed "overstrike" effects overlaying multiple characters.
|-
|89||9||^I, \t||HT||Horizontal Tab||The typewriter "tab key", usually moving to the next tab stop as defined in the particular software being used.
|-
|8A||10||^J, \n||LF||Line Feed||Move down one line. In Unix-style operating systems, it also moves to the beginning of the next line so that it can be used as a line break (newline) character, while in some other systems and terminals it just moves down without moving to the left, requiring the "CR LF" sequence to break a line.
|-
|8B||11||^K, \v||VT||Vertical Tab||Moves to vertical tab stops; not used nearly as often as the more-common horizontal tab.
|-
|8C||12||^L, \f||FF||Form Feed||Causes page to eject in printers, and may clear the screen in some terminal emulators. Sometimes used as a logical division of sections of a document.
|-
|8D||13||^M, \r||CR||Carriage Return||Moves to the beginning of the line. In some systems (e.g., Apple II, Commodore 64, and TRS-80, and early Macintosh systems before its OS switched to a Unix-based system), also moves to the next line so that it can be used as a line break character, while in other systems it stays on the same line so that it must be accompanied by a LF character to break a line (but on some printing terminals CR with no LF was used for overstrike effects including underlining by printing underscores). Thus the three different line-break conventions (LF, CR, and CR+LF) arose, which bedevil users of text files to this day. As an input character, CR is generally mapped onto the Enter key, signaling the completion of input.
|-
|8E||14||^N||SO||Shift Out||Switch to alternate character set (reversed by SI). Used in various systems and terminals to set different characters (e.g., APL or Cyrillic), or change the color or font.
|-
|8F||15||^O||SI||Shift In||Return to normal character set (reverses operation of SO).
|-
|90||16||^P||DLE||Data Link Escape||Signals the start of a sequence of raw data as opposed to normal printable or control characters.
|-
|91||17||^Q||DC1||Device Control 1||One of four device-control codes intended to be system-specific. This one (CTRL-Q, also known as XON) is often used to resume operations of a process, device, or output stream that has been paused with CTRL-S (XOFF).
|-
|92||18||^R||DC2||Device Control 2||Another device-control code; not used as much as DC1 and DC3.
|-
|93||19||^S||DC3||Device Control 3||The third of the device-control codes; this one (CTRL-S, also known as XOFF) is often used to pause processes, devices, or output streams, with CTRL-Q (XON) resuming them (though in some cases, any keypress causes output to resume).
|-
|94||20||^T||DC4||Device Control 4||The fourth device-control code; not used as much as DC1 or DC3. In DEC TOPS-20 mainframes, usually resulted in output of system status to terminal.
|-
|95||21||^U||NAK||Negative Acknowledge||In transmission protocols, indicates a failure requiring a re-send, or a negative response to a query of whether the process is ready to proceed.
|-
|96||22||^V||SYN||Synchronous Idle||Signals that a correction may now be received in synchronous transmission protocols.
|-
|97||23||^W||ETB||End of Transmission Block||Marks the end of a block of data divided into blocks for transmission.
|-
|98||24||^X||CAN||Cancel||Cancels an operation and signals that previously-sent data can be disregarded.
|-
|99||25||^Y||EM||End of Medium||Marks the end of a physical medium such as a data-storage tape.
|-
|9A||26||^Z||SUB||Substitute Character||Used to mark the spot where garbled, missing, or incomplete characters were received due to transmission errors, or various other uses involving place-holder characters. This character (Ctrl-Z) is also used by MS/PC-DOS to mark the end of a file or input stream, calling it EOF (although CTRL-D, EOT, would have been more standards-compliant and is used by Unix-style OSs for this purpose; however, some DEC operating systems used the CTRL-Z convention and this is what was followed by PC-DOS).
|-
|9B||27||^[||ESC||Escape||Mapped onto the ESC key on keyboards, this usually signals a user attempting to exit a menu or mode. It is also commonly used in printer and terminal control protocols to signal the beginning of a special "escape sequence" where immediately-following characters are interpreted as commands.
|-
|9C||28||^\||FS||File Separator||One of four separator characters intended to delimit structured data. FS is the highest-level separator, intended to separate structures which are in turn internally delimited with GS, RS, and US (in descending order). Also used as a "quit and dump core" signal in Unix shells.
|-
|9D||29||^]||GS||Group Separator||The second of four separator characters, subordinate to FS, but higher-level than RS and US.
|-
|9E||30||^^||RS||Record Separator||The third of four separator characters, subordinate to FS and GS, but higher-level than US.
|-
|9F||31||^_||US||Unit Separator||The lowest-level of the separator characters, used to divide strings of ASCII characters which are the base elements of a data structure. A sequence of such US-delimited strings can in turn be used as a higher-level data element separated by other such elements by the RS character, and this structure in turn can be delimited from other such elements by GS, and finally if a fourth level is needed the FS character separates those elements.
|}

[[Category:File format details]]

JIS X 0201

2012-11-11T03:53:13Z

Gmcgath: New

{{FormatInfo
|formattype=electronic
|subcat=Character Encodings
}}
JIS X 0201 is an 8-bit encoding for Japanese text. Character codes 0-127 are identical with ISO 646-JP. Character codes 128-255 provide encoding for Katakana characters and some Japanese punctuation. This allows phonetic rendering of Japanese but omits the Hiragana and Kanji scripts, which are used in normal writing.

JIS

2012-11-11T03:46:20Z

Gmcgath:

{{FormatInfo
|formattype=electronic
|subcat=Character Encodings
}}
'''JIS''' (Japanese Industrial Standard) is a name given to various character encoding standards based on JSA (Japanese Standards Association) standards. JIS defines the following character sets:

* JIS X 0201, encoding Roman letters (ROMAJI) and Katakana.
* JIS X 0208, supporting Kanji and Hiragana as well. There are 1978, 1983, 1990, and 1997 versions of the standard.
* JIS X 0212, which includes rarely used Kanji not included in the JIS X 0208 character set.
* JIS X 0213. There are 2000 and 2004 versions of the standard.

The following encodings are commonly used with JIS character sets:

* ISO-2022-JP
* ISO-2022-JP-1
* ISO-2022-JP-2
* EUC (UJIS)
* Shift-JIS

==Resources==
* One of the best resources on JIS charsets is the book "CJKV Information Processing" by Ken Lunde, O'Reilly 1999, ISBN 1-56592-224-7

ISO 8859-1

2012-11-10T23:36:12Z

Gmcgath: Code table link

{{FormatInfo
|formattype=electronic
|subcat=Character Encodings
}}
ISO 8859-1, aka Latin-1, is the most widely used encoding of the [[ISO 8859]] family for English-language text. [[Windows 1252]] encoding includes all the printing characters of ISO 8859-1 and includes additional printing characters by replacing some control characters of the C1 (128-159) range.

ISO 8859-1 was updated to [[ISO 8859-15]], called Latin-9, replacing some of the less used characters and adding the Euro (€) sign. However, those who have decided to update from Latin-1 have generally gone straight to Unicode, so Latin-9 has not become a widespread replacement. ISO [[8859-9]] or Latin-5, designed for use with Turkish scripts, also differs from Latin-1 in only a few code points.

[http://www.kreativekorp.com/charset/encoding.php?file=iso-8859-1.kte Code table]

ISO 8859-1

2012-11-10T23:34:57Z

Gmcgath: New page

File Information Tool Set

2012-11-10T23:22:23Z

Gmcgath: New page

FITS, the File Information Tool Set, is software created at the Harvard University Library to identify, validate, and extract technical metadata for various file formats. It wraps several third-party open source tools, normalizes and consolidates their output, and reports any errors. The tools which it incorporates include JHOVE, Exiftool, National Library of New Zealand Metadata Extractor, DROID, FFIdent, and Windows File Utility.

NOTE: This article should be titled "FITS" to follow the conventions of the wiki, but there is already an article on a data format by that name. Disambiguation, someone?

[[https://code.google.com/p/fits/ FITS home page on Google Code]]

Talk:Flexible Image Transport System

2012-11-10T23:16:31Z

Gmcgath: Disambiguation issue

We need a disambiguation here. This article is on FITS, the astronomical data format. There's at least one link to it which is clearly intended to refer to [[https://code.google.com/p/fits/ FITS]], the file validation and metadata extraction software developed at Harvard. Both are of interest to this project.

Should there be a Wikipedia-style disambiguation page? --[[User:Gmcgath|Gmcgath]] ([[User talk:Gmcgath|talk]]) 23:16, 10 November 2012 (UTC)

UTF-1

2012-11-10T16:35:20Z

Gmcgath: New page

UTF-1 is an obsolete encoding for [[Unicode]]. It was removed from ISO/IEC 10646-1:1993 by Amendment 4.

[[Category:Character Encodings]]

Category:Software

2012-11-10T16:27:19Z

Gmcgath: "Create"

Software

JHOVE

2012-11-10T16:26:36Z

Gmcgath: Category software

JHOVE is a tool to identify, validate and characterize file formats. It was originally a collaborative project by JSTOR and Harvard University Library. It is currently an active open source project no longer backed by those institutions.
JHOVE is licensed under the GNU Lesser General Public License.

== Related ==
{{related_file_format_identification}}

== Resources ==
* [http://jhove.sourceforge.net/ Offical Homepage]
* [http://www.garymcgath.com/jhovenote.html JHOVE usage notes]

[[Category:File Format Identification]]
[[Category:Software]]

JHOVE

2012-11-10T16:25:27Z

Gmcgath: Note on status, link to (my own) usage notes

UTF-32

2012-11-10T06:21:11Z

Gmcgath: Removed "Text Encoding"; redundant with "Character Encodings"

{{FormatInfo
|formattype=electronic
|subcat=Character Encodings
}}
'''[[UCS]] Transformation Format—32-bit''' (UTF-32) is a [[Unicode]] character encoding. There is a one-to-one mapping of Unicode code points to 32-bit values, so all characters require the same number of bits. Since the largest code points can be expressed in only 21 bits, this encoding is inherently wasteful of space; [[UTF-8]] or [[UTF-16]] is a more efficient coding in most cases. UTF-32 does provide computational simplicity and is more often used for in-memory storage of characters than for stored documents.

[[http://www.unicode.org/faq/utf_bom.html#UTF32 UTF-32 FAQ]]

UTF-32

2012-11-10T06:19:41Z

Gmcgath: New page

UTF-8

2012-11-10T06:05:04Z

Gmcgath: Add details

{{FormatInfo
|formattype=electronic
|subcat=Character Encodings
}}
'''[[UCS]] Transformation Format—8-bit''' (UTF-8) is a [[Unicode]] character encoding. Codes 0-127 (0-7F hexadecimal) represent the equivalent [[ASCII]] characters, and these codes in a UTF-8 stream are never used in any other context. Codes FE and FF are never used, except in the optional Byte Order Mark at the beginning of a document. In UTF-8 the BOM is encoded as the bytes 0xEF, 0xBB, 0xBF. Since UTF-8 has no "endianness," this is not actually a byte order indicator but can be treated as a signature indicating the document is UTF-8 encoded.

UTF-8 is best suited for scripts that make heavy use of the Roman alphabet. With other scripts it may not provide as efficient an encoding as [[UTF-16]] or [[UTF-32]].

== Specifications ==

* [http://tools.ietf.org/html/std63 STD 63]
** RFC 3629 (2003-11)
** RFC 2279 (1998-01)
** RFC 2044 (1996-10)
* [http://www.unicode.org/versions/Unicode6.0.0/ch03.pdf Unicode 6.0, Chapter 3] (2011) – §3.9 D92, §3.10 D95
* ISO/IEC 10646:2003 Annex D (2003)

== External links ==

* [http://doc.cat-v.org/plan_9/4th_edition/papers/utf Hello World or Καλημέρα κόσμε or こんにちは世界] ([http://plan9.bell-labs.com/sys/doc/utf.pdf PDF])

[[Category:Text encoding]]

Unicode

2012-11-10T05:33:54Z

Gmcgath: caveat about 8859-1 and UTF-8

{{FormatInfo
|formattype=electronic
|subcat=Character Encodings
}}
'''Unicode''' is a system for representing characters numerically, and is the basis for various character encodings including the popular [[UTF-8]]. It was devised beginning in 1987, with the first version of its standard published starting in 1991. Subsequent revisions have continually expanded its character repertoire.

Unicode was devised in reaction both to the unwieldy multiplicity of character sets that had arisen to include various subsets of the many characters left out of the English-centric [[ASCII]] set, and to the clumsiness of the proposed ISO 10646 character set which was the awkward product of international politics rather than a sensible technical system. It has been successful to the point where just about all technical standards dealing with characters now are defined with regard to Unicode code points, with even the older proprietary encodings cross-referenced to the Unicode characters they encode.

Early versions of Unicode attempted to be a 16-bit character encoding where characters in a potential repertoire of 65,536 code points could be represented as 16-bit (2-byte) unsigned integers, though the "big-endian vs. little-endian" problem caused there to be two possible byte streams corresponding to a particular document, which could be disambiguated using the "byte-order marker" character which came out as (hexadecimal) FFFE or FEFF depending on which byte order was used. However, from an early time, users of Unicode preferred encodings which did not take up two bytes for every character, even common ones from the ASCII repertoire, so variable-byte-length encodings came into common use. Ultimately, the Unicode standard expanded its repertoire using multiple "planes" so that even 16 bits weren't enough to encode all possible characters.

At its root, Unicode is simply an assignment of numeric values to characters, where a huge number of characters from various [[Written Languages|writing systems]] (modern or ancient) as well as special symbols of many types are each given a number. Usually in Unicode charts these are given as hexadecimal numbers rather than the decimal ones most humans tend to prefer (computer geeks are funny that way), but they can be expressed in any base you want; in [[HTML]] you can use ampersand encoding with either decimal or hexadecimal numbers to express a Unicode character (e.g., &#65; or &#x41;).

Once these numbers are assigned, they can be encoded as sequences of bytes in various ways, defined in the specifications of particular character encodings.

The first 128 Unicode code points, 0-127, correspond to the same code points in ASCII. The next 128, 128-255, correspond to the same points in [[ISO-8859-1]], which in turn contains the same characters at 0-127 as ASCII, so the entire first 256 characters in Unicode are equivalent to that standard. Since UTF-8 encodes characters 0-127 as raw bytes of the appropriate number, a file consisting entirely of ASCII characters will be identical in the us-ascii and UTF-8 encodings. However, characters in the range 128-255 are encoded differently in ISO 8859-1 and UTF-8, so files with these characters will be rendered incorrectly in the wrong encoding.

== References ==
* [http://www.unicode.org/ Unicode official site] -- has lots of standards documents and code charts
* [http://en.wikipedia.org/wiki/Unicode Wikipedia entry on Unicode]
* [http://en.wikipedia.org/wiki/List_of_Unicode_characters Wikipedia list of Unicode Characters]
* [http://shapecatcher.com/ Shapecatcher] - Site for finding unicode characters by drawing them
* http://rishida.net/tools/conversion/ - Tool for converting Unicode into other character formats such as UTF-8, HTML, etc.
* http://www.fileformat.info/info/unicode/char/search.htm - Unicode search at Fileformat.Info

Unicode

2012-11-10T05:26:32Z

Gmcgath: Category

{|
|[[File Formats]]
| >
|[[Electronic File Formats]]
| >
|[[Character Encoding]]
| >
|[[Unicode]]
|}

'''Unicode''' is a system for representing characters numerically, and is the basis for various character encodings including the popular [[UTF-8]]. It was devised beginning in 1987, with the first version of its standard published starting in 1991. Subsequent revisions have continually expanded its character repertoire.

Unicode was devised in reaction both to the unwieldy multiplicity of character sets that had arisen to include various subsets of the many characters left out of the English-centric [[ASCII]] set, and to the clumsiness of the proposed ISO 10646 character set which was the awkward product of international politics rather than a sensible technical system. It has been successful to the point where just about all technical standards dealing with characters now are defined with regard to Unicode code points, with even the older proprietary encodings cross-referenced to the Unicode characters they encode.

Early versions of Unicode attempted to be a 16-bit character encoding where characters in a potential repertoire of 65,536 code points could be represented as 16-bit (2-byte) unsigned integers, though the "big-endian vs. little-endian" problem caused there to be two possible byte streams corresponding to a particular document, which could be disambiguated using the "byte-order marker" character which came out as (hexadecimal) FFFE or FEFF depending on which byte order was used. However, from an early time, users of Unicode preferred encodings which did not take up two bytes for every character, even common ones from the ASCII repertoire, so variable-byte-length encodings came into common use. Ultimately, the Unicode standard expanded its repertoire using multiple "planes" so that even 16 bits weren't enough to encode all possible characters.

At its root, Unicode is simply an assignment of numeric values to characters, where a huge number of characters from various [[Written Languages|writing systems]] (modern or ancient) as well as special symbols of many types are each given a number. Usually in Unicode charts these are given as hexadecimal numbers rather than the decimal ones most humans tend to prefer (computer geeks are funny that way), but they can be expressed in any base you want; in [[HTML]] you can use ampersand encoding with either decimal or hexadecimal numbers to express a Unicode character (e.g., &#65; or &#x41;).

Once these numbers are assigned, they can be encoded as sequences of bytes in various ways, defined in the specifications of particular character encodings.

The first 128 Unicode code points, 0-127, correspond to the same code points in ASCII. The next 128, 128-255, correspond to the same points in [[ISO-8859-1]], which in turn contains the same characters at 0-127 as ASCII, so the entire first 256 characters in Unicode are equivalent to that standard. Since UTF-8 encodes characters 0-127 as raw bytes of the appropriate number, a file consisting entirely of ASCII characters will be identical in the us-ascii and UTF-8 encodings.

== References ==
* [http://www.unicode.org/ Unicode official site] -- has lots of standards documents and code charts
* [http://en.wikipedia.org/wiki/Unicode Wikipedia entry on Unicode]
* [http://en.wikipedia.org/wiki/List_of_Unicode_characters Wikipedia list of Unicode Characters]
* [http://shapecatcher.com/ Shapecatcher] - Site for finding unicode characters by drawing them
* http://rishida.net/tools/conversion/ - Tool for converting Unicode into other character formats such as UTF-8, HTML, etc.
* http://www.fileformat.info/info/unicode/char/search.htm - Unicode search at Fileformat.Info

[[Category:Text encoding]]

Windows 1252

2012-11-10T05:24:42Z

Gmcgath: Category

{|
|[[File Formats]]
| >
|[[Electronic File Formats]]
| >
|[[Character Encoding]]
| >
|[[Windows 1252]]
|}

Windows 1252 is a character encoding used in Microsoft Windows systems, particularly English-langauge installations. It includes all the printable characters of ISO 8859-1 (Latin-1), as well as additional characters in the range 128-159, which in ISO 8859 is reserved for control characters. It's often falsely claimed to be an ANSI standard.

Websites and e-mail messages often mistakenly use headers and parameters claiming a document or message to be in ISO-8859-1 when it actually uses characters that are part of Windows 1252.

== References ==
* [[http://msdn.microsoft.com/en-us/library/cc195054.aspx Microsoft Windows 1252 code page]]

[[Category:Text encoding]]

Character encoding

2012-11-10T05:24:19Z

Gmcgath: Category

{|
|[[File Formats]]
| >
|[[Electronic File Formats]]
| >
|[[Character Encoding]]
|}

* [[Adobe Standard Encoding]]
* [[ARMSCII]]
* [[ASCII]]
* [[ANSEL]]
** [[MARC-8]]
* [[ATASCII]] (used by Atari computers)
* [[Baudot code]]
* [[Braille]]
* [[EBCDIC]]
* [[IBM PC code pages]]
* [[ISO 646]]
** [[ISO 646-CA]] (Canada / French)
** [[ISO 646-CA-2]] (Canada / French)
** [[ISO 646-CH]] (Switzerland)
** [[ISO 646-CN]] (China / Basic Latin)
** [[ISO 646-CU]] (Cuba / Spanish)
** [[ISO 646-DE]] (Germany)
** [[ISO 646-DK]] (Denmark)
** [[ISO 646-FI]] (Finland)
** [[ISO 646-FR]] (France)
** [[ISO 646-GB]] (Great Britain)
** [[ISO 646-HU]] (Hungary)
** [[ISO 646-IRV]] (International Reference Version)
** [[ISO 646-IT]] (Italy)
** [[ISO 646-JP]] (Japan / Romaji)
** [[ISO 646-JP OCR-B]] (Japan / Romaji)
** [[ISO 646-KR]] (Korea / Latin)
** [[ISO 646-MT]] (Malta)
** [[ISO 646-NL]] (Netherlands)
** [[ISO 646-NO]] (Norway)
** [[ISO 646-NO-2]] (Norway)
** [[ISO 646-PT]] (Portugal)
** [[ISO 646-SE]] (Sweden)
** [[ISO 646-SE-2]] (Sweden)
** [[ISO 646-US]] (Same as [[ASCII]])
** [[ISO 646-YU]] (Yugoslavia)
* [[ISO 2022]]
* [[ISO 8859]]
** [[ISO 8859-1]] (Latin-1)
** [[ISO 8859-2]] (Latin-2, Central/East European)
** [[ISO 8859-3]] (Latin-3, Esperanto, Galician, Maltese, and Turkish)
** [[ISO 8859-4]] (Latin-4, Scandinavian and Baltic)
** [[ISO 8859-5]] (Cyrillic)
** [[ISO 8859-6]] (Arabic)
** [[ISO 8859-7]] (Modern Greek)
** [[ISO 8859-8]] (Hebrew)
** [[ISO 8859-9]] (Latin-5, Turkish)
** [[ISO 8859-10]] (Latin-6, Lappish, Nordic, and Inuit)
** [[ISO 8859-11]] (Thai)
** [[ISO 8859-13]] (Latin-7, Baltic Rim)
** [[ISO 8859-14]] (Celtic)
** [[ISO 8859-15]] (Latin-9, Latin-1 with a Euro sign)
** [[ISO 8859-16]] (Romanian)
* [[JIS]]
** [[JIS X 0201]]
** [[JIS X 0208]]
** [[Shift-JIS]]
* [[KOI8]]
** [[KOI8-CS]] (Czechoslovakia)
** [[KOI8-R]] (Russia)
** [[KOI8-U]] (Ukraine)
* [[Macintosh encodings]]
** [[MacCE]]
** [[MacCyrillic]]
** [[MacDingbat]]
** [[MacGreek]]
** [[MacGujarati]]
** [[MacGurmukhi]]
** [[MacIceland]]
** [[MacRoman]]
** [[MacRomania]]
** [[MacSymbol]]
** [[MacThai]]
** [[MacTurkish]]
** [[MacUkraine]]
* [[Morse code]]
* [[MS-DOS encodings]]
** [[MS-DOS Latin US ]]
** [[MS-DOS Greek ]]
** [[MS-DOS Baltic Rim ]]
** [[MS-DOS Latin-1 ]]
** [[MS-DOS Greek 1 ]]
** [[MS-DOS Latin-2 ]]
** [[MS-DOS Cyrillic ]]
** [[MS-DOS Turkish ]]
** [[MS-DOS Portuguese ]]
** [[MS-DOS Icelandic ]]
** [[MS-DOS Hebrew ]]
** [[MS-DOS French Canada ]]
** [[MS-DOS Arabic ]]
** [[MS-DOS Nordic ]]
** [[MS-DOS Cyrillic CIS 1 ]]
** [[MS-DOS Greek 2 ]]
* [[PETSCII]] (or PET ASCII or CBM ASCII; used by Commodore computers)
* [[Unicode]]
** [[UTF-1]]
** [[UTF-16]]
** [[UTF-8]]
** [[UTF-7]]
** [[UTF-EBCDIC]]
* [[VISCII]]
* [[Windows encodings]]
** [[Windows 1252]] (ISO 8859-1 plus additional characters)
** [[Windows 1255]] (Hebrew)
** [[Windows 1256]] (Arabic, Farsi, Urdu)
** [[Windows 1257]] (Baltic Rim)
** [[Windows 1258]] (Vietnamese)
== External links ==

* [http://www.transbay.net/~enf/ascii/ascii.pdf The Evolution of Character Codes, 1874–1968]
* [http://www.kreativekorp.com/charset/ Collection of character encodings]]

[[Category:Text encoding]]

XML

2012-11-10T05:11:44Z

Gmcgath: DTDs and schemas

{{FormatInfo
|formattype=electronic
|subcat=Document
|extensions={{ext|xml}}
|mimetypes=
{{mimetype|application/xml}} 
{{mimetype|text/xml}}
}}
'''Extensible Markup Language''' (XML) is a markup language used to encode data.

XML is a language from which languages are made. A body of rules for how an XML document for specific purpose may be constructed is often called a "language" or a "format" in its own right. These rules may be specified in several different ways, the most common being Document Type Definition (DTD) and Schema. A document which follows the syntactic rules of XML is considered "well-formed." A document which is well-formed and also conforms to its DTD or schema declarations is considered "valid."

A Document Type Definition may be included in an XML document or be referenced by a Document Type Declaration, or both approaches may be combined. An external reference to a DTD is provided by a Document Type Declaration, which confusingly has the same initials.

A Schema, unlike a DTD, is itself written in XML. A document can have a Schema for each of its namespaces. DTDs have been largely superseded by Schemas because of the former's limitation of one DTD per document and the latter's greater capacity for describing rules and namespace support.

XML documents refer to both Schemas and DTDs by a URI. It is crucial to remember that this reference is a Universal Resource Identifier, nor a Universal Resource Locator (URL). There is no requirement that the URI point to a resource on the Internet, or even that such a resource exist. This is a potential preservation risk with XML documents, as they may outlive the DTD and Schema documents that characterize them, or the documents may move and be difficult to locate.

== Specifications ==

* [http://www.w3.org/TR/xml/ XML 1.0]
** [http://www.w3.org/TR/2008/REC-xml-20081126/ XML 1.0, Fifth Edition (2008-11-26)]
** [http://www.w3.org/TR/2006/REC-xml-20060816/ XML 1.0, Fourth Edition (2006-09-29)]
** [http://www.w3.org/TR/2004/REC-xml-20040204/ XML 1.0, Third Edition (2004-02-04)]
** [http://www.w3.org/TR/2000/REC-xml-20001006 XML 1.0, Second Edition (2000-10-06)]
** [http://www.w3.org/TR/1998/REC-xml-19980210 XML 1.0, First Edition (1998-02-10)]
* [http://www.w3.org/TR/xml11/ XML 1.1]
** [http://www.w3.org/TR/2006/REC-xml11-20060816/ XML 1.1, Second Edition (2006-09-29)]
** [http://www.w3.org/TR/2004/REC-xml11-20040204/ XML 1.1, First Edition (2004-04-15)]

== External links ==

* [http://www.w3.org/XML/ W3C XML Homepage]
* [http://www.w3.org/XML/Core/ W3C XML Core Working Group Public Page]

[[Category:Text-based data]]

KOI8

2012-11-10T03:00:23Z

Gmcgath:

{|
|[[File Formats]]
| >
|[[Electronic File Formats]]
| >
|[[Character Encoding]]
| >
|[[KOI8]]
|}

'''KOI8''' is a family of 8-bit character encodings primarily for Cyrillic alphabets. Character codes 32-126 are identical with the corresponding ASCII characters. Cyrillic characters are ordered in phonetic correspondence to ASCII characters in the bottom half of the code table, not in native alphabetical order. The most widely used of these encodings is the Russian KOI8-R.

[[Category:Text encoding]]

VISCII

2012-11-10T02:59:56Z

Gmcgath: Category

{|
|[[File Formats]]
| >
|[[Electronic File Formats]]
| >
|[[Character Encoding]]
| >
|[[VISCII]]
|}

'''VISCII''' is the Vietnamese Standard Code for Information Interchange. It is an 8-bit encoding which is compatible with the printable code range of ASCII but replaces some control codes (in the range 0-31) with printable characters.

[http://www.trichlor.org/what-is-viscii.htm VISCII code table]

[[Category:Text encoding]]

ISO 646

2012-11-10T02:59:19Z

Gmcgath: Category

{|
|[[File Formats]]
| >
|[[Electronic File Formats]]
| >
|[[Character Encoding]]
| >
|[[ISO 646]]
|}

'''ISO 646''' was a set of 7-bit encodings designed to internationalize ASCII. The control characters, digits, letters, and some punctuation are the same in all variants, but some other ASCII characters are replaced by characters needed in particular languages. It is equivalent to the [http://www.itu.int/rec/T-REC-T.50-199209-I/en ISO T.50] International Reference Alphabet.

[[Category:Text encoding]]

ATASCII

2012-11-10T02:58:36Z

Gmcgath: Category

{|
|[[File Formats]]
| >
|[[Electronic File Formats]]
| >
|[[Character Encoding]]
| >
|[[ATASCII]]
|}
__NOTOC__

ATASCII is a eight-bit encoding that was used on Atari computers. It uses most, but not all, of the ASCII printing characters; codes 7B through 7F are different. The characters with the high bit set (80 through FF) are, with some exceptions, inverse video versions of the same code ANDed with 7F. There have been a number of programs to portray ATASCII on modern operating systems, as well as fonts designed from the originals.

=== Documentation ===

* [http://www.faqs.org/faqs/atari-8-bit/faq/section-62.html#b Atari 8-bit FAQ] - What is the ATASCII Character Set?
* [http://en.wikipedia.org/wiki/ATASCII Wikipedia entry on ATASCII]
* [http://spriteshack.com/?p=353 SpriteShack: Using ATASCII in Games]
* [http://www.atarimagazines.com/compute/issue70/113_1_INSIGHT_Atari.php Atari Character Codes] from Computer Magazine Issue #70
* [http://archive.org/details/analog-computing-magazine-31 Issue of Analog Computing Magazine (#31)] covering ATASCII Animations
* [http://www.atariarchives.org/basicxl/showpage.php?page=363 Process of typing in ATASCII on an Atari computer]

=== Code Tables ===

* [http://www.atariarchives.org/mapping/appendix10.php Code table]
* [http://www.atari-forum.com/wiki/index.php?title=The_Atari_character_set The Atari Character Set page at Atari-Forum.com]
* [http://joyfulcoder.net/atascii/ Joyful Coder ATASCII Map]
* [http://atariwiki.strotmann.de/wiki/Wiki.jsp?page=Atari%20ATASCII%20Table ATASCII map in PDF]

=== Programs ===

* [http://sourceforge.net/projects/atascii/ Atari ATASCII Viewer (Java)] on Sourceforge. Last updated in 2001.
* [http://www.leehanken.pwp.blueyonder.co.uk/atascii/ ATASCII View] Windows program for viewing ATASCII characters
* [http://www.vitoco.cl/atari/atasciiprinter.html Web-based ATASCII Printer]
* [http://xent.com/~bsittler/atascii.ti Terminfo for ATASCII]
* [http://syncterm.bbsdev.net/ Syncterm Software] - Terminal program for Windows that supports ATASCII
* [http://joyfulcoder.com/memopad/ Memo Pad], a windows memo pad for ATASCII characters
* [http://bf.amfband.com/bffiles.html File directory containing two ATASCII Telnet Clients for Windows]
* http://ape.dyndns.org:8083/ABBUC/379/IBM2A.C IBM2A.C - Program to make "best guess" conversion from ATASCII to IBM PC Characters (1993)

=== Video ===

* [https://vimeo.com/343238 Display of ATASCII on a 40h light panel] (Video)
* [http://www.youtube.com/watch?v=16CSYOBqdqQ ATARI BBS SIM (850 Express)] Video of a simulation of dialing an Atari BBS
* [http://www.youtube.com/watch?v=NKv6_f5ucMw ATARI TOONS "Aliens"]
* [http://www.youtube.com/watch?v=VHUl7WnJqwE ATARI TOONS "Trek"]

=== Fonts ===

* [http://members.bitstream.net/marksim/atarimac/fonts.html Atari Classic truetype fonts]
* [http://user.xmission.com/~trevin/atari/atari.html TrueType Font of ATASCII Characters]

[[Category:Text encoding]]

XML

2012-11-10T02:55:30Z

Gmcgath:

{{FormatInfo
|formattype=electronic
|subcat=Document
|extensions={{ext|xml}}
|mimetypes=
{{mimetype|application/xml}} 
{{mimetype|text/xml}}
}}
'''Extensible Markup Language''' (XML) is a markup language used to encode data.

== Specifications ==

* [http://www.w3.org/TR/xml/ XML 1.0]
** [http://www.w3.org/TR/2008/REC-xml-20081126/ XML 1.0, Fifth Edition (2008-11-26)]
** [http://www.w3.org/TR/2006/REC-xml-20060816/ XML 1.0, Fourth Edition (2006-09-29)]
** [http://www.w3.org/TR/2004/REC-xml-20040204/ XML 1.0, Third Edition (2004-02-04)]
** [http://www.w3.org/TR/2000/REC-xml-20001006 XML 1.0, Second Edition (2000-10-06)]
** [http://www.w3.org/TR/1998/REC-xml-19980210 XML 1.0, First Edition (1998-02-10)]
* [http://www.w3.org/TR/xml11/ XML 1.1]
** [http://www.w3.org/TR/2006/REC-xml11-20060816/ XML 1.1, Second Edition (2006-09-29)]
** [http://www.w3.org/TR/2004/REC-xml11-20040204/ XML 1.1, First Edition (2004-04-15)]

== External links ==

* [http://www.w3.org/XML/ W3C XML Homepage]
* [http://www.w3.org/XML/Core/ W3C XML Core Working Group Public Page]

[[Category:Text-based data]]

Adobe Standard Encoding

2012-11-10T02:50:32Z

Gmcgath: Caegory

{|
|[[File Formats]]
| >
|[[Electronic File Formats]]
| >
|[[Character Encoding]]
| >
|[[Adobe Standard Encoding]]
|}

'''Adobe Standard Encoding''' is an 8-bit encoding extended from ASCII, used as the default encoding for many PostScript fonts. Codes 32-126 and 161-251 are used.

[http://www.ascii.ca/adobestd.htm Adobe Standard Encoding vector]

[[Category:Text encoding]]

Category:Text encoding

2012-11-10T02:49:08Z

Gmcgath: Am I doing this right?

[[Text encoding]]

ISO 8859

2012-11-10T02:47:03Z

Gmcgath:

{|
|[[File Formats]]
| >
|[[Electronic File Formats]]
| >
|[[Character Encoding]]
| >
|[[ISO 8859]]
|}

ISO/IEC 8859 is a family of 8-bit character encodings that are extensions of ASCII. All of them reserve code values 80 through 9F (128 through 159 decimal) for control characters, though a lot of websites and e-mail programs bogusly indicate one of these encodings for content that actually uses printable characters in those positions via some proprietary system-specific encoding (usually Windows). The ISO 8859 encodings are the following:

* ISO 8859-1: Latin-1, used for western European languages including English [[http://www.kreativekorp.com/charset/encoding.php?file=iso-8859-1.kte Code table]]
* ISO 8859-2: Latin-2, used for central and eastern European languages [[http://www.kreativekorp.com/charset/encoding.php?file=iso-8859-2.kte Code table]]
* ISO 8859-3: Latin-3, for Esperanto, Galician, Maltese, and Turkish [[http://www.kreativekorp.com/charset/encoding.php?file=iso-8859-3.kte Code table]]
* ISO 8859-4: Latin-4, for Scandinavian and Baltic languages [[http://www.kreativekorp.com/charset/encoding.php?file=iso-8859-4.kte Code table]]
* ISO 8859-5: Cyrillic characters [[http://www.kreativekorp.com/charset/encoding.php?file=iso-8859-5.kte Code table]]
* ISO 8859-6: Arabic characters [[http://www.kreativekorp.com/charset/encoding.php?file=iso-8859-6.kte Code table]]
* ISO 8859-7: Modern Greek [[http://www.kreativekorp.com/charset/encoding.php?file=iso-8859-7.kte Code table]]
* ISO 8859-8: Hebrew [[http://www.kreativekorp.com/charset/encoding.php?file=iso-8859-8.kte Code table]]
* ISO 8859-9: Latin-5, Turkish variant of Latin-1 [[http://www.kreativekorp.com/charset/encoding.php?file=iso-8859-9.kte Code table]]
* ISO 8859-10: Latin-6, for Lappish, Nordic, and Inuit languages [[http://www.kreativekorp.com/charset/encoding.php?file=iso-8859-10.kte Code table]]
* ISO 8859-11: Thai [[http://www.kreativekorp.com/charset/encoding.php?file=iso-8859-11.kte Code table]]
* (ISO 8859-12 was abandoned)
* ISO 8859-13: Latin-7, Baltic Rim languages [[http://www.kreativekorp.com/charset/encoding.php?file=iso-8859-13.kte Code table]]
* ISO 8859-14: Latin-8, Celtic [[http://www.kreativekorp.com/charset/encoding.php?file=iso-8859-14.kte Code table]]
* ISO 8859-15: Latin-9, Revision of Latin-1, includes Euro sign [[http://www.kreativekorp.com/charset/encoding.php?file=iso-8859-15.kte Code table]]
* ISO 8859-16: Latin-10, Romanian [[http://www.kreativekorp.com/charset/encoding.php?file=iso-8859-16.kte Code table]]

[[ftp://ftp.unicode.org/Public/MAPPINGS/ISO8859/ ISO 8859 to Unicode mapping tables]]

[[Category:Text encoding]]