Sources

From Just Solve the File Format Problem
(Difference between revisions)
Jump to: navigation, search
(→‎Other Useful Materials and Services: Updated link)
(→‎List of Places Keeping Track of Formats)
 
(21 intermediate revisions by 10 users not shown)
Line 4: Line 4:
 
==List of Places Keeping Track of Formats==
 
==List of Places Keeping Track of Formats==
  
−
* [http://www.forensicswiki.org/wiki/Category:File_Formats Forensics Wiki]
+
{| class="wikitable" style="width: 100%;"
−
* http://archivematica.org/preservation
+
|-
−
* http://www.fileformat.info/
+
! URL !! Notes
−
* http://www.ace.net.nz/tech/TechFileFormat.html
+
|-
−
* http://wotsit.org/ - their download page is broken. need to ping webmaster about that.
+
| [https://fileinfo.com/ FileInfo]
−
* Wikipedia: http://en.wikipedia.org/wiki/List_of_file_formats and http://en.wikipedia.org/wiki/List_of_file_formats_%28alphabetical%29
+
| A vast catalog of file extensions, but the data they give is very minimal.
−
** Also [[wikipedia:Template:Infobox file format]] ([http://en.wikipedia.org/w/index.php?title=Special%3AWhatLinksHere&limit=500&target=Template%3AInfobox+file+format&namespace=0 pages using this template])
+
|-
−
* http://unarchiver.c3.cx/formats
+
| [https://filext.com/ FilExt]
−
* [[PRONOM|The PRONOM format registry]]
+
| Similar to FileInfo
−
* http://wiki.multimedia.cx/
+
|-
−
* http://archive.org/download/file-format-encyclopedia/fileformat.zip/
+
| [https://forensics.wiki/ Forensics Wiki]
−
* http://www.digitalpreservation.gov/formats/fdd/descriptions.shtml
+
|
−
* http://www.udfr.org/, see also http://gdfr.info/.
+
|-
−
* http://gitorious.org/re-lab Tools & specs from re-lab- graphics formats, and office formats
+
| http://archivematica.org/preservation
−
* http://www-mmsp.ece.mcgill.ca/documents/AudioFormats/index.html - Documentation, details, specs, and samples for a handful of audio formats
+
|
−
* http://wiki.xentax.com/index.php/Game_File_Format_Central This wiki is the home of the most game (archive) file format knowledge in the world.
+
|-
−
* http://rewiki.regengedanken.de has some more game related reverse engineering
+
| http://www.fileformat.info/
−
* http://mark0.net/soft-trid-e.html Home to TrID, which can identify a vast number of file formats using an updateable database
+
|
−
* http://mark0.net/soft-trid-deflist.html Online searchable version of the TrID database file
+
|-
−
* http://file-extension.net/seeker/ Metasearch engine for file extensions - Find info about unknown file extensions or filetypes
+
| http://www.ace.net.nz/tech/TechFileFormat.html
−
* http://www.vgmpf.com/Wiki/index.php?title=Main_Page - the Videogame Music Preservation Foundation Wiki has information on a number of formats used to play back videogame music
+
|
−
* http://hackipedia.org/File%20formats/ - loads of info on various formats here
+
|-
−
* http://wiki.mobileread.com/wiki/E-book_formats - detailed information on eBook formats.
+
| http://en.wikipedia.org/wiki/List_of_file_formats
−
* http://mediapedia.nla.gov.au/ - "The Mediapedia resource is intended to enable the identification of various physical media carrier types for assisting with collection planning, assessment, documentation, infrastructure and preservation planning for the content they hold. These could include media across various genres such as cine, video, photo, audio, data, paper carriers, microfilm, etc."
+
| Also [[wikipedia:Template:Infobox file format]] ([http://en.wikipedia.org/w/index.php?title=Special%3AWhatLinksHere&limit=500&target=Template%3AInfobox+file+format&namespace=0 pages using this template])
−
* http://wiki.opf-labs.org/display/TR/Formats - Small list of formats with detailed descriptions of some known access/preservation problems.
+
|-
−
* [http://registry.dcc.ac.uk:8080/RegistryWeb/Registry/ CASPAR/DCC Representation Information Repository]
+
| [https://wikidata.org Wikidata]
−
* [http://theswo.sourceforge.net/ The Software Ontology]
+
| we can use SPARQL to [https://w.wiki/6L8g query for MIME types] or [https://w.wiki/6L8h file extensions].
−
* [http://keep-totem.co.uk/ KEEP Totem] also covers software, hardware etc.
+
|-
−
* [http://cultureandcommunication.org/deadmedia/index.php/Main_Page Dead Media Archives]
+
| [[PRONOM|The PRONOM format registry]]
−
* [http://www.fid3.com/file-formats The >4000 formats known to FI Tools], [http://www.fid3.com/formats-ext1 by extension], [http://www.fid3.com/formats-mime by MIME type]
+
|
−
* [http://hg.python.org/cpython/file/2.7/Lib/mimetypes.py#l402 Python sourcecode for mimetypes module]
+
|-
−
* /etc/mimetypes on *nix systems defines a number of file formats and [[MIME types]]
+
| http://wiki.multimedia.cx
−
* http://www.iasa-web.org/tc04/key-digital-principles - Recommended Audio formats by Audiovisual Archives and Institutions
+
|
−
* http://www.gpsbabel.org/capabilities.html - List of formats used by GPS devices
+
|-
−
* http://www.microsoft.com/openspecifications/en/us/programs/osp/office-file-formats/default.aspx Microsoft's Open Document Specifications
+
| http://archive.org/download/file-format-encyclopedia/fileformat.zip
−
* http://mamedev.org/source/src/lib/formats/index.html http://mamedev.org/source/src/mess/formats/index.html - format handlers for disk images, cassette images, and memory snapshots/quickloads in the MESS source code
+
|
−
* http://minidisc.org alot of minidisc related info
+
|-
−
* http://cool.conservation-us.org/ Conservation Online
+
| http://www.digitalpreservation.gov/formats/fdd/descriptions.shtml
−
* http://filext.com/ File Extensions with MIME types
+
|
−
* http://en.wikipedia.org/wiki/List_of_file_formats_(alphabetical) Wikipedia list of file formats (alphabetical)
+
|-
−
* http://ist.uwaterloo.ca/~schepers/formats.html The C64 file formats list
+
| http://www.udfr.org/
−
* http://3dbrew.org/wiki/Category:File_formats Nintendo 3DS file formats
+
| See also http://gdfr.info/
−
* http://www.iana.org/assignments/media-types/index.html IANA MIME Media Types - lots of useful info here and often quite detailed as well
+
|-
−
* http://gitorious.org/msoscheme msoscheme is a project that creates a machine readable description of the binary file formats of Microsoft Office
+
| http://gitorious.org/re-lab  
−
* http://www.schnarff.com/file-formats/ File Formats documentation collection
+
| Tools & specs from re-lab- graphics formats, and office formats
−
* http://the-fr.org/ The Format Registry
+
|-
−
* http://www.freebase.com/computer/file_format?instances= Freebase instances in category "File Format"
+
| http://www-mmsp.ece.mcgill.ca/documents/AudioFormats/index.html
−
* https://bitbucket.org/haypo/hachoir/src/default/hachoir-parser/hachoir_parser/ Hachoir file format parsers
+
| Documentation, details, specs, and samples for a handful of audio formats
−
* https://bitbucket.org/haypo/hachoir/wiki/FileFormatResources Hachoir list of file format resources
+
|-
−
* https://theiphonewiki.com/wiki/Category:File_Formats iPhone related file formats
+
| http://wiki.xentax.com/index.php/Game_File_Format_Central
−
* https://www.synalysis.net/formats.xml Synalize It file format grammars
+
| This wiki is the home of the most game (archive) file format knowledge in the world.
−
* [http://www.binwalk.org Binwalk] — can recognise and extract data in many formats embeddded inside firmware images
+
|-
−
* http://www.math.utah.edu/~beebe/fonts/X-Window-System-fonts.html - Good info about the Fonts on Windows X system
+
| http://mark0.net/soft-trid-e.html
−
* http://www.online-convert.com/file-type
+
| Home to TrID, which can identify a vast number of file formats using an updateable database
 +
|-
 +
| http://mark0.net/soft-trid-deflist.html
 +
| Online searchable version of the TrID database file
 +
|-
 +
| http://file-extension.net/seeker/
 +
| Metasearch engine for file extensions. Find info about unknown file extensions or filetypes
 +
|-
 +
| http://www.vgmpf.com/Wiki/index.php?title=Main_Page
 +
| the Videogame Music Preservation Foundation Wiki has information on a number of formats used to play back videogame music
 +
|-
 +
| http://hackipedia.org/File%20formats/
 +
| loads of info on various formats here
 +
|-
 +
| http://wiki.mobileread.com/wiki/E-book_formats
 +
| detailed information on e-Book formats.
 +
|-
 +
| http://mediapedia.nla.gov.au/
 +
| "The Mediapedia resource is intended to enable the identification of various physical media carrier types for assisting with collection planning, assessment, documentation, infrastructure and preservation planning for the content they hold. These could include media across various genres such as cine, video, photo, audio, data, paper carriers, microfilm, etc."
 +
|-
 +
| http://wiki.opf-labs.org/display/TR/Formats
 +
| Small list of formats with detailed descriptions of some known access/preservation problems.
 +
|-
 +
| [http://registry.dcc.ac.uk:8080/RegistryWeb/Registry/ CASPAR/DCC Representation Information Repository]
 +
|
 +
|-
 +
| [http://theswo.sourceforge.net/ The Software Ontology]
 +
|
 +
|-
 +
| [http://keep-totem.co.uk/ KEEP Totem]
 +
| Also covers software, hardware etc.
 +
|-
 +
| [http://cultureandcommunication.org/deadmedia/index.php/Main_Page Dead Media Archives]
 +
|
 +
|-
 +
| [http://www.fid3.com/file-formats The >4000 formats known to FI Tools]<br>[http://www.fid3.com/formats-ext1 by extension]<br>[http://www.fid3.com/formats-mime by MIME type]
 +
|
 +
|-
 +
| [http://hg.python.org/cpython/file/2.7/Lib/mimetypes.py#l402 Python source code for mimetypes module]
 +
| /etc/mimetypes on *nix systems defines a number of file formats and [[MIME types]]
 +
|-
 +
| http://www.iasa-web.org/tc04/key-digital-principles
 +
| Recommended Audio formats by Audiovisual Archives and Institutions
 +
|-
 +
| http://www.gpsbabel.org/capabilities.html
 +
| List of formats used by GPS devices
 +
|-
 +
| [http://www.microsoft.com/openspecifications/en/us/programs/osp/office-file-formats/default.aspx Microsoft's Open Document Specifications]
 +
|
 +
|-
 +
| http://mamedev.org/source/src/lib/formats/index.html<br>http://mamedev.org/source/src/mess/formats/index.html
 +
| Format handlers for disk images, cassette images, and memory snapshots/quickloads in the MESS source code.
 +
|-
 +
| http://minidisc.org
 +
| A lot of minidisc related info
 +
|-
 +
| http://cool.conservation-us.org/
 +
| Conservation Online
 +
|-
 +
| http://filext.com/
 +
| File Extensions with MIME types
 +
|-
 +
| http://en.wikipedia.org/wiki/List_of_file_formats_(alphabetical)
 +
| Wikipedia list of file formats (alphabetical)
 +
|-
 +
| http://ist.uwaterloo.ca/~schepers/formats.html
 +
| The C64 file formats list
 +
|-
 +
| http://3dbrew.org/wiki/Category:File_formats
 +
| Nintendo 3DS file formats
 +
|-
 +
| [http://www.iana.org/assignments/media-types/index.html IANA MIME Media Types]
 +
| lots of useful info here and often quite detailed as well
 +
|-
 +
| [http://gitorious.org/msoscheme msoscheme]
 +
| Project that creates a machine-readable description of the binary file formats of Microsoft Office
 +
|-
 +
| http://www.schnarff.com/file-formats/
 +
| File Formats documentation collection
 +
|-
 +
| [http://the-fr.org/ The Format Registry]
 +
|-
 +
| http://www.freebase.com/computer/file_format?instances=
 +
| Freebase instances in category "File Format"
 +
|-
 +
| https://bitbucket.org/haypo/hachoir/src/default/hachoir-parser/hachoir_parser/
 +
| Hachoir file format parsers
 +
|-
 +
| https://bitbucket.org/haypo/hachoir/wiki/FileFormatResources
 +
| Hachoir list of file format resources
 +
|-
 +
| https://theiphonewiki.com/wiki/Category:File_Formats
 +
| iPhone related file formats
 +
|-
 +
| [https://www.synalysis.net/formats.xml Synalize It]
 +
| File-format grammars
 +
|-
 +
| [http://www.binwalk.org Binwalk]
 +
| Can recognise and extract data in many formats embedded inside firmware images
 +
|-
 +
| http://www.math.utah.edu/~beebe/fonts/X-Window-System-fonts.html
 +
| Good info about the Fonts on Windows X system
 +
|-
 +
| http://www.online-convert.com/file-type
 +
|
 +
|-
 +
| https://melog.info/archives/2004/09/30223144.html
 +
| PC98 disk image formats (Japanese)
 +
|-
 +
| http://lclevy.free.fr/amiga/formats.html
 +
| Amiga file formats
 +
|-
 +
| http://hitmen.c02.at/files/yagcd/yagcd/
 +
| Contains Nintendo GameCube file formats
 +
|-
 +
| https://wiibrew.org/wiki/Category:File_formats
 +
| Files related to the Nintendo Wii
 +
|-
 +
| http://wiki.tockdom.com/wiki/Category:File_Format
 +
| File formats related to the Mario Kart games
 +
|-
 +
| https://www.cgsecurity.org/wiki/File_Formats_Recovered_By_PhotoRec
 +
| File Format Signatures used in data recovery
 +
|-
 +
| [[Atari Forum Wiki]]
 +
|
 +
|-
 +
| http://www.undocprint.org/
 +
| Information on printing-related formats, languages, and protocols
 +
|-
 +
| https://extension.nirsoft.net
 +
| Contains program data related to files.
 +
|}
 +
 
 +
The following resources are no longer available:
 +
 
 +
{| class="wikitable" style="width: 100%;"
 +
|-
 +
! URL !! Notes
 +
|-
 +
| http://wotsit.org/
 +
| (<s>their download page is broken. need to ping webmaster about that.</s> now the entire website is just a parking page)
 +
|-
 +
| http://unarchiver.c3.cx/formats
 +
| ([https://web.archive.org/web/20170721220043/http://unarchiver.c3.cx/formats View on the Wayback Machine])
 +
|-
 +
| http://rewiki.regengedanken.de/ Game reverse-engineering
 +
| ([https://web.archive.org/web/20180105180951/http://rewiki.regengedanken.de/wiki/Main_Page View on the Wayback Machine])
 +
|}
  
 
==Emulation==
 
==Emulation==
Line 76: Line 224:
 
* [http://www.amazon.com/Lotus-File-Formats-1-2-3-Symphony/dp/0201168243/ Lotus File Formats for 1-2-3 Symphony & Jazz: File Structure Descriptions for Developers] from Lotus, Inc. (1986)
 
* [http://www.amazon.com/Lotus-File-Formats-1-2-3-Symphony/dp/0201168243/ Lotus File Formats for 1-2-3 Symphony & Jazz: File Structure Descriptions for Developers] from Lotus, Inc. (1986)
 
* [http://www.amazon.com/Inside-Windows-File-Formats-Swan/dp/0672303388/ Inside Windows File Formats] by Tom Swan (Nov 1993)
 
* [http://www.amazon.com/Inside-Windows-File-Formats-Swan/dp/0672303388/ Inside Windows File Formats] by Tom Swan (Nov 1993)
−
* [http://www.amazon.com/Internet-File-Formats-Complete-Receiving/dp/188357756X/ Internet File Formats: Your Complete Resource for Sending, Receiving, and Using Internet Files] by Tim Kientzle (Oct 15, 1995)
+
* [http://www.amazon.com/Internet-File-Formats-Complete-Receiving/dp/188357756X/ Internet File Formats: Your Complete Resource for Sending, Receiving, and Using Internet Files] by Tim Kientzle (Oct 15, 1995) [https://archive.org/details/mac_Internet_File_Formats_1995 Archive.org Scan], [https://archive.org/details/internet-file-formats-cd Companion CD-ROM]
 
* [http://www.amazon.com/File-Formats-Popular-PC-Software/dp/0471528064/ File Formats for Popular PC Software] by Jeff Walden (Nov 1992)
 
* [http://www.amazon.com/File-Formats-Popular-PC-Software/dp/0471528064/ File Formats for Popular PC Software] by Jeff Walden (Nov 1992)
 
* [http://www.amazon.com/File-Formats-Conversions-Book-Developers/dp/155755059X/ PC File Formats and Conversions/Book and Disk (Developers Series)] by Ralf Kussmann (Mar 1990)
 
* [http://www.amazon.com/File-Formats-Conversions-Book-Developers/dp/155755059X/ PC File Formats and Conversions/Book and Disk (Developers Series)] by Ralf Kussmann (Mar 1990)
Line 85: Line 233:
 
* [http://www.amazon.com/Window-System-Formats-Application-Conventions/dp/1872630154/ X Window System File Formats and Application Conventions (X11R4)] by The Open Group (Jul 17, 1991)
 
* [http://www.amazon.com/Window-System-Formats-Application-Conventions/dp/1872630154/ X Window System File Formats and Application Conventions (X11R4)] by The Open Group (Jul 17, 1991)
 
* [http://www.iasa-web.org/audio-preservation-tc04 Guidelines on the Production and Preservation of Digital Audio Objects] by IASA (2009)
 
* [http://www.iasa-web.org/audio-preservation-tc04 Guidelines on the Production and Preservation of Digital Audio Objects] by IASA (2009)
 +
* Android Forensics: Investigation, Analysis and Mobile Security for Google Android - Andrew Hoog - contains info on the Android file structure and how to obtain data from a locked phone, usually of a dead person, or a poor guy who forgot his password.
  
 
==Other Useful Materials and Services==
 
==Other Useful Materials and Services==
Line 123: Line 272:
 
* [http://access.bitcurator.net/index.php?title=Main_Page BitCurator access tools]
 
* [http://access.bitcurator.net/index.php?title=Main_Page BitCurator access tools]
 
* [http://www.documentliberation.org/projects/ Document Liberation Project projects]
 
* [http://www.documentliberation.org/projects/ Document Liberation Project projects]
 +
* [https://mediaarea.net/ollistd Open LossLess in STanDards] ([https://github.com/mediaarea/ollistd GitHub])
 +
* http://www.manmrk.net/tutorials/compress/index.htm A large collection of 1990s compression software
  
 
[[Category:Sources| ]]
 
[[Category:Sources| ]]

Latest revision as of 10:26, 17 May 2024

If you're looking for places to pull either file format information, or to verify stuff you've found, or to go that extra mile to track down variations, here's some similar endeavors and items.

[edit] List of Places Keeping Track of Formats

URL Notes
FileInfo A vast catalog of file extensions, but the data they give is very minimal.
FilExt Similar to FileInfo
Forensics Wiki
http://archivematica.org/preservation
http://www.fileformat.info/
http://www.ace.net.nz/tech/TechFileFormat.html
http://en.wikipedia.org/wiki/List_of_file_formats Also wikipedia:Template:Infobox file format (pages using this template)
Wikidata we can use SPARQL to query for MIME types or file extensions.
The PRONOM format registry
http://wiki.multimedia.cx
http://archive.org/download/file-format-encyclopedia/fileformat.zip
http://www.digitalpreservation.gov/formats/fdd/descriptions.shtml
http://www.udfr.org/ See also http://gdfr.info/
http://gitorious.org/re-lab Tools & specs from re-lab- graphics formats, and office formats
http://www-mmsp.ece.mcgill.ca/documents/AudioFormats/index.html Documentation, details, specs, and samples for a handful of audio formats
http://wiki.xentax.com/index.php/Game_File_Format_Central This wiki is the home of the most game (archive) file format knowledge in the world.
http://mark0.net/soft-trid-e.html Home to TrID, which can identify a vast number of file formats using an updateable database
http://mark0.net/soft-trid-deflist.html Online searchable version of the TrID database file
http://file-extension.net/seeker/ Metasearch engine for file extensions. Find info about unknown file extensions or filetypes
http://www.vgmpf.com/Wiki/index.php?title=Main_Page the Videogame Music Preservation Foundation Wiki has information on a number of formats used to play back videogame music
http://hackipedia.org/File%20formats/ loads of info on various formats here
http://wiki.mobileread.com/wiki/E-book_formats detailed information on e-Book formats.
http://mediapedia.nla.gov.au/ "The Mediapedia resource is intended to enable the identification of various physical media carrier types for assisting with collection planning, assessment, documentation, infrastructure and preservation planning for the content they hold. These could include media across various genres such as cine, video, photo, audio, data, paper carriers, microfilm, etc."
http://wiki.opf-labs.org/display/TR/Formats Small list of formats with detailed descriptions of some known access/preservation problems.
CASPAR/DCC Representation Information Repository
The Software Ontology
KEEP Totem Also covers software, hardware etc.
Dead Media Archives
The >4000 formats known to FI Tools
by extension
by MIME type
Python source code for mimetypes module /etc/mimetypes on *nix systems defines a number of file formats and MIME types
http://www.iasa-web.org/tc04/key-digital-principles Recommended Audio formats by Audiovisual Archives and Institutions
http://www.gpsbabel.org/capabilities.html List of formats used by GPS devices
Microsoft's Open Document Specifications
http://mamedev.org/source/src/lib/formats/index.html
http://mamedev.org/source/src/mess/formats/index.html 		Format handlers for disk images, cassette images, and memory snapshots/quickloads in the MESS source code.
http://minidisc.org A lot of minidisc related info
http://cool.conservation-us.org/ Conservation Online
http://filext.com/ File Extensions with MIME types
http://en.wikipedia.org/wiki/List_of_file_formats_(alphabetical) Wikipedia list of file formats (alphabetical)
http://ist.uwaterloo.ca/~schepers/formats.html The C64 file formats list
http://3dbrew.org/wiki/Category:File_formats Nintendo 3DS file formats
IANA MIME Media Types lots of useful info here and often quite detailed as well
msoscheme Project that creates a machine-readable description of the binary file formats of Microsoft Office
http://www.schnarff.com/file-formats/ File Formats documentation collection
The Format Registry
http://www.freebase.com/computer/file_format?instances= Freebase instances in category "File Format"
https://bitbucket.org/haypo/hachoir/src/default/hachoir-parser/hachoir_parser/ Hachoir file format parsers
https://bitbucket.org/haypo/hachoir/wiki/FileFormatResources Hachoir list of file format resources
https://theiphonewiki.com/wiki/Category:File_Formats iPhone related file formats
Synalize It File-format grammars
Binwalk Can recognise and extract data in many formats embedded inside firmware images
http://www.math.utah.edu/~beebe/fonts/X-Window-System-fonts.html Good info about the Fonts on Windows X system
http://www.online-convert.com/file-type
https://melog.info/archives/2004/09/30223144.html PC98 disk image formats (Japanese)
http://lclevy.free.fr/amiga/formats.html Amiga file formats
http://hitmen.c02.at/files/yagcd/yagcd/ Contains Nintendo GameCube file formats
https://wiibrew.org/wiki/Category:File_formats Files related to the Nintendo Wii
http://wiki.tockdom.com/wiki/Category:File_Format File formats related to the Mario Kart games
https://www.cgsecurity.org/wiki/File_Formats_Recovered_By_PhotoRec File Format Signatures used in data recovery
Atari Forum Wiki
http://www.undocprint.org/ Information on printing-related formats, languages, and protocols
https://extension.nirsoft.net Contains program data related to files.

The following resources are no longer available:

URL Notes
http://wotsit.org/ (their download page is broken. need to ping webmaster about that. now the entire website is just a parking page)
http://unarchiver.c3.cx/formats (View on the Wayback Machine)
http://rewiki.regengedanken.de/ Game reverse-engineering (View on the Wayback Machine)

[edit] Emulation

  • http://nonmess.retrogames.com/ - Emulator listing for various computer formats, including MESS and single-machine emulators.
  • The Emulation Framework is software developed by the international KEEP project, co-funded by the European Union's 7th Framework Programme.

[edit] Books

[edit] Other Useful Materials and Services

Retrieved from "http://fileformats.archiveteam.org/index.php?title=Sources&oldid=48597"
Personal tools
Namespaces

Variants
Actions
Navigation
Toolbox