|
|
This article may contain original research. Please improve it by verifying the claims made and adding references. Statements consisting only of original research may be removed. (November 2009) |
MPQ (Mo'PaQ, short for Mike O'Brien Pack, named after its creator[citation needed]), is an archiving file format used in several of Blizzard Entertainment's games.
MPQs used in Blizzard's games generally contain a game's data files, including graphics, sounds, and level data. The format's capabilities include compression, encryption, file segmentation, extensible file metadata, cryptographic signature and the ability to store multiple versions of the same file for internationalization and platform-specific differences. MPQ archives can use a variety of compression algorithms which may also be combined.
|
Contents
- 1 File indexing
- 2 Encryption
- 3 Revisions
- 4 Archive metadata
- 5 Compression
- 6 Warcraft III cinematics
- 7 Usage in gaming
- 8 References
- 9 External links
- 9.1 Specifications
- 9.2 Libraries
- 9.3 Programs
|
File indexing
In order to meet the requirements of speed generally demanded by a computer game, files are indexed in a hash table using a quick, low-collision hashing algorithm. The index of a specific file within the hash table is the hash of the lowercased filename modulo the size of the hash table, allowing for quick verification of a file's existence within the archive. If multiple files within the archive have the same hash, colliding entries will follow each other in increasing index order (forming a colliding hash cluster). In order to identify the exact entry for the requested file within a colliding hash cluster, each hash table entry stores 2 additional hashes of the lowercased filename, each using the same hashing algorithm but with a different seed value, as well as a locale code and platform code. The end of a colliding hash cluster is detected either by encountering an empty hash table entry or by traversing the entire hash table (including the modulo loopback) back to the initial hash table index.
Encryption
Both the block table (which contains information on where the file data is located in the archive) and the hash table used for file indexing are encrypted when stored. The encryption process which is used by default uses a known algorithm.
Revisions
The file header reserves space to contain format version data. Warcraft III ignores format version data of .mpq compliant files it loads and assumes all are version 1.
- Version 1 was used before World of Warcraft.
- Version 2 added an extended header to the format which contained data for an extended block table to allow for larger archive sizes.
Archive metadata
MPQ archives do not have specific structures to store metadata beyond what is absolutely necessary to access archived files. Instead, the convention is to use regular files whose filename is enclosed by parentheses.
Below are known metadata files.
- (listfile): Contains a list of the archive's files, one filename per line. May or may not be exhaustive.
- (signature): Contains the weak cryptographic signature of the archive. This type of signature is deprecated.
- (attributes): Contains extended file metadata. Currently known attributes are file creation date, CRC32 checksum and MD5 checksum.
Compression
In modern MPQ archives, each segment (or sector) of a file can be compressed using a combination of compression algorithms. A header byte is prepended to every compressed sector to indicate which compressions were used. The order in which those compressors are applied is hardcoded.
The following algorithms are currently in use by Blizzard games:
- PKZIP (licensed from PKWARE). The first compression algorithm available.[1]
- Huffman tree compression combined with ADPCM 4:1 compression (both introduced in StarCraft). Latter algorithm is lossy and only suitable for raw PCM input data.
- zlib (introduced in Warcraft III).
- bzip2 (introduced in World of Warcraft).
- LZMA (introduced in StarCraft II).
Since there was only one compression algorithm available when MPQs were first deployed in Diablo, those archives used a different archive file metadata flag to indicate compression and did not use a compression header byte.
Warcraft III cinematics
Cutscene cinematics with the MPQ extension are included with Warcraft III and, despite the file extension, are not actual MPQ files. Rather they are AVI files compressed with Blizzard's renamed MPEG-4 codec, BLZ0 (which actually is DivX). These files are playable in ordinary media players, provided the proper codecs are installed. Blizzard probably used the MPQ extension on those files to hide them from players, who could have spoiled the game by seeing them out of the gameplay. Nevertheless, World of Warcraft cinematics use the AVI extension.
Usage in gaming
Blizzard has utilized the MPQ file format for archiving game files in a number of their games, including:
- Diablo
- Diablo: Hellfire developed by Synergistic Software
- Lords of Magic developed by Sierra Entertainment
- Diablo II
- Diablo II: Lord of Destruction
- StarCraft
- StarCraft: Brood War
- WarCraft II: Battle.net Edition
- Warcraft III: Reign of Chaos
- Warcraft III: The Frozen Throne
- World of Warcraft and all its expansions
- StarCraft II: Wings of Liberty
- Diablo III
References
- ^ "Phillip Katz, Computer Software Pioneer, 37". The New York Times. Monday, May 1, 2000. http://www.nytimes.com/2000/05/01/us/phillip-katz-computer-software-pioneer-37.html. Retrieved 2011-07-14.
External links
Specifications
- The MoPaQ Archive Format
- Inside MoPaQ — A guide to the file format. (Not maintained, Win32 only, no source code.).
Libraries
- StormLib high level opensource cross-platform C++ library for manipulating MPQ archives as well as partial, encrypted and patched MPQ archives.
- python-mpq StormLib-based Python bindings for MPQ file reading.
- MpqLib StormLib-based .NET 3.5 binbings for MPQ file reading.
- libmpq opensource cross-platform C library (and associated tools) for extracting files from MPQ archives.
- jlibmpq java wrapper for libmpq.
- MPQKit Objective-C framework for Mac OS X to manipulate MPQ archives, with signature verification support.
Programs
- Ladik's MPQ Editor, a .MPQ file editor for Windows, Storm.dll, StormLib, libraries for reading and writing .MPQ files.
- MPQ Extractor — A Mac OS X extractor utility.
- MPQDraft, an open source program that allows users to work with MPQ files.
- SMPQ, an open source command line program and KDE4 KIO plugin based on StormLib that can create, append or extract files from MPQ archives.
|
Archive formats
|
|
| Archiving only |
- ar
- cpio
- shar
- tar
- LBR
- BagIt
|
|
| Compression only |
- bzip2
- gzip
- lzip
- lzop
- xz
- SQ
- compress
|
|
| Archiving and compression |
- 7z
- ACE
- ARC
- ARJ
- Cabinet
- cfs
- cpt
- DGCA
- .dmg
- .egg
- GCA
- kgb
- LHA
- LZX
- MPQ
- PEA
- qda
- RAR
- rzip
- sit
- SQX
- UDA
- UHARC
- Xar
- zoo
- ZIP
|
|
| Software packaging and distribution |
- deb
- pkg (Mac OS X)
- RPM
- RUNZ
- MSI
- JAR
|
|
| Document packaging and distribution |
- OEB Package Format
- OEBPS Container Format
- Open Packaging Conventions
- PAQ
|
|
|
|
|
|
Warcraft series
|
|
| Warcraft |
- Orcs & Humans
- II: Tides of Darkness (Beyond the Dark Portal)
- Adventures: Lord of the Clans
- III: Reign of Chaos (The Frozen Throne)
|
|
| World of Warcraft |
- World of Warcraft
- The Burning Crusade
- Wrath of the Lich King
- Cataclysm
- Mists of Pandaria
|
|
| Tabletop games |
- The Board Game
- The Roleplaying Game
- Trading Card Game
|
|
| Books |
- Day of the Dragon
- Lord of the Clans
- Of Blood and Honor
- The Last Guardian
- War of the Ancients Trilogy
- Rise of the Horde
- Tides of Darkness
- Beyond the Dark Portal
- Arthas: Rise of the Lich King
|
|
| Graphic novel |
- The Sunwell Trilogy
- World of Warcraft
- Legends
- Dragons of Outland
|
|
| Professional competitions |
- 2006 e-Sports World Champions
- Korean Warcraft III Championships
- Korean Warcraft III Rankings
- Warcraft III World Championships
|
|
| Related articles |
- BLP (file format)
- Defense of the Ancients
- JASS
- MDX (file format)
- MPQ (file format)
- Characters of Warcraft
- Races and factions of Warcraft
- Stratagus
- Warcraft III World Editor
|
|
|
Book:Warcraft
|
|
|
StarCraft series
|
|
| Video games |
- StarCraft
- Ghost
- II: Wings of Liberty
- Heart of the Swarm
- Legacy of the Void
|
|
| Information and fiction |
- Media
- Races
- Characters
- Jim Raynor
- Sarah Kerrigan
- Zeratul
- Locations
|
|
| Tabletop games |
- Adventures
- The Board Game
|
|
| Miscellaneous |
- Chris Metzen
- MPQ file format
- Brood War professional competition
- Battle.net
|
|
- Book:StarCraft
- Category:StarCraft
|
|
