cog/Frameworks/OpenMPT.old/OpenMPT/doc/module_formats.md

How to add support for new module formats

This document describes the basics of writing a new module loader and related work that has to be done. We will not discuss in detail how to write the loader, have a look at existing loaders to get an idea how they work in general.

General hints

• We strive for quality over quantity. The goal is not to support as many module formats as possible, but to support them as well as possible.

• Write defensive code. Guard against out-of-bound values, division by zero and similar stuff. libopenmpt is constantly fuzz-tested to catch any crashes, but of course we want our code to be reliable from the start.

• Every format should have its own MODTYPE flag, unless it can be reasonably represented as a subset of another format (like Ice Tracker ICE files being a subset of ProTracker MOD).

• When reading binary structs from the file, use our data types with defined endianness, which can be found in common/Endianness.h:

  • Big-Endian: (u)int8/16/32/64be, float32be, float64be
  • Little-Endian: (u)int8/16/32/64le, float32le, float64le

  Entire structs containing integers with defined endianness can be read in one go if they are tagged with MPT_BINARY_STRUCT (see existing loaders for an example).

• m_nChannels MUST NOT be changed after a pattern has been created, as existing patterns will be interpreted incorrectly. For module formats that support per-pattern channel amounts, the maximum number of channels must be determined beforehand.

• Strings can be safely handled using:

  • FileReader::ReadString and friends for reading them directly from a file
  • mpt::String::ReadBuf for reading them from a struct or char array

  These functions take care of string padding (zero / space padding) and will avoid reading past the end of the string if there is no terminating null character.

• Do not use non-const static variables in your loader. Loaders need to be thread-safe for libopenmpt.

• FileReader instances may be used to treat a portion of another file as its own independent file (through FileReader::ReadChunk). This can be useful with "embedded files" such as WAV or Ogg samples. Container formats are another good example for this usage.

• Samples either use middle-C frequency or finetune + transpose. For the few weird formats that use both, it may make sense to translate everything into middle-C frequency.

• Add the new MODTYPE to CSoundFile::UseFinetuneAndTranspose if applicable, and see if any effect handlers in soundlib/Snd_fx.cpp need to know the new MODTYPE.

• Do not rely on hard-coded magic numbers. For example, when comparing if an index is valid for a given array, do not hard-code the array size but rather use std::size (or mpt::array_size in contexts where std::size is not usable) or, for ensuring that char arrays are null-terminated, mpt::String::SetNullTerminator. Similarly, do not assume any specific quantities for OpenMPT's constants like MAX_SAMPLES, MAX_PATTERN_ROWS, etc. These may change at any time.

• Pay attention to off-by-one errors when comparing against MAX_SAMPLES and MAX_INSTRUMENTS, since sample and instrument numbers are 1-based.

• Placement of the loader function in CSoundFile::Create depends on various factors. In general, module formats that have very bad magic numbers (and thus might cause other formats to get mis-interpreted) should be placed at the bottom of the list. Two notable examples are 669 files, where the first two bytes of the file are "if" (which may e.g. cause a song title starting with "if ..." in various other formats to be interpreted as a 669 module), and of course Ultimate SoundTracker modules, which have no magic bytes at all.

• Avoid use of functions tagged with MPT_DEPRECATED.

Probing

libopenmpt provides fast probing functions that can be used by library users to quickly check if a file is most likely playable with libopenmpt, even if only a fraction of the file is available (e.g. when streaming from the internet).

In order to satisfy these requirements, probing functions should do as little work as possible (e.g. only parse the header of the file), but as much as required to tell with some certainty that the file is really of a certain mod format. However, probing functions should not rely on having access to more than the first CSoundFile::ProbeRecommendedSize bytes of the file.

• Probing functions must not allocate any memory on the heap.
• Probing functions must not return ProbeFailure or ProbeWantMoreData for any file that would normally be accepted by the loader. In particular, this means that any header checks must not be any more aggressive than they would be in the real loader (hence it is a good idea to not copy-paste this code but rather put it in a separate function), and the minimum additional size passed to CSoundFile::ProbeAdditionalSize must not be higher than the biggest size that would cause a hard failure (i.e. returning false) in the module loader.
• Probing functions may return ProbeSuccess for files that would be rejected by a loader after a more thorough inspection. For example, probing functions do not need to verify that all required chunks of an IFF-like file format are actually present, if the header makes it obvious enough that the file is highly likely to be a module.

Adding loader to the build systems and various other locations

Apart from writing the module loader itself, there are a couple of other places that need to be updated:

• Add loader file to build/android_ndk/Android.mk.
• Add loader file to build/autotools/Makefile.am.
• Run build/regenerate_vs_projects.sh / build/regenerate_vs_projects.cmd (depending on your platform)
• Add file extension to installer/filetypes.iss (in four places).
• Add file extension to CTrackApp::OpenModulesDialog in mptrack/Mptrack.cpp.
• Add format information to soundlib/Tables.cpp.