OMPC Import Hooks


The aim of OMPC is to give user the ability to seamlessly cross from the world of MALTAB into the free and much more colorful world of Python. Python has all the functionality available, but it is hidden behind different syntax.

There will be two major ways of using OMPC.

  1. do "import ompc" in a regular python, this will install hooks that allow one to reuse m-files as regular Python functions,
  2. run ompcpython from the command line, this should work as MATLAB python files can be imported using a special function.

To start using OMPC the user is supposed to do the following:

import ompc

Behind this very simple statement the python instance is supposed to import all the numerical modules and install a hook to the import routine that will allow importing MATLAB's m-files.

Importing M-files


The module ihooks provides this functionality, we attach our implementation of ModuleLoader to the ihooks module. After installing our hook when a module is searched for we need to include '*.m' (possible mex, dll and others) files in the search. The implementation is in the file ompc_ihooks.py.
To make it possible to import MATLAB importable files a hook has to be installed to the import process. All the import in Python go through the __import__ function. This function searches on the Python path for files with matching filename and supported extensions. The module ompc_ihooks.py installs these hooks and makes it possible to transparently import m-files. At every first import of a Python module this module has to be compiled. The same is true for every m-file, at the time of first import or reload the m-file is compiled. Compilation consists of MATLAB to Python adaptation and comilation of the wrapper to a regular .pyc or .pyo file.

The implementation is inspired by three examples:

  • http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/59867
  • http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/496692
  • http://svn.python.org/projects/python/trunk/Demo/imputil/importers.py

MATLAB makes functions accessible based on filesystem information. The function addpath

addpath dir1 dir2 dir3 ... -0
sys.path[:0] = [dir1, dir2, ...]
addpath dir1 dir2 dir3 ... -1
sys.path += [dir1, dir2, ...]

We should consider creating a completely separate variable sys.mpath to allow possible optimization of lookup and filesystem watching in the future.

Initialization


OMPC initialization takes the following steps:

  1. load numeric library for basic functionality,
  2. load certain scipy and GSL modules for extras and toolboxes,
  3. install import hooks

Numerics


In the initial stage numpy module is used as numerical backend. Numpy has all the features that matlab offers and implementing especially slicing should require only little effort.
Ultimatelly it

Hotswapping m-files


Hotswapping is a feature of MATLAB that reloads all scripts from files whenever they are called. Python, however, compiles a source script and unless a reload of the module is forced Python reuses compiled code object irrespectable of changes made to its source.
The second role of hot-swapping also called lazy import is to allow transparent import of modules whose lookup in the globals() fails.

A service constantly watching all the dependencies of a running script needs to be implemented. The performance hit of this module might be considerable in case of scripts and therefore an option should be available to turn of this feature. With this feature turned of Python compiler determines the location of dependencies of a script and insert necessary import statements.
For interactive session to assure full MATLAB experience this feature should be always on.


The process of translation

The OMPC exploits the fact that statements and most of the syntax of MATLAB's m-file is very similar to Python. The process of translation adapts m-files statements to Python statements simply by rewriting each staement. No interpretation is done at this step as it will become clear after this paragraph. The OMPC compiler makes sure that the interpretation of MATLAB's statement will be equivalent to interpretation of m-file within the MATLAB engine.

The steps are summarized in the following list:
  • comments are changed from % to #
  • simple numerical bracketed expression [] is translated to it's python equivalent numpy's matrix() contructor accepts strings that are MATLAB syntax compatible (FIXME!) or better concatenation function that will replace each ";" with "__SEMICOLON__", any concatenations containing a string result in a string, numerical values are converted to integers and ran through the chr() function
  • any use of Java is not planned to be supported, although it is not impossible, the lead developper Peter Jurica thinks that Java is evil and has no place in high level language like MATLAB. Low level extension of OMPC is possible using C and C++. Java was presumably incorporated for its righ library that can be easily replaced by less verbose and easier to learn Python.
  • all occurances of colons will be enclosed in OMPC's m_[] array factory,
  • cell arrays are translated into Python lists wrapped by a special class,
  • left value functions cannot be assigned to but if an object is returned it's properties can be altered, thich means that var(1:10).lvalue = 1 is allowed as well as var(1:10)[:] or anything similar.

Look at the OMPC Numerical Array Features page for list of all features emulation by the ompcArray object.


Catches


There is no way to tell functions apart from variables in MATLAB, we cannot make any assumption.

>> sum(1:10)
ans =
    55
>> 1:10
ans =
     1     2     3     4     5     6     7     8     9    10
>> sum = 5:20;
>> sum(1:10)
ans =
     5     6     7     8     9    10    11    12    13    14

All the problems here can be solved by treating the all MATLAB objects, functions and arrays, equally. The impact of implementing functions as objects has to be studied, The Python equivalent after translation from M-file should look as follows:

>>> sum(m_[1:10])
ans =
    55
>>> 1:10
ans =
     1     2     3     4     5     6     7     8     9    10
>>> sum = 5:20;
>>> sum(1:10)
ans =
     5     6     7     8     9    10    11    12    13    14

Regular expression


The syntax of MATLAB's and Python's regural expressions is slightly different. This translation is done on the level of the OMPC's implementation of the corresponding Python function.


Comments