next | previous | forward | backward | up | top | index | toc | Macaulay2 website
RunExternalM2 :: runExternalM2

runExternalM2 -- run a Macaulay2 function in a new Macaulay2 process



This function starts a short-lived new (``child'') Macaulay2 process, loads the file fname, runs the function func with the parameters params, captures the value returned by func, and stores it inside h in the original Macaulay2 process. Optionally, strict resource limits may be imposed on the child process from within Macaulay2, or data may be collected about the resources used by the child process.

Since the child is a new Macaulay2 process, it has no defined variables or functions except those defined in fname. Hence, func and anything it needs (e.g., ring definitions) must be defined in the file fname.

The hash table h stores the exit code of the created Macaulay2 process, the return code of the created Macaulay2 process (see run for details; this is usually 256 times the exit code, plus information about any signals received by the child), the wall-clock time used (as opposed to the CPU time), the name of the output file (unless it was deleted), the name of the answer file (unless it was deleted), any statistics recorded about the resource usage, and the value returned by the function func. If the child process terminates abnormally, then usually the exit code is nonzero and the value returned is null.

For example, we can write a few functions to a temporary file:

i1 : fn=temporaryFileName()|".m2"

o1 = /tmp/M2-288308-0/0.m2
i2 : fn<</// square = (x) -> (stderr<<"Running"<<endl; sleep(1); x^2); ///<<endl;
i3 : fn<</// justexit = () -> ( exit(27); ); ///<<endl;
i4 : fn<</// spin = (x) -> (stderr<<"Spinning!!"<<endl; startTime:=cpuTime(); while(cpuTime()-startTime<x) do for i to 10000000 do i; return(x);); ///<<endl;
i5 : fn<<flush;

and then call them:

i6 : h=runExternalM2(fn,"square",(4));
Running (true && (/usr/bin/M2-binary  --stop --no-debug --silent  -q  <"/tmp/M2-288308-0/1.m2" >"/tmp/M2-288308-0/1.out" 2>&1 ))
Finished running.
i7 : h

o7 = HashTable{answer file => null}
               exit code => 0
               output file => null
               return code => 0
               statistics => null
               time used => 2
               value => 16

o7 : HashTable
i8 : h#value===4^2

o8 = true
i9 : h#"exit code"===0

o9 = true

An abnormal program exit will have a nonzero exit code; also, the value will be null, the output file should exist, but the answer file may not exist unless the routine finished successfully.

i10 : h=runExternalM2(fn,"justexit",());
Running (true && (/usr/bin/M2-binary  --stop --no-debug --silent  -q  <"/tmp/M2-288308-0/2.m2" >"/tmp/M2-288308-0/2.out" 2>&1 ))
Finished running.
RunExternalM2: expected answer file does not exist
i11 : h

o11 = HashTable{answer file => /tmp/M2-288308-0/2.ans}
                exit code => 27
                output file => /tmp/M2-288308-0/2.out
                return code => 6912
                statistics => null
                time used => 0
                value => null

o11 : HashTable
i12 : fileExists(h#"output file")

o12 = true
i13 : fileExists(h#"answer file")

o13 = false

Here, we use resource limits to limit the routine to 2 seconds of computational time, while the system is asked to use 10 seconds of computational time:

i14 : h=runExternalM2(fn,"spin",10,PreRunScript=>"ulimit -t 2");
Running (ulimit -t 2 && (/usr/bin/M2-binary  --stop --no-debug --silent  -q  <"/tmp/M2-288308-0/3.m2" >"/tmp/M2-288308-0/3.out" 2>&1 ))
Finished running.
RunExternalM2: expected answer file does not exist
i15 : h

o15 = HashTable{answer file => /tmp/M2-288308-0/3.ans}
                exit code => 0
                output file => /tmp/M2-288308-0/3.out
                return code => 9
                statistics => null
                time used => 2
                value => null

o15 : HashTable
i16 : if h#"output file" =!= null and fileExists(h#"output file") then get(h#"output file")

o16 = 
      i1 : -- Script /tmp/M2-288308-0/3.m2 automatically generated by RunExternalM2

      i2 : load "/tmp/M2-288308-0/0.m2";

      i3 : runExternalM2ReturnAnswer("/tmp/M2-288308-0/3.ans",spin (10));
i17 : if h#"answer file" =!= null and fileExists(h#"answer file") then get(h#"answer file")

We can get quite a lot of detail on the resources used with the KeepStatistics command:

i18 : h=runExternalM2(fn,"spin",3,KeepStatistics=>true);
Running (true && ( (/usr/bin/time --verbose sh -c '/usr/bin/M2-binary  --stop --no-debug --silent  -q  <"/tmp/M2-288308-0/4.m2" >"/tmp/M2-288308-0/4.out" 2>&1') >"/tmp/M2-288308-0/4.stat" 2>&1 ))
Finished running.
i19 : h#"statistics"

o19 =         Command being timed: "sh -c /usr/bin/M2-binary  --stop --no-debug --silent  -q  <"/tmp/M2-288308-0/4.m2" >"/tmp/M2-288308-0/4.out" 2>&1"
              User time (seconds): 4.82
              System time (seconds): 0.05
              Percent of CPU this job got: 99%
              Elapsed (wall clock) time (h:mm:ss or m:ss): 0:04.88
              Average shared text size (kbytes): 0
              Average unshared data size (kbytes): 0
              Average stack size (kbytes): 0
              Average total size (kbytes): 0
              Maximum resident set size (kbytes): 91100
              Average resident set size (kbytes): 0
              Major (requiring I/O) page faults: 0
              Minor (reclaiming a frame) page faults: 20751
              Voluntary context switches: 678
              Involuntary context switches: 973
              Swaps: 0
              File system inputs: 0
              File system outputs: 16
              Socket messages sent: 0
              Socket messages received: 0
              Signals delivered: 0
              Page size (bytes): 4096
              Exit status: 0

We can handle most kinds of objects as return values, although MutableMatrix does not work. Here, we use the built-in identity function:

i20 : v=/// A complicated string^%&C@#CERQVASDFQ#BQBSDH"' ewrjwklsf///;
i21 : (runExternalM2(fn,identity,v))#value===v
Running (true && (/usr/bin/M2-binary  --stop --no-debug --silent  -q  <"/tmp/M2-288308-0/6.m2" >"/tmp/M2-288308-0/6.out" 2>&1 ))
Finished running.

o21 = true

Some care is required, however:

i22 : R=QQ[x,y];
i23 : v=coker random(R^2,R^{3:-1})

o23 = cokernel | 9/2x+1/2y x+3/4y    7/4x+7/9y  |
               | 9/4x+1/2y 3/2x+3/4y 7/10x+1/2y |

o23 : R-module, quotient of R
i24 : h=runExternalM2(fn,identity,v)
Running (true && (/usr/bin/M2-binary  --stop --no-debug --silent  -q  <"/tmp/M2-288308-0/7.m2" >"/tmp/M2-288308-0/7.out" 2>&1 ))
Finished running.
RunExternalM2: expected answer file does not exist

o24 = HashTable{answer file => /tmp/M2-288308-0/7.ans}
                exit code => 1
                output file => /tmp/M2-288308-0/7.out
                return code => 256
                statistics => null
                time used => 0
                value => null

o24 : HashTable

To view the error message:

i25 : get(h#"output file")

o25 = 
      i1 : -- Script /tmp/M2-288308-0/7.m2 automatically generated by RunExternalM2

      i2 : load "/tmp/M2-288308-0/0.m2";

      i3 : runExternalM2ReturnAnswer("/tmp/M2-288308-0/7.ans",identity (cokernel(map(R^2,R^{{-1}, {-1}, {-1}},{{(9/2)*x+(1/2)*y, x+(3/4)*y, (7/4)*x+(7/9)*y}, {(9/4)*x+(1/2)*y, (3/2)*x+(3/4)*y, (7/10)*x+(1/2)*y}}))));
      stdio:4:76:(3):[1]: error: no method for binary operator ^ applied to objects:
      --            R (of class Symbol)
      --      ^     2 (of class ZZ)

Keep in mind that the object you are passing must make sense in the context of the file containing your function! For instance, here we need to define the ring:

i26 : fn<<///R=QQ[x,y];///<<endl<<flush;
i27 : (runExternalM2(fn,identity,v))#value===v
Running (true && (/usr/bin/M2-binary  --stop --no-debug --silent  -q  <"/tmp/M2-288308-0/8.m2" >"/tmp/M2-288308-0/8.out" 2>&1 ))
Finished running.

o27 = true

This problem can be avoided by following some suggestions for using RunExternalM2.

The objects may unavoidably lose some internal references, though:

i28 : v=R;
i29 : h=runExternalM2(fn,identity,v);
Running (true && (/usr/bin/M2-binary  --stop --no-debug --silent  -q  <"/tmp/M2-288308-0/9.m2" >"/tmp/M2-288308-0/9.out" 2>&1 ))
Finished running.
i30 : h#value

o30 = QQ[x..y]

o30 : PolynomialRing
i31 : v===h#value

o31 = false

but this happens because

i32 : R===value(toExternalString(R))

o32 = false

See also

For the programmer

The object runExternalM2 is a function closure.