ProActive Quickstart Distributed Scilab

Scilab is an open source, cross-platform numerical computational package and a high-level, numerically oriented programming language. It can be used for signal processing, statistical analysis, image enhancement, fluid dynamics simulations, numerical optimization, and modeling, simulation of explicit and implicit dynamical systems and (if the corresponding toolbox is installed) symbolic manipulations. Scilab is the most complete open source alternative to MATLAB.

In this tutorial we will use ProActive PAConnector API to write simple Scilab computations which execute on four different machines (a.k.a ProActive Nodes) at the same time. Notice that, this tutorial is supposed to be executed on a Windows machine.

1 Install and configure your working environment


  1. The recommended version of Scilab is 5.5.2 but ProActive Scilab Connector will be soon compatible with version 6.0.0 and higher! You can find an installer compatible with your system [ here ].

  2. Go to your Scilab installation folder and edit SCILAB/etc/scilab.quit.
    Before the line " // starts modules .quit " add this

    if isdef("ProActiveConnectorlib") then
   PAterminate();
end

  3. Download Java Runtime Environment 8 on your machine. You can download a JRE archive at the following location: Latest releases (Choose Operating System:Windows, Architecture: x64, Package Type: JRE, Version: 8, and download the Zip archive). Extract the java archive, then replace the contents of the folder c:\Program Files\scilab-5.5.2\java\jre with the contents of the main archive folder (e.g. jdk8u345-b01-jre)

  4. From your ProActive account, download ProActive Scilab Connector and unzip it (i.e. c:\users\your_name\PAConnector). Do the same with ProActive for Workflows & Scheduling (i.e. c:\users\your_name\scheduling). To get them contact@activeeon.com.
    Start your ProActive server using the proactive-server script fitting to your system (in c:\users\your_name\scheduling\bin).

  5. Ensure your Scilab installation is properly set in c:\users\your_name\PAConnector\lib\worker\ScilabWorkerConfiguration.xml.

2 Load ProActive Scilab connector and Connect to the Scheduler.


  1. Open Scilab.

  2. Before loading a compatible version with our online ProActive platform, remove any already loaded ProActive connector from ATOMS 

    --> atomsRemove('ProActiveConnector')

    Old Scilab versions require to manually install JIMS module (Java Integration Mechanism in Scilab) from ATOMS 

    --> atomsInstall('JIMS')

  3. To load your ProActive Scilab connector and connect to the Scheduler, type the following commands
    (in the popup window, enter your ProActive login and password, or admin/admin by default) 

    --> cd c:\users\your_name\PAConnector

--> exec builder.sce

--> exec loader.sce

-->PAconnect("pnp://MY_MACHINE:64738")

 Connection to the middleman JVM has succeeded, now trying to connect to the scheduler

 Connection successful to pnp://MY_MACHINE:64738

 Please enter login/password

 Login successful
 ans  =

     []

  4. Enter a simple command to verify the scheduler state 

    -->PAstate()
	 ID       NAME            OWNER      PRIORITY     STATUS       START AT                         DURATION
	 2858     Scilab 19       demo       Normal       Finished     14/04/14 17:11 (23h43mn ago)      6s 200ms
	 2857     Scilab 18       demo       Normal       Finished     14/04/14 17:11 (23h43mn ago)      5s 721ms
	 2856     Scilab 17       demo       Normal       Finished     14/04/14 17:10 (23h43mn ago)      3s 737ms
	 2855     Scilab 16       demo       Normal       Finished     14/04/14 17:10 (23h44mn ago)      3s 852ms
	 2854     Scilab 15       demo       Normal       Finished     14/04/14 17:10 (23h44mn ago)     12s 522ms
	 2851     Scilab 12       demo       Normal       Finished     14/04/14 17:10 (23h44mn ago)     14s 323ms
	 2850     Scilab 11       demo       Normal       Finished     14/04/14 17:09 (23h44mn ago)     19s  30ms

3 Hello World Job


Single-parameter function


In this first example, we'll remotely execute a simple Hello World function. We define the function hellow which prints Hello followed by the function argument 

--> deff('[y]=hellow(x)','disp(''Hello '' + x);y=%t')
We submit this function to the Scheduler using the function PAsolve. PAsolve returns an object containing the job and task(s) status 

-->job = PAsolve( 'hellow', 'World')

 Job submitted : 3
 job  =

(1):
Awaited (J:3)

-->job
 job  =

(1):
Awaited (J:3)
We wait for the job completion (i.e. all of his tasks completion) by calling the function PAwaitFor 
-->val = PAwaitFor(job)
Job 3: Task 0_0
[2019-02-25 11:15:007 XPS-P31F][SCILAB OUT]
[2019-02-25 11:15:007 XPS-P31F][SCILAB OUT] Hello World


 val  =


       val(1)

  T

Multiple-parameter function


Now let's execute hellow on several machines with multiple parameters 

-->job = PAsolve( 'hellow', 'World1','World2','World3','World4','World5','World6')

 Job submitted : 7
 job  =

(1):
Awaited (J:7)
(2):
Awaited (J:7)
(3):
Awaited (J:7)
(4):
Awaited (J:7)
(5):
Awaited (J:7)
(6):
Awaited (J:7)

We can follow the remote execution log by displaying multiple times the job variable, which contains the up-to-date informations of the execution 

-->job
 job  =

Job 7: Task 0_0
[2019-02-25 11:30:005 XPS-P31F][SCILAB OUT]
[2019-02-25 11:30:005 XPS-P31F][SCILAB OUT] Hello World1


Job 7: Task 1_0
[2019-02-25 11:30:005 XPS-P31F][SCILAB OUT]
[2019-02-25 11:30:005 XPS-P31F][SCILAB OUT] Hello World2


Job 7: Task 2_0
[2019-02-25 11:30:005 XPS-P31F][SCILAB OUT]
[2019-02-25 11:30:005 XPS-P31F][SCILAB OUT] Hello World3


Job 7: Task 3_0
[2019-02-25 11:30:006 XPS-P31F][SCILAB OUT]
[2019-02-25 11:30:006 XPS-P31F][SCILAB OUT] Hello World4


(1):

  T
(2):

  T
(3):

  T
(4):

  T
(5):
Awaited (J:7)
(6):
Awaited (J:7)

-->job
 job  =

Job 7: Task 4_0
[2019-02-25 11:30:011 XPS-P31F][SCILAB OUT]
[2019-02-25 11:30:011 XPS-P31F][SCILAB OUT] Hello World5


Job 7: Task 5_0
[2019-02-25 11:30:012 XPS-P31F][SCILAB OUT]
[2019-02-25 11:30:012 XPS-P31F][SCILAB OUT] Hello World6


(1):

  T
(2):

  T
(3):

  T
(4):

  T
(5):

  T
(6):

  T

-->job
 job  =

(1):

  T
(2):

  T
(3):

  T
(4):

  T
(5):

  T
(6):

  T

Despite of using PAwaitFor, we can use PAwaitAny to wait for at least one task to finish 


-->job = PAsolve( 'hellow', 'World1','World2','World3','World4','World5','World6')

 Job submitted : 10
 job  =

(1):
Awaited (J:10)
(2):
Awaited (J:10)
(3):
Awaited (J:10)
(4):
Awaited (J:10)
(5):
Awaited (J:10)
(6):
Awaited (J:10)

-->val = PAwaitAny(job)
Job 10: Task 2_0
[2019-02-25 13:05:023 XPS-P31F][SCILAB OUT]
[2019-02-25 13:05:024 XPS-P31F][SCILAB OUT] Hello World3


 val  =

  T

If we want to see the log outputs of all tasks, we can use the function PAgetLogs 

->PAgetLogs(job)
 ans  =


       ans(1)

 Job 10: Task 0_0
[2019-02-25 13:05:024 XPS-P31F][SCILAB OUT]
[
      2019-02-25 13:05:024 XPS-P31F][SCILAB OUT] Hello World1



       ans(2)

 Job 10: Task 1_0
[2019-02-25 13:05:024 XPS-P31F][SCILAB OUT]
[
      2019-02-25 13:05:024 XPS-P31F][SCILAB OUT] Hello World2



       ans(3)

 Job 10: Task 2_0
[2019-02-25 13:05:023 XPS-P31F][SCILAB OUT]
[
      2019-02-25 13:05:024 XPS-P31F][SCILAB OUT] Hello World3



       ans(4)

 Job 10: Task 3_0
[2019-02-25 13:05:025 XPS-P31F][SCILAB OUT]
[
      2019-02-25 13:05:025 XPS-P31F][SCILAB OUT] Hello World4



       ans(5)

 Job 10: Task 4_0
[2019-02-25 13:05:031 XPS-P31F][SCILAB OUT]
[
      2019-02-25 13:05:031 XPS-P31F][SCILAB OUT] Hello World5



       ans(6)

 Job 10: Task 5_0
[2019-02-25 13:05:031 XPS-P31F][SCILAB OUT]
[
      2019-02-25 13:05:031 XPS-P31F][SCILAB OUT] Hello World6

Now let's specify multiple parameters function to PAsolve. We define a two-parameters function hellow2 following this 

-->deff('[z]=hellow2(x,y)','disp(x + '' '' + y);z=%t')

-->hellow2('Hello','World')

 Hello World
 ans  =

  T

In order to submit this function to PAsolve, we need to wrap the parameters inside a list

-->job=PAsolve('hellow2',list('Hello','World'))

 Job submitted : 11
 job  =

(1):
Awaited (J:11)

-->job
 job  =

Job 11: Task 0_0
[2019-02-25 13:16:046 XPS-P31F][SCILAB OUT]
[2019-02-25 13:16:046 XPS-P31F][SCILAB OUT] Hello World


(1):

  T

As you can see, the list is interpreted by PAsolve as a multiple-parameter call.
If we want to execute hellow2 on several machines 

 -->job=PAsolve('hellow2',list('Hello1','World1'),list('Hello2','World2'),list('Hello3','World3'))

 Job submitted : 12
 job  =

(1):
Awaited (J:12)
(2):
Awaited (J:12)
(3):
Awaited (J:12)

-->job
 job  =

Job 12: Task 0_0
[2019-02-25 13:22:026 XPS-P31F][SCILAB OUT]
[2019-02-25 13:22:026 XPS-P31F][SCILAB OUT] Hello1 World1


Job 12: Task 1_0
[2019-02-25 13:22:027 XPS-P31F][SCILAB OUT]
[2019-02-25 13:22:027 XPS-P31F][SCILAB OUT] Hello2 World2


Job 12: Task 2_0
[2019-02-25 13:22:027 XPS-P31F][SCILAB OUT]
[2019-02-25 13:22:027 XPS-P31F][SCILAB OUT] Hello3 World3


(1):

  T
(2):

  T
(3):

  T

4 Job with File Transfer


With inline function (defined at the Scilab console level)


In this chapter, we will demonstrate how to submit simple jobs relying on data transfers (input and output files). We will introduce the PATask object, necessary to use file transfer.
Let's define a function mycopy performing a single file copy 

--> deff('[y]=mycopy(x)','y=copyfile(''in''+string(x)+''.txt'',''out''+string(x)+''.txt'')')
Check the current directory of your Scilab session with pwd() 
--> pwd()
 cd c:\users\your_name\PAConnector

Create a file in1.txt in this directory. You can put some text content in this file if you want.

We will then create a PATask object of size 1 which contains the definition of the task to submit, including input and output files 

--> t=PATask(1,1);

We set the function to be executed 

--> t(1,1).Func = 'mycopy';

Now we add the parameter : 1. Parameters are wrapped inside a list in PATask to support both single and multiple parameter functions 

--> t(1,1).Params = list(1);

Finally we add input and output files. They can be specified as strings, or list of strings for multiple files 

--> t(1,1).InputFiles = 'in1.txt';
--> t(1,1).OutputFiles = 'out1.txt';

To display the full task description 

--> t
 t  =

(1,1):
Func: mycopy
Params:

    1.
InputFiles: { [in1.txt (automatic)] }
OutputFiles: { [out1.txt (automatic)] }
Static: false
ThresholdProximity: 0
Compose: false

We are ready to submit the task t using PAsolve 

--> job = PAsolve(t)

 Job submitted : 2880
 job  =

(1):
Awaited (J:2880)

--> job
 job  =

Job 2880: Task 0_0


(1):

    1.

No remote execution log will be displayed as mycopy function does not print anything. The result is simply the result of the command copyfile. We can check that out1.txt is now present 

-->listfiles(pwd())
 ans  =

!unloader.sce  !
!              !
!unit_tests    !
!              !
!script        !
!              !
!out1.txt      !         <-- output file
!              !
!macros        !
!              !
!loader.sce    !
!              !
!licence.txt   !
!              !
!lib           !
!              !
!jar           !
!              !
!in1.txt       !
!              !
!help          !
!              !
!etc           !
!              !
!config        !
!              !
!cleaner.sce   !
!              !
!builder.sce   !
!              !
!.PAScheduler  !

WIth extern function (defined in a .sci file)


In all the examples above, inline functions were automatically transferred to the remote workers. PAsolve indeed transfers the function variable used as its function argument. This is the case for both inline or extern functions.
However, PAsolve does not automatically transfer the content of auxiliary functions used by the main function. So if the main function calls other user-defined function, it is neceassary to transfer their definition manually.
To illustrate this, we define two function fun1 and fun2 in a dedicated file myfunctions.sci located in the current directory 

function y=fun1(x)
    y = fun2(x) * fun2(x)
endfunction

function y=fun2(x)
    y = x*x
endfunction

Now we construct a PATask of size 3 which uses fun1 and fun2 

-->t=PATask(1,3);

For all PATasks, we set their functions to fun1 (this syntax allow multiple assignment) 

-->t.Func='fun1';

Then we set their input parameters 

-->t(1,1).Params = list(1);
-->t(1,2).Params = list(2);
-->t(1,3).Params = list(3);

Finally we define the source file to be used 

-->t.Sources='myfunctions.sci';

To display the full task description 

-->t
 t  =

(1,1):
Func: fun1
Params:

    1.
InputFiles: { }
OutputFiles: { }
Static: false
ThresholdProximity: 0
Sources: myfunctions.sci
Compose: false

(1,2):
Func: fun1
Params:

    2.
InputFiles: { }
OutputFiles: { }
Static: false
ThresholdProximity: 0
Sources: myfunctions.sci
Compose: false

(1,3):
Func: fun1
Params:

    3.
InputFiles: { }
OutputFiles: { }
Static: false
ThresholdProximity: 0
Sources: myfunctions.sci
Compose: false

Before submitting it, we need to load fun1 and fun2 into the scilab memory 

-->exec myfunctions.sci

-->function y=fun1(x)
-->    y = fun2(x) * fun2(x)
-->endfunction


-->function y=fun2(x)
-->    y = x*x
-->endfunction


-->job = PAsolve(t)

 Job submitted : 2881
 job  =

(1):
Awaited (J:2881)
(2):
Awaited (J:2881)
(3):
Awaited (J:2881)

We can check that fun1 and fun2 were correctly executed 

-->job
 job  =

Job 2881: Task 0_0


Job 2881: Task 1_0


Job 2881: Task 2_0


(1):

    1.
(2):

    16.
(3):

    81.

That's all for the tutorial ! For the full documentation of ProActive Scilab Connector, go to the module's help section of Scilab.