I want to use InterpolatingFunctions in parallel evaluation. However I cant find a way to do it effective.I am aware parallel evaluation generates overhead but i cant understand a few things:1. DistributeDefinitions takes forever for copying definitions to kernels, ~2min for just 300 MB2. Parallel evaluation of InterpolatingFunctions is much slower on parallel kernels. The larger the function the slower it is, between 20 and 5000x slower3. Defining the InterpolatingFunctions directly on the parallel kernels can be fast but not always. However then the evaluation does work as fast on each kernel.4. When a distributed InterpolatingFunction is called the memory of each sub kernel increases by a factor ~3 one by one, only when all are done the evaluation starts. But when repeating the same calculation it does not make any difference.Does anyone have any explanation or a solutions to this?
Definitions
Definitions
In[]:=
ParallelEvaluate[$KernelID];v1=v2=RandomReal[{-1,1},3];t=RandomReal[{-1,1},6];pt={10.5,10.5,10.5};func[{xx_,yy_,zz_,xy_,xz_,yz_}]:=First[Eigenvectors[{{xx,xy,xz},{xy,yy,yz},{xz,yz,zz}}]];comp=Compile[{{v1,_Real,1},{v2,_Real,1}},v1.v2,RuntimeAttributes->{Listable},RuntimeOptions->"Speed"];int1p[n_]:=ListInterpolation[RandomReal[{-1,1},{n,n,n}],InterpolationOrder->1];int2p[n_]:=Interpolation[Thread[{Flatten[Array[{#1,#2,#3}&,{n,n,n}],2],Flatten[RandomReal[{-1,1},{n,n,n}],2]}],InterpolationOrder->1];int3p[n_]:=Interpolation[Thread[{Flatten[Array[{#1,#2,#3}&,{n,n,n}],2],Flatten[RandomReal[{-1,1},{n,n,n,6}],2]}],InterpolationOrder->1];DistributeDefinitions[v1,v2,t,pt,func,comp];parTest[f_]:=parTest[f,RepeatedTiming];parTest[f_,t_]:={t1=t[ReleaseHold@f][[1]],t2=Mean@ParallelEvaluate[t[ReleaseHold@f][[1]]],t2/t1};
Test1 : small int function distributed on kernels
Test1 : small int function distributed on kernels
In[]:=
n=20;int1=int1p[n];int2=int2p[n];int3=int3p[n];{AbsoluteTiming[DistributeDefinitions[#]][[1]],UnitConvert[Quantity[N@ByteCount[#],"Byte"],"MB"]}&/@{int1,int2,int3}TableForm[{Prepend[parTest[Hold@Pause[0.01]],Pause],Prepend[parTest[Hold@func[t]],func],Prepend[parTest[Hold@comp[v1,v2]],comp],Prepend[parTest[Hold[int1@@pt]],int1],Prepend[parTest[Hold[int2@@pt]],int2],Prepend[parTest[Hold[int3@@pt]],int3]},TableHeadings->{None,{"Function","Normal time","Par time","time ratio"}}]
Out[]=
0.0369059,,0.0365953,,0.43643,
Out[]//TableForm=
Function | Normal time | Par time | time ratio | ||
Pause | 0.0166602 | 0.0186088 | 1.11696 | ||
func | 0.0000136058 | 0.0000152046 | 1.11751 | ||
CompiledFunction | 6.47819× -7 10 | 1.04371× -6 10 | 1.61111 | ||
InterpolatingFunction | 8.62073× -6 10 | 0.0000397164 | 4.60708 | ||
InterpolatingFunction | 7.72271× -6 10 | 0.0000422187 | 5.46683 | ||
InterpolatingFunction | 0.000021532 | 0.00662395 | 307.633 |
Test2 : large int function distributed on kernels
Test2 : large int function distributed on kernels
In[]:=
n=100;int1=int1p[n];int2=int2p[n];int3=int3p[n];{AbsoluteTiming[DistributeDefinitions[#]][[1]],UnitConvert[Quantity[N@ByteCount[#],"Byte"],"MB"]}&/@{int1,int2,int3}TableForm[{Prepend[parTest[Hold[int1@@pt],AbsoluteTiming],int1],Prepend[parTest[Hold[int2@@pt],AbsoluteTiming],int2],Prepend[parTest[Hold[int3@@pt],AbsoluteTiming],int3]},TableHeadings->{None,{"Function","Normal time","Par time","time ratio"}}]TableForm[{Prepend[parTest[Hold[int1@@pt],AbsoluteTiming],int1],Prepend[parTest[Hold[int2@@pt],AbsoluteTiming],int2],Prepend[parTest[Hold[int3@@pt],AbsoluteTiming],int3]},TableHeadings->{None,{"Function","Normal time","Par time","time ratio"}}]
Out[]=
3.04477,,3.12655,,47.7633,
Out[]//TableForm=
Function | Normal time | Par time | time ratio | ||
InterpolatingFunction | 0.0000197 | 0.00817088 | 414.765 | ||
InterpolatingFunction | 0.0000177 | 0.00810938 | 458.157 | ||
InterpolatingFunction | 0.0000649 | 0.821239 | 12653.9 |
Out[]//TableForm=
Function | Normal time | Par time | time ratio | ||
InterpolatingFunction | 0.0000208 | 0.00740688 | 356.1 | ||
InterpolatingFunction | 0.0000168 | 0.0077943 | 463.946 | ||
InterpolatingFunction | 0.0000375 | 0.811144 | 21630.5 |
Test3 : large int function defined on kernels
Test3 : large int function defined on kernels
In[]:=
n=100;TableForm[{{AbsoluteTiming[int1=int1p[n]][[1]],AbsoluteTiming[int2=int2p[n]][[1]],AbsoluteTiming[int3=int3p[n]][[1]]},{AbsoluteTiming[ParallelEvaluate[int1=int1p[n]]][[1]],AbsoluteTiming[ParallelEvaluate[int2=int2p[n]]][[1]],AbsoluteTiming[ParallelEvaluate[int3=int3p[n]]][[1]]}},TableHeadings->{{"Normal time","Par Time"},{"int1","int2","int3"}}]TableForm[{Prepend[parTest[Hold[int1@@pt]],int1],Prepend[parTest[Hold[int2@@pt]],int2],Prepend[parTest[Hold[int3@@pt]],int3]},TableHeadings->{None,{"Function","Normal time","Par time","time ratio"}}]