# OP2

## Developing an open-source framework for the execution of unstructured grid applications

The OP2 project is developing an open-source framework for the execution of unstructured grid applications on clusters of GPUs or multi-core CPUs. Although OP2 is designed to look like a conventional library, the implementation uses source-source translation to generate the appropriate back-end code for the different target platforms.

**Funding for OP2 development is coming from:**

- Rolls-Royce plc and the Technology Strategy Board through support for the Siloet project
- EPSRC through support for the project on "Multilayer Abstractions for Partial Differential Equations on Multicore and Manycore Systems", a collaboration with Prof Paul Kelly at Imperial College
- NVIDIA through hardware donations

**Main developers:**

- Mike Giles (Oxford) -- software design, C core routines, CUDA and OpenMP implementations, MATLAB generator
- Gihan Mudalige (Oxford) -- software design, MPI distributed computing, performance assessment, Python generator
- István Reguly (now at PPKE Hungary) -- Applications (Volna, RR Hydra) CUDA optimisations, CPU vectorisation, check-pointing, tiling
- Carlo Bertolli (now at IBM Research) -- software design, FORTRAN and FORTRAN CUDA, ROSE generator
- Lawrence Mitchell (Edinburgh) -- sparse matrix support

**Other collaborators include:**

- Endre László and Ben Spencer at the University of Oxford
- Adam Betts, David Ham, Paul Kelly, Nicolas Loriant, Graham Markall and Florian Rathgeber at Imperial College
- Yoon Ho, Leigh Lapworth, David Radford and Massimiliano Leoni at Rolls-Royce
- Nick Hills at University of Surrey

Other collaborators are very welcome, either as developers of OP2, or as users to provide feedback on the software, documentation, etc.

**Contact: **

- Mike Giles Mike.Giles@maths.ox.ac.uk
- Gihan Mudalige gihan.mudalige@oerc.ox.ac.uk
- Istvan Reguly istvan.reguly@oerc.ox.ac.uk

### OP2

- Users manual (PDF) -- last updated Nov 26, 2014
- Developers guide (PDF) -- last updated April 19, 2012
- Distributed memory (MPI) developer guide (PDF) -- last updated June 12, 2012
- Airfoil example guide (PDF) -- last updated June 12, 2012
- Latest code available on git-hub repository

### Presentations

- I. Z. Reguly, G. R. Mudalige, C. Bertolli, M. B. Giles, A. Betts, P. H. J. Kelly and D. Radford.
*Acceleration of a Full-scale Industrial CFD Application with OP2,*UK Many-Core Developer Conference 2013 (UKMAC’13). December 16, Oxford, UK, 2013. (slides) *A framework for parallel unstructured grid applications on GPUs*, SIAM conference on Parallel Processing for Scientific Computing, Seattle, Feb 26, 2010 (PDF)*Performance Analysis of the OP2 Framework on Many-core Architectures*, Supercomputing SC10, New Orleans, Nov 13-19, 2010 (pptx)*OP2: an open-source library for unstructured grid applications*, IMA workshop on HPC and Emerging Architectures, Minnesota, Jan 10-14, 2011 (PDF)*Optimising the OP2 Framework for GPU Architectures*, MRSC 2011, Bristol, April 12, 2011 (PDF)*OP2 -- an open-source library for unstructured grid applications*, ENUMATH 2011, Leicester, Sept 7, 2011 (PDF)*Software abstractions for manycore software engineering*, LMS meeting on High Performance Scientific Computing at the Exascale, London, Oct 11, 2011 (PDF)*OP2: an active library framework for solvign unstructured mesh-based applications on multi-core and many-core architectures*, InPar '12, May 13-14, 2012 (PDF)

### Papers

- 2015)
*Vectorizing unstructured mesh computations for many-core architectures*. Concurrency Computat.: Pract. Exper., doi: 10.1002/cpe.3621.
, E. , G.R. and M.B. ( - I. Z. Reguly, G.R. Mudalige, C. Bertolli, M.B. Giles, A. Betts, P.H.J. Kelly, and D. Radford.
*Acceleration of a Full-scale Industrial CFD Application with OP2*. (2015) IEEE Transactions on Parallel and Distributed SystemsPrePrints, doi:10.1109/TPDS.2015.2453972 (arxiv) (data) (pre-print PDF) - I.Z. Reguly, E. Laszlo, G.R. Mudalige, M.B. Giles,.
*Vectorizing Unstructured Mesh Computations for Many-core Architectures*, PMAM'14, Proceedings of Programming Models and Applications on Multicores and Manycores, February 2014 (online) - M.B. Giles, G.R. Mudalige, Z. Sharif, G. Markall, P.H.J. Kelly.
*Performance Analysis of the OP2 Framework on Many-core Architectures*, ACM SIGMETRICS Performance Evaluation Review, 38(4):9-15, March 2011. (PDF) - M.B. Giles, G.R. Mudalige, Z. Sharif, G. Markall, P.H.J. Kelly. (2012)
*Performance Analysis and Optimisation of the OP2 Framework on Many-core Architectures*. Computer Journal, 55(2):168-180. (PDF) - C. Bertolli, A. Betts, G.R. Mudalige, M.B. Giles, P.H.J. Kelly.
*Design and Performance of the OP2 Library for Unstructured Mesh Applications*, Euro-Par 2001 Parallel Processing Workshops, Lecture Notes in Computer Science, Springer, 2011. (PDF) - G.R. Mudalige, M.B. Giles, B. Spencer, C. Bertolli, I. Reguly. D
*esigning OP2 for GPU Architectures*, Journal of Parallel and Distributed Computing, 2012. (PDF) - G.R. Mudalige, M.B. Giles, C. Bertolli, and P.H.J. Kelly.
*Predictive Modeling and Analysis of OP2 on Distributed Memory GPU Clusters*, ACM SIGMETRICS Performance Evaluation Review 40(2):61-67, 2012. (PDF) - C. Bertolli, A. Betts, P.H.J. Kelly, G.R. Mudalige, and M.B. Giles.
*Mesh Independent Loop Fusion for Unstructured mesh Applications*. In Proceedings of the 9th Conference on Computing Frontiers (CF '12). ACM, pp.43-52, 2012. (PDF) - G.R. Mudalige, I. Reguly, M.B. Giles, C. Bertolli, and P.H.J. Kelly.
*OP2: An Active Library Framework for Solving Unstructured Mesh-based Applications on Multi-Core and Many-Core Architectures,*Innovative Parallel Computing conference (InPar '12), 2012. (PDF) - C. Bertolli, A. Betts, N. Loriant, G.R. Mudalige, D. Radford, D. Ham, M.B. Giles, and P.H.J. Kelly.
*Compiler Optimizations for Industrial Unstructured Mesh CFD Applications on GPUs,*In proceedings of the International Workshop on Languages and Compilers for Parallel Computing (LCPC), 2012. (PDF)

** **

### OPlus papers and documentation

OP2 is based on the ideas from the OPlus library for the parallel execution of unstructured grid applications on distributed-memory clusters. The original OPlus library development was led by Mike Giles and Paul Crumpton, and it was subsequently ported to MPI and further improved by Nick Hills.

- P.I. Crumpton and M.B. Giles. `Multigrid aircraft computations using the OPlus parallel library'. in
*Parallel Computational Fluid Dynamics: Implementations and Results Using Parallel Computers*, 339-346. A. Ecer, J. Periaux, N. Satofuka, and S. Taylor, editors, North-Holland, 1996. (PDF) - D.A. Burgess, P.I. Crumpton, and M.B. Giles. `A parallel framework for unstructured grid solvers'. In
*Computational Fluid Dynamics '94: Proceedings of the Second European Computational Fluid Dynamics Conference*, pages 391-396. S. Wagner, E.H. Hirschel, J. Periaux, and R. Piva, editors. John Wiley and Sons, 1994. (PDF) - D.A. Burgess, P.I. Crumpton and M.B. Giles. `A parallel framework for unstructured grid solvers'. pp.97-106 in
*Programming Environments for Massively Parallel Distributed Systems*, K.M. Decker and R.M. Rehmann editors, Birkhauser, 1994. - OPlus User Guide (PDF)
- draft paper/report (PDF) on internals within OPlus
- an old webpage